Deep learning-based assessment of CT markers of sarcopenia and myosteatosis for outcome assessment in patients with advanced pancreatic cancer after high-intensity focused ultrasound treatment.

OBJECTIVES: To evaluate the prognostic value of CT-based markers of sarcopenia and myosteatosis in comparison to the Eastern Cooperative Oncology Group (ECOG) score for survival of patients with advanced pancreatic cancer treated with high-intensity focused ultrasound (HIFU). MATERIALS AND METHODS: For 142 retrospective patients, the skeletal muscle index (SMI), skeletal muscle radiodensity (SMRD), fatty muscle fraction (FMF), and intermuscular fat fraction (IMFF) were determined on superior mesenteric artery level in pre-interventional CT. Each marker was tested for associations with sex, age, body mass index (BMI), and ECOG. The prognostic value of the markers was examined in Kaplan-Meier analyses with the log-rank test and in uni- and multivariable Cox proportional hazards (CPH) models. RESULTS: The following significant associations were observed: Male patients had higher BMI and SMI. Patients with lower ECOG had lower BMI and SMI. Patients with BMI lower than 21.8 kg/m2 (median) also showed lower SMI and IMFF. Patients younger than 63.3 years (median) were found to have higher SMRD, lower FMF, and lower IMFF. In the Kaplan-Meier analysis, significantly lower survival times were observed in patients with higher ECOG or lower SMI. Increased patient risk was observed for higher ECOG, lower BMI, and lower SMI in univariable CPH analyses for 1-, 2-, and 3-year survival. Multivariable CPH analysis for 1-year survival revealed increased patient risk for higher ECOG, lower SMI, lower IMFF, and higher FMF. In multivariable analysis for 2- and 3-year survival, only ECOG and FMF remained significant. CONCLUSION: CT-based markers of sarcopenia and myosteatosis show a prognostic value for assessment of survival in advanced pancreatic cancer patients undergoing HIFU therapy. CLINICAL RELEVANCE STATEMENT: The results indicate a greater role of myosteatosis for additional risk assessment beyond clinical scores, as only FMF was associated with long-term survival in multivariable CPH analyses along ECOG and also showed independence to ECOG in group analysis. KEY POINTS: • This study investigates the prognostic value of CT-based markers of sarcopenia and myosteatosis for patients with pancreatic cancer treated with high-intensity focused ultrasound. • Markers for sarcopenia and myosteatosis showed a prognostic value besides clinical assessment of the physical status by the Eastern Cooperative Oncology Group score. In contrast to muscle size measurements, the myosteatosis marker fatty muscle fraction demonstrated independence to the clinical score. • The results indicate that myosteatosis might play a greater role for additional patient risk assessments beyond clinical assessments of physical status.

Development of image-based decision support systems utilizing information extracted from radiological free-text report databases with text-based transformers.

OBJECTIVES: To investigate the potential and limitations of utilizing transformer-based report annotation for on-site development of image-based diagnostic decision support systems (DDSS). METHODS: The study included 88,353 chest X-rays from 19,581 intensive care unit (ICU) patients. To label the presence of six typical findings in 17,041 images, the corresponding free-text reports of the attending radiologists were assessed by medical research assistants ("gold labels"). Automatically generated "silver" labels were extracted for all reports by transformer models trained on gold labels. To investigate the benefit of such silver labels, the image-based models were trained using three approaches: with gold labels only (MG), with silver labels first, then with gold labels (MS/G), and with silver and gold labels together (MS+G). To investigate the influence of invested annotation effort, the experiments were repeated with different numbers (N) of gold-annotated reports for training the transformer and image-based models and tested on 2099 gold-annotated images. Significant differences in macro-averaged area under the receiver operating characteristic curve (AUC) were assessed by non-overlapping 95% confidence intervals. RESULTS: Utilizing transformer-based silver labels showed significantly higher macro-averaged AUC than training solely with gold labels (N = 1000: MG 67.8 [66.0-69.6], MS/G 77.9 [76.2-79.6]; N = 14,580: MG 74.5 [72.8-76.2], MS/G 80.9 [79.4-82.4]). Training with silver and gold labels together was beneficial using only 500 gold labels (MS+G 76.4 [74.7-78.0], MS/G 75.3 [73.5-77.0]). CONCLUSIONS: Transformer-based annotation has potential for unlocking free-text report databases for the development of image-based DDSS. However, on-site development of image-based DDSS could benefit from more sophisticated annotation pipelines including further information than a single radiological report. CLINICAL RELEVANCE STATEMENT: Leveraging clinical databases for on-site development of artificial intelligence (AI)-based diagnostic decision support systems by text-based transformers could promote the application of AI in clinical practice by circumventing highly regulated data exchanges with third parties. KEY POINTS: • The amount of data from a database that can be used to develop AI-assisted diagnostic decision systems is often limited by the need for time-consuming identification of pathologies by radiologists. • The transformer-based structuring of free-text radiological reports shows potential to unlock corresponding image databases for on-site development of image-based diagnostic decision support systems. • However, the quality of image annotations generated solely on the content of a single radiology report may be limited by potential inaccuracies and incompleteness of this report.

Deep learning for standardized, MRI-based quantification of subcutaneous and subfascial tissue volume for patients with lipedema and lymphedema.

OBJECTIVES: To contribute to a more in-depth assessment of shape, volume, and asymmetry of the lower extremities in patients with lipedema or lymphedema utilizing volume information from MR imaging. METHODS: A deep learning (DL) pipeline was developed including (i) localization of anatomical landmarks (femoral heads, symphysis, knees, ankles) and (ii) quality-assured tissue segmentation to enable standardized quantification of subcutaneous (SCT) and subfascial tissue (SFT) volumes. The retrospectively derived dataset for method development consisted of 45 patients (42 female, 44.2 ± 14.8 years) who underwent clinical 3D DIXON MR-lymphangiography examinations of the lower extremities. Five-fold cross-validated training was performed on 16,573 axial slices from 40 patients and testing on 2187 axial slices from 5 patients. For landmark detection, two EfficientNet-B1 convolutional neural networks (CNNs) were applied in an ensemble. One determines the relative foot-head position of each axial slice with respect to the landmarks by regression, the other identifies all landmarks in coronal reconstructed slices using keypoint detection. After landmark detection, segmentation of SCT and SFT was performed on axial slices employing a U-Net architecture with EfficientNet-B1 as encoder. Finally, the determined landmarks were used for standardized analysis and visualization of tissue volume, distribution, and symmetry, independent of leg length, slice thickness, and patient position. RESULTS: Excellent test results were observed for landmark detection (z-deviation = 4.5 ± 3.1 mm) and segmentation (Dice score: SCT = 0.989 ± 0.004, SFT = 0.994 ± 0.002). CONCLUSIONS: The proposed DL pipeline allows for standardized analysis of tissue volume and distribution and may assist in diagnosis of lipedema and lymphedema or monitoring of conservative and surgical treatments. KEY POINTS: • Efficient use of volume information that MRI inherently provides can be extracted automatically by deep learning and enables in-depth assessment of tissue volumes in lipedema and lymphedema. • The deep learning pipeline consisting of body part regression, keypoint detection, and quality-assured tissue segmentation provides detailed information about the volume, distribution, and asymmetry of lower extremity tissues, independent of leg length, slice thickness, and patient position.

End-to-end automated body composition analyses with integrated quality control for opportunistic assessment of sarcopenia in CT.

OBJECTIVES: To develop a pipeline for automated body composition analysis and skeletal muscle assessment with integrated quality control for large-scale application in opportunistic imaging. METHODS: First, a convolutional neural network for extraction of a single slice at the L3/L4 lumbar level was developed on CT scans of 240 patients applying the nnU-Net framework. Second, a 2D competitive dense fully convolutional U-Net for segmentation of visceral and subcutaneous adipose tissue (VAT, SAT), skeletal muscle (SM), and subsequent determination of fatty muscle fraction (FMF) was developed on single CT slices of 1143 patients. For both steps, automated quality control was integrated by a logistic regression model classifying the presence of L3/L4 and a linear regression model predicting the segmentation quality in terms of Dice score. To evaluate the performance of the entire pipeline end-to-end, body composition metrics, and FMF were compared to manual analyses including 364 patients from two centers. RESULTS: Excellent results were observed for slice extraction (z-deviation = 2.46 ± 6.20 mm) and segmentation (Dice score for SM = 0.95 ± 0.04, VAT = 0.98 ± 0.02, SAT = 0.97 ± 0.04) on the dual-center test set excluding cases with artifacts due to metallic implants. No data were excluded for end-to-end performance analyses. With a restrictive setting of the integrated segmentation quality control, 39 of 364 patients were excluded containing 8 cases with metallic implants. This setting ensured a high agreement between manual and fully automated analyses with mean relative area deviations of ΔSM = 3.3 ± 4.1%, ΔVAT = 3.0 ± 4.7%, ΔSAT = 2.7 ± 4.3%, and ΔFMF = 4.3 ± 4.4%. CONCLUSIONS: This study presents an end-to-end automated deep learning pipeline for large-scale opportunistic assessment of body composition metrics and sarcopenia biomarkers in clinical routine. KEY POINTS: • Body composition metrics and skeletal muscle quality can be opportunistically determined from routine abdominal CT scans. • A pipeline consisting of two convolutional neural networks allows an end-to-end automated analysis. • Machine-learning-based quality control ensures high agreement between manual and automatic analysis.

Deep Learning Virtual Contrast-Enhanced T1 Mapping for Contrast-Free Myocardial Extracellular Volume Assessment.

BACKGROUND: The acquisition of contrast-enhanced T1 maps to calculate extracellular volume (ECV) requires contrast agent administration and is time consuming. This study investigates generative adversarial networks for contrast-free, virtual extracellular volume (vECV) by generating virtual contrast-enhanced T1 maps. METHODS AND RESULTS: This retrospective study includes 2518 registered native and contrast-enhanced T1 maps from 1000 patients who underwent cardiovascular magnetic resonance at 1.5 Tesla. Recent hematocrit values of 123 patients (hold-out test) and 96 patients from a different institution (external evaluation) allowed for calculation of conventional ECV. A generative adversarial network was trained to generate virtual contrast-enhanced T1 maps from native T1 maps for vECV creation. Mean and SD of the difference per patient (ΔECV) were calculated and compared by permutation of the 2-sided t test with 10 000 resamples. For ECV and vECV, differences in area under the receiver operating characteristic curve (AUC) for discriminating hold-out test patients with normal cardiovascular magnetic resonance versus myocarditis or amyloidosis were tested with Delong's test. ECV and vECV showed a high agreement in patients with myocarditis (ΔECV: hold-out test, 2.0%±1.5%; external evaluation, 1.9%±1.7%) and normal cardiovascular magnetic resonance (ΔECV: hold-out test, 1.9%±1.4%; external evaluation, 1.5%±1.2%), but variations in amyloidosis were higher (ΔECV: hold-out test, 6.2%±6.0%; external evaluation, 15.5%±6.4%). In the hold-out test, ECV and vECV had a comparable AUC for the diagnosis of myocarditis (ECV AUC, 0.77 versus vECV AUC, 0.76; P=0.76) and amyloidosis (ECV AUC, 0.99 versus vECV AUC, 0.96; P=0.52). CONCLUSIONS: Generation of vECV on the basis of native T1 maps is feasible. Multicenter training data are required to further enhance generalizability of vECV in amyloidosis.

CT-derived Epicardial Adipose Tissue Inflammation Predicts Outcome in Patients Undergoing Transcatheter Aortic Valve Replacement.

PURPOSE: Inflammatory changes in epicardial (EAT) and pericardial adipose tissue (PAT) are associated with increased overall cardiovascular risk. Using routine, preinterventional cardiac CT data, we examined the predictive value of quantity and quality of EAT and PAT for outcome after transcatheter aortic valve replacement (TAVR). MATERIALS AND METHODS: Cardiac CT data of 1197 patients who underwent TAVR at the in-house heart center between 2011 and 2020 were retrospectively analyzed. The amount and density of EAT and PAT were quantified from single-slice CT images at the level of the aortic valve. Using established risk scores and known independent risk factors, a clinical benchmark model (BMI, Chronic kidney disease stage, EuroSCORE 2, STS Prom, year of intervention) for outcome prediction (2-year mortality) after TAVR was established. Subsequently, we tested whether the additional inclusion of area and density values of EAT and PAT in the clinical benchmark model improved prediction. For this purpose, the cohort was divided into a training (n=798) and a test cohort (n=399). RESULTS: Within the 2-year follow-up, 264 patients died. In the training cohort, particularly the addition of EAT density to the clinical benchmark model showed a significant association with outcome (hazard ratio 1.04, 95% CI: 1.01-1.07; P =0.013). In the test cohort, the outcome prediction of the clinical benchmark model was also significantly improved with the inclusion of EAT density (c-statistic: 0.589 vs. 0.628; P =0.026). CONCLUSIONS: EAT density as a surrogate marker of EAT inflammation was associated with 2-year mortality after TAVR and may improve outcome prediction independent of established risk parameters.

Direct deep learning-based survival prediction from pre-interventional CT prior to transcatheter aortic valve replacement.

PURPOSE: To investigate survival prediction in patients undergoing transcatheter aortic valve replacement (TAVR) using deep learning (DL) methods applied directly to pre-interventional CT images and to compare performance with survival models based on scalar markers of body composition. METHOD: This retrospective single-center study included 760 patients undergoing TAVR (mean age 81 ± 6 years; 389 female). As a baseline, a Cox proportional hazards model (CPHM) was trained to predict survival on sex, age, and the CT body composition markers fatty muscle fraction (FMF), skeletal muscle radiodensity (SMRD), and skeletal muscle area (SMA) derived from paraspinal muscle segmentation of a single slice at L3/L4 level. The convolutional neural network (CNN) encoder of the DL model for survival prediction was pre-trained in an autoencoder setting with and without a focus on paraspinal muscles. Finally, a combination of DL and CPHM was evaluated. Performance was assessed by C-index and area under the receiver operating curve (AUC) for 1-year and 2-year survival. All methods were trained with five-fold cross-validation and were evaluated on 152 hold-out test cases. RESULTS: The CNN for direct image-based survival prediction, pre-trained in a focussed autoencoder scenario, outperformed the baseline CPHM (CPHM: C-index = 0.608, 1Y-AUC = 0.606, 2Y-AUC = 0.594 vs. DL: C-index = 0.645, 1Y-AUC = 0.687, 2Y-AUC = 0.692). Combining DL and CPHM led to further improvement (C-index = 0.668, 1Y-AUC = 0.713, 2Y-AUC = 0.696). CONCLUSIONS: Direct DL-based survival prediction shows potential to improve image feature extraction compared to segmentation-based scalar markers of body composition for risk assessment in TAVR patients.

Deep learning enables automated MRI-based estimation of uterine volume also in patients with uterine fibroids undergoing high-intensity focused ultrasound therapy.

BACKGROUND: High-intensity focused ultrasound (HIFU) is used for the treatment of symptomatic leiomyomas. We aim to automate uterine volumetry for tracking changes after therapy with a 3D deep learning approach. METHODS: A 3D nnU-Net model in the default setting and in a modified version including convolutional block attention modules (CBAMs) was developed on 3D T2-weighted MRI scans. Uterine segmentation was performed in 44 patients with routine pelvic MRI (standard group) and 56 patients with uterine fibroids undergoing ultrasound-guided HIFU therapy (HIFU group). Here, preHIFU scans (n = 56), postHIFU imaging maximum one day after HIFU (n = 54), and the last available follow-up examination (n = 53, days after HIFU: 420 ± 377) were included. The training was performed on 80% of the data with fivefold cross-validation. The remaining data were used as a hold-out test set. Ground truth was generated by a board-certified radiologist and a radiology resident. For the assessment of inter-reader agreement, all preHIFU examinations were segmented independently by both. RESULTS: High segmentation performance was already observed for the default 3D nnU-Net (mean Dice score = 0.95 ± 0.05) on the validation sets. Since the CBAM nnU-Net showed no significant benefit, the less complex default model was applied to the hold-out test set, which resulted in accurate uterus segmentation (Dice scores: standard group 0.92 ± 0.07; HIFU group 0.96 ± 0.02), which was comparable to the agreement between the two readers. CONCLUSIONS: This study presents a method for automatic uterus segmentation which allows a fast and consistent assessment of uterine volume. Therefore, this method could be used in the clinical setting for objective assessment of therapeutic response to HIFU therapy.

Opportunistic CT-derived analysis of fat and muscle tissue composition predicts mortality in patients with cardiogenic shock.

Prognosis estimation in patients with cardiogenic shock (CS) is important to guide clinical decision making. Aim of this study was to investigate the predictive value of opportunistic CT-derived body composition analysis in CS patients. Amount and density of fat and muscle tissue of 152 CS patients were quantified from single-slice CT images at the level of the intervertebral disc space L3/L4. Multivariable Cox regression and Kaplan-Meier survival analyses were performed to evaluate the predictive value of opportunistically CT-derived body composition parameters on the primary endpoint of 30-day mortality. Within the 30-day follow-up, 90/152 (59.2%) patients died. On multivariable analyses, lactate (Hazard Ratio 1.10 [95% Confidence Interval 1.04-1.17]; p = 0.002) and patient age (HR 1.04 [95% CI 1.01-1.07], p = 0.017) as clinical prognosticators, as well as visceral adipose tissue (VAT) area (HR 1.004 [95% CI 1.002-1.007]; p = 0.001) and skeletal muscle (SM) area (HR 0.987 [95% CI 0.975-0.999]; p = 0.043) as imaging biomarkers remained as independent predictors of 30-day mortality. Kaplan-Meier survival analyses showed significantly increased 30-day mortality in patients with higher VAT area (p = 0.015) and lower SM area (p = 0.035). CT-derived VAT and SM area are independent predictors of dismal outcomes in CS patients and have the potential to emerge as new imaging biomarkers available from routine diagnostic CT.

Deep Learning-Based Body Composition Analysis Predicts Outcome in Melanoma Patients Treated with Immune Checkpoint Inhibitors.

Previous studies suggest an impact of body composition on outcome in melanoma patients. We aimed to determine the prognostic value of CT-based body composition assessment in patients receiving immune checkpoint inhibitor therapy for treatment of metastatic disease using a deep learning approach. One hundred seven patients with staging CT examinations prior to initiation of checkpoint inhibition between January 2013 and August 2019 were retrospectively evaluated. Using an automated deep learning-based body composition analysis pipeline, parameters for estimation of skeletal muscle mass (skeletal muscle index, SMI) and adipose tissue compartments (visceral adipose tissue index, VAI; subcutaneous adipose tissue index, SAI) were derived from staging CT. The cohort was binarized according to gender-specific median cut-off values. Patients below the median were defined as having low SMI, VAI, or SAI, respectively. The impact on outcome was assessed using the Kaplan-Meier method with log-rank tests. A multivariable logistic regression model was built to test the impact of body composition parameters on 3-year mortality. Patients with low SMI displayed significantly increased 1-year (25% versus 9%, p = 0.035), 2-year (32% versus 13%, p = 0.017), and 3-year mortality (38% versus 19%, p = 0.016). No significant differences with regard to adipose tissue compartments were observed (3-year mortality: VAI, p = 0.448; SAI, p = 0.731). On multivariable analysis, low SMI (hazard ratio (HR), 2.245; 95% confidence interval (CI), 1.005-5.017; p = 0.049), neutrophil-to-lymphocyte ratio (HR, 1.170; 95% CI, 1.076-1.273; p < 0.001), and Karnofsky index (HR, 0.965; 95% CI, 0.945-0.985; p = 0.001) remained as significant predictors of 3-year mortality. Lowered skeletal muscle index as an indicator of sarcopenia was associated with worse outcome in patients with metastatic melanoma receiving immune checkpoint inhibitor therapy.