[Large language models from OpenAI, Google, Meta, X and Co. : The role of "closed" and "open" models in radiology]. / Große Sprachmodelle von OpenAI, Google, Meta, X und Co. : Die Rolle von "closed" und "open" Modellen in der Radiologie.

BACKGROUND: In 2023, the release of ChatGPT triggered an artificial intelligence (AI) boom. The underlying large language models (LLM) of the nonprofit organization "OpenAI" are not freely available under open-source licenses, which does not allow on-site implementation inside secure clinic networks. However, efforts are being made by open-source communities, start-ups and large tech companies to democratize the use of LLMs. This opens up the possibility of using LLMs in a data protection-compliant manner and even adapting them to our own data. OBJECTIVES: This paper aims to explain the potential of privacy-compliant local LLMs for radiology and to provide insights into the "open" versus "closed" dynamics of the currently rapidly developing field of AI. MATERIALS AND METHODS: PubMed search for radiology articles with LLMs and subjective selection of references in the sense of a narrative key topic article. RESULTS: Various stakeholders, including large tech companies such as Meta, Google and X, but also European start-ups such as Mistral AI, contribute to the democratization of LLMs by publishing the models (open weights) or by publishing the model and source code (open source). Their performance is lower than current "closed" LLMs, such as GPT­4 from OpenAI. CONCLUSION: Despite differences in performance, open and thus locally implementable LLMs show great promise for improving the efficiency and quality of diagnostic reporting as well as interaction with patients and enable retrospective extraction of diagnostic information for secondary use of clinical free-text databases for research, teaching or clinical application.

3D-Volumetric Shunt Measurement for Detection of High-Risk Esophageal Varices in Liver Cirrhosis.

Background and Objectives: Esophageal varices (EV) and variceal hemorrhages are major causes of mortality in liver cirrhosis patients. Detecting EVs early is crucial for effective management. Computed tomography (CT) scans, commonly performed for various liver-related indications, provide an opportunity for non-invasive EV assessment. However, previous CT studies focused on variceal diameter, neglecting the three-dimensional (3D) nature of varices and shunt vessels. This study aims to evaluate the potential of 3D volumetric shunt-vessel measurements from routine CT scans for detecting high-risk esophageal varices in portal hypertension. Methods: 3D volumetric measurements of esophageal varices were conducted using routine CT scans and compared to endoscopic variceal grading. Receiver operating characteristic (ROC) analyses were performed to determine the optimal cutoff value for identifying high-risk varices based on shunt volume. The study included 142 patients who underwent both esophagogastroduodenoscopy (EGD) and contrast-enhanced CT within six months. Results: The study established a cutoff value for identifying high-risk varices. The CT measurements exhibited a significant correlation with endoscopic EV grading (correlation coefficient r = 0.417, p < 0.001). A CT cutoff value of 2060 mm3 for variceal volume showed a sensitivity of 72.1% and a specificity of 65.5% for detecting high-risk varices during endoscopy. Conclusions: This study demonstrates the feasibility of opportunistically measuring variceal volumes from routine CT scans. CT volumetry for assessing EVs may have prognostic value, especially in cirrhosis patients who are not suitable candidates for endoscopy.

Generative Pre-trained Transformer 4 makes cardiovascular magnetic resonance reports easy to understand.

BACKGROUND: Patients are increasingly using Generative Pre-trained Transformer 4 (GPT-4) to better understand their own radiology findings. PURPOSE: To evaluate the performance of GPT-4 in transforming cardiovascular magnetic resonance (CMR) reports into text that is comprehensible to medical laypersons. METHODS: ChatGPT with GPT-4 architecture was used to generate three different explained versions of 20 various CMR reports (n = 60) using the same prompt: "Explain the radiology report in a language understandable to a medical layperson". Two cardiovascular radiologists evaluated understandability, factual correctness, completeness of relevant findings, and lack of potential harm, while 13 medical laypersons evaluated the understandability of the original and the GPT-4 reports on a Likert scale (1 "strongly disagree", 5 "strongly agree"). Readability was measured using the Automated Readability Index (ARI). Linear mixed-effects models (values given as median [interquartile range]) and intraclass correlation coefficient (ICC) were used for statistical analysis. RESULTS: GPT-4 reports were generated on average in 52 s ± 13. GPT-4 reports achieved a lower ARI score (10 [9-12] vs 5 [4-6]; p < 0.001) and were subjectively easier to understand for laypersons than original reports (1 [1] vs 4 [4,5]; p < 0.001). Eighteen out of 20 (90%) standard CMR reports and 2/60 (3%) GPT-generated reports had an ARI score corresponding to the 8th grade level or higher. Radiologists' ratings of the GPT-4 reports reached high levels for correctness (5 [4, 5]), completeness (5 [5]), and lack of potential harm (5 [5]); with "strong agreement" for factual correctness in 94% (113/120) and completeness of relevant findings in 81% (97/120) of reports. Test-retest agreement for layperson understandability ratings between the three simplified reports generated from the same original report was substantial (ICC: 0.62; p < 0.001). Interrater agreement between radiologists was almost perfect for lack of potential harm (ICC: 0.93, p < 0.001) and moderate to substantial for completeness (ICC: 0.76, p < 0.001) and factual correctness (ICC: 0.55, p < 0.001). CONCLUSION: GPT-4 can reliably transform complex CMR reports into more understandable, layperson-friendly language while largely maintaining factual correctness and completeness, and can thus help convey patient-relevant radiology information in an easy-to-understand manner.

CT-derived Epicardial Adipose Tissue Inflammation Predicts Outcome in Patients Undergoing Transcatheter Aortic Valve Replacement.

PURPOSE: Inflammatory changes in epicardial (EAT) and pericardial adipose tissue (PAT) are associated with increased overall cardiovascular risk. Using routine, preinterventional cardiac CT data, we examined the predictive value of quantity and quality of EAT and PAT for outcome after transcatheter aortic valve replacement (TAVR). MATERIALS AND METHODS: Cardiac CT data of 1197 patients who underwent TAVR at the in-house heart center between 2011 and 2020 were retrospectively analyzed. The amount and density of EAT and PAT were quantified from single-slice CT images at the level of the aortic valve. Using established risk scores and known independent risk factors, a clinical benchmark model (BMI, Chronic kidney disease stage, EuroSCORE 2, STS Prom, year of intervention) for outcome prediction (2-year mortality) after TAVR was established. Subsequently, we tested whether the additional inclusion of area and density values of EAT and PAT in the clinical benchmark model improved prediction. For this purpose, the cohort was divided into a training (n=798) and a test cohort (n=399). RESULTS: Within the 2-year follow-up, 264 patients died. In the training cohort, particularly the addition of EAT density to the clinical benchmark model showed a significant association with outcome (hazard ratio 1.04, 95% CI: 1.01-1.07; P =0.013). In the test cohort, the outcome prediction of the clinical benchmark model was also significantly improved with the inclusion of EAT density (c-statistic: 0.589 vs. 0.628; P =0.026). CONCLUSIONS: EAT density as a surrogate marker of EAT inflammation was associated with 2-year mortality after TAVR and may improve outcome prediction independent of established risk parameters.

Optical Flow-Guided Cine MRI Segmentation With Learned Corrections.

In cardiac cine magnetic resonance imaging (MRI), the heart is repeatedly imaged at numerous time points during the cardiac cycle. Frequently, the temporal evolution of a certain region of interest such as the ventricles or the atria is highly relevant for clinical diagnosis. In this paper, we devise a novel approach that allows for an automatized propagation of an arbitrary region of interest (ROI) along the cardiac cycle from respective annotated ROIs provided by medical experts at two different points in time, most frequently at the end-systolic (ES) and the end-diastolic (ED) cardiac phases. At its core, a 3D TV- L1 -based optical flow algorithm computes the apparent motion of consecutive MRI images in forward and backward directions. Subsequently, the given terminal annotated masks are propagated by this bidirectional optical flow in 3D, which results, however, in improper initial estimates of the segmentation masks due to numerical inaccuracies. These initially propagated segmentation masks are then refined by a 3D U-Net-based convolutional neural network (CNN), which was trained to enforce consistency with the forward and backward warped masks using a novel loss function. Moreover, a penalization term in the loss function controls large deviations from the initial segmentation masks. This method is benchmarked both on a new dataset with annotated single ventricles containing patients with severe heart diseases and on a publicly available dataset with different annotated ROIs. We emphasize that our novel loss function enables fine-tuning the CNN on a single patient, thereby yielding state-of-the-art results along the complete cardiac cycle.

Deep learning-based assessment of CT markers of sarcopenia and myosteatosis for outcome assessment in patients with advanced pancreatic cancer after high-intensity focused ultrasound treatment.

OBJECTIVES: To evaluate the prognostic value of CT-based markers of sarcopenia and myosteatosis in comparison to the Eastern Cooperative Oncology Group (ECOG) score for survival of patients with advanced pancreatic cancer treated with high-intensity focused ultrasound (HIFU). MATERIALS AND METHODS: For 142 retrospective patients, the skeletal muscle index (SMI), skeletal muscle radiodensity (SMRD), fatty muscle fraction (FMF), and intermuscular fat fraction (IMFF) were determined on superior mesenteric artery level in pre-interventional CT. Each marker was tested for associations with sex, age, body mass index (BMI), and ECOG. The prognostic value of the markers was examined in Kaplan-Meier analyses with the log-rank test and in uni- and multivariable Cox proportional hazards (CPH) models. RESULTS: The following significant associations were observed: Male patients had higher BMI and SMI. Patients with lower ECOG had lower BMI and SMI. Patients with BMI lower than 21.8 kg/m2 (median) also showed lower SMI and IMFF. Patients younger than 63.3 years (median) were found to have higher SMRD, lower FMF, and lower IMFF. In the Kaplan-Meier analysis, significantly lower survival times were observed in patients with higher ECOG or lower SMI. Increased patient risk was observed for higher ECOG, lower BMI, and lower SMI in univariable CPH analyses for 1-, 2-, and 3-year survival. Multivariable CPH analysis for 1-year survival revealed increased patient risk for higher ECOG, lower SMI, lower IMFF, and higher FMF. In multivariable analysis for 2- and 3-year survival, only ECOG and FMF remained significant. CONCLUSION: CT-based markers of sarcopenia and myosteatosis show a prognostic value for assessment of survival in advanced pancreatic cancer patients undergoing HIFU therapy. CLINICAL RELEVANCE STATEMENT: The results indicate a greater role of myosteatosis for additional risk assessment beyond clinical scores, as only FMF was associated with long-term survival in multivariable CPH analyses along ECOG and also showed independence to ECOG in group analysis. KEY POINTS: • This study investigates the prognostic value of CT-based markers of sarcopenia and myosteatosis for patients with pancreatic cancer treated with high-intensity focused ultrasound. • Markers for sarcopenia and myosteatosis showed a prognostic value besides clinical assessment of the physical status by the Eastern Cooperative Oncology Group score. In contrast to muscle size measurements, the myosteatosis marker fatty muscle fraction demonstrated independence to the clinical score. • The results indicate that myosteatosis might play a greater role for additional patient risk assessments beyond clinical assessments of physical status.

Standards of perioperative management in total knee and hip arthroplasty procedures. A survey-based study. Part I: Preoperative management.

INTRODUCTION: Recent years have seen an increasing number of elective total knee (TKA) and hip arthroplasty (THA) procedures. Since a wide variety of methods and procedures are used in perioperative management, a survey-based study was carried out to identify the patterns of practice in Polish hospitals. MATERIAL AND METHODS: With the help of the LimeSurvey application, questionnaires for anaesthesio-logists and orthopaedists were prepared to gain insight into the preparation of patients for TKA and THA procedures and perioperative care. Questionnaires included both single and multiple-choice questions concerning among other things type of laboratory tests, additional examinations and consultations performed on a routine basis before elective TKA and THA procedures. RESULTS: A total of 162 medical centres took part in the study. Questionnaire responses were obtained from 93 (57%) orthopaedics teams and 112 (69%) anaesthesiology teams. A mean (standard deviation, SD) of 7.2 (3.5) laboratory tests are routinely ordered before surgery. For example, 47% of orthopaedists and 20% of anaesthesiologists order urinalysis, while 53% of orthopaedists and 26% of anaesthesiologists order a CRP test. Seventy-nine per cent of orthopaedists refer patients for at least one specialist consultation before the procedure. Dental consultation is requested by 40%, gynaecological consultation by 27%. Patient preoperative education is provided by 85% of orthopaedists and preoperative rehabilitation is prescribed by 46% of them. A total of 56% surveyed anaesthesiologists perform pre-anaesthetic evaluation upon patients' hospital admission. CONCLUSIONS: The study found that the number of examinations and specialist consultations conducted in Polish hospitals exceeded the scope of recommendations of scientific societies. Furthermore, the authors identified a need to standardise perioperative management in the form of Polish guidelines or recommendations, with the intention to improve patient safety and optimize health care expenses.

Opportunistic CT-derived analysis of fat and muscle tissue composition predicts mortality in patients with cardiogenic shock.

Prognosis estimation in patients with cardiogenic shock (CS) is important to guide clinical decision making. Aim of this study was to investigate the predictive value of opportunistic CT-derived body composition analysis in CS patients. Amount and density of fat and muscle tissue of 152 CS patients were quantified from single-slice CT images at the level of the intervertebral disc space L3/L4. Multivariable Cox regression and Kaplan-Meier survival analyses were performed to evaluate the predictive value of opportunistically CT-derived body composition parameters on the primary endpoint of 30-day mortality. Within the 30-day follow-up, 90/152 (59.2%) patients died. On multivariable analyses, lactate (Hazard Ratio 1.10 [95% Confidence Interval 1.04-1.17]; p = 0.002) and patient age (HR 1.04 [95% CI 1.01-1.07], p = 0.017) as clinical prognosticators, as well as visceral adipose tissue (VAT) area (HR 1.004 [95% CI 1.002-1.007]; p = 0.001) and skeletal muscle (SM) area (HR 0.987 [95% CI 0.975-0.999]; p = 0.043) as imaging biomarkers remained as independent predictors of 30-day mortality. Kaplan-Meier survival analyses showed significantly increased 30-day mortality in patients with higher VAT area (p = 0.015) and lower SM area (p = 0.035). CT-derived VAT and SM area are independent predictors of dismal outcomes in CS patients and have the potential to emerge as new imaging biomarkers available from routine diagnostic CT.

Development of image-based decision support systems utilizing information extracted from radiological free-text report databases with text-based transformers.

OBJECTIVES: To investigate the potential and limitations of utilizing transformer-based report annotation for on-site development of image-based diagnostic decision support systems (DDSS). METHODS: The study included 88,353 chest X-rays from 19,581 intensive care unit (ICU) patients. To label the presence of six typical findings in 17,041 images, the corresponding free-text reports of the attending radiologists were assessed by medical research assistants ("gold labels"). Automatically generated "silver" labels were extracted for all reports by transformer models trained on gold labels. To investigate the benefit of such silver labels, the image-based models were trained using three approaches: with gold labels only (MG), with silver labels first, then with gold labels (MS/G), and with silver and gold labels together (MS+G). To investigate the influence of invested annotation effort, the experiments were repeated with different numbers (N) of gold-annotated reports for training the transformer and image-based models and tested on 2099 gold-annotated images. Significant differences in macro-averaged area under the receiver operating characteristic curve (AUC) were assessed by non-overlapping 95% confidence intervals. RESULTS: Utilizing transformer-based silver labels showed significantly higher macro-averaged AUC than training solely with gold labels (N = 1000: MG 67.8 [66.0-69.6], MS/G 77.9 [76.2-79.6]; N = 14,580: MG 74.5 [72.8-76.2], MS/G 80.9 [79.4-82.4]). Training with silver and gold labels together was beneficial using only 500 gold labels (MS+G 76.4 [74.7-78.0], MS/G 75.3 [73.5-77.0]). CONCLUSIONS: Transformer-based annotation has potential for unlocking free-text report databases for the development of image-based DDSS. However, on-site development of image-based DDSS could benefit from more sophisticated annotation pipelines including further information than a single radiological report. CLINICAL RELEVANCE STATEMENT: Leveraging clinical databases for on-site development of artificial intelligence (AI)-based diagnostic decision support systems by text-based transformers could promote the application of AI in clinical practice by circumventing highly regulated data exchanges with third parties. KEY POINTS: • The amount of data from a database that can be used to develop AI-assisted diagnostic decision systems is often limited by the need for time-consuming identification of pathologies by radiologists. • The transformer-based structuring of free-text radiological reports shows potential to unlock corresponding image databases for on-site development of image-based diagnostic decision support systems. • However, the quality of image annotations generated solely on the content of a single radiology report may be limited by potential inaccuracies and incompleteness of this report.

Direct deep learning-based survival prediction from pre-interventional CT prior to transcatheter aortic valve replacement.

PURPOSE: To investigate survival prediction in patients undergoing transcatheter aortic valve replacement (TAVR) using deep learning (DL) methods applied directly to pre-interventional CT images and to compare performance with survival models based on scalar markers of body composition. METHOD: This retrospective single-center study included 760 patients undergoing TAVR (mean age 81 ± 6 years; 389 female). As a baseline, a Cox proportional hazards model (CPHM) was trained to predict survival on sex, age, and the CT body composition markers fatty muscle fraction (FMF), skeletal muscle radiodensity (SMRD), and skeletal muscle area (SMA) derived from paraspinal muscle segmentation of a single slice at L3/L4 level. The convolutional neural network (CNN) encoder of the DL model for survival prediction was pre-trained in an autoencoder setting with and without a focus on paraspinal muscles. Finally, a combination of DL and CPHM was evaluated. Performance was assessed by C-index and area under the receiver operating curve (AUC) for 1-year and 2-year survival. All methods were trained with five-fold cross-validation and were evaluated on 152 hold-out test cases. RESULTS: The CNN for direct image-based survival prediction, pre-trained in a focussed autoencoder scenario, outperformed the baseline CPHM (CPHM: C-index = 0.608, 1Y-AUC = 0.606, 2Y-AUC = 0.594 vs. DL: C-index = 0.645, 1Y-AUC = 0.687, 2Y-AUC = 0.692). Combining DL and CPHM led to further improvement (C-index = 0.668, 1Y-AUC = 0.713, 2Y-AUC = 0.696). CONCLUSIONS: Direct DL-based survival prediction shows potential to improve image feature extraction compared to segmentation-based scalar markers of body composition for risk assessment in TAVR patients.

Myosteatosis independently predicts transplant-free survival in patients with primary sclerosing cholangitis.

BACKGROUND: Primary Sclerosing Cholangitis (PSC) is a progressive cholestatic liver disease with liver transplantation (LT) as the only curative therapy. Some regions use body-weight-loss as standard-exception criteria for organ allocation but data on the impact of body composition on survival of patients with PSC is scarce. METHODS: Abdominal MRI of PSC patients were quantitatively analyzed for intramuscular fat fraction (IMFF) as surrogate of myosteatosis. Clinical and laboratory data were retrieved from patient records. Primary outcome was transplant-free survival (TFS). RESULTS: 116 PSC patients were included. Median age was 38 (18-71) years with 74 (64%) male patients. 15 (13%) patients had significant weigh loss. IMFF was significantly associated with survival. Multivariate regression analysis showed IMFF ≥ 15% as independent predictor for TFS (p = 0.032, HR 3.215 CI 1.104-9.366), but not significant weight loss (p = 0.618). CONCLUSION: IMFF is independently associated with TFS in patients with PSC and may identify patients with more urgent need for LT. NCT03584204.

Deep learning enables automated MRI-based estimation of uterine volume also in patients with uterine fibroids undergoing high-intensity focused ultrasound therapy.

BACKGROUND: High-intensity focused ultrasound (HIFU) is used for the treatment of symptomatic leiomyomas. We aim to automate uterine volumetry for tracking changes after therapy with a 3D deep learning approach. METHODS: A 3D nnU-Net model in the default setting and in a modified version including convolutional block attention modules (CBAMs) was developed on 3D T2-weighted MRI scans. Uterine segmentation was performed in 44 patients with routine pelvic MRI (standard group) and 56 patients with uterine fibroids undergoing ultrasound-guided HIFU therapy (HIFU group). Here, preHIFU scans (n = 56), postHIFU imaging maximum one day after HIFU (n = 54), and the last available follow-up examination (n = 53, days after HIFU: 420 ± 377) were included. The training was performed on 80% of the data with fivefold cross-validation. The remaining data were used as a hold-out test set. Ground truth was generated by a board-certified radiologist and a radiology resident. For the assessment of inter-reader agreement, all preHIFU examinations were segmented independently by both. RESULTS: High segmentation performance was already observed for the default 3D nnU-Net (mean Dice score = 0.95 ± 0.05) on the validation sets. Since the CBAM nnU-Net showed no significant benefit, the less complex default model was applied to the hold-out test set, which resulted in accurate uterus segmentation (Dice scores: standard group 0.92 ± 0.07; HIFU group 0.96 ± 0.02), which was comparable to the agreement between the two readers. CONCLUSIONS: This study presents a method for automatic uterus segmentation which allows a fast and consistent assessment of uterine volume. Therefore, this method could be used in the clinical setting for objective assessment of therapeutic response to HIFU therapy.

Deep learning for standardized, MRI-based quantification of subcutaneous and subfascial tissue volume for patients with lipedema and lymphedema.

OBJECTIVES: To contribute to a more in-depth assessment of shape, volume, and asymmetry of the lower extremities in patients with lipedema or lymphedema utilizing volume information from MR imaging. METHODS: A deep learning (DL) pipeline was developed including (i) localization of anatomical landmarks (femoral heads, symphysis, knees, ankles) and (ii) quality-assured tissue segmentation to enable standardized quantification of subcutaneous (SCT) and subfascial tissue (SFT) volumes. The retrospectively derived dataset for method development consisted of 45 patients (42 female, 44.2 ± 14.8 years) who underwent clinical 3D DIXON MR-lymphangiography examinations of the lower extremities. Five-fold cross-validated training was performed on 16,573 axial slices from 40 patients and testing on 2187 axial slices from 5 patients. For landmark detection, two EfficientNet-B1 convolutional neural networks (CNNs) were applied in an ensemble. One determines the relative foot-head position of each axial slice with respect to the landmarks by regression, the other identifies all landmarks in coronal reconstructed slices using keypoint detection. After landmark detection, segmentation of SCT and SFT was performed on axial slices employing a U-Net architecture with EfficientNet-B1 as encoder. Finally, the determined landmarks were used for standardized analysis and visualization of tissue volume, distribution, and symmetry, independent of leg length, slice thickness, and patient position. RESULTS: Excellent test results were observed for landmark detection (z-deviation = 4.5 ± 3.1 mm) and segmentation (Dice score: SCT = 0.989 ± 0.004, SFT = 0.994 ± 0.002). CONCLUSIONS: The proposed DL pipeline allows for standardized analysis of tissue volume and distribution and may assist in diagnosis of lipedema and lymphedema or monitoring of conservative and surgical treatments. KEY POINTS: • Efficient use of volume information that MRI inherently provides can be extracted automatically by deep learning and enables in-depth assessment of tissue volumes in lipedema and lymphedema. • The deep learning pipeline consisting of body part regression, keypoint detection, and quality-assured tissue segmentation provides detailed information about the volume, distribution, and asymmetry of lower extremity tissues, independent of leg length, slice thickness, and patient position.

Deep learning supports the differentiation of alcoholic and other-than-alcoholic cirrhosis based on MRI.

Although CT and MRI are standard procedures in cirrhosis diagnosis, differentiation of etiology based on imaging is not established. This proof-of-concept study explores the potential of deep learning (DL) to support imaging-based differentiation of the etiology of liver cirrhosis. This retrospective, monocentric study included 465 patients with confirmed diagnosis of (a) alcoholic (n = 221) and (b) other-than-alcoholic (n = 244) cirrhosis. Standard T2-weighted single-slice images at the caudate lobe level were randomly split for training with fivefold cross-validation (85%) and testing (15%), balanced for (a) and (b). After automated upstream liver segmentation, two different ImageNet pre-trained convolutional neural network (CNN) architectures (ResNet50, DenseNet121) were evaluated for classification of alcohol-related versus non-alcohol-related cirrhosis. The highest classification performance on test data was observed for ResNet50 with unfrozen pre-trained parameters, yielding an area under the receiver operating characteristic curve of 0.82 (95% confidence interval (CI) 0.71-0.91) and an accuracy of 0.75 (95% CI 0.64-0.85). An ensemble of both models did not lead to significant improvement in classification performance. This proof-of-principle study shows that deep-learning classifiers have the potential to aid in discriminating liver cirrhosis etiology based on standard MRI.

End-to-end automated body composition analyses with integrated quality control for opportunistic assessment of sarcopenia in CT.

OBJECTIVES: To develop a pipeline for automated body composition analysis and skeletal muscle assessment with integrated quality control for large-scale application in opportunistic imaging. METHODS: First, a convolutional neural network for extraction of a single slice at the L3/L4 lumbar level was developed on CT scans of 240 patients applying the nnU-Net framework. Second, a 2D competitive dense fully convolutional U-Net for segmentation of visceral and subcutaneous adipose tissue (VAT, SAT), skeletal muscle (SM), and subsequent determination of fatty muscle fraction (FMF) was developed on single CT slices of 1143 patients. For both steps, automated quality control was integrated by a logistic regression model classifying the presence of L3/L4 and a linear regression model predicting the segmentation quality in terms of Dice score. To evaluate the performance of the entire pipeline end-to-end, body composition metrics, and FMF were compared to manual analyses including 364 patients from two centers. RESULTS: Excellent results were observed for slice extraction (z-deviation = 2.46 ± 6.20 mm) and segmentation (Dice score for SM = 0.95 ± 0.04, VAT = 0.98 ± 0.02, SAT = 0.97 ± 0.04) on the dual-center test set excluding cases with artifacts due to metallic implants. No data were excluded for end-to-end performance analyses. With a restrictive setting of the integrated segmentation quality control, 39 of 364 patients were excluded containing 8 cases with metallic implants. This setting ensured a high agreement between manual and fully automated analyses with mean relative area deviations of ΔSM = 3.3 ± 4.1%, ΔVAT = 3.0 ± 4.7%, ΔSAT = 2.7 ± 4.3%, and ΔFMF = 4.3 ± 4.4%. CONCLUSIONS: This study presents an end-to-end automated deep learning pipeline for large-scale opportunistic assessment of body composition metrics and sarcopenia biomarkers in clinical routine. KEY POINTS: • Body composition metrics and skeletal muscle quality can be opportunistically determined from routine abdominal CT scans. • A pipeline consisting of two convolutional neural networks allows an end-to-end automated analysis. • Machine-learning-based quality control ensures high agreement between manual and automatic analysis.

Deep Learning-Based Body Composition Analysis Predicts Outcome in Melanoma Patients Treated with Immune Checkpoint Inhibitors.

Previous studies suggest an impact of body composition on outcome in melanoma patients. We aimed to determine the prognostic value of CT-based body composition assessment in patients receiving immune checkpoint inhibitor therapy for treatment of metastatic disease using a deep learning approach. One hundred seven patients with staging CT examinations prior to initiation of checkpoint inhibition between January 2013 and August 2019 were retrospectively evaluated. Using an automated deep learning-based body composition analysis pipeline, parameters for estimation of skeletal muscle mass (skeletal muscle index, SMI) and adipose tissue compartments (visceral adipose tissue index, VAI; subcutaneous adipose tissue index, SAI) were derived from staging CT. The cohort was binarized according to gender-specific median cut-off values. Patients below the median were defined as having low SMI, VAI, or SAI, respectively. The impact on outcome was assessed using the Kaplan-Meier method with log-rank tests. A multivariable logistic regression model was built to test the impact of body composition parameters on 3-year mortality. Patients with low SMI displayed significantly increased 1-year (25% versus 9%, p = 0.035), 2-year (32% versus 13%, p = 0.017), and 3-year mortality (38% versus 19%, p = 0.016). No significant differences with regard to adipose tissue compartments were observed (3-year mortality: VAI, p = 0.448; SAI, p = 0.731). On multivariable analysis, low SMI (hazard ratio (HR), 2.245; 95% confidence interval (CI), 1.005-5.017; p = 0.049), neutrophil-to-lymphocyte ratio (HR, 1.170; 95% CI, 1.076-1.273; p < 0.001), and Karnofsky index (HR, 0.965; 95% CI, 0.945-0.985; p = 0.001) remained as significant predictors of 3-year mortality. Lowered skeletal muscle index as an indicator of sarcopenia was associated with worse outcome in patients with metastatic melanoma receiving immune checkpoint inhibitor therapy.

Detection of liver cirrhosis in standard T2-weighted MRI using deep transfer learning.

OBJECTIVES: To investigate the diagnostic performance of deep transfer learning (DTL) to detect liver cirrhosis from clinical MRI. METHODS: The dataset for this retrospective analysis consisted of 713 (343 female) patients who underwent liver MRI between 2017 and 2019. In total, 553 of these subjects had a confirmed diagnosis of liver cirrhosis, while the remainder had no history of liver disease. T2-weighted MRI slices at the level of the caudate lobe were manually exported for DTL analysis. Data were randomly split into training, validation, and test sets (70%/15%/15%). A ResNet50 convolutional neural network (CNN) pre-trained on the ImageNet archive was used for cirrhosis detection with and without upstream liver segmentation. Classification performance for detection of liver cirrhosis was compared to two radiologists with different levels of experience (4th-year resident, board-certified radiologist). Segmentation was performed using a U-Net architecture built on a pre-trained ResNet34 encoder. Differences in classification accuracy were assessed by the χ2-test. RESULTS: Dice coefficients for automatic segmentation were above 0.98 for both validation and test data. The classification accuracy of liver cirrhosis on validation (vACC) and test (tACC) data for the DTL pipeline with upstream liver segmentation (vACC = 0.99, tACC = 0.96) was significantly higher compared to the resident (vACC = 0.88, p < 0.01; tACC = 0.91, p = 0.01) and to the board-certified radiologist (vACC = 0.96, p < 0.01; tACC = 0.90, p < 0.01). CONCLUSION: This proof-of-principle study demonstrates the potential of DTL for detecting cirrhosis based on standard T2-weighted MRI. The presented method for image-based diagnosis of liver cirrhosis demonstrated expert-level classification accuracy. KEY POINTS: • A pipeline consisting of two convolutional neural networks (CNNs) pre-trained on an extensive natural image database (ImageNet archive) enables detection of liver cirrhosis on standard T2-weighted MRI. • High classification accuracy can be achieved even without altering the pre-trained parameters of the convolutional neural networks. • Other abdominal structures apart from the liver were relevant for detection when the network was trained on unsegmented images.

Body composition analysis using CT and MRI: intra-individual intermodal comparison of muscle mass and myosteatosis.

Computed tomography (CT) and magnetic resonance imaging (MRI) can quantify muscle mass and quality. However, it is still unclear if CT and MRI derived measurements can be used interchangeable. In this prospective study, fifty consecutive participants of a cancer screening program underwent same day low-dose chest CT and MRI. Cross-sectional areas (CSA) of the paraspinal skeletal muscles were obtained. CT and MRI muscle fat infiltration (MFI) were assessed by mean radiodensity in Hounsfield units (HU) and proton density fat fraction (MRIPDFF), respectively. CSA and MFI were highly correlated between CT and MRI (CSA: r = 0.93, P < 0.001; MFI: r = - 0.90, P < 0.001). Mean CSA was higher in CT compared to MRI (46.6cm2 versus 43.0cm2; P = 0.05) without significance. Based on MRIPDFF, a linear regression model was established to directly estimate skeletal muscle fat content from CT. Bland-Altman plots showed a difference between measurements of - 0.5 cm2 to 7.6 cm2 and - 4.2% to 2.4% regarding measurements of CSA and MFI, respectively. In conclusion, the provided results indicate interchangeability of CT and MRI derived imaging biomarkers of skeletal muscle quantity and quality. Comparable to MRIPDFF, skeletal muscle fat content can be quantified from CT, which might have an impact of analyses in larger cohort studies, particularly in sarcopenia patients.

Fully Automated Segmentation of Connective Tissue Compartments for CT-Based Body Composition Analysis: A Deep Learning Approach.

OBJECTIVE: Body composition comprises prognostic information in patients with various malignancies and can be opportunistically determined from routine computed tomography (CT) scans. However, accurate assessment of patients with alterations, for example, due to ascites or anasarca, and accurate identification of intermuscular fat remain challenging. In this study, we aimed to develop a fully automated and highly accurate segmentation tool for connective tissue compartments from abdominal CT scans using the open-source Convolutional Neural Network (CNN) DeepMedic. MATERIALS AND METHODS: In this retrospective study, a CNN was developed using data of 1143 consecutive patients undergoing either preinterventional CT for transcatheter aortic valve implantation (TAVI) (82%) or diagnostic CT for liver cirrhosis with portosystemic shunting (PTSS) (18%). All analyses were performed on single-slice images at the L3/L4 level. The data were subdivided into subsets of training (70%), validation (15%), and test data (15%), balanced for TAVI and PTSS patients. To demonstrate the generalizability of the applied method with respect to nonspecific clinical routine data, the model with the highest performance in TAVI and PTSS patients was further tested on 100 randomly selected patients who underwent CT for routine diagnostic purposes at a hospital of maximum care, including critically ill patients. The applicability of the method to native CT examinations was additionally tested on 50 patients. RESULTS: Compared with the ground truth of the test data, the presented method achieved highly accurate segmentation results (subcutaneous adipose tissue [SAT], Dice score [DSC]: 0.98 ± 0.01; visceral adipose tissue [VAT], DSC: 0.96 ± 0.04; skeletal muscles [SM], DSC: 0.95 ± 0.02) and showed excellent generalizability on the routine CT diagnostic patients (SAT, DSC: 0.97 ± 0.04; VAT, DSC: 0.95 ± 0.05; SM, DSC: 0.95 ± 0.04) and also on native CT scans (SAT, DSC: 0.99 ± 0.01; VAT, DSC: 0.97 ± 0.03; SM, DSC: 0.97 ± 0.02). CONCLUSIONS: Fully automated determination of body composition based on CT can be performed with excellent results using the open-source CNN DeepMedic. The trained model is made usable for research by a deployable and sharable application.

Synchronization and Alignment of Follow-up Examinations: a Practical and Educational Approach Using the DICOM Reference Coordinate System.

This work presents an approach for synchronization and alignment of Digital Imaging and Communications in Medicine (DICOM) series from different studies that allows, e.g., easier reading of follow-up examinations. The proposed concept developed within the DICOM's patient-based reference coordinate system allows to synchronize all image data of two different studies/examinations based on a single registration. The most suitable DICOM series for registration could be set as default per protocol. Necessary basics regarding the DICOM standard and the used mathematical transformations are presented in an educative way to allow straightforward implementation in Picture Archiving And Communications Systems (PACS) and other DICOM tools. The proposed method for alignment of DICOM images is potentially also useful for various scientific tasks and machine-learning applications.