Enhancing NSCLC recurrence prediction with PET/CT habitat imaging, ctDNA, and integrative radiogenomics-blood insights.

While we recognize the prognostic importance of clinicopathological measures and circulating tumor DNA (ctDNA), the independent contribution of quantitative image markers to prognosis in non-small cell lung cancer (NSCLC) remains underexplored. In our multi-institutional study of 394 NSCLC patients, we utilize pre-treatment computed tomography (CT) and 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) to establish a habitat imaging framework for assessing regional heterogeneity within individual tumors. This framework identifies three PET/CT subtypes, which maintain prognostic value after adjusting for clinicopathologic risk factors including tumor volume. Additionally, these subtypes complement ctDNA in predicting disease recurrence. Radiogenomics analysis unveil the molecular underpinnings of these imaging subtypes, highlighting downregulation in interferon alpha and gamma pathways in the high-risk subtype. In summary, our study demonstrates that these habitat imaging subtypes effectively stratify NSCLC patients based on their risk levels for disease recurrence after initial curative surgery or radiotherapy, providing valuable insights for personalized treatment approaches.

Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept.

[18F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

Automated Cellular-Level Dual Global Fusion of Whole-Slide Imaging for Lung Adenocarcinoma Prognosis.

Histopathologic whole-slide images (WSI) are generally considered the gold standard for cancer diagnosis and prognosis. Survival prediction based on WSI has recently attracted substantial attention. Nevertheless, it remains a central challenge owing to the inherent difficulties of predicting patient prognosis and effectively extracting informative survival-specific representations from WSI with highly compounded gigapixels. In this study, we present a fully automated cellular-level dual global fusion pipeline for survival prediction. Specifically, the proposed method first describes the composition of different cell populations on WSI. Then, it generates dimension-reduced WSI-embedded maps, allowing for efficient investigation of the tumor microenvironment. In addition, we introduce a novel dual global fusion network to incorporate global and inter-patch features of cell distribution, which enables the sufficient fusion of different types and locations of cells. We further validate the proposed pipeline using The Cancer Genome Atlas lung adenocarcinoma dataset. Our model achieves a C-index of 0.675 (±0.05) in the five-fold cross-validation setting and surpasses comparable methods. Further, we extensively analyze embedded map features and survival probabilities. These experimental results manifest the potential of our proposed pipeline for applications using WSI in lung adenocarcinoma and other malignancies.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study.

BACKGROUND: Only around 20-30% of patients with non-small-cell lung cancer (NCSLC) have durable benefit from immune-checkpoint inhibitors. Although tissue-based biomarkers (eg, PD-L1) are limited by suboptimal performance, tissue availability, and tumour heterogeneity, radiographic images might holistically capture the underlying cancer biology. We aimed to investigate the application of deep learning on chest CT scans to derive an imaging signature of response to immune checkpoint inhibitors and evaluate its added value in the clinical context. METHODS: In this retrospective modelling study, 976 patients with metastatic, EGFR/ALK negative NSCLC treated with immune checkpoint inhibitors at MD Anderson and Stanford were enrolled from Jan 1, 2014, to Feb 29, 2020. We built and tested an ensemble deep learning model on pretreatment CTs (Deep-CT) to predict overall survival and progression-free survival after treatment with immune checkpoint inhibitors. We also evaluated the added predictive value of the Deep-CT model in the context of existing clinicopathological and radiological metrics. FINDINGS: Our Deep-CT model demonstrated robust stratification of patient survival of the MD Anderson testing set, which was validated in the external Stanford set. The performance of the Deep-CT model remained significant on subgroup analyses stratified by PD-L1, histology, age, sex, and race. In univariate analysis, Deep-CT outperformed the conventional risk factors, including histology, smoking status, and PD-L1 expression, and remained an independent predictor after multivariate adjustment. Integrating the Deep-CT model with conventional risk factors demonstrated significantly improved prediction performance, with overall survival C-index increases from 0·70 (clinical model) to 0·75 (composite model) during testing. On the other hand, the deep learning risk scores correlated with some radiomics features, but radiomics alone could not reach the performance level of deep learning, indicating that the deep learning model effectively captured additional imaging patterns beyond known radiomics features. INTERPRETATION: This proof-of-concept study shows that automated profiling of radiographic scans through deep learning can provide orthogonal information independent of existing clinicopathological biomarkers, bringing the goal of precision immunotherapy for patients with NSCLC closer. FUNDING: National Institutes of Health, Mark Foundation Damon Runyon Foundation Physician Scientist Award, MD Anderson Strategic Initiative Development Program, MD Anderson Lung Moon Shot Program, Andrea Mugnaini, and Edward L C Smith.

Habitat Imaging Biomarkers for Diagnosis and Prognosis in Cancer Patients Infected with COVID-19.

OBJECTIVES: Cancer patients have worse outcomes from the COVID-19 infection and greater need for ventilator support and elevated mortality rates than the general population. However, previous artificial intelligence (AI) studies focused on patients without cancer to develop diagnosis and severity prediction models. Little is known about how the AI models perform in cancer patients. In this study, we aim to develop a computational framework for COVID-19 diagnosis and severity prediction particularly in a cancer population and further compare it head-to-head to a general population. METHODS: We have enrolled multi-center international cohorts with 531 CT scans from 502 general patients and 420 CT scans from 414 cancer patients. In particular, the habitat imaging pipeline was developed to quantify the complex infection patterns by partitioning the whole lung regions into phenotypically different subregions. Subsequently, various machine learning models nested with feature selection were built for COVID-19 detection and severity prediction. RESULTS: These models showed almost perfect performance in COVID-19 infection diagnosis and predicting its severity during cross validation. Our analysis revealed that models built separately on the cancer population performed significantly better than those built on the general population and locked to test on the cancer population. This may be because of the significant difference among the habitat features across the two different cohorts. CONCLUSIONS: Taken together, our habitat imaging analysis as a proof-of-concept study has highlighted the unique radiologic features of cancer patients and demonstrated effectiveness of CT-based machine learning model in informing COVID-19 management in the cancer population.

Deep learning enabled brain shunt valve identification using mobile phones.

BACKGROUND AND OBJECTIVE: Accurate information concerning implanted medical devices prior to a Magnetic resonance imaging (MRI) examination is crucial to assure safety of the patient and to address MRI induced unintended changes in device settings. The identification of these devices still remains a very challenging task. In this paper, with the aim of providing a faster device detection, we propose the adoption of deep learning for medical device detection from X-rays. METHOD: In particular, we propose a pipeline for the identification of implanted programmable cerebrospinal fluid shunt valves using X-ray images of the radiologist workstation screens captured with mobile phone integrated cameras at different angles and illuminations. We compare the proposed convolutional neural network with published methods. RESULTS: Experimental results show that this approach outperforms methods trained on images digitally transferred directly from the scanners and then applied on mobile phones images (mean accuracy 95% vs 77%, Avg. Precision 0.96 vs 0.77, Avg. Recall 0.95 vs 0.77, Avg. F1-score 0.95 vs 0.77) and existing published methods based on transfer learning fine-tuned directly on the mobile phone images (mean accuracy 94% vs 75%, Avg. Precision 0.94 vs 0.75, Avg. Recall 0.94 vs 0.75, Avg. F1-score 0.94 vs 0.75). CONCLUSION: An automated shunt valve identification system is a promising safety tool for radiologists to efficiently coordinate the care of patients with implanted devices. An image-based safety system able to be deployed on a mobile phone would have significant advantages over methods requiring direct input from X-ray scanners or clinical picture archiving and communication system (PACS) in terms of ease of integration in the hospital or clinical ecosystems.

Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study.

OBJECTIVE: To investigate the performance of deep learning (DL) based on fully convolutional neural network (FCNN) in segmenting brain tissues in a large cohort of multiple sclerosis (MS) patients. METHODS: We developed a FCNN model to segment brain tissues, including T2-hyperintense MS lesions. The training, validation, and testing of FCNN were based on ~1000 magnetic resonance imaging (MRI) datasets acquired on relapsing-remitting MS patients, as a part of a phase 3 randomized clinical trial. Multimodal MRI data (dual-echo, FLAIR, and T1-weighted images) served as input to the network. Expert validated segmentation was used as the target for training the FCNN. We cross-validated our results using the leave-one-center-out approach. RESULTS: We observed a high average (95% confidence limits) Dice similarity coefficient for all the segmented tissues: 0.95 (0.92-0.98) for white matter, 0.96 (0.93-0.98) for gray matter, 0.99 (0.98-0.99) for cerebrospinal fluid, and 0.82 (0.63-1.0) for T2 lesions. High correlations between the DL segmented tissue volumes and ground truth were observed (R2 > 0.92 for all tissues). The cross validation showed consistent results across the centers for all tissues. CONCLUSION: The results from this large-scale study suggest that deep FCNN can automatically segment MS brain tissues, including lesions, with high accuracy.

Deep Learning for Predicting Enhancing Lesions in Multiple Sclerosis from Noncontrast MRI.

Background Enhancing lesions on MRI scans obtained after contrast material administration are commonly thought to represent disease activity in multiple sclerosis (MS); it is desirable to develop methods that can predict enhancing lesions without the use of contrast material. Purpose To evaluate whether deep learning can predict enhancing lesions on MRI scans obtained without the use of contrast material. Materials and Methods This study involved prospective analysis of existing MRI data. A convolutional neural network was used for classification of enhancing lesions on unenhanced MRI scans. This classification was performed for each slice, and the slice scores were combined by using a fully connected network to produce participant-wise predictions. The network input consisted of 1970 multiparametric MRI scans from 1008 patients recruited from 2005 to 2009. Enhanced lesions on postcontrast T1-weighted images served as the ground truth. The network performance was assessed by using fivefold cross-validation. Statistical analysis of the network performance included calculation of lesion detection rates and areas under the receiver operating characteristic curve (AUCs). Results MRI scans from 1008 participants (mean age, 37.7 years ± 9.7; 730 women) were analyzed. At least one enhancing lesion was observed in 519 participants. The sensitivity and specificity averaged across the five test sets were 78% ± 4.3 and 73% ± 2.7, respectively, for slice-wise prediction. The corresponding participant-wise values were 72% ± 9.0 and 70% ± 6.3. The diagnostic performances (AUCs) were 0.82 ± 0.02 and 0.75 ± 0.03 for slice-wise and participant-wise enhancement prediction, respectively. Conclusion Deep learning used with conventional MRI identified enhanced lesions in multiple sclerosis from images from unenhanced multiparametric MRI with moderate to high accuracy. © RSNA, 2019.

Deep-Learning-Based Neural Tissue Segmentation of MRI in Multiple Sclerosis: Effect of Training Set Size.

BACKGROUND: The dependence of deep-learning (DL)-based segmentation accuracy of brain MRI on the training size is not known. PURPOSE: To determine the required training size for a desired accuracy in brain MRI segmentation in multiple sclerosis (MS) using DL. STUDY TYPE: Retrospective analysis of MRI data acquired as part of a multicenter clinical trial. STUDY POPULATION: In all, 1008 patients with clinically definite MS. FIELD STRENGTH/SEQUENCE: MRIs were acquired at 1.5T and 3T scanners manufactured by GE, Philips, and Siemens with dual turbo spin echo, FLAIR, and T1 -weighted turbo spin echo sequences. ASSESSMENT: Segmentation results using an automated analysis pipeline and validated by two neuroimaging experts served as the ground truth. A DL model, based on a fully convolutional neural network, was trained separately using 16 different training sizes. The segmentation accuracy as a function of the training size was determined. These data were fitted to the learning curve for estimating the required training size for desired accuracy. STATISTICAL TESTS: The performance of the network was evaluated by calculating the Dice similarity coefficient (DSC), and lesion true-positive and false-positive rates. RESULTS: The DSC for lesions showed much stronger dependency on the sample size than gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF). When the training size was increased from 10 to 800 the DSC values varied from 0.00 to 0.86 ± 0.016 for T2 lesions, 0.87 ± 009 to 0.94 ± 0.004 for GM, 0.86 ± 0.08 to 0.94 ± 0.005 for WM, and 0.91 ± 0.009 to 0.96 ± 0.003 for CSF. DATA CONCLUSION: Excellent segmentation was achieved with a training size as small as 10 image volumes for GM, WM, and CSF. In contrast, a training size of at least 50 image volumes was necessary for adequate lesion segmentation. LEVEL OF EVIDENCE: 1 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:1487-1496.

Are multi-contrast magnetic resonance images necessary for segmenting multiple sclerosis brains? A large cohort study based on deep learning.

BACKGROUND: Magnetic resonance images with multiple contrasts or sequences are commonly used for segmenting brain tissues, including lesions, in multiple sclerosis (MS). However, acquisition of images with multiple contrasts increases the scan time and complexity of the analysis, possibly introducing factors that could compromise segmentation quality. OBJECTIVE: To investigate the effect of various combinations of multi-contrast images as input on the segmented volumes of gray (GM) and white matter (WM), cerebrospinal fluid (CSF), and lesions using a deep neural network. METHODS: U-net, a fully convolutional neural network was used to automatically segment GM, WM, CSF, and lesions in 1000 MS patients. The input to the network consisted of 15 combinations of FLAIR, T1-, T2-, and proton density-weighted images. The Dice similarity coefficient (DSC) was evaluated to assess the segmentation performance. For lesions, true positive rate (TPR) and false positive rate (FPR) were also evaluated. In addition, the effect of lesion size on lesion segmentation was investigated. RESULTS: Highest DSC was observed for all the tissue volumes, including lesions, when the input was combination of all four image contrasts. All other input combinations that included FLAIR also provided high DSC for all tissue classes. However, the quality of lesion segmentation showed strong dependence on the input images. The DSC and TPR values for inputs with the four contrast combination and FLAIR alone were very similar, but FLAIR showed a moderately higher FPR for lesion size <100 µl. For lesions smaller than 20 µl all image combinations resulted in poor performance. The segmentation quality improved with lesion size. CONCLUSIONS: Best performance for segmented tissue volumes was obtained with all four image contrasts as the input, and comparable performance was attainable with FLAIR only as the input, albeit with a moderate increase in FPR for small lesions. This implies that acquisition of only FLAIR images provides satisfactory tissue segmentation. Lesion segmentation was poor for very small lesions and improved rapidly with lesion size.

Automated image quality evaluation of structural brain MRI using an ensemble of deep learning networks.

BACKGROUND: Deep learning (DL) is a promising methodology for automatic detection of abnormalities in brain MRI. PURPOSE: To automatically evaluate the quality of multicenter structural brain MRI images using an ensemble DL model based on deep convolutional neural networks (DCNNs). STUDY TYPE: Retrospective. POPULATION: The study included 1064 brain images of autism patients and healthy controls from the Autism Brain Imaging Data Exchange (ABIDE) database. MRI data from 110 multiple sclerosis patients from the CombiRx study were included for independent testing. SEQUENCE: T1 -weighted MR brain images acquired at 3T. ASSESSMENT: The ABIDE data were separated into training (60%), validation (20%), and testing (20%) sets. The ensemble DL model combined the results from three cascaded networks trained separately on the three MRI image planes (axial, coronal, and sagittal). Each cascaded network consists of a DCNN followed by a fully connected network. The quality of image slices from each plane was evaluated by the DCNN and the resultant image scores were combined into a volumewise quality rating using the fully connected network. The DL predicted ratings were compared with manual quality evaluation by two experts. STATISTICAL TESTS: Receiver operating characteristic (ROC) curve, area under ROC curve (AUC), sensitivity, specificity, accuracy, and positive (PPV) and negative (NPV) predictive values. RESULTS: The AUC, sensitivity, specificity, accuracy, PPV, and NPV for image quality evaluation of the ABIDE test set using the ensemble model were 0.90, 0.77, 0.85, 0.84, 0.42, and 0.96, respectively. On the CombiRx set the same model achieved performance of 0.71, 0.41, 0.84, 0.73, 0.48, and 0.80. DATA CONCLUSION: This study demonstrated the high accuracy of DL in evaluating image quality of structural brain MRI in multicenter studies. LEVEL OF EVIDENCE: 3 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;50:1260-1267.