Synthetic OCT-A blood vessel maps using fundus images and generative adversarial networks.

Vessel segmentation in fundus images permits understanding retinal diseases and computing image-based biomarkers. However, manual vessel segmentation is a time-consuming process. Optical coherence tomography angiography (OCT-A) allows direct, non-invasive estimation of retinal vessels. Unfortunately, compared to fundus images, OCT-A cameras are more expensive, less portable, and have a reduced field of view. We present an automated strategy relying on generative adversarial networks to create vascular maps from fundus images without training using manual vessel segmentation maps. Further post-processing used for standard en face OCT-A allows obtaining a vessel segmentation map. We compare our approach to state-of-the-art vessel segmentation algorithms trained on manual vessel segmentation maps and vessel segmentations derived from OCT-A. We evaluate them from an automatic vascular segmentation perspective and as vessel density estimators, i.e., the most common imaging biomarker for OCT-A used in studies. Using OCT-A as a training target over manual vessel delineations yields improved vascular maps for the optic disc area and compares to the best-performing vessel segmentation algorithm in the macular region. This technique could reduce the cost and effort incurred when training vessel segmentation algorithms. To incentivize research in this field, we will make the dataset publicly available to the scientific community.

Detection of Stroke with Retinal Microvascular Density and Self-Supervised Learning Using OCT-A and Fundus Imaging.

Acute cerebral stroke is a leading cause of disability and death, which could be reduced with a prompt diagnosis during patient transportation to the hospital. A portable retina imaging system could enable this by measuring vascular information and blood perfusion in the retina and, due to the homology between retinal and cerebral vessels, infer if a cerebral stroke is underway. However, the feasibility of this strategy, the imaging features, and retina imaging modalities to do this are not clear. In this work, we show initial evidence of the feasibility of this approach by training machine learning models using feature engineering and self-supervised learning retina features extracted from OCT-A and fundus images to classify controls and acute stroke patients. Models based on macular microvasculature density features achieved an area under the receiver operating characteristic curve (AUC) of 0.87-0.88. Self-supervised deep learning models were able to generate features resulting in AUCs ranging from 0.66 to 0.81. While further work is needed for the final proof for a diagnostic system, these results indicate that microvasculature density features from OCT-A images have the potential to be used to diagnose acute cerebral stroke from the retina.

Towards Stroke Biomarkers on Fundus Retinal Imaging: A Comparison Between Vasculature Embeddings and General Purpose Convolutional Neural Networks.

Fundus Retinal imaging is an easy-to-acquire modality typically used for monitoring eye health. Current evidence indicates that the retina, and its vasculature in particular, is associated with other disease processes making it an ideal candidate for biomarker discovery. The development of these biomarkers has typically relied on predefined measurements, which makes the development process slow. Recently, representation learning algorithms such as general purpose convolutional neural networks or vasculature embeddings have been proposed as an approach to learn imaging biomarkers directly from the data, hence greatly speeding up their discovery. In this work, we compare and contrast different state-of-the-art retina biomarker discovery methods to identify signs of past stroke in the retinas of a curated patient cohort of 2,472 subjects from the UK Biobank dataset. We investigate two convolutional neural networks previously used in retina biomarker discovery and directly trained on the stroke outcome, and an extension of the vasculature embedding approach which infers its feature representation from the vasculature and combines the information of retinal images from both eyes.In our experiments, we show that the pipeline based on vasculature embeddings has comparable or better performance than other methods with a much more compact feature representation and ease of training.Clinical Relevance-This study compares and contrasts three retinal biomarker discovery strategies, using a curated dataset of subject evidence, for the analysis of the retina as a proxy in the assessment of clinical outcomes, such as stroke risk.

Deep learning enabled brain shunt valve identification using mobile phones.

BACKGROUND AND OBJECTIVE: Accurate information concerning implanted medical devices prior to a Magnetic resonance imaging (MRI) examination is crucial to assure safety of the patient and to address MRI induced unintended changes in device settings. The identification of these devices still remains a very challenging task. In this paper, with the aim of providing a faster device detection, we propose the adoption of deep learning for medical device detection from X-rays. METHOD: In particular, we propose a pipeline for the identification of implanted programmable cerebrospinal fluid shunt valves using X-ray images of the radiologist workstation screens captured with mobile phone integrated cameras at different angles and illuminations. We compare the proposed convolutional neural network with published methods. RESULTS: Experimental results show that this approach outperforms methods trained on images digitally transferred directly from the scanners and then applied on mobile phones images (mean accuracy 95% vs 77%, Avg. Precision 0.96 vs 0.77, Avg. Recall 0.95 vs 0.77, Avg. F1-score 0.95 vs 0.77) and existing published methods based on transfer learning fine-tuned directly on the mobile phone images (mean accuracy 94% vs 75%, Avg. Precision 0.94 vs 0.75, Avg. Recall 0.94 vs 0.75, Avg. F1-score 0.94 vs 0.75). CONCLUSION: An automated shunt valve identification system is a promising safety tool for radiologists to efficiently coordinate the care of patients with implanted devices. An image-based safety system able to be deployed on a mobile phone would have significant advantages over methods requiring direct input from X-ray scanners or clinical picture archiving and communication system (PACS) in terms of ease of integration in the hospital or clinical ecosystems.

Deep learning segmentation of gadolinium-enhancing lesions in multiple sclerosis.

OBJECTIVE: The aim of this study is to assess the performance of deep learning convolutional neural networks (CNNs) in segmenting gadolinium-enhancing lesions using a large cohort of multiple sclerosis (MS) patients. METHODS: A three-dimensional (3D) CNN model was trained for segmentation of gadolinium-enhancing lesions using multispectral magnetic resonance imaging data (MRI) from 1006 relapsing-remitting MS patients. The network performance was evaluated for three combinations of multispectral MRI used as input: (U5) fluid-attenuated inversion recovery (FLAIR), T2-weighted, proton density-weighted, and pre- and post-contrast T1-weighted images; (U2) pre- and post-contrast T1-weighted images; and (U1) only post-contrast T1-weighted images. Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and lesion-wise true-positive (TPR) and false-positive (FPR) rates. Performance was also evaluated as a function of enhancing lesion volume. RESULTS: The DSC/TPR/FPR values averaged over all the enhancing lesion sizes were 0.77/0.90/0.23 using the U5 model. These values for the largest enhancement volumes (>500 mm3) were 0.81/0.97/0.04. For U2, the average DSC/TPR/FPR values were 0.72/0.86/0.31. Comparable performance was observed with U1. For all types of input, the network performance degraded with decreased enhancement size. CONCLUSION: Excellent segmentation of enhancing lesions was observed for enhancement volume ⩾70 mm3. The best performance was achieved when the input included all five multispectral image sets.

Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study.

OBJECTIVE: To investigate the performance of deep learning (DL) based on fully convolutional neural network (FCNN) in segmenting brain tissues in a large cohort of multiple sclerosis (MS) patients. METHODS: We developed a FCNN model to segment brain tissues, including T2-hyperintense MS lesions. The training, validation, and testing of FCNN were based on ~1000 magnetic resonance imaging (MRI) datasets acquired on relapsing-remitting MS patients, as a part of a phase 3 randomized clinical trial. Multimodal MRI data (dual-echo, FLAIR, and T1-weighted images) served as input to the network. Expert validated segmentation was used as the target for training the FCNN. We cross-validated our results using the leave-one-center-out approach. RESULTS: We observed a high average (95% confidence limits) Dice similarity coefficient for all the segmented tissues: 0.95 (0.92-0.98) for white matter, 0.96 (0.93-0.98) for gray matter, 0.99 (0.98-0.99) for cerebrospinal fluid, and 0.82 (0.63-1.0) for T2 lesions. High correlations between the DL segmented tissue volumes and ground truth were observed (R2 > 0.92 for all tissues). The cross validation showed consistent results across the centers for all tissues. CONCLUSION: The results from this large-scale study suggest that deep FCNN can automatically segment MS brain tissues, including lesions, with high accuracy.

Deep Learning for Predicting Enhancing Lesions in Multiple Sclerosis from Noncontrast MRI.

Background Enhancing lesions on MRI scans obtained after contrast material administration are commonly thought to represent disease activity in multiple sclerosis (MS); it is desirable to develop methods that can predict enhancing lesions without the use of contrast material. Purpose To evaluate whether deep learning can predict enhancing lesions on MRI scans obtained without the use of contrast material. Materials and Methods This study involved prospective analysis of existing MRI data. A convolutional neural network was used for classification of enhancing lesions on unenhanced MRI scans. This classification was performed for each slice, and the slice scores were combined by using a fully connected network to produce participant-wise predictions. The network input consisted of 1970 multiparametric MRI scans from 1008 patients recruited from 2005 to 2009. Enhanced lesions on postcontrast T1-weighted images served as the ground truth. The network performance was assessed by using fivefold cross-validation. Statistical analysis of the network performance included calculation of lesion detection rates and areas under the receiver operating characteristic curve (AUCs). Results MRI scans from 1008 participants (mean age, 37.7 years ± 9.7; 730 women) were analyzed. At least one enhancing lesion was observed in 519 participants. The sensitivity and specificity averaged across the five test sets were 78% ± 4.3 and 73% ± 2.7, respectively, for slice-wise prediction. The corresponding participant-wise values were 72% ± 9.0 and 70% ± 6.3. The diagnostic performances (AUCs) were 0.82 ± 0.02 and 0.75 ± 0.03 for slice-wise and participant-wise enhancement prediction, respectively. Conclusion Deep learning used with conventional MRI identified enhanced lesions in multiple sclerosis from images from unenhanced multiparametric MRI with moderate to high accuracy. © RSNA, 2019.

Deep-Learning-Based Neural Tissue Segmentation of MRI in Multiple Sclerosis: Effect of Training Set Size.

BACKGROUND: The dependence of deep-learning (DL)-based segmentation accuracy of brain MRI on the training size is not known. PURPOSE: To determine the required training size for a desired accuracy in brain MRI segmentation in multiple sclerosis (MS) using DL. STUDY TYPE: Retrospective analysis of MRI data acquired as part of a multicenter clinical trial. STUDY POPULATION: In all, 1008 patients with clinically definite MS. FIELD STRENGTH/SEQUENCE: MRIs were acquired at 1.5T and 3T scanners manufactured by GE, Philips, and Siemens with dual turbo spin echo, FLAIR, and T1 -weighted turbo spin echo sequences. ASSESSMENT: Segmentation results using an automated analysis pipeline and validated by two neuroimaging experts served as the ground truth. A DL model, based on a fully convolutional neural network, was trained separately using 16 different training sizes. The segmentation accuracy as a function of the training size was determined. These data were fitted to the learning curve for estimating the required training size for desired accuracy. STATISTICAL TESTS: The performance of the network was evaluated by calculating the Dice similarity coefficient (DSC), and lesion true-positive and false-positive rates. RESULTS: The DSC for lesions showed much stronger dependency on the sample size than gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF). When the training size was increased from 10 to 800 the DSC values varied from 0.00 to 0.86 ± 0.016 for T2 lesions, 0.87 ± 009 to 0.94 ± 0.004 for GM, 0.86 ± 0.08 to 0.94 ± 0.005 for WM, and 0.91 ± 0.009 to 0.96 ± 0.003 for CSF. DATA CONCLUSION: Excellent segmentation was achieved with a training size as small as 10 image volumes for GM, WM, and CSF. In contrast, a training size of at least 50 image volumes was necessary for adequate lesion segmentation. LEVEL OF EVIDENCE: 1 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:1487-1496.

Are multi-contrast magnetic resonance images necessary for segmenting multiple sclerosis brains? A large cohort study based on deep learning.

BACKGROUND: Magnetic resonance images with multiple contrasts or sequences are commonly used for segmenting brain tissues, including lesions, in multiple sclerosis (MS). However, acquisition of images with multiple contrasts increases the scan time and complexity of the analysis, possibly introducing factors that could compromise segmentation quality. OBJECTIVE: To investigate the effect of various combinations of multi-contrast images as input on the segmented volumes of gray (GM) and white matter (WM), cerebrospinal fluid (CSF), and lesions using a deep neural network. METHODS: U-net, a fully convolutional neural network was used to automatically segment GM, WM, CSF, and lesions in 1000 MS patients. The input to the network consisted of 15 combinations of FLAIR, T1-, T2-, and proton density-weighted images. The Dice similarity coefficient (DSC) was evaluated to assess the segmentation performance. For lesions, true positive rate (TPR) and false positive rate (FPR) were also evaluated. In addition, the effect of lesion size on lesion segmentation was investigated. RESULTS: Highest DSC was observed for all the tissue volumes, including lesions, when the input was combination of all four image contrasts. All other input combinations that included FLAIR also provided high DSC for all tissue classes. However, the quality of lesion segmentation showed strong dependence on the input images. The DSC and TPR values for inputs with the four contrast combination and FLAIR alone were very similar, but FLAIR showed a moderately higher FPR for lesion size <100 µl. For lesions smaller than 20 µl all image combinations resulted in poor performance. The segmentation quality improved with lesion size. CONCLUSIONS: Best performance for segmented tissue volumes was obtained with all four image contrasts as the input, and comparable performance was attainable with FLAIR only as the input, albeit with a moderate increase in FPR for small lesions. This implies that acquisition of only FLAIR images provides satisfactory tissue segmentation. Lesion segmentation was poor for very small lesions and improved rapidly with lesion size.

Automated image quality evaluation of structural brain MRI using an ensemble of deep learning networks.

BACKGROUND: Deep learning (DL) is a promising methodology for automatic detection of abnormalities in brain MRI. PURPOSE: To automatically evaluate the quality of multicenter structural brain MRI images using an ensemble DL model based on deep convolutional neural networks (DCNNs). STUDY TYPE: Retrospective. POPULATION: The study included 1064 brain images of autism patients and healthy controls from the Autism Brain Imaging Data Exchange (ABIDE) database. MRI data from 110 multiple sclerosis patients from the CombiRx study were included for independent testing. SEQUENCE: T1 -weighted MR brain images acquired at 3T. ASSESSMENT: The ABIDE data were separated into training (60%), validation (20%), and testing (20%) sets. The ensemble DL model combined the results from three cascaded networks trained separately on the three MRI image planes (axial, coronal, and sagittal). Each cascaded network consists of a DCNN followed by a fully connected network. The quality of image slices from each plane was evaluated by the DCNN and the resultant image scores were combined into a volumewise quality rating using the fully connected network. The DL predicted ratings were compared with manual quality evaluation by two experts. STATISTICAL TESTS: Receiver operating characteristic (ROC) curve, area under ROC curve (AUC), sensitivity, specificity, accuracy, and positive (PPV) and negative (NPV) predictive values. RESULTS: The AUC, sensitivity, specificity, accuracy, PPV, and NPV for image quality evaluation of the ABIDE test set using the ensemble model were 0.90, 0.77, 0.85, 0.84, 0.42, and 0.96, respectively. On the CombiRx set the same model achieved performance of 0.71, 0.41, 0.84, 0.73, 0.48, and 0.80. DATA CONCLUSION: This study demonstrated the high accuracy of DL in evaluating image quality of structural brain MRI in multicenter studies. LEVEL OF EVIDENCE: 3 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;50:1260-1267.