Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning.

PURPOSE: Development and validation of a fully automated method to detect and quantify macular fluid in conventional OCT images. DESIGN: Development of a diagnostic modality. PARTICIPANTS: The clinical dataset for fluid detection consisted of 1200 OCT volumes of patients with neovascular age-related macular degeneration (AMD, n = 400), diabetic macular edema (DME, n = 400), or retinal vein occlusion (RVO, n = 400) acquired with Zeiss Cirrus (Carl Zeiss Meditec, Dublin, CA) (n = 600) or Heidelberg Spectralis (Heidelberg Engineering, Heidelberg, Germany) (n = 600) OCT devices. METHODS: A method based on deep learning to automatically detect and quantify intraretinal cystoid fluid (IRC) and subretinal fluid (SRF) was developed. The performance of the algorithm in accurately identifying fluid localization and extent was evaluated against a manual consensus reading of 2 masked reading center graders. MAIN OUTCOME MEASURES: Performance of a fully automated method to accurately detect, differentiate, and quantify intraretinal and SRF using area under the receiver operating characteristics curves, precision, and recall. RESULTS: The newly designed, fully automated diagnostic method based on deep learning achieved optimal accuracy for the detection and quantification of IRC for all 3 macular pathologies with a mean accuracy (AUC) of 0.94 (range, 0.91-0.97), a mean precision of 0.91, and a mean recall of 0.84. The detection and measurement of SRF were also highly accurate with an AUC of 0.92 (range, 0.86-0.98), a mean precision of 0.61, and a mean recall of 0.81, with superior performance in neovascular AMD and RVO compared with DME, which was represented rarely in the population studied. High linear correlation was confirmed between automated and manual fluid localization and quantification, yielding an average Pearson's correlation coefficient of 0.90 for IRC and of 0.96 for SRF. CONCLUSIONS: Deep learning in retinal image analysis achieves excellent accuracy for the differential detection of retinal fluid types across the most prevalent exudative macular diseases and OCT devices. Furthermore, quantification of fluid achieves a high level of concordance with manual expert assessment. Fully automated analysis of retinal OCT images from clinical routine provides a promising horizon in improving accuracy and reliability of retinal diagnosis for research and clinical practice in ophthalmology.

Musculoskeletal imaging in preventive medicine.

The aim is to review the modalities in musculoskeletal imaging with view on the prognostic impact for the patient's and for social outcome and with view on three major fields of preventive medicine: nutrition and metabolism, sports, and patient education. The added value provided by preventive imaging is (1) to monitor bone health and body composition with a broad spectrum of biomarkers, (2) to detect and quantify variants or abnormalities of nerves, muscles, tendons, bones, and joints with a risk of overuse, rupture, or fracture, and (3) to develop radiology reports from the widely used narrative format to structured text and multimedia datasets. The awareness problem is a term for describing the underreporting and the underdiagnosis of fragility fractures in osteoporosis.

A data-driven approach for the partial reconstruction of individual human molar teeth using generative deep learning.

Background and objective: Due to the high prevalence of dental caries, fixed dental restorations are regularly required to restore compromised teeth or replace missing teeth while retaining function and aesthetic appearance. The fabrication of dental restorations, however, remains challenging due to the complexity of the human masticatory system as well as the unique morphology of each individual dentition. Adaptation and reworking are frequently required during the insertion of fixed dental prostheses (FDPs), which increase cost and treatment time. This article proposes a data-driven approach for the partial reconstruction of occlusal surfaces based on a data set that comprises 92 3D mesh files of full dental crown restorations. Methods: A Generative Adversarial Network (GAN) is considered for the given task in view of its ability to represent extensive data sets in an unsupervised manner with a wide variety of applications. Having demonstrated good capabilities in terms of image quality and training stability, StyleGAN-2 has been chosen as the main network for generating the occlusal surfaces. A 2D projection method is proposed in order to generate 2D representations of the provided 3D tooth data set for integration with the StyleGAN architecture. The reconstruction capabilities of the trained network are demonstrated by means of 4 common inlay types using a Bayesian Image Reconstruction method. This involves pre-processing the data in order to extract the necessary information of the tooth preparations required for the used method as well as the modification of the initial reconstruction loss. Results: The reconstruction process yields satisfactory visual and quantitative results for all preparations with a root mean square error (RMSE) ranging from 0.02 mm to 0.18 mm. When compared against a clinical procedure for CAD inlay fabrication, the group of dentists preferred the GAN-based restorations for 3 of the total 4 inlay geometries. Conclusions: This article shows the effectiveness of the StyleGAN architecture with a downstream optimization process for the reconstruction of 4 different inlay geometries. The independence of the reconstruction process and the initial training of the GAN enables the application of the method for arbitrary inlay geometries without time-consuming retraining of the GAN.

Microaneurysm detection using high-speed megahertz optical coherence tomography angiography in advanced diabetic retinopathy.

PURPOSE: To compare detection rates of microaneurysms (MAs) on high-speed megahertz optical coherence tomography angiography (MHz-OCTA), fluorescein angiography (FA) and colour fundus photography (CF) in patients with diabetic retinopathy (DR). METHODS: For this exploratory cross-sectional study, MHz-OCTA data were acquired with a swept-source OCT prototype (A-scan rate: 1.7 MHz), and FA and CF imaging was performed using Optos® California. MA count was manually evaluated on en face MHz-OCTA/FA/CF images within an extended ETDRS grid. Detectability of MAs visible on FA images was evaluated on corresponding MHz-OCTA and CF images. MA distribution and leakage were correlated with detectability on OCTA and CF imaging. RESULTS: 47 eyes with severe DR (n = 12) and proliferative DR (n = 35) were included. MHz-OCTA and CF imaging detected on average 56% and 36% of MAs, respectively. MHz-OCTA detection rate was significantly higher than CF (p < 0.01). The combination of MHz-OCTA and CF leads to an increased detection rate of 70%. There was no statistically significant association between leakage and MA detectability on OCTA (p = 0.13). For CF, the odds of detecting leaking MAs were significantly lower than non-leaking MAs (p = 0.012). Using MHz-OCTA, detection of MAs outside the ETDRS grid was less likely than MAs located within the ETDRS grid (outer ring, p < 0.01; inner ring, p = 0.028). No statistically significant difference between rings was observed for CF measurements. CONCLUSIONS: More MAs were detected on MHz-OCTA than on CF imaging. Detection rate was lower for MAs located outside the macular region with MHz-OCTA and for leaking MAs with CF imaging. Combining both non-invasive modalities can improve MA detection.

Live 4D-OCT denoising with self-supervised deep learning.

By providing three-dimensional visualization of tissues and instruments at high resolution, live volumetric optical coherence tomography (4D-OCT) has the potential to revolutionize ophthalmic surgery. However, the necessary imaging speed is accompanied by increased noise levels. A high data rate and the requirement for minimal latency impose major limitations for real-time noise reduction. In this work, we propose a low complexity neural network for denoising, directly incorporated into the image reconstruction pipeline of a microscope-integrated 4D-OCT prototype with an A-scan rate of 1.2 MHz. For this purpose, we trained a blind-spot network on unpaired OCT images using a self-supervised learning approach. With an optimized U-Net, only a few milliseconds of additional latency were introduced. Simultaneously, these architectural adaptations improved the numerical denoising performance compared to the basic setup, outperforming non-local filtering algorithms. Layers and edges of anatomical structures in B-scans were better preserved than with Gaussian filtering despite comparable processing time. By comparing scenes with and without denoising employed, we show that neural networks can be used to improve visual appearance of volumetric renderings in real time. Enhancing the rendering quality is an important step for the clinical acceptance and translation of 4D-OCT as an intra-surgical guidance tool.

Association of Diabetic Lesions and Retinal Nonperfusion Using Widefield Multimodal Imaging.

PURPOSE: To evaluate the association of microvascular lesions on ultrawidefield (UWF) color fundus (CF) images with retinal nonperfusion (RNP) up to the midperiphery on single-capture widefield (WF) OCT angiography (OCTA) in patients with diabetic retinopathy (DR). DESIGN: Cross-sectional study. SUBJECTS: Seventy-five eyes of 50 patients with mild to severe nonproliferative DR (NPDR) and proliferative DR (PDR) were included in this analysis. METHODS: ETDRS level and presence of predominantly peripheral lesions (PPLs) were assessed on UWF-CF images acquired with a Zeiss Clarus 700. Single-capture 65°-WF-OCTA was performed using a PlexElite prototype (Carl Zeiss Meditec, Inc.). A custom grid consisting of a central ETDRS grid extended by 2 rings reaching up to the midperiphery was overlaid to subdivide retinal areas visible on WF-OCTA en face images. Retinal nonperfusion was measured in each area and in total. Nonperfusion index (NPI) was calculated from total RNP. On UWF-CF images, the number of microaneurysms, hemorrhages, neovascularizations, and areas with intraretinal microvascular abnormalities (IRMAs) were evaluated using the same grid. MAIN OUTCOME MEASURES: Association of diabetic lesions with RNP was calculated using Spearman correlations (rs). RESULTS: Median RNP on WF-OCTA was 0 mm2 (0-0.9), 4.9 mm2 (1.9-5.4), 23.4 mm2 (17.8-37), and 68.4 mm2 (40.8-91.7) in mild, moderate, and severe NPDR and PDR, respectively. We found a statistically significant correlation (P < 0.01) of overall RNP (rs = 0.96,) and NPI (rs = 0.97) on WF-OCTA with ETDRS level. Number of grid-fields affected by IRMAs on CF images was highly associated with NPI (rs = 0.86, P < 0.01). Intraretinal microvascular abnormalities and RNPs had similar topographic distributions with high correlations in affected areas. Eyes with PPLs (n = 43 eyes, 57%) on CF images had a significantly higher NPI (P = 0.014) than eyes without PPLs. CONCLUSION: The combination of UWF-CF imaging and single-capture WF-OCTA allows precise and noninvasive analysis of the retinal vasculature up to the midperiphery in patients with DR. The presence and extent of IRMAs on CF images may serve as an indicator for underlying RNP, which is more pronounced in eyes with PPLs. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Multiple instance learning based classification of diabetic retinopathy in weakly-labeled widefield OCTA en face images.

Diabetic retinopathy (DR), a pathologic change of the human retinal vasculature, is the leading cause of blindness in working-age adults with diabetes mellitus. Optical coherence tomography angiography (OCTA), a functional extension of optical coherence tomography, has shown potential as a tool for early diagnosis of DR through its ability to visualize the retinal vasculature in all spatial dimensions. Previously introduced deep learning-based classifiers were able to support the detection of DR in OCTA images, but require expert labeling at the pixel level, a labor-intensive and expensive process. We present a multiple instance learning-based network, MIL-ResNet,14 that is capable of detecting biomarkers in an OCTA dataset with high accuracy, without the need for annotations other than the information whether a scan is from a diabetic patient or not. The dataset we used for this study was acquired with a diagnostic ultra-widefield swept-source OCT device with a MHz A-scan rate. We were able to show that our proposed method outperforms previous state-of-the-art networks for this classification task, ResNet14 and VGG16. In addition, our network pays special attention to clinically relevant biomarkers and is robust against adversarial attacks. Therefore, we believe that it could serve as a powerful diagnostic decision support tool for clinical ophthalmic screening.

Detection of diabetic neovascularisation using single-capture 65°-widefield optical coherence tomography angiography.

AIM: To assess the detection rate of retinal neovascularisation (NV) in eyes with proliferative diabetic retinopathy (PDR) using widefield optical coherence tomography angiography (WF-OCTA) in comparison to ultrawidefield fluorescein angiography (UWF-FA). METHODS: Single-capture 65°-WF-OCTA-imaging was performed in patients with NV at the disc or elsewhere (NVE) detected on UWF-FA using a modified PlexElite system and B-scans were examined for blood flow signals breaching the internal limiting membrane. Sensitivity of WF-OCTA and UWF colour fundus (UWF-CF) photography for correct diagnosis of PDR was determined and interdevice agreement (Fleiss' κ) between WF-OCTA and UWF-FA for detection of NV in the total gradable area and each retinal quadrant was evaluated. RESULTS: Fifty-nine eyes of 41 patients with PDR detected on UWF-FA were included. Sensitivity of detecting PDR on WF-OCTA scans was 0.95 in contrast to 0.78 on UWF-CF images. Agreement in detecting NVE between WF-OCTA and UWF-FA was high in the superotemporal (κ=0.98) and inferotemporal (κ=0.94) and weak in the superonasal (κ=0.24) and inferonasal quadrants (κ=0.42). On UWF-FA, 63% of NVEs (n=153) were located in the temporal quadrants with 93% (n=142) of them being detected on WF-OCTA scans. CONCLUSION: The high reliability of non-invasive WF-OCTA imaging in detecting PDR can improve clinical examination with the potential to replace FA as a single diagnostic tool.

Deep learning detection and quantification of pneumothorax in heterogeneous routine chest computed tomography.

BACKGROUND: Automatically detecting and quantifying pneumothorax on chest computed tomography (CT) may impact clinical decision-making. Machine learning methods published so far struggle with the heterogeneity of technical parameters and the presence of additional pathologies, highlighting the importance of stable algorithms. METHODS: A deep residual UNet was developed and evaluated for automated, volume-level pneumothorax grading (i.e., labelling a volume whether a pneumothorax was present or not), and pixel-level classification (i.e., segmentation and quantification of pneumothorax), on a retrospective series of routine chest CT data. Ground truth annotations were provided by radiologists. The fully automated pixel-level pneumothorax segmentation method was trained using 43 chest CT scans and evaluated on 9 chest CT scans with pixel-level annotation basis and 567 chest CT scans on a volume-level basis. RESULTS: This method achieved a receiver operating characteristic area under the curve (AUC) of 0.98, an average precision of 0.97, and a Dice similarity coefficient (DSC) of 0.94. This segmentation performance resulted to be similar to the inter-rater segmentation accuracy of two radiologists, who achieved a DSC of 0.92. The comparison of manual and automated pneumothorax quantification yielded a Pearson correlation coefficient of 0.996. The volume-level pneumothorax grading accuracy was evaluated on 567 chest CT scans and yielded an AUC of 0.98 and an average precision of 0.95. CONCLUSIONS: We proposed a deep learning method for the detection and quantification of pneumothorax in heterogeneous routine clinical data that may facilitate the automated triage of urgent examinations and enable treatment decision support.

Multivariable Risk Prediction of Dysphagia in Hospitalized Patients Using Machine Learning.

BACKGROUND: Dysphagia is a dysfunction of the swallowing act and is highly prevalent in acute post-stroke patients and patients with chronic neurological diseases. Dysphagia is associated with several potentially life threatening complications. Thus, an early identification and treatment could reduce morbidity and mortality rates. OBJECTIVES: The aim of the study was to develop a multivariable model predicting the individual risk of dysphagia in hospitalized patients. METHODS: We trained different machine learning algorithms on the electronic health records of over 33,000 patients. RESULTS: The tree-based Random Forest Classifier and Adaboost Classifier algorithms achieved an area under the receiver operating characteristic curve of 0.94. CONCLUSION: The developed models outperformed previously published models predicting dysphagia. In future, an implementation in the clinical workflow is needed to determine the clinical benefit.

Correction to "Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT".

The authors of "Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT" which appeared in the January 2020 issue of this journal [1] would like to provide an updated Fig. 3 because there was an error in the published version. The output of the last convolutional layers says "2" in the number of channels but it should be "11" (10 retinal layer and the background).

Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT.

Diagnosis and treatment guidance are aided by detecting relevant biomarkers in medical images. Although supervised deep learning can perform accurate segmentation of pathological areas, it is limited by requiring a priori definitions of these regions, large-scale annotations, and a representative patient cohort in the training set. In contrast, anomaly detection is not limited to specific definitions of pathologies and allows for training on healthy samples without annotation. Anomalous regions can then serve as candidates for biomarker discovery. Knowledge about normal anatomical structure brings implicit information for detecting anomalies. We propose to take advantage of this property using Bayesian deep learning, based on the assumption that epistemic uncertainties will correlate with anatomical deviations from a normal training set. A Bayesian U-Net is trained on a well-defined healthy environment using weak labels of healthy anatomy produced by existing methods. At test time, we capture epistemic uncertainty estimates of our model using Monte Carlo dropout. A novel post-processing technique is then applied to exploit these estimates and transfer their layered appearance to smooth blob-shaped segmentations of the anomalies. We experimentally validated this approach in retinal optical coherence tomography (OCT) images, using weak labels of retinal layers. Our method achieved a Dice index of 0.789 in an independent anomaly test set of age-related macular degeneration (AMD) cases. The resulting segmentations allowed very high accuracy for separating healthy and diseased cases with late wet AMD, dry geographic atrophy (GA), diabetic macular edema (DME) and retinal vein occlusion (RVO). Finally, we qualitatively observed that our approach can also detect other deviations in normal scans such as cut edge artifacts.

f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks.

Obtaining expert labels in clinical imaging is difficult since exhaustive annotation is time-consuming. Furthermore, not all possibly relevant markers may be known and sufficiently well described a priori to even guide annotation. While supervised learning yields good results if expert labeled training data is available, the visual variability, and thus the vocabulary of findings, we can detect and exploit, is limited to the annotated lesions. Here, we present fast AnoGAN (f-AnoGAN), a generative adversarial network (GAN) based unsupervised learning approach capable of identifying anomalous images and image segments, that can serve as imaging biomarker candidates. We build a generative model of healthy training data, and propose and evaluate a fast mapping technique of new data to the GAN's latent space. The mapping is based on a trained encoder, and anomalies are detected via a combined anomaly score based on the building blocks of the trained model - comprising a discriminator feature residual error and an image reconstruction error. In the experiments on optical coherence tomography data, we compare the proposed method with alternative approaches, and provide comprehensive empirical evidence that f-AnoGAN outperforms alternative approaches and yields high anomaly detection accuracy. In addition, a visual Turing test with two retina experts showed that the generated images are indistinguishable from real normal retinal OCT images. The f-AnoGAN code is available at https://github.com/tSchlegl/f-AnoGAN.

Unsupervised Identification of Disease Marker Candidates in Retinal OCT Imaging Data.

The identification and quantification of markers in medical images is critical for diagnosis, prognosis, and disease management. Supervised machine learning enables the detection and exploitation of findings that are known a priori after annotation of training examples by experts. However, supervision does not scale well, due to the amount of necessary training examples, and the limitation of the marker vocabulary to known entities. In this proof-of-concept study, we propose unsupervised identification of anomalies as candidates for markers in retinal optical coherence tomography (OCT) imaging data without a constraint to a priori definitions. We identify and categorize marker candidates occurring frequently in the data and demonstrate that these markers show a predictive value in the task of detecting disease. A careful qualitative analysis of the identified data driven markers reveals how their quantifiable occurrence aligns with our current understanding of disease course, in early- and late age-related macular degeneration (AMD) patients. A multi-scale deep denoising autoencoder is trained on healthy images, and a one-class support vector machine identifies anomalies in new data. Clustering in the anomalies identifies stable categories. Using these markers to classify healthy-, early AMD- and late AMD cases yields an accuracy of 81.40%. In a second binary classification experiment on a publicly available data set (healthy versus intermediate AMD), the model achieves an area under the ROC curve of 0.944.

Machine Learning to Analyze the Prognostic Value of Current Imaging Biomarkers in Neovascular Age-Related Macular Degeneration.

PURPOSE: To evaluate the potential of machine learning to predict best-corrected visual acuity (BCVA) outcomes from structural and functional assessments during the initiation phase in patients receiving standardized ranibizumab therapy for neovascular age-related macular degeneration (AMD). DESIGN: Post hoc analysis of a randomized, prospective clinical trial. PARTICIPANTS: Data of 614 evaluable patients receiving intravitreal ranibizumab monthly or pro re nata according to protocol-specified criteria in the HARBOR trial. METHODS: Monthly spectral-domain (SD) OCT volume scans were processed by validated, fully automated computational image analysis. This system performs spatially resolved 3-dimensional segmentation of retinal layers, intraretinal cystoid fluid (IRF), subretinal fluid (SRF), and pigment epithelial detachments (PED). The extracted quantitative OCT biomarkers and BCVA measurements at baseline and months 1, 2, and 3 were used to predict BCVA at 12 months using random forest machine learning. This approach was also used to correlate OCT morphology to BCVA at baseline (structure-function correlation). MAIN OUTCOME MEASURES: Accuracy (R2) of the prediction models; ranking of input variables. RESULTS: Computational image analysis enabled fully automated quantitative characterization of neovascular lesions in a large-scale clinical SD-OCT data set. At baseline, OCT features and BCVA were correlated with R2 = 0.21. The most relevant biomarker for BCVA was the horizontal extension of IRF in the foveal region, whereas SRF and PED ranked low. In predicting functional outcomes, the model's accuracy increased in a linear fashion with each month. If only the baseline visit was considered, the accuracy was R2 = 0.34. At month 3, final visual acuity outcomes could be predicted with an accuracy of R2 = 0.70. The strongest predictive factor for functional outcomes at 1 year was consistently the individual BCVA level during the initiation phase. CONCLUSIONS: In this large-scale study based on a wide spectrum of morphologic and functional features, baseline BCVA correlated modestly with baseline SD-OCT, whereas functional outcomes were determined by BCVA levels during the initiation phase with a minor influence of fluid-related features. This finding suggests a re-evaluation of current diagnostic imaging features and a search for novel imaging approaches, where machine learning is a promising approach.

Analyzing and Predicting Visual Acuity Outcomes of Anti-VEGF Therapy by a Longitudinal Mixed Effects Model of Imaging and Clinical Data.

Purpose: We develop a longitudinal statistical model describing best-corrected visual acuity (BCVA) changes in anti-VEGF therapy in relation to imaging data, and predict the future BCVA outcome for individual patients by combining population-wide trends and initial subject-specific time points. Methods: Automatic segmentation algorithms were used to measure intraretinal (IRF) and subretinal (SRF) fluid volume on monthly spectral-domain optical coherence tomography scans of eyes with central retinal vein occlusion (CRVO) receiving standardized anti-VEGF treatment. The trajectory of BCVA over time was modeled as a multivariable repeated-measure mixed-effects regression model including fluid volumes as covariates. Subject-specific BCVA trajectories and final treatment outcomes were predicted using a population-wide model and individual observations from early follow-up. Results: A total of 193 eyes (one per patient, 12-month follow-up, 2420 visits) were analyzed. The population-wide mixed model revealed that the impact of fluid on BCVA is highest for IRF in the central millimeter around the fovea, with -31.17 letters/mm3 (95% confidence interval [CI], -39.70 to -23.32), followed by SRF in the central millimeter, with -17.50 letters/mm3 (-31.17 to -4.60) and by IRF in the parafovea, with -2.87 letters/mm3 (-4.71 to -0.44). The influence of SRF in the parafoveal area was -1.24 letters/mm3 (-3.37-1.05). The conditional R2 of the model, including subject-specific deviations, was 0.887. The marginal R2 considering the population-wide trend and fluid changes was 0.109. BCVA at 1 year could be predicted for an individual patient after three visits with a mean absolute error of six letters and a predicted R2 of 0.658 using imaging information. Conclusions: The mixed-effects model revealed that retinal fluid volumes and population-wide trend only explains a small proportion of the variation in BCVA. Individual BCVA outcomes after 1 year could be predicted from initial BCVA and fluid measurements combined with the population-wide model. Accounting for fluid in the predictive model increased prediction accuracy.

Computational image analysis for prognosis determination in DME.

In this pilot study, we evaluated the potential of computational image analysis of optical coherence tomography (OCT) data to determine the prognosis of patients with diabetic macular edema (DME). Spectral-domain OCT scans with fully automated retinal layer segmentation and segmentation of intraretinal cystoid fluid (IRC) and subretinal fluid of 629 patients receiving anti-vascular endothelial growth factor therapy for DME in a randomized prospective clinical trial were analyzed. The results were used to define 312 potentially predictive features at three timepoints (baseline, weeks 12 and 24) for best-corrected visual acuity (BCVA) at baseline and after one year used in a random forest prediction path. Preliminarily, IRC in the outer nuclear layer in the 3-mm area around the fovea seemed to have the greatest predictive value for BCVA at baseline, and IRC and the total retinal thickness in the 3-mm area at weeks 12 and 24 for BCVA after one year. The overall model accuracy was R2=0.21/0.23 (p<0.001). The outcomes of this pilot analysis highlight the great potential of the proposed machine-learning approach for large-scale image data analysis in DME and other retinal diseases.

Prediction of Anti-VEGF Treatment Requirements in Neovascular AMD Using a Machine Learning Approach.

Purpose: The purpose of this study was to predict low and high anti-VEGF injection requirements during a pro re nata (PRN) treatment, based on sets of optical coherence tomography (OCT) images acquired during the initiation phase in neovascular AMD. Methods: Two-year clinical trial data of subjects receiving PRN ranibizumab according to protocol specified criteria in the HARBOR study after three initial monthly injections were included. OCT images were analyzed at baseline, month 1, and month 2. Quantitative spatio-temporal features computed from automated segmentation of retinal layers and fluid-filled regions were used to describe the macular microstructure. In addition, best-corrected visual acuity and demographic characteristics were included. Patients were grouped into low and high treatment categories based on first and third quartile, respectively. Random forest classification was used to learn and predict treatment categories and was evaluated with cross-validation. Results: Of 317 evaluable subjects, 71 patients presented low (≤5), 176 medium, and 70 high (≥16) injection requirements during the PRN maintenance phase from month 3 to month 23. Classification of low and high treatment requirement subgroups demonstrated an area under the receiver operating characteristic curve of 0.7 and 0.77, respectively. The most relevant feature for prediction was subretinal fluid volume in the central 3 mm, with the highest predictive values at month 2. Conclusions: We proposed and evaluated a machine learning methodology to predict anti-VEGF treatment needs from OCT scans taken during treatment initiation. The results of this pilot study are an important step toward image-guided prediction of treatment intervals in the management of neovascular AMD.

Spatial Correspondence Between Intraretinal Fluid, Subretinal Fluid, and Pigment Epithelial Detachment in Neovascular Age-Related Macular Degeneration.

Purpose: To identify the spatial distribution of exudative features of choroidal neovascularization in neovascular age-related macular degeneration (nAMD) based on the localization of intraretinal cystoid fluid (IRC), subretinal fluid (SRF), and pigment-epithelial detachment (PED). Methods: This retrospective cross-sectional study included spectral-domain optical coherence tomography volume scans (6 × 6 mm) of 1341 patients with treatment-naïve nAMD. IRC, SRF, and PED were detected on a per-voxel basis using fully automated segmentation algorithms. Two subsets of 37 volumes each were manually segmented to validate the automated results. The spatial correspondence of components was quantified by computing proportions of IRC-, SRF-, or PED-presenting A-scans simultaneously affected by the respective other pathomorphologic components on a per-patient basis. The median across the population is reported. Odds ratios between pairs of lesions were calculated and tested for significance pixel wise. Results: Automated image segmentation was successful in 1182 optical coherence tomography volumes, yielding more than 61 million A-scans for analysis. Overall, 81% of eyes showed IRC, 95% showed SRF, and 92% showed PED. IRC-presenting A-scans also showed SRF in a median 2.5%, PED in 32.9%. Of the SRF-presenting A-scans, 0.3% demonstrated IRC, 1.4% PED. Of the PED-presenting A-scans, 5.2% contained IRC, 2.0% SRF. Similar patterns were observed in the manually segmented subsets and via pixel-wise odds ratio analysis. Conclusions: Automated analyses of large-scale datasets in a cross-sectional study of 1182 patients with active treatment-naïve nAMD demonstrated low spatial correlation of SRF with IRC and PED in contrast to increased colocalization of IRC and PED. These morphological associations may contribute to our understanding of functional deficits in nAMD.

Predicting Semantic Descriptions from Medical Images with Convolutional Neural Networks.

Learning representative computational models from medical imaging data requires large training data sets. Often, voxel-level annotation is unfeasible for sufficient amounts of data. An alternative to manual annotation, is to use the enormous amount of knowledge encoded in imaging data and corresponding reports generated during clinical routine. Weakly supervised learning approaches can link volume-level labels to image content but suffer from the typical label distributions in medical imaging data where only a small part consists of clinically relevant abnormal structures. In this paper we propose to use a semantic representation of clinical reports as a learning target that is predicted from imaging data by a convolutional neural network. We demonstrate how we can learn accurate voxel-level classifiers based on weak volume-level semantic descriptions on a set of 157 optical coherence tomography (OCT) volumes. We specifically show how semantic information increases classification accuracy for intraretinal cystoid fluid (IRC), subretinal fluid (SRF) and normal retinal tissue, and how the learning algorithm links semantic concepts to image content and geometry.