A foundation model for generalizable disease detection from retinal images.

Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

Detection of Nonexudative Macular Neovascularization on Structural OCT Images Using Vision Transformers.

Purpose: A deep learning model was developed to detect nonexudative macular neovascularization (neMNV) using OCT B-scans. Design: Retrospective review of a prospective, observational study. Participants: Normal control eyes and patients with age-related macular degeneration (AMD) with and without neMNV. Methods: Swept-source OCT angiography (SS-OCTA) imaging (PLEX Elite 9000, Carl Zeiss Meditec, Inc) was performed using the 6 × 6-mm scan pattern. Individual B-scans were annotated to distinguish between drusen and the double-layer sign (DLS) associated with the neMNV. The machine learning model was tested on a dataset graded by humans, and model performance was compared with the human graders. Main Outcome Measures: Intersection over Union (IoU) score was measured to evaluate segmentation network performance. Area under the receiver operating characteristic curve values, sensitivity, specificity, and positive predictive value (PPV) and negative predictive value (NPV) were measured to assess the performance of the final classification performance. Chance-corrected agreement between the algorithm and the human grader determinations was measured with Cohen's kappa. Results: A total of 251 eyes from 210 patients, including 182 eyes with DLS and 115 eyes with drusen, were used for model training. Of 125 500 B-scans, 6879 B-scans were manually annotated. A vision transformer segmentation model was built to extract DLS and drusen from B-scans. The extracted prediction masks from all B-scans in a volume were projected to an en face image, and an eye-level projection map was obtained for each eye. A binary classification algorithm was established to identify eyes with neMNV from the projection map. The algorithm achieved 82%, 90%, 79%, and 91% sensitivity, specificity, PPV, and NPV, respectively, on a separate test set of 100 eyes that were evaluated by human graders in a previous study. The area under the curve value was calculated as 0.91 (95% confidence interval, 0.85-0.98). The results of the algorithm showed excellent agreement with the senior human grader (kappa = 0.83, P < 0.001) and moderate agreement with the junior grader consensus (kappa = 0.54, P < 0.001). Conclusions: Our network (code is available at https://github.com/uw-biomedical-ml/double_layer_vit) was able to detect the presence of neMNV from structural B-scans alone by applying a purely transformer-based model.

Quantification and Predictors of OCT-Based Macular Curvature and Dome-Shaped Configuration: Results From the UK Biobank.

Purpose: To investigate macular curvature, including the evaluation of potential associations and the dome-shaped macular configuration, given the increasing myopia prevalence and expected associated macular malformations. Methods: The study included a total of 65,440 subjects with a mean age (± SD) of 57.3 ± 8.11 years with spectral-domain optical coherence tomography (OCT) data from a unique contemporary resource for the study of health and disease that recruited more than half a million people in the United Kingdom (UK Biobank). A deep learning model was used to segment the retinal pigment epithelium. The macular curvature of the OCT scans was calculated by polynomial fit and evaluated. Further, associations with demographic, functional, ocular, and infancy factors were examined. Results: The overall macular curvature values followed a Gaussian distribution with high inter-eye agreement. Although all of the investigated parameters, except maternal smoking, were associated with the curvature in a multilinear analysis, ethnicity and refractive error consistently revealed the most significant effect. The prevalence of a macular dome-shaped configuration was 4.8% overall, most commonly in Chinese subjects as well as hypermetropic eyes. An increasing frequency up to 22.0% was found toward high refractive error. Subretinal fluid was rarely found in these eyes. Conclusions: Macular curvature revealed associations with demographic, functional, ocular, and infancy factors, as well as increasing prevalence of a dome-shaped macular configuration in high refractive error including high myopia and hypermetropia. These findings imply different pathophysiologic processes that lead to macular development and might open new fields to future myopia and macula research.

Foveal Curvature and Its Associations in UK Biobank Participants.

Purpose: To examine whether sociodemographic, and ocular factors relate to optical coherence tomography (OCT)-derived foveal curvature (FC) in healthy individuals. Methods: We developed a deep learning model to quantify OCT-derived FC from 63,939 participants (age range, 39-70 years). Associations of FC with sociodemographic, and ocular factors were obtained using multilevel regression analysis (to allow for right and left eyes) adjusting for age, sex, ethnicity, height (model 1), visual acuity, spherical equivalent, corneal astigmatism, center point retinal thickness (CPRT), intraocular pressure (model 2), deprivation (Townsend index), higher education, annual income, and birth order (model 3). Fovea curvature was modeled as a z-score. Results: Males had on average steeper FC (0.077; 95% confidence interval [CI] 0.077-0.078) than females (0.068; 95% CI 0.068-0.069). Compared with whites, non-white individuals showed flatter FC, particularly those of black ethnicity. In black males, -0.80 standard deviation (SD) change when compared with whites (95% CI -0.89, -0.71; P 5.2e10-68). In black females, -0.70 SD change when compared with whites (95% CI -0.77, -0.63; p 2.3e10-93). Ocular factors (visual acuity, refractive status, and CPRT) showed a graded inverse association with FC that persisted after adjustment. Macular curvature showed a positive association with FC. Income showed a linear trend increase in males (P for linear trend = 0.005). Conclusions: We demonstrate marked differences in FC with ethnicity on the largest cohort studied for this purpose to date. Ocular factors showed a graded association with FC. Implementation of FC quantification in research and on the clinical setting can enhance the understanding of clinical macular phenotypes in health and disease.

Policy-Driven, Multimodal Deep Learning for Predicting Visual Fields from the Optic Disc and OCT Imaging.

PURPOSE: To develop and validate a deep learning (DL) system for predicting each point on visual fields (VFs) from disc and OCT imaging and derive a structure-function mapping. DESIGN: Retrospective, cross-sectional database study. PARTICIPANTS: A total of 6437 patients undergoing routine care for glaucoma in 3 clinical sites in the United Kingdom. METHODS: OCT and infrared reflectance (IR) optic disc imaging were paired with the closest VF within 7 days. EfficientNet B2 was used to train 2 single-modality DL models to predict each of the 52 sensitivity points on the 24-2 VF pattern. A policy DL model was designed and trained to fuse the 2 model predictions. MAIN OUTCOME MEASURES: Pointwise mean absolute error (PMAE). RESULTS: A total of 5078 imaging scans to VF pairs were used as a held-out test set to measure the final performance. The improvement in PMAE with the policy model was 0.485 (0.438, 0.533) decibels (dB) compared with the IR image of the disc alone and 0.060 (0.047, 0.073) dB with to the OCT alone. The improvement with the policy fusion model was statistically significant (P < 0.0001). Occlusion masking shows that the DL models learned the correct structure-function mapping in a data-driven, feature agnostic fashion. CONCLUSIONS: The multimodal, policy DL model performed the best; it provided explainable maps of its confidence in fusing data from single modalities and provides a pathway for probing the structure-function relationship in glaucoma.

Phytohormone-dependent plant defense signaling orchestrated by oral bacteria of the herbivore Spodoptera litura.

A vast array of herbivorous arthropods live with symbiotic microorganisms. However, little is known about the nature and functional mechanism of bacterial effects on plant defense responses towards herbivores. We explored the role of microbes present in extracts of oral secretion (OS) isolated from larvae of Spodoptera litura, a generalist herbivore, in phytohormone signaling-dependent defense responses in Arabidopsis thaliana (Arabidopsis). In response to mechanical damage (MD) with application of bacteria-free OS (OS- ) prepared by sterilization or filtration of OS, Arabidopsis leaves exhibited enhanced de novo synthesis of oxylipins, and induction of transcript abundance of the responsible genes, in comparison to those in leaves with MD + nonsterilized OS (OS+ ), indicating that OS bacteria serve as suppressors of these genes. By contrast, de novo synthesis/signaling of salicylic acid and signaling of abscisic acid were enhanced by OS bacteria. These signaling networks were cross-regulated by each other. Meta-analysis of OS bacteria identified 70 bacterial strains. Among them was Staphylococcus epidermidis, an anaerobic staphylococcus that was shown to contribute to the suppression/manipulation of phytohormone-dependent plant defense signaling. The presence of OS bacteria was consequently beneficial for S. litura larvae hosted by Brassicaceae.

Exploring a Structural Basis for Delayed Rod-Mediated Dark Adaptation in Age-Related Macular Degeneration Via Deep Learning.

Purpose: Delayed rod-mediated dark adaptation (RMDA) is a functional biomarker for incipient age-related macular degeneration (AMD). We used anatomically restricted spectral domain optical coherence tomography (SD-OCT) imaging data to localize de novo imaging features associated with and to test hypotheses about delayed RMDA. Methods: Rod intercept time (RIT) was measured in participants with and without AMD at 5 degrees from the fovea, and macular SD-OCT images were obtained. A deep learning model was trained with anatomically restricted information using a single representative B-scan through the fovea of each eye. Mean-occlusion masking was utilized to isolate the relevant imaging features. Results: The model identified hyporeflective outer retinal bands on macular SD-OCT associated with delayed RMDA. The validation mean standard error (MSE) registered to the foveal B-scan localized the lowest error to 0.5 mm temporal to the fovea center, within an overall low-error region across the rod-free zone and adjoining parafovea. Mean absolute error (MAE) on the test set was 4.71 minutes (8.8% of the dynamic range). Conclusions: We report a novel framework for imaging biomarker discovery using deep learning and demonstrate its ability to identify and localize a previously undescribed biomarker in retinal imaging. The hyporeflective outer retinal bands in central macula on SD-OCT demonstrate a structural basis for dysfunctional rod vision that correlates to published histopathologic findings. Translational Relevance: This agnostic approach to anatomic biomarker discovery strengthens the rationale for RMDA as an outcome measure in early AMD clinical trials, and also expands the utility of deep learning beyond automated diagnosis to fundamental discovery.

Estimating Retinal Sensitivity Using Optical Coherence Tomography With Deep-Learning Algorithms in Macular Telangiectasia Type 2.

Importance: As currently used, microperimetry is a burdensome clinical testing modality for testing retinal sensitivity requiring long testing times and trained technicians. Objective: To create a deep-learning network that could directly estimate function from structure de novo to provide an en face high-resolution map of estimated retinal sensitivity. Design, Setting, and Participants: A cross-sectional imaging study using data collected between January 1, 2016, and November 30, 2017, from the Natural History Observation and Registry of macular telangiectasia type 2 (MacTel) evaluated 38 participants with confirmed MacTel from 2 centers. Main Outcomes and Measures: Mean absolute error of estimated compared with observed retinal sensitivity. Observed retinal sensitivity was obtained with fundus-controlled perimetry (microperimetry). Estimates of retinal sensitivity were made with deep-learning models that learned on superpositions of high-resolution optical coherence tomography (OCT) scans and microperimetry results. Those predictions were used to create high-density en face sensitivity maps of the macula. Training, validation, and test sets were segregated at the patient level. Results: A total of 2499 microperimetry sensitivities were mapped onto 1708 OCT B-scans from 63 eyes of 38 patients (mean [SD] age, 74.3 [9.7] years; 15 men [39.5%]). The numbers of examples for our algorithm were 67 899 (103 053 after data augmentation) for training, 1695 for validation, and 1212 for testing. Mean absolute error results were 4.51 dB (95% CI, 4.36-4.65 dB) when using linear regression and 3.66 dB (95% CI, 3.53-3.78 dB) when using the LeNet model. Using a 49.9 million-variable deep-learning model, a mean absolute error of 3.36 dB (95% CI, 3.25-3.48 dB) of retinal sensitivity for validation and test was achieved. Correlation showed a high degree of agreement (Pearson correlation r = 0.78). By paired Wilcoxon rank sum test, our model significantly outperformed these 2 baseline models (P < .001). Conclusions and Relevance: High-resolution en face maps of estimated retinal sensitivities were created in eyes with MacTel. The maps were of unequalled resolution compared with microperimetry and were able to correctly delineate functionally healthy and impaired retina. This model may be useful to monitor structural and functional disease progression and has potential as an objective surrogate outcome measure in investigational trials.