Isolated retinal neovascularization in retinopathy of prematurity: clinical associations and prognostic implications.

OBJECTIVE: Isolated retinal neovascularization (IRNV) is a common finding in patients with stage 2 and 3 retinopathy of prematurity (ROP). This study aims to further classify the clinical course and significance of these lesions (previously described as "popcorn" based on clinical appearance) in patients with ROP as visualized with ultra-widefield optical coherence tomography (UWF-OCT). DESIGN: Single center, retrospective case series. PARTICIPANTS: Images were collected from 136 babies in the Oregon Health and Science University neonatal intensive care unit. METHODS: A prototype UWF-OCT device captured en face scans (>140°), which were reviewed for the presence of IRNV along with standard zone, stage, and plus classification. In a cross-sectional analysis we compared demographics and the clinical course of eyes with and without IRNV. Longitudinally, we compared ROP severity using a clinician-assigned vascular severity score (VSS) and compared the risk of progression among eyes with and without IRNV using multivariable logistic regression (MLR). MAIN OUTCOME MEASURES: Differences in clinical demographics and disease progression between patients with and without IRNV. RESULTS: Of the 136 patients, 60 developed stage 2 or worse ROP during their disease course, 22 of whom had IRNV visualized on UWF-OCT (37%). On average, patients with IRNV had lower birth weights (BW) (660.1g vs 916.8g, p = 0.001), gestational age (GA) (24.9 vs 26.1 weeks, p = 0.01), and were more likely to present with ROP in zone I (63.4% vs 15.8%, p < 0.001). They were also more likely to progress to stage 3 (68.2% vs 13.2%, p < 0.001) and receive treatment (54.5% vs 15.8%, p = 0.002). Eyes with IRNV had a higher peak VSS (5.61 vs 3.73, p < 0.001) and averaged a higher VSS throughout their disease course. On MLR, IRNV was independently associated with progression to stage 3 (p = 0.02) and requiring treatment (p = 0.03), controlling for GA, BW, and initial zone 1 disease. CONCLUSION: In this single center study, we found that IRNV occurs in higher risk babies and was an independent risk factor for ROP progression and treatment. These findings may have implications for OCT-based ROP classifications in the future.

Multinational External Validation of Autonomous Retinopathy of Prematurity Screening.

Importance: Retinopathy of prematurity (ROP) is a leading cause of blindness in children, with significant disparities in outcomes between high-income and low-income countries, due in part to insufficient access to ROP screening. Objective: To evaluate how well autonomous artificial intelligence (AI)-based ROP screening can detect more-than-mild ROP (mtmROP) and type 1 ROP. Design, Setting, and Participants: This diagnostic study evaluated the performance of an AI algorithm, trained and calibrated using 2530 examinations from 843 infants in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) study, on 2 external datasets (6245 examinations from 1545 infants in the Stanford University Network for Diagnosis of ROP [SUNDROP] and 5635 examinations from 2699 infants in the Aravind Eye Care Systems [AECS] telemedicine programs). Data were taken from 11 and 48 neonatal care units in the US and India, respectively. Data were collected from January 2012 to July 2021, and data were analyzed from July to December 2023. Exposures: An imaging processing pipeline was created using deep learning to autonomously identify mtmROP and type 1 ROP in eye examinations performed via telemedicine. Main Outcomes and Measures: The area under the receiver operating characteristics curve (AUROC) as well as sensitivity and specificity for detection of mtmROP and type 1 ROP at the eye examination and patient levels. Results: The prevalence of mtmROP and type 1 ROP were 5.9% (91 of 1545) and 1.2% (18 of 1545), respectively, in the SUNDROP dataset and 6.2% (168 of 2699) and 2.5% (68 of 2699) in the AECS dataset. Examination-level AUROCs for mtmROP and type 1 ROP were 0.896 and 0.985, respectively, in the SUNDROP dataset and 0.920 and 0.982 in the AECS dataset. At the cross-sectional examination level, mtmROP detection had high sensitivity (SUNDROP: mtmROP, 83.5%; 95% CI, 76.6-87.7; type 1 ROP, 82.2%; 95% CI, 81.2-83.1; AECS: mtmROP, 80.8%; 95% CI, 76.2-84.9; type 1 ROP, 87.8%; 95% CI, 86.8-88.7). At the patient level, all infants who developed type 1 ROP screened positive (SUNDROP: 100%; 95% CI, 81.4-100; AECS: 100%; 95% CI, 94.7-100) prior to diagnosis. Conclusions and Relevance: Where and when ROP telemedicine programs can be implemented, autonomous ROP screening may be an effective force multiplier for secondary prevention of ROP.

Genome-wide association identifies novel ROP risk loci in a multiethnic cohort.

We conducted a genome-wide association study (GWAS) in a multiethnic cohort of 920 at-risk infants for retinopathy of prematurity (ROP), a major cause of childhood blindness, identifying 1 locus at genome-wide significance level (p < 5×10-8) and 9 with significance of p < 5×10-6 for ROP ≥ stage 3. The most significant locus, rs2058019, reached genome-wide significance within the full multiethnic cohort (p = 4.96×10-9); Hispanic and European Ancestry infants driving the association. The lead single nucleotide polymorphism (SNP) falls in an intronic region within the Glioma-associated oncogene family zinc finger 3 (GLI3) gene. Relevance for GLI3 and other top-associated genes to human ocular disease was substantiated through in-silico extension analyses, genetic risk score analysis and expression profiling in human donor eye tissues. Thus, we identify a novel locus at GLI3 with relevance to retinal biology, supporting genetic susceptibilities for ROP risk with possible variability by race and ethnicity.

Applications of Deep Learning: Automated Assessment of Vascular Tortuosity in Mouse Models of Oxygen-Induced Retinopathy.

Objective: To develop a generative adversarial network (GAN) to segment major blood vessels from retinal flat-mount images from oxygen-induced retinopathy (OIR) and demonstrate the utility of these GAN-generated vessel segmentations in quantifying vascular tortuosity. Design: Development and validation of GAN. Subjects: Three datasets containing 1084, 50, and 20 flat-mount mice retina images with various stains used and ages at sacrifice acquired from previously published manuscripts. Methods: Four graders manually segmented major blood vessels from flat-mount images of retinas from OIR mice. Pix2Pix, a high-resolution GAN, was trained on 984 pairs of raw flat-mount images and manual vessel segmentations and then tested on 100 and 50 image pairs from a held-out and external test set, respectively. GAN-generated and manual vessel segmentations were then used as an input into a previously published algorithm (iROP-Assist) to generate a vascular cumulative tortuosity index (CTI) for 20 image pairs containing mouse eyes treated with aflibercept versus control. Main Outcome Measures: Mean dice coefficients were used to compare segmentation accuracy between the GAN-generated and manually annotated segmentation maps. For the image pairs treated with aflibercept versus control, mean CTIs were also calculated for both GAN-generated and manual vessel maps. Statistical significance was evaluated using Wilcoxon signed-rank tests (P ≤ 0.05 threshold for significance). Results: The dice coefficient for the GAN-generated versus manual vessel segmentations was 0.75 ± 0.27 and 0.77 ± 0.17 for the held-out test set and external test set, respectively. The mean CTI generated from the GAN-generated and manual vessel segmentations was 1.12 ± 0.07 versus 1.03 ± 0.02 (P = 0.003) and 1.06 ± 0.04 versus 1.01 ± 0.01 (P < 0.001), respectively, for eyes treated with aflibercept versus control, demonstrating that vascular tortuosity was rescued by aflibercept when quantified by GAN-generated and manual vessel segmentations. Conclusions: GANs can be used to accurately generate vessel map segmentations from flat-mount images. These vessel maps may be used to evaluate novel metrics of vascular tortuosity in OIR, such as CTI, and have the potential to accelerate research in treatments for ischemic retinopathies. Financial Disclosures: The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Oxygenation Fluctuations Associated with Severe Retinopathy of Prematurity: Insights from a Multimodal Deep Learning Approach.

Purpose: Retinopathy of prematurity (ROP) is one of the leading causes of blindness in children. Although the role of oxygen in the pathophysiology of ROP is well established, a precise understanding of the dynamic relationship between oxygen exposure ROP incidence and severity is lacking. The purpose of this study was to evaluate the correlation between time-dependent oxygen variables and the onset of ROP. Design: Retrospective cohort study. Participants: Two hundred thirty infants who were born at a single academic center and met the inclusion criteria were included. Infants are mainly born between January 2011 and October 2022. Methods: Patient data were extracted from electronic health records (EHRs), with sufficient time-dependent oxygen data. Clinical outcomes for ROP were recorded as none/mild or moderate/severe (defined as type II or worse). Mixed-effects linear models were used to compare the 2 groups in terms of dynamic oxygen variables, such as daily average and the coefficient of variation (COV) fraction of inspired oxygen (FiO2). Support vector machine (SVM) and long-short-term memory (LSTM)-based multimodal models were trained with fivefold cross-validation to predict which infants would develop moderate/severe ROP. Gestational age (GA), birth weight, and time-dependent oxygen variables were used to develop predictive models. Main Outcome Measures: Model cross-validation performance was evaluated by computing the mean area under the receiver operating characteristic (AUROC) curve, precision, recall, and F1 score. Results: We found that both daily average and COV of FiO2 were associated with more severe ROP (adjusted P < 0.001). With fivefold cross-validation, the multimodal LSTM models had higher performance than the best static models (SVM using GA and 3 average FiO2 features) and SVM models trained on GA alone (mean AUROC = 0.89 ± 0.04 vs. 0.86 ± 0.05 vs. 0.83 ± 0.04). Conclusions: The development of severe ROP might not only be influenced by oxygen exposure but also by its fluctuation, which provides direction for future study of pathophysiological factors associated with severe ROP development. Additionally, we demonstrated that multimodal neural networks can be a method to extract useful information from time-series data, which may be a valuable methodology for the investigation of other diseases using EHR data. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Genome-wide association identifies novel ROP risk loci in a multi-ethnic cohort.

We conducted a genome-wide association study (GWAS) in a multiethnic cohort of 920 at-risk infants for retinopathy of prematurity (ROP), a major cause of childhood blindness, identifying 2 loci at genome-wide significance level (p<5×10-8) and 7 at suggestive significance (p<5×10-6) for ROP ≥ stage 3. The most significant locus, rs2058019, reached genome-wide significance within the full multiethnic cohort (p=4.96×10-9); Hispanic and Caucasian infants driving the association. The lead single nucleotide polymorphism (SNP) falls in an intronic region within the Glioma-associated oncogene family zinc finger 3 (GLI3) gene. Relevance for GLI3 and other top-associated genes to human ocular disease was substantiated through in-silico extension analyses, genetic risk score analysis and expression profiling in human donor eye tissues. Thus, we report the largest ROP GWAS to date, identifying a novel locus at GLI3 with relevance to retinal biology supporting genetic susceptibilities for ROP risk with possible variability by race and ethnicity.

Efficacy of Smartphone-Based Telescreening for Retinopathy of Prematurity With and Without Artificial Intelligence in India.

Importance: Retinopathy of prematurity (ROP) telemedicine screening programs have been found to be effective, but they rely on widefield digital fundus imaging (WDFI) cameras, which are expensive, making them less accessible in low- to middle-income countries. Cheaper, smartphone-based fundus imaging (SBFI) systems have been described, but these have a narrower field of view (FOV) and have not been tested in a real-world, operational telemedicine setting. Objective: To assess the efficacy of SBFI systems compared with WDFI when used by technicians for ROP screening with both artificial intelligence (AI) and human graders. Design, Setting, and Participants: This prospective cross-sectional comparison study took place as a single-center ROP teleophthalmology program in India from January 2021 to April 2022. Premature infants who met normal ROP screening criteria and enrolled in the teleophthalmology screening program were included. Those who had already been treated for ROP were excluded. Exposures: All participants had WDFI images and from 1 of 2 SBFI devices, the Make-In-India (MII) Retcam or Keeler Monocular Indirect Ophthalmoscope (MIO) devices. Two masked readers evaluated zone, stage, plus, and vascular severity scores (VSS, from 1-9) in all images. Smartphone images were then stratified by patient into training (70%), validation (10%), and test (20%) data sets and used to train a ResNet18 deep learning architecture for binary classification of normal vs preplus or plus disease, which was then used for patient-level predictions of referral warranted (RW)- and treatment requiring (TR)-ROP. Main Outcome and Measures: Sensitivity and specificity of detection of RW-ROP, and TR-ROP by both human graders and an AI system and area under the receiver operating characteristic curve (AUC) of grader-assigned VSS. Sensitivity and specificity were compared between the 2 SBFI systems using Pearson χ2testing. Results: A total of 156 infants (312 eyes; mean [SD] gestational age, 33.0 [3.0] weeks; 75 [48%] female) were included with paired examinations. Sensitivity and specificity were not found to be statistically different between the 2 SBFI systems. Human graders were effective with SBFI at detecting TR-ROP with a sensitivity of 100% and specificity of 83.49%. The AUCs with grader-assigned VSS only were 0.95 (95% CI, 0.91-0.99) and 0.96 (95% CI, 0.93-0.99) for RW-ROP and TR-ROP, respectively. For the AI system, the sensitivity of detecting TR-ROP sensitivity was 100% with specificity of 58.6%, and RW-ROP sensitivity was 80.0% with specificity of 59.3%. Conclusions and Relevance: In this cross-sectional study, 2 different SBFI systems used by technicians in an ROP screening program were highly sensitive for TR-ROP. SBFI systems with AI may be a cost-effective method to improve the global capacity for ROP screening.

Association of Biomarker-Based Artificial Intelligence With Risk of Racial Bias in Retinal Images.

Importance: Although race is a social construct, it is associated with variations in skin and retinal pigmentation. Image-based medical artificial intelligence (AI) algorithms that use images of these organs have the potential to learn features associated with self-reported race (SRR), which increases the risk of racially biased performance in diagnostic tasks; understanding whether this information can be removed, without affecting the performance of AI algorithms, is critical in reducing the risk of racial bias in medical AI. Objective: To evaluate whether converting color fundus photographs to retinal vessel maps (RVMs) of infants screened for retinopathy of prematurity (ROP) removes the risk for racial bias. Design, Setting, and Participants: The retinal fundus images (RFIs) of neonates with parent-reported Black or White race were collected for this study. A u-net, a convolutional neural network (CNN) that provides precise segmentation for biomedical images, was used to segment the major arteries and veins in RFIs into grayscale RVMs, which were subsequently thresholded, binarized, and/or skeletonized. CNNs were trained with patients' SRR labels on color RFIs, raw RVMs, and thresholded, binarized, or skeletonized RVMs. Study data were analyzed from July 1 to September 28, 2021. Main Outcomes and Measures: Area under the precision-recall curve (AUC-PR) and area under the receiver operating characteristic curve (AUROC) at both the image and eye level for classification of SRR. Results: A total of 4095 RFIs were collected from 245 neonates with parent-reported Black (94 [38.4%]; mean [SD] age, 27.2 [2.3] weeks; 55 majority sex [58.5%]) or White (151 [61.6%]; mean [SD] age, 27.6 [2.3] weeks, 80 majority sex [53.0%]) race. CNNs inferred SRR from RFIs nearly perfectly (image-level AUC-PR, 0.999; 95% CI, 0.999-1.000; infant-level AUC-PR, 1.000; 95% CI, 0.999-1.000). Raw RVMs were nearly as informative as color RFIs (image-level AUC-PR, 0.938; 95% CI, 0.926-0.950; infant-level AUC-PR, 0.995; 95% CI, 0.992-0.998). Ultimately, CNNs were able to learn whether RFIs or RVMs were from Black or White infants regardless of whether images contained color, vessel segmentation brightness differences were nullified, or vessel segmentation widths were uniform. Conclusions and Relevance: Results of this diagnostic study suggest that it can be very challenging to remove information relevant to SRR from fundus photographs. As a result, AI algorithms trained on fundus photographs have the potential for biased performance in practice, even if based on biomarkers rather than raw images. Regardless of the methodology used for training AI, evaluating performance in relevant subpopulations is critical.

Epidemiologic Evaluation of Retinopathy of Prematurity Severity in a Large Telemedicine Program in India Using Artificial Intelligence.

PURPOSE: Epidemiological changes in retinopathy of prematurity (ROP) depend on neonatal care, neonatal mortality, and the ability to carefully titrate and monitor oxygen. We evaluate whether an artificial intelligence (AI) algorithm for assessing ROP severity in babies can be used to evaluate changes in disease epidemiology in babies from South India over a 5-year period. DESIGN: Retrospective cohort study. PARTICIPANTS: Babies (3093) screened for ROP at neonatal care units (NCUs) across the Aravind Eye Care System (AECS) in South India. METHODS: Images and clinical data were collected as part of routine tele-ROP screening at the AECS in India over 2 time periods: August 2015 to October 2017 and March 2019 to December 2020. All babies in the original cohort were matched 1:3 by birthweight (BW) and gestational age (GA) with babies in the later cohort. We compared the proportion of eyes with moderate (type 2) or treatment-requiring (TR) ROP, and an AI-derived ROP vascular severity score (from retinal fundus images) at the initial tele-retinal screening exam for all babies in a district, VSS), in the 2 time periods. MAIN OUTCOME MEASURES: Differences in the proportions of type 2 or worse and TR-ROP cases, and VSS between time periods. RESULTS: Among BW and GA matched babies, the proportion [95% confidence interval {CI}] of babies with type 2 or worse and TR-ROP decreased from 60.9% [53.8%-67.7%] to 17.1% [14.0%-20.5%] (P < 0.001) and 16.8% [11.9%-22.7%] to 5.1% [3.4%-7.3%] (P < 0.001), over the 2 time periods. Similarly, the median [interquartile range] VSS in the population decreased from 2.9 [1.2] to 2.4 [1.8] (P < 0.001). CONCLUSIONS: In South India, over a 5-year period, the proportion of babies developing moderate to severe ROP has dropped significantly for babies at similar demographic risk, strongly suggesting improvements in primary prevention of ROP. These results suggest that AI-based assessment of ROP severity may be a useful epidemiologic tool to evaluate temporal changes in ROP epidemiology. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Assessment of Retinopathy of Prematurity Regression and Reactivation Using an Artificial Intelligence-Based Vascular Severity Score.

Importance: One of the biggest challenges when using anti-vascular endothelial growth factor (VEGF) agents to treat retinopathy of prematurity (ROP) is the need to perform long-term follow-up examinations to identify eyes at risk of ROP reactivation requiring retreatment. Objective: To evaluate whether an artificial intelligence (AI)-based vascular severity score (VSS) can be used to analyze ROP regression and reactivation after anti-VEGF treatment and potentially identify eyes at risk of ROP reactivation requiring retreatment. Design, Setting, and Participants: This prognostic study was a secondary analysis of posterior pole fundus images collected during the multicenter, double-blind, investigator-initiated Comparing Alternative Ranibizumab Dosages for Safety and Efficacy in Retinopathy of Prematurity (CARE-ROP) randomized clinical trial, which compared 2 different doses of ranibizumab (0.12 mg vs 0.20 mg) for the treatment of ROP. The CARE-ROP trial screened and enrolled infants between September 5, 2014, and July 14, 2016. A total of 1046 wide-angle fundus images obtained from 19 infants at predefined study time points were analyzed. The analyses of VSS were performed between January 20, 2021, and November 18, 2022. Interventions: An AI-based algorithm assigned a VSS between 1 (normal) and 9 (most severe) to fundus images. Main Outcomes and Measures: Analysis of VSS in infants with ROP over time and VSS comparisons between the 2 treatment groups (0.12 mg vs 0.20 mg of ranibizumab) and between infants who did and did not receive retreatment for ROP reactivation. Results: Among 19 infants with ROP in the CARE-ROP randomized clinical trial, the median (range) postmenstrual age at first treatment was 36.4 (34.7-39.7) weeks; 10 infants (52.6%) were male, and 18 (94.7%) were White. The mean (SD) VSS was 6.7 (1.9) at baseline and significantly decreased to 2.7 (1.9) at week 1 (P < .001) and 2.9 (1.3) at week 4 (P < .001). The mean (SD) VSS of infants with ROP reactivation requiring retreatment was 6.5 (1.9) at the time of retreatment, which was significantly higher than the VSS at week 4 (P < .001). No significant difference was found in VSS between the 2 treatment groups, but the change in VSS between baseline and week 1 was higher for infants who later required retreatment (mean [SD], 7.8 [1.3] at baseline vs 1.7 [0.7] at week 1) vs infants who did not (mean [SD], 6.4 [1.9] at baseline vs 3.0 [2.0] at week 1). In eyes requiring retreatment, higher baseline VSS was correlated with earlier time of retreatment (Pearson r = -0.9997; P < .001). Conclusions and Relevance: In this study, VSS decreased after ranibizumab treatment, consistent with clinical disease regression. In cases of ROP reactivation requiring retreatment, VSS increased again to values comparable with baseline values. In addition, a greater change in VSS during the first week after initial treatment was found to be associated with a higher risk of later ROP reactivation, and high baseline VSS was correlated with earlier retreatment. These findings may have implications for monitoring ROP regression and reactivation after anti-VEGF treatment.

Evaluation of an Artificial Intelligence System for Retinopathy of Prematurity Screening in Nepal and Mongolia.

Purpose: To evaluate the performance of a deep learning (DL) algorithm for retinopathy of prematurity (ROP) screening in Nepal and Mongolia. Design: Retrospective analysis of prospectively collected clinical data. Participants: Clinical information and fundus images were obtained from infants in 2 ROP screening programs in Nepal and Mongolia. Methods: Fundus images were obtained using the Forus 3nethra neo (Forus Health) in Nepal and the RetCam Portable (Natus Medical, Inc.) in Mongolia. The overall severity of ROP was determined from the medical record using the International Classification of ROP (ICROP). The presence of plus disease was determined independently in each image using a reference standard diagnosis. The Imaging and Informatics for ROP (i-ROP) DL algorithm was trained on images from the RetCam to classify plus disease and to assign a vascular severity score (VSS) from 1 through 9. Main Outcome Measures: Area under the receiver operating characteristic curve and area under the precision-recall curve for the presence of plus disease or type 1 ROP and association between VSS and ICROP disease category. Results: The prevalence of type 1 ROP was found to be higher in Mongolia (14.0%) than in Nepal (2.2%; P < 0.001) in these data sets. In Mongolia (RetCam images), the area under the receiver operating characteristic curve for examination-level plus disease detection was 0.968, and the area under the precision-recall curve was 0.823. In Nepal (Forus images), these values were 0.999 and 0.993, respectively. The ROP VSS was associated with ICROP classification in both datasets (P < 0.001). At the population level, the median VSS was found to be higher in Mongolia (2.7; interquartile range [IQR], 1.3-5.4]) as compared with Nepal (1.9; IQR, 1.2-3.4; P < 0.001). Conclusions: These data provide preliminary evidence of the effectiveness of the i-ROP DL algorithm for ROP screening in neonatal populations in Nepal and Mongolia using multiple camera systems and are useful for consideration in future clinical implementation of artificial intelligence-based ROP screening in low- and middle-income countries.

Association of Optical Coherence Tomography-Measured Fibrovascular Ridge Thickness and Clinical Disease Stage in Retinopathy of Prematurity.

Importance: Accurate diagnosis of retinopathy of prematurity (ROP) is essential to provide timely treatment and reduce the risk of blindness. However, the components of an ROP examination are subjective and qualitative. Objective: To evaluate whether optical coherence tomography (OCT)-derived retinal thickness measurements at the vascular-avascular junction are associated with clinical diagnosis of ROP stage. Design, Setting, and Participants: This cross-sectional longitudinal study compared OCT-based ridge thickness calculated from OCT B-scans by a masked examiner to the clinical diagnosis of 2 masked examiners using both traditional stage classifications and a more granular continuous scale at the neonatal intensive care unit (NICU) of Oregon Health & Science University (OHSU) Hospital. Infants who met ROP screening criteria in the OHSU NICU between June 2021 and April 2022 and had guardian consent were included. One OCT volume and en face image per patient per eye showing at least 1 to 2 clock hours of ridge were included in the final analysis. Main Outcomes and Measures: Comparison of OCT-derived ridge thickness to the clinical diagnosis of ROP stage using an ordinal and continuous scale. Repeatability was assessed using 20 repeated examinations from the same visit and compared using intraclass correlation coefficient (ICC) and coefficient of variation (CV). Comparison of ridge thickness with ordinal categories was performed using generalized estimating equations and with continuous stage using Spearman correlation. Results: A total of 128 separate OCT eye examinations from 50 eyes of 25 patients were analyzed. The ICC was 0.87 with a CV of 7.0%. Higher ordinal disease classification was associated with higher axial ridge thickness on OCT, with mean (SD) thickness measurements of 264.2 (11.2) µm (P < .001), 334.2 (11.4) µm (P < .001), and 495.0 (32.2) µm (P < .001) for stages 1, 2, and 3, respectively and with continuous stage labels (ρ = 0.739, P < .001). Conclusions and Relevance: These results suggest that OCT-based quantification of peripheral stage in ROP may be an objective and quantitative biomarker that may be useful for clinical diagnosis and longitudinal monitoring and may have implications for disease classification in the future.

Synthetic Medical Images for Robust, Privacy-Preserving Training of Artificial Intelligence: Application to Retinopathy of Prematurity Diagnosis.

Purpose: Developing robust artificial intelligence (AI) models for medical image analysis requires large quantities of diverse, well-chosen data that can prove challenging to collect because of privacy concerns, disease rarity, or diagnostic label quality. Collecting image-based datasets for retinopathy of prematurity (ROP), a potentially blinding disease, suffers from these challenges. Progressively growing generative adversarial networks (PGANs) may help, because they can synthesize highly realistic images that may increase both the size and diversity of medical datasets. Design: Diagnostic validation study of convolutional neural networks (CNNs) for plus disease detection, a component of severe ROP, using synthetic data. Participants: Five thousand eight hundred forty-two retinal fundus images (RFIs) collected from 963 preterm infants. Methods: Retinal vessel maps (RVMs) were segmented from RFIs. PGANs were trained to synthesize RVMs with normal, pre-plus, or plus disease vasculature. Convolutional neural networks were trained, using real or synthetic RVMs, to detect plus disease from 2 real RVM test datasets. Main Outcome Measures: Features of real and synthetic RVMs were evaluated using uniform manifold approximation and projection (UMAP). Similarities were evaluated at the dataset and feature level using Fréchet inception distance and Euclidean distance, respectively. CNN performance was assessed via area under the receiver operating characteristic curve (AUC); AUCs were compared via bootstrapping and Delong's test for correlated receiver operating characteristic curves. Confusion matrices were compared using McNemar's chi-square test and Cohen's κ value. Results: The CNN trained on synthetic RVMs showed a significantly higher AUC (0.971; P = 0.006 and P = 0.004) and classified plus disease more similarly to a set of 8 international experts (κ = 0.922) than the CNN trained on real RVMs (AUC = 0.934; κ = 0.701). Real and synthetic RVMs overlapped, by plus disease diagnosis, on the UMAP manifold, showing that synthetic images spanned the disease severity spectrum. Fréchet inception distance and Euclidean distances suggested that real and synthetic RVMs were more dissimilar to one another than real RVMs were to one another, further suggesting that synthetic RVMs were distinct from the training data with respect to privacy considerations. Conclusions: Synthetic datasets may be useful for training robust medical AI models. Furthermore, PGANs may be able to synthesize realistic data for use without protected health information concerns.

External Validation of a Retinopathy of Prematurity Screening Model Using Artificial Intelligence in 3 Low- and Middle-Income Populations.

Importance: Retinopathy of prematurity (ROP) is a leading cause of preventable blindness that disproportionately affects children born in low- and middle-income countries (LMICs). In-person and telemedical screening examinations can reduce this risk but are challenging to implement in LMICs owing to the multitude of at-risk infants and lack of trained ophthalmologists. Objective: To implement an ROP risk model using retinal images from a single baseline examination to identify infants who will develop treatment-requiring (TR)-ROP in LMIC telemedicine programs. Design, Setting, and Participants: In this diagnostic study conducted from February 1, 2019, to June 30, 2021, retinal fundus images were collected from infants as part of an Indian ROP telemedicine screening program. An artificial intelligence (AI)-derived vascular severity score (VSS) was obtained from images from the first examination after 30 weeks' postmenstrual age. Using 5-fold cross-validation, logistic regression models were trained on 2 variables (gestational age and VSS) for prediction of TR-ROP. The model was externally validated on test data sets from India, Nepal, and Mongolia. Data were analyzed from October 20, 2021, to April 20, 2022. Main Outcomes and Measures: Primary outcome measures included sensitivity, specificity, positive predictive value, and negative predictive value for predictions of future occurrences of TR-ROP; the number of weeks before clinical diagnosis when a prediction was made; and the potential reduction in number of examinations required. Results: A total of 3760 infants (median [IQR] postmenstrual age, 37 [5] weeks; 1950 male infants [51.9%]) were included in the study. The diagnostic model had a sensitivity and specificity, respectively, for each of the data sets as follows: India, 100.0% (95% CI, 87.2%-100.0%) and 63.3% (95% CI, 59.7%-66.8%); Nepal, 100.0% (95% CI, 54.1%-100.0%) and 77.8% (95% CI, 72.9%-82.2%); and Mongolia, 100.0% (95% CI, 93.3%-100.0%) and 45.8% (95% CI, 39.7%-52.1%). With the AI model, infants with TR-ROP were identified a median (IQR) of 2.0 (0-11) weeks before TR-ROP diagnosis in India, 0.5 (0-2.0) weeks before TR-ROP diagnosis in Nepal, and 0 (0-5.0) weeks before TR-ROP diagnosis in Mongolia. If low-risk infants were never screened again, the population could be effectively screened with 45.0% (India, 664/1476), 38.4% (Nepal, 151/393), and 51.3% (Mongolia, 266/519) fewer examinations required. Conclusions and Relevance: Results of this diagnostic study suggest that there were 2 advantages to implementation of this risk model: (1) the number of examinations for low-risk infants could be reduced without missing cases of TR-ROP, and (2) high-risk infants could be identified and closely monitored before development of TR-ROP.

Variability in Plus Disease Diagnosis using Single and Serial Images.

PURPOSE: To assess changes in retinopathy of prematurity (ROP) diagnosis in single and serial retinal images. DESIGN: Cohort study. PARTICIPANTS: Cases of ROP recruited from the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) consortium evaluated by 7 graders. METHODS: Seven ophthalmologists reviewed both single and 3 consecutive serial retinal images from 15 cases with ROP, and severity was assigned as plus, preplus, or none. Imaging data were acquired during routine ROP screening from 2011 to 2015, and a reference standard diagnosis was established for each image. A secondary analysis was performed using the i-ROP deep learning system to assign a vascular severity score (VSS) to each image, ranging from 1 to 9, with 9 being the most severe disease. This score has been previously demonstrated to correlate with the International Classification of ROP. Mean plus disease severity was calculated by averaging 14 labels per image in serial and single images to decrease noise. MAIN OUTCOME MEASURES: Grading severity of ROP as defined by plus, preplus, or no ROP. RESULTS: Assessment of serial retinal images changed the grading severity for > 50% of the graders, although there was wide variability. Cohen's kappa ranged from 0.29 to 1.0, which showed a wide range of agreement from slight to perfect by each grader. Changes in the grading of serial retinal images were noted more commonly in cases of preplus disease. The mean severity in cases with a diagnosis of plus disease and no disease did not change between single and serial images. The ROP VSS demonstrated good correlation with the range of expert classifications of plus disease and overall agreement with the mode class (P = 0.001). The VSS correlated with mean plus disease severity by expert diagnosis (correlation coefficient, 0.89). The more aggressive graders tended to be influenced by serial images to increase the severity of their grading. The VSS also demonstrated agreement with disease progression across serial images, which progressed to preplus and plus disease. CONCLUSIONS: Clinicians demonstrated variability in ROP diagnosis when presented with both single and serial images. The use of deep learning as a quantitative assessment of plus disease has the potential to standardize ROP diagnosis and treatment.

Federated Learning for Multicenter Collaboration in Ophthalmology: Implications for Clinical Diagnosis and Disease Epidemiology.

OBJECTIVE: To utilize a deep learning (DL) model trained via federated learning (FL), a method of collaborative training without sharing patient data, to delineate institutional differences in clinician diagnostic paradigms and disease epidemiology in retinopathy of prematurity (ROP). DESIGN: Evaluation of a diagnostic test or technology. SUBJECTS AND CONTROLS: We included 5245 patients with wide-angle retinal imaging from the neonatal intensive care units of 7 institutions as part of the Imaging and Informatics in ROP study. Images were labeled with the clinical diagnoses of plus disease (plus, preplus, no plus), which were documented in the chart, and a reference standard diagnosis was determined by 3 image-based ROP graders and the clinical diagnosis. METHODS: Demographics (birth weight, gestational age) and clinical diagnoses for all eye examinations were recorded from each institution. Using an FL approach, a DL model for plus disease classification was trained using only the clinical labels. The 3 class probabilities were then converted into a vascular severity score (VSS) for each eye examination, as well as an "institutional VSS," in which the average of the VSS values assigned to patients' higher severity ("worse") eyes at each examination was calculated for each institution. MAIN OUTCOME MEASURES: We compared demographics, clinical diagnoses of plus disease, and institutional VSSs between institutions using the McNemar-Bowker test, 2-proportion Z test, and 1-way analysis of variance with post hoc analysis by the Tukey-Kramer test. Single regression analysis was performed to explore the relationship between demographics and VSSs. RESULTS: We found that the proportion of patients diagnosed with preplus disease varied significantly between institutions (P < 0.001). Using the DL-derived VSS trained on the data from all institutions using FL, we observed differences in the institutional VSS and the level of vascular severity diagnosed as no plus (P < 0.001) across institutions. A significant, inverse relationship between the institutional VSS and mean gestational age was found (P = 0.049, adjusted R2 = 0.49). CONCLUSIONS: A DL-derived ROP VSS developed without sharing data between institutions using FL identified differences in the clinical diagnoses of plus disease and overall levels of ROP severity between institutions. Federated learning may represent a method to standardize clinical diagnoses and provide objective measurements of disease for image-based diseases.

Federated Learning for Multicenter Collaboration in Ophthalmology: Improving Classification Performance in Retinopathy of Prematurity.

OBJECTIVE: To compare the performance of deep learning classifiers for the diagnosis of plus disease in retinopathy of prematurity (ROP) trained using 2 methods for developing models on multi-institutional data sets: centralizing data versus federated learning (FL) in which no data leave each institution. DESIGN: Evaluation of a diagnostic test or technology. SUBJECTS: Deep learning models were trained, validated, and tested on 5255 wide-angle retinal images in the neonatal intensive care units of 7 institutions as part of the Imaging and Informatics in ROP study. All images were labeled for the presence of plus, preplus, or no plus disease with a clinical label and a reference standard diagnosis (RSD) determined by 3 image-based ROP graders and the clinical diagnosis. METHODS: We compared the area under the receiver operating characteristic curve (AUROC) for models developed on multi-institutional data, using a central approach initially, followed by FL, and compared locally trained models with both approaches. We compared the model performance (κ) with the label agreement (between clinical and RSD), data set size, and number of plus disease cases in each training cohort using the Spearman correlation coefficient (CC). MAIN OUTCOME MEASURES: Model performance using AUROC and linearly weighted κ. RESULTS: Four settings of experiment were used: FL trained on RSD against central trained on RSD, FL trained on clinical labels against central trained on clinical labels, FL trained on RSD against central trained on clinical labels, and FL trained on clinical labels against central trained on RSD (P = 0.046, P = 0.126, P = 0.224, and P = 0.0173, respectively). Four of the 7 (57%) models trained on local institutional data performed inferiorly to the FL models. The model performance for local models was positively correlated with the label agreement (between clinical and RSD labels, CC = 0.389, P = 0.387), total number of plus cases (CC = 0.759, P = 0.047), and overall training set size (CC = 0.924, P = 0.002). CONCLUSIONS: We found that a trained FL model performs comparably to a centralized model, confirming that FL may provide an effective, more feasible solution for interinstitutional learning. Smaller institutions benefit more from collaboration than larger institutions, showing the potential of FL for addressing disparities in resource access.