P Score: A Reference Image-Based Clinical Grading Scale for Vascular Change in Retinopathy of Prematurity.

PURPOSE: The International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), acknowledged that plus-like retinopathy of prematurity (ROP) vascular changes occurs along a spectrum. Historically, clinician-experts demonstrate variable agreement for plus diagnosis. We developed a 9-photograph reference image set for grading plus-like changes and compared intergrader agreement of the set with standard grading with no plus, preplus, and plus disease. DESIGN: Retinal photographic grading and expert consensus opinion. PARTICIPANTS: The development set included 34 international ICROP3 committee members. The validation set included 30 ophthalmologists with ROP expertise (15 ICROP3 committee members and 15 non-ICROP3 members) METHODS: Nine ROP fundus images (P1 through P9) representing increasing degrees of zone I vascular tortuosity and dilation, based on the 34 ICROP3 committee members' gradings and consensus image reviews, were used to establish standard photographs for the plus (P) score. Study participants graded 150 fundus photographs 2 ways, separated by a 1-week washout period: (1) no plus, preplus, or plus disease and (2) choosing the closest P score image. MAIN OUTCOME MEASURES: Intergrader agreement measured by intraclass correlation coefficient. RESULTS: Intergrader agreement was higher using the P score (intraclass correlation coefficient, 0.75; 95% confidence interval, 0.71-0.79) than no plus, preplus, or plus disease (intraclass correlation coefficient, 0.67; 95% confidence interval, 0.62-0.72). Mean ± standard deviation P scores for images with mode gradings of no plus, preplus, and plus disease were 2.5 ± 0.7, 4.8 ± 0.8, and 7.4 ± 0.8, respectively. CONCLUSIONS: Intergrader agreement of plus-like vascular change in ROP using the P score is high. We now incorporate this 9-image reference set into ICROP3 for use in clinician daily practice alongside zone, stage, and plus assessment. P score is not yet meant to replace plus diagnosis for treatment decisions, but its use at our institutions has permitted better comparison between examinations for progression and regression, communication between examiners, and documentation of vascular change without fundus imaging. P score also could provide more detailed ROP classification for clinical trials, consistent with the spectrum of plus-like change that is now formally part of the International Classification of Retinopathy of Prematurity. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Use of an artificial intelligence-generated vascular severity score improved plus disease diagnosis in retinopathy of prematurity.

OBJECTIVE: To evaluate whether providing clinicians with an artificial intelligence-based vascular severity score (AI-VSS) improves consistency in diagnosis of plus disease in retinopathy of prematurity (ROP). DESIGN: This is a multi-reader diagnostic accuracy imaging study. PARTICIPANTS: Eleven ROP experts (4 pediatric ophthalmologists, 7 retina specialists), 9 of which had been in practice for 10 or more years. METHODS: Retcam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available exams, a subset of 150 eye exams from 110 infants were selected for grading. An AI-VSS was assigned to each set of images using the i-ROP DL system. The clinicians were asked to diagnose plus disease for each exam and assign an estimated VSS (range 1-9) at baseline, and then again one month later with AI-VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye exam from the i-ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis. MAIN OUTCOME MEASURE: Mean linearly weighted kappa for plus disease diagnosis compared to the RSD. Area under the receiver operating characteristic and precision-recall curves (AUROC, AUPR) for 1-9 labels compared to RSD for plus disease. RESULTS: Expert agreement improved significantly from substantial (κ: 0.69 [0.59, 0.75]) to near perfect (κ: 0.81 [0.71, 0.86]) when AI-VSS was integrated. Additionally, there was a significant improvement in plus disease discrimination as measured by mean [95% confidence interval] AUROC (0.94 [0.92, 0.96] to 0.98 [0.96, 0.99], difference: 0.04 [0.01, 0.06]) and AUPR (0.86 [0.81, 0.90] to 0.95 [0.91, 0.97], difference: 0.09 [0.03, 0.14]). CONCLUSIONS: Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling, as compared to a reference standard diagnosis for plus disease. If implemented in practice, AI-VSS could reduce inter-observer variability and standardize treatment for infants with ROP.

Artificial Intelligence for Retinopathy of Prematurity: Validation of a Vascular Severity Scale against International Expert Diagnosis.

PURPOSE: To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), committee. DESIGN: Validation study of an AI-based ROP vascular severity score. PARTICIPANTS: A total of 34 ROP experts from the ICROP3 committee. METHODS: Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, preplus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labeled with a vascular severity score from the Imaging and Informatics in ROP deep learning system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage. MAIN OUTCOME MEASURES: Weighted kappa and Pearson correlation coefficients (CCs) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe. RESULTS: The mean weighted kappa and CC for all interobserver pairs for plus disease image comparison were 0.67 and 0.88, respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, P < 0.001) and the ophthalmoscopic diagnosis of stage (P < 0.001 by analysis of variance) among all experts. CONCLUSIONS: The ROP vascular severity score correlates well with the International Classification of Retinopathy of Prematurity committee member's labels for plus disease and stage, which had significant intergrader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.

Evaluation of a Deep Learning-Derived Quantitative Retinopathy of Prematurity Severity Scale.

PURPOSE: To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale. DESIGN: Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity. PARTICIPANTS: Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9. METHODS: A quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement. MAIN OUTCOME MEASURES: Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale. RESULTS: For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale. CONCLUSIONS: A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.

International Classification of Retinopathy of Prematurity, Third Edition.

PURPOSE: The International Classification of Retinopathy of Prematurity is a consensus statement that creates a standard nomenclature for classification of retinopathy of prematurity (ROP). It was initially published in 1984, expanded in 1987, and revisited in 2005. This article presents a third revision, the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), which is now required because of challenges such as: (1) concerns about subjectivity in critical elements of disease classification; (2) innovations in ophthalmic imaging; (3) novel pharmacologic therapies (e.g., anti-vascular endothelial growth factor agents) with unique regression and reactivation features after treatment compared with ablative therapies; and (4) recognition that patterns of ROP in some regions of the world do not fit neatly into the current classification system. DESIGN: Review of evidence-based literature, along with expert consensus opinion. PARTICIPANTS: International ROP expert committee assembled in March 2019 representing 17 countries and comprising 14 pediatric ophthalmologists and 20 retinal specialists, as well as 12 women and 22 men. METHODS: The committee was initially divided into 3 subcommittees-acute phase, regression or reactivation, and imaging-each of which used iterative videoconferences and an online message board to identify key challenges and approaches. Subsequently, the entire committee used iterative videoconferences, 2 in-person multiday meetings, and an online message board to develop consensus on classification. MAIN OUTCOME MEASURES: Consensus statement. RESULTS: The ICROP3 retains current definitions such as zone (location of disease), stage (appearance of disease at the avascular-vascular junction), and circumferential extent of disease. Major updates in the ICROP3 include refined classification metrics (e.g., posterior zone II, notch, subcategorization of stage 5, and recognition that a continuous spectrum of vascular abnormality exists from normal to plus disease). Updates also include the definition of aggressive ROP to replace aggressive-posterior ROP because of increasing recognition that aggressive disease may occur in larger preterm infants and beyond the posterior retina, particularly in regions of the world with limited resources. ROP regression and reactivation are described in detail, with additional description of long-term sequelae. CONCLUSIONS: These principles may improve the quality and standardization of ROP care worldwide and may provide a foundation to improve research and clinical care.

105° field of view non-contact handheld swept-source optical coherence tomography.

We demonstrate a handheld swept-source optical coherence tomography (OCT) system with a 400 kHz vertical-cavity surface-emitting laser (VCSEL) light source, a non-contact approach, and an unprecedented single shot 105° field of view (FOV). We also implemented a spiral scanning pattern allowing real-time visualization with improved scanning efficiency. To the best of our knowledge, this is the widest FOV achieved in a portable non-contact OCT retinal imaging system to date. Improvements to the FOV may aid the evaluation of retinal diseases such as retinopathy of prematurity, where important vitreoretinal changes often occur in the peripheral retina.

Aggressive Posterior Retinopathy of Prematurity: Clinical and Quantitative Imaging Features in a Large North American Cohort.

PURPOSE: Aggressive posterior retinopathy of prematurity (AP-ROP) is a vision-threatening disease with a significant rate of progression to retinal detachment. The purpose of this study was to characterize AP-ROP quantitatively by demographics, rate of disease progression, and a deep learning-based vascular severity score. DESIGN: Retrospective analysis. PARTICIPANTS: The Imaging and Informatics in ROP cohort from 8 North American centers, consisting of 947 patients and 5945 clinical eye examinations with fundus images, was used. Pretreatment eyes were categorized by disease severity: none, mild, type 2 or pre-plus, treatment-requiring (TR) without AP-ROP, TR with AP-ROP. Analyses compared TR with AP-ROP and TR without AP-ROP to investigate differences between AP-ROP and other TR disease. METHODS: A reference standard diagnosis was generated for each eye examination using previously published methods combining 3 independent image-based gradings and 1 ophthalmoscopic grading. All fundus images were analyzed using a previously published deep learning system and were assigned a score from 1 through 9. MAIN OUTCOME MEASURES: Birth weight, gestational age, postmenstrual age, and vascular severity score. RESULTS: Infants who demonstrated AP-ROP were more premature by birth weight (617 g vs. 679 g; P = 0.01) and gestational age (24.3 weeks vs. 25.0 weeks; P < 0.01) and reached peak severity at an earlier postmenstrual age (34.7 weeks vs. 36.9 weeks; P < 0.001) compared with infants with TR without AP-ROP. The mean vascular severity score was greatest in TR with AP-ROP infants compared with TR without AP-ROP infants (8.79 vs. 7.19; P < 0.001). Analyzing the severity score over time, the rate of progression was fastest in infants with AP-ROP (P < 0.002 at 30-32 weeks). CONCLUSIONS: Premature infants in North America with AP-ROP are born younger and demonstrate disease earlier than infants with less severe ROP. Disease severity is quantifiable with a deep learning-based score, which correlates with clinically identified categories of disease, including AP-ROP. The rate of progression to peak disease is greatest in eyes that demonstrate AP-ROP compared with other treatment-requiring eyes. Analysis of quantitative characteristics of AP-ROP may help improve diagnosis and treatment of an aggressive, vision-threatening form of ROP.

Assessment of a Tele-education System to Enhance Retinopathy of Prematurity Training by International Ophthalmologists-in-Training in Mexico.

PURPOSE: To evaluate a tele-education system developed to improve diagnostic competency in retinopathy of prematurity (ROP) by ophthalmologists-in-training in Mexico. DESIGN: Prospective, randomized cohort study. PARTICIPANTS: Fifty-eight ophthalmology residents and fellows from a training program in Mexico consented to participate. Twenty-nine of 58 trainees (50%) were randomized to the educational intervention (pretest, ROP tutorial, ROP educational chapters, and posttest), and 29 of 58 trainees (50%) were randomized to a control group (pretest and posttest only). METHODS: A secure web-based educational system was created using clinical cases (20 pretest, 20 posttest, and 25 training chapter-based) developed from a repository of over 2500 unique image sets of ROP. For each image set used, a reference standard ROP diagnosis was established by combining the clinical diagnosis by indirect ophthalmoscope examination and image-based diagnosis by multiple experts. Trainees were presented with image-based clinical cases of ROP during a pretest, posttest, and training chapters. MAIN OUTCOME MEASURES: The accuracy of ROP diagnosis (e.g., plus disease, zone, stage, category) was determined using sensitivity and specificity calculations from the pretest and posttest results of the educational intervention group versus control group. The unweighted kappa statistic was used to analyze the intragrader agreement for ROP diagnosis by the ophthalmologists-in-training during the pretest and posttest for both groups. RESULTS: Trainees completing the tele-education system had statistically significant improvements (P < 0.01) in the accuracy of ROP diagnosis for plus disease, zone, stage, category, and aggressive posterior ROP (AP-ROP). Compared with the control group, trainees who completed the ROP tele-education system performed better on the posttest for accurately diagnosing plus disease (67% vs. 48%; P = 0.04) and the presence of ROP (96% vs. 91%; P < 0.01). The specificity for diagnosing AP-ROP (94% vs. 78%; P < 0.01), type 2 ROP or worse (92% vs. 84%; P = 0.04), and ROP requiring treatment (89% vs. 79%; P < 0.01) was better for the trainees completing the tele-education system compared with the control group. Intragrader agreement improved for identification of plus disease, zone, stage, and category of ROP after completion of the educational intervention. CONCLUSIONS: A tele-education system for ROP education was effective in improving the diagnostic accuracy of ROP by ophthalmologists-in-training in Mexico. This system has the potential to increase competency in ROP diagnosis and management for ophthalmologists-in-training from middle-income nations.

Diagnostic Discrepancies in Retinopathy of Prematurity Classification.

PURPOSE: To identify the most common areas for discrepancy in retinopathy of prematurity (ROP) classification between experts. DESIGN: Prospective cohort study. PARTICIPANTS: A total of 281 infants were identified as part of a multicenter, prospective, ROP cohort study from 7 participating centers. Each site had participating ophthalmologists who provided the clinical classification after routine examination using binocular indirect ophthalmoscopy (BIO) and obtained wide-angle retinal images, which were independently classified by 2 study experts. METHODS: Wide-angle retinal images (RetCam; Clarity Medical Systems, Pleasanton, CA) were obtained from study subjects, and 2 experts evaluated each image using a secure web-based module. Image-based classifications for zone, stage, plus disease, and overall disease category (no ROP, mild ROP, type II or pre-plus, and type I) were compared between the 2 experts and with the clinical classification obtained by BIO. MAIN OUTCOME MEASURES: Inter-expert image-based agreement and image-based versus ophthalmoscopic diagnostic agreement using absolute agreement and weighted kappa statistic. RESULTS: A total of 1553 study eye examinations from 281 infants were included in the study. Experts disagreed on the stage classification in 620 of 1553 comparisons (40%), plus disease classification (including pre-plus) in 287 of 1553 comparisons (18%), zone in 117 of 1553 comparisons (8%), and overall ROP category in 618 of 1553 comparisons (40%). However, agreement for presence versus absence of type 1 disease was >95%. There were no differences between image-based and clinical classification except for zone III disease. CONCLUSIONS: The most common area of discrepancy in ROP classification is stage, although inter-expert agreement for clinically significant disease, such as presence versus absence of type 1 and type 2 disease, is high. There were no differences between image-based grading and clinical examination in the ability to detect clinically significant disease. This study provides additional evidence that image-based classification of ROP reliably detects clinically significant levels of ROP with high accuracy compared with the clinical examination.

Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader.

PURPOSE: To determine if (1) tortuosity assessment by a computer program (ROPtool, developed at the University of North Carolina, Chapel Hill, and Duke University, and licensed by FocusROP) that traces retinal blood vessels and (2) assessment by a lay reader are comparable with assessment by a panel of 3 retinopathy of prematurity (ROP) experts for remote clinical grading of vascular abnormalities such as plus disease. DESIGN: Validity and reliability analysis of diagnostic tools. PARTICIPANTS: Three hundred thirty-five fundus images of prematurely born infants. METHODS: Three hundred thirty-five fundus images of prematurely born infants were obtained by neonatal intensive care unit nurses. A panel of 3 ROP experts graded 84 images showing vascular dilatation, tortuosity, or both and 251 images showing no evidence of vascular abnormalities. These images were sent electronically to an experienced lay reader who independently graded them for vascular abnormalities. The images also were analyzed using the ROPtool, which assigns a numerical value to the level of vascular abnormality and tortuosity present in each of 4 quadrants or sectors. The ROPtool measurements of vascular abnormalities were graded and compared with expert panel grades with a receiver operating characteristic (ROC) curve. Grades between human readers were cross-tabulated. The area under the ROC curve was calculated for the ROPtool, and sensitivity and specificity were computed for the lay reader. MAIN OUTCOME MEASURES: Measurements of vascular abnormalities by ROPtool and grading of vascular abnormalities by 3 ROP experts and 1 experienced lay reader. RESULTS: The ROC curve for ROPtool's tortuosity assessment had an area under the ROC curve of 0.917. Using a threshold value of 4.97 for the second most tortuous quadrant, ROPtool's sensitivity was 91% and its specificity was 82%. Lay reader sensitivity and specificity were 99% and 73%, respectively, and had high reliability (κ, 0.87) in repeated measurements. CONCLUSIONS: ROPtool had very good accuracy for detection of vascular abnormalities suggestive of plus disease when compared with expert physician graders. The lay reader's results showed excellent sensitivity and good specificity when compared with those of the expert graders. These options for remote reading of images to detect vascular abnormalities deserve consideration in the quest to use telemedicine with remote reading for efficient delivery of high-quality care and to detect infants requiring bedside examination.

Plus Disease in Retinopathy of Prematurity: A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability.

PURPOSE: To identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP). DESIGN: We developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD. PARTICIPANTS: Eight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units. METHODS: Expert classification of images of plus disease in ROP. MAIN OUTCOME MEASURES: Interexpert agreement (weighted κ statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement). RESULTS: There was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted κ, 0-0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R2 = 0.82; and dataset B: P < 0.05 and adjusted R2 = 0.6615). CONCLUSIONS: There is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future.

Plus Disease in Retinopathy of Prematurity: Improving Diagnosis by Ranking Disease Severity and Using Quantitative Image Analysis.

PURPOSE: To determine expert agreement on relative retinopathy of prematurity (ROP) disease severity and whether computer-based image analysis can model relative disease severity, and to propose consideration of a more continuous severity score for ROP. DESIGN: We developed 2 databases of clinical images of varying disease severity (100 images and 34 images) as part of the Imaging and Informatics in ROP (i-ROP) cohort study and recruited expert physician, nonexpert physician, and nonphysician graders to classify and perform pairwise comparisons on both databases. PARTICIPANTS: Six participating expert ROP clinician-scientists, each with a minimum of 10 years of clinical ROP experience and 5 ROP publications, and 5 image graders (3 physicians and 2 nonphysician graders) who analyzed images that were obtained during routine ROP screening in neonatal intensive care units. METHODS: Images in both databases were ranked by average disease classification (classification ranking), by pairwise comparison using the Elo rating method (comparison ranking), and by correlation with the i-ROP computer-based image analysis system. MAIN OUTCOME MEASURES: Interexpert agreement (weighted κ statistic) compared with the correlation coefficient (CC) between experts on pairwise comparisons and correlation between expert rankings and computer-based image analysis modeling. RESULTS: There was variable interexpert agreement on diagnostic classification of disease (plus, preplus, or normal) among the 6 experts (mean weighted κ, 0.27; range, 0.06-0.63), but good correlation between experts on comparison ranking of disease severity (mean CC, 0.84; range, 0.74-0.93) on the set of 34 images. Comparison ranking provided a severity ranking that was in good agreement with ranking obtained by classification ranking (CC, 0.92). Comparison ranking on the larger dataset by both expert and nonexpert graders demonstrated good correlation (mean CC, 0.97; range, 0.95-0.98). The i-ROP system was able to model this continuous severity with good correlation (CC, 0.86). CONCLUSIONS: Experts diagnose plus disease on a continuum, with poor absolute agreement on classification but good relative agreement on disease severity. These results suggest that the use of pairwise rankings and a continuous severity score, such as that provided by the i-ROP system, may improve agreement on disease severity in the future.

Influence of Fluorescein Angiography on the Diagnosis and Management of Retinopathy of Prematurity.

PURPOSE: To examine the influence of fluorescein angiography (FA) on the diagnosis and management of retinopathy of prematurity (ROP). DESIGN: Prospective cohort study. PARTICIPANTS: Nine recognized ROP experts (3 pediatric ophthalmologists and 6 retina specialists) interpreted 32 sets (16 color fundus photographs and 16 color fundus photographs paired with the corresponding FA images) of wide-angle retinal images from infants with ROP. METHODS: All experts independently reviewed the 32 image sets on a secure website and provided a diagnosis and management plan for the case presented, first based on color fundus photographs alone, and then based on color fundus photographs and corresponding FA images. MAIN OUTCOME MEASURES: Sensitivity and specificity of the ROP diagnosis (zone, stage, plus disease, and category, i.e., no ROP, mild ROP, type 2 ROP, and ROP requiring treatment) were calculated using a consensus reference standard diagnosis, determined from the diagnosis of the color fundus photographs by 3 experienced readers in combination with the clinical diagnosis based on ophthalmoscopic examination. The κ statistic was used to analyze the average intergrader agreement among experts for the diagnosis of zone, stage, plus disease, and category. RESULTS: Addition of FA to color fundus photography resulted in a significant improvement in sensitivity for diagnosis of stage 3 or worse disease (39.8% vs. 74.1%; P = 0.008), type 2 or worse ROP (69.4% vs. 86.8%; P = 0.013), and pre-plus or worse disease (50.5 vs. 62.6%; P = 0.031). There was a nonsignificant trend toward improved sensitivity for diagnosis of ROP requiring treatment (22.2% vs. 40.3%; P = 0.063). Using the κ statistic, addition of FA to color fundus photography significantly improved intergrader agreement for diagnosis of ROP requiring treatment. Addition of FA to color fundus photography did not affect intergrader agreement significantly for the diagnosis of stage, zone, or plus disease. CONCLUSIONS: Compared with color fundus photography alone, FA may improve the sensitivity of diagnosis of ROP by experts, particularly for stage 3 disease. In addition, intergrader agreement for diagnosis of ROP requiring treatment may improve with FA interpretation.

Risk factors for retinopathy of prematurity: insights from outlier infants.

PURPOSE: To investigate the characteristics of outlier infants for insights into ROP risk. METHODS: Chart data were collected from 1,354 infants screened for ROP at Weill Cornell Medical Center and Columbia University Medical Center. ROP exam results and clinical risk factors were recorded. The cohort was stratified by weight, highest ROP stage, and need for ROP treatment. Descriptive and correlational statistics were performed. RESULTS: For the overall cohort, regression analysis found that birth weight (OR: 0.741 per 100 g; 95 % CI: 0.606, 0.905), gestational age at birth (OR: 0.563 per week; 95 % CI: 0.454, 0.697), multiple gestation (OR 2.02, 95 % CI: 1.15, 3.56), bronchopulmonary dysplasia (OR: 4.68, 95 % CI: 1.93, 11.35), and necrotizing enterocolitis (OR 2.80, 95 % CI: 1.40, 5.16) were independent risk factors for treatment-requiring ROP. Black race was found to be a protective factor for treatment-requiring ROP (OR 0.244, 95 % CI: 0.095, 0.626). Among 15 infants with BW <500 g, there were no significant differences in any clinical risk factors between the 12 (80 %) with ROP vs the three (20 %) without ROP. Similarly, among infants with BW >1500 g, the 17 (9 %) with ROP only differed from the 166 (91 %) without ROP with respect to a higher incidence of necrotizing enterocolitis among those with ROP (11.8 % vs 0 %). CONCLUSIONS: Although known clinical risk factors were predictive of ROP stage and need for laser treatment in this cohort, they were not significantly associated with ROP at extremes of birth weight. This suggests that other clinical, maternal, or genetic factors may protect from or predispose to ROP.

Assessing spectral effectiveness in color fundus photography for deep learning classification of retinopathy of prematurity.

Significance: Retinopathy of prematurity (ROP) poses a significant global threat to childhood vision, necessitating effective screening strategies. This study addresses the impact of color channels in fundus imaging on ROP diagnosis, emphasizing the efficacy and safety of utilizing longer wavelengths, such as red or green for enhanced depth information and improved diagnostic capabilities. Aim: This study aims to assess the spectral effectiveness in color fundus photography for the deep learning classification of ROP. Approach: A convolutional neural network end-to-end classifier was utilized for deep learning classification of normal, stage 1, stage 2, and stage 3 ROP fundus images. The classification performances with individual-color-channel inputs, i.e., red, green, and blue, and multi-color-channel fusion architectures, including early-fusion, intermediate-fusion, and late-fusion, were quantitatively compared. Results: For individual-color-channel inputs, similar performance was observed for green channel (88.00% accuracy, 76.00% sensitivity, and 92.00% specificity) and red channel (87.25% accuracy, 74.50% sensitivity, and 91.50% specificity), which is substantially outperforming the blue channel (78.25% accuracy, 56.50% sensitivity, and 85.50% specificity). For multi-color-channel fusion options, the early-fusion and intermediate-fusion architecture showed almost the same performance when compared to the green/red channel input, and they outperformed the late-fusion architecture. Conclusions: This study reveals that the classification of ROP stages can be effectively achieved using either the green or red image alone. This finding enables the exclusion of blue images, acknowledged for their increased susceptibility to light toxicity.

Isolated Retinal Neovascularization in Retinopathy of Prematurity: Clinical Associations and Prognostic Implications.

OBJECTIVE: Isolated retinal neovascularization (IRNV) is a common finding in patients with stage 2 and 3 retinopathy of prematurity (ROP). This study aimed to further classify the clinical course and significance of these lesions (previously described as "popcorn" based on clinical appearance) in patients with ROP as visualized with ultrawidefield OCT (UWF-OCT). DESIGN: Single center, retrospective case series. PARTICIPANTS: Images were collected from 136 babies in the Oregon Health and Science University neonatal intensive care unit. METHODS: A prototype UWF-OCT device captured en face scans (>140°), which were reviewed for the presence of IRNV along with standard zone, stage, and plus classification. In a cross-sectional analysis we compared demographics and the clinical course of eyes with and without IRNV. Longitudinally, we compared ROP severity using a clinician-assigned vascular severity score (VSS) and compared the risk of progression among eyes with and without IRNV using multivariable logistic regression. MAIN OUTCOME MEASURES: Differences in clinical demographics and disease progression between patients with and without IRNV. RESULTS: Of the 136 patients, 60 developed stage 2 or worse ROP during their disease course, 22 of whom had IRNV visualized on UWF-OCT (37%). On average, patients with IRNV had lower birth weights (BWs) (660.1 vs. 916.8 g, P = 0.001), gestational age (GA) (24.9 vs. 26.1 weeks, P = 0.01), and were more likely to present with ROP in zone I (63.4% vs. 15.8%, P < 0.001). They were also more likely to progress to stage 3 (68.2% vs. 13.2%, P < 0.001) and receive treatment (54.5% vs. 15.8%, P = 0.002). Eyes with IRNV had a higher peak VSS (5.61 vs. 3.73, P < 0.001) and averaged a higher VSS throughout their disease course. On multivariable logistic regression, IRNV was independently associated with progression to stage 3 (P = 0.02) and requiring treatment (P = 0.03), controlling for GA, BW, and initial zone 1 disease. CONCLUSIONS: In this single center study, we found that IRNV occurs in higher risk babies and was an independent risk factor for ROP progression and treatment. These findings may have implications for OCT-based ROP classifications in the future. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Panretinal handheld OCT angiography for pediatric retinal imaging.

Comprehensive visualization of retina morphology is essential in the diagnosis and management of retinal diseases in pediatric populations. Conventional imaging techniques often face challenges in effectively capturing the peripheral retina, primarily due to the limitations in current optical designs, which lack the necessary field of view to characterize the far periphery. To address this gap, our study introduces a novel ultra-widefield optical coherence tomography angiography (OCTA) system. This system, specifically tailored for pediatric applications, incorporates an ultrahigh-speed 800 kHz swept-source laser. The system's innovative design achieves a 140° field of view while maintaining excellent optical performance. Over the last 15 months, we have conducted 379 eye examinations on 96 babies using this system. It demonstrates marked efficacy in the diagnosis of retinopathy of prematurity, providing detailed and comprehensive peripheral retinal angiography. The capabilities of the ultra-widefield handheld OCTA system in enhancing the clarity and thoroughness of retina vascularization assessments have significantly improved the precision of diagnoses and the customization of treatment strategies. Our findings underscore the system's potential to advance pediatric ophthalmology and broaden the scope of retinal imaging.

Multinational External Validation of Autonomous Retinopathy of Prematurity Screening.

Importance: Retinopathy of prematurity (ROP) is a leading cause of blindness in children, with significant disparities in outcomes between high-income and low-income countries, due in part to insufficient access to ROP screening. Objective: To evaluate how well autonomous artificial intelligence (AI)-based ROP screening can detect more-than-mild ROP (mtmROP) and type 1 ROP. Design, Setting, and Participants: This diagnostic study evaluated the performance of an AI algorithm, trained and calibrated using 2530 examinations from 843 infants in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) study, on 2 external datasets (6245 examinations from 1545 infants in the Stanford University Network for Diagnosis of ROP [SUNDROP] and 5635 examinations from 2699 infants in the Aravind Eye Care Systems [AECS] telemedicine programs). Data were taken from 11 and 48 neonatal care units in the US and India, respectively. Data were collected from January 2012 to July 2021, and data were analyzed from July to December 2023. Exposures: An imaging processing pipeline was created using deep learning to autonomously identify mtmROP and type 1 ROP in eye examinations performed via telemedicine. Main Outcomes and Measures: The area under the receiver operating characteristics curve (AUROC) as well as sensitivity and specificity for detection of mtmROP and type 1 ROP at the eye examination and patient levels. Results: The prevalence of mtmROP and type 1 ROP were 5.9% (91 of 1545) and 1.2% (18 of 1545), respectively, in the SUNDROP dataset and 6.2% (168 of 2699) and 2.5% (68 of 2699) in the AECS dataset. Examination-level AUROCs for mtmROP and type 1 ROP were 0.896 and 0.985, respectively, in the SUNDROP dataset and 0.920 and 0.982 in the AECS dataset. At the cross-sectional examination level, mtmROP detection had high sensitivity (SUNDROP: mtmROP, 83.5%; 95% CI, 76.6-87.7; type 1 ROP, 82.2%; 95% CI, 81.2-83.1; AECS: mtmROP, 80.8%; 95% CI, 76.2-84.9; type 1 ROP, 87.8%; 95% CI, 86.8-88.7). At the patient level, all infants who developed type 1 ROP screened positive (SUNDROP: 100%; 95% CI, 81.4-100; AECS: 100%; 95% CI, 94.7-100) prior to diagnosis. Conclusions and Relevance: Where and when ROP telemedicine programs can be implemented, autonomous ROP screening may be an effective force multiplier for secondary prevention of ROP.

Oxygenation Fluctuations Associated with Severe Retinopathy of Prematurity: Insights from a Multimodal Deep Learning Approach.

Purpose: Retinopathy of prematurity (ROP) is one of the leading causes of blindness in children. Although the role of oxygen in the pathophysiology of ROP is well established, a precise understanding of the dynamic relationship between oxygen exposure ROP incidence and severity is lacking. The purpose of this study was to evaluate the correlation between time-dependent oxygen variables and the onset of ROP. Design: Retrospective cohort study. Participants: Two hundred thirty infants who were born at a single academic center and met the inclusion criteria were included. Infants are mainly born between January 2011 and October 2022. Methods: Patient data were extracted from electronic health records (EHRs), with sufficient time-dependent oxygen data. Clinical outcomes for ROP were recorded as none/mild or moderate/severe (defined as type II or worse). Mixed-effects linear models were used to compare the 2 groups in terms of dynamic oxygen variables, such as daily average and the coefficient of variation (COV) fraction of inspired oxygen (FiO2). Support vector machine (SVM) and long-short-term memory (LSTM)-based multimodal models were trained with fivefold cross-validation to predict which infants would develop moderate/severe ROP. Gestational age (GA), birth weight, and time-dependent oxygen variables were used to develop predictive models. Main Outcome Measures: Model cross-validation performance was evaluated by computing the mean area under the receiver operating characteristic (AUROC) curve, precision, recall, and F1 score. Results: We found that both daily average and COV of FiO2 were associated with more severe ROP (adjusted P < 0.001). With fivefold cross-validation, the multimodal LSTM models had higher performance than the best static models (SVM using GA and 3 average FiO2 features) and SVM models trained on GA alone (mean AUROC = 0.89 ± 0.04 vs. 0.86 ± 0.05 vs. 0.83 ± 0.04). Conclusions: The development of severe ROP might not only be influenced by oxygen exposure but also by its fluctuation, which provides direction for future study of pathophysiological factors associated with severe ROP development. Additionally, we demonstrated that multimodal neural networks can be a method to extract useful information from time-series data, which may be a valuable methodology for the investigation of other diseases using EHR data. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Genome-wide association identifies novel ROP risk loci in a multiethnic cohort.

We conducted a genome-wide association study (GWAS) in a multiethnic cohort of 920 at-risk infants for retinopathy of prematurity (ROP), a major cause of childhood blindness, identifying 1 locus at genome-wide significance level (p < 5×10-8) and 9 with significance of p < 5×10-6 for ROP ≥ stage 3. The most significant locus, rs2058019, reached genome-wide significance within the full multiethnic cohort (p = 4.96×10-9); Hispanic and European Ancestry infants driving the association. The lead single nucleotide polymorphism (SNP) falls in an intronic region within the Glioma-associated oncogene family zinc finger 3 (GLI3) gene. Relevance for GLI3 and other top-associated genes to human ocular disease was substantiated through in-silico extension analyses, genetic risk score analysis and expression profiling in human donor eye tissues. Thus, we identify a novel locus at GLI3 with relevance to retinal biology, supporting genetic susceptibilities for ROP risk with possible variability by race and ethnicity.