Usability Evaluation of a Dashboard for Home Care Nurses.

The introduction of electronic health records has produced many challenges for clinicians. These include integrating technology into clinical workflow and fragmentation of relevant information across systems. Dashboards, which use visualized data to summarize key patient information, have the potential to address these issues. In this article, we outline a usability evaluation of a dashboard designed for home care nurses. An iterative design process was used, which consisted of (1) contextual inquiry (observation and interviews) with two home care nurses; (2) rapid feedback on paper prototypes of the dashboard (10 nurses); and (3) usability evaluation of the final dashboard prototype (20 nurses). Usability methods and assessments included observation of nurses interacting with the dashboard, the system usability scale, and the Questionnaire for User Interaction Satisfaction short form. The dashboard prototype was deemed to have high usability (mean system usability scale, 73.2 [SD, 18.8]) and was positively evaluated by nurse users. It is important to ensure that technology solutions such as the one proposed in this article are designed with clinical users in mind, to meet their information needs. The design elements of the dashboard outlined in this article could be translated to other electronic health records used in home care settings.

Assessment of a Tele-education System to Enhance Retinopathy of Prematurity Training by International Ophthalmologists-in-Training in Mexico.

PURPOSE: To evaluate a tele-education system developed to improve diagnostic competency in retinopathy of prematurity (ROP) by ophthalmologists-in-training in Mexico. DESIGN: Prospective, randomized cohort study. PARTICIPANTS: Fifty-eight ophthalmology residents and fellows from a training program in Mexico consented to participate. Twenty-nine of 58 trainees (50%) were randomized to the educational intervention (pretest, ROP tutorial, ROP educational chapters, and posttest), and 29 of 58 trainees (50%) were randomized to a control group (pretest and posttest only). METHODS: A secure web-based educational system was created using clinical cases (20 pretest, 20 posttest, and 25 training chapter-based) developed from a repository of over 2500 unique image sets of ROP. For each image set used, a reference standard ROP diagnosis was established by combining the clinical diagnosis by indirect ophthalmoscope examination and image-based diagnosis by multiple experts. Trainees were presented with image-based clinical cases of ROP during a pretest, posttest, and training chapters. MAIN OUTCOME MEASURES: The accuracy of ROP diagnosis (e.g., plus disease, zone, stage, category) was determined using sensitivity and specificity calculations from the pretest and posttest results of the educational intervention group versus control group. The unweighted kappa statistic was used to analyze the intragrader agreement for ROP diagnosis by the ophthalmologists-in-training during the pretest and posttest for both groups. RESULTS: Trainees completing the tele-education system had statistically significant improvements (P < 0.01) in the accuracy of ROP diagnosis for plus disease, zone, stage, category, and aggressive posterior ROP (AP-ROP). Compared with the control group, trainees who completed the ROP tele-education system performed better on the posttest for accurately diagnosing plus disease (67% vs. 48%; P = 0.04) and the presence of ROP (96% vs. 91%; P < 0.01). The specificity for diagnosing AP-ROP (94% vs. 78%; P < 0.01), type 2 ROP or worse (92% vs. 84%; P = 0.04), and ROP requiring treatment (89% vs. 79%; P < 0.01) was better for the trainees completing the tele-education system compared with the control group. Intragrader agreement improved for identification of plus disease, zone, stage, and category of ROP after completion of the educational intervention. CONCLUSIONS: A tele-education system for ROP education was effective in improving the diagnostic accuracy of ROP by ophthalmologists-in-training in Mexico. This system has the potential to increase competency in ROP diagnosis and management for ophthalmologists-in-training from middle-income nations.

Diagnostic Discrepancies in Retinopathy of Prematurity Classification.

PURPOSE: To identify the most common areas for discrepancy in retinopathy of prematurity (ROP) classification between experts. DESIGN: Prospective cohort study. PARTICIPANTS: A total of 281 infants were identified as part of a multicenter, prospective, ROP cohort study from 7 participating centers. Each site had participating ophthalmologists who provided the clinical classification after routine examination using binocular indirect ophthalmoscopy (BIO) and obtained wide-angle retinal images, which were independently classified by 2 study experts. METHODS: Wide-angle retinal images (RetCam; Clarity Medical Systems, Pleasanton, CA) were obtained from study subjects, and 2 experts evaluated each image using a secure web-based module. Image-based classifications for zone, stage, plus disease, and overall disease category (no ROP, mild ROP, type II or pre-plus, and type I) were compared between the 2 experts and with the clinical classification obtained by BIO. MAIN OUTCOME MEASURES: Inter-expert image-based agreement and image-based versus ophthalmoscopic diagnostic agreement using absolute agreement and weighted kappa statistic. RESULTS: A total of 1553 study eye examinations from 281 infants were included in the study. Experts disagreed on the stage classification in 620 of 1553 comparisons (40%), plus disease classification (including pre-plus) in 287 of 1553 comparisons (18%), zone in 117 of 1553 comparisons (8%), and overall ROP category in 618 of 1553 comparisons (40%). However, agreement for presence versus absence of type 1 disease was >95%. There were no differences between image-based and clinical classification except for zone III disease. CONCLUSIONS: The most common area of discrepancy in ROP classification is stage, although inter-expert agreement for clinically significant disease, such as presence versus absence of type 1 and type 2 disease, is high. There were no differences between image-based grading and clinical examination in the ability to detect clinically significant disease. This study provides additional evidence that image-based classification of ROP reliably detects clinically significant levels of ROP with high accuracy compared with the clinical examination.

Plus Disease in Retinopathy of Prematurity: A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability.

PURPOSE: To identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP). DESIGN: We developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD. PARTICIPANTS: Eight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units. METHODS: Expert classification of images of plus disease in ROP. MAIN OUTCOME MEASURES: Interexpert agreement (weighted κ statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement). RESULTS: There was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted κ, 0-0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R2 = 0.82; and dataset B: P < 0.05 and adjusted R2 = 0.6615). CONCLUSIONS: There is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future.

Plus Disease in Retinopathy of Prematurity: Improving Diagnosis by Ranking Disease Severity and Using Quantitative Image Analysis.

PURPOSE: To determine expert agreement on relative retinopathy of prematurity (ROP) disease severity and whether computer-based image analysis can model relative disease severity, and to propose consideration of a more continuous severity score for ROP. DESIGN: We developed 2 databases of clinical images of varying disease severity (100 images and 34 images) as part of the Imaging and Informatics in ROP (i-ROP) cohort study and recruited expert physician, nonexpert physician, and nonphysician graders to classify and perform pairwise comparisons on both databases. PARTICIPANTS: Six participating expert ROP clinician-scientists, each with a minimum of 10 years of clinical ROP experience and 5 ROP publications, and 5 image graders (3 physicians and 2 nonphysician graders) who analyzed images that were obtained during routine ROP screening in neonatal intensive care units. METHODS: Images in both databases were ranked by average disease classification (classification ranking), by pairwise comparison using the Elo rating method (comparison ranking), and by correlation with the i-ROP computer-based image analysis system. MAIN OUTCOME MEASURES: Interexpert agreement (weighted κ statistic) compared with the correlation coefficient (CC) between experts on pairwise comparisons and correlation between expert rankings and computer-based image analysis modeling. RESULTS: There was variable interexpert agreement on diagnostic classification of disease (plus, preplus, or normal) among the 6 experts (mean weighted κ, 0.27; range, 0.06-0.63), but good correlation between experts on comparison ranking of disease severity (mean CC, 0.84; range, 0.74-0.93) on the set of 34 images. Comparison ranking provided a severity ranking that was in good agreement with ranking obtained by classification ranking (CC, 0.92). Comparison ranking on the larger dataset by both expert and nonexpert graders demonstrated good correlation (mean CC, 0.97; range, 0.95-0.98). The i-ROP system was able to model this continuous severity with good correlation (CC, 0.86). CONCLUSIONS: Experts diagnose plus disease on a continuum, with poor absolute agreement on classification but good relative agreement on disease severity. These results suggest that the use of pairwise rankings and a continuous severity score, such as that provided by the i-ROP system, may improve agreement on disease severity in the future.

Immediate Sequential Bilateral Pediatric Vitreoretinal Surgery: An International Multicenter Study.

PURPOSE: To determine the feasibility and safety of bilateral simultaneous vitreoretinal surgery in pediatric patients. DESIGN: International, multicenter, interventional, retrospective case series. PARTICIPANTS: Patients 17 years of age or younger from 24 centers worldwide who underwent immediate sequential bilateral vitreoretinal surgery (ISBVS)-defined as vitrectomy, scleral buckle, or lensectomy using the vitreous cutter-performed in both eyes sequentially during the same anesthesia session. METHODS: Clinical history, surgical details and indications, time under anesthesia, and intraoperative and postoperative ophthalmic and systemic adverse events were reviewed. MAIN OUTCOME MEASURES: Ocular and systemic adverse events. RESULTS: A total of 344 surgeries from 172 ISBVS procedures in 167 patients were included in the study. The mean age of the cohort was 1.3±2.6 years. Nonexclusive indications for ISBVS were rapidly progressive disease (74.6%), systemic morbidity placing the child at high anesthesia risk (76.0%), and residence remote from surgery location (30.2%). The most common diagnoses were retinopathy of prematurity (ROP; 72.7% [P < 0.01]; stage 3, 4.8%; stage 4A, 44.4%; stage 4B, 22.4%; stage 5, 26.4%), familial exudative vitreoretinopathy (7.0%), abusive head trauma (4.1%), persistent fetal vasculature (3.5%), congenital cataract (1.7%), posterior capsular opacification (1.7%), rhegmatogenous retinal detachment (1.7%), congenital X-linked retinoschisis (1.2%), Norrie disease (2.3%), and viral retinitis (1.2%). Mean surgical time was 143±59 minutes for both eyes. Higher ROP stage correlated with longer surgical time (P = 0.02). There were no reported intraoperative ocular complications. During the immediate postoperative period, 2 eyes from different patients demonstrated unilateral vitreous hemorrhage (0.6%). No cases of endophthalmitis, choroidal hemorrhage, or hypotony occurred. Mean total anesthesia time was 203±87 minutes. There were no cases of anesthesia-related death, malignant hyperthermia, anaphylaxis, or cardiac event. There was 1 case of reintubation (0.6%) and 1 case of prolonged oxygen desaturation (0.6%). Mean follow-up after surgery was 103 weeks, and anatomic success and globe salvage rates were 89.8% and 98.0%, respectively. CONCLUSIONS: This study found ISBVS to be a feasible and safe treatment paradigm for pediatric patients with bilateral vitreoretinal pathologic features when repeated general anesthesia is undesirable or impractical.

Influence of Fluorescein Angiography on the Diagnosis and Management of Retinopathy of Prematurity.

PURPOSE: To examine the influence of fluorescein angiography (FA) on the diagnosis and management of retinopathy of prematurity (ROP). DESIGN: Prospective cohort study. PARTICIPANTS: Nine recognized ROP experts (3 pediatric ophthalmologists and 6 retina specialists) interpreted 32 sets (16 color fundus photographs and 16 color fundus photographs paired with the corresponding FA images) of wide-angle retinal images from infants with ROP. METHODS: All experts independently reviewed the 32 image sets on a secure website and provided a diagnosis and management plan for the case presented, first based on color fundus photographs alone, and then based on color fundus photographs and corresponding FA images. MAIN OUTCOME MEASURES: Sensitivity and specificity of the ROP diagnosis (zone, stage, plus disease, and category, i.e., no ROP, mild ROP, type 2 ROP, and ROP requiring treatment) were calculated using a consensus reference standard diagnosis, determined from the diagnosis of the color fundus photographs by 3 experienced readers in combination with the clinical diagnosis based on ophthalmoscopic examination. The κ statistic was used to analyze the average intergrader agreement among experts for the diagnosis of zone, stage, plus disease, and category. RESULTS: Addition of FA to color fundus photography resulted in a significant improvement in sensitivity for diagnosis of stage 3 or worse disease (39.8% vs. 74.1%; P = 0.008), type 2 or worse ROP (69.4% vs. 86.8%; P = 0.013), and pre-plus or worse disease (50.5 vs. 62.6%; P = 0.031). There was a nonsignificant trend toward improved sensitivity for diagnosis of ROP requiring treatment (22.2% vs. 40.3%; P = 0.063). Using the κ statistic, addition of FA to color fundus photography significantly improved intergrader agreement for diagnosis of ROP requiring treatment. Addition of FA to color fundus photography did not affect intergrader agreement significantly for the diagnosis of stage, zone, or plus disease. CONCLUSIONS: Compared with color fundus photography alone, FA may improve the sensitivity of diagnosis of ROP by experts, particularly for stage 3 disease. In addition, intergrader agreement for diagnosis of ROP requiring treatment may improve with FA interpretation.

Reducing NICU Unplanned Extubations From Tube Dislodgement.

OBJECTIVES: Unplanned extubations (UEs) can be a frequent problem and are associated with adverse outcomes. This quality improvement initiative sought to reduce UEs from tube dislodgement in a level IV NICU utilizing methods applicable to other ICUs and performed with minimal monetary funds. METHODS: From January 2019 to July 2023, an interdisciplinary quality improvement team used the Model for Improvement and performed sequential interventions to improve the outcome measure of UEs per 100 ventilator days. Process measures included adherence to a modified, site-specific UE care bundle derived from the Solutions for Patient Safety network, whereas the number of endotracheal tube-related pressure injuries was used as a balancing measure. Statistical process control charts and established rules for special cause variation were applied to analyze data. RESULTS: Sequential interventions reduced the rate of UEs from a baseline of 2.3 to 0.6 UEs per 100 ventilator days. Greater than 90% adherence with the UE care bundle and apparent cause analysis form completion occurred since December 2020. There were no endotracheal tube-related pressure injuries. CONCLUSIONS: A sustained reduction in UEs was demonstrated. Leveraging a multidisciplinary team allowed for continuous UE analysis, which promoted tailored consecutive interventions. UE care bundle audits and the creation of a postevent debrief guide, which helped providers share a common language, were the most impactful interventions. Next steps include disseminating these interventions to other ICUs across our hospital enterprise. These low-cost interventions can be scalable to other NICUs and PICUs.

Characterization of Errors in Retinopathy of Prematurity Diagnosis by Ophthalmologists-in-Training in Middle-Income Countries.

PURPOSE: To characterize common errors in the diagnosis of retinopathy of prematurity (ROP) among ophthalmologistsin-training in middle-income countries. METHODS: In this prospective cohort study, 200 ophthalmologists-in-training from programs in Brazil, Mexico, and the Philippines participated. A secure web-based educational system was developed using a repository of more than 2,500 unique image sets of ROP, and a reference standard diagnosis was established by combining the clinical diagnosis and the image-based diagnosis by multiple experts. Twenty web-based cases of wide-field retinal images were presented, and ophthalmologists-in-training were asked to diagnose plus disease, zone, stage, and category for each eye. Trainees' responses were compared to the consensus reference standard diagnosis. Main outcome measures were frequency and types of diagnostic errors were analyzed. RESULTS: The error rate in the diagnosis of any category of ROP was between 48% and 59% for all countries. The error rate in identifying type 2 or pre-plus disease was 77%, with a tendency for overdiagnosis (27% underdiagnosis vs 50% overdiagnosis; mean difference: 23.4; 95% CI: 12.1 to 34.7; P = .005). Misdiagnosis of treatment-requiring ROP as type 2 ROP was most commonly associated with incorrectly identifying plus disease (plus disease error rate = 18% with correct category diagnosis vs 69% when misdiagnosed; mean difference: 51.0; 95% CI: 49.3 to 52.7; P = .003). CONCLUSIONS: Ophthalmologists-in-training from middle-income countries misdiagnosed ROP more than half of the time. Identification of plus disease was the salient factor leading to incorrect diagnosis. These findings emphasize the need for improved access to ROP education to improve competency in diagnosis among ophthalmologists-in-training in middle-income countries. [J Pediatr Ophthalmol Strabismus. 2023;60(5):344-352.].

Characterization of Errors in Retinopathy of Prematurity Diagnosis by Ophthalmologists-in-Training in the United States and Canada.

PURPOSE: To identify the prominent factors that lead to misdiagnosis of retinopathy of prematurity (ROP) by ophthalmologists-in-training in the United States and Canada. METHODS: This prospective cohort study included 32 ophthalmologists-in-training at six ophthalmology training programs in the United States and Canada. Twenty web-based cases of ROP using wide-field retinal images were presented, and ophthalmologists-in-training were asked to diagnose plus disease, zone, stage, and category for each eye. Responses were compared to a consensus reference standard diagnosis for accuracy, which was established by combining the clinical diagnosis and the image-based diagnosis by multiple experts. The types of diagnostic errors that occurred were analyzed with descriptive and chi-squared analysis. Main outcome measures were frequency of types (category, zone, stage, plus disease) of diagnostic errors; association of errors in zone, stage, and plus disease diagnosis with incorrectly identified category; and performance of ophthalmologists-in-training across postgraduate years. RESULTS: Category of ROP was misdiagnosed at a rate of 48%. Errors in classification of plus disease were most commonly associated with misdiagnosis of treatment-requiring (plus error rate = 16% when treatment-requiring was correctly diagnosed vs 81% when underdiagnosed as type 2 or pre-plus; mean difference: 64.3; 95% CI: 51.9 to 76.7; P < .001) and type 2 or pre-plus (plus error rate = 35% when type 2 or pre-plus was correctly diagnosed vs 76% when overdiagnosed as treatment-requiring; mean difference: 41.0; 95% CI: 28.4 to 53.5; P < .001) disease. The diagnostic error rate of postgraduate year (PGY)-2 trainees was significantly higher than PGY-3 trainees (PGY-2 category error rate = 61% vs PGY-3 = 35%; mean difference, 25.4; 95% CI: 17.7 to 33.0; P < .001). CONCLUSIONS: Ophthalmologists-in-training in the United States and Canada misdiagnosed ROP nearly half of the time, with incorrect identification of plus disease as a leading cause. Integration of structured learning for ROP in residency education may improve diagnostic competency. [J Pediatr Ophthalmol Strabismus. 2023;60(5):337-343.].

Improved Training Efficiency for Retinopathy of Prematurity Deep Learning Models Using Comparison versus Class Labels.

Purpose: To compare the efficacy and efficiency of training neural networks for medical image classification using comparison labels indicating relative disease severity versus diagnostic class labels from a retinopathy of prematurity (ROP) image dataset. Design: Evaluation of diagnostic test or technology. Participants: Deep learning neural networks trained on expert-labeled wide-angle retinal images obtained from patients undergoing diagnostic ROP examinations obtained as part of the Imaging and Informatics in ROP (i-ROP) cohort study. Methods: Neural networks were trained with either class or comparison labels indicating plus disease severity in ROP retinal fundus images from 2 datasets. After training and validation, all networks underwent evaluation using a separate test dataset in 1 of 2 binary classification tasks: normal versus abnormal or plus versus nonplus. Main Outcome Measures: Area under the receiver operating characteristic curve (AUC) values were measured to assess network performance. Results: Given the same number of labels, neural networks learned more efficiently by comparison, generating significantly higher AUCs in both classification tasks across both datasets. Similarly, given the same number of images, comparison learning developed networks with significantly higher AUCs across both classification tasks in 1 of 2 datasets. The difference in efficiency and accuracy between models trained on either label type decreased as the size of the training set increased. Conclusions: Comparison labels individually are more informative and more abundant per sample than class labels. These findings indicate a potential means of overcoming the common obstacle of data variability and scarcity when training neural networks for medical image classification tasks.

Variability in Plus Disease Diagnosis using Single and Serial Images.

PURPOSE: To assess changes in retinopathy of prematurity (ROP) diagnosis in single and serial retinal images. DESIGN: Cohort study. PARTICIPANTS: Cases of ROP recruited from the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) consortium evaluated by 7 graders. METHODS: Seven ophthalmologists reviewed both single and 3 consecutive serial retinal images from 15 cases with ROP, and severity was assigned as plus, preplus, or none. Imaging data were acquired during routine ROP screening from 2011 to 2015, and a reference standard diagnosis was established for each image. A secondary analysis was performed using the i-ROP deep learning system to assign a vascular severity score (VSS) to each image, ranging from 1 to 9, with 9 being the most severe disease. This score has been previously demonstrated to correlate with the International Classification of ROP. Mean plus disease severity was calculated by averaging 14 labels per image in serial and single images to decrease noise. MAIN OUTCOME MEASURES: Grading severity of ROP as defined by plus, preplus, or no ROP. RESULTS: Assessment of serial retinal images changed the grading severity for > 50% of the graders, although there was wide variability. Cohen's kappa ranged from 0.29 to 1.0, which showed a wide range of agreement from slight to perfect by each grader. Changes in the grading of serial retinal images were noted more commonly in cases of preplus disease. The mean severity in cases with a diagnosis of plus disease and no disease did not change between single and serial images. The ROP VSS demonstrated good correlation with the range of expert classifications of plus disease and overall agreement with the mode class (P = 0.001). The VSS correlated with mean plus disease severity by expert diagnosis (correlation coefficient, 0.89). The more aggressive graders tended to be influenced by serial images to increase the severity of their grading. The VSS also demonstrated agreement with disease progression across serial images, which progressed to preplus and plus disease. CONCLUSIONS: Clinicians demonstrated variability in ROP diagnosis when presented with both single and serial images. The use of deep learning as a quantitative assessment of plus disease has the potential to standardize ROP diagnosis and treatment.

External Validation of a Retinopathy of Prematurity Screening Model Using Artificial Intelligence in 3 Low- and Middle-Income Populations.

Importance: Retinopathy of prematurity (ROP) is a leading cause of preventable blindness that disproportionately affects children born in low- and middle-income countries (LMICs). In-person and telemedical screening examinations can reduce this risk but are challenging to implement in LMICs owing to the multitude of at-risk infants and lack of trained ophthalmologists. Objective: To implement an ROP risk model using retinal images from a single baseline examination to identify infants who will develop treatment-requiring (TR)-ROP in LMIC telemedicine programs. Design, Setting, and Participants: In this diagnostic study conducted from February 1, 2019, to June 30, 2021, retinal fundus images were collected from infants as part of an Indian ROP telemedicine screening program. An artificial intelligence (AI)-derived vascular severity score (VSS) was obtained from images from the first examination after 30 weeks' postmenstrual age. Using 5-fold cross-validation, logistic regression models were trained on 2 variables (gestational age and VSS) for prediction of TR-ROP. The model was externally validated on test data sets from India, Nepal, and Mongolia. Data were analyzed from October 20, 2021, to April 20, 2022. Main Outcomes and Measures: Primary outcome measures included sensitivity, specificity, positive predictive value, and negative predictive value for predictions of future occurrences of TR-ROP; the number of weeks before clinical diagnosis when a prediction was made; and the potential reduction in number of examinations required. Results: A total of 3760 infants (median [IQR] postmenstrual age, 37 [5] weeks; 1950 male infants [51.9%]) were included in the study. The diagnostic model had a sensitivity and specificity, respectively, for each of the data sets as follows: India, 100.0% (95% CI, 87.2%-100.0%) and 63.3% (95% CI, 59.7%-66.8%); Nepal, 100.0% (95% CI, 54.1%-100.0%) and 77.8% (95% CI, 72.9%-82.2%); and Mongolia, 100.0% (95% CI, 93.3%-100.0%) and 45.8% (95% CI, 39.7%-52.1%). With the AI model, infants with TR-ROP were identified a median (IQR) of 2.0 (0-11) weeks before TR-ROP diagnosis in India, 0.5 (0-2.0) weeks before TR-ROP diagnosis in Nepal, and 0 (0-5.0) weeks before TR-ROP diagnosis in Mongolia. If low-risk infants were never screened again, the population could be effectively screened with 45.0% (India, 664/1476), 38.4% (Nepal, 151/393), and 51.3% (Mongolia, 266/519) fewer examinations required. Conclusions and Relevance: Results of this diagnostic study suggest that there were 2 advantages to implementation of this risk model: (1) the number of examinations for low-risk infants could be reduced without missing cases of TR-ROP, and (2) high-risk infants could be identified and closely monitored before development of TR-ROP.

Evaluation of an Artificial Intelligence System for Retinopathy of Prematurity Screening in Nepal and Mongolia.

Purpose: To evaluate the performance of a deep learning (DL) algorithm for retinopathy of prematurity (ROP) screening in Nepal and Mongolia. Design: Retrospective analysis of prospectively collected clinical data. Participants: Clinical information and fundus images were obtained from infants in 2 ROP screening programs in Nepal and Mongolia. Methods: Fundus images were obtained using the Forus 3nethra neo (Forus Health) in Nepal and the RetCam Portable (Natus Medical, Inc.) in Mongolia. The overall severity of ROP was determined from the medical record using the International Classification of ROP (ICROP). The presence of plus disease was determined independently in each image using a reference standard diagnosis. The Imaging and Informatics for ROP (i-ROP) DL algorithm was trained on images from the RetCam to classify plus disease and to assign a vascular severity score (VSS) from 1 through 9. Main Outcome Measures: Area under the receiver operating characteristic curve and area under the precision-recall curve for the presence of plus disease or type 1 ROP and association between VSS and ICROP disease category. Results: The prevalence of type 1 ROP was found to be higher in Mongolia (14.0%) than in Nepal (2.2%; P < 0.001) in these data sets. In Mongolia (RetCam images), the area under the receiver operating characteristic curve for examination-level plus disease detection was 0.968, and the area under the precision-recall curve was 0.823. In Nepal (Forus images), these values were 0.999 and 0.993, respectively. The ROP VSS was associated with ICROP classification in both datasets (P < 0.001). At the population level, the median VSS was found to be higher in Mongolia (2.7; interquartile range [IQR], 1.3-5.4]) as compared with Nepal (1.9; IQR, 1.2-3.4; P < 0.001). Conclusions: These data provide preliminary evidence of the effectiveness of the i-ROP DL algorithm for ROP screening in neonatal populations in Nepal and Mongolia using multiple camera systems and are useful for consideration in future clinical implementation of artificial intelligence-based ROP screening in low- and middle-income countries.

Evaluation of Potential Systemic Adverse Events Related to Fluorescein Angiography in Pediatric Patients.

PURPOSE: To evaluate adverse events of fluorescein angiography (FA) in pediatric patients. DESIGN: Single-institution retrospective chart review. PARTICIPANTS: Patients 0 to 18 years of age who underwent FA between January 2010 and December 2015 at a single institution in the United States. METHODS: Pediatric patients who underwent FA by 3 surgeons were included in the study. Patients with fewer than 24 hours of documented follow-up were excluded. Significant adverse events within 24 hours of FA were evaluated. Detailed intraoperative and perioperative physiological parameters, including heart rate, blood pressure, oxygen saturation, and ventilation parameters, in inpatients undergoing simultaneous examination under anesthesia were reviewed. Peri-injection effects of FA were evaluated by 2-tailed paired t test comparison of mean 5-minute preinjection and 5-minute postinjection physiological data. MAIN OUTCOME MEASURES: Significant adverse events associated with FA. RESULTS: One hundred fifteen patients with a total of 214 FA examinations were included. No significant adverse events were associated directly with FA. Comparison of mean 5-minute preinjection and postinjection physiologic parameters in 27 patients who underwent intravenous FA during EUA did not reveal significant changes associated with FA. A significant difference was found in average patient age between inpatient (2.5 years) and outpatient (10.7 years) FA (P < 0.00001). The youngest patients who underwent successful FA were 3.8 years old in the outpatient setting and 32 weeks' postmenstrual age in the inpatient setting. Patients younger than 3.8 years accounted for most (77.6%; n = 85) inpatient FA examinations. Excluding patients with a need or likely need for laser or surgery, the reasons for inpatient FA in patients older than 3.8 years included the lack of availability of outpatient ultra-widefield FA (UWFA) and more challenging situations in patients with developmental delay. CONCLUSIONS: Fluorescein angiography was not found to be associated directly with systemic adverse events in pediatric patients in this study. Younger patients more commonly were found to require an inpatient FA, whereas older patients older than 4 years underwent outpatient UWFA.

Development of Screening Criteria for Retinopathy of Prematurity in Ulaanbaatar, Mongolia, Using a Web-based Data Management System.

PURPOSE: To describe a process for identifying birth weight (BW) and gestational age (GA) screening guidelines in Mongolia. METHODS: This was a prospective cohort study in a tertiary care hospital in Ulaanbataar, Mongolia, of 193 premature infants with GA of 36 weeks or younger and/or BW of 2,000 g or less) with regression analysis to determine associations between BW and GA and the development of retinopathy of prematurity (ROP). RESULTS: As BW and GA decreased, the relative risk of developing ROP increased. The relative risk of developing any stage of ROP in infants born at 29 weeks or younger was 2.91 (95% CI: 1.55 to 5.44; P < .001] compared to older infants. The relative risk of developing any type of ROP in infants with BW of less than 1,200 g was 2.41 (95% CI: 1.35 to 4.29; P = .003] and developing type 2 or worse ROP was 2.05 (95% CI: 0.99 to 4.25; P = .05). CONCLUSIONS: Infants in Mongolia with heavier BW and older GA who fall outside of current United States screening guidelines of GA of 30 weeks or younger and/or BW of 1,500 g or less developed clinically relevant ROP. [J Pediatr Ophthalmol Strabismus. 2020;57(5):333-339.].

Training of Residents and Fellows in Retinopathy of Prematurity Around the World: An International Web-Based Survey.

PURPOSE: To characterize retinopathy of prematurity (ROP) training practices in international residency and fellowship programs. METHODS: A publicly available online-based platform (http://www.SurveyMonkey.com) was used to develop a 28-question multiple-choice survey that targeted ROP screening and treatment methods. The authors solicited training programs in the Philippines, Thailand, and Taiwan. RESULTS: Programs from three countries participated in the survey, and a total of 95 responses collected from residents, fellows, and attending ophthalmologists were analyzed. A descriptive analysis demonstrated that 45 participants (47%) reported 1% to 33% of ROP screenings were performed under direct supervision of attending ophthalmologists, and 35 (37%) reported the use of formal assessments. The majority of participants (Country A: 87%, Country B: 71%, and Country C: 75%) estimated 1% to 33% of their practice was spent screening for ROP. Notably, 44 participants (46%) reported performing zero laser photocoagulation treatments for ROP during training (Country A: 65%, Country B: 38%, and Country C: 38%). CONCLUSIONS: International ophthalmology trainees perform a limited number of ROP examinations and laser interventions. ROP screenings are often unsupervised and lead to no formal evaluation by an attending ophthalmologist. Limited ROP training among ophthalmologists may lead to misdiagnosis and ultimately mismanagement of a patient. Loss of vision and exposure to unwarranted treatments are among the implications of such errors. The findings highlight the need to improve ROP training in international ophthalmology residency and fellowship programs. [J Pediatr Ophthalmol Strabismus. 2019;56(5):282-287.].

Automated Fundus Image Quality Assessment in Retinopathy of Prematurity Using Deep Convolutional Neural Networks.

PURPOSE: Accurate image-based ophthalmic diagnosis relies on fundus image clarity. This has important implications for the quality of ophthalmic diagnoses and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a deep convolutional neural network (CNN) for automated assessment of fundus image quality in retinopathy of prematurity (ROP). DESIGN: Experimental study. PARTICIPANTS: Retinal fundus images were collected from preterm infants during routine ROP screenings. METHODS: Six thousand one hundred thirty-nine retinal fundus images were collected from 9 academic institutions. Each image was graded for quality (acceptable quality [AQ], possibly acceptable quality [PAQ], or not acceptable quality [NAQ]) by 3 independent experts. Quality was defined as the ability to assess an image confidently for the presence of ROP. Of the 6139 images, NAQ, PAQ, and AQ images represented 5.6%, 43.6%, and 50.8% of the image set, respectively. Because of low representation of NAQ images in the data set, images labeled NAQ were grouped into the PAQ category, and a binary CNN classifier was trained using 5-fold cross-validation on 4000 images. A test set of 2109 images was held out for final model evaluation. Additionally, 30 images were ranked from worst to best quality by 6 experts via pairwise comparisons, and the CNN's ability to rank quality, regardless of quality classification, was assessed. MAIN OUTCOME MEASURES: The CNN performance was evaluated using area under the receiver operating characteristic curve (AUC). A Spearman's rank correlation was calculated to evaluate the overall ability of the CNN to rank images from worst to best quality as compared with experts. RESULTS: The mean AUC for 5-fold cross-validation was 0.958 (standard deviation, 0.005) for the diagnosis of AQ versus PAQ images. The AUC was 0.965 for the test set. The Spearman's rank correlation coefficient on the set of 30 images was 0.90 as compared with the overall expert consensus ranking. CONCLUSIONS: This model accurately assessed retinal fundus image quality in a comparable manner with that of experts. This fully automated model has potential for application in clinical settings, telemedicine, and computer-based image analysis in ROP and for generalizability to other ophthalmic diseases.

Early Diagnosis and Management of Aggressive Posterior Vitreoretinopathy Presenting in Premature Neonates.

BACKGROUND AND OBJECTIVE: Aggressive posterior vitreoretinopathy (APVR) manifests with a broad area of retinal avascularity, progressive neovascularization, and/or tractional retinal detachment during the neonatal period. PATIENTS AND METHODS: A multicenter, retrospective, observational, consecutive case series study was performed to evaluate the retinal findings and structural retinal outcomes in patients treated for APVR within the first 3 months of life. RESULTS: Three premature neonates with a non-retinopathy of prematurity (ROP) APVR identified during routine ROP screening exams exhibited relatively severe, rapidly progressive retinal vascular abnormalities. Immediate laser photocoagulation of the avascular retina and vitrectomy for traction retinal detachment within several days to weeks improved or stabilized the retinal anatomy in all cases. CONCLUSIONS: This series describes clinical features in APVR in premature infants and suggests that early diagnosis and intervention may mitigate the typical aggressive course and poor prognosis of this condition. [Ophthalmic Surg Lasers Imaging Retina. 2019;50:201-207.].

Retinal Telemedicine.

PURPOSE OF REVIEW: An update and overview of the literature on current telemedicine applications in retina. RECENT FINDINGS: The application of telemedicine to the field of Ophthalmology and Retina has been growing with advancing technologies in ophthalmic imaging. Retinal telemedicine has been most commonly applied to diabetic retinopathy and retinopathy of prematurity in adult and pediatric patients respectively. Telemedicine has the potential to alleviate the growing demand for clinical evaluation of retinal diseases. Subsequently, automated image analysis and deep learning systems may facilitate efficient processing of large, increasing numbers of images generated in telemedicine systems. Telemedicine may additionally improve access to education and standardized training through tele-education systems. SUMMARY: Telemedicine has the potential to be utilized as a useful adjunct but not a complete replacement for physical clinical examinations. Retinal telemedicine programs should be carefully and appropriately integrated into current clinical systems.