Comparison of 2-Field and 5-Field Mydriatic Handheld Retinal Imaging in a Community-Based Diabetic Retinopathy Screening Program.

INTRODUCTION: The purpose of this study was to compare 2-field (2F) and 5-field (5F) mydriatic handheld retinal imaging for the assessment of diabetic retinopathy (DR) severity in a community-based DR screening program (DRSP). METHODS: This was a prospective, cross-sectional diagnostic study, evaluating images of 805 eyes from 407 consecutive patients with diabetes acquired from a community-based DRSP. Mydriatic standardized 5F imaging (macula, disc, superior, inferior, temporal) with handheld retinal camera was performed. 2F (disc, macula), and 5F images were independently assessed using the International DR classification at a centralized reading center. Simple (K) and weighted (Kw) kappa statistics were calculated for DR. Sensitivity and specificity for referable DR ([refDR] moderate nonproliferative DR [NPDR] or worse) and vision-threatening DR ([vtDR] severe NPDR or worse) for 2F compared to 5F imaging were calculated. RESULTS: Distribution of DR severity by 2F/5F images (%): no DR 66.0/61.7, mild NPDR 10.7/14.4, moderate NPDR 7.9/8.1, severe NPDR 3.3/5.6, proliferative DR 5.6/4.6, ungradable 6.5/5.6. Exact agreement of DR grading between 2F and 5F was 81.7%, within 1-step 97.1% (K = 0.64, Kw = 0.78). Sensitivity/specificity for 2F compared 5F was refDR 0.80/0.97, vtDR 0.73/0.98. The ungradable images rate with 2F was 16.1% higher than with 5F (6.5 vs. 5.6%, p < 0.001). CONCLUSIONS: Mydriatic 2F and 5F handheld imaging have substantial agreement in assessing severity of DR. However, the use of mydriatic 2F handheld imaging only meets the minimum standards for sensitivity and specificity for refDR but not for vtDR. When using handheld cameras, the addition of peripheral fields in 5F imaging further refines the referral approach by decreasing ungradable rate and increasing sensitivity for vtDR.

Comparisons of Handheld Retinal Imaging with Optical Coherence Tomography for the Identification of Macular Pathology in Patients with Diabetes.

INTRODUCTION: Handheld retinal imaging cameras are relatively inexpensive and highly portable devices that have the potential to significantly expand diabetic retinopathy (DR) screening, allowing a much broader population to be evaluated. However, it is essential to evaluate if these devices can accurately identify vision-threatening macular diseases if DR screening programs will rely on these instruments. Thus, the purpose of this study was to evaluate the detection of diabetic macular pathology using monoscopic macula-centered images using mydriatic handheld retinal imaging compared with spectral domain optical coherence tomography (SDOCT). METHODS: Mydriatic 40°-60° macula-centered images taken with 3 handheld retinal imaging devices (Aurora [AU], SmartScope [SS], RetinaVue 700 [RV]) were compared with the Cirrus 6000 SDOCT taken during the same visit. Images were evaluated for the presence of diabetic macular edema (DME) on monoscopic fundus photographs adapted from Early Treatment Diabetic Retinopathy Study (ETDRS) definitions (no DME, noncenter-involved DME [non-ciDME], and center-involved DME [ciDME]). Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for each device with SDOCT as gold standard. RESULTS: Severity by ETDRS photos: no DR 33.3%, mild NPDR 20.4%, moderate 14.2%, severe 11.6%, proliferative 20.4%, and ungradable for DR 0%; no DME 83.1%, non-ciDME 4.9%, ciDME 12.0%, and ungradable for DME 0%. Gradable images by SDOCT (N = 217, 96.4%) showed no DME in 75.6%, non-ciDME in 9.8%, and ciDME in 11.1%. The ungradable rate for images (poor visualization in >50% of the macula) was AU: 0.9%, SS: 4.4%, and RV: 6.2%. For DME, sensitivity and specificity were similar across devices (0.5-0.64, 0.93-0.97). For nondiabetic macular pathology (ERM, pigment epithelial detachment, traction retinal detachment) across all devices, sensitivity was low to moderate (0.2-0.5) but highly specific (0.93-1.00). CONCLUSIONS: Compared to SDOCT, handheld macular imaging attained high specificity but low sensitivity in identifying macular pathology. This suggests the importance of SDOCT evaluation for patients suspected to have DME on fundus photography, leading to more appropriate referral refinement.

Accuracy of Integrated Artificial Intelligence Grading Using Handheld Retinal Imaging in a Community Diabetic Eye Screening Program.

Purpose: To evaluate mydriatic handheld retinal imaging performance assessed by point-of-care (POC) artificial intelligence (AI) as compared with retinal image graders at a centralized reading center (RC) in identifying diabetic retinopathy (DR) and diabetic macular edema (DME). Design: Prospective, comparative study. Subjects: Five thousand five hundred eighty-five eyes from 2793 adult patients with diabetes. Methods: Point-of-care AI assessment of disc and macular handheld retinal images was compared with RC evaluation of validated 5-field handheld retinal images (disc, macula, superior, inferior, and temporal) in identifying referable DR (refDR; defined as moderate nonproliferative DR [NPDR], or worse, or any level of DME) and vision-threatening DR (vtDR; defined as severe NPDR or worse, or any level of center-involving DME [ciDME]). Reading center evaluation of the 5-field images followed the international DR/DME classification. Sensitivity (SN) and specificity (SP) for ungradable images, refDR, and vtDR were calculated. Main Outcome Measures: Agreement for DR and DME; SN and SP for refDR, vtDR, and ungradable images. Results: Diabetic retinopathy severity by RC evaluation: no DR, 67.3%; mild NPDR, 9.7%; moderate NPDR, 8.6%; severe NPDR, 4.8%; proliferative DR, 3.8%; and ungradable, 5.8%. Diabetic macular edema severity by RC evaluation was as follows: no DME (80.4%), non-ciDME (7.7%), ciDME (4.4%), and ungradable (7.5%). Referable DR was present in 25.3% and vtDR was present in 17.5% of eyes. Images were ungradable for DR or DME in 7.5% by RC evaluation and 15.4% by AI. There was substantial agreement between AI and RC for refDR (κ = 0.66) and moderate agreement for vtDR (κ = 0.54). The SN/SP of AI grading compared with RC evaluation was 0.86/0.86 for refDR and 0.92/0.80 for vtDR. Conclusions: This study demonstrates that POC AI following a defined handheld retinal imaging protocol at the time of imaging has SN and SP for refDR that meets the current United States Food and Drug Administration thresholds of 85% and 82.5%, but not for vtDR. Integrating AI at the POC could substantially reduce centralized RC burden and speed information delivery to the patient, allowing more prompt eye care referral. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Comparisons of handheld retinal imaging devices with ultrawide field images for determining diabetic retinopathy severity.

PURPOSE: To compare diabetic retinopathy (DR) severity identified on handheld retinal imaging with ultrawide field (UWF) images. METHODS: Mydriatic images of 225 eyes of 118 diabetic patients were prospectively imaged with the Aurora (AU) handheld retinal camera [5-field protocol (macula-centred, disc-centred, temporal, superior, inferior)] and compared with UWF images. Images were classified based on the international classification for DR. Sensitivity, specificity, kappa statistics (K/Kw) were calculated on an eye and person-level. RESULTS: Distribution of DR severity by AU/UWF images (%) by eye was no DR 41.3/36.0, mild non-proliferative DR (NPDR) 18.7/17.8, moderate 10.2/10.7, severe 16.4/15.1, proliferative DR (PDR) 13.3/20.4. Agreement between UWF and AU was exact in 64.4%, within 1-step 90.7%, k = 0.55 (95% CI:0.45-0.65), and kw = 0.79 (95% CI:0.73-0.85) by eye, and exact in 68%, within 1-step 92.9%, k = 0.58 (95% CI:0.50-0.66), and kw = 0.76 (95% CI:0.70-0.81) by person. Sensitivity/specificity for any DR, refDR, vtDR and PDR were as follows: 0.90/0.83, 0.90/0.97, 0.82/0.95 and 0.69/1.00 by person and 0.86/0.90, 0.84/0.98, 0.75/0.95 and 0.63/0.99 by eye. Handheld imaging missed 37% (17/46) eyes and 30.8% (8/26) persons with PDR. Only 3.9% (1/26) persons or 6.5% (3/46) eyes with PDR were missed if a referral threshold of moderate NPDR was used. CONCLUSIONS: Data from this study show that comparing UWF and handheld images, when PDR was the referral threshold for handheld devices, 37.0% of eyes or 30.8% of patients with PDR were missed. Due to the identification of neovascular lesions outside of the handheld fields, lower referral thresholds are needed if handheld devices are used.

One-field, two-field and five-field handheld retinal imaging compared with standard seven-field Early Treatment Diabetic Retinopathy Study photography for diabetic retinopathy screening.

BACKGROUND/AIMS: To determine agreement of one-field (1F, macula-centred), two-field (2F, disc-macula) and five-field (5F, macula, disc, superior, inferior and nasal) mydriatic handheld retinal imaging protocols for the assessment of diabetic retinopathy (DR) as compared with standard seven-field Early Treatment Diabetic Retinopathy Study (ETDRS) photography. METHODS: Prospective, comparative instrument validation study. Mydriatic retinal images were taken using three handheld retinal cameras: Aurora (AU; 50° field of view (FOV), 5F), Smartscope (SS; 40° FOV, 5F), and RetinaVue (RV; 60° FOV, 2F) followed by ETDRS photography. Images were evaluated at a centralised reading centre using the international DR classification. Each field protocol (1F, 2F and 5F) was graded independently by masked graders. Weighted kappa (Kw) statistics assessed agreement for DR. Sensitivity (SN) and specificity (SP) for referable diabetic retinopathy (refDR; moderate non-proliferative diabetic retinopathy (NPDR) or worse, or ungradable images) were calculated. RESULTS: Images from 225 eyes of 116 patients with diabetes were evaluated. Severity by ETDRS photography: no DR, 33.3%; mild NPDR, 20.4%; moderate, 14.2%; severe, 11.6%; proliferative, 20.4%. Ungradable rate for DR: ETDRS, 0%; AU: 1F 2.23%, 2F 1.79%, 5F 0%; SS: 1F 7.6%, 2F 4.0%, 5F 3.6%; RV: 1F 6.7%, 2F 5.8%. Agreement rates of DR grading between handheld retinal imaging and ETDRS photography were (Kw, SN/SP refDR) AU: 1F 0.54, 0.72/0.92; 2F 0.59, 0.74/0.92; 5F 0.75, 0.86/0.97; SS: 1F 0.51, 0.72/0.92; 2F 0.60, 0.75/0.92; 5F 0.73, 0.88/0.92; RV: 1F 0.77, 0.91/0.95; 2F 0.75, 0.87/0.95. CONCLUSION: When using handheld devices, the addition of peripheral fields decreased the ungradable rate and increased SN and SP for refDR. These data suggest the benefit of additional peripheral fields in DR screening programmes that use handheld retinal imaging.

Performance of Automated Machine Learning for Diabetic Retinopathy Image Classification from Multi-field Handheld Retinal Images.

PURPOSE: To create and validate code-free automated deep learning models (AutoML) for diabetic retinopathy (DR) classification from handheld retinal images. DESIGN: Prospective development and validation of AutoML models for DR image classification. PARTICIPANTS: A total of 17 829 deidentified retinal images from 3566 eyes with diabetes, acquired using handheld retinal cameras in a community-based DR screening program. METHODS: AutoML models were generated based on previously acquired 5-field (macula-centered, disc-centered, superior, inferior, and temporal macula) handheld retinal images. Each individual image was labeled using the International DR and diabetic macular edema (DME) Classification Scale by 4 certified graders at a centralized reading center under oversight by a senior retina specialist. Images for model development were split 8-1-1 for training, optimization, and testing to detect referable DR ([refDR], defined as moderate nonproliferative DR or worse or any level of DME). Internal validation was performed using a published image set from the same patient population (N = 450 images from 225 eyes). External validation was performed using a publicly available retinal imaging data set from the Asia Pacific Tele-Ophthalmology Society (N = 3662 images). MAIN OUTCOME MEASURES: Area under the precision-recall curve (AUPRC), sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 scores. RESULTS: Referable DR was present in 17.3%, 39.1%, and 48.0% of the training set, internal validation, and external validation sets, respectively. The model's AUPRC was 0.995 with a precision and recall of 97% using a score threshold of 0.5. Internal validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.96 (95% confidence interval [CI], 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.96 (95% CI, 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.97, and 0.96, respectively. External validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.94 (95% CI, 0.929-0.951), 0.97 (95% CI, 0.957-0.974), 0.96 (95% CI, 0.952-0.971), 0.95 (95% CI, 0.935-0.956), 0.97, and 0.96, respectively. CONCLUSIONS: This study demonstrates the accuracy and feasibility of code-free AutoML models for identifying refDR developed using handheld retinal imaging in a community-based screening program. Potentially, the use of AutoML may increase access to machine learning models that may be adapted for specific programs that are guided by the clinical need to rapidly address disparities in health care delivery. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Comparison of Handheld Retinal Imaging with ETDRS 7-Standard Field Photography for Diabetic Retinopathy and Diabetic Macular Edema.

PURPOSE: To compare nonmydriatic (NM) and mydriatic (MD) handheld retinal imaging with standard ETDRS 7-field color fundus photography (ETDRS photographs) for the assessment of diabetic retinopathy (DR) and diabetic macular edema (DME). DESIGN: Prospective, comparative, instrument validation study. SUBJECTS: A total of 225 eyes from 116 patients with diabetes mellitus. METHODS: Following a standardized protocol, NM and MD images were acquired using handheld retinal cameras (NM images: Aurora, Smartscope, and RetinaVue-700; MD images: Aurora, Smartscope, RetinaVue-700, and iNview) and dilated ETDRS photographs. Grading was performed at a centralized reading center using the International Clinical Classification for DR and DME. Kappa statistics (simple [K], weighted [Kw]) assessed the level of agreement for DR and DME. Sensitivity and specificity were calculated for any DR, referable DR (refDR), and vision-threatening DR (vtDR). MAIN OUTCOME MEASURES: Agreement for DR and DME; sensitivity and specificity for any DR, refDR, and vtDR; ungradable rates. RESULTS: Severity by ETDRS photographs: no DR, 33.3%; mild nonproliferative DR, 20.4%; moderate DR, 14.2%; severe DR, 11.6%; proliferative DR, 20.4%; no DME, 68.0%; DME, 9.3%; non-center involving clinically significant DME, 4.9%; center-involving clinically significant DME, 12.4%; and ungradable, 5.3%. For NM handheld retinal imaging, Kw was 0.70 to 0.73 for DR and 0.76 to 0.83 for DME. For MD handheld retinal imaging, Kw was 0.68 to 0.75 for DR and 0.77 to 0.91 for DME. Thresholds for sensitivity (0.80) and specificity (0.95) were met by NM images acquired using Smartscope and MD images acquired using Aurora and RetinaVue-700 cameras for any DR and by MD images acquired using Aurora and RetinaVue-700 cameras for refDR. Thresholds for sensitivity and specificity were met by MD images acquired using Aurora and RetinaVue-700 for DME. Nonmydriatic and MD ungradable rates for DR were 15.1% to 38.3% and 0% to 33.8%, respectively. CONCLUSIONS: Following standardized protocols, NM and MD handheld retinal imaging devices have substantial agreement levels for DR and DME. With mydriasis, not all handheld retinal imaging devices meet standards for sensitivity and specificity in identifying any DR and refDR. None of the handheld devices met the established 95% specificity for vtDR, suggesting that lower referral thresholds should be used if handheld devices must be utilized. When using handheld devices, the ungradable rate is significantly reduced with mydriasis and DME sensitivity thresholds are only achieved following dilation.