Comparisons of Handheld Retinal Imaging with Optical Coherence Tomography for the Identification of Macular Pathology in Patients with Diabetes.

INTRODUCTION: Handheld retinal imaging cameras are relatively inexpensive and highly portable devices that have the potential to significantly expand diabetic retinopathy (DR) screening, allowing a much broader population to be evaluated. However, it is essential to evaluate if these devices can accurately identify vision-threatening macular diseases if DR screening programs will rely on these instruments. Thus, the purpose of this study was to evaluate the detection of diabetic macular pathology using monoscopic macula-centered images using mydriatic handheld retinal imaging compared with spectral domain optical coherence tomography (SDOCT). METHODS: Mydriatic 40°-60° macula-centered images taken with 3 handheld retinal imaging devices (Aurora [AU], SmartScope [SS], RetinaVue 700 [RV]) were compared with the Cirrus 6000 SDOCT taken during the same visit. Images were evaluated for the presence of diabetic macular edema (DME) on monoscopic fundus photographs adapted from Early Treatment Diabetic Retinopathy Study (ETDRS) definitions (no DME, noncenter-involved DME [non-ciDME], and center-involved DME [ciDME]). Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for each device with SDOCT as gold standard. RESULTS: Severity by ETDRS photos: no DR 33.3%, mild NPDR 20.4%, moderate 14.2%, severe 11.6%, proliferative 20.4%, and ungradable for DR 0%; no DME 83.1%, non-ciDME 4.9%, ciDME 12.0%, and ungradable for DME 0%. Gradable images by SDOCT (N = 217, 96.4%) showed no DME in 75.6%, non-ciDME in 9.8%, and ciDME in 11.1%. The ungradable rate for images (poor visualization in >50% of the macula) was AU: 0.9%, SS: 4.4%, and RV: 6.2%. For DME, sensitivity and specificity were similar across devices (0.5-0.64, 0.93-0.97). For nondiabetic macular pathology (ERM, pigment epithelial detachment, traction retinal detachment) across all devices, sensitivity was low to moderate (0.2-0.5) but highly specific (0.93-1.00). CONCLUSIONS: Compared to SDOCT, handheld macular imaging attained high specificity but low sensitivity in identifying macular pathology. This suggests the importance of SDOCT evaluation for patients suspected to have DME on fundus photography, leading to more appropriate referral refinement.

Performance of Automated Machine Learning for Diabetic Retinopathy Image Classification from Multi-field Handheld Retinal Images.

PURPOSE: To create and validate code-free automated deep learning models (AutoML) for diabetic retinopathy (DR) classification from handheld retinal images. DESIGN: Prospective development and validation of AutoML models for DR image classification. PARTICIPANTS: A total of 17 829 deidentified retinal images from 3566 eyes with diabetes, acquired using handheld retinal cameras in a community-based DR screening program. METHODS: AutoML models were generated based on previously acquired 5-field (macula-centered, disc-centered, superior, inferior, and temporal macula) handheld retinal images. Each individual image was labeled using the International DR and diabetic macular edema (DME) Classification Scale by 4 certified graders at a centralized reading center under oversight by a senior retina specialist. Images for model development were split 8-1-1 for training, optimization, and testing to detect referable DR ([refDR], defined as moderate nonproliferative DR or worse or any level of DME). Internal validation was performed using a published image set from the same patient population (N = 450 images from 225 eyes). External validation was performed using a publicly available retinal imaging data set from the Asia Pacific Tele-Ophthalmology Society (N = 3662 images). MAIN OUTCOME MEASURES: Area under the precision-recall curve (AUPRC), sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 scores. RESULTS: Referable DR was present in 17.3%, 39.1%, and 48.0% of the training set, internal validation, and external validation sets, respectively. The model's AUPRC was 0.995 with a precision and recall of 97% using a score threshold of 0.5. Internal validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.96 (95% confidence interval [CI], 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.96 (95% CI, 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.97, and 0.96, respectively. External validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.94 (95% CI, 0.929-0.951), 0.97 (95% CI, 0.957-0.974), 0.96 (95% CI, 0.952-0.971), 0.95 (95% CI, 0.935-0.956), 0.97, and 0.96, respectively. CONCLUSIONS: This study demonstrates the accuracy and feasibility of code-free AutoML models for identifying refDR developed using handheld retinal imaging in a community-based screening program. Potentially, the use of AutoML may increase access to machine learning models that may be adapted for specific programs that are guided by the clinical need to rapidly address disparities in health care delivery. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Comparison of Handheld Retinal Imaging with ETDRS 7-Standard Field Photography for Diabetic Retinopathy and Diabetic Macular Edema.

PURPOSE: To compare nonmydriatic (NM) and mydriatic (MD) handheld retinal imaging with standard ETDRS 7-field color fundus photography (ETDRS photographs) for the assessment of diabetic retinopathy (DR) and diabetic macular edema (DME). DESIGN: Prospective, comparative, instrument validation study. SUBJECTS: A total of 225 eyes from 116 patients with diabetes mellitus. METHODS: Following a standardized protocol, NM and MD images were acquired using handheld retinal cameras (NM images: Aurora, Smartscope, and RetinaVue-700; MD images: Aurora, Smartscope, RetinaVue-700, and iNview) and dilated ETDRS photographs. Grading was performed at a centralized reading center using the International Clinical Classification for DR and DME. Kappa statistics (simple [K], weighted [Kw]) assessed the level of agreement for DR and DME. Sensitivity and specificity were calculated for any DR, referable DR (refDR), and vision-threatening DR (vtDR). MAIN OUTCOME MEASURES: Agreement for DR and DME; sensitivity and specificity for any DR, refDR, and vtDR; ungradable rates. RESULTS: Severity by ETDRS photographs: no DR, 33.3%; mild nonproliferative DR, 20.4%; moderate DR, 14.2%; severe DR, 11.6%; proliferative DR, 20.4%; no DME, 68.0%; DME, 9.3%; non-center involving clinically significant DME, 4.9%; center-involving clinically significant DME, 12.4%; and ungradable, 5.3%. For NM handheld retinal imaging, Kw was 0.70 to 0.73 for DR and 0.76 to 0.83 for DME. For MD handheld retinal imaging, Kw was 0.68 to 0.75 for DR and 0.77 to 0.91 for DME. Thresholds for sensitivity (0.80) and specificity (0.95) were met by NM images acquired using Smartscope and MD images acquired using Aurora and RetinaVue-700 cameras for any DR and by MD images acquired using Aurora and RetinaVue-700 cameras for refDR. Thresholds for sensitivity and specificity were met by MD images acquired using Aurora and RetinaVue-700 for DME. Nonmydriatic and MD ungradable rates for DR were 15.1% to 38.3% and 0% to 33.8%, respectively. CONCLUSIONS: Following standardized protocols, NM and MD handheld retinal imaging devices have substantial agreement levels for DR and DME. With mydriasis, not all handheld retinal imaging devices meet standards for sensitivity and specificity in identifying any DR and refDR. None of the handheld devices met the established 95% specificity for vtDR, suggesting that lower referral thresholds should be used if handheld devices must be utilized. When using handheld devices, the ungradable rate is significantly reduced with mydriasis and DME sensitivity thresholds are only achieved following dilation.