A Comparison of Artificial Intelligence and Human Diabetic Retinal Image Interpretation in an Urban Health System.

INTRODUCTION: Artificial intelligence (AI) diabetic retinopathy (DR) software has the potential to decrease time spent by clinicians on image interpretation and expand the scope of DR screening. We performed a retrospective review to compare Eyenuk's EyeArt software (Woodland Hills, CA) to Temple Ophthalmology optometry grading using the International Classification of Diabetic Retinopathy scale. METHODS: Two hundred and sixty consecutive diabetic patients from the Temple Faculty Practice Internal Medicine clinic underwent 2-field retinal imaging. Classifications of the images by the software and optometrist were analyzed using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and McNemar's test. Ungradable images were analyzed to identify relationships with HbA1c, age, and ethnicity. Disagreements and a sample of 20% of agreements were adjudicated by a retina specialist. RESULTS: On patient level comparison, sensitivity for the software was 100%, while specificity was 77.78%. PPV was 19.15%, and NPV was 100%. The 38 disagreements between software and optometrist occurred when the optometrist classified a patient's images as non-referable while the software classified them as referable. Of these disagreements, a retina specialist agreed with the optometrist 57.9% the time (22/38). Of the agreements, the retina specialist agreed with both the program and the optometrist 96.7% of the time (28/29). There was a significant difference in numbers of ungradable photos in older patients (≥60) vs younger patients (<60) (p=0.003). CONCLUSIONS: The AI program showed high sensitivity with acceptable specificity for a screening algorithm. The high NPV indicates that the software is unlikely to miss DR but may refer patients unnecessarily.

A 15 month experience with a primary care-based telemedicine screening program for diabetic retinopathy.

BACKGROUND: Using telemedicine for diabetic retinal screening is becoming popular especially amongst at-risk urban communities with poor access to care. The goal of the diabetic telemedicine project at Temple University Hospital is to improve cost-effective access to appropriate retinal care to those in need of close monitoring and/or treatment. METHODS: This will be a retrospective review of 15 months of data from March 2016 to May 2017. We will investigate how many patients were screened, how interpretable the photographs were, how often the photographs generated a diagnosis of diabetic retinopathy (DR) based on the screening photo, and how many patients followed-up for an exam in the office, if indicated. RESULTS: Six-hundred eighty-nine (689) digital retinal screening exams on 1377 eyes of diabetic patients were conducted in Temple's primary care clinic. The majority of the photographs were read to have no retinopathy (755, 54.8%). Among all of the screening exams, 357 (51.8%) triggered a request for a referral to ophthalmology. Four-hundred forty-nine (449, 32.6%) of the photos were felt to be uninterpretable by the clinician. Referrals were meant to be requested for DR found in one or both eyes, inability to assess presence of retinopathy in one or both eyes, or for suspicion of a different ophthalmic diagnosis. Sixty-seven patients (9.7%) were suspected to have another ophthalmic condition based on other findings in the retinal photographs. Among the 34 patients that were successfully completed a referral visit to Temple ophthalmology, there was good concordance between the level of DR detected by their screening fundus photographs and visit diagnosis. CONCLUSIONS: Although a little more than half of the patients did not have diabetic eye disease, about half needed a referral to ophthalmology. However, only 9.5% of the referral-warranted exams actually received an eye exam. Mere identification of referral-warranted diabetic retinopathy and other ophthalmic conditions is not enough. A successful telemedicine screening program must close the communication gap between screening and diagnosis by reviewer to provide timely follow-up by eye care specialists.