What are the perceptions and concerns of people living with diabetes and National Health Service staff around the potential implementation of AI-assisted screening for diabetic eye disease? Development and validation of a survey for use in a secondary care screening setting.

INTRODUCTION: The English National Health Service (NHS) Diabetic Eye Screening Programme (DESP) performs around 2.3 million eye screening appointments annually, generating approximately 13 million retinal images that are graded by humans for the presence or severity of diabetic retinopathy. Previous research has shown that automated retinal image analysis systems, including artificial intelligence (AI), can identify images with no disease from those with diabetic retinopathy as safely and effectively as human graders, and could significantly reduce the workload for human graders. Some algorithms can also determine the level of severity of the retinopathy with similar performance to humans. There is a need to examine perceptions and concerns surrounding AI-assisted eye-screening among people living with diabetes and NHS staff, if AI was to be introduced into the DESP, to identify factors that may influence acceptance of this technology. METHODS AND ANALYSIS: People living with diabetes and staff from the North East London (NEL) NHS DESP were invited to participate in two respective focus groups to codesign two online surveys exploring their perceptions and concerns around the potential introduction of AI-assisted screening.Focus group participants were representative of the local population in terms of ages and ethnicity. Participants' feedback was taken into consideration to update surveys which were circulated for further feedback. Surveys will be piloted at the NEL DESP and followed by semistructured interviews to assess accessibility, usability and to validate the surveys.Validated surveys will be distributed by other NHS DESP sites, and also via patient groups on social media, relevant charities and the British Association of Retinal Screeners. Post-survey evaluative interviews will be undertaken among those who consent to participate in further research. ETHICS AND DISSEMINATION: Ethical approval has been obtained by the NHS Research Ethics Committee (IRAS ID: 316631). Survey results will be shared and discussed with focus groups to facilitate preparation of findings for publication and to inform codesign of outreach activities to address concerns and perceptions identified.

Ethnic disparities in progression rates for sight-threatening diabetic retinopathy in diabetic eye screening: a population-based retrospective cohort study.

INTRODUCTION: The English Diabetic Eye Screening Programme (DESP) offers people living with diabetes (PLD) annual eye screening. We examined incidence and determinants of sight-threatening diabetic retinopathy (STDR) in a sociodemographically diverse multi-ethnic population. RESEARCH DESIGN AND METHODS: North East London DESP cohort data (January 2012 to December 2021) with 137 591 PLD with no retinopathy, or non-STDR at baseline in one/both eyes, were used to calculate STDR incidence rates by sociodemographic factors, diabetes type, and duration. HR from Cox models examined associations with STDR. RESULTS: There were 16 388 incident STDR cases over a median of 5.4 years (IQR 2.8-8.2; STDR rate 2.214, 95% CI 2.214 to 2.215 per 100 person-years). People with no retinopathy at baseline had a lower risk of sight-threatening diabetic retinopathy (STDR) compared with those with non-STDR in one eye (HR 3.03, 95% CI 2.91 to 3.15, p<0.001) and both eyes (HR 7.88, 95% CI 7.59 to 8.18, p<0.001). Black and South Asian individuals had higher STDR hazards than white individuals (HR 1.57, 95% CI 1.50 to 1.64 and HR 1.36, 95% CI 1.31 to 1.42, respectively). Additionally, every 5-year increase in age at inclusion was associated with an 8% reduction in STDR hazards (p<0.001). CONCLUSIONS: Ethnic disparities exist in a health system limited by capacity rather than patient economic circumstances. Diabetic retinopathy at first screen is a strong determinant of STDR development. By using basic demographic characteristics, screening programmes or clinical practices can stratify risk for sight-threatening diabetic retinopathy development.

Two-year recall for people with no diabetic retinopathy: a multi-ethnic population-based retrospective cohort study using real-world data to quantify the effect.

BACKGROUND/AIMS: The English Diabetic Eye Screening Programme (DESP) offers people living with diabetes (PLD) annual screening. Less frequent screening has been advocated among PLD without diabetic retinopathy (DR), but evidence for each ethnic group is limited. We examined the potential effect of biennial versus annual screening on the detection of sight-threatening diabetic retinopathy (STDR) and proliferative diabetic retinopathy (PDR) among PLD without DR from a large urban multi-ethnic English DESP. METHODS: PLD in North-East London DESP (January 2012 to December 2021) with no DR on two prior consecutive screening visits with up to 8 years of follow-up were examined. Annual STDR and PDR incidence rates, overall and by ethnicity, were quantified. Delays in identification of STDR and PDR events had 2-year screening intervals been used were determined. FINDINGS: Among 82 782 PLD (37% white, 36% South Asian, and 16% black people), there were 1788 incident STDR cases over mean (SD) 4.3 (2.4) years (STDR rate 0.51, 95% CI 0.47 to 0.55 per 100-person-years). STDR incidence rates per 100-person-years by ethnicity were 0.55 (95% CI 0.48 to 0.62) for South Asian, 0.34 (95% CI 0.29 to 0.40) for white, and 0.77 (95% CI 0.65 to 0.90) for black people. Biennial screening would have delayed diagnosis by 1 year for 56.3% (1007/1788) with STDR and 43.6% (45/103) with PDR. Standardised cumulative rates of delayed STDR per 100 000 persons for each ethnic group were 1904 (95% CI 1683 to 2154) for black people, 1276 (95% CI 1153 to 1412) for South Asian people, and 844 (95% CI 745 to 955) for white people. INTERPRETATION: Biennial screening would have delayed detection of some STDR and PDR by 1 year, especially among those of black ethnic origin, leading to healthcare inequalities.

Effect of ethnicity and other sociodemographic factors on attendance at diabetic eye screening: a 12-month retrospective cohort study.

OBJECTIVES: To examine the association of sociodemographic characteristics with attendance at diabetic eye screening in a large ethnically diverse urban population. DESIGN: Retrospective cohort study. SETTING: Screening visits in the North East London Diabetic Eye Screening Programme (NELDESP). PARTICIPANTS: 84 449 people with diabetes aged 12 years or older registered in the NELDESP and scheduled for screening between 1 April 2017 and 31 March 2018. MAIN OUTCOME MEASURE: Attendance at diabetic eye screening appointments. RESULTS: The mean age of people with diabetes was 60 years (SD 14.2 years), 53.4% were men, 41% South Asian, 29% White British and 17% Black; 83.4% attended screening. Black people with diabetes had similar levels of attendance compared with White British people. However, South Asian, Chinese and 'Any other Asian' background ethnicities showed greater odds of attendance compared with White British. When compared with their respective reference group, high levels of deprivation, younger age, longer duration of diabetes and worse visual acuity, were all associated with non-attendance. There was a higher likelihood of attendance per quintile improvement in deprivation (OR, 1.06; 95% CI, 1.03 to 1.08), with increasing age (OR per decade, 1.17; 95% CI, 1.15 to 1.19), with better visual acuity (OR per Bailey-Lovie chart line 1.12; 95% CI, 1.11 to 1.14) and with longer time of NELDESP registration (OR per year, 1.02; 95% CI, 1.01 to 1.03). CONCLUSION: Ethnic differences in diabetic eye screening uptake, though small, are evident. Despite preconceptions, a higher likelihood of screening attendance was observed among Asian ethnic groups when compared with the White ethnic group. Poorer socioeconomic profile was associated with higher likelihood of non-attendance for screening. Further work is needed to understand how to target individuals at risk of non-attendance and reduce inequalities.

Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients.

BACKGROUND/AIMS: Human grading of digital images from diabetic retinopathy (DR) screening programmes represents a significant challenge, due to the increasing prevalence of diabetes. We evaluate the performance of an automated artificial intelligence (AI) algorithm to triage retinal images from the English Diabetic Eye Screening Programme (DESP) into test-positive/technical failure versus test-negative, using human grading following a standard national protocol as the reference standard. METHODS: Retinal images from 30 405 consecutive screening episodes from three English DESPs were manually graded following a standard national protocol and by an automated process with machine learning enabled software, EyeArt v2.1. Screening performance (sensitivity, specificity) and diagnostic accuracy (95% CIs) were determined using human grades as the reference standard. RESULTS: Sensitivity (95% CIs) of EyeArt was 95.7% (94.8% to 96.5%) for referable retinopathy (human graded ungradable, referable maculopathy, moderate-to-severe non-proliferative or proliferative). This comprises sensitivities of 98.3% (97.3% to 98.9%) for mild-to-moderate non-proliferative retinopathy with referable maculopathy, 100% (98.7%,100%) for moderate-to-severe non-proliferative retinopathy and 100% (97.9%,100%) for proliferative disease. EyeArt agreed with the human grade of no retinopathy (specificity) in 68% (67% to 69%), with a specificity of 54.0% (53.4% to 54.5%) when combined with non-referable retinopathy. CONCLUSION: The algorithm demonstrated safe levels of sensitivity for high-risk retinopathy in a real-world screening service, with specificity that could halve the workload for human graders. AI machine learning and deep learning algorithms such as this can provide clinically equivalent, rapid detection of retinopathy, particularly in settings where a trained workforce is unavailable or where large-scale and rapid results are needed.

Diagnostic accuracy of diabetic retinopathy grading by an artificial intelligence-enabled algorithm compared with a human standard for wide-field true-colour confocal scanning and standard digital retinal images.

BACKGROUND: Photographic diabetic retinopathy screening requires labour-intensive grading of retinal images by humans. Automated retinal image analysis software (ARIAS) could provide an alternative to human grading. We compare the performance of an ARIAS using true-colour, wide-field confocal scanning images and standard fundus images in the English National Diabetic Eye Screening Programme (NDESP) against human grading. METHODS: Cross-sectional study with consecutive recruitment of patients attending annual diabetic eye screening. Imaging with mydriasis was performed (two-field protocol) with the EIDON platform (CenterVue, Padua, Italy) and standard NDESP cameras. Human grading was carried out according to NDESP protocol. Images were processed by EyeArt V.2.1.0 (Eyenuk Inc, Woodland Hills, California). The reference standard for analysis was the human grade of standard NDESP images. RESULTS: We included 1257 patients. Sensitivity estimates for retinopathy grades were: EIDON images; 92.27% (95% CI: 88.43% to 94.69%) for any retinopathy, 99% (95% CI: 95.35% to 100%) for vision-threatening retinopathy and 100% (95% CI: 61% to 100%) for proliferative retinopathy. For NDESP images: 92.26% (95% CI: 88.37% to 94.69%) for any retinopathy, 100% (95% CI: 99.53% to 100%) for vision-threatening retinopathy and 100% (95% CI: 61% to 100%) for proliferative retinopathy. One case of vision-threatening retinopathy (R1M1) was missed by the EyeArt when analysing the EIDON images, but identified by the human graders. The EyeArt identified all cases of vision-threatening retinopathy in the standard images. CONCLUSION: EyeArt identified diabetic retinopathy in EIDON images with similar sensitivity to standard images in a large-scale screening programme, exceeding the sensitivity threshold recommended for a screening test. Further work to optimise the identification of 'no retinopathy' and to understand the differential lesion detection in the two imaging systems would enhance the use of these two innovative technologies in a diabetic retinopathy screening setting.

Comparison of true-colour wide-field confocal scanner imaging with standard fundus photography for diabetic retinopathy screening.

BACKGROUND: Screening of diabetic retinopathy (DR) reduces blindness by early identification of retinopathy. This study compares DR grades derived from a two-field imaging protocol from two imaging platforms, one providing a single 60-degree horizontal field of view (FOV) and the other, a standard 45-degree FOV. METHODS: Cross-sectional study which included 1257 diabetic patients aged ≥18 years attending their DR screening visit in the English National Diabetic Eye Screening Programme (NDESP). Patients with maculopathy (M1), preproliferative (R2) or proliferative DR (R3) were referred to an ophthalmologist. Patients with ungradable images (U) are examined in a slit-lamp biomicroscopy clinic. Image acquisition under mydriasis of two images per eye was carried out with the EIDON and with standard fundus cameras. Evaluation was performed by masked graders. RESULTS: Agreement after consensus with kappa statistic was 0.89 (quadratic weights (95% CI 0.87 to 0.92)) for NDESP severity grade, 0.88 (quadratic weights (95% CI 0.82 to 0.94)) for referable disease and 0.92 (linear weights (95% CI 0.88 to 0.95)) for maculopathy. The EIDON detected clinically relevant DR features outside the 45-degree fields in two patients (0.16%): one with intraretinal microvascular abnormalities (IRMAs) and one with neovascularisation. In eight patients (0.64%), the EIDON allowed DR feature visualisation inside the 45-degree fields that were not identified in the NDESP images: three patients (0.24%) with IRMA and five patients (0.40%) with maculopathy. The rates of ungradable encounters were 12 (0.95%) and 13 (1.03%) with the EIDON and NDESP images, respectively. CONCLUSION: The EIDON identifies a small number of additional patients with referable disease which are not detected with standard imaging. This is due to the EIDON finding disease outside the standard FOV and greater clarity finding disease within the standard FOV.

Automated Diabetic Retinopathy Image Assessment Software: Diagnostic Accuracy and Cost-Effectiveness Compared with Human Graders.

OBJECTIVE: With the increasing prevalence of diabetes, annual screening for diabetic retinopathy (DR) by expert human grading of retinal images is challenging. Automated DR image assessment systems (ARIAS) may provide clinically effective and cost-effective detection of retinopathy. We aimed to determine whether ARIAS can be safely introduced into DR screening pathways to replace human graders. DESIGN: Observational measurement comparison study of human graders following a national screening program for DR versus ARIAS. PARTICIPANTS: Retinal images from 20 258 consecutive patients attending routine annual diabetic eye screening between June 1, 2012, and November 4, 2013. METHODS: Retinal images were manually graded following a standard national protocol for DR screening and were processed by 3 ARIAS: iGradingM, Retmarker, and EyeArt. Discrepancies between manual grades and ARIAS results were sent to a reading center for arbitration. MAIN OUTCOME MEASURES: Screening performance (sensitivity, false-positive rate) and diagnostic accuracy (95% confidence intervals of screening-performance measures) were determined. Economic analysis estimated the cost per appropriate screening outcome. RESULTS: Sensitivity point estimates (95% confidence intervals) of the ARIAS were as follows: EyeArt 94.7% (94.2%-95.2%) for any retinopathy, 93.8% (92.9%-94.6%) for referable retinopathy (human graded as either ungradable, maculopathy, preproliferative, or proliferative), 99.6% (97.0%-99.9%) for proliferative retinopathy; Retmarker 73.0% (72.0 %-74.0%) for any retinopathy, 85.0% (83.6%-86.2%) for referable retinopathy, 97.9% (94.9%-99.1%) for proliferative retinopathy. iGradingM classified all images as either having disease or being ungradable. EyeArt and Retmarker saved costs compared with manual grading both as a replacement for initial human grading and as a filter prior to primary human grading, although the latter approach was less cost-effective. CONCLUSIONS: Retmarker and EyeArt systems achieved acceptable sensitivity for referable retinopathy when compared with that of human graders and had sufficient specificity to make them cost-effective alternatives to manual grading alone. ARIAS have the potential to reduce costs in developed-world health care economies and to aid delivery of DR screening in developing or remote health care settings.

An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness.

BACKGROUND: Diabetic retinopathy screening in England involves labour-intensive manual grading of retinal images. Automated retinal image analysis systems (ARIASs) may offer an alternative to manual grading. OBJECTIVES: To determine the screening performance and cost-effectiveness of ARIASs to replace level 1 human graders or pre-screen with ARIASs in the NHS diabetic eye screening programme (DESP). To examine technical issues associated with implementation. DESIGN: Observational retrospective measurement comparison study with a real-time evaluation of technical issues and a decision-analytic model to evaluate cost-effectiveness. SETTING: A NHS DESP. PARTICIPANTS: Consecutive diabetic patients who attended a routine annual NHS DESP visit. INTERVENTIONS: Retinal images were manually graded and processed by three ARIASs: iGradingM (version 1.1; originally Medalytix Group Ltd, Manchester, UK, but purchased by Digital Healthcare, Cambridge, UK, at the initiation of the study, purchased in turn by EMIS Health, Leeds, UK, after conclusion of the study), Retmarker (version 0.8.2, Retmarker Ltd, Coimbra, Portugal) and EyeArt (Eyenuk Inc., Woodland Hills, CA, USA). The final manual grade was used as the reference standard. Arbitration on a subset of discrepancies between manual grading and the use of an ARIAS by a reading centre masked to all grading was used to create a reference standard manual grade modified by arbitration. MAIN OUTCOME MEASURES: Screening performance (sensitivity, specificity, false-positive rate and likelihood ratios) and diagnostic accuracy [95% confidence intervals (CIs)] of ARIASs. A secondary analysis explored the influence of camera type and patients' ethnicity, age and sex on screening performance. Economic analysis estimated the cost per appropriate screening outcome identified. RESULTS: A total of 20,258 patients with 102,856 images were entered into the study. The sensitivity point estimates of the ARIASs were as follows: EyeArt 94.7% (95% CI 94.2% to 95.2%) for any retinopathy, 93.8% (95% CI 92.9% to 94.6%) for referable retinopathy and 99.6% (95% CI 97.0% to 99.9%) for proliferative retinopathy; and Retmarker 73.0% (95% CI 72.0% to 74.0%) for any retinopathy, 85.0% (95% CI 83.6% to 86.2%) for referable retinopathy and 97.9% (95% CI 94.9 to 99.1%) for proliferative retinopathy. iGradingM classified all images as either 'disease' or 'ungradable', limiting further iGradingM analysis. The sensitivity and false-positive rates for EyeArt were not affected by ethnicity, sex or camera type but sensitivity declined marginally with increasing patient age. The screening performance of Retmarker appeared to vary with patient's age, ethnicity and camera type. Both EyeArt and Retmarker were cost saving relative to manual grading either as a replacement for level 1 human grading or used prior to level 1 human grading, although the latter was less cost-effective. A threshold analysis testing the highest ARIAS cost per patient before which ARIASs became more expensive per appropriate outcome than human grading, when used to replace level 1 grader, was Retmarker £3.82 and EyeArt £2.71 per patient. LIMITATIONS: The non-randomised study design limited the health economic analysis but the same retinal images were processed by all ARIASs in this measurement comparison study. CONCLUSIONS: Retmarker and EyeArt achieved acceptable sensitivity for referable retinopathy and false-positive rates (compared with human graders as reference standard) and appear to be cost-effective alternatives to a purely manual grading approach. Future work is required to develop technical specifications to optimise deployment and address potential governance issues. FUNDING: The National Institute for Health Research (NIHR) Health Technology Assessment programme, a Fight for Sight Grant (Hirsch grant award) and the Department of Health's NIHR Biomedical Research Centre for Ophthalmology at Moorfields Eye Hospital and the University College London Institute of Ophthalmology.

A study of whether automated Diabetic Retinopathy Image Assessment could replace manual grading steps in the English National Screening Programme.

OBJECTIVES: Diabetic retinopathy screening in England involves labour intensive manual grading of digital retinal images. We present the plan for an observational retrospective study of whether automated systems could replace one or more steps of human grading. METHODS: Patients aged 12 or older who attended the Diabetes Eye Screening programme, Homerton University Hospital (London) between 1 June 2012 and 4 November 2013 had macular and disc-centred retinal images taken. All screening episodes were manually graded and will additionally be graded by three automated systems. Each system will process all screening episodes, and screening performance (sensitivity, false positive rate, likelihood ratios) and diagnostic accuracy (95% confidence intervals of screening performance measures) will be quantified. A sub-set of gradings will be validated by an approved Reading Centre. Additional analyses will explore the effect of altering thresholds for disease detection within each automated system on screening performance. RESULTS: 2,782/20,258 diabetes patients were referred to ophthalmologists for further examination. Prevalence of maculopathy (M1), pre-proliferative retinopathy (R2), and proliferative retinopathy (R3) were 7.9%, 3.1% and 1.2%, respectively; 4749 (23%) patients were diagnosed with background retinopathy (R1); 1.5% were considered ungradable by human graders. CONCLUSIONS: Retinopathy prevalence was similar to other English diabetic screening programmes, so findings should be generalizable. The study population size will allow the detection of differences in screening performance between the human and automated grading systems as small as 2%. The project will compare performance and economic costs of manual versus automated systems.