Clinician-Driven AI: Code-Free Self-Training on Public Data for Diabetic Retinopathy Referral.

Importance: Democratizing artificial intelligence (AI) enables model development by clinicians with a lack of coding expertise, powerful computing resources, and large, well-labeled data sets. Objective: To determine whether resource-constrained clinicians can use self-training via automated machine learning (ML) and public data sets to design high-performing diabetic retinopathy classification models. Design, Setting, and Participants: This diagnostic quality improvement study was conducted from January 1, 2021, to December 31, 2021. A self-training method without coding was used on 2 public data sets with retinal images from patients in France (Messidor-2 [n = 1748]) and the UK and US (EyePACS [n = 58 689]) and externally validated on 1 data set with retinal images from patients of a private Egyptian medical retina clinic (Egypt [n = 210]). An AI model was trained to classify referable diabetic retinopathy as an exemplar use case. Messidor-2 images were assigned adjudicated labels available on Kaggle; 4 images were deemed ungradable and excluded, leaving 1744 images. A total of 300 images randomly selected from the EyePACS data set were independently relabeled by 3 blinded retina specialists using the International Classification of Diabetic Retinopathy protocol for diabetic retinopathy grade and diabetic macular edema presence; 19 images were deemed ungradable, leaving 281 images. Data analysis was performed from February 1 to February 28, 2021. Exposures: Using public data sets, a teacher model was trained with labeled images using supervised learning. Next, the resulting predictions, termed pseudolabels, were used on an unlabeled public data set. Finally, a student model was trained with the existing labeled images and the additional pseudolabeled images. Main Outcomes and Measures: The analyzed metrics for the models included the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, and F1 score. The Fisher exact test was performed, and 2-tailed P values were calculated for failure case analysis. Results: For the internal validation data sets, AUROC values for performance ranged from 0.886 to 0.939 for the teacher model and from 0.916 to 0.951 for the student model. For external validation of automated ML model performance, AUROC values and accuracy were 0.964 and 93.3% for the teacher model, 0.950 and 96.7% for the student model, and 0.890 and 94.3% for the manually coded bespoke model, respectively. Conclusions and Relevance: These findings suggest that self-training using automated ML is an effective method to increase both model performance and generalizability while decreasing the need for costly expert labeling. This approach advances the democratization of AI by enabling clinicians without coding expertise or access to large, well-labeled private data sets to develop their own AI models.

GliaMorph: a modular image analysis toolkit to quantify Müller glial cell morphology.

Cell morphology is crucial for all cell functions. This is particularly true for glial cells as they rely on complex shape to contact and support neurons. However, methods to quantify complex glial cell shape accurately and reproducibly are lacking. To address this, we developed the image analysis pipeline 'GliaMorph'. GliaMorph is a modular analysis toolkit developed to perform (1) image pre-processing, (2) semi-automatic region-of-interest selection, (3) apicobasal texture analysis, (4) glia segmentation, and (5) cell feature quantification. Müller glia (MG) have a stereotypic shape linked to their maturation and physiological status. Here, we characterized MG on three levels: (1) global image-level, (2) apicobasal texture, and (3) regional apicobasal vertical-to-horizontal alignment. Using GliaMorph, we quantified MG development on a global and single-cell level, showing increased feature elaboration and subcellular morphological rearrangement in the zebrafish retina. As proof of principle, we analysed expression changes in a mouse glaucoma model, identifying subcellular protein localization changes in MG. Together, these data demonstrate that GliaMorph enables an in-depth understanding of MG morphology in the developing and diseased retina.

Evaluating an automated machine learning model that predicts visual acuity outcomes in patients with neovascular age-related macular degeneration.

PURPOSE: Neovascular age-related macular degeneration (nAMD) is a major global cause of blindness. Whilst anti-vascular endothelial growth factor (anti-VEGF) treatment is effective, response varies considerably between individuals. Thus, patients face substantial uncertainty regarding their future ability to perform daily tasks. In this study, we evaluate the performance of an automated machine learning (AutoML) model which predicts visual acuity (VA) outcomes in patients receiving treatment for nAMD, in comparison to a manually coded model built using the same dataset. Furthermore, we evaluate model performance across ethnic groups and analyse how the models reach their predictions. METHODS: Binary classification models were trained to predict whether patients' VA would be 'Above' or 'Below' a score of 70 one year after initiating treatment, measured using the Early Treatment Diabetic Retinopathy Study (ETDRS) chart. The AutoML model was built using the Google Cloud Platform, whilst the bespoke model was trained using an XGBoost framework. Models were compared and analysed using the What-if Tool (WIT), a novel model-agnostic interpretability tool. RESULTS: Our study included 1631 eyes from patients attending Moorfields Eye Hospital. The AutoML model (area under the curve [AUC], 0.849) achieved a highly similar performance to the XGBoost model (AUC, 0.847). Using the WIT, we found that the models over-predicted negative outcomes in Asian patients and performed worse in those with an ethnic category of Other. Baseline VA, age and ethnicity were the most important determinants of model predictions. Partial dependence plot analysis revealed a sigmoidal relationship between baseline VA and the probability of an outcome of 'Above'. CONCLUSION: We have described and validated an AutoML-WIT pipeline which enables clinicians with minimal coding skills to match the performance of a state-of-the-art algorithm and obtain explainable predictions.

Applications of interpretability in deep learning models for ophthalmology.

PURPOSE OF REVIEW: In this article, we introduce the concept of model interpretability, review its applications in deep learning models for clinical ophthalmology, and discuss its role in the integration of artificial intelligence in healthcare. RECENT FINDINGS: The advent of deep learning in medicine has introduced models with remarkable accuracy. However, the inherent complexity of these models undermines its users' ability to understand, debug and ultimately trust them in clinical practice. Novel methods are being increasingly explored to improve models' 'interpretability' and draw clearer associations between their outputs and features in the input dataset. In the field of ophthalmology, interpretability methods have enabled users to make informed adjustments, identify clinically relevant imaging patterns, and predict outcomes in deep learning models. SUMMARY: Interpretability methods support the transparency necessary to implement, operate and modify complex deep learning models. These benefits are becoming increasingly demonstrated in models for clinical ophthalmology. As quality standards for deep learning models used in healthcare continue to evolve, interpretability methods may prove influential in their path to regulatory approval and acceptance in clinical practice.

Medical applicant general practice experience and career aspirations: a questionnaire study.

BACKGROUND: Increasing access to general practice work experience placements for school students is a strategy for improving general practice recruitment, despite limited evidence and concerns surrounding equity of access to general practice experiences. AIMS: To examine the association between undertaking general practice experience and the perceptions of general practice as an appealing future career among prospective medical applicants. To identify socioeconomic factors associated with obtaining general practice experience. DESIGN & SETTING: Cross-sectional questionnaire study in the UK. METHOD: Participants were UK residents aged ≥16 years and seriously considering applying to study medicine in 2019/2020. They were invited to take part via the University Clinical Aptitude Test (UCAT). Questionnaire data were analysed using a linear regression of general practice appeal on general practice experience, adjusting for career motivations and demographics, and a logistic regression of general practice experience on measures of social capital and demographics. RESULTS: Of 6391 responders, 4031 were in their last year of school. General practice experience predicted general practice appeal after adjusting for career motivation and demographics (b = 0.37, standard error [SE] = 0.06, P<0.00001). General practice experience was more common among students at private (odds ratio [OR] = 1.65, 95% confidence interval [CI] = 1.31 to 2.08, P<0.0001) or grammar schools (OR = 1.33, 95% CI = 1.02 to 1.72, P = 0.03) and in the highest socioeconomic group (OR = 1.62, 95% CI = 1.28 to 2.05, P<0.0001), and less likely among students of 'other' ethnicity (OR = 0.37, 95% CI = 0.20 to 0.67, P = 0.0011). CONCLUSION: Having general practice experience prior to medical school was associated with finding general practice appealing, which supports its utility in recruitment. Applicants from more deprived backgrounds were less likely to have had a general practice experience, possibly through lack of accessible opportunities.