Comparing code-free deep learning models to expert-designed models for detecting retinal diseases from optical coherence tomography.

BACKGROUND: Code-free deep learning (CFDL) is a novel tool in artificial intelligence (AI). This study directly compared the discriminative performance of CFDL models designed by ophthalmologists without coding experience against bespoke models designed by AI experts in detecting retinal pathologies from optical coherence tomography (OCT) videos and fovea-centered images. METHODS: Using the same internal dataset of 1,173 OCT macular videos and fovea-centered images, model development was performed simultaneously but independently by an ophthalmology resident (CFDL models) and a postdoctoral researcher with expertise in AI (bespoke models). We designed a multi-class model to categorize video and fovea-centered images into five labels: normal retina, macular hole, epiretinal membrane, wet age-related macular degeneration and diabetic macular edema. We qualitatively compared point estimates of the performance metrics of the CFDL and bespoke models. RESULTS: For videos, the CFDL model demonstrated excellent discriminative performance, even outperforming the bespoke models for some metrics: area under the precision-recall curve was 0.984 (vs. 0.901), precision and sensitivity were both 94.1% (vs. 94.2%) and accuracy was 94.1% (vs. 96.7%). The fovea-centered CFDL model overall performed better than video-based model and was as accurate as the best bespoke model. CONCLUSION: This comparative study demonstrated that code-free models created by clinicians without coding expertise perform as accurately as expert-designed bespoke models at classifying various retinal pathologies from OCT videos and images. CFDL represents a step forward towards the democratization of AI in medicine, although its numerous limitations must be carefully addressed to ensure its effective application in healthcare.

Automated Machine Learning versus Expert-Designed Models in Ocular Toxoplasmosis: Detection and Lesion Localization Using Fundus Images.

PURPOSE: Automated machine learning (AutoML) allows clinicians without coding experience to build their own deep learning (DL) models. This study assesses the performance of AutoML in detecting and localizing ocular toxoplasmosis (OT) lesions in fundus images and compares it to expert-designed models. METHODS: Ophthalmology trainees without coding experience designed AutoML models using 304 labelled fundus images. We designed a binary model to differentiate OT from normal and an object detection model to visually identify OT lesions. RESULTS: The AutoML model had an area under the precision-recall curve (AuPRC) of 0.945, sensitivity of 100%, specificity of 83% and accuracy of 93.5% (vs. 94%, 86% and 91% for the bespoke models). The AutoML object detection model had an AuPRC of 0.600 with a precision of 93.3% and recall of 56%. Using a diversified external validation dataset, our model correctly labeled 15 normal fundus images (100%) and 15 OT fundus images (100%), with a mean confidence score of 0.965 and 0.963, respectively. CONCLUSION: AutoML models created by ophthalmologists without coding experience were comparable or better than expert-designed bespoke models trained on the same dataset. By creatively using AutoML to identify OT lesions on fundus images, our approach brings the whole spectrum of DL model design into the hands of clinicians.

Assessing the medical reasoning skills of GPT-4 in complex ophthalmology cases.

BACKGROUND/AIMS: This study assesses the proficiency of Generative Pre-trained Transformer (GPT)-4 in answering questions about complex clinical ophthalmology cases. METHODS: We tested GPT-4 on 422 Journal of the American Medical Association Ophthalmology Clinical Challenges, and prompted the model to determine the diagnosis (open-ended question) and identify the next-step (multiple-choice question). We generated responses using two zero-shot prompting strategies, including zero-shot plan-and-solve+ (PS+), to improve the reasoning of the model. We compared the best-performing model to human graders in a benchmarking effort. RESULTS: Using PS+ prompting, GPT-4 achieved mean accuracies of 48.0% (95% CI (43.1% to 52.9%)) and 63.0% (95% CI (58.2% to 67.6%)) in diagnosis and next step, respectively. Next-step accuracy did not significantly differ by subspecialty (p=0.44). However, diagnostic accuracy in pathology and tumours was significantly higher than in uveitis (p=0.027). When the diagnosis was accurate, 75.2% (95% CI (68.6% to 80.9%)) of the next steps were correct. Conversely, when the diagnosis was incorrect, 50.2% (95% CI (43.8% to 56.6%)) of the next steps were accurate. The next step was three times more likely to be accurate when the initial diagnosis was correct (p<0.001). No significant differences were observed in diagnostic accuracy and decision-making between board-certified ophthalmologists and GPT-4. Among trainees, senior residents outperformed GPT-4 in diagnostic accuracy (p≤0.001 and 0.049) and in accuracy of next step (p=0.002 and 0.020). CONCLUSION: Improved prompting enhances GPT-4's performance in complex clinical situations, although it does not surpass ophthalmology trainees in our context. Specialised large language models hold promise for future assistance in medical decision-making and diagnosis.

Capabilities of GPT-4 in ophthalmology: an analysis of model entropy and progress towards human-level medical question answering.

BACKGROUND: Evidence on the performance of Generative Pre-trained Transformer 4 (GPT-4), a large language model (LLM), in the ophthalmology question-answering domain is needed. METHODS: We tested GPT-4 on two 260-question multiple choice question sets from the Basic and Clinical Science Course (BCSC) Self-Assessment Program and the OphthoQuestions question banks. We compared the accuracy of GPT-4 models with varying temperatures (creativity setting) and evaluated their responses in a subset of questions. We also compared the best-performing GPT-4 model to GPT-3.5 and to historical human performance. RESULTS: GPT-4-0.3 (GPT-4 with a temperature of 0.3) achieved the highest accuracy among GPT-4 models, with 75.8% on the BCSC set and 70.0% on the OphthoQuestions set. The combined accuracy was 72.9%, which represents an 18.3% raw improvement in accuracy compared with GPT-3.5 (p<0.001). Human graders preferred responses from models with a temperature higher than 0 (more creative). Exam section, question difficulty and cognitive level were all predictive of GPT-4-0.3 answer accuracy. GPT-4-0.3's performance was numerically superior to human performance on the BCSC (75.8% vs 73.3%) and OphthoQuestions (70.0% vs 63.0%), but the difference was not statistically significant (p=0.55 and p=0.09). CONCLUSION: GPT-4, an LLM trained on non-ophthalmology-specific data, performs significantly better than its predecessor on simulated ophthalmology board-style exams. Remarkably, its performance tended to be superior to historical human performance, but that difference was not statistically significant in our study.

Natural History of Untreated Optic Neuritis Associated With Mild COVID-19 Infection.

This article describes a case of untreated optic neuritis occurring in the setting of coronavirus disease 2019 (COVID-19) infection and provides new insights into the natural history of this condition. A 29-year-old male patient with no known ocular or systemic disease presented with pain on extraocular movements and sudden loss of vision in the inferior visual field affecting the right eye. He had tested positive for COVID-19 six days prior after experiencing mild upper respiratory symptoms. On examination, visual acuity was 20/20, and color vision was normal. A relative afferent pupillary defect was observed in the right eye. Fundoscopy revealed mild optic disc edema in the same eye. Optical coherence tomography showed increased retinal nerve fiber layer thickness of the right optic nerve head and visual field testing revealed an inferonasal defect. Extensive laboratory and imaging investigations failed to reveal an underlying etiology, supporting a diagnosis of COVID-19-associated optic neuritis. The patient improved spontaneously without treatment. At the five-month follow-up, minor optic atrophy and a small residual visual field defect remained.

Evaluating the Performance of ChatGPT in Ophthalmology: An Analysis of Its Successes and Shortcomings.

Purpose: Foundation models are a novel type of artificial intelligence algorithms, in which models are pretrained at scale on unannotated data and fine-tuned for a myriad of downstream tasks, such as generating text. This study assessed the accuracy of ChatGPT, a large language model (LLM), in the ophthalmology question-answering space. Design: Evaluation of diagnostic test or technology. Participants: ChatGPT is a publicly available LLM. Methods: We tested 2 versions of ChatGPT (January 9 "legacy" and ChatGPT Plus) on 2 popular multiple choice question banks commonly used to prepare for the high-stakes Ophthalmic Knowledge Assessment Program (OKAP) examination. We generated two 260-question simulated exams from the Basic and Clinical Science Course (BCSC) Self-Assessment Program and the OphthoQuestions online question bank. We carried out logistic regression to determine the effect of the examination section, cognitive level, and difficulty index on answer accuracy. We also performed a post hoc analysis using Tukey's test to decide if there were meaningful differences between the tested subspecialties. Main Outcome Measures: We reported the accuracy of ChatGPT for each examination section in percentage correct by comparing ChatGPT's outputs with the answer key provided by the question banks. We presented logistic regression results with a likelihood ratio (LR) chi-square. We considered differences between examination sections statistically significant at a P value of < 0.05. Results: The legacy model achieved 55.8% accuracy on the BCSC set and 42.7% on the OphthoQuestions set. With ChatGPT Plus, accuracy increased to 59.4% ± 0.6% and 49.2% ± 1.0%, respectively. Accuracy improved with easier questions when controlling for the examination section and cognitive level. Logistic regression analysis of the legacy model showed that the examination section (LR, 27.57; P = 0.006) followed by question difficulty (LR, 24.05; P < 0.001) were most predictive of ChatGPT's answer accuracy. Although the legacy model performed best in general medicine and worst in neuro-ophthalmology (P < 0.001) and ocular pathology (P = 0.029), similar post hoc findings were not seen with ChatGPT Plus, suggesting more consistent results across examination sections. Conclusion: ChatGPT has encouraging performance on a simulated OKAP examination. Specializing LLMs through domain-specific pretraining may be necessary to improve their performance in ophthalmic subspecialties. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Efficacy of ab-interno gelatin microstent implantation in primary and refractory glaucoma.

OBJECTIVE: To evaluate the outcomes of ab interno gelatin microstent implantation alone and in combination with phacoemulsification for the reduction of intraocular pressure (IOP). DESIGN: Retrospective cohort study. PARTICIPANTS: 141 eyes of 141 patients with any glaucoma subtype, including refractory glaucoma, operated in the Centre Hospitalier de l'Université de Montréal (CHUM) from 2015-2018. Patients were included if they were over 40 years of age and had a preoperative IOP of >18 mm Hg on maximum tolerated medical therapy. METHODS: All patients received ab-interno microstent implantation (XEN-45, Allergan, Madison, NJ) with mitomycin C +/- combined phacoemulsification. The primary outcome was complete surgical success (IOP 6-18 mm Hg and <20% reduction from baseline without IOP medications or reoperations or cyclophotocoagulation); secondary outcomes included qualified success allowing for medications, percentage reduction in mean IOP and medications, and reduction in number of complications, interventions, and reoperations. RESULTS: Mean follow-up was 30.5 ± 10.2 months (±SD). Mean IOP was 23.3 ± 7.0 mm Hg on 3.4 ± 0.8 medications at baseline and 13.3 ± 4.7 mm Hg on 1.9 ± 1.5 medications at 24 months of follow-up (p < 0.001). From 24-month survival analysis estimates, complete success was achieved in 34.1% of microstent eyes versus 20.7% with combined phacoemulsification (p = 0.02); 79.1% versus 75.1% achieved qualified success, respectively (p = 0.86). Cases with combined phacoemulsification had a higher rate of failure (hazard ratio [HR] = 1.6, 95% CI 1.1-2.3, p = 0.02). Needling with mitomycin-C or 5-fluorouracil postoperatively occurred in 54 eyes (38.3%). Complications included transient hypotony (10.6%), transient hyphema (6.4%), macular edema (4.3%), and microstent exposure (2.8%). There were 33 eyes (23.4%) with reoperations and 14 (9.9%) requiring subsequent cyclophotocoagulation lasers. CONCLUSIONS: Microstent implantation required topical therapy in most cases 24 months following surgery in primary and refractory glaucoma and, when combined with phacoemulsification, had a higher risk of failure.

Indications and pathologic diagnoses of diagnostic chorioretinal biopsies in the province of Quebec, Canada.

OBJECTIVE: To review the clinical usefulness of chorioretinal biopsies in diagnostically undefined cases of intraocular inflammation or chorioretinal lesions. DESIGN: Retrospective case series. PARTICIPANTS: Seven patients who underwent chorioretinal biopsies. METHODS: This case series included all consecutive patients who underwent chorioretinal biopsies in 2 academic tertiary care centres in the province of Quebec between 2014 and 2020. RESULTS: A total of 7 patients were included in the study. Five patients with intraocular inflammation underwent chorioretinal biopsies to rule out an infectious or neoplastic etiology, whereas 2 patients underwent biopsies for suspicion of neoplastic chorioretinal masses. Final diagnoses included primary chorioretinal lymphoma (n = 2), toxoplasmosis (n = 1), benign choroidal mass (n = 1), nonnecrotizing granuloma (n = 1), and peripheral exudative hemorrhagic chorioretinopathy (n = 1). No specific diagnosis was defined in 1 case of panuveitis with scleritis. No postoperative complications were reported. CONCLUSIONS: Chorioretinal biopsies clarified the diagnosis in 6 of 7 patients, including a definitive diagnosis of lymphoma in 2 patients. This is a high rate of diagnosis that also represents clinically meaningful results that influence management. Future directions include identifying patients in whom adjuvant chorioretinal biopsy would yield a high rate of diagnosis.

Risk factors for epiretinal membrane formation and peeling following pars plana vitrectomy for primary rhegmatogenous retinal detachment, an OCT guided analysis.

BACKGROUND: To evaluate the rate and risk factors of epiretinal membrane (ERM) formation and need for ERM peeling after pars plana vitrectomy (PPV) for uncomplicated primary rhegmatogenous retinal detachment (RRD). METHODS: Retrospective, single-center, cohort study of 119 consecutive patients (119 eyes) that underwent RRD repair using PPV. The primary outcomes were ERM formation, classified using an optical coherence tomography grading system, and the rate of ERM peeling. Visual acuity, postoperative complications, and risk factors for ERM formation and peeling were also identified. RESULTS: Postoperative ERM formation occurred in 69 eyes (58.0%); 56 (47.1%) were stage 1, 9 (7.6%) stage 2, 3 (2.5%) stage 3, and 1 (0.8%) stage 4. Only 6 (5.0%) eyes required secondary PPV for a visually significant ERM, with a mean time to reoperation of 488 ± 351 days. Risk factors for ERM formation included intraoperative cryotherapy, more than 1000 laser shots, 360° laser photocoagulation, and choroidal detachment (p < 0.01). Eyes with more than 3 tears had a trend towards increased ERM surgery (p = 0.10). CONCLUSIONS: Visually significant ERM formation following PPV for primary RRD was uncommon in this cohort (5%). Half of the ERMs were detected after the first post-operative year, indicating that this complication may be underreported in studies with only 1-year follow-up.

Development of a code-free machine learning model for the classification of cataract surgery phases.

This study assessed the performance of automated machine learning (AutoML) in classifying cataract surgery phases from surgical videos. Two ophthalmology trainees without coding experience designed a deep learning model in Google Cloud AutoML Video Classification for the classification of 10 different cataract surgery phases. We used two open-access publicly available datasets (total of 122 surgeries) for model training, validation and testing. External validation was performed on 10 surgeries issued from another dataset. The AutoML model demonstrated excellent discriminating performance, even outperforming bespoke deep learning models handcrafter by experts. The area under the precision-recall curve was 0.855. At the 0.5 confidence threshold cut-off, the overall performance metrics were as follows: sensitivity (81.0%), recall (77.1%), accuracy (96.0%) and F1 score (0.79). The per-segment metrics varied across the surgical phases: precision 66.7-100%, recall 46.2-100% and specificity 94.1-100%. Hydrodissection and phacoemulsification were the most accurately predicted phases (100 and 92.31% correct predictions, respectively). During external validation, the average precision was 54.2% (0.00-90.0%), the recall was 61.1% (0.00-100%) and specificity was 96.2% (91.0-99.0%). In conclusion, a code-free AutoML model can accurately classify cataract surgery phases from videos with an accuracy comparable or better than models developed by experts.

Outcomes and complications of Boston keratoprosthesis type I implantation in unilateral versus bilateral corneal blindness.

OBJECTIVE: To compare the outcomes of Boston keratoprosthesis (KPro) type I implantation between patients who are legally blind versus sighted in the contralateral eye. DESIGN: Single centre retrospective comparative case series. PARTICIPANTS: Patients who underwent Boston KPro type I implantation between 2008 and 2017. METHODS: Single-center retrospective comparative case series. Patients were divided into 2 groups based on the preoperative best-corrected visual acuity (BCVA) in the contralateral eye: group I (>20/200) and group II (20/200). MAIN OUTCOME MEASURES: Postoperative BCVA, device retention, and complications. RESULTS: Group I (56 eyes) and group II (53 eyes) had similar demographics, median preoperative BCVA (hand movements) in the operated eye, and median duration of postoperative follow-up (76.92 vs 85.6 months, respectively). Final postoperative BCVA of the operated eye was statistically better in group I compared with group II (20/400 and hand movements, respectively, p = 0.03). There was no statistical significance in device retention mean survival time. The most common complication in both groups was retroprosthetic membrane. Cystoid macular edema occurred more frequently in group I (p = 0.004), whereas retinal detachment was more common in group II (p = 0.052). CONCLUSIONS: Most patients who underwent Boston KPro type I implantation experienced an improvement in their vision, with final BCVA being superior in the unilateral blind group. Despite similar complication rates and device retention, there are additional socioeconomic factors that need to be considered in sighted individuals. Because the prognosis is tied to the underlying etiology, it is important to recognize that some diagnoses may influence a better outcome.

Endogenous Tuberculous Endophthalmitis and Panophthalmitis: A Systematic Review of Case Reports and Case Series.

PURPOSE: To systematically review the characteristics of patients with endogenous tuberculous (TB) endophthalmitis and panophthalmitis in an effort to help clinicians with diagnosis and treatment. PATIENTS AND METHODS: We conducted a systematic literature search in MEDLINE/PubMed, EMBASE and Web of Science from inception to August 2020. References and abstracts were screened independently by two authors. Included studies were case reports and case series reporting endogenous TB endophthalmitis and panophthalmitis secondary to Mycobacterium tuberculosis complex (MTBC). Available-case analysis was employed to handle missing data. RESULTS: A total of 1343 articles were found using the search strategy. Following abstract screening, 51 articles were selected for full text-review, from which 26 were deemed eligible for inclusion in the study. Forty-four cases from 26 articles were included in the quantitative analysis. The median age of presentation was 29.5 (range: 1 to 81), and 11/44 patients (25.0%) were pediatric. Immunosuppression was seen in 9/36 cases (25.0%). Most patients (24/38, 63.2%) had no prior history of tuberculosis. Systemic symptoms were absent in half of the patients (16/32, 50.0%). Visual acuity was poor, with 23/27 cases (85.2%) being 20/200 or worse at presentation. Poor organ and visual outcomes were reported: 36/43 cases (83.7%) resulted in enucleation/evisceration or exenteration. Intraocular tumors were suspected in 5/34 cases (14.7%). Pulmonary tuberculosis was seen in 15/35 cases (42.8%), and miliary tuberculosis was seen in 7/35 cases (20.0%). The earliest source of TB diagnosis was through histopathologic specimen after eye removal in 32/44 cases (72.7%), vitreous specimen in 6/44 cases (13.6%) and aqueous specimen in 3/44 cases (6.8%). CONCLUSION: TB endophthalmitis is a rare and sight-threatening manifestation of ocular tuberculosis. It can occur in apparently healthy individuals and can mimic intraocular tumors and other infectious etiologies. Diagnosis remains a significant challenge, which, often delayed, leads to profound visual loss. Early detection and treatment of intraocular tuberculosis may be associated with better ocular and systemic outcomes.

Negative afterimages facilitate the detection of real images.

Negative, or complementary afterimages are experienced following brief adaptation to chromatic or achromatic stimuli, and are believed to be formed in the post-receptoral layers of the retinae. Afterimages can be cancelled by the addition of real images, suggesting that afterimages and real images are processed by similar mechanisms. However given their retinal origin, afterimage signals represented at the cortical level might have different spatio-temporal properties from their real images counterparts. To test this we determined whether afterimages reduce the contrast threshold of added real images, i.e. produce the classic "dipper" function characteristic of contrast discrimination, a behavior believed to be cortically mediated. Stimuli were chromatic and achromatic disks on a grey background. Observers adapted for 1.0 s to two side-by-side disks of a particular color. Following stimulus offset, a test disk added to one side was ramped downwards for 1.5 s to approximately match the temporal characteristic of the afterimage, and the observer was required to indicate the side containing the test disk. The test hue/brightness was either the same as that of the afterimage or a different hue/brightness. The independent variable was the contrast of the adaptor. A dipper followed by masking was observed in most conditions in which the afterimage and test colors had the same hue or brightness. We conclude that afterimages are represented similarly to their real image counterparts at the cortical level.