Reasoning with large language models for medical question answering.

OBJECTIVES: To investigate approaches of reasoning with large language models (LLMs) and to propose a new prompting approach, ensemble reasoning, to improve medical question answering performance with refined reasoning and reduced inconsistency. MATERIALS AND METHODS: We used multiple choice questions from the USMLE Sample Exam question files on 2 closed-source commercial and 1 open-source clinical LLM to evaluate our proposed approach ensemble reasoning. RESULTS: On GPT-3.5 turbo and Med42-70B, our proposed ensemble reasoning approach outperformed zero-shot chain-of-thought with self-consistency on Steps 1, 2, and 3 questions (+3.44%, +4.00%, and +2.54%) and (2.3%, 5.00%, and 4.15%), respectively. With GPT-4 turbo, there were mixed results with ensemble reasoning again outperforming zero-shot chain-of-thought with self-consistency on Step 1 questions (+1.15%). In all cases, the results demonstrated improved consistency of responses with our approach. A qualitative analysis of the reasoning from the model demonstrated that the ensemble reasoning approach produces correct and helpful reasoning. CONCLUSION: The proposed iterative ensemble reasoning has the potential to improve the performance of LLMs in medical question answering tasks, particularly with the less powerful LLMs like GPT-3.5 turbo and Med42-70B, which may suggest that this is a promising approach for LLMs with lower capabilities. Additionally, the findings show that our approach helps to refine the reasoning generated by the LLM and thereby improve consistency even with the more powerful GPT-4 turbo. We also identify the potential and need for human-artificial intelligence teaming to improve the reasoning beyond the limits of the model.

Disparities in Retinal Vein Occlusion Presentation and Initiation of Anti-VEGF Therapy: An Academy IRIS® Registry Analysis.

OBJECTIVE: Investigate disparities in retinal vein occlusion (RVO) presentation and initiation of anti-VEGF treatment. DESIGN: Retrospective cohort study. SUBJECTS: Patients in the American Academy of Ophthalmology IRIS® (Intelligent Research in Sight) Registry database (2015-2021) with branch or central RVO and macular edema (ME). METHODS: The association of demographic characteristics and presenting visual acuity (VA) with anti-VEGF treatment initiation were quantified using multivariable logistic regression. MAIN OUTCOME MEASURES: Treatment with ≥ 1 anti-VEGF injection within 12 months after RVO diagnosis. RESULTS: A total of 304 558 eligible patients with RVO and ME were identified. Age at presentation varied by race, ethnicity, sex, and RVO type (all P values < 0.001). Within the first year after RVO presentation, 192 602 (63.2%) patients received ≥ 1 anti-VEGF injection. In a multivariable regression model adjusting for relevant covariates, female (vs. male) patients had lower odds of receiving injections (odds ratio [OR], 0.95; 95% confidence interval [CI], 0.93-0.96; P < 0.0001) as did Black/African American (vs. White) patients (OR, 0.90; 95% CI, 0.88-0.92; P < 0.0001) and Asian (vs. White) patients (OR, 0.95; 95% CI, 0.91-0.99; P = 0.02), whereas older patients (vs. patients aged < 51 years) had higher odds (61-70 years: OR, 1.20; 95% CI, 1.16-1.24; 71-80 years: OR, 1.20; 95% CI, 1.16-1.24; > 80 years: OR, 1.15; 95% CI, 1.11-1.18; all P values < 0.0001). Hispanic (vs. non-Hispanic) patients had a small increased odds of treatment initiation (OR, 1.08; 95% CI, 1.04-1.11; P < 0.0001). Results were similar in the subset of 226 143 patients with VA data. In this subset, patients with presenting VA < 20/40 to 20/200 were most frequently treated in the first year after diagnosis (∼ 70%) and patients with light perception/no light perception (LP-NLP) vision or VA of 20/20 or better were treated least frequently (36.9% and 41.9%, respectively). CONCLUSIONS: In this large national clinical registry, 37% of RVO patients with ME had no anti-VEGF treatment documented in the first year after diagnosis. Black/African American, Asian, and female patients and patients with VA of LP-NLP were least likely to receive treatment. Awareness of this undertreatment and these disparities highlight the need for initiatives to ensure all RVO patients receive timely anti-VEGF injections for optimized visual outcomes. FINANCIAL DISCLOSURES: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.