Generative artificial intelligence versus clinicians: Who diagnoses multiple sclerosis faster and with greater accuracy?

BACKGROUND: Those receiving the diagnosis of multiple sclerosis (MS) over the next ten years will predominantly be part of Generation Z (Gen Z). Recent observations within our clinic suggest that younger people with MS utilize online generative artificial intelligence (AI) platforms for personalized medical advice prior to their first visit with a specialist in neuroimmunology. The use of such platforms is anticipated to increase given the technology driven nature, desire for instant communication, and cost-conscious nature of Gen Z. Our objective was to determine if ChatGPT (Generative Pre-trained Transformer) could diagnose MS in individuals earlier than their clinical timeline, and to assess if the accuracy differed based on age, sex, and race/ethnicity. METHODS: People with MS between 18 and 59 years of age were studied. The clinical timeline for people diagnosed with MS was retrospectively identified and simulated using ChatGPT-3.5 (GPT-3.5). Chats were conducted using both actual and derivatives of their age, sex, and race/ethnicity to test diagnostic accuracy. A Kaplan-Meier survival curve was estimated for time to diagnosis, clustered by subject. The p-value testing for differences in time to diagnosis was accomplished using a general Wilcoxon test. Logistic regression (subject-specific intercept) was used to capture intra-subject correlation to test the accuracy prior to and after the inclusion of MRI data. RESULTS: The study cohort included 100 unique people with MS. Of those, 50 were members of Gen Z (38 female; 22 White; mean age at first symptom was 20.6 years (y) (standard deviation (SD)=2.2y)), and 50 were non-Gen Z (34 female; 27 White; mean age at first symptom was 37.0y (SD=10.4y)). In addition, a total of 529 people that represented digital simulations of the original cohort of 100 people (333 female; 166 White; 136 Black/African American; 107 Asian; 120 Hispanic, mean age at first symptom was 31.6y (SD=12.4y)) were generated allowing for 629 scripted conversations to be analyzed. The estimated median time to diagnosis in clinic was significantly longer at 0.35y (95% CI=[0.28, 0.48]) versus that by ChatGPT at 0.08y (95% CI=[0.04, 0.24]) (p<0.0001). There was no difference in the diagnostic accuracy between ages and by race/ethnicity prior to the inclusion of MRI data. However, prior to including the MRI data, males had a 47% less likely chance of a correct diagnosis relative to females (p=0.05). Post-MRI data inclusion within GPT-3.5, the odds of an accurate diagnosis was 4.0-fold greater for Gen Z participants, relative to non-Gen Z participants (p=0.01) with the diagnostic accuracy being 68% less in males relative to females (p=0.009), and 75% less for White subjects, relative to non-White subjects (p=0.0004). CONCLUSION: Although generative AI platforms enable rapid information access and are not principally designed for use in healthcare, an increase in use by Gen Z is anticipated. However, the obtained responses may not be generalizable to all users and bias may exist in select groups.

Design and Implementation of a Brief, Self-Directed Course on Immunotherapy Best Practices for Neurology Trainees.

OBJECTIVES: To create and implement a brief, self-directed course on immunotherapy (IT) best practices for trainees on a neuroimmunology elective rotation. METHODS: A working group of neurology faculty developed a curriculum covering the mechanism of action, indications, and necessary monitoring for different IT used in neurology practice. The content was presented as a web-based course and hosted on local servers. Neurology residents and fellows participating in a neuroimmunology elective were given access to the curriculum over a 2-week period. A multiple-choice assessment and questionnaire assessing learner confidence with IT was administered prior to starting the course, and again upon course completion. Twelve months after implementation, the pretest and posttest were revised following an item analysis. RESULTS: Twenty-two neurology residents and fellows completed the course since July 2022. The average score on the first version of the pretest and posttest was 78% versus 92% (P = .02), and 51% versus 70% (P = .02) on the revised version. Trainee self-reported confidence with IT also improved, although only 59.1% of trainees completed the postcourse questionnaire. Respondents provided positive feedback on the format and content of the course and expressed a desire for a reference to the material for future use. CONCLUSION: In this pilot study, our course improved resident confidence and knowledge of IT best practices. The course was well-received, and our methods can be implemented in a variety of clinical environments to supplement trainee learning.

Cladribine tablets after treatment with natalizumab (CLADRINA) - rationale and design.

Background: Individual disease modifying therapies approved for multiple sclerosis (MS) have limited effectiveness and potentially serious side effects, especially when administered over long periods. Sequential combination therapy is a plausible alternative approach. Natalizumab is a monoclonal therapeutic antibody that reduces leukocyte access to the central nervous system that is associated with an increased risk of progressive multifocal leukoencephalopathy and disease reactivation after its discontinuation. Cladribine tablets act as a synthetic adenosine analog, disrupting DNA synthesis and repair, thereby reducing the number of lymphocytes. The generation of prospective, rigorous safety, and efficacy data in transitioning from natalizumab to cladribine is an unmet clinical need. Objectives: To test the feasibility of transitioning patients with relapsing forms of MS natalizumab to cladribine tablets. Design: Cladribine tablets after treatment with natalizumab (CLADRINA) is an open-label, single-arm, multicenter, collaborative phase IV, research study that will generate hypothesis regarding the safety, efficacy, and immunological impact of transition from natalizumab to cladribine tablets in patients with relapsing forms of MS. Methods and analysis: Participants will be recruited from three different sites. The primary endpoint is the absolute and percent change from baseline of lymphocytes and myeloid cell subsets, as well as blood neurofilament light levels. The secondary endpoint is the annualized relapse rate over the 12- and 24-month trial periods. Exploratory endpoints include the expanded disability status scale, and magnetic resonance imaging outcomes. Discussion: The CLADRINA trial will generate data regarding the safety, efficacy, and immunological impact of the transition from natalizumab to cladribine. As the pace of immunological knowledge of MS continues, insight into disease modifying therapy transition strategies is needed.