Generative artificial intelligence versus clinicians: Who diagnoses multiple sclerosis faster and with greater accuracy?

BACKGROUND: Those receiving the diagnosis of multiple sclerosis (MS) over the next ten years will predominantly be part of Generation Z (Gen Z). Recent observations within our clinic suggest that younger people with MS utilize online generative artificial intelligence (AI) platforms for personalized medical advice prior to their first visit with a specialist in neuroimmunology. The use of such platforms is anticipated to increase given the technology driven nature, desire for instant communication, and cost-conscious nature of Gen Z. Our objective was to determine if ChatGPT (Generative Pre-trained Transformer) could diagnose MS in individuals earlier than their clinical timeline, and to assess if the accuracy differed based on age, sex, and race/ethnicity. METHODS: People with MS between 18 and 59 years of age were studied. The clinical timeline for people diagnosed with MS was retrospectively identified and simulated using ChatGPT-3.5 (GPT-3.5). Chats were conducted using both actual and derivatives of their age, sex, and race/ethnicity to test diagnostic accuracy. A Kaplan-Meier survival curve was estimated for time to diagnosis, clustered by subject. The p-value testing for differences in time to diagnosis was accomplished using a general Wilcoxon test. Logistic regression (subject-specific intercept) was used to capture intra-subject correlation to test the accuracy prior to and after the inclusion of MRI data. RESULTS: The study cohort included 100 unique people with MS. Of those, 50 were members of Gen Z (38 female; 22 White; mean age at first symptom was 20.6 years (y) (standard deviation (SD)=2.2y)), and 50 were non-Gen Z (34 female; 27 White; mean age at first symptom was 37.0y (SD=10.4y)). In addition, a total of 529 people that represented digital simulations of the original cohort of 100 people (333 female; 166 White; 136 Black/African American; 107 Asian; 120 Hispanic, mean age at first symptom was 31.6y (SD=12.4y)) were generated allowing for 629 scripted conversations to be analyzed. The estimated median time to diagnosis in clinic was significantly longer at 0.35y (95% CI=[0.28, 0.48]) versus that by ChatGPT at 0.08y (95% CI=[0.04, 0.24]) (p<0.0001). There was no difference in the diagnostic accuracy between ages and by race/ethnicity prior to the inclusion of MRI data. However, prior to including the MRI data, males had a 47% less likely chance of a correct diagnosis relative to females (p=0.05). Post-MRI data inclusion within GPT-3.5, the odds of an accurate diagnosis was 4.0-fold greater for Gen Z participants, relative to non-Gen Z participants (p=0.01) with the diagnostic accuracy being 68% less in males relative to females (p=0.009), and 75% less for White subjects, relative to non-White subjects (p=0.0004). CONCLUSION: Although generative AI platforms enable rapid information access and are not principally designed for use in healthcare, an increase in use by Gen Z is anticipated. However, the obtained responses may not be generalizable to all users and bias may exist in select groups.

Dynamic Expansion and Contraction of Multiple Sclerosis T2-weighted Hyperintense Lesions are Present Below the Threshold of Visual Perception.

BACKGROUND AND PURPOSE: The study of T2-weighted hyperintense lesions resulting from autoimmune inflammatory injury and associated volumes within the CNS remains fundamental to the diagnosis and disease surveillance of multiple sclerosis (MS). We investigated the dynamic changes of individual T2-weighted hyperintense MS lesions on MRI and hypothesized that variations may be present below the threshold of visual perception when evaluating longitudinal data. MATERIALS AND METHODS: A retrospective study was performed of people with MS, incorporating data from three consecutive MRI time points acquired within a single academic center. All included MRI studies lacked formal imaging interpretations of newly enlarging or contracting T2-weighted hyperintensities. Well defined, non-coalescing, individual T2-weighted hyperintense lesions were targeted. A total of 8-12 lesions were randomly selected in a blinded fashion at MRI time point 1 and 3-dimensional lesion volumes followed over MRI time points 2 and 3. The impact of treatment on lesion expansion and relationship to brain MRI advancement, patient-reported progression of disease, and physician-identified progression was also studied. RESULTS: The study cohort was comprised of 115 people (81 (70.4%) female; mean disease duration of 6.62 years(y) (standard deviation: 6.68y)) who were primarily White (79.1%). A total of 1,426 focal T2-weighted hyperintense MS lesions were identified on MRI time point 1 and longitudinally followed over MRI time points 2 and 3. In the evaluation of raw changes in individual T2-weighted hyperintense lesion volumes from MRI time point 1 to MRI time point 2, a similar number of individuals were observed with predominantly expanding (49/115; 42.6%) or contracting (51/115; 44.3%) lesions. However, the majority of lesions expanded in volume (48/115; 41.7%) versus those that contracted (45/115; 39.1%) when evaluating MRI time point 3 to time point 1. Those individuals not on active treatment had a 67.15% reduction in the odds of more individual lesions predominantly contracting in volume relative to those on low-efficacy disease modifying therapy treatment (95% CI=[-83.89, -33.01], p=0.0008) and 74.02% reduction for those on high-efficacy treatment (95% CI=[-87.37%,-46.56%], p<0.0001). CONCLUSIONS: Dynamic changes in T2-weighted hyperintense lesions are abundant, occurring below the threshold of visual perception and are present more frequently in untreated individuals. ABBREVIATIONS: MS = multiple sclerosis; DMT = disease modifying therapy; 2D = 2-dimensional; 3D = 3-dimensional; LMS = Lambda, Mu, and Sigma.

Underdosed generic specialty medications: A prescription for patient harm?

The use of generic specialty medications amongst individuals with multiple sclerosis (MS) has expanded due to an increase in the number of available agents. We describe a woman who was denied continued use of brand name teriflunomide (AubagioⓇ), despite being clinically stable for 2.5 years, and switched to generic teriflunomide. She experienced a significant spinal cord exacerbation within a few months of starting treatment. We analyzed 3 generic teriflunomide agents, including the one used for treatment, in addition to AubagioⓇ. The generic teriflunomide used by our patient contained 55.5 % content of the labeled amount, well below U.S. FDA specifications.

Dimethyl fumarate preserves brainstem and cervical spinal cord integrity in radiologically isolated syndrome.

BACKGROUND AND PURPOSE: The first randomized placebo-controlled therapeutic trial in radiologically isolated syndrome (RIS), ARISE, demonstrated that treatment with dimethyl fumarate (DMF) delayed the onset of a first clinical event related to CNS demyelination and was associated with a significant reduction in new and/or newly enlarging T2-weighted hyperintense lesions. The purpose of this study was to explore the effect of DMF on volumetric measures, including whole brain, thalamic, and subcortical gray matter volumes, brainstem and upper cervical spine three-dimensional (3D) volumes, and brainstem and upper cervical spine surface characteristics. METHODS: Standardized 3T MRIs including 3D isotropic T1-weighted gradient echo images were acquired at baseline and end-of-study according to the ARISE study protocol. The acquired data were analyzed using Structural Image Evaluation Using Normalization of Atrophy (SIENA), FreeSurfer v7.3, and an in-house pipeline for 3D conformational metrics. Multivariate mixed models for repeated measures were used to analyze rates of change in whole brain, thalamic, subcortical gray matter, as well as change in the 3D surface curvature of the dorsal pons and dorsal medulla and 3D volume change at the medulla-upper cervical spinal cord. RESULTS: The study population consisted of 64 RIS subjects (DMF:30, placebo:34). No significant difference was seen in whole brain, thalamic, or subcortical gray matter volumes in treated vs. untreated RIS patients. A significant difference was observed in dorsal pons curvature with the DMF group having a lower least squares mean change of - 4.46 (standard estimate (SE): 3.77) when compared to placebo [6.94 (3.71)] (p = 0.036). In individuals that experienced a first clinical event, a greater reduction in medulla-upper cervical spinal cord volume (p = 0.044) and a decrease in surface curvature was observed at the dorsal medulla (p = 0.009) but not at the dorsal pons (p = 0.443). CONCLUSIONS: The benefit of disease-modifying therapy in RIS may extend to CNS structures impacted by neurodegeneration that is below the resolution of conventional volumetric measures.