Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study.

BACKGROUND: Large language models (LLMs) have demonstrated impressive performances in various medical domains, prompting an exploration of their potential utility within the high-demand setting of emergency department (ED) triage. This study evaluated the triage proficiency of different LLMs and ChatGPT, an LLM-based chatbot, compared to professionally trained ED staff and untrained personnel. We further explored whether LLM responses could guide untrained staff in effective triage. OBJECTIVE: This study aimed to assess the efficacy of LLMs and the associated product ChatGPT in ED triage compared to personnel of varying training status and to investigate if the models' responses can enhance the triage proficiency of untrained personnel. METHODS: A total of 124 anonymized case vignettes were triaged by untrained doctors; different versions of currently available LLMs; ChatGPT; and professionally trained raters, who subsequently agreed on a consensus set according to the Manchester Triage System (MTS). The prototypical vignettes were adapted from cases at a tertiary ED in Germany. The main outcome was the level of agreement between raters' MTS level assignments, measured via quadratic-weighted Cohen κ. The extent of over- and undertriage was also determined. Notably, instances of ChatGPT were prompted using zero-shot approaches without extensive background information on the MTS. The tested LLMs included raw GPT-4, Llama 3 70B, Gemini 1.5, and Mixtral 8x7b. RESULTS: GPT-4-based ChatGPT and untrained doctors showed substantial agreement with the consensus triage of professional raters (κ=mean 0.67, SD 0.037 and κ=mean 0.68, SD 0.056, respectively), significantly exceeding the performance of GPT-3.5-based ChatGPT (κ=mean 0.54, SD 0.024; P<.001). When untrained doctors used this LLM for second-opinion triage, there was a slight but statistically insignificant performance increase (κ=mean 0.70, SD 0.047; P=.97). Other tested LLMs performed similar to or worse than GPT-4-based ChatGPT or showed odd triaging behavior with the used parameters. LLMs and ChatGPT models tended toward overtriage, whereas untrained doctors undertriaged. CONCLUSIONS: While LLMs and the LLM-based product ChatGPT do not yet match professionally trained raters, their best models' triage proficiency equals that of untrained ED doctors. In its current form, LLMs or ChatGPT thus did not demonstrate gold-standard performance in ED triage and, in the setting of this study, failed to significantly improve untrained doctors' triage when used as decision support. Notable performance enhancements in newer LLM versions over older ones hint at future improvements with further technological development and specific training.

The Use of Nitrosative Stress Molecules as Potential Diagnostic Biomarkers in Multiple Sclerosis.

Multiple sclerosis (MS) is an autoimmune disease of the central nervous system (CNS) of still unclear etiology. In recent years, the search for biomarkers facilitating its diagnosis, prognosis, therapy response, and other parameters has gained increasing attention. In this regard, in a previous meta-analysis comprising 22 studies, we found that MS is associated with higher nitrite/nitrate (NOx) levels in the cerebrospinal fluid (CSF) compared to patients with non-inflammatory other neurological diseases (NIOND). However, many of the included studies did not distinguish between the different clinical subtypes of MS, included pre-treated patients, and inclusion criteria varied. As a follow-up to our meta-analysis, we therefore aimed to analyze the serum and CSF NOx levels in clinically well-defined cohorts of treatment-naïve MS patients compared to patients with somatic symptom disorder. To this end, we analyzed the serum and CSF levels of NOx in 117 patients (71 relapsing-remitting (RR) MS, 16 primary progressive (PP) MS, and 30 somatic symptom disorder). We found that RRMS and PPMS patients had higher serum NOx levels compared to somatic symptom disorder patients. This difference remained significant in the subgroup of MRZ-negative RRMS patients. In conclusion, the measurement of NOx in the serum might indeed be a valuable tool in supporting MS diagnosis.

Real-world evidence on siponimod treatment in patients with secondary progressive multiple sclerosis.

BACKGROUND: Therapeutic options targeting inflammation in multiple sclerosis (MS) have evolved rapidly for relapsing-remitting MS, whereas few therapies are available for progressive forms of MS, in particular secondary progressive MS (SPMS). The approval of siponimod for SPMS has allowed for optimism in the otherwise discouraging therapeutic landscape. METHODS: We conducted a retrospective, multicenter, non-interventional study analyzing the efficacy and safety of siponimod under real-world conditions in 227 SPMS patients. According to the retrospective study framework, data was acquired at prespecified time points. Clinical readouts were assessed every three months. Disease progression was determined as increase in expanded disability status scale (EDSS), radiological progression, or the occurrence of new relapses under treatment. For safety analyses, adverse events (AE) and reasons for discontinuation were documented. The collected data points were analyzed at baseline and after 6, 12 and 18 months. However, data were predominately collected at the 6- and 12-month time points as many patients were lost to follow-up. In a group consisting of 41 patients, a more detailed investigation regarding disease progression was conducted, including data from measurement of cognitive and motoric functions. RESULTS: Under siponimod therapy, 64.8% of patients experienced sustained clinical disease stability at 12 months. Out of the stable patients 21.4% of patients improved. Of the remaining patients, 31.5% experienced EDSS progression, 3.7% worsened without meeting the threshold for progression. Relapses occurred in 7.4%. Radiological disease activity was detected in 24.1% of patients after six months of treatment and in 29.6% of patients at 12 months follow-up. The in-depth cohort consisting of 41 patients demonstrated no substantial changes in cognitive abilities measured by Paced Auditory Serial Addition Test and Symbol Digit Modalities Test or motoric functions measured with Timed 25-Foot Walk, 100-m timed test, and 9-Hole Peg Test throughout the 12-month study period. Radiological assessment showed a stable volume of white and grey matter, as well as a stable lesion count at 12 months follow-up. AE were observed in nearly half of the included patients, with lymphopenia being the most common. Due to disease progression or AE, 31.2% of patients discontinued therapy. CONCLUSION: Treatment with siponimod had an overall stabilizing effect regarding clinical and radiological outcome measures. However, there is a need for more intensive treatment management and monitoring to identify disease progression and AE.

Immune response to SARS-CoV-2 vaccination in relation to peripheral immune cell profiles among patients with multiple sclerosis receiving ocrelizumab.

BACKGROUND: Vaccination has proven to be effective in preventing SARS-CoV-2 transmission and severe disease courses. However, immunocompromised patients have not been included in clinical trials and real-world clinical data point to an attenuated immune response to SARS-CoV-2 vaccines among patients with multiple sclerosis (MS) receiving immunomodulatory therapies. METHODS: We performed a retrospective study including 59 ocrelizumab (OCR)-treated patients with MS who received SARS-CoV-2 vaccination. Anti-SARS-CoV-2-antibody titres, routine blood parameters and peripheral immune cell profiles were measured prior to the first (baseline) and at a median of 4 weeks after the second vaccine dose (follow-up). Moreover, the SARS-CoV-2-specific T cell response and peripheral B cell subsets were analysed at follow-up. Finally, vaccination-related adverse events were assessed. RESULTS: After vaccination, we found anti-SARS-CoV-2(S) antibodies in 27.1% and a SARS-CoV-2-specific T cell response in 92.7% of MS cases. T cell-mediated interferon (IFN)-γ release was more pronounced in patients without anti-SARS-CoV-2(S) antibodies. Antibody titres positively correlated with peripheral B cell counts, time since last infusion and total IgM levels. They negatively correlated with the number of previous infusion cycles. Peripheral plasma cells were increased in antibody-positive patients. A positive correlation between T cell response and peripheral lymphocyte counts was observed. Moreover, IFN-γ release was negatively correlated with the time since the last infusion. CONCLUSION: In OCR-treated patients with MS, the humoral immune response to SARS-CoV-2 vaccination is attenuated while the T cell response is preserved. However, it is still unclear whether T or B cell-mediated immunity is required for effective clinical protection. Nonetheless, given the long-lasting clinical effects of OCR, monitoring of peripheral B cell counts could facilitate individualised treatment regimens and might be used to identify the optimal time to vaccinate.

Vaccine-based clinical protection against SARS-CoV-2 infection and the humoral immune response: A 1-year follow-up study of patients with multiple sclerosis receiving ocrelizumab.

Introduction: Given the varying severity of coronavirus disease 2019 (COVID-19) and the rapid spread of Severe-Acute-Respiratory-Syndrome-Corona-Virus-2 (SARS-CoV-2), vaccine-mediated protection of particularly vulnerable individuals has gained increasing attention during the course of the pandemic. Methods: We performed a 1-year follow-up study of 51 ocrelizumab-treated patients with multiple sclerosis (OCR-pwMS) who received COVID-19 vaccination in 2021. We retrospectively identified 37 additional OCR-pwMS, 42 pwMS receiving natalizumab, 27 pwMS receiving sphingosine 1-phosphate receptor modulators, 59 pwMS without a disease-modifying therapy, and 61 controls without MS (HC). In OCR-pwMS, anti-SARS-CoV-2(S)-antibody titers were measured prior to the first and after the second, third, and fourth vaccine doses (pv2/3/4). The SARS-CoV-2-specific T cell response was analyzed pv2. SARS-CoV-2 infection status, COVID-19 disease severity, and vaccination-related adverse events were assessed in all pwMS and HC. Results: We found a pronounced and increasing anti-SARS-CoV-2(S)-antibody response after COVID-19 booster vaccinations in OCR-pwMS (pv2: 30.4%, pv3: 56.5%, and pv4 90.0% were antibody positive). More than one third of OCR-pwMS without detectable antibodies pv2 developed positive antibodies pv3. 23.5% of OCR-pwMS had a confirmed SARS-CoV-2 infection, of which 84.2% were symptomatic. Infection rates were comparable between OCR-pwMS and control groups. None of the pwMS had severe COVID-19. An attenuated humoral immune response was not associated with a higher risk of SARS-CoV-2 infection. Discussion: Additional COVID-19 vaccinations can boost the humoral immune response in OCR-pwMS and improve clinical protection against COVID-19. Vaccines effectively protect even OCR-pwMS without a detectable COVID-19 specific humoral immune response, indicating compensatory, e.g., T cell-mediated immunological mechanisms.

Cord Blood Low-Density Granulocytes Correspond to an Immature Granulocytic Subset with Low Expression of S100A12.

Although substantial progress has been achieved concerning neonatal sepsis, its lethality remains considerably high, and further insights into peculiarities and malfunctions of neonatal immunity are needed. This study aims to contribute to a better understanding of the role of human neonatal granulocyte subpopulations and calgranulin C (S100A12). For this purpose, we gathered 136 human cord blood (CB) samples. CD66b+ CB low-density granulocytes (LDG) and CB normal-density granulocytes were isolated and functionally and phenotypically compared with healthy adult control granulocytes. We could identify CB-LDG as CD66bbright CD64high CD16low CD35low CD10low S100A12med-low and, based on these markers, recovered in whole CB stainings. Consistent with flow cytometric findings, microscopic imaging supported an immature phenotype of CB-LDG with decreased S100A12 expression. In CB serum of healthy neonates, S100A12 was found to be higher in female newborns when compared with males. Additionally, S100A12 levels correlated positively with gestational age independently from sex. We could solidify functional deficits of CB-LDG concerning phagocytosis and generation of neutrophil extracellular traps. Our study reveals that previously described suppressive effects of CB-LDG on CD4+ T cell proliferation are exclusively due to phagocytosis of stimulation beads used in cocultures and absent when using soluble or coated Abs. In conclusion, we characterize CB-LDG as immature neutrophils with functional deficits and decreased expression and storage of S100A12. Concerning their cross-talk with the adaptive immunity, we found no direct inhibitory effect of LDG. Neonatal LDG may thus represent a distinct population that differs from LDG populations found in adults.