Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores.

Importance: Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR. Objective: Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates. Methods: Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation. Results: For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date. Conclusions: In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

Quadriceps fatigue during hypoxic and ischemic knee-extension exercise is similar in males and females.

Hypoxia is known to increase muscle fatigue via both central and peripheral mechanisms. Females are typically less fatigable than males during isometric fatiguing contractions due to greater peripheral blood flow. However, sex differences in fatigue are blunted during dynamic fatiguing tasks. Thus, this study determined the interactions of sex and hypoxia on knee extensor muscle contractile function during a dynamic, ischemic fatiguing contraction. Electrical stimulation was used to determine contractile properties of the knee extensor muscles in eight males and eight females before and after an ischemic, dynamic fatiguing task while inspiring room air or a hypoxic gas mixture (10% O2:90% N2). Fatigue (assessed as time-to-task failure) was ∼10% greater during the hypoxic condition (94.3 ± 33.4 s) compared with normoxic condition (107.0 ± 42.8 s, P = 0.041) and ∼40% greater for females than males (77.1 ± 18.8 vs. 124.2 ± 38.7, P < 0.001). Immediately after the dynamic fatiguing task, there were reductions in maximal voluntary contraction force (P = 0.034) and electrically evoked twitch force (P < 0.001), and these reductions did not differ based on sex or inspirate. Cerebral tissue oxygenation showed a significant interaction of time and inspirate (P = 0.003) whereby it increased during normoxia and remained unchanged in hypoxia. No sex-related differences in the changes of cerebral tissue oxygenation were observed (P = 0.528). These data suggest that acute hypoxia increases central fatigue during ischemic single-leg exercise resulting in earlier exercise termination, but the effect does not differ based on sex.NEW & NOTEWORTHY Hypoxia exacerbates fatigue via central mechanisms after ischemic single-leg exercise. The greater fatigue observed during ischemic dynamic fatiguing exercise with hypoxia inspirate did not differ between the sexes. Hypoxia-induced central limitations are present in acute ischemic exercise and do not appear different in males and females.

Rates Among Hospitalized Patients With COVID-19 Treated With Convalescent Plasma: A Systematic Review and Meta-Analysis.

Objective: To examine the association of COVID-19 convalescent plasma transfusion with mortality and the differences between subgroups in hospitalized patients with COVID-19. Patients and Methods: On October 26, 2022, a systematic search was performed for clinical studies of COVID-19 convalescent plasma in the literature from January 1, 2020, to October 26, 2022. Randomized clinical trials and matched cohort studies investigating COVID-19 convalescent plasma transfusion compared with standard of care treatment or placebo among hospitalized patients with confirmed COVID-19 were included. The electronic search yielded 3841 unique records, of which 744 were considered for full-text screening. The selection process was performed independently by a panel of 5 reviewers. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Data were extracted by 5 independent reviewers in duplicate and pooled using an inverse-variance random effects model. The prespecified end point was all-cause mortality during hospitalization. Results: Thirty-nine randomized clinical trials enrolling 21,529 participants and 70 matched cohort studies enrolling 50,160 participants were included in the systematic review. Separate meta-analyses reported that transfusion of COVID-19 convalescent plasma was associated with a decrease in mortality compared with the control cohort for both randomized clinical trials (odds ratio [OR], 0.87; 95% CI, 0.76-1.00) and matched cohort studies (OR, 0.76; 95% CI, 0.66-0.88). The meta-analysis of subgroups revealed 2 important findings. First, treatment with convalescent plasma containing high antibody levels was associated with a decrease in mortality compared with convalescent plasma containing low antibody levels (OR, 0.85; 95% CI, 0.73 to 0.99). Second, earlier treatment with COVID-19 convalescent plasma was associated with a decrease in mortality compared with the later treatment cohort (OR, 0.63; 95% CI, 0.48 to 0.82). Conclusion: During COVID-19 convalescent plasma use was associated with a 13% reduced risk of mortality, implying a mortality benefit for hospitalized patients with COVID-19, particularly those treated with convalescent plasma containing high antibody levels treated earlier in the disease course.

Access to and safety of COVID-19 convalescent plasma in the United States Expanded Access Program: A national registry study.

BACKGROUND: The United States (US) Expanded Access Program (EAP) to coronavirus disease 2019 (COVID-19) convalescent plasma was initiated in response to the rapid spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of COVID-19. While randomized clinical trials were in various stages of development and enrollment, there was an urgent need for widespread access to potential therapeutic agents. The objective of this study is to report on the demographic, geographical, and chronological characteristics of patients in the EAP, and key safety metrics following transfusion of COVID-19 convalescent plasma. METHODS AND FINDINGS: Mayo Clinic served as the central institutional review board for all participating facilities, and any US physician could participate as a local physician-principal investigator. Eligible patients were hospitalized, were aged 18 years or older, and had-or were at risk of progression to-severe or life-threatening COVID-19; eligible patients were enrolled through the EAP central website. Blood collection facilities rapidly implemented programs to collect convalescent plasma for hospitalized patients with COVID-19. Demographic and clinical characteristics of all enrolled patients in the EAP were summarized. Temporal patterns in access to COVID-19 convalescent plasma were investigated by comparing daily and weekly changes in EAP enrollment in response to changes in infection rate at the state level. Geographical analyses on access to convalescent plasma included assessing EAP enrollment in all national hospital referral regions, as well as assessing enrollment in metropolitan areas and less populated areas that did not have access to COVID-19 clinical trials. From April 3 to August 23, 2020, 105,717 hospitalized patients with severe or life-threatening COVID-19 were enrolled in the EAP. The majority of patients were 60 years of age or older (57.8%), were male (58.4%), and had overweight or obesity (83.8%). There was substantial inclusion of minorities and underserved populations: 46.4% of patients were of a race other than white, and 37.2% of patients were of Hispanic ethnicity. Chronologically and geographically, increases in the number of both enrollments and transfusions in the EAP closely followed confirmed infections across all 50 states. Nearly all national hospital referral regions enrolled and transfused patients in the EAP, including both in metropolitan and in less populated areas. The incidence of serious adverse events was objectively low (<1%), and the overall crude 30-day mortality rate was 25.2% (95% CI, 25.0% to 25.5%). This registry study was limited by the observational and pragmatic study design that did not include a control or comparator group; thus, the data should not be used to infer definitive treatment effects. CONCLUSIONS: These results suggest that the EAP provided widespread access to COVID-19 convalescent plasma in all 50 states, including for underserved racial and ethnic minority populations. The study design of the EAP may serve as a model for future efforts when broad access to a treatment is needed in response to an emerging infectious disease. TRIAL REGISTRATION: ClinicalTrials.gov NCT#: NCT04338360.

Mortality in individuals treated with COVID-19 convalescent plasma varies with the geographic provenance of donors.

Successful therapeutics and vaccines for coronavirus disease 2019 (COVID-19) have harnessed the immune response to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Evidence that SARS-CoV-2 exists as locally evolving variants suggests that immunological differences may impact the effectiveness of antibody-based treatments such as convalescent plasma and vaccines. Considering that near-sourced convalescent plasma likely reflects the antigenic composition of local viral strains, we hypothesize that convalescent plasma has a higher efficacy, as defined by death within 30 days of transfusion, when the convalescent plasma donor and treated patient were in close geographic proximity. Results of a series of modeling techniques applied to approximately 28,000 patients from the Expanded Access to Convalescent Plasma program (ClinicalTrials.gov number: NCT04338360) support this hypothesis. This work has implications for the interpretation of clinical studies, the ability to develop effective COVID-19 treatments, and, potentially, for the effectiveness of COVID-19 vaccines as additional locally-evolving variants continue to emerge.

Program and patient characteristics for the United States Expanded Access Program to COVID-19 convalescent plasma.

BACKGROUND: The United States (US) Expanded Access Program (EAP) to COVID-19 convalescent plasma was initiated in response to the rapid spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease-2019 (COVID-19). While randomized clinical trials were in various stages of development and enrollment, there was an urgent need for widespread access to potential therapeutic agents particularly for vulnerable racial and ethnic minority populations who were disproportionately affected by the pandemic. The objective of this study is to report on the demographic, geographic, and chronological access to COVID-19 convalescent plasma in the US via the EAP. METHODS AND FINDINGS: Mayo Clinic served as the central IRB for all participating facilities and any US physician could participate as local physician-principal investigator. Registration occurred through the EAP central website. Blood banks rapidly developed logistics to provide convalescent plasma to hospitalized patients with COVID-19. Demographic and clinical characteristics of all enrolled patients in the EAP were summarized. Temporal trends in access to COVID-19 convalescent plasma were investigated by comparing daily and weekly changes in EAP enrollment in response to changes in infection rate on a state level. Geographical analyses on access to convalescent plasma included assessing EAP enrollment in all national hospital referral regions as well as assessing enrollment in metropolitan and less populated areas which did not have access to COVID-19 clinical trials.From April 3 to August 23, 2020, 105,717 hospitalized patients with severe or life-threatening COVID-19 were enrolled in the EAP. A majority of patients were older than 60 years of age (57.8%), male (58.4%), and overweight or obese (83.8%). There was substantial inclusion of minorities and underserved populations, including 46.4% of patients with a race other than White, and 37.2% of patients were of Hispanic ethnicity. Severe or life-threatening COVID-19 was present in 61.8% of patients and 18.9% of patients were mechanically ventilated at time of convalescent plasma infusion. Chronologically and geographically, increases in enrollment in the EAP closely followed confirmed infections across all 50 states. Nearly all national hospital referral regions enrolled patients in the EAP, including both in metropolitan and less populated areas. CONCLUSIONS: The EAP successfully provided widespread access to COVID-19 convalescent plasma in all 50 states, including for underserved racial and ethnic minority populations. The efficient study design of the EAP may serve as an example framework for future efforts when broad access to a treatment is needed in response to a dynamic disease affecting demographic groups and areas historically underrepresented in clinical studies.

Convalescent Plasma Antibody Levels and the Risk of Death from Covid-19.

BACKGROUND: Convalescent plasma has been widely used to treat coronavirus disease 2019 (Covid-19) under the presumption that such plasma contains potentially therapeutic antibodies to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that can be passively transferred to the plasma recipient. Whether convalescent plasma with high antibody levels rather than low antibody levels is associated with a lower risk of death is unknown. METHODS: In a retrospective study based on a U.S. national registry, we determined the anti-SARS-CoV-2 IgG antibody levels in convalescent plasma used to treat hospitalized adults with Covid-19. The primary outcome was death within 30 days after plasma transfusion. Patients who were enrolled through July 4, 2020, and for whom data on anti-SARS-CoV-2 antibody levels in plasma transfusions and on 30-day mortality were available were included in the analysis. RESULTS: Of the 3082 patients included in this analysis, death within 30 days after plasma transfusion occurred in 115 of 515 patients (22.3%) in the high-titer group, 549 of 2006 patients (27.4%) in the medium-titer group, and 166 of 561 patients (29.6%) in the low-titer group. The association of anti-SARS-CoV-2 antibody levels with the risk of death from Covid-19 was moderated by mechanical ventilation status. A lower risk of death within 30 days in the high-titer group than in the low-titer group was observed among patients who had not received mechanical ventilation before transfusion (relative risk, 0.66; 95% confidence interval [CI], 0.48 to 0.91), and no effect on the risk of death was observed among patients who had received mechanical ventilation (relative risk, 1.02; 95% CI, 0.78 to 1.32). CONCLUSIONS: Among patients hospitalized with Covid-19 who were not receiving mechanical ventilation, transfusion of plasma with higher anti-SARS-CoV-2 IgG antibody levels was associated with a lower risk of death than transfusion of plasma with lower antibody levels. (Funded by the Department of Health and Human Services and others; ClinicalTrials.gov number, NCT04338360.).

The Role of Disease Severity and Demographics in the Clinical Course of COVID-19 Patients Treated With Convalescent Plasma.

Treatment of patients with COVID-19 using convalescent plasma from recently recovered patients has been shown to be safe, but the time course of change in clinical status following plasma transfusion in relation to baseline disease severity has not yet been described. We analyzed short, descriptive daily reports of patient status in 7,180 hospitalized recipients of COVID-19 convalescent plasma in the Mayo Clinic Expanded Access Program. We assessed, from the day following transfusion, whether the patient was categorized by his or her physician as better, worse or unchanged compared to the day before, and whether, on the reporting day, the patient received mechanical ventilation, was in the ICU, had died or had been discharged. Most patients improved following transfusion, but clinical improvement was most notable in mild to moderately ill patients. Patients classified as severely ill upon enrollment improved, but not as rapidly, while patients classified as critically ill/end-stage and patients on ventilators showed worsening of disease status even after treatment with convalescent plasma. Patients age 80 and over showed little or no clinical improvement following transfusion. Clinical status at the time of convalescent plasma treatment and age appear to be the primary factors in determining the therapeutic effectiveness of COVID-19 convalescent plasma among hospitalized patients.

Effect of Convalescent Plasma on Mortality among Hospitalized Patients with COVID-19: Initial Three-Month Experience.

IMPORTANCE: Passive antibody transfer is a longstanding treatment strategy for infectious diseases that involve the respiratory system. In this context, human convalescent plasma has been used to treat coronavirus disease 2019 (COVID-19), but the efficacy remains uncertain. OBJECTIVE: To explore potential signals of efficacy of COVID-19 convalescent plasma. DESIGN: Open-label, Expanded Access Program (EAP) for the treatment of COVID-19 patients with human convalescent plasma. SETTING: Multicenter, including 2,807 acute care facilities in the US and territories. PARTICIPANTS: Adult participants enrolled and transfused under the purview of the US Convalescent Plasma EAP program between April 4 and July 4, 2020 who were hospitalized with (or at risk of) severe or life threatening acute COVID-19 respiratory syndrome. INTERVENTION: Transfusion of at least one unit of human COVID-19 convalescent plasma using standard transfusion guidelines at any time during hospitalization. Convalescent plasma was donated by recently-recovered COVID-19 survivors, and the antibody levels in the units collected were unknown at the time of transfusion. Main Outcomes and Measures: Seven and thirty-day mortality. RESULTS: The 35,322 transfused patients had heterogeneous demographic and clinical characteristics. This cohort included a high proportion of critically-ill patients, with 52.3% in the intensive care unit (ICU) and 27.5% receiving mechanical ventilation at the time of plasma transfusion. The seven-day mortality rate was 8.7% [95% CI 8.3%-9.2%] in patients transfused within 3 days of COVID-19 diagnosis but 11.9% [11.4%-12.2%] in patients transfused 4 or more days after diagnosis (p<0.001). Similar findings were observed in 30-day mortality (21.6% vs. 26.7%, p<0.0001). Importantly, a gradient of mortality was seen in relation to IgG antibody levels in the transfused plasma. For patients who received high IgG plasma (>18.45 S/Co), seven-day mortality was 8.9% (6.8%, 11.7%); for recipients of medium IgG plasma (4.62 to 18.45 S/Co) mortality was 11.6% (10.3%, 13.1%); and for recipients of low IgG plasma (<4.62 S/Co) mortality was 13.7% (11.1%, 16.8%) (p=0.048). This unadjusted dose-response relationship with IgG was also observed in thirty-day mortality (p=0.021). The pooled relative risk of mortality among patients transfused with high antibody level plasma units was 0.65 [0.47-0.92] for 7 days and 0.77 [0.63-0.94] for 30 days compared to low antibody level plasma units. CONCLUSIONS AND RELEVANCE: The relationships between reduced mortality and both earlier time to transfusion and higher antibody levels provide signatures of efficacy for convalescent plasma in the treatment of hospitalized COVID-19 patients. This information may be informative for the treatment of COVID-19 and design of randomized clinical trials involving convalescent plasma. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04338360.

Safety Update: COVID-19 Convalescent Plasma in 20,000 Hospitalized Patients.

OBJECTIVE: To provide an update on key safety metrics after transfusion of convalescent plasma in hospitalized coronavirus 2019 (COVID-19) patients, having previously demonstrated safety in 5000 hospitalized patients. PATIENTS AND METHODS: From April 3 to June 2, 2020, the US Food and Drug Administration Expanded Access Program for COVID-19 convalescent plasma transfused a convenience sample of 20,000 hospitalized patients with COVID-19 convalescent plasma. RESULTS: The incidence of all serious adverse events was low; these included transfusion reactions (n=78; <1%), thromboembolic or thrombotic events (n=113; <1%), and cardiac events (n=677, ~3%). Notably, the vast majority of the thromboembolic or thrombotic events (n=75) and cardiac events (n=597) were judged to be unrelated to the plasma transfusion per se. The 7-day mortality rate was 13.0% (12.5%, 13.4%), and was higher among more critically ill patients relative to less ill counterparts, including patients admitted to the intensive care unit versus those not admitted (15.6 vs 9.3%), mechanically ventilated versus not ventilated (18.3% vs 9.9%), and with septic shock or multiple organ dysfunction/failure versus those without dysfunction/failure (21.7% vs 11.5%). CONCLUSION: These updated data provide robust evidence that transfusion of convalescent plasma is safe in hospitalized patients with COVID-19, and support the notion that earlier administration of plasma within the clinical course of COVID-19 is more likely to reduce mortality.

Early safety indicators of COVID-19 convalescent plasma in 5000 patients.

BACKGROUNDConvalescent plasma is the only antibody-based therapy currently available for patients with coronavirus disease 2019 (COVID-19). It has robust historical precedence and sound biological plausibility. Although promising, convalescent plasma has not yet been shown to be safe as a treatment for COVID-19.METHODSThus, we analyzed key safety metrics after transfusion of ABO-compatible human COVID-19 convalescent plasma in 5000 hospitalized adults with severe or life-threatening COVID-19, with 66% in the intensive care unit, as part of the US FDA expanded access program for COVID-19 convalescent plasma.RESULTSThe incidence of all serious adverse events (SAEs), including mortality rate (0.3%), in the first 4 hours after transfusion was <1%. Of the 36 reported SAEs, there were 25 reported incidences of related SAEs, including mortality (n = 4), transfusion-associated circulatory overload (n = 7), transfusion-related acute lung injury (n = 11), and severe allergic transfusion reactions (n = 3). However, only 2 of 36 SAEs were judged as definitely related to the convalescent plasma transfusion by the treating physician. The 7-day mortality rate was 14.9%.CONCLUSIONGiven the deadly nature of COVID-19 and the large population of critically ill patients included in these analyses, the mortality rate does not appear excessive. These early indicators suggest that transfusion of convalescent plasma is safe in hospitalized patients with COVID-19.TRIAL REGISTRATIONClinicalTrials.gov NCT04338360.FUNDINGMayo Clinic, Biomedical Advanced Research and Development Authority (75A50120C00096), National Center for Advancing Translational Sciences (UL1TR002377), National Heart, Lung, and Blood Institute (5R35HL139854 and R01 HL059842), National Institute of Diabetes and Digestive and Kidney Diseases (5T32DK07352), Natural Sciences and Engineering Research Council of Canada (PDF-532926-2019), National Institute of Allergy and Infectious Disease (R21 AI145356, R21 AI152318, and AI152078), Schwab Charitable Fund, United Health Group, National Basketball Association, Millennium Pharmaceuticals, and Octapharma USA Inc.

Sex differences in youth elite swimming.

BACKGROUND: The timing and magnitude of sex differences in athletic performance during early human development, prior to adulthood, is unknown. OBJECTIVE: To compare swimming velocity of boys and girls for all Olympic-length freestyle swimming events to determine the age of divergence in swimming performance. METHODS: We collected the all-time top 100 U.S. freestyle swimming performance times of boys and girls age 5 to 18 years for the 50m to 1500m events. RESULTS: Swimming performance improved with increasing age for boys and girls (p<0.001) until reaching a plateau, which initiated at a younger age for girls (15 years) than boys (17 years; sex×age; p<0.001). Prior to age 10, the top 5 swimming records for girls were 3% faster than the top boys (p<0.001). For the 10th-50th places, however, there were no sex-related differences in swimming performance prior to age 10 (p = 0.227). For both the top 5 and 10th-50th places, the sex difference in performance increased from age 10 (top 5, 2.5%; 10th-50th places, 1.0%) until age 17 (top 5, 7.6%; 10th-50th places, 8.0%). For all places, the sex difference in performance at age 18 was larger for sprint events (9.6%; 50-200m) than endurance events (7.1%; 400-1500m; p<0.001). Additionally, the sex-related difference in performance increased across age and US ranking from 2.4% for 1st place to 4.3% for 100th place (p<0.001), indicating less depth of performance in girls than boys. However, annual participation was ~20% higher in girls than boys for all ages (p<0.001). CONCLUSION: The top 5 girls demonstrated faster swimming velocities and the 10th-50th place girls demonstrated similar swimming velocities than boys (until ~10 years). After age 10, however, boys demonstrated increasingly faster swimming velocities than girls until 17 years. Collectively, these data suggest girls are faster, or at least not slower, than boys prior to the performance-enhancing effects of puberty.