Evaluating Large Language Models for Drafting Emergency Department Discharge Summaries.

Importance: Large language models (LLMs) possess a range of capabilities which may be applied to the clinical domain, including text summarization. As ambient artificial intelligence scribes and other LLM-based tools begin to be deployed within healthcare settings, rigorous evaluations of the accuracy of these technologies are urgently needed. Objective: To investigate the performance of GPT-4 and GPT-3.5-turbo in generating Emergency Department (ED) discharge summaries and evaluate the prevalence and type of errors across each section of the discharge summary. Design: Cross-sectional study. Setting: University of California, San Francisco ED. Participants: We identified all adult ED visits from 2012 to 2023 with an ED clinician note. We randomly selected a sample of 100 ED visits for GPT-summarization. Exposure: We investigate the potential of two state-of-the-art LLMs, GPT-4 and GPT-3.5-turbo, to summarize the full ED clinician note into a discharge summary. Main Outcomes and Measures: GPT-3.5-turbo and GPT-4-generated discharge summaries were evaluated by two independent Emergency Medicine physician reviewers across three evaluation criteria: 1) Inaccuracy of GPT-summarized information; 2) Hallucination of information; 3) Omission of relevant clinical information. On identifying each error, reviewers were additionally asked to provide a brief explanation for their reasoning, which was manually classified into subgroups of errors. Results: From 202,059 eligible ED visits, we randomly sampled 100 for GPT-generated summarization and then expert-driven evaluation. In total, 33% of summaries generated by GPT-4 and 10% of those generated by GPT-3.5-turbo were entirely error-free across all evaluated domains. Summaries generated by GPT-4 were mostly accurate, with inaccuracies found in only 10% of cases, however, 42% of the summaries exhibited hallucinations and 47% omitted clinically relevant information. Inaccuracies and hallucinations were most commonly found in the Plan sections of GPT-generated summaries, while clinical omissions were concentrated in text describing patients' Physical Examination findings or History of Presenting Complaint. Conclusions and Relevance: In this cross-sectional study of 100 ED encounters, we found that LLMs could generate accurate discharge summaries, but were liable to hallucination and omission of clinically relevant information. A comprehensive understanding of the location and type of errors found in GPT-generated clinical text is important to facilitate clinician review of such content and prevent patient harm.

Outcomes of severely injured pregnant trauma patients: a multicenter analysis.

Nearly 10% of pregnant women suffer traumatic injury. Clinical outcomes for pregnant trauma patients (PTPs) with severe injuries have not been well studied. We sought to describe outcomes for PTPs presenting with severe injuries, hypothesizing that PTPs with severe injuries will have higher rates of complications and mortality compared to less injured PTPs. A post-hoc analysis of a multi-institutional retrospective study at 12 Level-I/II trauma centers was performed. Patients were stratified into severely injured (injury severity score [ISS] > 15) and not severely injured (ISS < 15) and compared with bivariate analyses. From 950 patients, 32 (3.4%) had severe injuries. Compared to non-severely injured PTPs, severely injured PTPs were of similar maternal age but had younger gestational age (21 vs 26 weeks, p = 0.009). Penetrating trauma was more common in the severely injured cohort (15.6% vs 1.4%, p < 0.001). The severely injured cohort more often underwent an operation (68.8% vs 3.8%, p < 0.001), including a hysterectomy (6.3% vs 0.3%, p < 0.001). The severely injured group had higher rates of complications (34.4% vs 0.9%, p < 0.001), mortality (15.6% vs 0.1%, p < 0.001), a higher rate of fetal delivery (37.5% vs. 6.0%, p < 0.001) and resuscitative hysterotomy (9.4% vs. 0%, p < 0.001). Only approximately 3% of PTPs were severely injured. However, severely injured PTPs had a nearly 40% rate of fetal delivery as well as increased complications and mortality. This included a resuscitative hysterotomy rate of nearly 10%. Significant vigilance must remain when caring for this population.

Predictors of fetal delivery in pregnant trauma patients: A multicenter study.

BACKGROUND: Pregnant trauma patients (PTPs) undergo observation and fetal monitoring following trauma due to possible fetal delivery (FD) or adverse outcome. There is a paucity of data on PTP outcomes, especially related to risk factors for FD. We aimed to identify predictors of posttraumatic FD in potentially viable pregnancies. METHODS: All PTPs (≥18 years) with ≥24-weeks gestational age were included in this multicenter retrospective study at 12 Level-I and II trauma centers between 2016 and 2021. Pregnant trauma patients who underwent FD ((+) FD) were compared to those who did not deliver ((-) FD) during the index hospitalization. Univariate analyses and multivariable logistic regression were performed to identify predictors of FD. RESULTS: Of 591 PTPs, 63 (10.7%) underwent FD, with 4 (6.3%) maternal deaths. The (+) FD group was similar in maternal age (27 vs. 28 years, p = 0.310) but had older gestational age (37 vs. 30 weeks, p < 0.001) and higher mean injury severity score (7.0 vs. 1.5, p < 0.001) compared with the (-) FD group. The (+) FD group had higher rates of vaginal bleeding (6.3% vs. 1.1%, p = 0.002), uterine contractions (46% vs. 23.5%, p < 0.001), and abnormal fetal heart tracing (54.7% vs. 14.6%, p < 0.001). On multivariate analysis, independent predictors for (+) FD included abdominal injury (odds ratio [OR], 4.07; confidence interval [CI], 1.11-15.02; p = 0.035), gestational age (OR, 1.68 per week ≥24 weeks; CI, 1.44-1.95; p < 0.001), abnormal FHT (OR, 12.72; CI, 5.19-31.17; p < 0.001), and premature rupture of membranes (OR, 35.97; CI, 7.28-177.74; p < 0.001). CONCLUSION: The FD rate was approximately 10% for PTPs with viable fetal gestational age. Independent risk factors for (+) FD included maternal and fetal factors, many of which are available on initial trauma bay evaluation. These risk factors may help predict FD in the trauma setting and shape future guidelines regarding the recommended observation of PTPs. LEVEL OF EVIDENCE: Therapeutic/Care Management; Level III.

Significant variation in computed tomography imaging of pregnant trauma patients: a retrospective multicenter study.

PURPOSE: Following motor vehicle collisions (MVCs), patients often undergo extensive computed tomography (CT) imaging. However, pregnant trauma patients (PTPs) represent a unique population where the risk of fetal radiation may supersede the benefits of liberal CT imaging. This study sought to evaluate imaging practices for PTPs, hypothesizing variability in CT imaging among trauma centers. If demonstrated, this might suggest the need to develop specific guidelines to standardize practice. METHODS: A multicenter retrospective study (2016-2021) was performed at 12 Level-I/II trauma centers. Adult (≥18 years old) PTPs involved in MVCs were included, with no patients excluded. The primary outcome was the frequency of CT. Chi-square tests were used to compare categorical variables, and ANOVA was used to compare the means of normally distributed continuous variables. RESULTS: A total of 729 PTPs sustained MVCs (73% at high speed of ≥ 25 miles per hour). Most patients were mildly injured but a small variation of injury severity score (range 1.1-4.6, p < 0.001) among centers was observed. There was a variation of imaging rates for CT head (range 11.8-62.5%, p < 0.001), cervical spine (11.8-75%, p < 0.001), chest (4.4-50.2%, p < 0.001), and abdomen/pelvis (0-57.3%, p < 0.001). In high-speed MVCs, there was variation for CT head (12.5-64.3%, p < 0.001), cervical spine (16.7-75%, p < 0.001), chest (5.9-83.3%, p < 0.001), and abdomen/pelvis (0-60%, p < 0.001). There was no difference in mortality (0-2.9%, p =0.19). CONCLUSION: Significant variability of CT imaging in PTPs after MVCs was demonstrated across 12 trauma centers, supporting the need for standardization of CT imaging for PTPs to reduce unnecessary radiation exposure while ensuring optimal injury identification is achieved.

Outcomes for advanced aged (35 and older) versus younger aged pregnant trauma patients: A multicenter study.

BACKGROUND: Effects of advanced maternal age (AMA) pregnancies (defined as ≥35 years) on pregnant trauma patients (PTPs) are unknown. This study compared AMA versus younger PTPs, hypothesizing AMA PTPs have increased risk of fetal delivery (FD). METHODS: A retrospective (2016-2021) multicenter study included all PTPs. Multivariable logistic regression was used to evaluate risk of FD after trauma. RESULTS: A total of 950 PTPs were included. Both cohorts had similar gestational age and injury severity scores. The AMA group had increased injuries to the pancreas, bladder, and stomach (p < 0.05). There was no difference in rate or associated risk of FD between cohorts (5.3% vs. 11.4%; OR 0.59, CI 0.19-1.88, p > 0.05). CONCLUSION: Compared to their younger counterparts, some intra-abdominal injuries (pancreas, bladder, and stomach) were more common among AMA PTPs. However, there was no difference in rate or associated risk of FD in AMA PTPs, thus they do not require increased observation.

Visual inspection with acetic acid screening for cervical cancer among women receiving anti-retroviral therapy for human immunodeficiency virus infection in northern Tanzania.

AIM: To evaluate visual inspection with acetic acid (VIA) screening for cervical cancer among human immunodeficiency virus (HIV)-positive patients in an East African community. METHODS: During a July 2018 cervical cancer screen-and-treat in Mwanza, Tanzania, participants were offered free cervical VIA screening, cryotherapy when indicated, and HIV testing. Acetowhite lesions and/or abnormal vascularity were designated VIA positive in accordance with current guidelines. The association between VIA results and HIV status was compared using Chi-square and Fisher exact tests. RESULTS: Eight hundred and twenty-four of 921 consented participants underwent VIA screening and 25.0% (n = 206) were VIA positive. VIA-positive nonpregnant women (n = 147) received cryotherapy and 15 (1.8%) with cancerous-appearing lesions were referred to Bugando Hospital. Sixty-six women were HIV-positive and included 25 diagnosed with HIV at the cervical cancer VIA screening and 41 with a prior diagnosis of HIV who were receiving antiretroviral therapy (ART) at the time of cervical cancer VIA screening. Sixty-four of these 66 patients, were screened with VIA. HIV infection was not associated with VIA findings. Abnormal VIA positive screening was observed in 20.3% (n = 13) of HIV-positive patients and in 24.4% (n = 145) of HIV-negative patients (p = 0.508). A nonsignificant trend of higher VIA positive screens among newly diagnosed HIV patients of 26.1% (n = 6) versus patients with preexisting HIV on ART of 17.1% (n = 7) was observed (p = 0.580). CONCLUSION: The unexpected lack of correlation between HIV infection and VIA positivity in a community with access to ART warrants additional research regarding the previously described role of ART in attenuating HPV-mediated neoplasia.

Implementation of human papillomavirus video education for women participating in mass cervical cancer screening in Tanzania.

BACKGROUND: Because the global disease burden of cervical cancer is greatest in Africa, the World Health Organization has endorsed visual inspection with acetic acid screening with cryotherapy triage for the screen-and-treat approach. With the lowest doctor-to-patient ratio worldwide (1:50,000), Tanzania has nearly 10,000 new cases of cervical cancer and 7000 deaths annually. OBJECTIVE: We report on the feasibility of visual inspection with acetic acid in the severely resource-limited Mwanza district and on the impact of intervening education on baseline human papillomavirus and cervical cancer knowledge. STUDY DESIGN: Two 5-day free visual inspection with acetic acid (VIA) clinics in urban Buzuruga and rural Sangabuye on the shores of Lake Victoria were approved by our university institutional review board and local Tanzanian health authorities. Participants completed a demographic survey and a 6-question (1 point per question) multiple choice test written in Kiswahili to assess baseline knowledge. A 15-minute educational video in Kiswahili (MedicalAidFilms: Understanding screening, treatment, and prevention of cervical cancer) was followed by repeated assessment using the same test, visual inspection with acetic acid screening, and optional HIV testing. Pre- and postvideo scores and change of score were analyzed via t test, analysis of variance, and multivariate regression. Significance was considered at P<.05. RESULTS: From July 2, 2018 to July 6, 2018, 825 women were screened, and 207 women (25.1%) were VIA positive (VIA+). One hundred forty-seven VIA+ nonpregnant women received same-day cryotherapy. Seven hundred sixty women participated in an educational intervention-61.6% of whom were from an urban site and 38.2% from a rural site. The mean age was 36.4 (standard deviation, 11.1). Primary languages were Kiswahili (62.2%) and Kisukuma (30.6%). Literacy was approximately 73%, and average education level was equivalent to the seventh grade (United States). Less than 20% of urban and rural women reported access to healthcare providers. Mean score of the participants before watching the video was 2.22 (standard deviation, 1.76) and was not different between VIA+ and VIA negative groups. Mean score of the participants after watching the video was 3.86 (standard deviation, 1.78). Postvideo scores significantly improved regardless of age group, clinic site, primary language, education level, literacy, or access to healthcare provider (P<.0001). Change of score after watching the video was significantly greater in participants from urban areas (1.99±2.07) than in those from rural areas (1.07±1.95) (P<.0001). Multivariate analysis identified urban site as an independent factor in change of score (P=.0211). CONCLUSION: Visual inspection with acetic acid screening for cervical cancer is feasible and accepted in northern Tanzania. Short video-based educational intervention improved baseline knowledge on the consequences of human papillomavirus infection in the studied populations. The impact was greater in the urban setting than in the rural setting.

Cervical cancer in Tanzania: A systematic review of current challenges in six domains.

Cervical cancer is the most common cancer in Tanzania. After excluding human immunodeficiency virus, lower respiratory infections, malaria, diarrheal diseases, and tuberculosis, cervical cancer kills more women than any other form of illness in the country. Unfortunately, Tanzania has a low doctor-to-patient ratio (1:50,000) and nearly 7000 women die each year from this disease. The clinical problem is further magnified by the country's lack of resources and prevailing poverty, sporadic cervical cancer screening, prevalence of high-risk oncogenic human papillomavirus subtypes, and relatively high rates of human immunodeficiency virus co-infection. In recent years, addressing the cervical cancer problem has become a priority for the Tanzanian government. In this systematic review of 39 peer-reviewed publications that appeared in the PubMed/MEDLINE (NCBI) database from 2013 to 2018, we synthesize the growing body of literature to capture current trends in Tanzania's evolving cervical cancer landscape. Six domains were identified, including risk factors, primary prevention, barriers to screening, treatment, healthcare worker education, and sustainability. In addition to traditional risk factors associated with sexual behavior, acetowhite changes observed during visual inspection of the cervix with acetic acid, lower education, rural setting, and HIV positivity also have a noteworthy clinical impact.