Use of Generative AI to Identify Helmet Status Among Patients With Micromobility-Related Injuries From Unstructured Clinical Notes.

Importance: Large language models (LLMs) have potential to increase the efficiency of information extraction from unstructured clinical notes in electronic medical records. Objective: To assess the utility and reliability of an LLM, ChatGPT-4 (OpenAI), to analyze clinical narratives and identify helmet use status of patients injured in micromobility-related accidents. Design, Setting, and Participants: This cross-sectional study used publicly available, deidentified 2019 to 2022 data from the US Consumer Product Safety Commission's National Electronic Injury Surveillance System, a nationally representative stratified probability sample of 96 hospitals in the US. Unweighted estimates of e-bike, bicycle, hoverboard, and powered scooter-related injuries that resulted in an emergency department visit were used. Statistical analysis was performed from November 2023 to April 2024. Main Outcomes and Measures: Patient helmet status (wearing vs not wearing vs unknown) was extracted from clinical narratives using (1) a text string search using researcher-generated text strings and (2) the LLM by prompting the system with low-, intermediate-, and high-detail prompts. The level of agreement between the 2 approaches across all 3 prompts was analyzed using Cohen κ test statistics. Fleiss κ was calculated to measure the test-retest reliability of the high-detail prompt across 5 new chat sessions and days. Performance statistics were calculated by comparing results from the high-detail prompt to classifications of helmet status generated by researchers reading the clinical notes (ie, a criterion standard review). Results: Among 54 569 clinical notes, moderate (Cohen κ = 0.74 [95% CI, 0.73-0.75) and weak (Cohen κ = 0.53 [95% CI, 0.52-0.54]) agreement were found between the text string-search approach and the LLM for the low- and intermediate-detail prompts, respectively. The high-detail prompt had almost perfect agreement (κ = 1.00 [95% CI, 1.00-1.00]) but required the greatest amount of time to complete. The LLM did not perfectly replicate its analyses across new sessions and days (Fleiss κ = 0.91 across 5 trials; P < .001). The LLM often hallucinated and was consistent in replicating its hallucinations. It also showed high validity compared with the criterion standard (n = 400; κ = 0.98 [95% CI, 0.96-1.00]). Conclusions and Relevance: This study's findings suggest that although there are efficiency gains for using the LLM to extract information from clinical notes, the inadequate reliability compared with a text string-search approach, hallucinations, and inconsistent performance significantly hinder the potential of the currently available LLM.

Comparing alcohol involvement among injured pedalcycle and motorcycle riders across three national public-use datasets.

BACKGROUND: Annually since 2008; over 38% of fatally injured motorcycle riders and 20% of pedalcyclists involved in traffic crashes were under the influence of alcohol, yet public health surveillance of alcohol involvement in these injuries is underdeveloped. This study determined alcohol involvement among fatally and non-fatally injured pedalcycle and motorcycle riders and compared findings across three national public-use datasets. METHODS: Using the 2019 National Emergency Medical Services Information System (NEMSIS), the Fatality Analysis Reporting System (FARS), and National Electronic Injury Surveillance System (NEISS) datasets, we identified alcohol involvement in fatal and non-fatal injuries to pedalcycle and motorcycle riders (≥21 years). Alcohol involvement was positive based on the clinician's evaluation of alcohol at the scene (NEMSIS) or within the ED record (NEISS); or when Blood Alcohol Content (BAC) values were ≥.01 (FARS). Pedalcycle and motorcycle injuries were identified across datasets using: 1) ICD10 codes for pedalcycle (V10-V19) or motorcycle (V20-V29) within the cause of injury and EMS respondent's impression of the encounter variables (NEMSIS); 2) product codes for bicycles or moped/power-assisted cycle/minibike/two-wheeled, powered, off-road vehicles (NEISS); and 3) American National Standard Institute's classifications for pedalcycle and motorcycle in the person and vehicle type variables (FARS). The descriptive epidemiology was compared across datasets. RESULTS: There were 26,295 pedalcyclist and 50,122 motorcycle rider injuries resulting in an EMS response within NEMSIS data; 10.2% and 8.5% of these injuries respectively involved alcohol. These estimates were greater than the 7.3% of pedalcyclist and 6.1% of moped/power-assisted cycle/minibike/two-wheeled, powered, off-road vehicle injuries involving alcohol among patients who presented to an ED within the NEISS dataset. Based on FARS data, alcohol was involved in 27.0% of pedalcyclist and 42.0% of motorcyclist fatal injuries. Regardless of the data source, pedalcyclist and motorcycle fatal and non-fatal injuries were more likely to involve alcohol among middle-aged adults compared to older and early aged adults, and for men compared to women, with proportions that were generally 3-8% higher for men. CONCLUSIONS: Measures for pedalcycle and motorcycle injuries and alcohol involvement vary substantially across national public-use datasets. Standardized, valid, and feasible methods are needed to accurately inform injury prevention efforts.

Clinical trauma severity of indoor and outdoor injurious falls requiring emergency medical service response.

BACKGROUND: Injurious falls represent a significant public health burden. Research and policies have primarily focused on falls occurring indoors despite evidence that outdoor falls account for 47-58% of all falls requiring some medical attention. This study described the clinical trauma severity of indoor versus outdoor injurious falls requiring Emergency Medical Services (EMS) response. METHODS: Using the 2019 National Emergency Medical Services Information System (NEMSIS) dataset, we identified the location of patients injured from falls that required EMS response. We classified injury severity using (1) the Revised Trauma Score for Triage (T-RTS): ≤ 11 indicated the need for transport to a Trauma Center; (2) Glasgow Coma Scale (GCS): ≤ 8 and 9-12 indicated severe and moderate neurologic injury; and (3) patient clinical acuity by EMS: Dead, Critical, Emergent, Low. RESULTS: Of 1,854,909 encounters for patients with injurious falls, the vast majority occurred indoors (n = 1,596,860) compared to outdoors (n = 152,994). For patients who fell indoors vs outdoors on streets or sidewalks, the proportions were comparable for moderate or severe GCS scores (3.0% vs 3.9%), T-RTS scores indicating need for transport to a Trauma Center (5.2% vs 5.9%) and EMS acuity rated as Emergent or Critical (27.7% vs 27.1%). Injurious falls were more severe among male patients compared to females and males injured by falling on streets or sidewalks had higher percentages for moderate or severe GCS scores (5.2% vs 1.9%) and T-RTS scores indicating the need for transport to a Trauma Center (7.3% vs 3.9%) compared to falling indoors. Young and middle-aged patients who fell on streets or sidewalks had higher proportions for a T-RTS score indicating the need for Trauma Center care compared to those in this subgroup who fell indoors. Yet older patients injured by falling indoors were more likely to have a T-RTS score indicating the need for transport to a Trauma Center than older patients who fell on streets or sidewalks. CONCLUSIONS: There was a similar proportion of patients with severe injurious falls that occurred indoors and outdoors on streets or sidewalks. These findings suggest the need to determine outdoor environmental risks for outdoor falls to support location-specific interventions.

Clinical Trauma Severity of Indoor and Outdoor Injurious Falls Requiring Emergency Medical Service Response.

Background: Injurious falls represent a significant public health burden. Research and polices have primarily focused on falls occurring indoors despite evidence that outdoor falls account for 47-58% of all falls requiring some medical attention. This study compared the clinical trauma severity of indoor versus outdoor injurious falls requiring Emergency Medical Services (EMS) response. Methods: Using the 2019 National Emergency Medical Services Information System (NEMSIS) dataset, we identified the location of patients injured from falls that required EMS response. We classified injury severity using 1) the Revised Trauma Score for Triage (T-RTS): ≤ 11 indicated the need for transport to a Trauma Center; 2) Glasgow Coma Scale (GCS): ≤8 and 9-12 indicated moderate and severe neurologic injury; and 3) patient clinical acuity by EMS: Dead, Critical, Emergent, Low. Results: Of 1,854,909 encounters for patients with injurious falls, the vast majority occurred indoors (n=1,596,860) compared to outdoors (n=152,994). The proportions of patients with moderate or severe GCS scores, were comparable between those with indoor falls (3.0%) and with outdoor falls on streets or sidewalks (3.8%), T-RTS scores indicating need for transport to a Trauma Center (5.2% vs 5.9%) and EMS acuity rated as Emergent or Critical (27.7% vs 27.1%).Injurious falls were more severe among male patients compared to females: and males injured by falling on streets or sidewalks had higher percentages for moderate or severe GCS scores (4.8% vs 3.6%) and T-RTS scores indicating the need for transport to a Trauma Center (7.3% vs 6.5%) compared to indoor falls. Young and middle-aged patients whose injurious falls occurred on streets or sidewalks were more likely to have a T-RTS score indicating the need for Trauma Center care compared to indoor falls among this subgroup. Yet older patients injured by falling indoors were more likely to have a T-RTS score indicating the need for Trauma Center than older patients who fell on streets or sidewalks. Conclusions: There was a similar proportion of patients with severe injurious falls that occurred indoors and on streets or sidewalks. These findings suggest the need to determine outdoor environmental risks for outdoor falls to support location-specific interventions.

Prevalence of indications of alcohol and drug use among patients treated for injurious falls by Emergency Medical Services.

Objective: To describe the distribution of alcohol and drug involvement in injurious falls by location and subtype of fall. Methods: Using the 2019 National Emergency Medical Services Information System (NEMSIS) dataset we identified 1,854,909 patients injured from falls requiring an Emergency Medical Services (EMS) response and determined the fall location (e.g. indoors or on street/sidewalk) and the EMS clinician's notation of alcohol or drug involvement. We analyzed substance involvement by fall subtype, location of fall and patient demographics. Results: Overall, for 7.4% of injurious falls there was a notation of substance use: 6.5% for alcohol alone, 0.6% for drugs and 0.3% for alcohol and drugs. 21.2% of falls that occurred on a street or sidewalk had a notation of substance use; alcohol use alone for 18.5% of falls, drugs alone for 1.7% of falls and alcohol and drugs for 0.9% of falls. Substance use prevalence was highest, at 30.3%, in the age group 21 to 64 years, for falls occurring on streets and sidewalks, without syncope or heat illness as contributing factors; alcohol use alone for 26.3%, drugs alone for 2.6%, and alcohol and drugs for 1.4%. Reported substance use involvement was more frequent for men compared to women for each location type. Conclusions: Overall, 1-in-5 injurious falls on streets and sidewalks and requiring EMS attention involved substance use, and these numbers likely underestimate the true burden. As cities seek to expand nightlife districts, design strategies to protect pedestrians from falls should be enacted.