Classifying Firearm Injury Intent in Electronic Hospital Records Using Natural Language Processing.

Importance: International Classification of Diseases-coded hospital discharge data do not accurately reflect whether firearm injuries were caused by assault, unintentional injury, self-harm, legal intervention, or were of undetermined intent. Applying natural language processing (NLP) and machine learning (ML) techniques to electronic health record (EHR) narrative text could be associated with improved accuracy of firearm injury intent data. Objective: To assess the accuracy with which an ML model identified firearm injury intent. Design, Setting, and Participants: A cross-sectional retrospective EHR review was conducted at 3 level I trauma centers, 2 from health care institutions in Boston, Massachusetts, and 1 from Seattle, Washington, between January 1, 2000, and December 31, 2019; data analysis was performed from January 18, 2021, to August 22, 2022. A total of 1915 incident cases of firearm injury in patients presenting to emergency departments at the model development institution and 769 from the external validation institution with a firearm injury code assigned according to International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) or International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Clinical Modification (ICD-10-CM), in discharge data were included. Exposures: Classification of firearm injury intent. Main Outcomes and Measures: Intent classification accuracy by the NLP model was compared with ICD codes assigned by medical record coders in discharge data. The NLP model extracted intent-relevant features from narrative text that were then used by a gradient-boosting classifier to determine the intent of each firearm injury. Classification accuracy was evaluated against intent assigned by the research team. The model was further validated using an external data set. Results: The NLP model was evaluated in 381 patients presenting with firearm injury at the model development site (mean [SD] age, 39.2 [13.0] years; 348 [91.3%] men) and 304 patients at the external development site (mean [SD] age, 31.8 [14.8] years; 263 [86.5%] men). The model proved more accurate than medical record coders in assigning intent to firearm injuries at the model development site (accident F-score, 0.78 vs 0.40; assault F-score, 0.90 vs 0.78). The model maintained this improvement on an external validation set from a second institution (accident F-score, 0.64 vs 0.58; assault F-score, 0.88 vs 0.81). While the model showed some degradation between institutions, retraining the model using data from the second institution further improved performance on that site's records (accident F-score, 0.75; assault F-score, 0.92). Conclusions and Relevance: The findings of this study suggest that NLP ML can be used to improve the accuracy of firearm injury intent classification compared with ICD-coded discharge data, particularly for cases of accident and assault intents (the most prevalent and commonly misclassified intent types). Future research could refine this model using larger and more diverse data sets.

Assessment of the Accuracy of Firearm Injury Intent Coding at 3 US Hospitals.

Importance: The absence of reliable hospital discharge data regarding the intent of firearm injuries (ie, whether caused by assault, accident, self-harm, legal intervention, or an act of unknown intent) has been characterized as a glaring gap in the US firearms data infrastructure. Objective: To use incident-level information to assess the accuracy of intent coding in hospital data used for firearm injury surveillance. Design, Setting, and Participants: This cross-sectional retrospective medical review study was conducted using case-level data from 3 level I US trauma centers (for 2008-2019) for patients presenting to the emergency department with an incident firearm injury of any severity. Exposures: Classification of firearm injury intent. Main Outcomes and Measures: Researchers reviewed electronic health records for all firearm injuries and compared intent adjudicated by team members (the gold standard) with International Classification of Diseases, Ninth and Tenth Revision, Clinical Modification (ICD-9-CM and ICD-10-CM) codes for firearm injury intent assigned by medical records coders (in discharge data) and by trauma registrars. Accuracy was assessed using intent-specific sensitivity and positive predictive value (PPV). Results: Of the 1227 cases of firearm injury incidents seen during the ICD-10-CM study period (October 1, 2015, to December 31, 2019), the majority of patients (1090 [88.8%]) were male and 547 (44.6%) were White. The research team adjudicated 837 (68.2%) to be assaults. Of these assault incidents, 234 (28.0%) were ICD coded as unintentional injuries in hospital discharge data. These miscoded patient cases largely accounted for why discharge data had low sensitivity for assaults (66.3%) and low PPV for unintentional injuries (34.3%). Misclassification was substantial even for patient cases described explicitly as assaults in clinical notes (sensitivity of 74.3%), as well as in the ICD-9-CM study period (sensitivity of 77.0% for assaults and PPV of 38.0% for unintentional firearm injuries). By contrast, intent coded by trauma registrars differed minimally from researcher-adjudicated intent (eg, sensitivity for assault of 96.0% and PPV for unintentional firearm injury of 93.0%). Conclusions and Relevance: The findings of this cross-sectional study underscore questions raised by prior work using aggregate count data regarding the accuracy of ICD-coded discharge data as a source of firearm injury intent. Based on our observations, researchers and policy makers should be aware that databases drawn from hospital discharge data (most notably, the Nationwide Emergency Department Sample) cannot be used to reliably count or characterize intent-specific firearm injuries.

PASCLex: A comprehensive post-acute sequelae of COVID-19 (PASC) symptom lexicon derived from electronic health record clinical notes.

OBJECTIVE: To develop a comprehensive post-acute sequelae of COVID-19 (PASC) symptom lexicon (PASCLex) from clinical notes to support PASC symptom identification and research. METHODS: We identified 26,117 COVID-19 positive patients from the Mass General Brigham's electronic health records (EHR) and extracted 328,879 clinical notes from their post-acute infection period (day 51-110 from first positive COVID-19 test). PASCLex incorporated Unified Medical Language System® (UMLS) Metathesaurus concepts and synonyms based on selected semantic types. The MTERMS natural language processing (NLP) tool was used to automatically extract symptoms from a development dataset. The lexicon was iteratively revised with manual chart review, keyword search, concept consolidation, and evaluation of NLP output. We assessed the comprehensiveness of PASCLex and the NLP performance using a validation dataset and reported the symptom prevalence across the entire corpus. RESULTS: PASCLex included 355 symptoms consolidated from 1520 UMLS concepts of 16,466 synonyms. NLP achieved an averaged precision of 0.94 and an estimated recall of 0.84. Symptoms with the highest frequency included pain (43.1%), anxiety (25.8%), depression (24.0%), fatigue (23.4%), joint pain (21.0%), shortness of breath (20.8%), headache (20.0%), nausea and/or vomiting (19.9%), myalgia (19.0%), and gastroesophageal reflux (18.6%). DISCUSSION AND CONCLUSION: PASC symptoms are diverse. A comprehensive lexicon of PASC symptoms can be derived using an ontology-driven, EHR-guided and NLP-assisted approach. By using unstructured data, this approach may improve identification and analysis of patient symptoms in the EHR, and inform prospective study design, preventative care strategies, and therapeutic interventions for patient care.

Using Natural Language Processing and Machine Learning to Identify Hospitalized Patients with Opioid Use Disorder.

Opioid use disorder (OUD) represents a global public health crisis that challenges classic clinical decision making. As existing hospital screening methods are resource-intensive, patients with OUD are significantly under-detected. An automated and accurate approach is needed to improve OUD identification so that appropriate care can be provided to these patients in a timely fashion. In this study, we used a large-scale clinical database from Mass General Brigham (MGB; formerly Partners HealthCare) to develop an OUD patient identification algorithm, using multiple machine learning methods. Working closely with an addiction psychiatrist, we developed a set of hand-crafted rules for identifying information suggestive of OUD from free-text clinical notes. We implemented a natural language processing (NLP)-based classification algorithm within the Medical Text Extraction, Reasoning and Mapping System (MTERMS) tool suite to automatically label patients as positive or negative for OUD based on these rules. We further used the NLP output as features to build multiple machine learning and a neural classifier. Our methods yielded robust performance for classifying hospitalized patients as positive or negative for OUD, with the best performing feature set and model combination achieving an F1 score of 0.97. These results show promise for the future development of a real-time tool for quickly and accurately identifying patients with OUD in the hospital setting.