Extraction of Substance Use Information from Clinical Notes: A GPT-Based Investigation.

BACKGROUND: Understanding the multifaceted nature of health outcomes requires a comprehensive examination of the social, economic, and environmental determinants that shape individual well-being. Among these determinants, behavioral factors play a crucial role, particularly the consumption patterns of psychoactive substances, which have important implications on public health. The Global Burden of Disease Study shows a growing impact in disability-adjusted life years due to substance use. The successful identification of patients' substance use information equips clinical care teams to address substance-related issues more effectively, enabling targeted support and ultimately improving patient outcomes. OBJECTIVE: Traditional natural language processing (NLP) methods face limitations in accurately parsing diverse clinical language associated with substance use. Large Language Models (LLMs) offer promise in overcoming these challenges by adapting to diverse language patterns. This study investigates the application of the generative pre-trained transformer (GPT) model, in specific GPT-3.5- for extracting tobacco, alcohol, and substance use information from patient discharge summaries in zero-shot and few-shot learning settings. This study contributes to the evolving landscape of healthcare informatics by showcasing the potential of advanced language models in extracting nuanced information critical for enhancing patient care. METHODS: The main data source for analysis in this paper is Medical Information Mart for Intensive Care III (MIMIC-III) dataset. Among all notes in this dataset, we focused on discharge summaries. Prompt engineering was undertaken, involving an iterative exploration of diverse prompts. Leveraging carefully curated examples and refined prompts, we investigate the model's proficiency through zero-shot as well as few-shot prompting strategies. RESULTS: The presented results highlight the contrasting performance of GPT in extracting text span mentioning tobacco, alcohol, and substance use in both zero-shot and few-shot learning scenarios. In the zero-shot setting, the accuracy for extraction of tobacco, alcohol, and substance use information is notably high. However, in the few-shot setting, the accuracy diminishes significantly. On the contrary, few-shot learning led to significant increase in devising the status of substance use compared to zero-shot learning with significant increase in recall and F1-score. However, this improvement comes at the cost of a reduction in precision in extraction of not only the text span mentioning the use but also status of the use. CONCLUSIONS: Excellence of zero-shot learning in precisely extracting text span mentioning substance use demonstrates its effectiveness in situations where comprehensive recall is important. Conversely, few-shot learning offers advantages when accurately determining the status of substance use is the primary focus, even if it involves a trade-off in precision. The results contribute to enhancement of early detection and intervention strategies, tailor treatment plans with greater precision, and ultimately, contribute to a holistic understanding of patient health profiles. By integrating these AI-driven methods into electronic health record systems, clinicians can gain immediate, comprehensive insights into substance use that results in shaping interventions that are not only timely but also more personalized and effective.

NLP-Assisted Differential Diagnosis of Chronic Obstructive Pulmonary Disease Exacerbation.

Chronic Obstructive Pulmonary Disease (COPD) frequently coincides with other comorbidities such as congestive heart failure, hypertension, coronary artery disease, or atrial fibrillation. The exhibition of overlapping sets of symptoms associated with these conditions prevents early identification of an acute exacerbation upon admission to a hospital. Early identification of the underlying cause of exacerbation allows timely prescription of an optimal treatment plan as well as allows avoiding unnecessary clinical tests and specialist consultations. The aim of this study was to develop a predictive model for early identification of COPD exacerbation by using the clinical notes generated within 24 hours of admission to the hospital. The study cohort included patients with a prior diagnosis of COPD. Four predictive models have been developed, among which the support vector machine showed the best performance based on the resulting 80% F1 score.

Combining NLP and Machine Learning for Differential Diagnosis of COPD Exacerbation Using Emergency Room Data.

Chronic Obstructive Pulmonary Disease (COPD) exacerbation exhibits a set of overlapping symptoms with various forms of cardiovascular disease, which makes its early identification challenging. Timely identification of the underlying condition that caused acute admission of COPD patients in the emergency room (ER) may improve patient care and reduce care costs. This study aims to use machine learning combined with natural language processing (NLP) of ER notes to facilitate differential diagnosis in COPD patients admitted to ER. Using unstructured patient information extracted from the notes documented at the very first hours of admission to the hospital, four machine learning models were developed and tested. The random forest model demonstrated the best performance with F1 score of 93%.

Identification of Subphenotypes of Opioid Use Disorder Using Unsupervised Machine Learning.

This paper aimed to detect the latent clusters of patients with opioid use disorder and to identify the risk factors affecting drug misuse using unsupervised machine learning. The cluster with the highest proportion of successful treatment outcomes was characterized by the highest percentage of employment rate at admission and discharge, the highest percentage of patients who also recovered from alcohol and other drug co-use, and the highest proportion of patients who recovered from untreated health issues. Longer participation in opioid treatment programs was associated with the highest proportion of treatment success.

Using Electronic Medical Records and Clinical Notes to Predict the Outcome of Opioid Treatment Program.

Opioid addiction is a serious public health problem in the US, and this study aimed to explore how natural language processing (NLP) can be used to identify factors that contribute to distress in individuals with opioid addiction, and then use this information along with structured data to predict the outcome of opioid treatment programs (OTP). The study analyzed medical records data and clinical notes of 1,364 patients, out of which 136 succeeded in the program and 1,228 failed. The results showed that several factors influenced the success of patients in the program, including sex, race, education, employment, secondary substance, tobacco use, and type of residences. XGBoost with down sampling was the best model. The accuracy of the model was 0.71 and the AUC score was 0.64. The study highlights the importance of using both structured and unstructured data to evaluate the effectiveness of OTP.

Exercise Exertion Level Prediction Using Data from Wearable Physiologic Monitors.

This study aims to develop machine learning (ML) algorithms to predict exercise exertion levels using physiological parameters collected from wearable devices. Real-time ECG, oxygen saturation, pulse rate, and revolutions per minute (RPM) data were collected at three intensity levels during a 16-minute cycling exercise. Parallel to this, throughout each exercise session, the study subjects' ratings of perceived exertion (RPE) were gathered once per minute. Each 16-minute exercise session was divided into a total of eight 2-minute windows. Each exercise window was labeled as "high exertion," or "low exertion" classes based on the self-reported RPEs. For each window, the gathered ECG data were used to derive the heart rate variability (HRV) features in the temporal and frequency domains. Additionally, each window's averaged RPMs, heart rate, and oxygen saturation levels were calculated to form all the predictive features. The minimum redundancy maximum relevance algorithm was used to choose the best predictive features. Top selected features were then used to assess the accuracy of ten ML classifiers to predict the next window's exertion level. The k-nearest neighbors (KNN) model showed the highest accuracy of 85.7% and the highest F1 score of 83%. An ensemble model showed the highest area under the curve (AUC) of 0.92. The suggested method can be used to automatically track perceived exercise exertion in real-time.

NLP-Assisted Pipeline for COVID-19 Core Outcome Set Identification Using ClinicalTrials.gov.

Core outcome sets (COS) are necessary to ensure the systematic collection, metadata analysis and sharing the information across studies. However, development of an area-specific clinical research is costly and time consuming. ClinicalTrials.gov, as a public repository, provides access to a vast collection of clinical trials and their characteristics such as primary outcomes. With the growing number of COVID-19 clinical trials, identifying COSs from outcomes of such trials is crucial. This paper introduces a semi-automatic pipeline that can efficiently identify, aggregate and rank the COS from the primary outcomes of COVID-19 clinical trials. Using Natural language processing (NLP) techniques, our proposed pipeline successfully downloads and processes 5090 trials from all over the world and identifies COVID-19-specific outcomes that appeared in more than 1% of the trials. The top-of-the-list outcomes identified by the pipeline are mortality due to COVID-19, COVID-19 infection rate and COVID-19 symptoms.

Latent COVID-19 Clusters in Patients with Opioid Misuse.

The goal of this paper is to apply unsupervised machine learning techniques in order to discover latent clusters in patients who have opioid misuse and also undergone COVID-19 testing. Target dataset has been constructed based on COVID-19 testing results at Mount Sinai Health System and opioid treatment program (OTP) information from New York State Office of Addiction Service and Support (OASAS). The dataset was preprocessed using factor analysis for mixed data (FAMD) method and then K-means algorithm along with elbow method were used to determine the number of optimal clusters. Four patient clusters were identified among which the fourth cluster constituted the maximum percentage of positive COVID-19 test results (20%). Compared to the other clusters, this cluster has the highest percentage of African Americans. This cluster has also the highest mortality rate (16.52%), hospitalization rate after receiving the COVID-19 test result (72.17%, use of ventilator (7.83%) and ICU admission rate (47.83%). In addition, this cluster has the highest percentage of patients with at least one chronic disease (99.13%) and age-adjusted comorbidity score more than 1 (83.48%). Longer participation in OTP was associated with the highest morbidity and mortality from COVID-19.

Comparative Analysis of Patient Distress in Opioid Treatment Programs using Natural Language Processing.

Psychiatric and medical disorders, social and family environment, and legal distress are important determinants of distress that impact the effectiveness of the treatment in opioid treatment program (OTP). This information is not routinely captured in electronic health record, but may be found in clinical notes. This study aims to explore the feasibility and effectiveness of natural language processing (NLP) strategy for identifying legal, social, mental and medical determinates of distress along with emotional pain rooted in family environment from clinical narratives of patients with opioid addiction, and then using this information to find its impact on OTP outcomes. Analysis in this study showed that mental and legal distress significantly impact the result of the treatment in OTP.

Using Natural Language Processing of Clinical Notes to Predict Outcomes of Opioid Treatment Program.

Potential of natural language processing (NLP) in extracting patient's information from clinical notes of opioid treatment programs (OTP) and leveraging it in development of predictive models has not been fully explored. The goal of this study was to assess potential of NLP in identifying legal, social, mental, medical and family environment-based determinants of distress from clinical narratives of patients with opioid addiction, and then using this information in predicting OTP outcomes. Around 63% of patients reported improvements after completing OTP. We compared the results of logistics regression and random forest for predictive modeling. Random forest model performed slightly better than logistic regression (75% F1 score) with 74% accuracy. Clinical Relevance- Psychiatric and medical disorders, social, legal and family-based distress are important determinants of distress in patients enrolled in OTP. These information are often recorded in clinical notes. Extraction of this information and their utilization as features in machine learning models will lead to the enhancement of the performance of the OTP outcome predictive models.

Entity Extraction for Clinical Notes, a Comparison Between MetaMap and Amazon Comprehend Medical.

Extracting meaningful information from clinical notes is challenging due to their semi- or unstructured format. Clinical notes such as discharge summaries contain information about diseases, their risk factors, and treatment approaches associated to them. As such, it is critical for healthcare quality as well as for clinical research to extract those information and make them accessible to other computerized applications that rely on coded data. In this context, the goal of this paper is to compare the automatic medical entity extraction capacity of two available entity extraction tools: MetaMap (MM) and Amazon Comprehend Medical (ACM). Recall, precision and F-score have been used to evaluate the performance of the tools. The results show that ACM achieves higher average recall, average precision, and average F-score in comparison with MM.

Comparison of ACM and CLAMP for Entity Extraction in Clinical Notes.

Rapid increase in adoption of electronic health records in health care institutions has motivated the use of entity extraction tools to extract meaningful information from clinical notes with unstructured and narrative style. This paper investigates the performance of two such tools in automatic entity extraction. In specific, this work focuses on automatic medication extraction performance of Amazon Comprehend Medical (ACM) and Clinical Language Annotation, Modeling and Processing (CLAMP) toolkit using 2014 i2b2 NLP challenge dataset and its annotated medical entities. Recall, precision and F-score are used to evaluate the performance of the tools.Clinical Relevance- Majority of data in electronic health records (EHRs) are in the form of free text that features a gold mine of patient's information. While computerized applications in healthcare institutions as well as clinical research leverage structured data. As a result, information hidden in clinical free texts needs to be extracted and formatted as a structured data. This paper evaluates the performance of ACM and CLAMP in automatic entity extraction. The evaluation results show that CLAMP achieves an F-score of 91%, in comparison to an 87% F-score by ACM.