Unlocking the Power of EHRs: Harnessing Unstructured Data for Machine Learning-based Outcome Predictions.

The integration of Electronic Health Records (EHRs) with Machine Learning (ML) models has become imperative in examining patient outcomes due to the vast amounts of clinical data they provide. However, critical information regarding social and behavioral factors that affect health, such as social isolation, stress, and mental health complexities, is often recorded in unstructured clinical notes, hindering its accessibility. This has resulted in an over-reliance on clinical data in current EHR-based research, potentially leading to disparities in health outcomes. This study aims to evaluate the impact of incorporating patient-specific context from unstructured EHR data on the accuracy and stability of ML algorithms for predicting mortality, using the MIMIC III database. Results from the study confirmed the significance of incorporating patient-specific information into prediction models, leading to a notable improvement in the discriminatory power and robustness of the ML algorithms. Furthermore, the findings underline the importance of considering non-clinical factors related to a patient's daily life, in addition to clinical factors, when making predictions about patient outcomes. The advent of advanced generative models, such as GPT-4, presents new opportunities for effectively extracting social and behavioral factors from unstructured clinical notes, further enhancing the accuracy and stability of ML algorithms in predicting patient outcomes. The results of our study have significant ramifications for improving ML in clinical decision support and patient outcome predictions, specifically highlighting the potential role of generative models like GPT-4 in advancing ML-based outcome predictions.

Evaluating progress in automatic chest X-ray radiology report generation.

Artificial intelligence (AI) models for automatic generation of narrative radiology reports from images have the potential to enhance efficiency and reduce the workload of radiologists. However, evaluating the correctness of these reports requires metrics that can capture clinically pertinent differences. In this study, we investigate the alignment between automated metrics and radiologists' scoring of errors in report generation. We address the limitations of existing metrics by proposing new metrics, RadGraph F1 and RadCliQ, which demonstrate stronger correlation with radiologists' evaluations. In addition, we analyze the failure modes of the metrics to understand their limitations and provide guidance for metric selection and interpretation. This study establishes RadGraph F1 and RadCliQ as meaningful metrics for guiding future research in radiology report generation.

Predicting patient decompensation from continuous physiologic monitoring in the emergency department.

Anticipation of clinical decompensation is essential for effective emergency and critical care. In this study, we develop a multimodal machine learning approach to predict the onset of new vital sign abnormalities (tachycardia, hypotension, hypoxia) in ED patients with normal initial vital signs. Our method combines standard triage data (vital signs, demographics, chief complaint) with features derived from a brief period of continuous physiologic monitoring, extracted via both conventional signal processing and transformer-based deep learning on ECG and PPG waveforms. We study 19,847 adult ED visits, divided into training (75%), validation (12.5%), and a chronologically sequential held-out test set (12.5%). The best-performing models use a combination of engineered and transformer-derived features, predicting in a 90-minute window new tachycardia with AUROC of 0.836 (95% CI, 0.800-0.870), new hypotension with AUROC 0.802 (95% CI, 0.747-0.856), and new hypoxia with AUROC 0.713 (95% CI, 0.680-0.745), in all cases significantly outperforming models using only standard triage data. Salient features include vital sign trends, PPG perfusion index, and ECG waveforms. This approach could improve the triage of apparently stable patients and be applied continuously for the prediction of near-term clinical deterioration.

Foundation models for generalist medical artificial intelligence.

The exceptionally rapid development of highly flexible, reusable artificial intelligence (AI) models is likely to usher in newfound capabilities in medicine. We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI). GMAI models will be capable of carrying out a diverse set of tasks using very little or no task-specific labelled data. Built through self-supervision on large, diverse datasets, GMAI will flexibly interpret different combinations of medical modalities, including data from imaging, electronic health records, laboratory results, genomics, graphs or medical text. Models will in turn produce expressive outputs such as free-text explanations, spoken recommendations or image annotations that demonstrate advanced medical reasoning abilities. Here we identify a set of high-impact potential applications for GMAI and lay out specific technical capabilities and training datasets necessary to enable them. We expect that GMAI-enabled applications will challenge current strategies for regulating and validating AI devices for medicine and will shift practices associated with the collection of large medical datasets.

Detecting Uncertainty of Mortality Prediction Using Confident Learning.

Early mortality prediction is an actively researched problem that has led to the development of various severity scores and machine learning (ML) models for accurate and reliable detection of mortality in severely ill patients staying in intensive care units (ICUs). However, the uncertainty of such predictions due to irregular patient sampling, missing information, or high diversity of patient data has not yet been adequately addressed. In this paper, we used confident learning (CL) to incorporate sample-uncertainty information into our mortality prediction models and evaluated the performance of these models using a large dataset of 139,367 unique ICU admissions within the eICU Collaborative Research Database (eICU-CRD). The results of our study validate the importance of uncertainty quantification in patient outcome prediction and show that the state-of-the-art ML models augmented with CL are more robust against epistemic error and class imbalance.

Predicting Discharge Destination of Critically Ill Patients Using Machine Learning.

Decision making about discharge destination for critically ill patients is a highly subjective and multidisciplinary process, heavily reliant on the ICU care team, patients and their caregivers' preferences, resource demand, staffing, and bed capacity. Timely identification of discharge disposition can be useful in care planning, and as a surrogate for functional status outcomes following critical illness. Although prior research has proposed methods to predict discharge destination in a critical care setting, they are limited in scope and in the generalizability of their findings. We proposed and implemented different machine learning architectures to determine the efficacy of the Acute Physiology and Chronic Health Evaluation (APACHE) IV score as well as the patient characteristics that comprise it to predict the discharge destination for critically ill patients within 24 hours of ICU admission. We conducted a retrospective study of ICU admissions within the eICU Collaborative Research Database (eICU-CRD) populated with de-identified clinical data from adult patients admitted to an ICU between 2014 and 2015. Machine learning models were developed to predict four discharge categories: death, home, nursing facility, and rehabilitation. These models were trained and tested on 115,248 unique ICU admissions. To mitigate class imbalance, we used synthetic minority over-sampling techniques. Hierarchical and ensemble classifiers were used to further study the impact of imbalanced testing set on the performance of our predictive models. Amongst all of the tested models, XGBoost provided the best discrimination performance with an area under the receiver operating characteristic curve of 90% (recall: 71%, F1: 70%). Our findings indicate that the variables used in the APACHE IV model for estimating patient severity of illness are better predictors of hospital discharge destination than the APACHE IV score alone. Incorporating these models into clinical decision support systems may assist patients, caregivers, and the ICU team to begin disposition planning as early as possible during the hospitalization.