Development and validation of a deep learning algorithm for the prediction of serum creatinine in critically ill patients.

Objectives: Serum creatinine (SCr) is the primary biomarker for assessing kidney function; however, it may lag behind true kidney function, especially in instances of acute kidney injury (AKI). The objective of the work is to develop Nephrocast, a deep-learning model to predict next-day SCr in adult patients treated in the intensive care unit (ICU). Materials and Methods: Nephrocast was trained and validated, temporally and prospectively, using electronic health record data of adult patients admitted to the ICU in the University of California San Diego Health (UCSDH) between January 1, 2016 and June 22, 2024. The model features consisted of demographics, comorbidities, vital signs and laboratory measurements, and medications. Model performance was evaluated by mean absolute error (MAE) and root-mean-square error (RMSE) and compared against the prediction day's SCr as a reference. Results: A total of 28 191 encounters met the eligibility criteria, corresponding to 105 718 patient-days. The median (interquartile range [IQR]) MAE and RMSE in the internal test set were 0.09 (0.085-0.09) mg/dL and 0.15 (0.146-0.152) mg/dL, respectively. In the prospective validation, the MAE and RMSE were 0.09 mg/dL and 0.14 mg/dL, respectively. The model's performance was superior to the reference SCr. Discussion and Conclusion: Our model demonstrated good performance in predicting next-day SCr by leveraging clinical data routinely collected in the ICU. The model could aid clinicians in in identifying high-risk patients for AKI, predicting AKI trajectory, and informing the dosing of renally eliminated drugs.

The impact of laboratory data missingness on sepsis diagnosis timeliness.

Objective: To investigate the impact of missing laboratory measurements on sepsis diagnostic delays. Materials and Methods: In adult patients admitted to 2 University of California San Diego (UCSD) hospitals from January 1, 2021 to June 30, 2024, we evaluated the relative time of organ failure (T OF) and time of clinical suspicion of sepsis (T suspicion) in patients with sepsis according to the Centers for Medicare & Medicaid Services (CMS) definition. Results: Of the patients studied, 48.7% (n = 2017) in the emergency department (ED), 30.8% (n = 209) in the wards, and 14.4% (n = 167) in the intensive care unit (ICU) had T OF after T suspicion. Patients with T OF after T suspicion had significantly higher data missingness of 1 or more of the 5 laboratory components used to determine organ failure. The mean number of missing labs was 4.23 vs 2.83 in the ED, 4.04 vs 3.38 in the wards, and 3.98 vs 3.19 in the ICU. Discussion: Our study identified many sepsis patients with missing laboratory results vital for the identification of organ failure and the diagnosis of sepsis at or before the time of clinical suspicion of sepsis. Addressing data missingness via more timely laboratory assessment could precipitate an earlier recognition of organ failure and potentially earlier diagnosis of and treatment initiation for sepsis. Conclusions: More prompt laboratory assessment might improve the timeliness of sepsis recognition and treatment.

Impact of wearable device data and multi-scale entropy analysis on improving hospital readmission prediction.

OBJECTIVE: Unplanned readmissions following a hospitalization remain common despite significant efforts to curtail these. Wearable devices may offer help identify patients at high risk for an unplanned readmission. MATERIALS AND METHODS: We conducted a multi-center retrospective cohort study using data from the All of Us data repository. We included subjects with wearable data and developed a baseline Feedforward Neural Network (FNN) model and a Long Short-Term Memory (LSTM) time-series deep learning model to predict daily, unplanned rehospitalizations up to 90 days from discharge. In addition to demographic and laboratory data from subjects, post-discharge data input features include wearable data and multiscale entropy features based on intraday wearable time series. The most significant features in the LSTM model were determined by permutation feature importance testing. RESULTS: In sum, 612 patients met inclusion criteria. The complete LSTM model had a higher area under the receiver operating characteristic curve than the FNN model (0.83 vs 0.795). The 5 most important input features included variables from multiscale entropy (steps) and number of active steps per day. DISCUSSION: Data available from wearable devices can improve ability to predict readmissions. Prior work has focused on predictors available up to discharge or on additional data abstracted from wearable devices. Our results from 35 institutions highlight how multiscale entropy can improve readmission prediction and may impact future work in this domain. CONCLUSION: Wearable data and multiscale entropy can improve prediction of a deep-learning model to predict unplanned 90-day readmissions. Prospective studies are needed to validate these findings.

Development and Validation of a Deep Learning Model for Prediction of Adult Physiological Deterioration.

BACKGROUND: Prediction-based strategies for physiologic deterioration offer the potential for earlier clinical interventions that improve patient outcomes. Current strategies are limited because they operate on inconsistent definitions of deterioration, attempt to dichotomize a dynamic and progressive phenomenon, and offer poor performance. OBJECTIVE: Can a deep learning deterioration prediction model (Deep Learning Enhanced Triage and Emergency Response for Inpatient Optimization [DETERIO]) based on a consensus definition of deterioration (the Adult Inpatient Decompensation Event [AIDE] criteria) and that approaches deterioration as a state "value-estimation" problem outperform a commercially available deterioration score? DERIVATION COHORT: The derivation cohort contained retrospective patient data collected from both inpatient services (inpatient) and emergency departments (EDs) of two hospitals within the University of California San Diego Health System. There were 330,729 total patients; 71,735 were inpatient and 258,994 were ED. Of these data, 20% were randomly sampled as a retrospective "testing set." VALIDATION COHORT: The validation cohort contained temporal patient data. There were 65,898 total patients; 13,750 were inpatient and 52,148 were ED. PREDICTION MODEL: DETERIO was developed and validated on these data, using the AIDE criteria to generate a composite score. DETERIO's architecture builds upon previous work. DETERIO's prediction performance up to 12 hours before T0 was compared against Epic Deterioration Index (EDI). RESULTS: In the retrospective testing set, DETERIO's area under the receiver operating characteristic curve (AUC) was 0.797 and 0.874 for inpatient and ED subsets, respectively. In the temporal validation cohort, the corresponding AUC were 0.775 and 0.856, respectively. DETERIO outperformed EDI in the inpatient validation cohort (AUC, 0.775 vs. 0.721; p < 0.01) while maintaining superior sensitivity and a comparable rate of false alarms (sensitivity, 45.50% vs. 30.00%; positive predictive value, 20.50% vs. 16.11%). CONCLUSIONS: DETERIO demonstrates promise in the viability of a state value-estimation approach for predicting adult physiologic deterioration. It may outperform EDI while offering additional clinical utility in triage and clinician interaction with prediction confidence and explanations. Additional studies are needed to assess generalizability and real-world clinical impact.

Voice-Enabled Response Analysis Agent (VERAA): Leveraging Large Language Models to Map Voice Responses in SDoH Survey.

Social Determinants of Health (SDoH) have been shown to have profound impacts on health-related outcomes, yet this data suffers from high rates of missingness in electronic health records (EHR). Moreover, limited English proficiency in the United States can be a barrier to communication with health care providers. In this study, we have designed a multilingual conversational agent capable of conducting SDoH surveys for use in healthcare environments. The agent asks questions in the patient's native language, translates responses into English, and subsequently maps these responses via a large language model (LLM) to structured options in a SDoH survey. This tool can be extended to a variety of survey instruments in either hospital or home settings, enabling the extraction of structured insights from free-text answers. The proposed approach heralds a shift towards more inclusive and insightful data collection, marking a significant stride in SDoH data enrichment for optimizing health outcome predictions and interventions.

Prediction of Readmission Following Sepsis Using Social Determinants of Health.

OBJECTIVES: To determine the predictive value of social determinants of health (SDoH) variables on 30-day readmission following a sepsis hospitalization as compared with traditional clinical variables. DESIGN: Multicenter retrospective cohort study using patient-level data, including demographic, clinical, and survey data. SETTINGS: Thirty-five hospitals across the United States from 2017 to 2021. PATIENTS: Two hundred seventy-one thousand four hundred twenty-eight individuals in the AllofUs initiative, of which 8909 had an index sepsis hospitalization. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Unplanned 30-day readmission to the hospital. Multinomial logistic regression models were constructed to account for survival in determination of variables associate with 30-day readmission and are presented as adjusted odds rations (aORs). Of the 8909 sepsis patients in our cohort, 21% had an unplanned hospital readmission within 30 days. Median age (interquartile range) was 54 years (41-65 yr), 4762 (53.4%) were female, and there were self-reported 1612 (18.09%) Black, 2271 (25.49%) Hispanic, and 4642 (52.1%) White individuals. In multinomial logistic regression models accounting for survival, we identified that change to nonphysician provider type due to economic reasons (aOR, 2.55 [2.35-2.74]), delay of receiving medical care due to lack of transportation (aOR, 1.68 [1.62-1.74]), and inability to afford flow-up care (aOR, 1.59 [1.52-1.66]) were strongly and independently associated with a 30-day readmission when adjusting for survival. Patients who lived in a ZIP code with a high percentage of patients in poverty and without health insurance were also more likely to be readmitted within 30 days (aOR, 1.26 [1.22-1.29] and aOR, 1.28 [1.26-1.29], respectively). Finally, we found that having a primary care provider and health insurance were associated with low odds of an unplanned 30-day readmission. CONCLUSIONS: In this multicenter retrospective cohort, several SDoH variables were strongly associated with unplanned 30-day readmission. Models predicting readmission following sepsis hospitalization may benefit from the addition of SDoH factors to traditional clinical variables.

Impact of a deep learning sepsis prediction model on quality of care and survival.

Sepsis remains a major cause of mortality and morbidity worldwide. Algorithms that assist with the early recognition of sepsis may improve outcomes, but relatively few studies have examined their impact on real-world patient outcomes. Our objective was to assess the impact of a deep-learning model (COMPOSER) for the early prediction of sepsis on patient outcomes. We completed a before-and-after quasi-experimental study at two distinct Emergency Departments (EDs) within the UC San Diego Health System. We included 6217 adult septic patients from 1/1/2021 through 4/30/2023. The exposure tested was a nurse-facing Best Practice Advisory (BPA) triggered by COMPOSER. In-hospital mortality, sepsis bundle compliance, 72-h change in sequential organ failure assessment (SOFA) score following sepsis onset, ICU-free days, and the number of ICU encounters were evaluated in the pre-intervention period (705 days) and the post-intervention period (145 days). The causal impact analysis was performed using a Bayesian structural time-series approach with confounder adjustments to assess the significance of the exposure at the 95% confidence level. The deployment of COMPOSER was significantly associated with a 1.9% absolute reduction (17% relative decrease) in in-hospital sepsis mortality (95% CI, 0.3%-3.5%), a 5.0% absolute increase (10% relative increase) in sepsis bundle compliance (95% CI, 2.4%-8.0%), and a 4% (95% CI, 1.1%-7.1%) reduction in 72-h SOFA change after sepsis onset in causal inference analysis. This study suggests that the deployment of COMPOSER for early prediction of sepsis was associated with a significant reduction in mortality and a significant increase in sepsis bundle compliance.

Improving long COVID-related text classification: a novel end-to-end domain-adaptive paraphrasing framework.

The emergence of long COVID during the ongoing COVID-19 pandemic has presented considerable challenges for healthcare professionals and researchers. The task of identifying relevant literature is particularly daunting due to the rapidly evolving scientific landscape, inconsistent definitions, and a lack of standardized nomenclature. This paper proposes a novel solution to this challenge by employing machine learning techniques to classify long COVID literature. However, the scarcity of annotated data for machine learning poses a significant obstacle. To overcome this, we introduce a strategy called medical paraphrasing, which diversifies the training data while maintaining the original content. Additionally, we propose a Data-Reweighting-Based Multi-Level Optimization Framework for Domain Adaptive Paraphrasing, supported by a Meta-Weight-Network (MWN). This innovative approach incorporates feedback from the downstream text classification model to influence the training of the paraphrasing model. During the training process, the framework assigns higher weights to the training examples that contribute more effectively to the downstream task of long COVID text classification. Our findings demonstrate that this method substantially improves the accuracy and efficiency of long COVID literature classification, offering a valuable tool for physicians and researchers navigating this complex and ever-evolving field.

Intravenous immunoglobulin resistance in Kawasaki disease patients: prediction using clinical data.

BACKGROUND: About 10-20% of Kawasaki disease (KD) patients are resistant to the initial infusion of intravenous immunoglobin (IVIG). The aim of this study was to assess whether IVIG resistance in KD patients could be predicted using standard clinical and laboratory features. METHODS: Data were from two cohorts: a Korean cohort of 7101 KD patients from 2015 to 2017 and a cohort of 649 KD patients from San Diego enrolled from 1998 to 2021. Features included laboratory values, the worst Z-score from the initial echocardiogram or during hospitalization, and the five clinical KD signs at presentation. RESULTS: Five machine learning models achieved a maximum median AUC of 0.711 [IQR: 0.706-0.72] in the Korean cohort and 0.696 [IQR: 0.609-0.722] in the San Diego cohort during stratified 10-fold cross-validation using significant laboratory features identified from univariate analysis. Adding the Z-score, KD clinical signs, or both did not considerably improve the median AUC in either cohort. CONCLUSIONS: Using commonly measured clinical laboratory data alone or in conjunction with echocardiographic findings and clinical features is not sufficient to predict IVIG resistance. Further attempts to predict IVIG resistance will need to incorporate additional data such as transcriptomics, proteomics, and genetics to achieve meaningful predictive utility. IMPACT: We demonstrated that laboratory, echocardiographic, and clinical findings cannot predict intravenous immunoglobin (IVIG) resistance to a clinically meaningful extent using machine learning in a homogenous Asian or ethnically diverse population of patients with Kawasaki disease (KD). Visualizing these features using uniform manifold approximation and projection (UMAP) is an important step to evaluate predictive utility in a qualitative manner. Further attempts to predict IVIG resistance in KD patients will need to incorporate novel biomarkers or other specialized features such as genetic differences or transcriptomics to be clinically useful.

Unsupervised Detection and Correction of Model Calibration Shift at Test-Time.

The wide adoption of predictive models into clinical practice require generalizability across hospitals and maintenance of consistent performance across time. Model calibration shift, caused by factors such as changes in prevalence rates or data distribution shift, can affect the generalizability of such models. In this work, we propose a model calibration detection and correction (CaDC) method, specifically designed to utilize only unlabeled data at a target hospital. The proposed method is very flexible and can be used alongside any deep learning-based clinical predictive model. As a case study, we focus on the problem of detecting and correcting model calibration shift in the context of early prediction of sepsis. Three patient cohorts consisting of 545,089 adult patients admitted to the emergency departments at three geographically diverse healthcare systems in the United States were used to train and externally validate the proposed method. We successfully show that utilizing the CaDC model can help assist the sepsis prediction model in achieving a predefined positive predictive value (PPV). For instance, when trained to achieve a PPV of 20%, the performance of the sepsis prediction model with and without the calibration shift estimation model was 18.0% vs 12.9% and 23.1% vs 13.4% at the two external validation cohorts, respectively. As such, the proposed CaDC method has potential applications in maintaining performance claims of predictive models deployed across hospital systems.Clinical relevance- Model generalizability is a requirement of wider adoption of clinical predictive models.

A Deep Learning Framework for Image-Based Screening of Kawasaki Disease.

Kawasaki disease (KD) is a leading cause of acquired heart disease in children and is characterized by the presence of a combination of five clinical signs assessed during the physical examination. Timely treatment of intravenous immunoglobin is needed to prevent coronary artery aneurysm formation, but KD is usually diagnosed when pediatric patients are evaluated by a clinician in the emergency department days after onset. One or more of the five clinical signs usually manifests in pediatric patients prior to ED admission, presenting an opportunity for earlier intervention if families receive guidance to seek medical care as soon as clinical signs are observed along with a fever for at least five days. We present a deep learning framework for a novel screening tool to calculate the relative risk of KD by analyzing images of the five clinical signs. The framework consists of convolutional neural networks to separately calculate the risk for each clinical sign, and a new algorithm to determine what clinical sign is in an image. We achieved a mean accuracy of 90% during 10-fold cross-validation and 88% during external validation for the new algorithm. These results demonstrate the algorithms in the proposed screening tool can be utilized by families to determine if their child should be evaluated by a clinician based on the number of clinical signs consistent with KD.Clinical Relevance- This screening framework has the potential for earlier clinical evaluation and detection of KD to reduce the risk of coronary artery complications.

Development & Deployment of a Real-time Healthcare Predictive Analytics Platform.

The deployment of predictive analytic algorithms that can safely and seamlessly integrate into existing healthcare workflows remains a significant challenge. Here, we present a scalable, cloud-based, fault-tolerant platform that is capable of extracting and processing electronic health record (EHR) data for any patient at any time following admission and transferring results back into the EHR. This platform has been successfully deployed within the UC San Diego Health system and utilizes interoperable data standards to enable portability.Clinical relevance- This platform is currently hosting a deep learning model for the early prediction of sepsis that is operational in two emergency departments.

Predicting Hospital Readmission among Patients with Sepsis Using Clinical and Wearable Data.

Sepsis is a life-threatening condition that occurs due to a dysregulated host response to infection. Recent data demonstrate that patients with sepsis have a significantly higher readmission risk than other common conditions, such as heart failure, pneumonia and myocardial infarction and associated economic burden. Prior studies have demonstrated an association between a patient's physical activity levels and readmission risk. In this study, we show that distribution of activity level prior and post-discharge among patients with sepsis are predictive of unplanned rehospitalization in 90 days (P-value<1e-3). Our preliminary results indicate that integrating Fitbit data with clinical measurements may improve model performance on predicting 90 days readmission.Clinical relevance Sepsis, Activity level, Hospital readmission, Wearable data.

Voice-Enabled Response Analysis Agent (VERAA): Leveraging Large Language Models to Map Voice Responses in SDoH Survey.

Social Determinants of Health (SDoH) have been shown to have profound impacts on health-related outcomes, yet this data suffers from high rates of missingness in electronic health records (EHR). Moreover, limited English proficiency in the United States can be a barrier to communication with health care providers. In this study, we have designed a multilingual conversational agent capable of conducting SDoH surveys for use in healthcare environments. The agent asks questions in the patient's native language, translates responses into English, and subsequently maps these responses via a large language model (LLM) to structured options in a SDoH survey. This tool can be extended to a variety of survey instruments in either hospital or home settings, enabling the extraction of structured insights from free-text answers. The proposed approach heralds a shift towards more inclusive and insightful data collection, marking a significant stride in SDoH data enrichment for optimizing health outcome predictions and interventions.

Making the Improbable Possible: Generalizing Models Designed for a Syndrome-Based, Heterogeneous Patient Landscape.

Syndromic conditions, such as sepsis, are commonly encountered in the intensive care unit. Although these conditions are easy for clinicians to grasp, these conditions may limit the performance of machine-learning algorithms. Individual hospital practice patterns may limit external generalizability. Data missingness is another barrier to optimal algorithm performance and various strategies exist to mitigate this. Recent advances in data science, such as transfer learning, conformal prediction, and continual learning, may improve generalizability of machine-learning algorithms in critically ill patients. Randomized trials with these approaches are indicated to demonstrate improvements in patient-centered outcomes at this point.

Representation Learning and Spectral Clustering for the Development and External Validation of Dynamic Sepsis Phenotypes: Observational Cohort Study.

BACKGROUND: Recent attempts at clinical phenotyping for sepsis have shown promise in identifying groups of patients with distinct treatment responses. Nonetheless, the replicability and actionability of these phenotypes remain an issue because the patient trajectory is a function of both the patient's physiological state and the interventions they receive. OBJECTIVE: We aimed to develop a novel approach for deriving clinical phenotypes using unsupervised learning and transition modeling. METHODS: Forty commonly used clinical variables from the electronic health record were used as inputs to a feed-forward neural network trained to predict the onset of sepsis. Using spectral clustering on the representations from this network, we derived and validated consistent phenotypes across a diverse cohort of patients with sepsis. We modeled phenotype dynamics as a Markov decision process with transitions as a function of the patient's current state and the interventions they received. RESULTS: Four consistent and distinct phenotypes were derived from over 11,500 adult patients who were admitted from the University of California, San Diego emergency department (ED) with sepsis between January 1, 2016, and January 31, 2020. Over 2000 adult patients admitted from the University of California, Irvine ED with sepsis between November 4, 2017, and August 4, 2022, were involved in the external validation. We demonstrate that sepsis phenotypes are not static and evolve in response to physiological factors and based on interventions. We show that roughly 45% of patients change phenotype membership within the first 6 hours of ED arrival. We observed consistent trends in patient dynamics as a function of interventions including early administration of antibiotics. CONCLUSIONS: We derived and describe 4 sepsis phenotypes present within 6 hours of triage in the ED. We observe that the administration of a 30 mL/kg fluid bolus may be associated with worse outcomes in certain phenotypes, whereas prompt antimicrobial therapy is associated with improved outcomes.

Development & Deployment of a Real-time Healthcare Predictive Analytics Platform.

The deployment of predictive analytic algorithms that can safely and seamlessly integrate into existing healthcare workflows remains a significant challenge. Here, we present a scalable, cloud-based, fault-tolerant platform that is capable of extracting and processing electronic health record (EHR) data for any patient at any time following admission and transferring results back into the EHR. This platform has been successfully deployed within the UC San Diego Health system and utilizes interoperable data standards to enable portability.

Predicting Hospital Readmission among Patients with Sepsis using Clinical and Wearable Data.

Sepsis is a life-threatening condition that occurs due to a dysregulated host response to infection. Recent data demonstrate that patients with sepsis have a significantly higher readmission risk than other common conditions, such as heart failure, pneumonia and myocardial infarction and associated economic burden. Prior studies have demonstrated an association between a patient's physical activity levels and readmission risk. In this study, we show that distribution of activity level prior and post-discharge among patients with sepsis are predictive of unplanned rehospitalization in 90 days (P-value<1e-3). Our preliminary results indicate that integrating Fitbit data with clinical measurements may improve model performance on predicting 90 days readmission.