Voice-Enabled Response Analysis Agent (VERAA): Leveraging Large Language Models to Map Voice Responses in SDoH Survey.

Social Determinants of Health (SDoH) have been shown to have profound impacts on health-related outcomes, yet this data suffers from high rates of missingness in electronic health records (EHR). Moreover, limited English proficiency in the United States can be a barrier to communication with health care providers. In this study, we have designed a multilingual conversational agent capable of conducting SDoH surveys for use in healthcare environments. The agent asks questions in the patient's native language, translates responses into English, and subsequently maps these responses via a large language model (LLM) to structured options in a SDoH survey. This tool can be extended to a variety of survey instruments in either hospital or home settings, enabling the extraction of structured insights from free-text answers. The proposed approach heralds a shift towards more inclusive and insightful data collection, marking a significant stride in SDoH data enrichment for optimizing health outcome predictions and interventions.

Prediction of Readmission Following Sepsis Using Social Determinants of Health.

OBJECTIVES: To determine the predictive value of social determinants of health (SDoH) variables on 30-day readmission following a sepsis hospitalization as compared with traditional clinical variables. DESIGN: Multicenter retrospective cohort study using patient-level data, including demographic, clinical, and survey data. SETTINGS: Thirty-five hospitals across the United States from 2017 to 2021. PATIENTS: Two hundred seventy-one thousand four hundred twenty-eight individuals in the AllofUs initiative, of which 8909 had an index sepsis hospitalization. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Unplanned 30-day readmission to the hospital. Multinomial logistic regression models were constructed to account for survival in determination of variables associate with 30-day readmission and are presented as adjusted odds rations (aORs). Of the 8909 sepsis patients in our cohort, 21% had an unplanned hospital readmission within 30 days. Median age (interquartile range) was 54 years (41-65 yr), 4762 (53.4%) were female, and there were self-reported 1612 (18.09%) Black, 2271 (25.49%) Hispanic, and 4642 (52.1%) White individuals. In multinomial logistic regression models accounting for survival, we identified that change to nonphysician provider type due to economic reasons (aOR, 2.55 [2.35-2.74]), delay of receiving medical care due to lack of transportation (aOR, 1.68 [1.62-1.74]), and inability to afford flow-up care (aOR, 1.59 [1.52-1.66]) were strongly and independently associated with a 30-day readmission when adjusting for survival. Patients who lived in a ZIP code with a high percentage of patients in poverty and without health insurance were also more likely to be readmitted within 30 days (aOR, 1.26 [1.22-1.29] and aOR, 1.28 [1.26-1.29], respectively). Finally, we found that having a primary care provider and health insurance were associated with low odds of an unplanned 30-day readmission. CONCLUSIONS: In this multicenter retrospective cohort, several SDoH variables were strongly associated with unplanned 30-day readmission. Models predicting readmission following sepsis hospitalization may benefit from the addition of SDoH factors to traditional clinical variables.

Improving long COVID-related text classification: a novel end-to-end domain-adaptive paraphrasing framework.

The emergence of long COVID during the ongoing COVID-19 pandemic has presented considerable challenges for healthcare professionals and researchers. The task of identifying relevant literature is particularly daunting due to the rapidly evolving scientific landscape, inconsistent definitions, and a lack of standardized nomenclature. This paper proposes a novel solution to this challenge by employing machine learning techniques to classify long COVID literature. However, the scarcity of annotated data for machine learning poses a significant obstacle. To overcome this, we introduce a strategy called medical paraphrasing, which diversifies the training data while maintaining the original content. Additionally, we propose a Data-Reweighting-Based Multi-Level Optimization Framework for Domain Adaptive Paraphrasing, supported by a Meta-Weight-Network (MWN). This innovative approach incorporates feedback from the downstream text classification model to influence the training of the paraphrasing model. During the training process, the framework assigns higher weights to the training examples that contribute more effectively to the downstream task of long COVID text classification. Our findings demonstrate that this method substantially improves the accuracy and efficiency of long COVID literature classification, offering a valuable tool for physicians and researchers navigating this complex and ever-evolving field.

Impact of a deep learning sepsis prediction model on quality of care and survival.

Sepsis remains a major cause of mortality and morbidity worldwide. Algorithms that assist with the early recognition of sepsis may improve outcomes, but relatively few studies have examined their impact on real-world patient outcomes. Our objective was to assess the impact of a deep-learning model (COMPOSER) for the early prediction of sepsis on patient outcomes. We completed a before-and-after quasi-experimental study at two distinct Emergency Departments (EDs) within the UC San Diego Health System. We included 6217 adult septic patients from 1/1/2021 through 4/30/2023. The exposure tested was a nurse-facing Best Practice Advisory (BPA) triggered by COMPOSER. In-hospital mortality, sepsis bundle compliance, 72-h change in sequential organ failure assessment (SOFA) score following sepsis onset, ICU-free days, and the number of ICU encounters were evaluated in the pre-intervention period (705 days) and the post-intervention period (145 days). The causal impact analysis was performed using a Bayesian structural time-series approach with confounder adjustments to assess the significance of the exposure at the 95% confidence level. The deployment of COMPOSER was significantly associated with a 1.9% absolute reduction (17% relative decrease) in in-hospital sepsis mortality (95% CI, 0.3%-3.5%), a 5.0% absolute increase (10% relative increase) in sepsis bundle compliance (95% CI, 2.4%-8.0%), and a 4% (95% CI, 1.1%-7.1%) reduction in 72-h SOFA change after sepsis onset in causal inference analysis. This study suggests that the deployment of COMPOSER for early prediction of sepsis was associated with a significant reduction in mortality and a significant increase in sepsis bundle compliance.

Intravenous immunoglobulin resistance in Kawasaki disease patients: prediction using clinical data.

BACKGROUND: About 10-20% of Kawasaki disease (KD) patients are resistant to the initial infusion of intravenous immunoglobin (IVIG). The aim of this study was to assess whether IVIG resistance in KD patients could be predicted using standard clinical and laboratory features. METHODS: Data were from two cohorts: a Korean cohort of 7101 KD patients from 2015 to 2017 and a cohort of 649 KD patients from San Diego enrolled from 1998 to 2021. Features included laboratory values, the worst Z-score from the initial echocardiogram or during hospitalization, and the five clinical KD signs at presentation. RESULTS: Five machine learning models achieved a maximum median AUC of 0.711 [IQR: 0.706-0.72] in the Korean cohort and 0.696 [IQR: 0.609-0.722] in the San Diego cohort during stratified 10-fold cross-validation using significant laboratory features identified from univariate analysis. Adding the Z-score, KD clinical signs, or both did not considerably improve the median AUC in either cohort. CONCLUSIONS: Using commonly measured clinical laboratory data alone or in conjunction with echocardiographic findings and clinical features is not sufficient to predict IVIG resistance. Further attempts to predict IVIG resistance will need to incorporate additional data such as transcriptomics, proteomics, and genetics to achieve meaningful predictive utility. IMPACT: We demonstrated that laboratory, echocardiographic, and clinical findings cannot predict intravenous immunoglobin (IVIG) resistance to a clinically meaningful extent using machine learning in a homogenous Asian or ethnically diverse population of patients with Kawasaki disease (KD). Visualizing these features using uniform manifold approximation and projection (UMAP) is an important step to evaluate predictive utility in a qualitative manner. Further attempts to predict IVIG resistance in KD patients will need to incorporate novel biomarkers or other specialized features such as genetic differences or transcriptomics to be clinically useful.

Unsupervised Detection and Correction of Model Calibration Shift at Test-Time.

The wide adoption of predictive models into clinical practice require generalizability across hospitals and maintenance of consistent performance across time. Model calibration shift, caused by factors such as changes in prevalence rates or data distribution shift, can affect the generalizability of such models. In this work, we propose a model calibration detection and correction (CaDC) method, specifically designed to utilize only unlabeled data at a target hospital. The proposed method is very flexible and can be used alongside any deep learning-based clinical predictive model. As a case study, we focus on the problem of detecting and correcting model calibration shift in the context of early prediction of sepsis. Three patient cohorts consisting of 545,089 adult patients admitted to the emergency departments at three geographically diverse healthcare systems in the United States were used to train and externally validate the proposed method. We successfully show that utilizing the CaDC model can help assist the sepsis prediction model in achieving a predefined positive predictive value (PPV). For instance, when trained to achieve a PPV of 20%, the performance of the sepsis prediction model with and without the calibration shift estimation model was 18.0% vs 12.9% and 23.1% vs 13.4% at the two external validation cohorts, respectively. As such, the proposed CaDC method has potential applications in maintaining performance claims of predictive models deployed across hospital systems.Clinical relevance- Model generalizability is a requirement of wider adoption of clinical predictive models.

A Deep Learning Framework for Image-Based Screening of Kawasaki Disease.

Kawasaki disease (KD) is a leading cause of acquired heart disease in children and is characterized by the presence of a combination of five clinical signs assessed during the physical examination. Timely treatment of intravenous immunoglobin is needed to prevent coronary artery aneurysm formation, but KD is usually diagnosed when pediatric patients are evaluated by a clinician in the emergency department days after onset. One or more of the five clinical signs usually manifests in pediatric patients prior to ED admission, presenting an opportunity for earlier intervention if families receive guidance to seek medical care as soon as clinical signs are observed along with a fever for at least five days. We present a deep learning framework for a novel screening tool to calculate the relative risk of KD by analyzing images of the five clinical signs. The framework consists of convolutional neural networks to separately calculate the risk for each clinical sign, and a new algorithm to determine what clinical sign is in an image. We achieved a mean accuracy of 90% during 10-fold cross-validation and 88% during external validation for the new algorithm. These results demonstrate the algorithms in the proposed screening tool can be utilized by families to determine if their child should be evaluated by a clinician based on the number of clinical signs consistent with KD.Clinical Relevance- This screening framework has the potential for earlier clinical evaluation and detection of KD to reduce the risk of coronary artery complications.

Development & Deployment of a Real-time Healthcare Predictive Analytics Platform.

The deployment of predictive analytic algorithms that can safely and seamlessly integrate into existing healthcare workflows remains a significant challenge. Here, we present a scalable, cloud-based, fault-tolerant platform that is capable of extracting and processing electronic health record (EHR) data for any patient at any time following admission and transferring results back into the EHR. This platform has been successfully deployed within the UC San Diego Health system and utilizes interoperable data standards to enable portability.Clinical relevance- This platform is currently hosting a deep learning model for the early prediction of sepsis that is operational in two emergency departments.

Predicting Hospital Readmission among Patients with Sepsis Using Clinical and Wearable Data.

Sepsis is a life-threatening condition that occurs due to a dysregulated host response to infection. Recent data demonstrate that patients with sepsis have a significantly higher readmission risk than other common conditions, such as heart failure, pneumonia and myocardial infarction and associated economic burden. Prior studies have demonstrated an association between a patient's physical activity levels and readmission risk. In this study, we show that distribution of activity level prior and post-discharge among patients with sepsis are predictive of unplanned rehospitalization in 90 days (P-value<1e-3). Our preliminary results indicate that integrating Fitbit data with clinical measurements may improve model performance on predicting 90 days readmission.Clinical relevance Sepsis, Activity level, Hospital readmission, Wearable data.

Voice-Enabled Response Analysis Agent (VERAA): Leveraging Large Language Models to Map Voice Responses in SDoH Survey.

Social Determinants of Health (SDoH) have been shown to have profound impacts on health-related outcomes, yet this data suffers from high rates of missingness in electronic health records (EHR). Moreover, limited English proficiency in the United States can be a barrier to communication with health care providers. In this study, we have designed a multilingual conversational agent capable of conducting SDoH surveys for use in healthcare environments. The agent asks questions in the patient's native language, translates responses into English, and subsequently maps these responses via a large language model (LLM) to structured options in a SDoH survey. This tool can be extended to a variety of survey instruments in either hospital or home settings, enabling the extraction of structured insights from free-text answers. The proposed approach heralds a shift towards more inclusive and insightful data collection, marking a significant stride in SDoH data enrichment for optimizing health outcome predictions and interventions.

Making the Improbable Possible: Generalizing Models Designed for a Syndrome-Based, Heterogeneous Patient Landscape.

Syndromic conditions, such as sepsis, are commonly encountered in the intensive care unit. Although these conditions are easy for clinicians to grasp, these conditions may limit the performance of machine-learning algorithms. Individual hospital practice patterns may limit external generalizability. Data missingness is another barrier to optimal algorithm performance and various strategies exist to mitigate this. Recent advances in data science, such as transfer learning, conformal prediction, and continual learning, may improve generalizability of machine-learning algorithms in critically ill patients. Randomized trials with these approaches are indicated to demonstrate improvements in patient-centered outcomes at this point.

Representation Learning and Spectral Clustering for the Development and External Validation of Dynamic Sepsis Phenotypes: Observational Cohort Study.

BACKGROUND: Recent attempts at clinical phenotyping for sepsis have shown promise in identifying groups of patients with distinct treatment responses. Nonetheless, the replicability and actionability of these phenotypes remain an issue because the patient trajectory is a function of both the patient's physiological state and the interventions they receive. OBJECTIVE: We aimed to develop a novel approach for deriving clinical phenotypes using unsupervised learning and transition modeling. METHODS: Forty commonly used clinical variables from the electronic health record were used as inputs to a feed-forward neural network trained to predict the onset of sepsis. Using spectral clustering on the representations from this network, we derived and validated consistent phenotypes across a diverse cohort of patients with sepsis. We modeled phenotype dynamics as a Markov decision process with transitions as a function of the patient's current state and the interventions they received. RESULTS: Four consistent and distinct phenotypes were derived from over 11,500 adult patients who were admitted from the University of California, San Diego emergency department (ED) with sepsis between January 1, 2016, and January 31, 2020. Over 2000 adult patients admitted from the University of California, Irvine ED with sepsis between November 4, 2017, and August 4, 2022, were involved in the external validation. We demonstrate that sepsis phenotypes are not static and evolve in response to physiological factors and based on interventions. We show that roughly 45% of patients change phenotype membership within the first 6 hours of ED arrival. We observed consistent trends in patient dynamics as a function of interventions including early administration of antibiotics. CONCLUSIONS: We derived and describe 4 sepsis phenotypes present within 6 hours of triage in the ED. We observe that the administration of a 30 mL/kg fluid bolus may be associated with worse outcomes in certain phenotypes, whereas prompt antimicrobial therapy is associated with improved outcomes.

Development & Deployment of a Real-time Healthcare Predictive Analytics Platform.

The deployment of predictive analytic algorithms that can safely and seamlessly integrate into existing healthcare workflows remains a significant challenge. Here, we present a scalable, cloud-based, fault-tolerant platform that is capable of extracting and processing electronic health record (EHR) data for any patient at any time following admission and transferring results back into the EHR. This platform has been successfully deployed within the UC San Diego Health system and utilizes interoperable data standards to enable portability.

Predicting Hospital Readmission among Patients with Sepsis using Clinical and Wearable Data.

Sepsis is a life-threatening condition that occurs due to a dysregulated host response to infection. Recent data demonstrate that patients with sepsis have a significantly higher readmission risk than other common conditions, such as heart failure, pneumonia and myocardial infarction and associated economic burden. Prior studies have demonstrated an association between a patient's physical activity levels and readmission risk. In this study, we show that distribution of activity level prior and post-discharge among patients with sepsis are predictive of unplanned rehospitalization in 90 days (P-value<1e-3). Our preliminary results indicate that integrating Fitbit data with clinical measurements may improve model performance on predicting 90 days readmission.

Prediction of Unplanned Hospital Readmission using Clinical and Longitudinal Wearable Sensor Features.

Predictive models have been suggested as potential tools for identifying highest risk patients for hospital readmissions, in order to improve care coordination and ultimately long-term patient outcomes. However, the accuracy of current predictive models for readmission prediction is still moderate and further data enrichment is needed to identify at risk patients. This paper describes models to predict 90-day readmission, focusing on testing the predictive performance of wearable sensor features generated using multiscale entropy techniques and clinical features. Our study explores ways to incorporate pre-discharge and post-discharge wearable sensor features to make robust patient predictions. Data were used from participants enrolled in the AllofUs Research program. We extracted the inpatient cohort of patients and integrated clinical data from the electronic health records (EHR) and Fitbit sensor measurements. Entropy features were calculated from the longitudinal wearable sensor data, such as heart rate and mobility-related measurements, in order to characterize time series variability and complexity. Our best performing model acheived an AUC of 83%, and at 80% sensitivity acheived 75% specificity and 57% positive predictive value. Our results indicate that it would be possible to improve the ability to predict unplanned hospital readmissions by considering pre-discharge and post-discharge wearable features.

Differentiation of Fungal, Viral, and Bacterial Sepsis using Multimodal Deep Learning.

Sepsis is a major cause of morbidity and mortality worldwide, and is caused by bacterial infection in a majority of cases. However, fungal sepsis often carries a higher mortality rate both due to its prevalence in immunocompromised patients as well as delayed recognition. Using chest x-rays, associated radiology reports, and structured patient data from the MIMIC-IV clinical dataset, the authors present a machine learning methodology to differentiate between bacterial, fungal, and viral sepsis. Model performance shows AUCs of 0.81, 0.83, 0.79 for detecting bacterial, fungal, and viral sepsis respectively, with best performance achieved using embeddings from image reports and structured clinical data. By improving early detection of an often missed causative septic agent, predictive models could facilitate earlier treatment of non-bacterial sepsis with resultant associated mortality reduction.

Optimizing the Implementation of Clinical Predictive Models to Minimize National Costs: Sepsis Case Study.

BACKGROUND: Sepsis costs and incidence vary dramatically across diagnostic categories, warranting a customized approach for implementing predictive models. OBJECTIVE: The aim of this study was to optimize the parameters of a sepsis prediction model within distinct patient groups to minimize the excess cost of sepsis care and analyze the potential effect of factors contributing to end-user response to sepsis alerts on overall model utility. METHODS: We calculated the excess costs of sepsis to the Centers for Medicare and Medicaid Services (CMS) by comparing patients with and without a secondary sepsis diagnosis but with the same primary diagnosis and baseline comorbidities. We optimized the parameters of a sepsis prediction algorithm across different diagnostic categories to minimize these excess costs. At the optima, we evaluated diagnostic odds ratios and analyzed the impact of compliance factors such as noncompliance, treatment efficacy, and tolerance for false alarms on the net benefit of triggering sepsis alerts. RESULTS: Compliance factors significantly contributed to the net benefit of triggering a sepsis alert. However, a customized deployment policy can achieve a significantly higher diagnostic odds ratio and reduced costs of sepsis care. Implementing our optimization routine with powerful predictive models could result in US $4.6 billion in excess cost savings for CMS. CONCLUSIONS: We designed a framework for customizing sepsis alert protocols within different diagnostic categories to minimize excess costs and analyzed model performance as a function of false alarm tolerance and compliance with model recommendations. We provide a framework that CMS policymakers could use to recommend minimum adherence rates to the early recognition and appropriate care of sepsis that is sensitive to hospital department-level incidence rates and national excess costs. Customizing the implementation of clinical predictive models by accounting for various behavioral and economic factors may improve the practical benefit of predictive models.

Assessing Barriers to Implementation of Machine Learning and Artificial Intelligence-Based Tools in Critical Care: Web-Based Survey Study.

BACKGROUND: Although there is considerable interest in machine learning (ML) and artificial intelligence (AI) in critical care, the implementation of effective algorithms into practice has been limited. OBJECTIVE: We sought to understand physician perspectives of a novel intubation prediction tool. Further, we sought to understand health care provider and nonprovider perspectives on the use of ML in health care. We aim to use the data gathered to elucidate implementation barriers and determinants of this intubation prediction tool, as well as ML/AI-based algorithms in critical care and health care in general. METHODS: We developed 2 anonymous surveys in Qualtrics, 1 single-center survey distributed to 99 critical care physicians via email, and 1 social media survey distributed via Facebook and Twitter with branching logic to tailor questions for providers and nonproviders. The surveys included a mixture of categorical, Likert scale, and free-text items. Likert scale means with SD were reported from 1 to 5. We used student t tests to examine the differences between groups. In addition, Likert scale responses were converted into 3 categories, and percentage values were reported in order to demonstrate the distribution of responses. Qualitative free-text responses were reviewed by a member of the study team to determine validity, and content analysis was performed to determine common themes in responses. RESULTS: Out of 99 critical care physicians, 47 (48%) completed the single-center survey. Perceived knowledge of ML was low with a mean Likert score of 2.4 out of 5 (SD 0.96), with 7.5% of respondents rating their knowledge as a 4 or 5. The willingness to use the ML-based algorithm was 3.32 out of 5 (SD 0.95), with 75% of respondents answering 3 out of 5. The social media survey had 770 total responses with 605 (79%) providers and 165 (21%) nonproviders. We found no difference in providers' perceived knowledge based on level of experience in either survey. We found that nonproviders had significantly less perceived knowledge of ML (mean 3.04 out of 5, SD 1.53 vs mean 3.43, SD 0.941; P<.001) and comfort with ML (mean 3.28 out of 5, SD 1.02 vs mean 3.53, SD 0.935; P=.004) than providers. Free-text responses revealed multiple shared concerns, including accuracy/reliability, data bias, patient safety, and privacy/security risks. CONCLUSIONS: These data suggest that providers and nonproviders have positive perceptions of ML-based tools, and that a tool to predict the need for intubation would be of interest to critical care providers. There were many shared concerns about ML/AI in health care elucidated by the surveys. These results provide a baseline evaluation of implementation barriers and determinants of ML/AI-based tools that will be important in their optimal implementation and adoption in the critical care setting and health care in general.