Multisite development and validation of machine learning models to predict severe outcomes and guide decision-making for emergency department patients with influenza.

Objective: Millions of Americans are infected by influenza annually. A minority seek care in the emergency department (ED) and, of those, only a limited number experience severe disease or death. ED clinicians must distinguish those at risk for deterioration from those who can be safely discharged. Methods: We developed random forest machine learning (ML) models to estimate needs for critical care within 24 h and inpatient care within 72 h in ED patients with influenza. Predictor data were limited to those recorded prior to ED disposition decision: demographics, ED complaint, medical problems, vital signs, supplemental oxygen use, and laboratory results. Our study population was comprised of adults diagnosed with influenza at one of five EDs in our university health system between January 1, 2017 and May 18, 2022; visits were divided into two cohorts to facilitate model development and validation. Prediction performance was assessed by the area under the receiver operating characteristic curve (AUC) and the Brier score. Results: Among 8032 patients with laboratory-confirmed influenza, incidence of critical care needs was 6.3% and incidence of inpatient care needs was 19.6%. The most common reasons for ED visit were symptoms of respiratory tract infection, fever, and shortness of breath. Model AUCs were 0.89 (95% CI 0.86-0.93) for prediction of critical care and 0.90 (95% CI 0.88-0.93) for inpatient care needs; Brier scores were 0.026 and 0.042, respectively. Importantpredictors included shortness of breath, increasing respiratory rate, and a high number of comorbid diseases. Conclusions: ML methods can be used to accurately predict clinical deterioration in ED patients with influenza and have potential to support ED disposition decision-making.

Epidemiology and Clinical Outcomes of Community-Acquired Acute Kidney Injury in the Emergency Department: A Multisite Retrospective Cohort Study.

RATIONALE & OBJECTIVE: The prevalence of community-acquired acute kidney injury (CA-AKI) in the United States and its clinical consequences are not well described. Our objective was to describe the epidemiology of CA-AKI and the associated clinical outcomes. STUDY DESIGN: Retrospective cohort study. SETTING & PARTICIPANTS: 178,927 encounters by 139,632 adults at 5 US emergency departments (EDs) between July 1, 2017, and December 31, 2022. PREDICTORS: CA-AKI identified using KDIGO (Kidney Disease: Improving Global Outcomes) serum creatinine (Scr)-based criteria. OUTCOMES: For encounters resulting in hospitalization, the in-hospital trajectory of AKI severity, dialysis initiation, intensive care unit (ICU) admission, and death. For all encounters, occurrence over 180 days of hospitalization, ICU admission, new or progressive chronic kidney disease, dialysis initiation, and death. ANALYTICAL APPROACH: Multivariable logistic regression analysis to test the association between CA-AKI and measured outcomes. RESULTS: For all encounters, 10.4% of patients met the criteria for any stage of AKI on arrival to the ED. 16.6% of patients admitted to the hospital from the ED had CA-AKI on arrival to the ED. The likelihood of AKI recovery was inversely related to CA-AKI stage on arrival to the ED. Among encounters for hospitalized patients, CA-AKI was associated with in-hospital dialysis initiation (OR, 6.2; 95% CI, 5.1-7.5), ICU admission (OR, 1.9; 95% CI, 1.7-2.0), and death (OR, 2.2; 95% CI, 2.0-2.5) compared with patients without CA-AKI. Among all encounters, CA-AKI was associated with new or progressive chronic kidney disease (OR, 6.0; 95% CI, 5.6-6.4), dialysis initiation (OR, 5.1; 95% CI, 4.5-5.7), subsequent hospitalization (OR, 1.1; 95% CI, 1.1-1.2) including ICU admission (OR, 1.2; 95% CI, 1.1-1.4), and death (OR, 1.6; 95% CI, 1.5-1.7) during the subsequent 180 days. LIMITATIONS: Residual confounding. Study implemented at a single university-based health system. Potential selection bias related to exclusion of patients without an available baseline Scr measurement. Potential ascertainment bias related to limited repeat Scr data during follow-up after an ED visit. CONCLUSIONS: CA-AKI is a common and important entity that is associated with serious adverse clinical consequences during the 6-month period after diagnosis. PLAIN-LANGUAGE SUMMARY: Acute kidney injury (AKI) is a condition characterized by a rapid decline in kidney function. There are many causes of AKI, but few studies have examined how often AKI is already present when patients first arrive to an emergency department seeking medical attention for any reason. We analyzed approximately 175,000 visits to Johns Hopkins emergency departments and found that AKI is common on presentation to the emergency department and that patients with AKI have increased risks of hospitalization, intensive care unit admission, development of chronic kidney disease, requirement of dialysis, and death in the first 6 months after diagnosis. AKI is an important condition for health care professionals to recognize and is associated with serious adverse outcomes.

Do prehospital sepsis alerts decrease time to complete CMS sepsis measures?

INTRODUCTION: In an effort to improve sepsis outcomes the Centers for Medicare and Medicaid Services (CMS) established a time sensitive sepsis management bundle as a core quality measure that includes blood culture collection, serum lactate collection, initiation of intravenous fluid administration, and initiation of broad-spectrum antibiotics. Few studies examine the effects of a prehospital sepsis alert protocol on decreasing time to complete CMS sepsis core measures. METHODS: This study was a retrospective cohort study of patients transported via EMS from December 1, 2018 to December 1, 2019 who met the criteria of the Maryland Statewide EMS sepsis protocol and compared outcomes between patients who activated a prehospital sepsis alert and patients who did not activate a prehospital sepsis alert. The Maryland Institute for Emergency Medical Services Systems developed a sepsis protocol that instructs EMS providers to notify the nearest appropriate facility with a sepsis alert if a patient 18 years of age and older is suspected of having an infection and also presents with at least two of the following: temperature >38 °C or <35.5 °C, a heart rate >100 beats per minute, a respiratory rate >25 breaths per minute or end-tidal carbon dioxide less than or equal to 32 mmHg, a systolic blood pressure <90 mmHg, or a point of care lactate reading greater than or equal to 4 mmol/L. RESULTS: Median time to achieve all four studied CMS sepsis core measures was 103 min [IQR 61-153] for patients who received a prehospital sepsis alert and 106.5 min [IQR 75-189] for patients who did not receive a prehospital sepsis alert (p-value 0.105). Median time to completion was shorter for serum lactate collection (28 min. vs 35 min., p-value 0.019), blood culture collection (28 min. vs 38 min., p-value <0.01), and intravenous fluid administration (54 min. vs 61 min., p-value 0.025) but was not significantly different for antibiotic administration (94 min. vs 103 min., p-value 0.12) among patients who triggered a sepsis alert. CONCLUSION: This study questions the effectiveness of prehospital sepsis alert protocols on decreasing time to complete CMS sepsis core measures. Future studies should address if these times can be impacted by having EMS providers independently administer antibiotics.

Renal outcomes following intravenous contrast administration in patients with acute kidney injury: a multi-site retrospective propensity-adjusted analysis.

PURPOSE: Evidence of an association between intravenous contrast media (CM) and persistent renal dysfunction is lacking for patients with pre-existing acute kidney injury (AKI). This study was designed to determine the association between intravenous CM administration and persistent AKI in patients with pre-existing AKI. METHODS: A retrospective propensity-weighted and entropy-balanced observational cohort analysis of consecutive hospitalized patients ≥ 18 years old meeting Kidney Disease Improving Global Outcomes (KDIGO) creatinine-based criteria for AKI at time of arrival to one of three emergency departments between 7/1/2017 and 6/30/2021 who did or did not receive intravenous CM. Outcomes included persistent AKI at hospital discharge and initiation of dialysis within 180 days of index encounter. RESULTS: Our analysis included 14,449 patient encounters, with 12.8% admitted to the intensive care unit (ICU). CM was administered in 18.4% of all encounters. AKI resolved prior to hospital discharge for 69.1%. No association between intravenous CM administration and persistent AKI was observed after unadjusted multivariable logistic regression modeling (OR 1; 95% CI 0.89-1.11), propensity weighting (OR 0.93; 95% CI 0.83-1.05), and entropy balancing (OR 0.94; 95% CI 0.83-1.05). Sub-group analysis in those admitted to the ICU yielded similar results. Initiation of dialysis within 180 days was observed in 5.4% of the cohort. An association between CM administration and increased risk of dialysis within 180 days was not observed. CONCLUSION: Among patients with pre-existing AKI, contrast administration was not associated with either persistent AKI at hospital discharge or initiation of dialysis within 180 days. Current consensus recommendations for use of intravenous CM in patients with stable renal disease may also be applied to patients with pre-existing AKI.

Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage.

Objective: To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage. Materials and Methods: Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients' risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model's predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity. Results: There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients. Discussion: Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race. Conclusion: Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.

Monocyte distribution width as a pragmatic screen for SARS-CoV-2 or influenza infection.

Monocyte distribution width (MDW) is a novel marker of monocyte activation, which is known to occur in the immune response to viral pathogens. Our objective was to determine the performance of MDW and other leukocyte parameters as screening tests for SARS-CoV-2 and influenza infection. This was a prospective cohort analysis of adult patients who underwent complete blood count (CBC) and SARS-CoV-2 or influenza testing in an Emergency Department (ED) between January 2020 and July 2021. The primary outcome was SARS-CoV-2 or influenza infection. Secondary outcomes were measures of severity of illness including inpatient hospitalization, critical care admission, hospital lengths of stay and mortality. Descriptive statistics and test performance measures were evaluated for monocyte percentage, MDW, white blood cell (WBC) count, and neutrophil to lymphocyte ratio (NLR). 3,425 ED patient visits were included. SARS-CoV-2 testing was performed during 1,922 visits with a positivity rate of 5.4%; influenza testing was performed during 2,090 with a positivity rate of 2.3%. MDW was elevated in patients with SARS-Cov-2 (median 23.0U; IQR 20.5-25.1) or influenza (median 24.1U; IQR 22.0-26.9) infection, as compared to those without (18.9U; IQR 17.4-20.7 and 19.1U; 17.4-21, respectively, P < 0.001). Monocyte percentage, WBC and NLR values were within normal range in patients testing positive for either virus. MDW identified SARS-CoV-2 and influenza positive patients with an area under the curve (AUC) of 0.83 (95% CI 0.79-0.86) and 0.83 (95% CI 0.77-0.88), respectively. At the accepted cut-off value of 20U for MDW, sensitivities were 83.7% (95% CI 76.5-90.8%) for SARS-CoV-2 and 89.6% (95% CI 80.9-98.2%) for influenza, compared to sensitivities below 45% for monocyte percentage, WBC and NLR. MDW negative predictive values were 98.6% (95% CI 98.0-99.3%) and 99.6% (95% CI 99.3-100.0%) respectively for SARS-CoV-2 and influenza. Monocyte Distribution Width (MDW), available as part of a routine complete blood count (CBC) with differential, may be a useful indicator of SARS-CoV-2 or influenza infection.

Multisite implementation of a workflow-integrated machine learning system to optimize COVID-19 hospital admission decisions.

Demand has outstripped healthcare supply during the coronavirus disease 2019 (COVID-19) pandemic. Emergency departments (EDs) are tasked with distinguishing patients who require hospital resources from those who may be safely discharged to the community. The novelty and high variability of COVID-19 have made these determinations challenging. In this study, we developed, implemented and evaluated an electronic health record (EHR) embedded clinical decision support (CDS) system that leverages machine learning (ML) to estimate short-term risk for clinical deterioration in patients with or under investigation for COVID-19. The system translates model-generated risk for critical care needs within 24 h and inpatient care needs within 72 h into rapidly interpretable COVID-19 Deterioration Risk Levels made viewable within ED clinician workflow. ML models were derived in a retrospective cohort of 21,452 ED patients who visited one of five ED study sites and were prospectively validated in 15,670 ED visits that occurred before (n = 4322) or after (n = 11,348) CDS implementation; model performance and numerous patient-oriented outcomes including in-hospital mortality were measured across study periods. Incidence of critical care needs within 24 h and inpatient care needs within 72 h were 10.7% and 22.5%, respectively and were similar across study periods. ML model performance was excellent under all conditions, with AUC ranging from 0.85 to 0.91 for prediction of critical care needs and 0.80-0.90 for inpatient care needs. Total mortality was unchanged across study periods but was reduced among high-risk patients after CDS implementation.

A Patient Outcomes-Driven Feedback Platform for Emergency Medicine Clinicians: Human-Centered Design and Usability Evaluation of Linking Outcomes Of Patients (LOOP).

BACKGROUND: The availability of patient outcomes-based feedback is limited in episodic care environments such as the emergency department. Emergency medicine (EM) clinicians set care trajectories for a majority of hospitalized patients and provide definitive care to an even larger number of those discharged into the community. EM clinicians are often unaware of the short- and long-term health outcomes of patients and how their actions may have contributed. Despite large volumes of patients and data, outcomes-driven learning that targets individual clinician experiences is meager. Integrated electronic health record (EHR) systems provide opportunity, but they do not have readily available functionality intended for outcomes-based learning. OBJECTIVE: This study sought to unlock insights from routinely collected EHR data through the development of an individualizable patient outcomes feedback platform for EM clinicians. Here, we describe the iterative development of this platform, Linking Outcomes Of Patients (LOOP), under a human-centered design framework, including structured feedback obtained from its use. METHODS: This multimodal study consisting of human-centered design studios, surveys (24 physicians), interviews (11 physicians), and a LOOP application usability evaluation (12 EM physicians for ≥30 minutes each) was performed between August 2019 and February 2021. The study spanned 3 phases: (1) conceptual development under a human-centered design framework, (2) LOOP technical platform development, and (3) usability evaluation comparing pre- and post-LOOP feedback gathering practices in the EHR. RESULTS: An initial human-centered design studio and EM clinician surveys revealed common themes of disconnect between EM clinicians and their patients after the encounter. Fundamental postencounter outcomes of death (15/24, 63% respondents identified as useful), escalation of care (20/24, 83%), and return to ED (16/24, 67%) were determined high yield for demonstrating proof-of-concept in our LOOP application. The studio aided the design and development of LOOP, which integrated physicians throughout the design and content iteration. A final LOOP prototype enabled usability evaluation and iterative refinement prior to launch. Usability evaluation compared to status quo (ie, pre-LOOP) feedback gathering practices demonstrated a shift across all outcomes from "not easy" to "very easy" to obtain and from "not confident" to "very confident" in estimating outcomes after using LOOP. On a scale from 0 (unlikely) to 10 (most likely), the users were very likely (9.5) to recommend LOOP to a colleague. CONCLUSIONS: This study demonstrates the potential for human-centered design of a patient outcomes-driven feedback platform for individual EM providers. We have outlined a framework for working alongside clinicians with a multidisciplined team to develop and test a tool that augments their clinical experience and enables closed-loop learning.

Monocyte distribution width as part of a broad pragmatic sepsis screen in the emergency department.

STUDY OBJECTIVE: Enhancement of a routine complete blood count (CBC) for detection of sepsis in the emergency department (ED) has pragmatic utility for early management. This study evaluated the performance of monocyte distribution width (MDW) alone and in combination with other routine CBC parameters as a screen for sepsis and septic shock in ED patients. METHODS: A prospective cohort analysis of adult patients with a CBC collected at an urban ED from January 2020 through July 2021. The performance of MDW, white blood count (WBC) count, and neutrophil-to-lymphocyte-ratio (NLR) to detect sepsis and septic shock (Sepsis-3 Criteria) was evaluated using diagnostic performance measures. RESULTS: The cohort included 7952 ED patients, with 180 meeting criteria for sepsis; 43 with septic shock and 137 without shock. MDW was highest for patients with septic shock (median 24.8 U, interquartile range [IQR] 22.0-28.1) and trended downward for patients with sepsis without shock (23.9 U, IQR 20.2-26.8), infection (20.4 U, IQR 18.2-23.3), then controls (18.6 U, IQR 17.1-20.4). In isolation, MDW detected sepsis and septic shock with an area under the receiver operator characteristic curve (AUC) of 0.80 (95% confidence interval [CI] 0.77-0.84) and 0.85 (95% CI 0.80-0 .91), respectively. Optimal performance was achieved in combination with WBC count and NLR for detection of sepsis (AUC 0.86, 95% CI 0.83-0.89) and septic shock (0.86, 95% CI 0.80-0.92). CONCLUSION: A CBC differential panel that includes MDW demonstrated strong performance characteristics in a broad ED population suggesting pragmatic value as a rapid screen for sepsis and septic shock.

Development of an Electronic Algorithm to Identify in Real Time Adults Hospitalized With Suspected Community-Acquired Pneumonia.

BACKGROUND: Community-acquired pneumonia (CAP) is a major driver of hospital antibiotic use. Efficient methods to identify patients treated for CAP in real time using the electronic health record (EHR) are needed. Automated identification of these patients could facilitate systematic tracking, intervention, and feedback on CAP-specific metrics such as appropriate antibiotic choice and duration. METHODS: Using retrospective data, we identified suspected CAP cases by searching for patients who received CAP antibiotics AND had an admitting International Classification of Diseases, Tenth Revision (ICD-10) code for pneumonia OR chest imaging within 24 hours OR bacterial urinary antigen testing within 48 hours of admission (denominator query). We subsequently explored different structured and natural language processing (NLP)-derived data from the EHR to identify CAP cases. We evaluated combinations of these electronic variables through receiver operating characteristic (ROC) curves to assess which best identified CAP cases compared to cases identified by manual chart review. Exclusion criteria were age <18 years, absolute neutrophil count <500 cells/mm3, and admission to an oncology unit. RESULTS: Compared to the gold standard of chart review, the area under the ROC curve to detect CAP was 0.63 (95% confidence interval [CI], .55-.72; P < .01) using structured data (ie, laboratory and vital signs) and 0.83 (95% CI, .77-.90; P < .01) when NLP-derived data from radiographic reports were included. The sensitivity and specificity of the latter model were 80% and 81%, respectively. CONCLUSIONS: Creating an electronic tool that effectively identifies CAP cases in real time is possible, but its accuracy is dependent on NLP-derived radiographic data.

The Role of Healthcare Worker-Mediated Contact Networks in the Transmission of Vancomycin-Resistant Enterococci.

BACKGROUND: User- and time-stamped data from hospital electronic health records (EHRs) present opportunities to evaluate how healthcare worker (HCW)-mediated contact networks impact transmission of multidrug-resistant pathogens, such as vancomycin-resistant enterococci (VRE). METHODS: This is a retrospective analysis of incident acquisitions of VRE between July 1, 2016 and June 30, 2018. Clinical and demographic patient data were extracted from the hospital EHR system, including all recorded HCW contacts with patients. Contacts by an HCW with 2 different patients within 1 hour was considered a "connection". Incident VRE acquisition was determined by positive clinical or surveillance cultures collected ≥72 hours after a negative surveillance culture. RESULTS: There were 2952 hospitalizations by 2364 patients who had ≥2 VRE surveillance swabs, 112 (4.7%) patients of which had incident nosocomial acquisitions. Patients had a median of 24 (interquartile range [IQR], 18-33) recorded HCW contacts per day, 9 (IQR, 5-16) of which, or approximately 40%, were connections that occurred <1 hour after another patient contact. Patients that acquired VRE had a higher average number of daily connections to VRE-positive patients (3.1 [standard deviation {SD}, 2.4] versus 2.0 [SD, 2.1]). Controlling for other risk factors, connection to a VRE-positive patient was associated with increased odds of acquiring VRE (odds ratio, 1.64; 95% confidence interval, 1.39-1.92). CONCLUSIONS: We demonstrated that EHR data can be used to quantify the impact of HCW-mediated patient connections on transmission of VRE in the hospital. Defining incident acquisition risk of multidrug-resistant organisms through HCWs connections from EHR data in real-time may aid implementation and evaluation of interventions to contain their spread.