Prediction of episode of hemodynamic instability using an electrocardiogram based analytic: a retrospective cohort study.

BACKGROUND: Predicting the onset of hemodynamic instability before it occurs remains a sought-after goal in acute and critical care medicine. Technologies that allow for this may assist clinicians in preventing episodes of hemodynamic instability (EHI). We tested a novel noninvasive technology, the Analytic for Hemodynamic Instability-Predictive Indicator (AHI-PI), which analyzes a single lead of electrocardiogram (ECG) and extracts heart rate variability and morphologic waveform features to predict an EHI prior to its occurrence. METHODS: Retrospective cohort study at a quaternary care academic health system using data from hospitalized adult patients between August 2019 and April 2020 undergoing continuous ECG monitoring with intermittent noninvasive blood pressure (NIBP) or with continuous intraarterial pressure (IAP) monitoring. RESULTS: AHI-PI's low and high-risk indications were compared with the presence of EHI in the future as indicated by vital signs (heart rate > 100 beats/min with a systolic blood pressure < 90 mmHg or a mean arterial blood pressure of < 70 mmHg). 4,633 patients were analyzed (3,961 undergoing NIBP monitoring, 672 with continuous IAP monitoring). 692 patients had an EHI (380 undergoing NIBP, 312 undergoing IAP). For IAP patients, the sensitivity and specificity of AHI-PI to predict EHI was 89.7% and 78.3% with a positive and negative predictive value of 33.7% and 98.4% respectively. For NIBP patients, AHI-PI had a sensitivity and specificity of 86.3% and 80.5% with a positive and negative predictive value of 11.7% and 99.5% respectively. Both groups performed with an AUC of 0.87. AHI-PI predicted EHI in both groups with a median lead time of 1.1 h (average lead time of 3.7 h for IAP group, 2.9 h for NIBP group). CONCLUSIONS: AHI-PI predicted EHIs with high sensitivity and specificity and within clinically significant time windows that may allow for intervention. Performance was similar in patients undergoing NIBP and IAP monitoring.

External Validation and Comparison of a General Ward Deterioration Index Between Diversely Different Health Systems.

OBJECTIVES: Implementing a predictive analytic model in a new clinical environment is fraught with challenges. Dataset shifts such as differences in clinical practice, new data acquisition devices, or changes in the electronic health record (EHR) implementation mean that the input data seen by a model can differ significantly from the data it was trained on. Validating models at multiple institutions is therefore critical. Here, using retrospective data, we demonstrate how Predicting Intensive Care Transfers and other UnfoReseen Events (PICTURE), a deterioration index developed at a single academic medical center, generalizes to a second institution with significantly different patient population. DESIGN: PICTURE is a deterioration index designed for the general ward, which uses structured EHR data such as laboratory values and vital signs. SETTING: The general wards of two large hospitals, one an academic medical center and the other a community hospital. SUBJECTS: The model has previously been trained and validated on a cohort of 165,018 general ward encounters from a large academic medical center. Here, we apply this model to 11,083 encounters from a separate community hospital. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The hospitals were found to have significant differences in missingness rates (> 5% difference in 9/52 features), deterioration rate (4.5% vs 2.5%), and racial makeup (20% non-White vs 49% non-White). Despite these differences, PICTURE's performance was consistent (area under the receiver operating characteristic curve [AUROC], 0.870; 95% CI, 0.861-0.878), area under the precision-recall curve (AUPRC, 0.298; 95% CI, 0.275-0.320) at the first hospital; AUROC 0.875 (0.851-0.902), AUPRC 0.339 (0.281-0.398) at the second. AUPRC was standardized to a 2.5% event rate. PICTURE also outperformed both the Epic Deterioration Index and the National Early Warning Score at both institutions. CONCLUSIONS: Important differences were observed between the two institutions, including data availability and demographic makeup. PICTURE was able to identify general ward patients at risk of deterioration at both hospitals with consistent performance (AUROC and AUPRC) and compared favorably to existing metrics.

Emergency department length of stay and outcomes of emergency department-based intensive care unit patients.

OBJECTIVE: Emergency department (ED) boarding of patients who are critically ill is associated with poor outcomes. ED-based intensive care units (ED-ICUs) may mitigate the risks of ED boarding. We sought to analyze the impact of ED length of stay (LOS) before transfer to an ED-ICU on patient outcomes. METHODS: We retrospectively analyzed adult ED patients managed in the ED-ICU at a US medical center. Bivariate and multivariable linear regressions tested ED LOS as a predictor of inpatient ICU and hospital LOS, and separate bivariate and multivariable logistic regressions tested ED LOS as a predictor of inpatient ICU admission, 48-hour mortality, and hospital mortality. Multivariable analyses' covariates were age, sex, Charlson Comorbidity Index (CCI), Emergency Severity Index, and eSimplified Acute Physiology Score (eSAPS3). RESULTS: We included 5859 ED visits with subsequent care in the ED-ICU. Median age, CCI, eSAPS3, ED LOS, and ED-ICU LOS were 62 years (interquartile range [IQR], 48-73 years), 5 (IQR, 2-8), 46 (IQR, 36-56), 3.6 hours (IQR, 2.5-5.3 hours), and 8.5 hours (IQR, 5.3-13.4 hours), respectively, and 46.3% were women. Bivariate analyses showed negative associations of ED LOS with hospital LOS (ß = -3.4; 95% confidence interval [CI], -5.9 to -1.0), inpatient ICU admission (odds ratio [OR], 0.86, 95% CI, 0.84-0.88), 48-hour mortality (OR, 0.89; 95% CI, 0.82-0.98), and hospital mortality (OR, 0.89; 95% CI, 0.85-0.92), but no association with inpatient ICU LOS. Multivariable analyses showed a negative association of ED LOS with inpatient ICU admission (OR, 0.91; 95% CI, 0.88-0.93), but no associations with other outcomes. CONCLUSIONS: We observed no significant associations between ED LOS before ED-ICU transfer and worsened outcomes, suggesting an ED-ICU may mitigate the risks of ED boarding of patients who are critically ill.

Developing an electronic health record-derived health equity dashboard to improve learner access to data and metrics.

OBJECTIVES: It is essential to engage learners in efforts aimed at dismantling racism and other contributors to health care disparities. Barriers to their involvement include limited access to data. The objective of our study was to create a data dashboard using an existing quality improvement (QI) infrastructure and provide resident access to data to facilitate exploratory analysis on disparities in emergency department (ED) patient care. METHODS: Focusing on patient populations that have previously been shown in the literature to suffer significant disparities in the ED, we extracted outcomes across a variety of metrics already collected as part of routine ED operations. Using data visualization software, we developed an interactive dashboard for visual exploratory analyses. RESULTS: We designed a dashboard for our resident learners with views that are flexible and allow user selected filters to view clinical outcomes by patient age, treatment area, and chief complaint. Learners were also allowed to select grouping and outcomes of interest to investigate questions and form new hypotheses of their choosing. Available dashboard views included summary counts view to assess ED visits over time by selectable group, a rooming and triage acuity view, time-to-event survival curve view, histogram and box plot views for continuous variables, a view to assess outcome variables by time of day of ED arrival, customizable contingency table views, and correspondence analysis. CONCLUSIONS: Utilizing an existing QI infrastructure, we developed a dashboard that provides a new perspective into commonly collected ED operations data to allow for the exploration of disparities in ED care that is accessible to learners. Future directions include using these data to refine hypotheses on ED disparities, understand root causes, develop interventions, and measure their impact.

An emergency department-based intensive care unit is associated with decreased hospital length of stay for upper gastrointestinal bleeding.

INTRODUCTION: Upper gastrointestinal bleeding (UGIB) is associated with substantial morbidity, mortality, and intensive care unit (ICU) utilization. Initial risk stratification and disposition from the Emergency Department (ED) can prove challenging due to limited data points during a short period of observation. An ED-based ICU (ED-ICU) may allow more rapid delivery of ICU-level care, though its impact on patients with UGIB is unknown. METHODS: A retrospective observational study was conducted at a tertiary U.S. academic medical center. An ED-ICU (the Emergency Critical Care Center [EC3]) opened in February 2015. Patients presenting to the ED with UGIB undergoing esophagogastroduodenoscopy within 72 h were identified and analyzed. The Pre- and Post-EC3 cohorts included patients from 9/2/2012-2/15/2015 and 2/16/2015-6/30/2019. RESULTS: We identified 3788 ED visits; 1033 Pre-EC3 and 2755 Post-EC3. Of Pre-EC3 visits, 200 were critically ill and admitted to ICU [Cohort A]. Of Post-EC3 visits, 682 were critically ill and managed in EC3 [Cohort B], whereas 61 were critically ill and admitted directly to ICU without care in EC3 [Cohort C]. The mean interval from ED presentation to ICU level care was shorter in Cohort B than A or C (3.8 vs 6.3 vs 7.7 h, p < 0.05). More patients in Cohort B received ICU level care within six hours of ED arrival (85.3 vs 52.0 vs 57.4%, p < 0.05). Mean hospital length of stay (LOS) was shorter in Cohort B than A or C (6.2 vs 7.3 vs 10.0 days, p < 0.05). In the Post-EC3 cohort, fewer patients were admitted to an ICU (9.3 vs 19.4%, p < 0.001). The rate of floor admission with transfer to ICU within 24 h was similar. No differences in absolute or risk-adjusted mortality were observed. CONCLUSION: For critically ill ED patients with UGIB, implementation of an ED-ICU was associated with reductions in rate of ICU admission and hospital LOS, with no differences in safety outcomes.

Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods.

BACKGROUND: COVID-19 has led to an unprecedented strain on health care facilities across the United States. Accurately identifying patients at an increased risk of deterioration may help hospitals manage their resources while improving the quality of patient care. Here, we present the results of an analytical model, Predicting Intensive Care Transfers and Other Unforeseen Events (PICTURE), to identify patients at high risk for imminent intensive care unit transfer, respiratory failure, or death, with the intention to improve the prediction of deterioration due to COVID-19. OBJECTIVE: This study aims to validate the PICTURE model's ability to predict unexpected deterioration in general ward and COVID-19 patients, and to compare its performance with the Epic Deterioration Index (EDI), an existing model that has recently been assessed for use in patients with COVID-19. METHODS: The PICTURE model was trained and validated on a cohort of hospitalized non-COVID-19 patients using electronic health record data from 2014 to 2018. It was then applied to two holdout test sets: non-COVID-19 patients from 2019 and patients testing positive for COVID-19 in 2020. PICTURE results were aligned to EDI and NEWS scores for head-to-head comparison via area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve. We compared the models' ability to predict an adverse event (defined as intensive care unit transfer, mechanical ventilation use, or death). Shapley values were used to provide explanations for PICTURE predictions. RESULTS: In non-COVID-19 general ward patients, PICTURE achieved an AUROC of 0.819 (95% CI 0.805-0.834) per observation, compared to the EDI's AUROC of 0.763 (95% CI 0.746-0.781; n=21,740; P<.001). In patients testing positive for COVID-19, PICTURE again outperformed the EDI with an AUROC of 0.849 (95% CI 0.820-0.878) compared to the EDI's AUROC of 0.803 (95% CI 0.772-0.838; n=607; P<.001). The most important variables influencing PICTURE predictions in the COVID-19 cohort were a rapid respiratory rate, a high level of oxygen support, low oxygen saturation, and impaired mental status (Glasgow Coma Scale). CONCLUSIONS: The PICTURE model is more accurate in predicting adverse patient outcomes for both general ward patients and COVID-19 positive patients in our cohorts compared to the EDI. The ability to consistently anticipate these events may be especially valuable when considering potential incipient waves of COVID-19 infections. The generalizability of the model will require testing in other health care systems for validation.

Identifying trigger concepts to screen emergency department visits for diagnostic errors.

OBJECTIVES: The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm. METHODS: We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED. RESULTS: Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings. CONCLUSIONS: We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Demonstrating the consequences of learning missingness patterns in early warning systems for preventative health care: A novel simulation and solution.

When using tree-based methods to develop predictive analytics and early warning systems for preventive healthcare, it is important to use an appropriate imputation method to prevent learning the missingness pattern. To demonstrate this, we developed a novel simulation that generated synthetic electronic health record data using a variational autoencoder with a custom loss function, which took into account the high missing rate of electronic health data. We showed that when tree-based methods learn missingness patterns (correlated with adverse events) in electronic health record data, this leads to decreased performance if the system is used in a new setting that has different missingness patterns. Performance is worst in this scenario when the missing rate between those with and without an adverse event is the greatest. We found that randomized and Bayesian regression imputation methods mitigate the issue of learning the missingness pattern for tree-based methods. We used this information to build a novel early warning system for predicting patient deterioration in general wards and telemetry units: PICTURE (Predicting Intensive Care Transfers and other UnfoReseen Events). To develop, tune, and test PICTURE, we used labs and vital signs from electronic health records of adult patients over four years (n = 133,089 encounters). We analyzed primary outcomes of unplanned intensive care unit transfer, emergency vasoactive medication administration, cardiac arrest, and death. We compared PICTURE with existing early warning systems and logistic regression at multiple levels of granularity. When analyzing PICTURE on the testing set using all observations within a hospital encounter (event rate = 3.4%), PICTURE had an area under the receiver operating characteristic curve (AUROC) of 0.83 and an adjusted (event rate = 4%) area under the precision-recall curve (AUPR) of 0.27, while the next best tested method-regularized logistic regression-had an AUROC of 0.80 and an adjusted AUPR of 0.22. To ensure system interpretability, we applied a state-of-the-art prediction explainer that provided a ranked list of features contributing most to the prediction. Though it is currently difficult to compare machine learning-based early warning systems, a rudimentary comparison with published scores demonstrated that PICTURE is on par with state-of-the-art machine learning systems. To facilitate more robust comparisons and development of early warning systems in the future, we have released our variational autoencoder's code and weights so researchers can (a) test their models on data similar to our institution and (b) make their own synthetic datasets.

Association of an Emergency Department-Based Intensive Care Unit With Survival and Inpatient Intensive Care Unit Admissions.

Importance: Increased patient acuity, decreased intensive care unit (ICU) bed availability, and a shortage of intensivist physicians have led to strained ICU capacity. The resulting increase in emergency department (ED) boarding time for patients requiring ICU-level care has been associated with worse outcomes. Objective: To determine the association of a novel ED-based ICU, the Emergency Critical Care Center (EC3), with 30-day mortality and inpatient ICU admission. Design, Setting, and Participants: This retrospective cohort study used electronic health records of all ED visits between September 1, 2012, and July 31, 2017, with a documented clinician encounter at a large academic medical center in the United States with approximately 75 000 adult ED visits per year. The pre-EC3 cohort included ED patients from September 2, 2012, to February 15, 2015, when the EC3 opened, and the post-EC3 cohort included ED patients from February 16, 2015, to July 31, 2017. Data analyses were conducted from March 2, 2018, to May 28, 2019. Exposures: Implementation of EC3, an ED-based ICU designed to provide rapid initiation of ICU-level care in the ED setting and seamless transition to inpatient ICUs. Main Outcomes and Measures: The main outcomes were 30-day mortality among ED patients and rate of ED to ICU admission. Results: A total of 349 310 visits from a consecutive sample of ED patients (mean [SD] age, 48.5 [19.7] years; 189 709 [54.3%] women) were examined; the pre-EC3 cohort included 168 877 visits and the post-EC3 cohort included 180 433 visits. Implementation of EC3 was associated with a statistically significant reduction in risk-adjusted 30-day mortality among all ED patients (pre-EC3, 2.13%; post-EC3, 1.83%; adjusted odds ratio, 0.85; 95% CI, 0.80-0.90; number needed to treat, 333 patient encounters; 95% CI, 256-476). The risk-adjusted rate of ED admission to ICU decreased with implementation of EC3 (pre-EC3, 3.2%; post-EC3, 2.7%; adjusted odds ratio, 0.80; 95% CI, 0.76-0.83; number needed to treat, 179 patient encounters; 95% CI, 149-217). Conclusions and Relevance: Implementation of a novel ED-based ICU was associated with improved 30-day survival and reduced inpatient ICU admission. Additional research is warranted to further explore the value of this novel care delivery model in various health care systems.

Willingness and Ability of Older Adults in the Emergency Department to Provide Clinical Information Using a Tablet Computer.

OBJECTIVES: To estimate the proportion of older adults in the emergency department (ED) who are willing and able to use a tablet computer to answer questions. DESIGN: Prospective, ED-based cross-sectional study. SETTING: Two U.S. academic EDs. PARTICIPANTS: Individuals aged 65 and older. MEASUREMENTS: As part of screening for another study, potential study participants were asked whether they would be willing to use a tablet computer to answer eight questions instead of answering questions orally. A custom user interface optimized for older adults was used. Trained research assistants observed study participants as they used the tablets. Ability to use the tablet was assessed based on need for assistance and number of questions answered correctly. RESULTS: Of 365 individuals approached, 248 (68%) were willing to answer screening questions, 121 of these (49%) were willing to use a tablet computer; of these, 91 (75%) were able to answer at least six questions correctly, and 35 (29%) did not require assistance. Only 14 (12%) were able to answer all eight questions correctly without assistance. Individuals aged 65 to 74 and those reporting use of a touchscreen device at least weekly were more likely to be willing and able to use the tablet computer. Of individuals with no or mild cognitive impairment, the percentage willing to use the tablet was 45%, and the percentage answering all questions correctly was 32%. CONCLUSION: Approximately half of this sample of older adults in the ED was willing to provide information using a tablet computer, but only a small minority of these were able to enter all information correctly without assistance. Tablet computers may provide an efficient means of collecting clinical information from some older adults in the ED, but at present, it will be ineffective for a significant portion of this population.

Computer-facilitated review of electronic medical records reliably identifies emergency department interventions in older adults.

OBJECTIVES: An estimated 14% to 25% of all scientific studies in peer-reviewed emergency medicine (EM) journals are medical records reviews. The majority of the chart reviews in these studies are performed manually, a process that is both time-consuming and error-prone. Computer-based text search engines have the potential to enhance chart reviews of electronic emergency department (ED) medical records. The authors compared the efficiency and accuracy of a computer-facilitated medical record review of ED clinical records of geriatric patients with a traditional manual review of the same data and describe the process by which this computer-facilitated review was completed. METHODS: Clinical data from consecutive ED patients age 65 years or older were collected retrospectively by manual and computer-facilitated medical record review. The frequency of three significant ED interventions in older adults was determined using each method. Performance characteristics of each search method, including sensitivity and positive predictive value, were determined, and the overall sensitivities of the two search methods were compared using McNemar's test. RESULTS: For 665 patient visits, there were 49 (7.4%) Foley catheters placed, 36 (5.4%) sedative medications administered, and 15 (2.3%) patients who received positive pressure ventilation. The computer-facilitated review identified more of the targeted procedures (99 of 100, 99%), compared to manual review (74 of 100 procedures, 74%; p < 0.0001). CONCLUSIONS: A practical, non-resource-intensive, computer-facilitated free-text medical record review was completed and was more efficient and accurate than manually reviewing ED records.