Development and Structure of an Accurate Machine Learning Algorithm to Predict Inpatient Mortality and Hospice Outcomes in the Coronavirus Disease 2019 Era.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic has challenged the accuracy and racial biases present in traditional mortality scores. An accurate prognostic model that can be applied to hospitalized patients irrespective of race or COVID-19 status may benefit patient care. RESEARCH DESIGN: This cohort study utilized historical and ongoing electronic health record features to develop and validate a deep-learning model applied on the second day of admission predicting a composite outcome of in-hospital mortality, discharge to hospice, or death within 30 days of admission. Model features included patient demographics, diagnoses, procedures, inpatient medications, laboratory values, vital signs, and substance use history. Conventional performance metrics were assessed, and subgroup analysis was performed based on race, COVID-19 status, and intensive care unit admission. SUBJECTS: A total of 35,521 patients hospitalized between April 2020 and October 2020 at a single health care system including a tertiary academic referral center and 9 community hospitals. RESULTS: Of 35,521 patients, including 9831 non-White patients and 2020 COVID-19 patients, 2838 (8.0%) met the composite outcome. Patients who experienced the composite outcome were older (73 vs. 61 y old) with similar sex and race distributions between groups. The model achieved an area under the receiver operating characteristic curve of 0.89 (95% confidence interval: 0.88, 0.91) and an average positive predictive value of 0.46 (0.40, 0.52). Model performance did not differ significantly in White (0.89) and non-White (0.90) subgroups or when grouping by COVID-19 status and intensive care unit admission. CONCLUSION: A deep-learning model using large-volume, structured electronic health record data can effectively predict short-term mortality or hospice outcomes on the second day of admission in the general inpatient population without significant racial bias.

A Randomized Trial of Palliative Care Discussions Linked to an Automated Early Warning System Alert.

OBJECTIVE: To determine whether an Early Warning System could identify patients wishing to focus on palliative care measures. DESIGN: Prospective, randomized, pilot study. SETTING: Barnes-Jewish Hospital, Saint Louis, MO (January 15, 2015, to December 12, 2015). PATIENTS: A total of 206 patients; 89 intervention (43.2%) and 117 controls (56.8%). INTERVENTIONS: Palliative care in high-risk patients targeted by an Early Warning System. MEASUREMENTS AND MAIN RESULTS: Advanced directive documentation was significantly greater prior to discharge in the intervention group (37.1% vs 15.4%; p < 0.001) as were first-time requests for advanced directive documentation (14.6% vs 0.0%; p < 0.001). Documentation of resuscitation status was also greater prior to discharge in the intervention group (36.0% vs 23.1%; p = 0.043). There was no difference in the number of patients requesting a change in resuscitation status between groups (11.2% vs 9.4%; p = 0.666). However, changes in resuscitation status occurred earlier and on the general medicine units for the intervention group compared to the control group. The number of patients transferred to an ICU was significantly lower for intervention patients (12.4% vs 27.4%; p = 0.009). The median (interquartile range) ICU length of stay was significantly less for the intervention group (0 [0-0] vs 0 [0-1] d; p = 0.014). Hospital mortality was similar (12.4% vs 10.3%; p = 0.635). CONCLUSIONS: This study suggests that automated Early Warning System alerts can identify patients potentially benefitting from directed palliative care discussions and reduce the number of ICU transfers.

The number of discharge medications predicts thirty-day hospital readmission: a cohort study.

BACKGROUND: Hospital readmission occurs often and is difficult to predict. Polypharmacy has been identified as a potential risk factor for hospital readmission. However, the overall impact of the number of discharge medications on hospital readmission is still undefined. METHODS: To determine whether the number of discharge medications is predictive of thirty-day readmission using a retrospective cohort study design performed at Barnes-Jewish Hospital from January 15, 2013 to May 9, 2013. The primary outcome assessed was thirty-day hospital readmission. We also assessed potential predictors of thirty-day readmission to include the number of discharge medications. RESULTS: The final cohort had 5507 patients of which 1147 (20.8 %) were readmitted within thirty days of their hospital discharge date. The number of discharge medications was significantly greater for patients having a thirty-day readmission compared to those without a thirty-day readmission (7.2 ± 4.1 medications [7.0 medications (4.0 medications, 10.0 medications)] versus 6.0 ± 3.9 medications [6.0 medications (3.0 medications, 9.0 medications)]; P < 0.001). There was a statistically significant association between increasing numbers of discharge medications and the prevalence of thirty-day hospital readmission (P < 0.001). Multiple logistic regression identified more than six discharge medications to be independently associated with thirty-day readmission (OR, 1.26; 95 % CI, 1.17-1.36; P = 0.003). Other independent predictors of thirty-day readmission were: more than one emergency department visit in the previous six months, a minimum hemoglobin value less than or equal to 9 g/dL, presence of congestive heart failure, peripheral vascular disease, cirrhosis, and metastatic cancer. A risk score for thirty-day readmission derived from the logistic regression model had good predictive accuracy (AUROC = 0.661 [95 % CI, 0.643-0.679]). CONCLUSIONS: The number of discharge medications is associated with the prevalence of thirty-day hospital readmission. A risk score, that includes the number of discharge medications, accurately predicts patients at risk for thirty-day readmission. Our findings suggest that relatively simple and accessible parameters can identify patients at high risk for hospital readmission potentially distinguishing such individuals for interventions to minimize readmissions.

Identifying critically ill patients at risk for inappropriate antibiotic therapy: a pilot study of a point-of-care decision support alert.

OBJECTIVE: To develop an automated alert aimed at reducing inappropriate antibiotic therapy of serious healthcare-associated infections. DESIGN: Single-center cohort study from November 2011 to November 2012. SETTING: Barnes-Jewish Hospital (1,250-bed academic hospital). PATIENTS: A total of 3,616 critically ill patients receiving treatment with antibiotics targeting healthcare-associated infections due to Gram-negative bacteria. INTERVENTIONS: Upon antibiotic order entry in the ICU for a Gram-negative antibiotic, the antibiotic and microbiologic history for each patient was electronically queried in real time across all 13 BJC HealthCare hospitals. Patients were assigned to the alert group if they had exposure to the same antibiotic class currently being prescribed (cefepime, meropenem, or piperacillin-tazobactam) or had a positive culture isolating a Gram-negative organism with resistance to the prescribed antibiotic in the previous 6 months. MEASUREMENTS AND MAIN RESULTS: Nine hundred patients (24.2%) generated an alert. Alerted patients were significantly more likely to receive inappropriate antibiotic therapy (7.1% vs. 2.9%; p < 0.001). Based on clinical information available in the alert, 34 of 64 of the alerted patients that received inappropriate therapy (53.1%) could have received an alternative ß-lactam antibiotic with in vitro susceptibility to the identified pathogen. Independent predictors (adjusted odds ratio [95% CI]) of inappropriate therapy included alert generation (1.788 [1.167-2.740]; p = 0.008), medical ICU patients (1.528 [1.007-2.319]; p = 0.046), and a pulmonary source of infection (2.063 [1.363-3.122]; p = 0.0001). Patients in the alert group had significantly greater hospital mortality (29.9% vs. 23.6%; p < 0.001) and hospital length of stay (median, 13.1 vs. 10.7 d; p < 0.001) compared with nonalert patients. CONCLUSIONS: Our results suggest that a simple automated alert could identify more than 40% of critically ill patients prescribed inappropriate antibiotic therapy for healthcare-associated infections. These data suggest that an opportunity exists to employ hospital informatics systems to improve the prescription of antibiotic therapy in ICU patients with suspected healthcare-associated infections.

Advanced Care Planning for Hospitalized Patients Following Clinician Notification of Patient Mortality by a Machine Learning Algorithm.

Importance: Goal-concordant care is an ongoing challenge in hospital settings. Identification of high mortality risk within 30 days may call attention to the need to have serious illness conversations, including the documentation of patient goals of care. Objective: To examine goals of care discussions (GOCDs) in a community hospital setting with patients identified as having a high risk of mortality by a machine learning mortality prediction algorithm. Design, Setting, and Participants: This cohort study took place at community hospitals within 1 health care system. Participants included adult patients with a high risk of 30-day mortality who were admitted to 1 of 4 hospitals between January 2 and July 15, 2021. Patient encounters of inpatients in the intervention hospital where physicians were notified of the computed high risk mortality score were compared with patient encounters of inpatients in 3 community hospitals without the intervention (ie, matched control). Intervention: Physicians of patients with a high risk of mortality within 30 days received notification and were encouraged to arrange for GOCDs. Main Outcomes and Measures: The primary outcome was the percentage change of documented GOCDs prior to discharge. Propensity-score matching was completed on a preintervention and postintervention period using age, sex, race, COVID-19 status, and machine learning-predicted mortality risk scores. A difference-in-difference analysis validated the results. Results: Overall, 537 patients were included in this study with 201 in the preintervention period (94 in the intervention group; 104 in the control group) and 336 patients in the postintervention period. The intervention and control groups included 168 patients per group and were well-balanced in age (mean [SD], 79.3 [9.60] vs 79.6 [9.21] years; standardized mean difference [SMD], 0.03), sex (female, 85 [51%] vs 85 [51%]; SMD, 0), race (White patients, 145 [86%] vs 144 [86%]; SMD 0.006), and Charlson comorbidities (median [range], 8.00 [2.00-15.0] vs 9.00 [2.00 to 19.0]; SMD, 0.34). Patients in the intervention group from preintervention to postintervention period were associated with being 5 times more likely to have documented GOCDs (OR, 5.11 [95% CI, 1.93 to 13.42]; P = .001) by discharge compared with matched controls, and GOCD occurred significantly earlier in the hospitalization in the intervention patients as compared with matched controls (median, 4 [95% CI, 3 to 6] days vs 16 [95% CI, 15 to not applicable] days; P < .001). Similar findings were observed for Black patient and White patient subgroups. Conclusions and Relevance: In this cohort study, patients whose physicians had knowledge of high-risk predictions from machine learning mortality algorithms were associated with being 5 times more likely to have documented GOCDs than matched controls. Additional external validation is needed to determine if similar interventions would be helpful at other institutions.

Implementation of a real-time computerized sepsis alert in nonintensive care unit patients.

OBJECTIVE: Early therapy of sepsis involving fluid resuscitation and antibiotic administration has been shown to improve patient outcomes. A proactive tool to identify patients at risk for developing sepsis may decrease time to interventions and improve patient outcomes. The objective of this study was to evaluate whether the implementation of an automated sepsis screening and alert system facilitated early appropriate interventions. DESIGN: Prospective, observational, pilot study. SETTING: Six medicine wards in Barnes-Jewish Hospital, a 1250-bed academic medical center. PATIENTS: Patients identified by the sepsis screen while admitted to a medicine ward were included in the study. A total of 300 consecutive patients were identified comprising the nonintervention group (n=200) and the intervention group (n=100). INTERVENTIONS: A real-time sepsis alert was implemented for the intervention group, which notified the charge nurse on the patient's hospital ward by text page. MEASUREMENTS AND MAIN RESULTS: Within 12 hrs of the sepsis alert, interventions by the treating physicians were assessed, including new or escalated antibiotics, intravenous fluid administration, oxygen therapy, vasopressors, and diagnostic tests. After exclusion of patients without commitment to aggressive management, 181 patients in the nonintervention group and 89 patients in the intervention group were analyzed. Within 12 hrs of the sepsis alert, 70.8% of patients in the intervention group had received≥1 intervention vs. 55.8% in the nonintervention group (p=.018). Antibiotic escalation, intravenous fluid administration, oxygen therapy, and diagnostic tests were all increased in the intervention group. This was a single-center, institution- and patient-specific algorithm. CONCLUSIONS: The sepsis alert developed at Barnes-Jewish Hospital was shown to increase early therapeutic and diagnostic interventions among nonintensive care unit patients at risk for sepsis.

Mortality and Length of Stay Trends Following Implementation of a Rapid Response System and Real-Time Automated Clinical Deterioration Alerts.

A study was performed to determine the potential influence of a rapid response system (RRS) employing real-time clinical deterioration alerts (RTCDAs) on patient outcomes involving 8 general medicine units. Introduction of the RRS occurred in 2006 with staged addition of the RTCDAs in 2009. Statistically significant year-to-year decreases in mortality were observed through 2014 ( r = -.794; P = .002). Similarly, year-to-year decreases in the number of cardiopulmonary arrests (CPAs; r = -.792; P = .006) and median lengths of stay ( r = -.841; P = .001) were observed. There was a statistically significant year-to-year increase in the number of RRS activations for these units ( r = .939; P < .001) that was inversely correlated with the occurrence of CPAs ( r = -.784; P = .007). In this single-institution retrospective study, introduction of a RRS employing RTCDAs was associated with lower hospital mortality, CPAs, and hospital length of stay.

Real-time automated clinical deterioration alerts predict thirty-day hospital readmission.

INTRODUCTION: Clinical deterioration alerts (CDAs) are increasingly employed to identify deteriorating patients. METHODS: We performed a retrospective study to determine whether CDAs predict 30-day readmission. Patients admitted to 8 general medicine units were assessed for all-cause 30-day readmission. RESULTS: Among 3015 patients, 567 (18.8%) were readmitted within 30 days. Patients triggering a CDA (n = 1141; 34.4%) were more likely to have a 30-day readmission (23.6% vs 15.9%; P < 0.001). Logistic regression identified triggering of a CDA to be independently associated with 30-day readmission (odds ratio [OR]: 1.40; 95% confidence interval [CI]: 1.26-1.55; P = 0.001). Other predictors were: an emergency department visit in the previous 6 months (OR: 1.23; 95% CI:, 1.20-1.26; P < 0.001), increasing age (OR: 1.01; 95% CI: 1.01-1.02; P = 0.003), presence of connective tissue disease (OR: 1.63; 95% CI: 1.34-1.98; P = 0.012), diabetes mellitus with end-organ complications (OR: 1.23; 95% CI: 1.13-1.33; P = 0.010), chronic renal disease (OR: 1.16; 95% CI: 1.08-1.24; P = 0.034), cirrhosis (OR: 1.25; 95% CI: 1.17-1.33; P < 0.001), and metastatic cancer (OR: 1.12; 95% CI: 1.08-1.17; P = 0.002). Addition of the CDA to the other predictors added only modest incremental value for the prediction of hospital readmission. CONCLUSIONS: Readily identifiable clinical variables can be identified that predict 30-day readmission. It may be important to include these variables in existing prediction tools if pay for performance and across-institution comparisons are to be "fair" to institutions that care for more seriously ill patients. Journal of Hospital Medicine 2016;11:768-772. © 2016 Society of Hospital Medicine.

Mortality Prediction in ICUs Using A Novel Time-Slicing Cox Regression Method.

Over the last few decades, machine learning and data mining have been increasingly used for clinical prediction in ICUs. However, there is still a huge gap in making full use of the time-series data generated from ICUs. Aiming at filling this gap, we propose a novel approach entitled Time Slicing Cox regression (TS-Cox), which extends the classical Cox regression into a classification method on multi-dimensional time-series. Unlike traditional classifiers such as logistic regression and support vector machines, our model not only incorporates the discriminative features derived from the time-series, but also naturally exploits the temporal orders of these features based on a Cox-like function. Empirical evaluation on MIMIC-II database demonstrates the efficacy of the TS-Cox model. Our TS-Cox model outperforms all other baseline models by a good margin in terms of AUC_PR, sensitivity and PPV, which indicates that TS-Cox may be a promising tool for mortality prediction in ICUs.

Prevention of inpatient hypoglycemia with a real-time informatics alert.

BACKGROUND: Severe hypoglycemia (SH), defined as a blood glucose (BG) <40 mg/dL, is associated with an increased risk of adverse clinical outcomes in inpatients. OBJECTIVE: To determine whether a predictive informatics hypoglycemia risk-alert supported by trained nurse responders would reduce the incidence of SH in our hospital. DESIGN: A 5-month prospective cohort intervention study. SETTING: Acute care medical floors in a tertiary care academic hospital in St. Louis, Missouri. PATIENTS: From 655 inpatients on designated medical floors with a BG of <90 mg/dL, 390 were identified as high risk for hypoglycemia by the alert system. MEASUREMENTS: The primary outcome was the incidence of SH occurring in high-risk intervention versus high-risk control patients. Secondary outcomes included: number of episodes of SH in all study patients, incidence of BG < 60 mg/dL and severe hyperglycemia with a BG >299 mg/dL, length of stay, transfer to a higher level of care, the frequency that high-risk patient's orders were changed in response to the alert-intervention process, and mortality. RESULTS: The alert process, when augmented by nurse-physician collaboration, resulted in a significant decrease by 68% in the rate of SH in alerted high-risk patients versus nonalerted high-risk patients (3.1% vs 9.7%, P = 0.012). Rates of hyperglycemia were similar on intervention and control floors at 28% each. There was no difference in mortality, length of stay, or patients requiring transfer to a higher level of care. CONCLUSION: A real-time predictive informatics-generated alert, when supported by trained nurse responders, significantly reduced inpatient SH.

A randomized trial of real-time automated clinical deterioration alerts sent to a rapid response team.

BACKGROUND: Episodes of patient deterioration on hospital units are expected to increasingly contribute to morbidity and healthcare costs. OBJECTIVE: To determine if real-time alerts sent to the rapid response team (RRT) improved patient care. DESIGN: Randomized, controlled trial. SETTING: Eight medicine units (Barnes-Jewish Hospital). PATIENTS: Five hundred seventy-one patients. INTERVENTION: Real-time alerts generated by a validated deterioration algorithm were sent real-time to the RRT (intervention) or hidden (control). MEASUREMENTS: Intensive care unit (ICU) transfer, hospital mortality, hospital duration. RESULTS: ICU transfer (17.8% vs 18.2%; odds ratio: 0.972; 95% confidence interval [CI]: 0.635-1.490) and hospital mortality (7.3% vs 7.7%; odds ratio: 0.947; 95% CI: 0.509-1.764) were similar for the intervention and control groups. The number of patients requiring transfer to a nursing home or long-term acute care hospital was similar for patients in the intervention and control groups (26.9% vs 26.3%; odds ratio: 1.032; 95% CI: 0.712-1.495). Hospital duration (8.4 ± 9.5 days vs 9.4 ± 11.1 days; P = 0.038) was statistically shorter for the intervention group. The number of RRT calls initiated by the primary care team was similar for the intervention and control groups (19.9% vs 16.5%; odds ratio: 1.260; 95% CI: 0.823-1.931). CONCLUSIONS: Real-time alerts sent to the RRT did not reduce ICU transfers, hospital mortality, or the need for subsequent long term care. However, hospital length of stay was modestly reduced.

A trial of a real-time alert for clinical deterioration in patients hospitalized on general medical wards.

BACKGROUND: With limited numbers of intensive care unit (ICU) beds available, increasing patient acuity is expected to contribute to episodes of inpatient deterioration on general wards. OBJECTIVE: To prospectively validate a predictive algorithm for clinical deterioration in general-medical ward patients, and to conduct a trial of real-time alerts based on this algorithm. DESIGN: Randomized, controlled crossover study. SETTING/PATIENTS: Academic center with patients hospitalized on 8 general wards between July 2007 and December 2011. INTERVENTIONS: Real-time alerts were generated by an algorithm designed to predict the need for ICU transfer using electronically available data. The alerts were sent by text page to the nurse manager on intervention wards. MEASUREMENTS: Intensive care unit transfer, hospital mortality, and hospital length of stay. RESULTS: Patients meeting the alert threshold were at nearly 5.3-fold greater risk of ICU transfer (95% confidence interval [CI]: 4.6-6.0) than those not satisfying the alert threshold (358 of 2353 [15.2%] vs 512 of 17678 [2.9%]). Patients with alerts were at 8.9-fold greater risk of death (95% CI: 7.4-10.7) than those without alerts (244 of 2353 [10.4%] vs 206 of 17678 [1.2%]). Among patients identified by the early warning system, there were no differences in the proportion of patients who were transferred to the ICU or who died in the intervention group as compared with the control group. CONCLUSIONS: Real-time alerts were highly specific for clinical deterioration resulting in ICU transfer and death, and were associated with longer hospital length of stay. However, an intervention notifying a nurse of the risk did not result in improvement in these outcomes.

Computerized surveillance for adverse drug events in a pediatric hospital.

There are limited data on adverse drug event rates in pediatrics. The authors describe the implementation and evaluation of an automated surveillance system modified to detect adverse drug events (ADEs) in pediatric patients. The authors constructed an automated surveillance system to screen admissions to a large pediatric hospital. Potential ADEs identified by the system were reviewed by medication safety pharmacists and a physician and scored for causality and severity. Over the 6 month study period, 6,889 study children were admitted to the hospital for a total of 40,250 patient-days. The ADE surveillance system generated 1226 alerts, which yielded 160 true ADEs. This represents a rate of 2.3 ADEs per 100 admissions or 4 per 1,000 patient-days. Medications most frequently implicated were diuretics, antibiotics, immunosuppressants, narcotics, and anticonvulsants. The composite positive predictive value of the ADE surveillance system was 13%. Automated surveillance can be an effective method for detecting ADEs in hospitalized children.

Computerized surveillance for adverse drug events in a pediatric hospital.

Adverse drug event (ADE) surveillance is needed to inform processes and methods for prevention. Voluntary reporting and manual chart review have limitations. Automated surveillance systems are gaining recognition and provide useful information to supplement the other methods. Preliminary evaluation of a pediatric automated adverse drug event application shows a positive predictive value of 13%, discovering events with harm in 1.3% of inpatient admissions.

Improving adherence to dyslipidemia medication guidelines in hospitalized diabetic patients using a technology-assisted pharmacist intervention.

We tested whether a technology-assisted pharmacist intervention improved physician adherence to guidelines for lipid-lowering therapy in diabetic patients. Computerized alerts identified diabetic patients above LDL-Cholesterol (LDL-C) goal. During Period 1 prescribing behavior was observed in both control and intervention physician groups without intervening. In Period 2, pharmacists conducted academic detailing with intervention group physicians. Control group physicians were observed. The intervention significantly improved the proportion of diabetic patients discharged on statin therapy.

Migrating toward a next-generation clinical decision support application: the BJC HealthCare experience.

The next-generation model outlined in the AMIA Roadmap for National Action on Clinical Decision Support (CDS) is aimed to optimize the effectiveness of CDS interventions, and to achieve widespread adoption. BJC HealthCare re-engineered its existing CDS system in alignment with the AMIA roadmap and plans to use it for guidance on further enhancements. We present our experience and discuss an incremental approach to migrate towards the next generation of CDS applications from the viewpoint of a healthcare institution. Specifically, a CDS rule engine service with a standards-based rule representation format was built to simplify maintenance and deployment. Rules were separated from execution code and made customizable for multi-facility deployment. Those changes resulted in system improvement in the short term while aligning with long-term strategic objectives.

Alerts to improve chart documentation for National Quality Measures.

Clinical decision support (CDS) rules monitoring adherence to guidelines for secondary prevention of acute myocardial infarction (AMI) have been in use at BJC HealthCare's academic facility for five years. The alert web response form for these rules was enhanced to facilitate documentation of contraindications for ACE/ARB, beta blocker, aspirin, and lipid-lowering medications. An analysis of the impact of these enhancements and the changes to pharmacy workflow are presented here.

Using BPEL to define an executable CDS rule process.

Business Process Execution Language for Web Services (BPEL) is an XML-based language used to define business process and workflow logic. While its original purpose was to provide a method of linking several disparate business applications, we have found that BPEL provides several features and advantages that lend it to incorporation in a clinical decision support (CDS) rule engine.

Implementation of virtual medical record object model for a standards-based clinical decision support rule engine.

The Virtual Medical Record (vMR) is a structured data model for representing individual patient informations. Our implementation of vMR is based on HL7 Reference Information Model (RIM) v2.13 from which a minimum set of objects and attributes are selected to meet the requirement of a clinical decision support (CDS) rule engine. Our success of mapping local patient data to the vMR model and building a vMR adaptor middle layer demonstrate the feasibility and advantages of implementing a vMR in a portable CDS solution.