Postimplementation Evaluation of a Machine Learning-Based Deterioration Risk Alert to Enhance Sepsis Outcome Improvements.

Machine learning-based early warning systems (EWSs) can detect clinical deterioration more accurately than point-score tools. In patients with sepsis, however, the timing and scope of sepsis interventions relative to an advanced EWS alert are not well understood. The objectives of this study were to evaluate the timing and frequency of fluid bolus therapy, new antibiotics, and Do Not Resuscitate (DNR) status relative to the time of an advanced EWS alert. We conducted 2 rounds of chart reviews of patients with an EWS alert admitted to community hospitals of a large integrated health system in Northern California (round 1: n = 21; round 2: n = 47). We abstracted patient characteristics and process measures of sepsis intervention and performed summary statistics. Sepsis decedents were older and sicker at admission and alert time. Most EWS alerts occurred near admission, and most sepsis interventions occurred before the first alert. Of 14 decedents, 12 (86%) had a DNR order before death. Fluid bolus therapy and new intravenous antibiotics frequently occurred before the alert, suggesting a potential overlap between sepsis care in the emergency department and the first alert following admission. Two tactics to minimize alerts that may not motivate new sepsis interventions are (1) locking out the alert during the immediate time after hospital admission; and (2) triaging and reviewing patients with alerts outside of the unit before activating a bedside response. Some decedents may have been on a palliative/end-of-life trajectory, because DNR orders were very common among decedents. Nurse leaders sponsoring or leading machine learning projects should consider tactics to reduce false-positive and clinically meaningless alerts dispatched to clinical staff.

Ripe for Disruption? Adopting Nurse-Led Data Science and Artificial Intelligence to Predict and Reduce Hospital-Acquired Outcomes in the Learning Health System.

Nurse leaders are dually responsible for resource stewardship and the delivery of high-quality care. However, methods to identify patient risk for hospital-acquired conditions are often outdated and crude. Although hospitals and health systems have begun to use data science and artificial intelligence in physician-led projects, these innovative methods have not seen adoption in nursing. We propose the Petri dish model, a theoretical hybrid model, which combines population ecology theory and human factors theory to explain the cost/benefit dynamics influencing the slow adoption of data science for hospital-based nursing. The proliferation of nurse-led data science in health systems may be facing several barriers: a scarcity of doctorally prepared nurse scientists with expertise in data science; internal structural inertia; an unaligned national "precision health" strategy; and a federal reimbursement landscape, which constrains-but does not negate the hard dollar business case. Nurse executives have several options: deferring adoption, outsourcing services, and investing in internal infrastructure to develop and implement risk models. The latter offers the best performing models. Progress in nurse-led data science work has been sluggish. Balanced partnerships with physician experts and organizational stakeholders are needed, as is a balanced PhD-DNP research-practice collaboration model.

Statistical Modeling and Aggregate-Weighted Scoring Systems in Prediction of Mortality and ICU Transfer: A Systematic Review.

BACKGROUND: The clinical deterioration of patientsin general hospital wards is an important safety issue. Aggregate-weighted early warning systems (EWSs) may not detect risk until patients present with acute decline. PURPOSE: We aimed to compare the prognostic test accuracy and clinical workloads generated by EWSs using statistical modeling (multivariable regression or machine learning) versus aggregate-weighted tools. DATA SOURCES: We searched PubMed and CINAHL using terms that described clinical deterioration and use of an advanced EWS. STUDY SELECTION: The outcome was clinical deterioration (intensive care unit transfer or death) of adult patients on general hospital wards. We included studies published from January 1, 2012 to September 15, 2018. DATA EXTRACTION: Following 2015 PRIMSA systematic review protocol guidelines; 2015 TRIPOD criteria for predictive model evaluation; and the Cochrane Collaboration guidelines, we reported model performance, adjusted positive predictive value (PPV), and conducted simulations of workup-to-detection ratios. DATA SYNTHESIS: Of 285 articles, six studies reported the model performance of advanced EWSs, and five were of high quality. All EWSs using statistical modeling identified at-risk patients with greater precision than aggregate-weighted EWSs (mean AUC 0.80 vs 0.73). EWSs using statistical modeling generated 4.9 alerts to find one true positive case versus 7.1 alerts in aggregate-weighted EWSs; a nearly 50% relative workload increase for aggregate-weighted EWSs. CONCLUSIONS: Compared with aggregate-weighted tools, EWSs using statistical modeling consistently demonstrated superior prognostic performance and generated less workload to identify and treat one true positive case. A standardized approach to reporting EWS model performance is needed, including outcome definitions, pretest probability, observed and adjusted PPV, and workup-to-detection ratio.

Risk Adjustment for Hospital Characteristics Reduces Unexplained Hospital Variation in Pressure Injury Risk.

BACKGROUND: Research investigating risk factors for hospital-acquired pressure injury (HAPI) has primarily focused on the characteristics of patients and nursing staff. Limited data are available on the association of hospital characteristics with HAPI. OBJECTIVE: We aimed to quantify the association of hospital characteristics with HAPI and their effect on residual hospital variation in HAPI risk. METHODS: We employed a retrospective cohort study design with split validation using hierarchical survival analysis. This study extends the analysis "Hospital-Acquired Pressure Injury (HAPI): Risk Adjusted Comparisons in an Integrated Healthcare Delivery System" by Rondinelli et al. (2018) to include hospital-level factors. We analyzed 1,661 HAPI episodes among 728,266 adult hospitalization episodes across 35 California Kaiser Permanente hospitals, an integrated healthcare delivery system between January 1, 2013, and June 30, 2015. RESULTS: After adjusting for patient-level and hospital-level variables, 2 out of 12 candidate hospital variables were statistically significant predictors of HAPI. The hazard for HAPI decreased by 4.8% for every 0.1% increase in a hospital's mean mortality ([6.3%, 2.6%], p < .001), whereas every 1% increase in a hospital's proportion of patients with a history of diabetes increased HAPI hazard by 5% ([-0.04%, 10.0%], p = .072). Addition of these hierarchical variables decreased unexplained hospital variation of HAPI risk by 35%. DISCUSSION: We found hospitals with higher patient mortality had lower HAPI risk. Higher patient mortality may decrease the pool of patients who live to HAPI occurrence. Such hospitals may also provide more resources (specialty staff) to care for frail patient populations. Future research should aim to combine hospital data sets to overcome power limitations at the hospital level and should investigate additional measures of structure and process related to HAPI care.

The Rural Inpatient Mortality Study: Does Urban-Rural County Classification Predict Hospital Mortality in California?

CONTEXT: Evidence suggests an association between rurality and decreased life expectancy. OBJECTIVE: To determine whether rural hospitals have higher hospital mortality, given that very sick patients may be transferred to regional hospitals. DESIGN: In this ecologic study, we combined Medicare hospital mortality ratings (N = 1267) with US census data, critical access hospital classification, and National Center for Health Statistics urban-rural county classifications. Ratings included mortality for coronary artery bypass grafting, stroke, chronic obstructive pulmonary disease, heart attack, heart failure, and pneumonia across 277 California hospitals between July 2011 and June 2014. We used generalized estimating equations to evaluate the association of urban-rural county classifications on mortality ratings. MAIN OUTCOME MEASURES: Unfavorable Medicare hospital mortality rating "worse than the national rate" compared with "better" or "same." RESULTS: Compared with large central "metro" (metropolitan) counties, hospitals in medium-sized metro counties had 6.4 times the odds of rating "worse than the national rate" for hospital mortality (95% confidence interval = 2.8-14.8, p < 0.001). For hospitals in small metro counties, the odds of having such a rating were 3.7 times greater (95% confidence interval = 0.7-23.4, p = 0.12), although not statistically significant. Few ratings were provided for rural counties, and analysis of rural counties was underpowered. CONCLUSION: Hospitals in medium-sized metro counties are associated with unfavorable Medicare mortality ratings, but current methods to assign mortality ratings may hinder fair comparisons. Patient transfers from rural locations to regional medical centers may contribute to these results, a potential factor that future research should examine.

Many emergency department patients with severe sepsis and septic shock do not meet diagnostic criteria within 3 hours of arrival.

STUDY OBJECTIVE: Proposed national performance measures for severe sepsis or septic shock include interventions within 3 hours of emergency department (ED) arrival rather than from time of first meeting diagnostic criteria. We aim to determine the percentage of ED patients who first meet criteria greater than 3 hours after arrival. METHODS: We conducted a retrospective analysis of adult patients with severe sepsis or septic shock in 2 EDs (university hospital [September 2012 to June 2013] and public trauma center [December 2012 to May 2013]). Times of ED arrival and first meeting clinical criteria were collected for quality assurance programs, which differed between institutions. At the university hospital, patients with admission diagnoses consistent with infection were included. Clinical presentation was defined as time meeting 2 or more systemic inflammatory response syndrome criteria and evidence of end-organ dysfunction. At the trauma center, only patients with hospital discharge diagnoses consistent with infection were included. Clinical presentation was defined by time of end-organ dysfunction. RESULTS: Three hundred seventy-two patients met inclusion criteria at the university hospital and 133 at the trauma center. Median times from ED arrival to first meeting criteria were 68 minutes (interquartile range 34 to 130 minutes) and 31 minutes (interquartile range 8 to 73 minutes), respectively; 15.3% (95% confidence interval 11.9% to 19.3%) and 9.8% (95% confidence interval 5.5% to 15.7%) first met criteria greater than 3 hours from ED arrival, respectively. CONCLUSION: Compliance with a performance metric for severe sepsis and septic shock within 3 hours of ED arrival would require application of this measure to patients who do not meet diagnostic criteria, potentially resulting in unnecessary interventions. Measure developers should consider these findings.