Heterogeneity of Benefit from Earlier Time-to-Antibiotics for Sepsis.

Rationale: Shorter time-to-antibiotics improves survival from sepsis, particularly among patients in shock. There may be other subgroups for whom faster antibiotics are particularly beneficial.Objectives: Identify patient characteristics associated with greater benefit from shorter time-to-antibiotics.Methods: Observational cohort study of patients hospitalized with community-onset sepsis at 173 hospitals and treated with antimicrobials within 12 hours. We used three approaches to evaluate heterogeneity of benefit from shorter time-to-antibiotics: 1) conditional average treatment effects of shorter (⩽3 h) versus longer (>3-12 h) time-to-antibiotics on 30-day mortality using multivariable Poisson regression; 2) causal forest to identify characteristics associated with greatest benefit from shorter time-to-antibiotics; and 3) logistic regression with time-to-antibiotics modeled as a spline.Measurements and Main Results: Among 273,255 patients with community-onset sepsis, 131,094 (48.0%) received antibiotics within 3 hours. In Poisson models, shorter time-to-antibiotics was associated with greater absolute mortality reduction among patients with metastatic cancer (5.0% [95% confidence interval; CI: 4.3-5.7] vs. 0.4% [95% CI: 0.2-0.6] for patients without cancer, P < 0.001); patients with shock (7.0% [95% CI: 5.8-8.2%] vs. 2.8% [95% CI: 2.7-3.5%] for patients without shock, P = 0.005); and patients with more acute organ dysfunctions (4.8% [95% CI: 3.9-5.6%] for three or more dysfunctions vs. 0.5% [95% CI: 0.3-0.8] for one dysfunction, P < 0.001). In causal forest, metastatic cancer and shock were associated with greatest benefit from shorter time-to-antibiotics. Spline analysis confirmed differential nonlinear associations of time-to-antibiotics with mortality in patients with metastatic cancer and shock.Conclusions: In patients with community-onset sepsis, the mortality benefit of shorter time-to-antibiotics varied by patient characteristics. These findings suggest that shorter time-to-antibiotics for sepsis is particularly important among patients with cancer and/or shock.

Development and Validation of an Electronic Health Record-based Score for Triage to Perioperative Medicine.

OBJECTIVE: To develop an electronic health record-based risk model for perioperative medicine (POM) triage and compare this model with legacy triage practices that were based on clinician assessment. SUMMARY OF BACKGROUND DATA: POM clinicians seek to address the increasingly complex medical needs of patients prior to scheduled surgery. Identifying which patients might derive the most benefit from evaluation is challenging. METHODS: Elective surgical cases performed within a health system 2014- 2019 (N = 470,727) were used to develop a predictive score, called the Comorbidity Assessment for Surgical Triage (CAST) score, using split validation. CAST incorporates patient and surgical case characteristics to predict the risk of 30-day post-operative morbidity, defined as a composite of mortality and major NSQIP complications. Thresholds of CAST were then selected to define risk groups, which correspond with triage to POM appointments of different durations and modalities. The predictive discrimination CAST score was compared with the surgeon's assessments of patient complexity and the American Society of Anesthesiologists class. RESULTS: The CAST score demonstrated a significantly higher discrimination for predicting post-operative morbidity (area under the receiver operating characteristic curve 0.75) than the surgeon's complexity designation (0.63; P < 0.001) or the American Society of Anesthesiologists (0.65; P < 0.001) ( Fig. 1 ). Incorporating the complexity designation in the CAST model did not significantly alter the discrimination (0.75; P = 0.098). Compared with the complexity designation, classification based on CAST score groups resulted a net reclassification improvement index of 10.4% ( P < 0.001) ( Table 1 ). CONCLUSION: A parsimonious electronic health record-based predictive model demonstrates improved performance for identifying pre-surgical patients who are at risk than previously-used assessments for POM triage.

Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration.

BACKGROUND: Hospitalized adults whose condition deteriorates while they are in wards (outside the intensive care unit [ICU]) have considerable morbidity and mortality. Early identification of patients at risk for clinical deterioration has relied on manually calculated scores. Outcomes after an automated detection of impending clinical deterioration have not been widely reported. METHODS: On the basis of a validated model that uses information from electronic medical records to identify hospitalized patients at high risk for clinical deterioration (which permits automated, real-time risk-score calculation), we developed an intervention program involving remote monitoring by nurses who reviewed records of patients who had been identified as being at high risk; results of this monitoring were then communicated to rapid-response teams at hospitals. We compared outcomes (including the primary outcome, mortality within 30 days after an alert) among hospitalized patients (excluding those in the ICU) whose condition reached the alert threshold at hospitals where the system was operational (intervention sites, where alerts led to a clinical response) with outcomes among patients at hospitals where the system had not yet been deployed (comparison sites, where a patient's condition would have triggered a clinical response after an alert had the system been operational). Multivariate analyses adjusted for demographic characteristics, severity of illness, and burden of coexisting conditions. RESULTS: The program was deployed in a staggered fashion at 19 hospitals between August 1, 2016, and February 28, 2019. We identified 548,838 non-ICU hospitalizations involving 326,816 patients. A total of 43,949 hospitalizations (involving 35,669 patients) involved a patient whose condition reached the alert threshold; 15,487 hospitalizations were included in the intervention cohort, and 28,462 hospitalizations in the comparison cohort. Mortality within 30 days after an alert was lower in the intervention cohort than in the comparison cohort (adjusted relative risk, 0.84, 95% confidence interval, 0.78 to 0.90; P<0.001). CONCLUSIONS: The use of an automated predictive model to identify high-risk patients for whom interventions by rapid-response teams could be implemented was associated with decreased mortality. (Funded by the Gordon and Betty Moore Foundation and others.).

Prediction of In-hospital Mortality Among Intensive Care Unit Patients Using Modified Daily Laboratory-based Acute Physiology Score, Version 2.

BACKGROUND: Mortality prediction for intensive care unit (ICU) patients frequently relies on single ICU admission acuity measures without accounting for subsequent clinical changes. OBJECTIVE: Evaluate novel models incorporating modified admission and daily, time-updating Laboratory-based Acute Physiology Score, version 2 (LAPS2) to predict in-hospital mortality among ICU patients. RESEARCH DESIGN: Retrospective cohort study. PATIENTS: ICU patients in 5 hospitals from October 2017 through September 2019. MEASURES: We used logistic regression, penalized logistic regression, and random forest models to predict in-hospital mortality within 30 days of ICU admission using admission LAPS2 alone in patient-level and patient-day-level models, or admission and daily LAPS2 at the patient-day level. Multivariable models included patient and admission characteristics. We performed internal-external validation using 4 hospitals for training and the fifth for validation, repeating analyses for each hospital as the validation set. We assessed performance using scaled Brier scores (SBS), c -statistics, and calibration plots. RESULTS: The cohort included 13,993 patients and 107,699 ICU days. Across validation hospitals, patient-day-level models including daily LAPS2 (SBS: 0.119-0.235; c -statistic: 0.772-0.878) consistently outperformed models with admission LAPS2 alone in patient-level (SBS: 0.109-0.175; c -statistic: 0.768-0.867) and patient-day-level (SBS: 0.064-0.153; c -statistic: 0.714-0.861) models. Across all predicted mortalities, daily models were better calibrated than models with admission LAPS2 alone. CONCLUSIONS: Patient-day-level models incorporating daily, time-updating LAPS2 to predict mortality among an ICU population performs as well or better than models incorporating modified admission LAPS2 alone. The use of daily LAPS2 may offer an improved tool for clinical prognostication and risk adjustment in research in this population.

COVID-19 Vaccination and the Timing of Surgery Following COVID-19 Infection.

OBJECTIVE: To evaluate whether COVID-19 vaccination status or mode of anesthesia modified the temporal harms associated with surgery following coronavirus disease-2019 (COVID-19) infection. BACKGROUND: Surgery shortly after COVID-19 infection is associated with higher rates of complications, leading to recommendations to delay surgery following COVID-19 infection when possible. However, prior studies were based on populations with low or no prevalence of vaccination. METHODS: A retrospective cohort study of patients who underwent scheduled surgery in a health system from January 1, 2018 to February 28, 2022 (N=228,913) was performed. Patients were grouped by time of surgery relative to COVID-19 test positivity: 0 to 4 weeks after COVID-19 ("early post-COVID-19"), 4 to 8 weeks after COVID-19 ("mid post-COVID-19"), >8 weeks after COVID-19 ("late post-COVID-19"), surgery at least 30 days before subsequent COVID-19 ("pre-COVID-19"), and surgery with no prior or subsequent test positivity for COVID-19. RESULTS: Among patients who were not fully vaccinated at the time of COVID-19 infection, the adjusted rate of perioperative complications for the early post-COVID-19 group was significantly higher than for the pre-COVID-19 group (relative risk: 1.55; P =0.05). No significantly higher risk was identified between these groups for patients who were fully vaccinated (0.66; P =1.00), or for patients who were not fully vaccinated and underwent surgery without general anesthesia (0.52; P =0.83). CONCLUSIONS: Surgery shortly following COVID-19 infection was not associated with higher risks among fully vaccinated patients or among patients who underwent surgery without general anesthesia. Further research will be valuable to understand additional factors that modify perioperative risks associated with prior COVID-19 infection.

Adults hospitalized with breakthrough COVID-19 have lower mortality than matched unvaccinated adults.

BACKGROUND: Coronavirus disease 2019 (COVID-19) breakthrough infections are common. OBJECTIVE: Evaluate in-hospital mortality of patients with COVID-19 by vaccination status using retrospective cohort study. METHODS: We generated propensity scores for receipt of full vaccination in adults requiring supplemental oxygen hospitalized at Kaiser Permanente Northern California (1 April 2021 to 30 November 2021) with positive severe acute respiratory syndrome coronavirus 2 polymerase chain reaction tests. Optimal matching of fully vaccinated/unvaccinated patients was performed comparing in-hospital mortality. RESULTS: Of 7305 patients, 1463 (20.0%) were full, 138 (1.9%) were partial, and 5704 (78.1%) were unvaccinated. Fully vaccinated were older than partial or unvaccinated (71.0, 63.0, and 54.0 years, respectively, p < 0.001) with more comorbidities (Comorbidity Point Scores 33.0, 22.0, and 10.0, p < 0.001) and immunosuppressant (11.5%, 8.7%, and 3.0%, p < 0.001) or chemotherapy exposure (2.8%, 0.7%, and 0.4%, p < 0.001). Fewer fully vaccinated patients died compared to matched unvaccinated (9.0% vs. 16.3%, p < 0.0001). CONCLUSION: Fully vaccinated patients are less likely to die compared to matched unvaccinated patients.

Association between Troponin I Levels during Sepsis and Postsepsis Cardiovascular Complications.

Rationale: Sepsis commonly results in elevated serum troponin levels and increased risk for postsepsis cardiovascular complications; however, the association between troponin levels during sepsis and cardiovascular complications after sepsis is unclear.Objectives: To evaluate the association between serum troponin levels during sepsis and 1 year after sepsis cardiovascular events.Methods: We analyzed adults aged ⩾40 years without preexisting cardiovascular disease within 5 years, admitted with sepsis across 21 hospitals from 2011 to 2017. Peak serum troponin I levels during sepsis were grouped as normal (⩽0.04 ng/ml) or tertiles of abnormal (>0.04 to ⩽0.09 ng/ml, >0.09 to ⩽0.42 ng/ml, or >0.42 ng/ml). Multivariable adjusted cause-specific Cox proportional hazards models with death as a competing risk were used to assess associations between peak troponin I levels and a composite cardiovascular outcome (atherosclerotic cardiovascular disease, atrial fibrillation, and heart failure) in the year following sepsis. Models were adjusted for presepsis and intrasepsis factors considered potential confounders.Measurements and Main Results: Among 14,046 eligible adults with troponin I measured, 2,012 (14.3%) experienced the composite cardiovascular outcome, including 832 (10.9%) patients with normal troponin levels, as compared with 370 (17.3%), 376 (17.6%), and 434 (20.3%) patients within each sequential abnormal troponin tertile, respectively (P < 0.001). Patients within the elevated troponin tertiles had increased risks of adverse cardiovascular events (adjusted hazard ratio [aHR]troponin0.04-0.09 = 1.37; 95% confidence interval [CI], 1.20-1.55; aHRtroponin0.09-0.42 = 1.44; 95% CI, 1.27-1.63; and aHRtroponin>0.42 = 1.77; 95% CI, 1.56-2.00).Conclusions: Among patients without preexisting cardiovascular disease, troponin elevation during sepsis identified patients at increased risk for postsepsis cardiovascular complications. Strategies to mitigate cardiovascular complications among this high-risk subset of patients are warranted.

Prospective evaluation of social risks, physical function, and cognitive function in prediction of non-elective rehospitalization and post-discharge mortality.

BACKGROUND: Increasing evidence suggests that social factors and problems with physical and cognitive function may contribute to patients' rehospitalization risk. Understanding a patient's readmission risk may help healthcare providers develop tailored treatment and post-discharge care plans to reduce readmission and mortality. This study aimed to evaluate whether including patient-reported data on social factors; cognitive status; and physical function improves on a predictive model based on electronic health record (EHR) data alone. METHODS: We conducted a prospective study of 1,547 hospitalized adult patients in 3 Kaiser Permanente Northern California hospitals. The main outcomes were non-elective rehospitalization or death within 30 days post-discharge. Exposures included patient-reported social factors and cognitive and physical function (obtained in a pre-discharge interview) and EHR-derived data for comorbidity burden, acute physiology, care directives, prior utilization, and hospital length of stay. We performed bivariate comparisons using Chi-square, t-tests, and Wilcoxon rank-sum tests and assessed correlations between continuous variables using Spearman's rho statistic. For all models, the results reported were obtained after fivefold cross validation. RESULTS: The 1,547 adult patients interviewed were younger (age, p = 0.03) and sicker (COPS2, p < 0.0001) than the rest of the hospitalized population. Of the 6 patient-reported social factors measured, 3 (not living with a spouse/partner, transportation difficulties, health or disability-related limitations in daily activities) were significantly associated (p < 0.05) with the main outcomes, while 3 (living situation concerns, problems with food availability, financial problems) were not. Patient-reported cognitive (p = 0.027) and physical function (p = 0.01) were significantly lower in patients with the main outcomes. None of the patient-reported variables, singly or in combination, improved predictive performance of a model that included acute physiology and longitudinal comorbidity burden (area under the receiver operator characteristic curve was 0.716 for both the EHR model and maximal performance of a random forest model including all predictors). CONCLUSIONS: In this insured population, incorporating patient-reported social factors and measures of cognitive and physical function did not improve performance of an EHR-based model predicting 30-day non-elective rehospitalization or mortality. While incorporating patient-reported social and functional status data did not improve ability to predict these outcomes, such data may still be important for improving patient outcomes.

Long-Term Outcomes Among Patients Discharged From the Hospital With Moderate Anemia: A Retrospective Cohort Study.

Background: Randomized clinical trial findings support decreased red blood cell (RBC) transfusion and short-term tolerance of in-hospital anemia. However, long-term outcomes related to changes in transfusion practice have not been described. Objective: To describe the prevalence of anemia at and after hospital discharge and associated morbidity and mortality events. Design: Retrospective cohort study. Setting: Integrated health care delivery system with 21 hospitals serving 4 million members. Participants: 445 371 surviving adults who had 801 261 hospitalizations between January 2010 and December 2014. Measurements: Hemoglobin levels and RBC transfusion, rehospitalization, and mortality events within 6 months of hospital discharge. Generalized estimating equations were used to examine trends over time, accounting for correlated observations and patient-level covariates. Results: From 2010 to 2014, the prevalence of moderate anemia (hemoglobin levels between 7 and 10 g/dL) at hospital discharge increased from 20% to 25% (P < 0.001) and RBC transfusion declined by 28% (39.8 to 28.5 RBC units per 1000 patients; P < 0.001). The proportion of patients whose moderate anemia had resolved within 6 months of hospital discharge decreased from 42% to 34% (P < 0.001), and RBC transfusion and rehospitalization within 6 months of hospital discharge decreased from 19% to 17% and 37% to 33%, respectively (P < 0.001 for both). During this period, the adjusted 6-month mortality rate decreased from 16.1% to 15.6% (P = 0.004) in patients with moderate anemia, in parallel with that of all others. Limitation: Possible unmeasured confounding. Conclusion: Anemia after hospitalization increased in parallel with decreased RBC transfusion. This increase was not accompanied by a rise in subsequent RBC use, rehospitalization, or mortality within 6 months of hospital discharge. Longitudinal analyses support the safety of practice recommendations to limit RBC transfusion and tolerate anemia during and after hospitalization. Primary Funding Source: National Heart, Lung, and Blood Institute.

The Impact of Pharmacy-specific Predictors on the Performance of 30-Day Readmission Risk Prediction Models.

RESEARCH OBJECTIVE: Pharmacists are an expensive and limited resource in the hospital and outpatient setting. A pharmacist can spend up to 25% of their day planning. Time spent planning is time not spent delivering an intervention. A readmission risk adjustment model has potential to be used as a universal outcome-based prioritization tool to help pharmacists plan their interventions more efficiently. Pharmacy-specific predictors have not been used in the constructs of current readmission risk models. We assessed the impact of adding pharmacy-specific predictors on performance of readmission risk prediction models. STUDY DESIGN: We used an observational retrospective cohort study design to assess whether pharmacy-specific predictors such as an aggregate pharmacy score and drug classes would improve the prediction of 30-day readmission. A model of age, sex, length of stay, and admission category predictors was used as the reference model. We added predictor variables in sequential models to evaluate the incremental effect of additional predictors on the performance of the reference. We used logistic regression to regress the outcomes on predictors in our derivation dataset. We derived and internally validated our models through a 50:50 split validation of our dataset. POPULATION STUDIED: Our study population (n=350,810) was of adult admissions at hospitals in a large integrated health care delivery system. PRINCIPAL FINDINGS: Individually, the aggregate pharmacy score and drug classes caused a nearly identical but moderate increase in model performance over the reference. As a single predictor, the comorbidity burden score caused the greatest increase in model performance when added to the reference. Adding the severity of illness score, comorbidity burden score and the aggregate pharmacy score to the reference caused a cumulative increase in model performance with good discrimination (c statistic, 0.712; Nagelkerke R, 0.112). The best performing model included all predictors: severity of illness score, comorbidity burden score, aggregate pharmacy score, diagnosis groupings, and drug subgroups. CONCLUSIONS: Adding the aggregate pharmacy score to the reference model significantly increased the c statistic but was out-performed by the comorbidity burden score model in predicting readmission. The need for a universal prioritization tool for pharmacists may therefore be potentially met with the comorbidity burden score model. However, the aggregate pharmacy score and drug class models still out-performed current Medicare readmission risk adjustment models. IMPLICATIONS FOR POLICY OR PRACTICE: Pharmacists have a great role in preventing readmission, and therefore can potentially use one of our models: comorbidity burden score model, aggregate pharmacy score model, drug class model or complex model (a combination of all 5 major predictors) to prioritize their interventions while exceeding Medicare performance measures on readmission. The choice of model to use should be based on the availability of these predictors in the health care system.

Automated early detection of obstetric complications: theoretic and methodologic considerations.

Compared with adults who are admitted to general medical-surgical wards, women who are admitted to labor and delivery services are at much lower risk of experiencing unexpected critical illness. Nonetheless, critical illness and other complications that put either the mother or fetus at risk do occur. One potential approach to prevention is to use automated early warning systems, such as those used for nonpregnant adults. Predictive models that use data extracted in real time from electronic records constitute the cornerstone of such systems. This article addresses several issues that are involved in the development of such predictive models: specification of temporal characteristics, choice of denominator, selection of outcomes for model calibration, potential uses of existing adult severity of illness scores, approaches to data processing, statistical considerations, validation, and options for instantiation. These have not been addressed explicitly in the obstetrics literature, which has focused on the use of manually assigned scores. In addition, this article provides some results from work in progress to develop 2 obstetric predictive models with the use of data from 262,071 women who were admitted to a labor and delivery service at 15 Kaiser Permanente Northern California hospitals between 2010 and 2017.

Hospital-Acquired Pressure Injury: Risk-Adjusted Comparisons in an Integrated Healthcare Delivery System.

BACKGROUND: Although healthcare organizations have decreased hospital-acquired pressure injury (HAPI) rates, HAPIs are not eliminated, driving further examination in both nursing and health services research. OBJECTIVE: The objective was to describe HAPI incidence, risk factors, and risk-adjusted hospital variation within a California integrated healthcare system. METHODS: Inpatient episodes were included in this retrospective cohort if patients were hospitalized between January 1, 2013, and June 30, 2015. The primary outcome was development of a HAPI over time. Predictors included cited HAPI risk factors in addition to incorporation of a longitudinal comorbidity burden (Comorbidity Point Score, Version 2 [COPS2]), a severity-of-illness score (Laboratory-Based Acute Physiology Score, Version 2 [LAPS2]), and the Braden Scale for Predicting Pressure Ulcer Risk. RESULTS: Analyses included HAPI inpatient episodes (n = 1661) and non-HAPI episodes (n = 726,605). HAPI incidence was 0.57 per 1,000 patient days (95% CI [0.019, 3.805]) and 0.2% of episodes. A multivariate Cox proportional hazards model showed significant (p < .001) hazard ratios (HRs) for the change from the 25th to the 75th percentile for age (HR = 1.36, 95% CI [1.25, 1.45]), higher COPS2 scores (HR = 1.10, 95% CI [1.04, 1.16]), and higher LAPS2 scores (HR = 1.38, 95% CI [1.28, 1.50]). Female gender, an emergency room admission for a medical reason, and higher Braden scores showed significant protective HRs (HR < 1.00, p < .001). After risk adjustment, significant variation remained among the 35 hospitals. DISCUSSION: Results prompt the consideration of age, severity of illness (LAPS2), comorbidity indexes (COPS2), and the Braden score as important predictors for HAPI risk. HAPI rates may be low; however, because of significant individual site variation, HAPIs remain an area to explore through both research and quality improvement initiatives.

Risk Adjustment for Hospital Characteristics Reduces Unexplained Hospital Variation in Pressure Injury Risk.

BACKGROUND: Research investigating risk factors for hospital-acquired pressure injury (HAPI) has primarily focused on the characteristics of patients and nursing staff. Limited data are available on the association of hospital characteristics with HAPI. OBJECTIVE: We aimed to quantify the association of hospital characteristics with HAPI and their effect on residual hospital variation in HAPI risk. METHODS: We employed a retrospective cohort study design with split validation using hierarchical survival analysis. This study extends the analysis "Hospital-Acquired Pressure Injury (HAPI): Risk Adjusted Comparisons in an Integrated Healthcare Delivery System" by Rondinelli et al. (2018) to include hospital-level factors. We analyzed 1,661 HAPI episodes among 728,266 adult hospitalization episodes across 35 California Kaiser Permanente hospitals, an integrated healthcare delivery system between January 1, 2013, and June 30, 2015. RESULTS: After adjusting for patient-level and hospital-level variables, 2 out of 12 candidate hospital variables were statistically significant predictors of HAPI. The hazard for HAPI decreased by 4.8% for every 0.1% increase in a hospital's mean mortality ([6.3%, 2.6%], p < .001), whereas every 1% increase in a hospital's proportion of patients with a history of diabetes increased HAPI hazard by 5% ([-0.04%, 10.0%], p = .072). Addition of these hierarchical variables decreased unexplained hospital variation of HAPI risk by 35%. DISCUSSION: We found hospitals with higher patient mortality had lower HAPI risk. Higher patient mortality may decrease the pool of patients who live to HAPI occurrence. Such hospitals may also provide more resources (specialty staff) to care for frail patient populations. Future research should aim to combine hospital data sets to overcome power limitations at the hospital level and should investigate additional measures of structure and process related to HAPI care.

Adaptation of the Acute Organ Failure Score for Use in a Medicare Population.

OBJECTIVES: Without widely available physiologic data, a need exists for ICU risk adjustment methods that can be applied to administrative data. We sought to expand the generalizability of the Acute Organ Failure Score by adapting it to a commonly used administrative database. DESIGN: Retrospective cohort study. SETTING: One hundred fifty-one hospitals in Pennsylvania. PATIENTS: A total of 90,733 ICU admissions among 77,040 unique patients between January 1, 2009, and December 1, 2009, in the Medicare Provider Analysis and Review database. MEASUREMENTS AND MAIN RESULTS: We used multivariable logistic regression on a random split cohort to predict 30-day mortality, and to examine the impact of using different comorbidity measures in the model and adding historical claims data. Overall 30-day mortality was 17.6%. In the validation cohort, using the original Acute Organ Failure Score model's ß coefficients resulted in poor discrimination (C-statistic, 0.644; 95% CI, 0.639-0.649). The model's C-statistic improved to 0.721 (95% CI, 0.711-0.730) when the Medicare cohort was used to recalibrate the ß coefficients. Model discrimination improved further when comorbidity was expressed as the COmorbidity Point Score 2 (C-statistic, 0.737; 95% CI, 0.728-0.747; p < 0.001) or the Elixhauser index (C-statistic, 0.748; 95% CI, 0.739-0.757) instead of the Charlson index. Adding historical claims data increased the number of comorbidities identified, but did not enhance model performance. CONCLUSIONS: Modification of the Acute Organ Failure Score resulted in good model discrimination among a diverse population regardless of comorbidity measure used. This study expands the use of the Acute Organ Failure Score for risk adjustment in ICU research and outcomes reporting using standard administrative data.

Evaluation Following Staggered Implementation of the "Rethinking Critical Care" ICU Care Bundle in a Multicenter Community Setting.

OBJECTIVES: To evaluate process metrics and outcomes after implementation of the "Rethinking Critical Care" ICU care bundle in a community setting. DESIGN: Retrospective interrupted time-series analysis. SETTING: Three hospitals in the Kaiser Permanente Northern California integrated healthcare delivery system. PATIENTS: ICU patients admitted between January 1, 2009, and August 30, 2013. INTERVENTIONS: Implementation of the Rethinking Critical Care ICU care bundle which is designed to reduce potentially preventable complications by focusing on the management of delirium, sedation, mechanical ventilation, mobility, ambulation, and coordinated care. Rethinking Critical Care implementation occurred in a staggered fashion between October 2011 and November 2012. MEASUREMENTS AND MAIN RESULTS: We measured implementation metrics based on electronic medical record data and evaluated the impact of implementation on mortality with multivariable regression models for 24,886 first ICU episodes in 19,872 patients. After implementation, some process metrics (e.g., ventilation start and stop times) were achieved at high rates, whereas others (e.g., ambulation distance), available late in the study period, showed steep increases in compliance. Unadjusted mortality decreased from 12.3% to 10.9% (p < 0.01) before and after implementation, respectively. The adjusted odds ratio for hospital mortality after implementation was 0.85 (95% CI, 0.73-0.99) and for 30-day mortality was 0.88 (95% CI, 0.80-0.97) compared with before implementation. However, the mortality rate trends were not significantly different before and after Rethinking Critical Care implementation. The mean duration of mechanical ventilation and hospital stay also did not demonstrate incrementally greater declines after implementation. CONCLUSIONS: Rethinking Critical Care implementation was associated with changes in practice and a 12-15% reduction in the odds of short-term mortality. However, these findings may represent an evaluation of changes in practices and outcomes still in the midimplementation phase and cannot be directly attributed to the elements of bundle implementation.

Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU.

BACKGROUND: Patients in general medical-surgical wards who experience unplanned transfer to the intensive care unit (ICU) show evidence of physiologic derangement 6-24h prior to their deterioration. With increasing availability of electronic medical records (EMRs), automated early warning scores (EWSs) are becoming feasible. OBJECTIVE: To describe the development and performance of an automated EWS based on EMR data. MATERIALS AND METHODS: We used a discrete-time logistic regression model to obtain an hourly risk score to predict unplanned transfer to the ICU within the next 12h. The model was based on hospitalization episodes from all adult patients (18years) admitted to 21 Kaiser Permanente Northern California (KPNC) hospitals from 1/1/2010 to 12/31/2013. Eligible patients met these entry criteria: initial hospitalization occurred at a KPNC hospital; the hospitalization was not for childbirth; and the EMR had been operational at the hospital for at least 3months. We evaluated the performance of this risk score, called Advanced Alert Monitor (AAM) and compared it against two other EWSs (eCART and NEWS) in terms of their sensitivity, specificity, negative predictive value, positive predictive value, and area under the receiver operator characteristic curve (c statistic). RESULTS: A total of 649,418 hospitalization episodes involving 374,838 patients met inclusion criteria, with 19,153 of the episodes experiencing at least one outcome. The analysis data set had 48,723,248 hourly observations. Predictors included physiologic data (laboratory tests and vital signs); neurological status; severity of illness and longitudinal comorbidity indices; care directives; and health services indicators (e.g. elapsed length of stay). AAM showed better performance compared to NEWS and eCART in all the metrics and prediction intervals. The AAM AUC was 0.82 compared to 0.79 and 0.76 for eCART and NEWS, respectively. Using a threshold that generated 1 alert per day in a unit with a patient census of 35, the sensitivity of AAM was 49% (95% CI: 47.6-50.3%) compared to the sensitivities of eCART and NEWS scores of 44% (42.3-45.1) and 40% (38.2-40.9), respectively. For all three scores, about half of alerts occurred within 12h of the event, and almost two thirds within 24h of the event. CONCLUSION: The AAM score is an example of a score that takes advantage of multiple data streams now available in modern EMRs. It highlights the ability to harness complex algorithms to maximize signal extraction. The main challenge in the future is to develop detection approaches for patients in whom data are sparser because their baseline risk is lower.

Nonelective Rehospitalizations and Postdischarge Mortality: Predictive Models Suitable for Use in Real Time.

BACKGROUND: Hospital discharge planning has been hampered by the lack of predictive models. OBJECTIVE: To develop predictive models for nonelective rehospitalization and postdischarge mortality suitable for use in commercially available electronic medical records (EMRs). DESIGN: Retrospective cohort study using split validation. SETTING: Integrated health care delivery system serving 3.9 million members. PARTICIPANTS: A total of 360,036 surviving adults who experienced 609,393 overnight hospitalizations at 21 hospitals between June 1, 2010 and December 31, 2013. MAIN OUTCOME MEASURE: A composite outcome (nonelective rehospitalization and/or death within 7 or 30 days of discharge). RESULTS: Nonelective rehospitalization rates at 7 and 30 days were 5.8% and 12.4%; mortality rates were 1.3% and 3.7%; and composite outcome rates were 6.3% and 14.9%, respectively. Using data from a comprehensive EMR, we developed 4 models that can generate risk estimates for risk of the combined outcome within 7 or 30 days, either at the time of admission or at 8 AM on the day of discharge. The best was the 30-day discharge day model, which had a c-statistic of 0.756 (95% confidence interval, 0.754-0.756) and a Nagelkerke pseudo-R of 0.174 (0.171-0.178) in the validation dataset. The most important predictors-a composite acute physiology score and end of life care directives-accounted for 54% of the predictive ability of the 30-day model. Incorporation of diagnoses (not reliably available for real-time use) did not improve model performance. CONCLUSIONS: It is possible to develop robust predictive models, suitable for use in real time with commercially available EMRs, for nonelective rehospitalization and postdischarge mortality.

Accuracy of hospital standardized mortality rates: effects of model calibration.

BACKGROUND: Risk-adjusted mortality rates are commonly used in quality report cards to compare hospital performance. The risk adjustment depends on models that are assessed for goodness-of-fit using various discrimination and calibration measures. However, the relationship between model fit and the accuracy of hospital comparisons is not well characterized. OBJECTIVES: To evaluate the impact of imperfect model calibration (miscalibration) on the accuracy of hospital comparisons. METHODS: We constructed Monte Carlo simulations where a risk-adjustment model is used in a population with a different mortality distribution than in the original model. We estimated the power of calibration metrics to detect miscalibration. We estimated the sensitivity and specificity of a hospital comparisons method under different imperfect model calibration scenarios using an empirical method. RESULTS: The U-statistics showed the highest power to detect intercept and slope deviations in the calibration curve, followed by the Hosmer-Lemeshow, and the calibration intercept and slope tests. The specificity decreased with increased intercept and slope deviations and with hospital size. The effect of an imperfect model fit on sensitivity is a function of the true standardized mortality ratio, the underlying mortality rate, sample size, and observed intercept and slope deviations. Poorly performing hospitals can appear as good performers and vice versa, depending on the deviation magnitude and direction. CONCLUSIONS: Deviations from perfect model calibration have a direct impact on the accuracy of hospital comparisons. Publishing the calibration intercept and slope of risk-adjustment models would allow the users to monitor their performance against the true standard population.

Trends in red blood cell transfusion and 30-day mortality among hospitalized patients.

BACKGROUND: Blood conservation strategies have been shown to be effective in decreasing red blood cell (RBC) utilization in specific patient groups. However, few data exist describing the extent of RBC transfusion reduction or their impact on transfusion practice and mortality in a diverse inpatient population. STUDY DESIGN AND METHODS: We conducted a retrospective cohort study using comprehensive electronic medical record data from 21 medical facilities in Kaiser Permanente Northern California. We examined unadjusted and risk-adjusted RBC transfusion and 30-day mortality coincident with implementation of RBC conservation strategies. RESULTS: The inpatient study cohort included 391,958 patients who experienced 685,753 hospitalizations. From 2009 to 2013, the incidence of RBC transfusion decreased from 14.0% to 10.8% of hospitalizations; this change coincided with a decline in pretransfusion hemoglobin (Hb) levels from 8.1 to 7.6 g/dL. Decreased RBC utilization affected broad groups of admission diagnoses and was most pronounced in patients with a nadir Hb level between 8 and 9 g/dL (n = 73,057; 50.8% to 19.3%). During the study period, the standard deviation of risk-adjusted RBC transfusion incidence across hospitals decreased by 44% (p < 0.001). Thirty-day mortality did not change significantly with declines in RBC utilization in patient groups previously studied in clinical trials nor in other subgroups. CONCLUSIONS: After the implementation of blood conservation strategies, RBC transfusion incidence and pretransfusion Hb levels decreased broadly across medical and surgical patients. Variation in RBC transfusion incidence across hospitals decreased from 2010 to 2013. Consistent with clinical trial data, more restrictive transfusion practice did not appear to impact 30-day mortality.