Validation of a Multivariable Model to Predict Suicide Attempt in a Mental Health Intake Sample.

Importance: Given that suicide rates have been increasing over the past decade and the demand for mental health care is at an all-time high, targeted prevention efforts are needed to identify individuals seeking to initiate mental health outpatient services who are at high risk for suicide. Suicide prediction models have been developed using outpatient mental health encounters, but their performance among intake appointments has not been directly examined. Objective: To assess the performance of a predictive model of suicide attempts among individuals seeking to initiate an episode of outpatient mental health care. Design, Setting, and Participants: This prognostic study tested the performance of a previously developed machine learning model designed to predict suicide attempts within 90 days of any mental health outpatient visit. All mental health intake appointments scheduled between January 1, 2012, and April 1, 2022, at Kaiser Permanente Northern California, a large integrated health care delivery system serving over 4.5 million patients, were included. Data were extracted and analyzed from August 9, 2022, to July 31, 2023. Main Outcome and Measures: Suicide attempts (including completed suicides) within 90 days of the appointment, determined by diagnostic codes and government databases. All predictors were extracted from electronic health records. Results: The study included 1 623 232 scheduled appointments from 835 616 unique patients. There were 2800 scheduled appointments (0.17%) followed by a suicide attempt within 90 days. The mean (SD) age across appointments was 39.7 (15.8) years, and most appointments were for women (1 103 184 [68.0%]). The model had an area under the receiver operating characteristic curve of 0.77 (95% CI, 0.76-0.78), an area under the precision-recall curve of 0.02 (95% CI, 0.02-0.02), an expected calibration error of 0.0012 (95% CI, 0.0011-0.0013), and sensitivities of 37.2% (95% CI, 35.5%-38.9%) and 18.8% (95% CI, 17.3%-20.2%) at specificities of 95% and 99%, respectively. The 10% of appointments at the highest risk level accounted for 48.8% (95% CI, 47.0%-50.6%) of the appointments followed by a suicide attempt. Conclusions and Relevance: In this prognostic study involving mental health intakes, a previously developed machine learning model of suicide attempts showed good overall classification performance. Implementation research is needed to determine appropriate thresholds and interventions for applying the model in an intake setting to target high-risk cases in a manner that is acceptable to patients and clinicians.

Trends in Outpatient Visits and Hospital and Intensive Care Unit Admissions of Adults With COVID-19 in an Integrated US Health Care System, March 2020 to January 2022.

This cohort study of patients at a single integrated health system examines trends in COVID-19­related treatment location and mortality.

Evaluation of Outcomes Following Hospital-Wide Implementation of a Subcutaneous Insulin Protocol for Diabetic Ketoacidosis.

Importance: Standard diabetic ketoacidosis care in the US includes intravenous insulin treatment in the intensive care unit. Subcutaneous (SQ) insulin could decrease intensive care unit need, but the data are limited. Objective: To assess outcomes after implementation of an SQ insulin protocol for treating diabetic ketoacidosis. Design, Setting, and Participants: This cohort study is a retrospective evaluation of a prospectively implemented SQ insulin protocol. The study was conducted at an integrated health care system in Northern California. Participants included hospitalized patients with diabetic ketoacidosis at 21 hospitals between January 1, 2010, and December 31, 2019. The preimplementation phase was 2010 to 2015, and the postimplementation phase was 2017 to 2019. Data analysis was performed from October 2020 to January 2022. Exposure: An SQ insulin treatment protocol for diabetic ketoacidosis. Main Outcomes and Measures: Difference-in-differences evaluation of the need for intensive care, mortality, readmission, and length of stay at a single intervention site using an SQ insulin protocol from 2017 onward compared with 20 control hospitals using standard care. Results: A total of 7989 hospitalizations for diabetic ketoacidosis occurred, with 4739 (59.3%) occurring before and 3250 (40.7%) occurring after implementation. The overall mean (SD) age was 42.3 (17.7) years, with 4137 hospitalizations (51.8%) occurring among female patients. Before implementation, SQ insulin was the first insulin used in 40 intervention (13.4%) and 651 control (14.7%) hospitalizations. After implementation, 98 hospitalizations (80.3%) received SQ insulin first at the intervention site compared with 402 hospitalizations (12.8%) at control sites. The adjusted rate ratio for intensive care unit admission was 0.43 (95% CI, 0.33-0.56) at the intervention sites, a 57% reduction compared with control sites, and was 0.50 (95% CI, 0.25-0.99) for 30-day hospital readmission, a 50% reduction. There were no significant changes in hospital length of stay and rates of death. Conclusions and Relevance: These findings suggest that a protocol based on SQ insulin for diabetic ketoacidosis treatment was associated with significant decreases in intensive care unit need and readmission, with no evidence of increases in adverse events.

Prognostic Accuracy of Presepsis and Intrasepsis Characteristics for Prediction of Cardiovascular Events After a Sepsis Hospitalization.

OBJECTIVES: Sepsis survivors face increased risk for cardiovascular complications; however, the contribution of intrasepsis events to cardiovascular risk profiles is unclear. SETTING: Kaiser Permanente Northern California (KPNC) and Intermountain Healthcare (IH) integrated healthcare delivery systems. SUBJECTS: Sepsis survivors (2011-2017 [KPNC] and 2018-2020 [IH]) greater than or equal to 40 years old without prior cardiovascular disease. DESIGN: Data across KPNC and IH were harmonized and grouped into presepsis (demographics, atherosclerotic cardiovascular disease scores, comorbidities) or intrasepsis factors (e.g., laboratory values, vital signs, organ support, infection source) with random split for training/internal validation datasets (75%/25%) within KPNC and IH. Models were bidirectionally, externally validated between healthcare systems. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Changes to predictive accuracy (C-statistic) of cause-specific proportional hazards models predicting 1-year cardiovascular outcomes (atherosclerotic cardiovascular disease, heart failure, and atrial fibrillation events) were compared between models that did and did not contain intrasepsis factors. Among 39,590 KPNC and 16,388 IH sepsis survivors, 3,503 (8.8%) at Kaiser Permanente (KP) and 600 (3.7%) at IH experienced a cardiovascular event within 1-year after hospital discharge, including 996 (2.5%) at KP and 192 (1.2%) IH with an atherosclerotic event first, 564 (1.4%) at KP and 117 (0.7%) IH with a heart failure event, 2,310 (5.8%) at KP and 371 (2.3%) with an atrial fibrillation event. Death within 1 year after sepsis occurred for 7,948 (20%) KP and 2,085 (12.7%) IH patients. Combined models with presepsis and intrasepsis factors had better discrimination for cardiovascular events (KPNC C-statistic 0.783 [95% CI, 0.766-0.799]; IH 0.763 [0.726-0.801]) as compared with presepsis cardiovascular risk alone (KPNC: 0.666 [0.648-0.683], IH 0.660 [0.619-0.702]) during internal validation. External validation of models across healthcare systems showed similar performance (KPNC model within IH data C-statistic: 0.734 [0.725-0.744]; IH model within KPNC data: 0.787 [0.768-0.805]). CONCLUSIONS: Across two large healthcare systems, intrasepsis factors improved postsepsis cardiovascular risk prediction as compared with presepsis cardiovascular risk profiles. Further exploration of sepsis factors that contribute to postsepsis cardiovascular events is warranted for improved mechanistic and predictive models.

Outcomes After Implementation of a Benzodiazepine-Sparing Alcohol Withdrawal Order Set in an Integrated Health Care System.

Importance: Alcohol withdrawal syndrome (AWS) is a common inpatient diagnosis managed primarily with benzodiazepines. Concerns about the adverse effects associated with benzodiazepines have spurred interest in using benzodiazepine-sparing treatments. Objective: To evaluate changes in outcomes after implementation of a benzodiazepine-sparing AWS inpatient order set that included adjunctive therapies (eg, gabapentin, valproic acid, clonidine, and dexmedetomidine). Design, Setting, and Participants: This difference-in-differences quality improvement study was conducted among 22 899 AWS adult hospitalizations from October 1, 2014, to September 30, 2019, in the Kaiser Permanente Northern California integrated health care delivery system. Data were analyzed from September 2020 through November 2021. Exposures: Implementation of the benzodiazepine-sparing AWS order set on October 1, 2018. Main Outcomes and Measures: Adjusted rate ratios for medication use, inpatient mortality, length of stay, intensive care unit admission, and nonelective readmission within 30 days were calculated comparing postimplementation and preimplementation periods among hospitals with and without order set use. Results: Among 904 540 hospitalizations in the integrated health care delivery system during the study period, AWS was present in 22 899 hospitalizations (2.5%), occurring among 16 323 unique patients (mean [SD] age, 57.1 [14.8] years; 15 764 [68.8%] men). Of these hospitalizations, 12 889 (56.3%) used an order set for alcohol withdrawal. Among hospitalizations with order set use, any benzodiazepine use decreased after implementation from 6431 hospitalizations (78.1%) to 2823 hospitalizations (60.7%) (P < .001), with concomitant decreases in the mean (SD) total dosage of lorazepam before vs after implementation (19.7 [38.3] mg vs 6.0 [9.1] mg; P < .001). There were also significant changes from before to after implementation in the use of adjunctive medications, including gabapentin (2413 hospitalizations [29.3%] vs 2814 hospitalizations [60.5%]; P < .001), clonidine (1476 hospitalizations [17.9%] vs 2208 hospitalizations [47.5%]; P < .001), thiamine (6298 hospitalizations [76.5%] vs 4047 hospitalizations [87.0%]; P < .001), valproic acid (109 hospitalizations [1.3%] vs 256 hospitalizations [5.5%]; P < .001), and phenobarbital (412 hospitalizations [5.0%] vs 292 hospitalizations [6.3%]; P = .003). Compared with AWS hospitalizations without order set use, use of the benzodiazepine-sparing order set was associated with decreases in intensive care unit use (adjusted rate ratio [ARR], 0.71; 95% CI, 0.56-0.89; P = .003) and hospital length of stay (ARR, 0.71; 95% CI, 0.58-0.86; P < .001). Conclusions and Relevance: This study found that implementation of a benzodiazepine-sparing AWS order set was associated with decreased use of benzodiazepines and favorable trends in outcomes. These findings suggest that further prospective research is needed to identify the most effective treatments regimens for patients hospitalized with alcohol withdrawal.

Evaluation of Incident 7-Day Infection and Sepsis Hospitalizations in an Integrated Health System.

Rationale: Prehospital opportunities to predict infection and sepsis hospitalization may exist, but little is known about their incidence following common healthcare encounters. Objectives: To evaluate the incidence and timing of infection and sepsis hospitalization within 7 days of living hospital discharge, emergency department discharge, and ambulatory visit settings. Methods: In each setting, we identified patients in clinical strata based on the presence of infection and severity of illness. We estimated number needed to evaluate values with hypothetical predictive model operating characteristics. Results: We identified 97,614,228 encounters, including 1,117,702 (1.1%) hospital discharges, 4,635,517 (4.7%) emergency department discharges, and 91,861,009 (94.1%) ambulatory visits between 2012 and 2017. The incidence of 7-day infection hospitalization varied from 37,140 (3.3%) following inpatient discharge to 50,315 (1.1%) following emergency department discharge and 277,034 (0.3%) following ambulatory visits. The incidence of 7-day infection hospitalization was increased for inpatient discharges with high readmission risk (10.0%), emergency department discharges with increased acute or chronic severity of illness (3.5% and 4.7%, respectively), and ambulatory visits with acute infection (0.7%). The timing of 7-day infection and sepsis hospitalizations differed across settings with an early rise following ambulatory visits, a later peak following emergency department discharges, and a delayed peak following inpatient discharge. Theoretical number needed to evaluate values varied by strata, but following hospital and emergency department discharge, were as low as 15-25. Conclusions: Incident 7-day infection and sepsis hospitalizations following encounters in routine healthcare settings were surprisingly common and may be amenable to clinical predictive models.

Development and Validation of an Obstetric Comorbidity Risk Score for Clinical Use.

Background: A comorbidity summary score may support early and systematic identification of women at high risk for adverse obstetric outcomes. The objective of this study was to conduct the initial development and validation of an obstetrics comorbidity risk score for automated implementation in the electronic health record (EHR) for clinical use. Methods: The score was developed and validated using EHR data for a retrospective cohort of pregnancies with delivery between 2010 and 2018 at Kaiser Permanente Northern California, an integrated health care system. The outcome used for model development consisted of adverse obstetric events from delivery hospitalization (e.g., eclampsia, hemorrhage, death). Candidate predictors included maternal age, parity, multiple gestation, and any maternal diagnoses assigned in health care encounters in the 12 months before admission for delivery. We used penalized regression for variable selection, logistic regression to fit the model, and internal validation for model evaluation. We also evaluated prenatal model performance at 18 weeks of pregnancy. Results: The development cohort (n = 227,405 pregnancies) had an outcome rate of 3.8% and the validation cohort (n = 41,683) had an outcome rate of 2.9%. Of 276 candidate predictors, 37 were included in the final model. The final model had a validation c-statistic of 0.72 (95% confidence interval [CI] 0.70-0.73). When evaluated at 18 weeks of pregnancy, discrimination was modestly diminished (c-statistic 0.68 [95% CI 0.67-0.70]). Conclusions: The obstetric comorbidity score demonstrated good discrimination for adverse obstetric outcomes. After additional appropriate validation, the score can be automated in the EHR to support early identification of high-risk women and assist efforts to ensure risk-appropriate maternal care.

Development of a healthcare system COVID Hotspotting Score in California: an observational study with prospective validation.

OBJECTIVE: To examine the value of health systems data as indicators of emerging COVID-19 activity. DESIGN: Observational study of health system indicators for the COVID Hotspotting Score (CHOTS) with prospective validation. SETTING AND PARTICIPANTS: An integrated healthcare delivery system in Northern California including 21 hospitals and 4.5 million members. MAIN OUTCOME MEASURES: The CHOTS incorporated 10 variables including four major (cough/cold calls, emails, new positive COVID-19 tests, COVID-19 hospital census) and six minor (COVID-19 calls, respiratory infection and COVID-19 routine and urgent visits, and respiratory viral testing) indicators assessed with change point detection and slope metrics. We quantified cross-correlations lagged by 7-42 days between CHOTS and standardised COVID-19 hospital census using observational data from 1 April to 31 May 2020 and two waves of prospective data through 21 March 2021. RESULTS: Through 30 September 2020, peak cross-correlation between CHOTS and COVID-19 hospital census occurred with a 28-day lag at 0.78; at 42 days, the correlation was 0.69. Lagged correlation between medical centre CHOTS and their COVID-19 census was highest at 42 days for one facility (0.63), at 35 days for nine facilities (0.52-0.73), at 28 days for eight facilities (0.28-0.74) and at 14 days for two facilities (0.73-0.78). The strongest correlation for individual indicators was 0.94 (COVID-19 census) and 0.90 (new positive COVID-19 tests) lagged 1-14 days and 0.83 for COVID-19 calls and urgent clinic visits lagged 14-28 days. Cross-correlation was similar (0.73) with a 35-day lag using prospective validation from 1 October 2020 to 21 March 2021. CONCLUSIONS: Passively collected health system indicators were strongly correlated with forthcoming COVID-19 hospital census up to 6 weeks before three successive COVID-19 waves. These tools could inform communities, health systems and public health officials to identify, prepare for and mitigate emerging COVID-19 activity.

Evaluation of Vaccination Strategies to Compare Efficient and Equitable Vaccine Allocation by Race and Ethnicity Across Time.

Importance: Identifying the most efficient COVID-19 vaccine allocation strategy may substantially reduce hospitalizations and save lives while ensuring an equitable vaccine distribution. Objective: To simulate the association of different vaccine allocation strategies with COVID-19-associated morbidity and mortality and their distribution across racial and ethnic groups. Design Setting and Participants: We developed and internally validated the risk of COVID-19 infection and risk of hospitalization models on randomly split training and validation data sets. These were used in a computer simulation study of vaccine prioritization among adult health plan members who were drawn from an integrated health care delivery system. The study was conducted from January 3, 2021, to June 1, 2021, in Oakland, California, and the data were analyzed during the same period. Main Outcomes and Measures: We simulated the association of different vaccine allocation strategies, including (1) random, (2) a US Centers for Disease Control and Prevention (CDC) proxy, (3) age based, and (4) combinations of models for the risk of adverse outcomes (CRS) and COVID-19 infection (PROVID), with COVID-19-related hospitalizations between May 1, 2020, and December 31, 2020, that were randomly permuted by month across 250 simulations and assessed vaccine allocation by race and ethnicity and the neighborhood deprivation index across time. Results: The study included 3 202 679 adult patients (mean [SD] age, 48.2 [18.0] years; 1 677 637 women [52.4%]; 1 525 042 men [47.6%]; 611 154 Asian [19.1%], 206 363 Black [6.4%], 642 344 Hispanic [20.1%], and 1 390 638 White individuals [43.4%]), of whom 36 137 (1.1%) were positive for SARS-CoV-2. A risk-based strategy (CRS/PROVID) showed the largest avoidable hospitalization estimates (4954; 95% CI, 3452-5878) followed by age-based (4362; 95% CI, 2866-5175) and CDC proxy (4085; 95% CI, 2805-5109) strategies. Random vaccination showed substantially lower reductions in adverse outcomes. Risk-based strategies also showed the largest number of avoidable COVID-19 deaths (joint CRS/PROVID) and household transmissions. Risk-based (PROVID) and CDC proxy strategies were estimated to vaccinate the highest percentage of Hispanic and Black patients in 8 months (joint CRS/PROVID: 642 570 [100%] Hispanic, 185 530 [90%] Black; PROVID: 642 570 [100%] Hispanic, 198 480 [96%] Black; CDC proxy: 605 770 [95%] Hispanic and 151 772 [74%] Black) compared with an age-based approach (438 423 [68%] Hispanic, 154 714 [75%] Black). Overall, the PROVID and joint CRS/PROVID risk-based strategies were estimated to be followed by the most patients from areas with high neighborhood deprivation index being vaccinated early. Conclusions and Relevance: In this simulation modeling study of adults from a large integrated health care delivery system, risk-based strategies were associated with the largest estimated reductions in COVID-19 hospitalizations, deaths, and household transmissions compared with the CDC proxy and age-based strategies, with a higher proportion of Hispanic and Black patients were estimated to be vaccinated early in the process compared with the CDC strategy.

Assessment of Value of Neighborhood Socioeconomic Status in Models That Use Electronic Health Record Data to Predict Health Care Use Rates and Mortality.

Importance: Prediction models are widely used in health care as a way of risk stratifying populations for targeted intervention. Most risk stratification has been done using a small number of predictors from insurance claims. However, the utility of diverse nonclinical predictors, such as neighborhood socioeconomic contexts, remains unknown. Objective: To assess the value of using neighborhood socioeconomic predictors in the context of 1-year risk prediction for mortality and 6 different health care use outcomes in a large integrated care system. Design, Setting, and Participants: Diagnostic study using data from all adults age 18 years or older who had Kaiser Foundation Health Plan membership and/or use in the Kaiser Permantente Northern California: a multisite, integrated health care delivery system between January 1, 2013, and June 30, 2014. Data were recorded before the index date for each patient to predict their use and mortality in a 1-year post period using a test-train split for model training and evaluation. Analyses were conducted in fall of 2019. Main Outcomes and Measures: One-year encounter counts (doctor office, virtual, emergency department, elective hospitalizations, and nonelective), total costs, and mortality. Results: A total of 2 951 588 patients met inclusion criteria (mean [SD] age, 47.2 [17.4] years; 47.8% were female). The mean (SD) Neighborhood Deprivation Index was -0.32 (0.84). The areas under the receiver operator curve ranged from 0.71 for emergency department use (using the LASSO method and electronic health record predictors) to 0.94 for mortality (using the random forest method and electronic health record predictors). Neighborhood socioeconomic status predictors did not meaningfully increase the predictive performance of the models for any outcome. Conclusions and Relevance: In this study, neighborhood socioeconomic predictors did not improve risk estimates compared with what is obtainable using standard claims data regardless of model used.

Multiyear Rehospitalization Rates and Hospital Outcomes in an Integrated Health Care System.

Importance: Since the introduction of the rehospitalization rate as a quality measure, multiple changes have taken place in the US health care delivery system. Interpreting rehospitalization rates without taking a global view of these changes and new data elements from comprehensive electronic medical records yields a limited assessment of the quality of care. Objective: To examine hospitalization outcomes from a broad perspective, including the implications of numerator and denominator definitions, all adult patients with all diagnoses, and detailed clinical data. Design, Setting, and Participants: This cohort study obtained data from 21 hospitals in Kaiser Permanente Northern California (KPNC), an integrated health care delivery system that serves patients with Medicare Advantage plans, Medicaid, and/or Kaiser Foundation Health Plan. The KPNC electronic medical record system was used to capture hospitalization data for adult patients who were 18 years of age or older; discharged from June 1, 2010, through December 31, 2017; and hospitalized for reasons other than childbirth. Hospital stays for transferred patients were linked using public and internal sources. Exposures: Hospitalization type (inpatient, for observation only), comorbidity burden, acute physiology score, and care directives. Main Outcomes and Measures: Mortality (inpatient, 30-day, and 30-day postdischarge), nonelective rehospitalization, and discharge disposition (home, home with home health assistance, regular skilled nursing facility, or custodial skilled nursing facility). Results: In total, 1 384 025 hospitalizations were identified, of which 1 155 034 (83.5%) were inpatient and 228 991 (16.5%) were for observation only. These hospitalizations involved 679 831 patients (mean [SD] age, 61.4 [18.1] years; 362 582 female [53.3%]). The number of for-observation-only hospitalizations increased from 16 497 (9.4%) in the first year of the study to 120 215 (20.5%) in the last period of the study, whereas inpatient hospitalizations with length of stay less than 24 hours decreased by 33% (from 12 008 [6.9%] to 27 108 [4.6%]). Illness burden measured using administrative data or acute physiology score increased significantly. The proportion of patients with a Comorbidity Point Score of 65 or higher increased from 20.5% (range across hospitals, 18.4%-26.4%) to 28.8% (range, 22.3%-33.0%), as did the proportion with a Charlson Comorbidity Index score of 4 or higher, which increased from 28.8% (range, 24.6%-35.0%) to 38.4% (range, 31.9%-43.4%). The proportion of patients at or near critical illness (Laboratory-based Acute Physiology Score [LAPS2] ≥110) increased by 21.4% (10.3% [range across hospitals, 7.4%-14.7%] to 12.5% [range across hospitals, 8.3%-16.6%]; P < .001), reflecting a steady increase of 0.07 (95% CI, 0.04-0.10) LAPS2 points per month. Unadjusted inpatient mortality in the first year of the study was 2.78% and in the last year was 2.71%; the corresponding numbers for 30-day mortality were 5.88% and 6.15%, for 30-day postdischarge mortality were 3.94% and 4.22%, and for nonelective rehospitalization were 12.00% and 12.81%, respectively. All outcomes improved after risk adjustment. Compared with the first month, the final observed to expected ratio was 0.79 (95% CI, 0.73-0.84) for inpatient mortality, 0.86 (95% CI, 0.82-0.89) for 30-day mortality, 0.90 (95% CI, 0.85-0.95) for 30-day nonelective rehospitalization, and 0.87 (95% CI, 0.83-0.92) for 30-day postdischarge mortality. The proportion of nonelective rehospitalizations meeting public reporting criteria decreased substantially over the study period (from 58.0% in 2010-2011 to 45.2% in 2017); most of this decrease was associated with the exclusion of observation stays. Conclusions and Relevance: This study found that in this integrated system, the hospitalization rate decreased and risk-adjusted hospital outcomes improved steadily over the 7.5-year study period despite worsening case mix. The comprehensive results suggest that future assessments of care quality should consider the implications of numerator and denominator definitions, display multiple metrics concurrently, and include all hospitalization types and detailed data.

The Impact of Pharmacy-specific Predictors on the Performance of 30-Day Readmission Risk Prediction Models.

RESEARCH OBJECTIVE: Pharmacists are an expensive and limited resource in the hospital and outpatient setting. A pharmacist can spend up to 25% of their day planning. Time spent planning is time not spent delivering an intervention. A readmission risk adjustment model has potential to be used as a universal outcome-based prioritization tool to help pharmacists plan their interventions more efficiently. Pharmacy-specific predictors have not been used in the constructs of current readmission risk models. We assessed the impact of adding pharmacy-specific predictors on performance of readmission risk prediction models. STUDY DESIGN: We used an observational retrospective cohort study design to assess whether pharmacy-specific predictors such as an aggregate pharmacy score and drug classes would improve the prediction of 30-day readmission. A model of age, sex, length of stay, and admission category predictors was used as the reference model. We added predictor variables in sequential models to evaluate the incremental effect of additional predictors on the performance of the reference. We used logistic regression to regress the outcomes on predictors in our derivation dataset. We derived and internally validated our models through a 50:50 split validation of our dataset. POPULATION STUDIED: Our study population (n=350,810) was of adult admissions at hospitals in a large integrated health care delivery system. PRINCIPAL FINDINGS: Individually, the aggregate pharmacy score and drug classes caused a nearly identical but moderate increase in model performance over the reference. As a single predictor, the comorbidity burden score caused the greatest increase in model performance when added to the reference. Adding the severity of illness score, comorbidity burden score and the aggregate pharmacy score to the reference caused a cumulative increase in model performance with good discrimination (c statistic, 0.712; Nagelkerke R, 0.112). The best performing model included all predictors: severity of illness score, comorbidity burden score, aggregate pharmacy score, diagnosis groupings, and drug subgroups. CONCLUSIONS: Adding the aggregate pharmacy score to the reference model significantly increased the c statistic but was out-performed by the comorbidity burden score model in predicting readmission. The need for a universal prioritization tool for pharmacists may therefore be potentially met with the comorbidity burden score model. However, the aggregate pharmacy score and drug class models still out-performed current Medicare readmission risk adjustment models. IMPLICATIONS FOR POLICY OR PRACTICE: Pharmacists have a great role in preventing readmission, and therefore can potentially use one of our models: comorbidity burden score model, aggregate pharmacy score model, drug class model or complex model (a combination of all 5 major predictors) to prioritize their interventions while exceeding Medicare performance measures on readmission. The choice of model to use should be based on the availability of these predictors in the health care system.

Measuring Population Health in a Large Integrated Health System to Guide Goal Setting and Resource Allocation: A Proof of Concept.

In integrated health care systems, techniques that identify successes and opportunities for targeted improvement are needed. The authors propose a new method for estimating population health that provides a more accurate and dynamic assessment of performance and priority setting. Member data from a large integrated health system (n = 96,246, 73.8% female, mean age = 44 ± 0.01 years) were used to develop a mechanistic mathematical simulation, representing the top causes of US mortality in 2014 and their associated risk factors. An age- and sex-matched US cohort served as comparator group. The simulation was recalibrated and retested for validity employing the outcome measure of 5-year mortality. The authors sought to estimate potential population health that could be gained by improving health risk factors in the study population. Potential gains were assessed using both average life years (LY) gained and average quality-adjusted life years (QALYs) gained. The simulation validated well compared to integrated health system data, producing an AUC (area under the curve) of 0.88 for 5-year mortality. Current population health was estimated as a life expectancy of 84.7 years or 69.2 QALYs. Comparing potential health gain in the US cohort to the Kaiser Permanente cohort, eliminating physical inactivity, unhealthy diet, smoking, and uncontrolled diabetes resulted in an increase of 1.5 vs. 1.3 LY, 1.1 vs. 0.8 LY, 0.5 vs. 0.2 LY, and 0.5 vs. 0.5 LY on average per person, respectively. Using mathematical simulations may inform efforts by integrated health systems to target resources most effectively, and may facilitate goal setting.

Long-Term Outcomes Among Patients Discharged From the Hospital With Moderate Anemia: A Retrospective Cohort Study.

Background: Randomized clinical trial findings support decreased red blood cell (RBC) transfusion and short-term tolerance of in-hospital anemia. However, long-term outcomes related to changes in transfusion practice have not been described. Objective: To describe the prevalence of anemia at and after hospital discharge and associated morbidity and mortality events. Design: Retrospective cohort study. Setting: Integrated health care delivery system with 21 hospitals serving 4 million members. Participants: 445 371 surviving adults who had 801 261 hospitalizations between January 2010 and December 2014. Measurements: Hemoglobin levels and RBC transfusion, rehospitalization, and mortality events within 6 months of hospital discharge. Generalized estimating equations were used to examine trends over time, accounting for correlated observations and patient-level covariates. Results: From 2010 to 2014, the prevalence of moderate anemia (hemoglobin levels between 7 and 10 g/dL) at hospital discharge increased from 20% to 25% (P < 0.001) and RBC transfusion declined by 28% (39.8 to 28.5 RBC units per 1000 patients; P < 0.001). The proportion of patients whose moderate anemia had resolved within 6 months of hospital discharge decreased from 42% to 34% (P < 0.001), and RBC transfusion and rehospitalization within 6 months of hospital discharge decreased from 19% to 17% and 37% to 33%, respectively (P < 0.001 for both). During this period, the adjusted 6-month mortality rate decreased from 16.1% to 15.6% (P = 0.004) in patients with moderate anemia, in parallel with that of all others. Limitation: Possible unmeasured confounding. Conclusion: Anemia after hospitalization increased in parallel with decreased RBC transfusion. This increase was not accompanied by a rise in subsequent RBC use, rehospitalization, or mortality within 6 months of hospital discharge. Longitudinal analyses support the safety of practice recommendations to limit RBC transfusion and tolerate anemia during and after hospitalization. Primary Funding Source: National Heart, Lung, and Blood Institute.

Risk Adjustment for Hospital Characteristics Reduces Unexplained Hospital Variation in Pressure Injury Risk.

BACKGROUND: Research investigating risk factors for hospital-acquired pressure injury (HAPI) has primarily focused on the characteristics of patients and nursing staff. Limited data are available on the association of hospital characteristics with HAPI. OBJECTIVE: We aimed to quantify the association of hospital characteristics with HAPI and their effect on residual hospital variation in HAPI risk. METHODS: We employed a retrospective cohort study design with split validation using hierarchical survival analysis. This study extends the analysis "Hospital-Acquired Pressure Injury (HAPI): Risk Adjusted Comparisons in an Integrated Healthcare Delivery System" by Rondinelli et al. (2018) to include hospital-level factors. We analyzed 1,661 HAPI episodes among 728,266 adult hospitalization episodes across 35 California Kaiser Permanente hospitals, an integrated healthcare delivery system between January 1, 2013, and June 30, 2015. RESULTS: After adjusting for patient-level and hospital-level variables, 2 out of 12 candidate hospital variables were statistically significant predictors of HAPI. The hazard for HAPI decreased by 4.8% for every 0.1% increase in a hospital's mean mortality ([6.3%, 2.6%], p < .001), whereas every 1% increase in a hospital's proportion of patients with a history of diabetes increased HAPI hazard by 5% ([-0.04%, 10.0%], p = .072). Addition of these hierarchical variables decreased unexplained hospital variation of HAPI risk by 35%. DISCUSSION: We found hospitals with higher patient mortality had lower HAPI risk. Higher patient mortality may decrease the pool of patients who live to HAPI occurrence. Such hospitals may also provide more resources (specialty staff) to care for frail patient populations. Future research should aim to combine hospital data sets to overcome power limitations at the hospital level and should investigate additional measures of structure and process related to HAPI care.

Impact of emergency physician-provided patient education about alternative care venues.

OBJECTIVES: Interventions that focus on educating patients appear to be the most effective in directing healthcare utilization to more appropriate venues. We sought to evaluate the effects of mailed information and a brief scripted educational phone call from an emergency physician (EP) on subsequent emergency department (ED) utilization by low-risk adults with a recent treat-and-release ED visit. STUDY DESIGN: Patients were randomized into 3 groups for post-ED follow-up: EP phone call with mailed information, mailed information only, and no educational intervention. Each intervention group was compared with a set of matched controls. METHODS: We undertook this study in 6 EDs within an integrated healthcare delivery system. Overall, 9093 patients were identified; the final groups were the phone group (n = 609), mail group (n = 771), and matched control groups for each (n = 1827 and n = 1542, respectively). Analysis was stratified by age (<65 and ≥65 years). Patients were educated about available venues of care delivery for their future medical needs. The primary outcome was the rate of 6-month ED utilization after the intervention compared with the 6-month utilization rate preceding the intervention. RESULTS: Compared with matched controls, subsequent ED utilization decreased by 22% for patients 65 years or older in the phone group (P = .04) and by 27% for patients younger than 65 years in the mail group (P = .03). CONCLUSIONS: ED utilization subsequent to a low-acuity ED visit decreased after a brief post-ED education intervention by an EP explaining alternative venues of care for future medical needs. Response to the method of communication (phone vs mail) varied significantly by patient age.

Hospital-Acquired Pressure Injury: Risk-Adjusted Comparisons in an Integrated Healthcare Delivery System.

BACKGROUND: Although healthcare organizations have decreased hospital-acquired pressure injury (HAPI) rates, HAPIs are not eliminated, driving further examination in both nursing and health services research. OBJECTIVE: The objective was to describe HAPI incidence, risk factors, and risk-adjusted hospital variation within a California integrated healthcare system. METHODS: Inpatient episodes were included in this retrospective cohort if patients were hospitalized between January 1, 2013, and June 30, 2015. The primary outcome was development of a HAPI over time. Predictors included cited HAPI risk factors in addition to incorporation of a longitudinal comorbidity burden (Comorbidity Point Score, Version 2 [COPS2]), a severity-of-illness score (Laboratory-Based Acute Physiology Score, Version 2 [LAPS2]), and the Braden Scale for Predicting Pressure Ulcer Risk. RESULTS: Analyses included HAPI inpatient episodes (n = 1661) and non-HAPI episodes (n = 726,605). HAPI incidence was 0.57 per 1,000 patient days (95% CI [0.019, 3.805]) and 0.2% of episodes. A multivariate Cox proportional hazards model showed significant (p < .001) hazard ratios (HRs) for the change from the 25th to the 75th percentile for age (HR = 1.36, 95% CI [1.25, 1.45]), higher COPS2 scores (HR = 1.10, 95% CI [1.04, 1.16]), and higher LAPS2 scores (HR = 1.38, 95% CI [1.28, 1.50]). Female gender, an emergency room admission for a medical reason, and higher Braden scores showed significant protective HRs (HR < 1.00, p < .001). After risk adjustment, significant variation remained among the 35 hospitals. DISCUSSION: Results prompt the consideration of age, severity of illness (LAPS2), comorbidity indexes (COPS2), and the Braden score as important predictors for HAPI risk. HAPI rates may be low; however, because of significant individual site variation, HAPIs remain an area to explore through both research and quality improvement initiatives.

Prediction of Recurrent Clostridium Difficile Infection Using Comprehensive Electronic Medical Records in an Integrated Healthcare Delivery System.

BACKGROUND Predicting recurrent Clostridium difficile infection (rCDI) remains difficult. METHODS: We employed a retrospective cohort design. Granular electronic medical record (EMR) data had been collected from patients hospitalized at 21 Kaiser Permanente Northern California hospitals. The derivation dataset (2007-2013) included data from 9,386 patients who experienced incident CDI (iCDI) and 1,311 who experienced their first CDI recurrences (rCDI). The validation dataset (2014) included data from 1,865 patients who experienced incident CDI and 144 who experienced rCDI. Using multiple techniques, including machine learning, we evaluated more than 150 potential predictors. Our final analyses evaluated 3 models with varying degrees of complexity and 1 previously published model. RESULTS Despite having a large multicenter cohort and access to granular EMR data (eg, vital signs, and laboratory test results), none of the models discriminated well (c statistics, 0.591-0.605), had good calibration, or had good explanatory power. CONCLUSIONS Our ability to predict rCDI remains limited. Given currently available EMR technology, improvements in prediction will require incorporating new variables because currently available data elements lack adequate explanatory power. Infect Control Hosp Epidemiol 2017;38:1196-1203.

Enhanced Recovery After Surgery Program Implementation in 2 Surgical Populations in an Integrated Health Care Delivery System.

Importance: Novel approaches to perioperative surgical care focus on optimizing nutrition, mobility, and pain management to minimize adverse events after surgical procedures. Objective: To evaluate the outcomes of an enhanced recovery after surgery (ERAS) program among 2 target populations: patients undergoing elective colorectal resection and patients undergoing emergency hip fracture repair. Design, Setting, and Participants: A pre-post difference-in-differences study before and after ERAS implementation in the target populations compared with contemporaneous surgical comparator groups (patients undergoing elective gastrointestinal surgery and emergency orthopedic surgery). Implementation began in February and March 2014 and concluded by the end of 2014 at 20 medical centers within the Kaiser Permanente Northern California integrated health care delivery system. Exposures: A multifaceted ERAS program designed with a particular focus on perioperative pain management, mobility, nutrition, and patient engagement. Main Outcomes and Measures: The primary outcome was hospital length of stay. Secondary outcomes included hospital mortality, home discharge, 30-day readmission rates, and complication rates. Results: The study included a total of 3768 patients undergoing elective colorectal resection (mean [SD] age, 62.7 [14.1] years; 1812 [48.1%] male) and 5002 patients undergoing emergency hip fracture repair (mean [SD] age, 79.5 [11.8] years; 1586 [31.7%] male). Comparator surgical patients included 5556 patients undergoing elective gastrointestinal surgery and 1523 patients undergoing emergency orthopedic surgery. Most process metrics had significantly greater changes in the ERAS target populations after implementation compared with comparator surgical populations, including those for ambulation, nutrition, and opioid use. Hospital length of stay and postoperative complication rates were also significantly lower among ERAS target populations after implementation. The rate ratios for postoperative complications were 0.68 (95% CI, 0.46-0.99; P = .04) for patients undergoing colorectal resection and 0.67 (95% CI, 0.45-0.99, P = .05) for patients with hip fracture. Among patients undergoing colorectal resection, ERAS implementation was associated with decreased rates of hospital mortality (0.17; 95% CI, 0.03-0.86; P = .03), whereas among patients with hip fracture, implementation was associated with increased rates of home discharge (1.24; 95% CI, 1.06-1.44; P = .007). Conclusions and Relevance: Multicenter implementation of an ERAS program among patients undergoing elective colorectal resection and patients undergoing emergency hip fracture repair successfully altered processes of care and was associated with significant absolute and relative decreases in hospital length of stay and postoperative complication rates. Rapid, large-scale implementation of a multidisciplinary ERAS program is feasible and effective in improving surgical outcomes.

A Quantitative, Risk-Based Approach to the Management of Neonatal Early-Onset Sepsis.

Importance: Current algorithms for management of neonatal early-onset sepsis (EOS) result in medical intervention for large numbers of uninfected infants. We developed multivariable prediction models for estimating the risk of EOS among late preterm and term infants based on objective data available at birth and the newborn's clinical status. Objectives: To examine the effect of neonatal EOS risk prediction models on sepsis evaluations and antibiotic use and assess their safety in a large integrated health care system. Design, Setting, and Participants: The study cohort includes 204 485 infants born at 35 weeks' gestation or later at a Kaiser Permanente Northern California hospital from January 1, 2010, through December 31, 2015. The study compared 3 periods when EOS management was based on (1) national recommended guidelines (baseline period [January 1, 2010, through November 31, 2012]), (2) multivariable estimates of sepsis risk at birth (learning period [December 1, 2012, through June 30, 2014]), and (3) the multivariable risk estimate combined with the infant's clinical condition in the first 24 hours after birth (EOS calculator period [July 1, 2014, through December 31, 2015]). Main Outcomes and Measures: The primary outcome was antibiotic administration in the first 24 hours. Secondary outcomes included blood culture use, antibiotic administration between 24 and 72 hours, clinical outcomes, and readmissions for EOS. Results: The study cohort included 204 485 infants born at 35 weeks' gestation or later: 95 343 in the baseline period (mean [SD] age, 39.4 [1.3] weeks; 46 651 male [51.0%]; 37 007 white, non-Hispanic [38.8%]), 52 881 in the learning period (mean [SD] age, 39.3 [1.3] weeks; 27 067 male [51.2%]; 20 175 white, non-Hispanic [38.2%]), and 56 261 in the EOS calculator period (mean [SD] age, 39.4 [1.3] weeks; 28 575 male [50.8%]; 20 484 white, non-Hispanic [36.4%]). In a comparison of the baseline period with the EOS calculator period, blood culture use decreased from 14.5% to 4.9% (adjusted difference, -7.7%; 95% CI, -13.1% to -2.4%). Empirical antibiotic administration in the first 24 hours decreased from 5.0% to 2.6% (adjusted difference, -1.8; 95% CI, -2.4% to -1.3%). No increase in antibiotic use occurred between 24 and 72 hours after birth; use decreased from 0.5% to 0.4% (adjusted difference, 0.0%; 95% CI, -0.1% to 0.2%). The incidence of culture-confirmed EOS was similar during the 3 periods (0.03% in the baseline period, 0.03% in the learning period, and 0.02% in the EOS calculator period). Readmissions for EOS (within 7 days of birth) were rare in all periods (5.2 per 100 000 births in the baseline period, 1.9 per 100 000 births in the learning period, and 5.3 per 100 000 births in the EOS calculator period) and did not differ statistically (P = .70). Incidence of adverse clinical outcomes, including need for inotropes, mechanical ventilation, meningitis, and death, was unchanged after introduction of the EOS calculator. Conclusions and Relevance: Clinical care algorithms based on individual infant estimates of EOS risk derived from a multivariable risk prediction model reduced the proportion of newborns undergoing laboratory testing and receiving empirical antibiotic treatment without apparent adverse effects.