Content of Patient Electronic Messages to Physicians in a Large Integrated System.

This quality improvement study describes the content of electronic health record messages from patients to physicians in a large integrated health care system using natural language processing algorithms.

Validation of a Multivariable Model to Predict Suicide Attempt in a Mental Health Intake Sample.

Importance: Given that suicide rates have been increasing over the past decade and the demand for mental health care is at an all-time high, targeted prevention efforts are needed to identify individuals seeking to initiate mental health outpatient services who are at high risk for suicide. Suicide prediction models have been developed using outpatient mental health encounters, but their performance among intake appointments has not been directly examined. Objective: To assess the performance of a predictive model of suicide attempts among individuals seeking to initiate an episode of outpatient mental health care. Design, Setting, and Participants: This prognostic study tested the performance of a previously developed machine learning model designed to predict suicide attempts within 90 days of any mental health outpatient visit. All mental health intake appointments scheduled between January 1, 2012, and April 1, 2022, at Kaiser Permanente Northern California, a large integrated health care delivery system serving over 4.5 million patients, were included. Data were extracted and analyzed from August 9, 2022, to July 31, 2023. Main Outcome and Measures: Suicide attempts (including completed suicides) within 90 days of the appointment, determined by diagnostic codes and government databases. All predictors were extracted from electronic health records. Results: The study included 1 623 232 scheduled appointments from 835 616 unique patients. There were 2800 scheduled appointments (0.17%) followed by a suicide attempt within 90 days. The mean (SD) age across appointments was 39.7 (15.8) years, and most appointments were for women (1 103 184 [68.0%]). The model had an area under the receiver operating characteristic curve of 0.77 (95% CI, 0.76-0.78), an area under the precision-recall curve of 0.02 (95% CI, 0.02-0.02), an expected calibration error of 0.0012 (95% CI, 0.0011-0.0013), and sensitivities of 37.2% (95% CI, 35.5%-38.9%) and 18.8% (95% CI, 17.3%-20.2%) at specificities of 95% and 99%, respectively. The 10% of appointments at the highest risk level accounted for 48.8% (95% CI, 47.0%-50.6%) of the appointments followed by a suicide attempt. Conclusions and Relevance: In this prognostic study involving mental health intakes, a previously developed machine learning model of suicide attempts showed good overall classification performance. Implementation research is needed to determine appropriate thresholds and interventions for applying the model in an intake setting to target high-risk cases in a manner that is acceptable to patients and clinicians.

Training the next generation of delivery science researchers: 10-year experience of a post-doctoral research fellowship program within an integrated care system.

Introduction: Learning health systems require a workforce of researchers trained in the methods of identifying and overcoming barriers to effective, evidence-based care. Most existing postdoctoral training programs, such as NIH-funded postdoctoral T32 awards, support basic and epidemiological science with very limited focus on rigorous delivery science methods for improving care. In this report, we present the 10-year experience of developing and implementing a Delivery Science postdoctoral fellowship embedded within an integrated health care delivery system. Methods: In 2012, the Kaiser Permanente Northern California Division of Research designed and implemented a 2-year postdoctoral Delivery Science Fellowship research training program to foster research expertise in identifying and addressing barriers to evidence-based care within health care delivery systems. Results: Since 2014, 20 fellows have completed the program. Ten fellows had PhD-level scientific training, and 10 fellows had clinical doctorates (eg, MD, RN/PhD, PharmD). Fellowship alumni have graduated to faculty research positions at academic institutions (9), and research or clinical organizations (4). Seven alumni now hold positions in Kaiser Permanente's clinical operations or medical group (7). Conclusions: This delivery science fellowship program has succeeded in training graduates to address delivery science problems from both research and operational perspectives. In the next 10 years, additional goals of the program will be to expand its reach (eg, by developing joint research training models in collaboration with clinical fellowships) and strengthen mechanisms to support transition from fellowship to the workforce, especially for researchers from underrepresented groups.

Trends in Outpatient Visits and Hospital and Intensive Care Unit Admissions of Adults With COVID-19 in an Integrated US Health Care System, March 2020 to January 2022.

This cohort study of patients at a single integrated health system examines trends in COVID-19­related treatment location and mortality.

Quantifying the breadth of antibiotic exposure in sepsis and suspected infection using spectrum scores.

A retrospective cohort study. Studies to quantify the breadth of antibiotic exposure across populations remain limited. Therefore, we applied a validated method to describe the breadth of antimicrobial coverage in a multicenter cohort of patients with suspected infection and sepsis. We conducted a retrospective cohort study across 21 hospitals within an integrated healthcare delivery system of patients admitted to the hospital through the ED with suspected infection or sepsis and receiving antibiotics during hospitalization from January 1, 2012, to December 31, 2017. We quantified the breadth of antimicrobial coverage using the Spectrum Score, a numerical score from 0 to 64, in patients with suspected infection and sepsis using electronic health record data. Of 364,506 hospital admissions through the emergency department, we identified 159,004 (43.6%) with suspected infection and 205,502 (56.4%) with sepsis. Inpatient mortality was higher among those with sepsis compared to those with suspected infection (8.4% vs 1.2%; P < .001). Patients with sepsis had higher median global Spectrum Scores (43.8 [interquartile range IQR 32.0-49.5] vs 43.5 [IQR 26.8-47.2]; P < .001) and additive Spectrum Scores (114.0 [IQR 57.0-204.5] vs 87.5 [IQR 45.0-144.8]; P < .001) compared to those with suspected infection. Increased Spectrum Scores were associated with inpatient mortality, even after covariate adjustments (adjusted odds ratio per 10-point increase in Spectrum Score 1.31; 95%CI 1.29-1.33). Spectrum Scores quantify the variability in antibiotic breadth among individual patients, between suspected infection and sepsis populations, over the course of hospitalization, and across infection sources. They may play a key role in quantifying the variation in antibiotic prescribing in patients with suspected infection and sepsis.

Unsupervised probabilistic models for sequential Electronic Health Records.

We develop an unsupervised probabilistic model for heterogeneous Electronic Health Record (EHR) data. Utilizing a mixture model formulation, our approach directly models sequences of arbitrary length, such as medications and laboratory results. This allows for subgrouping and incorporation of the dynamics underlying heterogeneous data types. The model consists of a layered set of latent variables that encode underlying structure in the data. These variables represent subject subgroups at the top layer, and unobserved states for sequences in the second layer. We train this model on episodic data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The resulting properties of the trained model generate novel insight from these complex and multifaceted data. In addition, we show how the model can be used to analyze sequences that contribute to assessment of mortality likelihood.

Risk of severe COVID-19 infection among adults with prior exposure to children.

Susceptibility and severity of COVID-19 infection vary widely. Prior exposure to endemic coronaviruses, common in young children, may protect against SARS-CoV-2. We evaluated risk of severe COVID-19 among adults with and without exposure to young children in a large, integrated healthcare system. Adults with children 0-5 years were matched 1:1 to adults with children 6-11 years, 12-18 years, and those without children based upon a COVID-19 propensity score and risk factors for severe COVID-19. COVID-19 infections, hospitalizations, and need for intensive care unit (ICU) were assessed in 3,126,427 adults, of whom 24% (N = 743,814) had children 18 years or younger, and 8.8% (N = 274,316) had a youngest child 0-5 years. After 1:1 matching, propensity for COVID-19 infection and risk factors for severe COVID-19 were well balanced between groups. Rates of COVID-19 infection were slightly higher for adults with exposure to older children (incident risk ratio, 1.09, 95% confidence interval, [1.05-1.12] and IRR 1.09 [1.05-1.13] for adults with children 6-11 and 12-18, respectively), compared to those with children 0-5 years, although no difference in rates of COVID-19 illness requiring hospitalization or ICU admission was observed. However, adults without exposure to children had lower rates of COVID-19 infection (IRR 0.85, [0.83-0.87]) but significantly higher rates of COVID-19 hospitalization (IRR 1.49, [1.29-1.73]) and hospitalization requiring ICU admission (IRR 1.76, [1.19-2.58]) compared to those with children aged 0-5. In a large, real-world population, exposure to young children was associated with less severe COVID-19 illness. Endemic coronavirus cross-immunity may play a role in protection against severe COVID-19.

The Kaiser Permanente Northern California Advance Alert Monitor Program: An Automated Early Warning System for Adults at Risk for In-Hospital Clinical Deterioration.

BACKGROUND: In-hospital deterioration among ward patients is associated with substantially increased adverse outcome rates. In 2013 Kaiser Permanente Northern California (KPNC) developed and implemented a predictive analytics-driven program, Advance Alert Monitor (AAM), to improve early detection and intervention for in-hospital deterioration. The AAM predictive model is designed to give clinicians 12 hours of lead time before clinical deterioration, permitting early detection and a patient goals-concordant response to prevent worsening. DESIGN OF THE AAM INTERVENTION: Across the 21 hospitals of the KPNC integrated health care delivery system, AAM analyzes electronic health record (EHR) data for patients in medical/surgical and telemetry units 24 hours a day, 7 days a week. Patients identified as high risk by the AAM algorithm trigger an alert for a regional team of experienced critical care virtual quality nurse consultants (VQNCs), who then cascade validated, actionable information to rapid response team (RRT) nurses at local hospitals. RRT nurses conduct bedside assessments of at-risk patients and formulate interdisciplinary clinical responses with hospital-based physicians, bedside nurses, and supportive care teams to ensure a well-defined escalation plan that includes clarification of the patients' goals of care. SUCCESS OF THE INTERVENTION: Since 2019 the AAM program has been implemented at all 21 KPNC hospitals. The use of predictive modeling embedded within the EHR to identify high-risk patients has produced the standardization of monitoring workflows, clinical rescue protocols, and coordination to ensure that care is consistent with patients' individual goals of care. An evaluation of the program, using a staggered deployment sequence over 19 hospitals, demonstrates that the AAM program is associated with statistically significant decreases in mortality (9.8% vs. 14.4%), hospital length of stay, and ICU length of stay. Statistical analyses estimated that more than 500 deaths were prevented each year with the AAM program. LESSONS LEARNED: Unlocking the potential of predictive modeling in the EHR is the first step toward realizing the promise of artificial intelligence/machine learning (AI/ML) to improve health outcomes. The AAM program leveraged predictive analytics to produce highly reliable care by identifying at-risk patients, preventing deterioration, and reducing adverse outcomes and can be used as a model for how clinical decision support and inpatient population management can effectively improve care.

Evaluation of Outcomes Following Hospital-Wide Implementation of a Subcutaneous Insulin Protocol for Diabetic Ketoacidosis.

Importance: Standard diabetic ketoacidosis care in the US includes intravenous insulin treatment in the intensive care unit. Subcutaneous (SQ) insulin could decrease intensive care unit need, but the data are limited. Objective: To assess outcomes after implementation of an SQ insulin protocol for treating diabetic ketoacidosis. Design, Setting, and Participants: This cohort study is a retrospective evaluation of a prospectively implemented SQ insulin protocol. The study was conducted at an integrated health care system in Northern California. Participants included hospitalized patients with diabetic ketoacidosis at 21 hospitals between January 1, 2010, and December 31, 2019. The preimplementation phase was 2010 to 2015, and the postimplementation phase was 2017 to 2019. Data analysis was performed from October 2020 to January 2022. Exposure: An SQ insulin treatment protocol for diabetic ketoacidosis. Main Outcomes and Measures: Difference-in-differences evaluation of the need for intensive care, mortality, readmission, and length of stay at a single intervention site using an SQ insulin protocol from 2017 onward compared with 20 control hospitals using standard care. Results: A total of 7989 hospitalizations for diabetic ketoacidosis occurred, with 4739 (59.3%) occurring before and 3250 (40.7%) occurring after implementation. The overall mean (SD) age was 42.3 (17.7) years, with 4137 hospitalizations (51.8%) occurring among female patients. Before implementation, SQ insulin was the first insulin used in 40 intervention (13.4%) and 651 control (14.7%) hospitalizations. After implementation, 98 hospitalizations (80.3%) received SQ insulin first at the intervention site compared with 402 hospitalizations (12.8%) at control sites. The adjusted rate ratio for intensive care unit admission was 0.43 (95% CI, 0.33-0.56) at the intervention sites, a 57% reduction compared with control sites, and was 0.50 (95% CI, 0.25-0.99) for 30-day hospital readmission, a 50% reduction. There were no significant changes in hospital length of stay and rates of death. Conclusions and Relevance: These findings suggest that a protocol based on SQ insulin for diabetic ketoacidosis treatment was associated with significant decreases in intensive care unit need and readmission, with no evidence of increases in adverse events.

Prognostic Accuracy of Presepsis and Intrasepsis Characteristics for Prediction of Cardiovascular Events After a Sepsis Hospitalization.

OBJECTIVES: Sepsis survivors face increased risk for cardiovascular complications; however, the contribution of intrasepsis events to cardiovascular risk profiles is unclear. SETTING: Kaiser Permanente Northern California (KPNC) and Intermountain Healthcare (IH) integrated healthcare delivery systems. SUBJECTS: Sepsis survivors (2011-2017 [KPNC] and 2018-2020 [IH]) greater than or equal to 40 years old without prior cardiovascular disease. DESIGN: Data across KPNC and IH were harmonized and grouped into presepsis (demographics, atherosclerotic cardiovascular disease scores, comorbidities) or intrasepsis factors (e.g., laboratory values, vital signs, organ support, infection source) with random split for training/internal validation datasets (75%/25%) within KPNC and IH. Models were bidirectionally, externally validated between healthcare systems. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Changes to predictive accuracy (C-statistic) of cause-specific proportional hazards models predicting 1-year cardiovascular outcomes (atherosclerotic cardiovascular disease, heart failure, and atrial fibrillation events) were compared between models that did and did not contain intrasepsis factors. Among 39,590 KPNC and 16,388 IH sepsis survivors, 3,503 (8.8%) at Kaiser Permanente (KP) and 600 (3.7%) at IH experienced a cardiovascular event within 1-year after hospital discharge, including 996 (2.5%) at KP and 192 (1.2%) IH with an atherosclerotic event first, 564 (1.4%) at KP and 117 (0.7%) IH with a heart failure event, 2,310 (5.8%) at KP and 371 (2.3%) with an atrial fibrillation event. Death within 1 year after sepsis occurred for 7,948 (20%) KP and 2,085 (12.7%) IH patients. Combined models with presepsis and intrasepsis factors had better discrimination for cardiovascular events (KPNC C-statistic 0.783 [95% CI, 0.766-0.799]; IH 0.763 [0.726-0.801]) as compared with presepsis cardiovascular risk alone (KPNC: 0.666 [0.648-0.683], IH 0.660 [0.619-0.702]) during internal validation. External validation of models across healthcare systems showed similar performance (KPNC model within IH data C-statistic: 0.734 [0.725-0.744]; IH model within KPNC data: 0.787 [0.768-0.805]). CONCLUSIONS: Across two large healthcare systems, intrasepsis factors improved postsepsis cardiovascular risk prediction as compared with presepsis cardiovascular risk profiles. Further exploration of sepsis factors that contribute to postsepsis cardiovascular events is warranted for improved mechanistic and predictive models.

Validation of Respiratory Rate-Oxygenation Index in Patients With COVID-19-Related Respiratory Failure.

OBJECTIVES: The respiratory rate-oxygenation (ROX) index is a fraction of oxygen saturation, Fio2, and respiratory rate that has been validated to predict receipt of invasive mechanical ventilation in patients receiving high-flow nasal cannula (HFNC). This study aimed to validate ROX in a cohort of inpatients with COVID-19-related respiratory failure. DESIGN: Retrospective validation of the ROX index. We calculated sensitivity, specificity, positive predictive value, negative predictive value, and 95% CIs of ROX for invasive mechanical ventilation any time during hospitalization. SETTING: Twenty-one hospitals of Kaiser Permanente Northern California, an integrated healthcare delivery system. PATIENTS: We identified adults with positive severe acute respiratory syndrome coronavirus 2 polymerase chain reaction test within 3 weeks of, or during, hospitalization between February 1, 2020, and December 31, 2020. We calculated ROX at 12 hours after HFNC initiation. We grouped patients as low (≥ 4.88), intermediate (< 4.88 and ≥ 3.85), or high (< 3.85) risk using previously published thresholds. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We identified 1,847 patients who had no limitation of life support. Of these, 525 (31.7%) received invasive mechanical ventilation any time during hospitalization and 511 died (27.7%). The sensitivity, specificity, positive predictive value, and negative predictive value of 12-hour ROX threshold (< 3.85) predicting invasive mechanical ventilation were 32.3% (95% CI, 28.5-36.3%), 89.8% (95% CI, 88.0-91.4%), 59.4% (95% CI, 53.8-64.9%), and 74.1% (95% CI, 71.8-76.3%), respectively. CONCLUSIONS: The 12-hour ROX index has a positive predictive value (59.4%) using threshold of less than 3.85 for COVID-19 patients needing invasive mechanical ventilation. Our health system has embedded ROX into the electronic health record to prioritize rounding during periods of inpatient surge.

Outcomes After Implementation of a Benzodiazepine-Sparing Alcohol Withdrawal Order Set in an Integrated Health Care System.

Importance: Alcohol withdrawal syndrome (AWS) is a common inpatient diagnosis managed primarily with benzodiazepines. Concerns about the adverse effects associated with benzodiazepines have spurred interest in using benzodiazepine-sparing treatments. Objective: To evaluate changes in outcomes after implementation of a benzodiazepine-sparing AWS inpatient order set that included adjunctive therapies (eg, gabapentin, valproic acid, clonidine, and dexmedetomidine). Design, Setting, and Participants: This difference-in-differences quality improvement study was conducted among 22 899 AWS adult hospitalizations from October 1, 2014, to September 30, 2019, in the Kaiser Permanente Northern California integrated health care delivery system. Data were analyzed from September 2020 through November 2021. Exposures: Implementation of the benzodiazepine-sparing AWS order set on October 1, 2018. Main Outcomes and Measures: Adjusted rate ratios for medication use, inpatient mortality, length of stay, intensive care unit admission, and nonelective readmission within 30 days were calculated comparing postimplementation and preimplementation periods among hospitals with and without order set use. Results: Among 904 540 hospitalizations in the integrated health care delivery system during the study period, AWS was present in 22 899 hospitalizations (2.5%), occurring among 16 323 unique patients (mean [SD] age, 57.1 [14.8] years; 15 764 [68.8%] men). Of these hospitalizations, 12 889 (56.3%) used an order set for alcohol withdrawal. Among hospitalizations with order set use, any benzodiazepine use decreased after implementation from 6431 hospitalizations (78.1%) to 2823 hospitalizations (60.7%) (P < .001), with concomitant decreases in the mean (SD) total dosage of lorazepam before vs after implementation (19.7 [38.3] mg vs 6.0 [9.1] mg; P < .001). There were also significant changes from before to after implementation in the use of adjunctive medications, including gabapentin (2413 hospitalizations [29.3%] vs 2814 hospitalizations [60.5%]; P < .001), clonidine (1476 hospitalizations [17.9%] vs 2208 hospitalizations [47.5%]; P < .001), thiamine (6298 hospitalizations [76.5%] vs 4047 hospitalizations [87.0%]; P < .001), valproic acid (109 hospitalizations [1.3%] vs 256 hospitalizations [5.5%]; P < .001), and phenobarbital (412 hospitalizations [5.0%] vs 292 hospitalizations [6.3%]; P = .003). Compared with AWS hospitalizations without order set use, use of the benzodiazepine-sparing order set was associated with decreases in intensive care unit use (adjusted rate ratio [ARR], 0.71; 95% CI, 0.56-0.89; P = .003) and hospital length of stay (ARR, 0.71; 95% CI, 0.58-0.86; P < .001). Conclusions and Relevance: This study found that implementation of a benzodiazepine-sparing AWS order set was associated with decreased use of benzodiazepines and favorable trends in outcomes. These findings suggest that further prospective research is needed to identify the most effective treatments regimens for patients hospitalized with alcohol withdrawal.

Evaluation of Incident 7-Day Infection and Sepsis Hospitalizations in an Integrated Health System.

Rationale: Prehospital opportunities to predict infection and sepsis hospitalization may exist, but little is known about their incidence following common healthcare encounters. Objectives: To evaluate the incidence and timing of infection and sepsis hospitalization within 7 days of living hospital discharge, emergency department discharge, and ambulatory visit settings. Methods: In each setting, we identified patients in clinical strata based on the presence of infection and severity of illness. We estimated number needed to evaluate values with hypothetical predictive model operating characteristics. Results: We identified 97,614,228 encounters, including 1,117,702 (1.1%) hospital discharges, 4,635,517 (4.7%) emergency department discharges, and 91,861,009 (94.1%) ambulatory visits between 2012 and 2017. The incidence of 7-day infection hospitalization varied from 37,140 (3.3%) following inpatient discharge to 50,315 (1.1%) following emergency department discharge and 277,034 (0.3%) following ambulatory visits. The incidence of 7-day infection hospitalization was increased for inpatient discharges with high readmission risk (10.0%), emergency department discharges with increased acute or chronic severity of illness (3.5% and 4.7%, respectively), and ambulatory visits with acute infection (0.7%). The timing of 7-day infection and sepsis hospitalizations differed across settings with an early rise following ambulatory visits, a later peak following emergency department discharges, and a delayed peak following inpatient discharge. Theoretical number needed to evaluate values varied by strata, but following hospital and emergency department discharge, were as low as 15-25. Conclusions: Incident 7-day infection and sepsis hospitalizations following encounters in routine healthcare settings were surprisingly common and may be amenable to clinical predictive models.

Development and Validation of an Obstetric Comorbidity Risk Score for Clinical Use.

Background: A comorbidity summary score may support early and systematic identification of women at high risk for adverse obstetric outcomes. The objective of this study was to conduct the initial development and validation of an obstetrics comorbidity risk score for automated implementation in the electronic health record (EHR) for clinical use. Methods: The score was developed and validated using EHR data for a retrospective cohort of pregnancies with delivery between 2010 and 2018 at Kaiser Permanente Northern California, an integrated health care system. The outcome used for model development consisted of adverse obstetric events from delivery hospitalization (e.g., eclampsia, hemorrhage, death). Candidate predictors included maternal age, parity, multiple gestation, and any maternal diagnoses assigned in health care encounters in the 12 months before admission for delivery. We used penalized regression for variable selection, logistic regression to fit the model, and internal validation for model evaluation. We also evaluated prenatal model performance at 18 weeks of pregnancy. Results: The development cohort (n = 227,405 pregnancies) had an outcome rate of 3.8% and the validation cohort (n = 41,683) had an outcome rate of 2.9%. Of 276 candidate predictors, 37 were included in the final model. The final model had a validation c-statistic of 0.72 (95% confidence interval [CI] 0.70-0.73). When evaluated at 18 weeks of pregnancy, discrimination was modestly diminished (c-statistic 0.68 [95% CI 0.67-0.70]). Conclusions: The obstetric comorbidity score demonstrated good discrimination for adverse obstetric outcomes. After additional appropriate validation, the score can be automated in the EHR to support early identification of high-risk women and assist efforts to ensure risk-appropriate maternal care.

Evaluation of an intervention targeted with predictive analytics to prevent readmissions in an integrated health system: observational study.

OBJECTIVES: To determine the associations between a care coordination intervention (the Transitions Program) targeted to patients after hospital discharge and 30 day readmission and mortality in a large, integrated healthcare system. DESIGN: Observational study. SETTING: 21 hospitals operated by Kaiser Permanente Northern California. PARTICIPANTS: 1 539 285 eligible index hospital admissions corresponding to 739 040 unique patients from June 2010 to December 2018. 411 507 patients were discharged post-implementation of the Transitions Program; 80 424 (19.5%) of these patients were at medium or high predicted risk and were assigned to receive the intervention after discharge. INTERVENTION: Patients admitted to hospital were automatically assigned to be followed by the Transitions Program in the 30 days post-discharge if their predicted risk of 30 day readmission or mortality was greater than 25% on the basis of electronic health record data. MAIN OUTCOME MEASURES: Non-elective hospital readmissions and all cause mortality in the 30 days after hospital discharge. RESULTS: Difference-in-differences estimates indicated that the intervention was associated with significantly reduced odds of 30 day non-elective readmission (adjusted odds ratio 0.91, 95% confidence interval 0.89 to 0.93; absolute risk reduction 95% confidence interval -2.5%, -3.1% to -2.0%) but not with the odds of 30 day post-discharge mortality (1.00, 0.95 to 1.04). Based on the regression discontinuity estimate, the association with readmission was of similar magnitude (absolute risk reduction -2.7%, -3.2% to -2.2%) among patients at medium risk near the risk threshold used for enrollment. However, the regression discontinuity estimate of the association with post-discharge mortality (-0.7% -1.4% to -0.0%) was significant and suggested benefit in this subgroup of patients. CONCLUSIONS: In an integrated health system, the implementation of a comprehensive readmissions prevention intervention was associated with a reduction in 30 day readmission rates. Moreover, there was no association with 30 day post-discharge mortality, except among medium risk patients, where some evidence for benefit was found. Altogether, the study provides evidence to suggest the effectiveness of readmission prevention interventions in community settings, but further research might be required to confirm the findings beyond this setting.

Development of a healthcare system COVID Hotspotting Score in California: an observational study with prospective validation.

OBJECTIVE: To examine the value of health systems data as indicators of emerging COVID-19 activity. DESIGN: Observational study of health system indicators for the COVID Hotspotting Score (CHOTS) with prospective validation. SETTING AND PARTICIPANTS: An integrated healthcare delivery system in Northern California including 21 hospitals and 4.5 million members. MAIN OUTCOME MEASURES: The CHOTS incorporated 10 variables including four major (cough/cold calls, emails, new positive COVID-19 tests, COVID-19 hospital census) and six minor (COVID-19 calls, respiratory infection and COVID-19 routine and urgent visits, and respiratory viral testing) indicators assessed with change point detection and slope metrics. We quantified cross-correlations lagged by 7-42 days between CHOTS and standardised COVID-19 hospital census using observational data from 1 April to 31 May 2020 and two waves of prospective data through 21 March 2021. RESULTS: Through 30 September 2020, peak cross-correlation between CHOTS and COVID-19 hospital census occurred with a 28-day lag at 0.78; at 42 days, the correlation was 0.69. Lagged correlation between medical centre CHOTS and their COVID-19 census was highest at 42 days for one facility (0.63), at 35 days for nine facilities (0.52-0.73), at 28 days for eight facilities (0.28-0.74) and at 14 days for two facilities (0.73-0.78). The strongest correlation for individual indicators was 0.94 (COVID-19 census) and 0.90 (new positive COVID-19 tests) lagged 1-14 days and 0.83 for COVID-19 calls and urgent clinic visits lagged 14-28 days. Cross-correlation was similar (0.73) with a 35-day lag using prospective validation from 1 October 2020 to 21 March 2021. CONCLUSIONS: Passively collected health system indicators were strongly correlated with forthcoming COVID-19 hospital census up to 6 weeks before three successive COVID-19 waves. These tools could inform communities, health systems and public health officials to identify, prepare for and mitigate emerging COVID-19 activity.

Association of Positive Fluid Balance at Discharge After Sepsis Management With 30-Day Readmission.

Importance: Although early fluid administration has been shown to lower sepsis mortality, positive fluid balance has been associated with adverse outcomes. Little is known about associations in non-intensive care unit settings, with growing concern about readmission from excess fluid accumulation in patients with sepsis. Objective: To evaluate whether positive fluid balance among non-critically ill patients with sepsis was associated with increased readmission risk, including readmission for heart failure. Design, Setting, and Participants: This multicenter retrospective cohort study was conducted between January 1, 2012, and December 31, 2017, among 57 032 non-critically ill adults hospitalized for sepsis at 21 hospitals across Northern California. Kaiser Permanente Northern California is an integrated health care system with a community-based population of more than 4.4 million members. Statistical analysis was performed from January 1 to December 31, 2019. Exposures: Intake and output net fluid balance (I/O) measured daily and cumulatively at discharge (positive vs negative). Main Outcomes and Measures: The primary outcome was 30-day readmission. The secondary outcomes were readmission stratified by category and mortality after living discharge. Results: The cohort included 57 032 patients who were hospitalized for sepsis (28 779 women [50.5%]; mean [SD] age, 73.7 [15.5] years). Compared with patients with positive I/O (40 940 [71.8%]), those with negative I/O (16 092 [28.2%]) were older, with increased comorbidity, acute illness severity, preexisting heart failure or chronic kidney disease, diuretic use, and decreased fluid administration volume. During 30-day follow-up, 8719 patients (15.3%) were readmitted and 3639 patients (6.4%) died. There was no difference in readmission between patients with positive vs negative I/O (HR, 1.00; 95% CI, 0.95-1.05). No association was detected between readmission and I/O using continuous, splined, and quadratic function transformations. Positive I/O was associated with decreased heart failure-related readmission (HR, 0.80 [95% CI, 0.71-0.91]) and increased 30-day mortality (HR, 1.23 [95% CI, 1.15-1.31]). Conclusions and Relevance: In this large observational study of non-critically ill patients hospitalized with sepsis, there was no association between positive fluid balance at the time of discharge and readmission. However, these findings may have been limited by variable recording and documentation of fluid intake and output; additional studies are needed to examine the association of fluid status with outcomes in patients with sepsis to reduce readmission risk.

Racial Disparities in COVID-19 Testing and Outcomes : Retrospective Cohort Study in an Integrated Health System.

BACKGROUND: Racial disparities exist in outcomes after severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. OBJECTIVE: To evaluate the contribution of race/ethnicity in SARS-CoV-2 testing, infection, and outcomes. DESIGN: Retrospective cohort study (1 February 2020 to 31 May 2020). SETTING: Integrated health care delivery system in Northern California. PARTICIPANTS: Adult health plan members. MEASUREMENTS: Age, sex, neighborhood deprivation index, comorbid conditions, acute physiology indices, and race/ethnicity; SARS-CoV-2 testing and incidence of positive test results; and hospitalization, illness severity, and mortality. RESULTS: Among 3 481 716 eligible members, 42.0% were White, 6.4% African American, 19.9% Hispanic, and 18.6% Asian; 13.0% were of other or unknown race. Of eligible members, 91 212 (2.6%) were tested for SARS-CoV-2 infection and 3686 had positive results (overall incidence, 105.9 per 100 000 persons; by racial group, White, 55.1; African American, 123.1; Hispanic, 219.6; Asian, 111.7; other/unknown, 79.3). African American persons had the highest unadjusted testing and mortality rates, White persons had the lowest testing rates, and those with other or unknown race had the lowest mortality rates. Compared with White persons, adjusted testing rates among non-White persons were marginally higher, but infection rates were significantly higher; adjusted odds ratios [aORs] for African American persons, Hispanic persons, Asian persons, and persons of other/unknown race were 2.01 (95% CI, 1.75 to 2.31), 3.93 (CI, 3.59 to 4.30), 2.19 (CI, 1.98 to 2.42), and 1.57 (CI, 1.38 to 1.78), respectively. Geographic analyses showed that infections clustered in areas with higher proportions of non-White persons. Compared with White persons, adjusted hospitalization rates for African American persons, Hispanic persons, Asian persons, and persons of other/unknown race were 1.47 (CI, 1.03 to 2.09), 1.42 (CI, 1.11 to 1.82), 1.47 (CI, 1.13 to 1.92), and 1.03 (CI, 0.72 to 1.46), respectively. Adjusted analyses showed no racial differences in inpatient mortality or total mortality during the study period. For testing, comorbid conditions made the greatest relative contribution to model explanatory power (77.9%); race only accounted for 8.1%. Likelihood of infection was largely due to race (80.3%). For other outcomes, age was most important; race only contributed 4.5% for hospitalization, 12.8% for admission illness severity, 2.3% for in-hospital death, and 0.4% for any death. LIMITATION: The study involved an insured population in a highly integrated health system. CONCLUSION: Race was the most important predictor of SARS-CoV-2 infection. After infection, race was associated with increased hospitalization risk but not mortality. PRIMARY FUNDING SOURCE: The Permanente Medical Group, Inc.

Evaluation of Vaccination Strategies to Compare Efficient and Equitable Vaccine Allocation by Race and Ethnicity Across Time.

Importance: Identifying the most efficient COVID-19 vaccine allocation strategy may substantially reduce hospitalizations and save lives while ensuring an equitable vaccine distribution. Objective: To simulate the association of different vaccine allocation strategies with COVID-19-associated morbidity and mortality and their distribution across racial and ethnic groups. Design Setting and Participants: We developed and internally validated the risk of COVID-19 infection and risk of hospitalization models on randomly split training and validation data sets. These were used in a computer simulation study of vaccine prioritization among adult health plan members who were drawn from an integrated health care delivery system. The study was conducted from January 3, 2021, to June 1, 2021, in Oakland, California, and the data were analyzed during the same period. Main Outcomes and Measures: We simulated the association of different vaccine allocation strategies, including (1) random, (2) a US Centers for Disease Control and Prevention (CDC) proxy, (3) age based, and (4) combinations of models for the risk of adverse outcomes (CRS) and COVID-19 infection (PROVID), with COVID-19-related hospitalizations between May 1, 2020, and December 31, 2020, that were randomly permuted by month across 250 simulations and assessed vaccine allocation by race and ethnicity and the neighborhood deprivation index across time. Results: The study included 3 202 679 adult patients (mean [SD] age, 48.2 [18.0] years; 1 677 637 women [52.4%]; 1 525 042 men [47.6%]; 611 154 Asian [19.1%], 206 363 Black [6.4%], 642 344 Hispanic [20.1%], and 1 390 638 White individuals [43.4%]), of whom 36 137 (1.1%) were positive for SARS-CoV-2. A risk-based strategy (CRS/PROVID) showed the largest avoidable hospitalization estimates (4954; 95% CI, 3452-5878) followed by age-based (4362; 95% CI, 2866-5175) and CDC proxy (4085; 95% CI, 2805-5109) strategies. Random vaccination showed substantially lower reductions in adverse outcomes. Risk-based strategies also showed the largest number of avoidable COVID-19 deaths (joint CRS/PROVID) and household transmissions. Risk-based (PROVID) and CDC proxy strategies were estimated to vaccinate the highest percentage of Hispanic and Black patients in 8 months (joint CRS/PROVID: 642 570 [100%] Hispanic, 185 530 [90%] Black; PROVID: 642 570 [100%] Hispanic, 198 480 [96%] Black; CDC proxy: 605 770 [95%] Hispanic and 151 772 [74%] Black) compared with an age-based approach (438 423 [68%] Hispanic, 154 714 [75%] Black). Overall, the PROVID and joint CRS/PROVID risk-based strategies were estimated to be followed by the most patients from areas with high neighborhood deprivation index being vaccinated early. Conclusions and Relevance: In this simulation modeling study of adults from a large integrated health care delivery system, risk-based strategies were associated with the largest estimated reductions in COVID-19 hospitalizations, deaths, and household transmissions compared with the CDC proxy and age-based strategies, with a higher proportion of Hispanic and Black patients were estimated to be vaccinated early in the process compared with the CDC strategy.