Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system.

Template matching is a proposed approach for hospital benchmarking, which measures performance based on matching a subset of comparable patient hospitalizations from each hospital. We assessed the ability to create the required matched samples and thus the feasibility of template matching to benchmark hospital performance in a diverse healthcare system.Nationwide Veterans Affairs (VA) hospitals, 2017.Observational cohort study.We used administrative and clinical data from 668,592 hospitalizations at 134 VA hospitals in 2017. A standardized template of 300 hospitalizations was selected, and then 300 hospitalizations were matched to the template from each hospital.There was substantial case-mix variation across VA hospitals, which persisted after excluding small hospitals, hospitals with primarily psychiatric admissions, and hospitalizations for rare diagnoses. Median age ranged from 57 to 75 years across hospitals; percent surgical admissions ranged from 0.0% to 21.0%; percent of admissions through the emergency department, 0.1% to 98.7%; and percent Hispanic patients, 0.2% to 93.3%. Characteristics for which there was substantial variation across hospitals could not be balanced with any matching algorithm tested. Although most other variables could be balanced, we were unable to identify a matching algorithm that balanced more than ∼20 variables simultaneously.We were unable to identify a template matching approach that could balance hospitals on all measured characteristics potentially important to benchmarking. Given the magnitude of case-mix variation across VA hospitals, a single template is likely not feasible for general hospital benchmarking.

The Veterans Affairs Cardiac Risk Score: Recalibrating the Atherosclerotic Cardiovascular Disease Score for Applied Use.

BACKGROUND: Accurately estimating cardiovascular risk is fundamental to good decision-making in cardiovascular disease (CVD) prevention, but risk scores developed in one population often perform poorly in dissimilar populations. We sought to examine whether a large integrated health system can use their electronic health data to better predict individual patients' risk of developing CVD. METHODS: We created a cohort using all patients ages 45-80 who used Department of Veterans Affairs (VA) ambulatory care services in 2006 with no history of CVD, heart failure, or loop diuretics. Our outcome variable was new-onset CVD in 2007-2011. We then developed a series of recalibrated scores, including a fully refit "VA Risk Score-CVD (VARS-CVD)." We tested the different scores using standard measures of prediction quality. RESULTS: For the 1,512,092 patients in the study, the Atherosclerotic cardiovascular disease risk score had similar discrimination as the VARS-CVD (c-statistic of 0.66 in men and 0.73 in women), but the Atherosclerotic cardiovascular disease model had poor calibration, predicting 63% more events than observed. Calibration was excellent in the fully recalibrated VARS-CVD tool, but simpler techniques tested proved less reliable. CONCLUSIONS: We found that local electronic health record data can be used to estimate CVD better than an established risk score based on research populations. Recalibration improved estimates dramatically, and the type of recalibration was important. Such tools can also easily be integrated into health system's electronic health record and can be more readily updated.

Temporal Changes in the Influence of Hospitals and Regional Healthcare Networks on Severe Sepsis Mortality.

OBJECTIVES: There is systematic variation between hospitals in their care of severe sepsis, but little information on whether this variation impacts sepsis-related mortality, or how hospitals' and health-systems' impacts have changed over time. We examined whether hospital and regional organization of severe sepsis care is associated with meaningful differences in 30-day mortality in a large integrated health care system, and the extent to which those effects are stable over time. DESIGN: In this retrospective cohort study, we used risk- and reliability-adjusted hierarchical logistic regression to estimate hospital- and region-level random effects after controlling for severity of illness using a rich mix of administrative and clinical laboratory data. SETTING: One hundred fourteen U.S. Department of Veterans Affairs hospitals in 21 geographic regions. PATIENTS: Forty-three thousand seven hundred thirty-three patients with severe sepsis in 2012, compared to 33,095 such patients in 2008. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The median hospital in the worst quintile of performers had a risk-adjusted 30-day mortality of 16.7% (95% CI, 13.5%, 20.5%) in 2012 compared with the best quintile, which had a risk-adjusted mortality of 12.8% (95% CI, 10.7%, 15.3%). Hospitals and regions explained a statistically and clinically significant proportion of the variation in patient outcomes. Thirty-day mortality after severe sepsis declined from 18.3% in 2008 to 14.7% in 2012 despite very similar severity of illness between years. The proportion of the variance in sepsis-related mortality explained by hospitals and regions was stable between 2008 and 2012. CONCLUSIONS: In this large integrated healthcare system, there is clinically significant variation in sepsis-related mortality associated with hospitals and regions. The proportion of variance explained by hospitals and regions has been stable over time, although sepsis-related mortality has declined.

Despite variation in volume, Veterans Affairs hospitals show consistent outcomes among patients with non-postoperative mechanical ventilation.

OBJECTIVE: To assess the relationship between volume of nonoperative mechanically ventilated patients receiving care in a specific Veterans Health Administration hospital and their mortality. DESIGN: Retrospective cohort study. SETTING: One-hundred nineteen Veterans Health Administration medical centers. PATIENTS: We identified 5,131 hospitalizations involving mechanically ventilated patients in an intensive care unit during 2009, who did not receive surgery. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We extracted demographic and clinical data from the VA Inpatient Evaluation Center. For each hospital, we defined volume as the total number of nonsurgical admissions receiving mechanical ventilation in an intensive care unit during 2009. We examined the hospital contribution to 30-day mortality using multilevel logistic regression models with a random intercept for each hospital. We quantified the extent of interhospital variation in 30-day mortality using the intraclass correlation coefficient and median odds ratio. We used generalized estimating equations to examine the relationship between volume and 30-day mortality and risk-adjusted all models using a patient-level prognostic score derived from clinical data representing the risk of death conditional on treatment at a high-volume hospital. Mean age for the sample was 65 (SD 11) yrs, 97% were men, and 60% were white. The median VA hospital cared for 40 (interquartile range 19-62) mechanically ventilated patients in 2009. Crude 30-day mortality for these patients was 36.9%. After reliability and risk adjustment to the median patient, adjusted hospital-level mortality varied from 33.5% to 40.6%. The intraclass correlation coefficient for the hospital-level variation was 0.6% (95% confidence interval 0.1, 3.4%), with a median odds ratio of 1.15 (95% confidence interval 1.06, 1.38). The relationship between hospital volume of mechanically ventilated and 30-day mortality was not statistically significant: each 50-patient increase in volume was associated with a nonsignificant 2% decrease in the odds of death within 30 days (odds ratio 0.98, 95% confidence interval 0.87-1.10). CONCLUSIONS: Veterans Health Administration hospitals caring for lower volumes of mechanically ventilated patients do not have worse mortality. Mechanisms underlying this finding are unclear, but, if elucidated, may offer other integrated health systems ways to overcome the disadvantages of small-volume centers in achieving good outcomes.