Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system.

Template matching is a proposed approach for hospital benchmarking, which measures performance based on matching a subset of comparable patient hospitalizations from each hospital. We assessed the ability to create the required matched samples and thus the feasibility of template matching to benchmark hospital performance in a diverse healthcare system.Nationwide Veterans Affairs (VA) hospitals, 2017.Observational cohort study.We used administrative and clinical data from 668,592 hospitalizations at 134 VA hospitals in 2017. A standardized template of 300 hospitalizations was selected, and then 300 hospitalizations were matched to the template from each hospital.There was substantial case-mix variation across VA hospitals, which persisted after excluding small hospitals, hospitals with primarily psychiatric admissions, and hospitalizations for rare diagnoses. Median age ranged from 57 to 75 years across hospitals; percent surgical admissions ranged from 0.0% to 21.0%; percent of admissions through the emergency department, 0.1% to 98.7%; and percent Hispanic patients, 0.2% to 93.3%. Characteristics for which there was substantial variation across hospitals could not be balanced with any matching algorithm tested. Although most other variables could be balanced, we were unable to identify a matching algorithm that balanced more than ∼20 variables simultaneously.We were unable to identify a template matching approach that could balance hospitals on all measured characteristics potentially important to benchmarking. Given the magnitude of case-mix variation across VA hospitals, a single template is likely not feasible for general hospital benchmarking.

The Veterans Affairs Cardiac Risk Score: Recalibrating the Atherosclerotic Cardiovascular Disease Score for Applied Use.

BACKGROUND: Accurately estimating cardiovascular risk is fundamental to good decision-making in cardiovascular disease (CVD) prevention, but risk scores developed in one population often perform poorly in dissimilar populations. We sought to examine whether a large integrated health system can use their electronic health data to better predict individual patients' risk of developing CVD. METHODS: We created a cohort using all patients ages 45-80 who used Department of Veterans Affairs (VA) ambulatory care services in 2006 with no history of CVD, heart failure, or loop diuretics. Our outcome variable was new-onset CVD in 2007-2011. We then developed a series of recalibrated scores, including a fully refit "VA Risk Score-CVD (VARS-CVD)." We tested the different scores using standard measures of prediction quality. RESULTS: For the 1,512,092 patients in the study, the Atherosclerotic cardiovascular disease risk score had similar discrimination as the VARS-CVD (c-statistic of 0.66 in men and 0.73 in women), but the Atherosclerotic cardiovascular disease model had poor calibration, predicting 63% more events than observed. Calibration was excellent in the fully recalibrated VARS-CVD tool, but simpler techniques tested proved less reliable. CONCLUSIONS: We found that local electronic health record data can be used to estimate CVD better than an established risk score based on research populations. Recalibration improved estimates dramatically, and the type of recalibration was important. Such tools can also easily be integrated into health system's electronic health record and can be more readily updated.

The role of clinical uncertainty in treatment decisions for diabetic patients with uncontrolled blood pressure.

BACKGROUND: Factors underlying failure to intensify therapy in response to elevated blood pressure have not been systematically studied. OBJECTIVE: To examine the process of care for diabetic patients with elevated triage blood pressure (> or =140/90 mm Hg) during routine primary care visits to assess whether a treatment change occurred and to what degree specific patient and provider factors correlated with the likelihood of treatment change. DESIGN: Prospective cohort study. SETTING: 9 Veterans Affairs facilities in 3 midwestern states. PARTICIPANTS: 1169 diabetic patients with scheduled visits to 92 primary care providers from February 2005 to March 2006. MEASUREMENTS: Proportion of patients who had a change in a blood pressure treatment (medication intensification or planned follow-up within 4 weeks). Predicted probability of treatment change was calculated from a multilevel logistic model that included variables assessing clinical uncertainty, competing demands and prioritization, and medication-related factors (controlling for blood pressure). RESULTS: Overall, 573 (49%) patients had a blood pressure treatment change at the visit. The following factors made treatment change less likely: repeated blood pressure by provider recorded as less than 140/90 mm Hg versus 140/90 mm Hg or greater or no recorded repeated blood pressure (13% vs. 61%; P < 0.001); home blood pressure reported by patients as less than 140/90 mm Hg versus 140/90 mm Hg or greater or no recorded home blood pressure (18% vs. 52%; P < 0.001); provider systolic blood pressure goal greater than 130 mm Hg versus 130 mm Hg or less (33% vs. 52%; P = 0.002); discussion of conditions unrelated to hypertension and diabetes versus no discussion (44% vs. 55%; P = 0.008); and discussion of medication issues versus no discussion (23% vs. 52%; P < 0.001). LIMITATION: Providers knew that the study pertained to diabetes and hypertension, and treatment change was assessed for 1 visit per patient. CONCLUSION: Approximately 50% of diabetic patients presenting with a substantially elevated triage blood pressure received treatment change at the visit. Clinical uncertainty about the true blood pressure value was a prominent reason that providers did not intensify therapy.

Whom should we profile? Examining diabetes care practice variation among primary care providers, provider groups, and health care facilities.

OBJECTIVE: To evaluate the amount of variation in diabetes practice patterns at the primary care provider (PCP), provider group, and facility level, and to examine the reliability of diabetes care profiles constructed using electronic databases. DATA SOURCES/STUDY SETTING: Clinical and administrative data obtained from the electronic information systems at all facilities in a Department of Veterans Affairs' (VA) integrated service network for a study period of October 1997 through September 1998. STUDY DESIGN: This is a cohort study. The key variables of interest are different types of diabetes quality indicators, including measures of technical process, intermediate outcomes, and resource use. DATA COLLECTION/EXTRACTION METHODS: A coordinated registry of patients with diabetes was constructed by integrating laboratory, pharmacy, utilization, and primary care provider data extracted from the local clinical information system used at all VA medical centers. The study sample consisted of 12,110 patients with diabetes, 258 PCPs, 42 provider groups, and 13 facilities. PRINCIPAL FINDINGS: There were large differences in the amount of practice variation across levels of care and for different types of diabetes care indicators. The greatest amount of variance tended to be attributable to the facility level. For process measures, such as whether a hemoglobin A1c was measured, the facility and PCP effects were generally comparable. However, for three resource use measures the facility effect was at least six times the size of the PCP effect, and for inter-mediate outcome indicators, such as hyperlipidemia, facility effects ranged from two to sixty times the size of the PCP level effect. A somewhat larger PCP effect was found (5 percent of the variation) when we examined a "linked" process-outcome measure linking hyperlipidemia and treatment with statins). When the PCP effect is small (i.e., 2 percent), a panel of two hundred diabetes patients is needed to construct profiles with 80 percent reliability. CONCLUSIONS: little of the variation in many currently measured diabetes care practices is attributable to PCPs and, unless panel sizes are large, PCP profiling will be inaccurate. If profiling is to improve quality, it may be best to focus on examining facility-level performance variations and on developing indicators that promote specific, high-priority clinical actions.

Comparing clinical automated, medical record, and hybrid data sources for diabetes quality measures.

BACKGROUND: Little is known about the relative reliability of medical record and clinical automated data, sources commonly used to assess diabetes quality of care. The agreement between diabetes quality measures constructed from clinical automated versus medical record data sources was compared, and the performance of hybrid measures derived from a combination of the two data sources was examined. METHODS: Medical records were abstracted for 1,032 patients with diabetes who received care from 21 facilities in 4 Veterans Integrated Service Networks. Automated data were obtained from a central Veterans Health Administration diabetes registry containing information on laboratory tests and medication use. RESULTS: Success rates were higher for process measures derived from medical record data than from automated data, but no substantial differences among data sources were found for the intermediate outcome measures. Agreement for measures derived from the medical record compared with automated data was moderate for process measures but high for intermediate outcome measures. Hybrid measures yielded success rates similar to those of medical record-based measures but would have required about 50% fewer chart reviews. CONCLUSIONS: Agreement between medical record and automated data was generally high. Yet even in an integrated health care system with sophisticated information technology, automated data tended to underestimate the success rate in technical process measures for diabetes care and yielded different quartile performance rankings for facilities. Applying hybrid methodology yielded results consistent with the medical record but required less data to come from medical record reviews.