Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system.

Template matching is a proposed approach for hospital benchmarking, which measures performance based on matching a subset of comparable patient hospitalizations from each hospital. We assessed the ability to create the required matched samples and thus the feasibility of template matching to benchmark hospital performance in a diverse healthcare system.Nationwide Veterans Affairs (VA) hospitals, 2017.Observational cohort study.We used administrative and clinical data from 668,592 hospitalizations at 134 VA hospitals in 2017. A standardized template of 300 hospitalizations was selected, and then 300 hospitalizations were matched to the template from each hospital.There was substantial case-mix variation across VA hospitals, which persisted after excluding small hospitals, hospitals with primarily psychiatric admissions, and hospitalizations for rare diagnoses. Median age ranged from 57 to 75 years across hospitals; percent surgical admissions ranged from 0.0% to 21.0%; percent of admissions through the emergency department, 0.1% to 98.7%; and percent Hispanic patients, 0.2% to 93.3%. Characteristics for which there was substantial variation across hospitals could not be balanced with any matching algorithm tested. Although most other variables could be balanced, we were unable to identify a matching algorithm that balanced more than ∼20 variables simultaneously.We were unable to identify a template matching approach that could balance hospitals on all measured characteristics potentially important to benchmarking. Given the magnitude of case-mix variation across VA hospitals, a single template is likely not feasible for general hospital benchmarking.

Does adding clinical data to administrative data improve agreement among hospital quality measures?

BACKGROUND: Hospital performance measures based on patient mortality and readmission have indicated modest rates of agreement. We examined if combining clinical data on laboratory tests and vital signs with administrative data leads to improved agreement with each other, and with other measures of hospital performance in the nation's largest integrated health care system. METHODS: We used patient-level administrative and clinical data, and hospital-level data on quality indicators, for 2007-2010 from the Veterans Health Administration (VA). For patients admitted for acute myocardial infarction (AMI), heart failure (HF) and pneumonia we examined changes in hospital performance on 30-d mortality and 30-d readmission rates as a result of adding clinical data to administrative data. We evaluated whether this enhancement yielded improved measures of hospital quality, based on concordance with other hospital quality indicators. RESULTS: For 30-d mortality, data enhancement improved model performance, and significantly changed hospital performance profiles; for 30-d readmission, the impact was modest. Concordance between enhanced measures of both outcomes, and with other hospital quality measures - including Joint Commission process measures, VA Surgical Quality Improvement Program (VASQIP) mortality and morbidity, and case volume - remained poor. CONCLUSIONS: Adding laboratory tests and vital signs to measure hospital performance on mortality and readmission did not improve the poor rates of agreement across hospital quality indicators in the VA. INTERPRETATION: Efforts to improve risk adjustment models should continue; however, evidence of validation should precede their use as reliable measures of quality.

Improving Pregnancy Outcomes through Maternity Care Coordination: A Systematic Review.

BACKGROUND: Care during pregnancy is multifaceted and often goes beyond traditional prenatal care from an obstetrical care provider. Coordinating care between multiple providers can be challenging, but is beneficial for providers and patients. Care coordination is associated with decreased costs, greater patient satisfaction, and a reduction in medical errors. To our knowledge, no previous review has examined maternity care coordination (MCC) programs and their association with pregnancy outcomes. METHODS: Using a search algorithm comprised of relevant MCC terminology, studies were identified through a systematic search of PubMed, Scopus, ClinicalTrials.gov, and Google Scholar. Studies meeting eligibility criteria (e.g., defining the care coordination components and examining at least one quantitative outcome) were fully abstracted and quality rated using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist. MAIN FINDINGS: Thirty-three observational studies of MCC were included in this review. Quality scores ranged from 27% to 100%. Most studies included strategies with a team approach to decision making and/or individual case management. Social service referrals to outside organizations were also common. Twenty-seven studies reported infant birth weight as a main outcome; 12 found a significant improvement in birth weights among care coordination participants. CONCLUSIONS: Roughly one-third of the included studies reported improved birth weights among care coordination participants. However, it remains unknown what effect care coordination strategies have on patient and provider satisfaction in the prenatal care setting, two aspects of maternity care that may advance the quality and utilization of prenatal health services.

Use of risk-adjusted change in health status to assess the performance of integrated service networks in the Veterans Health Administration.

OBJECTIVE: Health outcome assessments have become an expectation of regulatory and accreditation agencies. We examined whether a clinically credible risk adjustment methodology for the outcome of change in health status can be developed for performance assessment of integrated service networks. STUDY DESIGN: Longitudinal study. SETTING: Outpatient. STUDY PARTICIPANTS: Thirty-one thousand eight hundred and twenty-three patients from 22 Veterans Health Administration (VHA) integrated service networks were followed for 18 months. MAIN MEASURE: The physical (PCS) and mental (MCS) component scales from the Veterans Rand 36-items Health Survey (VR-36) and mortality. The outcomes were decline in PCS (decline in PCS scores greater than -6.5 points or death) and MCS (decline in MCS scores greater than -7.9 points). RESULTS: Four thousand three hundred and twenty-eight (13.6%) patients showed a decline in PCS scores greater than -6.5 points, 4322 (13.5%) had a decline in MCS scores by more than -7.9 points, and 1737 died (5.5%). Multivariate logistic regression models were used to adjust for case-mix. The models performed reasonably well in cross-validated tests of discrimination (c-statistics = 0.72 and 0.68 for decline in PCS and MCS, respectively) and calibration. The resulting risk-adjusted rates of decline in PCS and MCS and ranks of the networks differed considerably from unadjusted ratings. CONCLUSION: It is feasible to develop clinically credible risk adjustment models for the outcomes of decline in PCS and MCS. Without adequate controls for case-mix, we could not determine whether poor patient outcomes reflect poor performance, sicker patients, or other factors. This methodology can help to measure and report the performance of health care systems.

Do different case-mix measures affect assessments of provider efficiency? Lessons from the Department of Veterans Affairs.

Although case-mix adjustment is critical for provider profiling, little is known regarding whether different case-mix measures affect assessments of provider efficiency. We examine whether two case-mix measures, Adjusted Clinical Groups (ACGs) and Diagnostic Cost Groups (DCGs), result in different assessments of efficiency across service networks within the Department of Veterans Affairs (VA). Three profiling indicators examine variation in resource use. Although results from the ACGs and DCGs generally agree on which networks have greater or lesser efficiency than average, assessments of individual network efficiency vary depending upon the case-mix measure used. This suggests that caution should be used so that providers are not misclassified based on reported efficiency.

Measuring the quality of depression care in a large integrated health system.

BACKGROUND: Guideline-based depression process measures provide a powerful way to monitor depression care and target areas needing improvement. OBJECTIVES: To assess the adequacy of depression care in the Veterans Health Administration (VHA) using guideline-based process measures derived from administrative and centralized pharmacy records, and to identify patient and provider characteristics associated with adequate depression care. RESEARCH DESIGN: This is a cohort study of patients from 14 VHA hospitals in the Northeastern United States which relied on existing databases. Subject eligibility criteria: at least one depression diagnosis during 1999, neither schizophrenia nor bipolar disease, and at least one antidepressant prescribed in the VHA during the period of depression care profiling (June 1, 1999 through August 31, 1999). Depression care was evaluated with process measures defined from the 1997 VHA depression guidelines: antidepressant dosage and duration adequacy. We used multivariable regression to identify patient and provider characteristics predicting adequate care. SUBJECTS: There were 12,678 patients eligible for depression care profiling. RESULTS: Adequate dosage was identified in 90%; 45% of patients had adequate duration of antidepressants. Significant patient and provider characteristics predicting inadequate depression care were younger age (<65), black race, and treatment exclusively in primary care. CONCLUSIONS: Under-treatment of depression exists in the VHA, despite considerable mental health access and generous pharmacy benefits. Certain patient populations may be at higher risk for inadequate depression care. More work is needed to align current practice with best-practice guidelines and to identify optimal ways of using available data sources to monitor depression care quality.

Profiling resource use: do different outcomes affect assessments of provider efficiency?

OBJECTIVES: To examine whether 2 outcome measures result in different assessments of efficiency across 22 service networks within the Department of Veterans Affairs (VA). STUDY DESIGN: A retrospective analysis using VA inpatient and outpatient administrative databases. METHODS: A 60% random sample of veterans who used healthcare services during fiscal year 1997 was split into a 40% sample (n = 1,046,803) for development and a 20% sample (n = 524,461) for validation. Weighted concurrent case-mix models using adjusted clinical groups were developed to explain variation in 2 outcomes: "days of care"--the sum of a patient's inpatient and outpatient annual visit days, and "average accounting costs"--the sum of the average service costs multiplied by the units of service for each patient. Two profiling indicators were calculated for each outcome: an unadjusted efficiency index and an adjusted efficiency index. These indices were compared to examine network efficiency. RESULTS: Although about half the networks were identified as "efficient" before and after case-mix adjustment, assessments of individual network efficiency were affected by the adjustment. The 2 outcomes differed on which networks were efficient. For example, 4 networks that appeared as efficient based on days of care appeared as inefficient based on average costs. CONCLUSIONS: Assessments of provider efficiency across the 22 networks depended on the outcome measure used. Knowledge about the extent to which assessments of provider efficiency depend on the outcome measure used is an important step toward improved and more equitable comparisons across providers.