Are Improvements in Measured Performance Driven by Better Treatment or "Denominator Management"?

BACKGROUND: Process measures of healthcare quality are usually formulated as the number of patients who receive evidence-based treatment (numerator) divided by the number of patients in the target population (denominator). When the systems being evaluated can influence which patients are included in the denominator, it is reasonable to wonder if improvements in measured quality are driven by expanding numerators or contracting denominators. OBJECTIVE: In 2003, the US Department of Veteran Affairs (VA) based executive compensation in part on performance on a substance use disorder (SUD) continuity-of-care quality measure. The first goal of this study was to evaluate if implementing the measure in this way resulted in expected improvements in measured performance. The second goal was to examine if the proportion of patients with SUD who qualified for the denominator contracted after the quality measure was implemented, and to describe the facility-level variation in and correlates of denominator contraction or expansion. DESIGN: Using 40 quarters of data straddling the implementation of the performance measure, an interrupted time series design was used to evaluate changes in two outcomes. PARTICIPANTS: All veterans with an SUD diagnosis in all VA facilities from fiscal year 2000 to 2009. MAIN MEASURES: The two outcomes were 1) measured performance-patients retained/patients qualified and 2) denominator prevalence-patients qualified/patients with SUD program contact. KEY RESULTS: Measured performance improved over time (P < 0.001). Notably, the proportion of patients with SUD program contact who qualified for the denominator decreased more rapidly after the measure was implemented (p = 0.02). Facilities with higher pre-implementation denominator prevalence had steeper declines in denominator prevalence after implementation (p < 0.001). CONCLUSIONS: These results should motivate the development of measures that are less vulnerable to denominator management, and also the exploration of "shadow measures" to monitor and reduce undesirable denominator management.

Measurement-based management of mental health quality and access in VHA: SAIL mental health domain.

We outline the development of a Mental Health Domain to track accessibility and quality of mental health care in the United States Veterans Health Administration (VHA) as part of a broad-based performance measurement system. This domain adds an important element to national performance improvement efforts by targeting regional and facility leadership and providing them a concise yet comprehensive measure to identify facilities facing challenges in their mental health programs. We present the conceptual framework and rationale behind measure selection and development. The Mental Health Domain covers three important aspects of mental health treatment: Population Coverage, Continuity of Care, and Experience of Care. Each component is a composite of existing and newly adapted measures with moderate to high internal consistency; components are statistically independent or moderately related. Development and dissemination of the Mental Health Domain involved a variety of approaches and benefited from close collaboration between local, regional, and national leadership and from coordination with existing quality-improvement initiatives. During the first year of use, facilities varied in the direction and extent of change. These patterns of change were generally consistent with qualitative information, providing support for the validity of the domain and its component measures. Measure maintenance remains an iterative process as the VHA mental health system and potential data resources continue to evolve. Lessons learned may be helpful to the broader mental health-provider community as mental health care consolidates and becomes increasingly integrated within healthcare systems. (PsycINFO Database Record

Telephone-based continuing care counseling in substance abuse treatment: Economic analysis of a randomized trial.

PURPOSE: To investigate whether telephone-based continuing care (TEL) is a promising alternative to traditional face-to-face counseling for clients in treatment for substance abuse. METHODS: Patients with alcohol and/or cocaine dependence who had completed a 4-week intensive outpatient program were randomly assigned through urn randomization into one of three 12-week interventions: standard continuing care (STD), in-person relapse prevention (RP), or telephone-based continuing care (TEL). This study performed cost, cost-effectiveness, and cost-benefit analyses of TEL and RP compared to STD, using results from the randomized clinical trial with two years of follow up (359 participants). In addition, the study examined the potential moderating effect of baseline patient costs on economic outcomes. RESULTS: The study found that TEL was less expensive per client from the societal perspective ($569) than STD ($870) or RP ($1684). TEL also was also significantly more effective, with an abstinence rate of 57.1% compared to 46.7% for STD (p<0.05). Thus TEL dominated STD, with a highly favorable negative incremental cost-effectiveness ratio (-$1400 per abstinent year). TEL also proved favorable under a benefit-cost perspective. CONCLUSIONS: TEL proved to be a cost-effective and cost-beneficial contributor to long-term recovery over two years. Because TEL dominated STD care interventions, wider adoption should be considered.

Directed funding to address under-provision of treatment for substance use disorders: a quantitative study.

BACKGROUND: Substance use disorders (SUDs) are a substantial problem in the United States (U.S.), affecting far more people than receive treatment. This is true broadly and within the U.S. military veteran population, which is our focus. To increase funding for treatment, the Veterans Health Administration (VA) has implemented several initiatives over the past decade to direct funds toward SUD treatment, supplementing the unrestricted funds VA medical centers receive. We study the 'flypaper effect' or the extent to which these directed funds have actually increased SUD treatment spending. METHODS: The study sample included all VA facilities and used observational data spanning years 2002 to 2010. Data were analyzed with a fixed effects, ordinary least squares specification with monetized workload as the dependent variable and funding dedicated to SUD specialty clinics the key dependent variable, controlling for unrestricted funding. RESULTS: We observed different effects of dedicated SUD specialty clinic funding over the period 2002 to 2008 versus 2009 to 2010. In the earlier period, there is no evidence of a significant portion of the dedicated funding sticking to its target. In the later period, a substantial proportion--38% in 2009 and 61% in 2010--of funding dedicated to SUD specialty clinics did translate into increased medical center spending for SUD treatment. In comparison, only five cents of every dollar of unrestricted funding is spent on SUD treatment. CONCLUSIONS: Relative to unrestricted funding, dedicated funding for SUD treatment was much more effective in increasing workload, but only in years 2009 and 2010. The differences in those years relative to prior ones may be due to the observed management focus on SUD and SUD-related treatment in the later years. If true, this suggests that in a centrally directed healthcare organization such as the VA, funding dedicated to a service is a necessary, but not sufficient condition for increasing resources expended for that service.