Statistical benchmarks for process measures of quality of care for mental and substance use disorders.

OBJECTIVE: Benchmarks, representing the level of performance achieved by the best-performing providers, can be used to set achievable goals for improving care, but they have not heretofore been available for mental health care. This article describes the application of a method for developing statistical benchmarks for 12 process measures of quality of care for mental and substance use disorders. METHODS: Twelve quality measures--taken from a core measure set selected by a multistakeholder panel through a formal consensus process--were constructed from 1994-1995 administrative data on care received by Medicaid beneficiaries in six states. Conformance rates were calculated at the provider level and presented as means, 90th-percentile results, and statistical benchmarks. Sample sizes for each measure ranged from 356 to 4,494 providers and from 1,205 to 78,627 cases. Three measures involved antidepressant treatment, two involved antipsychotic treatment, and one involved mood stabilizers for bipolar disorder. Six other measures involved follow-up treatment visits. RESULTS: Benchmarks for provider-level performance ranged from 59.7 percent to 97.7 percent, markedly higher than the mean results, which ranged from 9.4 percent to 65.4 percent. Benchmark results varied widely-in contrast to results for these measures at the 90th percentile of providers and in contrast to performance standards that apply the same numerical goal across varied clinical processes. CONCLUSIONS: Statistical benchmarks can be applied to results from quality assessment of mental health care. Further research should examine whether incorporating benchmarks into quality improvement activities leads to better mental health care and substance-related care and improved outcomes.

Achieving consensus across diverse stakeholders on quality measures for mental healthcare.

OBJECTIVE: Quality-improvement efforts are hindered by a lack of consensus on meaningful and feasible measures of care. The objective of this study was to develop a core set of quality measures for mental health and substance-related care that are meaningful to stakeholders, feasible to implement, and broadly representative of diverse dimensions of the mental health system. METHOD: A 12-member panel of stakeholders from national organizations evaluated 116 process measures in a 2-stage modified Delphi consensus development process. Drawing on a conceptual framework and literature review, panelists rated each measure on 7 domains using a 9-point scale (1 = best). Measures were then mapped to a framework of system dimensions to identify a core set with the highest ratings for system characteristics within each dimension. RESULTS: Twenty-eight measures were identified assessing treatment (12), access (2), assessment (2), continuity (4), coordination (2), prevention (1), and safety (5). Overall, mean ratings for meaningfulness were: clinical importance 2.29; perceived gap between actual and optimal care 2.59; association between improved performance and outcome 2.61. For feasibility, mean ratings were clarity of specifications 3.39; acceptability of data collection burden 4.77; and adequacy of case mix adjustment 4.20. The measures address a range of treatment modalities, clinical settings, diagnostic categories, vulnerable populations, and other dimensions of mental healthcare. CONCLUSIONS: A structured consensus process identified a core set of quality measures that are meaningful and feasible to multiple stakeholders, as well as broadly representative of the mental healthcare system. By yielding quantitative assessments of meaningfulness, feasibility and degree of consensus among stakeholders, these results can inform ongoing national efforts to adopt common quality measures for mental healthcare.