Uses and Limitations of Claims-based Performance Feedback Reports: Lessons From the Comprehensive Primary Care Initiative.

BACKGROUND: Performance feedback is central to data-driven models of quality improvement, but the use of claims-based data for feedback has received little attention. PURPOSE: To examine the challenges, uses, and limitations of quarterly Medicare claims-based performance feedback reports generated for practices participating in the Comprehensive Primary Care (CPC) initiative from 2012 to 2015. METHODS: Mixed methods study of nearly 500 CPC practices in seven regions, combining pilot testing; systematic monitoring; surveys; in-depth interviews; user feedback; and input from data feedback team. RESULTS: Designing reports required addressing issues about timing, data completeness and reliability, variations in patient risk across practices, and use of benchmarks and metrics understandable to users. Practices' ability to use reports constructively depended on their experience, analytic resources, expectations, and perceptions about the role of primary care in improving reported outcomes. CONCLUSIONS: Generating claims-based feedback reports that support practices' quality improvement efforts requires a significant investment of analytic expertise, time, resources, continuous improvement, and technical assistance. IMPLICATIONS: Claims-based performance feedback can provide insight into patterns of patients' care across provider settings and opportunities for improvement, but practices need data from other sources to manage patients in real time or assess the short-term effects of specific changes in care delivery.

Testing consumers' comprehension of quality measures using alternative reporting formats.

CMS has publicly reported nursing home quality measures since 2002, but research has shown that many users do not understand them. Alternative visual displays may improve comprehension. We developed seven reporting templates in different formats, including bar graphs like those displayed on the CMS Nursing Home Compare Web site www.medicare.gov, and tested them with 90 individuals age 45-75, using structured protocols. Tests of significance were conducted, and statistically significant findings identified. Fewer than one-half the respondents accurately interpreted bar graphs as currently displayed on the Nursing Home Compare Web site. Respondents made fewest errors on templates using words to characterize performance as better, average, or worse.