Testing the Use of Data Drawn from the Electronic Health Record to Compare Quality.

INTRODUCTION: Health systems spend $1.5 billion annually reporting data on quality, but efficacy and utility for benchmarking are limited due, in part, to limitations of data sources. Our objective was to implement and evaluate measures of pediatric quality for three conditions using electronic health record (EHR)-derived data. METHODS: PCORnet networks standardized EHR-derived data to a common data model. In 13 health systems from 2 networks for 2015, we implemented the National Quality Forum measures: % children with sickle cell anemia who received a transcranial Doppler; % children on antipsychotics who had metabolic screening; and % pediatric acute otitis media with amoxicillin prescribed. Manual chart review assessed measure accuracy. RESULTS: Only 39% (N = 2,923) of 7,278 children on antipsychotics received metabolic screening (range: 20%-54%). If the measure indicated screening was performed, the chart agreed 88% of the time [95% confidence interval (CI): 81%-94%]; if it indicated screening was not done, the chart agreed 86% (95% CI: 78%-93%). Only 69% (N = 793) of 1,144 children received transcranial Doppler screening (range across sites: 49%-88%). If the measure indicated screening was performed, the chart agreed 98% of the time (95% CI: 94%-100%); if it indicated screening was not performed, the chart agreed 89% (95% CI: 82%-95%). For acute otitis media, chart review identified many qualifying cases missed by the National Quality Forum measure, which excluded a common diagnostic code. CONCLUSIONS: Measures of healthcare quality developed using EHR-derived data were valid and identified wide variation among network sites. This data can facilitate the identification and spread of best practices.

Use of EHR-Based Pediatric Quality Measures: Views of Health System Leaders and Parents.

Measures of health care quality are produced from a variety of data sources, but often, physicians do not believe these measures reflect the quality of provided care. The aim was to assess the value to health system leaders (HSLs) and parents of benchmarking on health care quality measures using data mined from the electronic health record (EHR). Using in-context interviews with HSLs and parents, the authors investigated what new decisions and actions benchmarking using data mined from the EHR may enable and how benchmarking information should be presented to be most informative. Results demonstrate that although parents may have little experience using data on health care quality for decision making, they affirmed its potential value. HSLs expressed the need for high-confidence, validated metrics. They also perceived barriers to achieving meaningful metrics but recognized that mining data directly from the EHR could overcome those barriers. Parents and HSLs need high-confidence health care quality data to support decision making.