Testing the Use of Data Drawn from the Electronic Health Record to Compare Quality.

INTRODUCTION: Health systems spend $1.5 billion annually reporting data on quality, but efficacy and utility for benchmarking are limited due, in part, to limitations of data sources. Our objective was to implement and evaluate measures of pediatric quality for three conditions using electronic health record (EHR)-derived data. METHODS: PCORnet networks standardized EHR-derived data to a common data model. In 13 health systems from 2 networks for 2015, we implemented the National Quality Forum measures: % children with sickle cell anemia who received a transcranial Doppler; % children on antipsychotics who had metabolic screening; and % pediatric acute otitis media with amoxicillin prescribed. Manual chart review assessed measure accuracy. RESULTS: Only 39% (N = 2,923) of 7,278 children on antipsychotics received metabolic screening (range: 20%-54%). If the measure indicated screening was performed, the chart agreed 88% of the time [95% confidence interval (CI): 81%-94%]; if it indicated screening was not done, the chart agreed 86% (95% CI: 78%-93%). Only 69% (N = 793) of 1,144 children received transcranial Doppler screening (range across sites: 49%-88%). If the measure indicated screening was performed, the chart agreed 98% of the time (95% CI: 94%-100%); if it indicated screening was not performed, the chart agreed 89% (95% CI: 82%-95%). For acute otitis media, chart review identified many qualifying cases missed by the National Quality Forum measure, which excluded a common diagnostic code. CONCLUSIONS: Measures of healthcare quality developed using EHR-derived data were valid and identified wide variation among network sites. This data can facilitate the identification and spread of best practices.

Development and evaluation of an EHR-based computable phenotype for identification of pediatric Crohn's disease patients in a National Pediatric Learning Health System.

OBJECTIVES: To develop and evaluate the classification accuracy of a computable phenotype for pediatric Crohn's disease using electronic health record data from PEDSnet, a large, multi-institutional research network and Learning Health System. STUDY DESIGN: Using clinician and informatician input, algorithms were developed using combinations of diagnostic and medication data drawn from the PEDSnet clinical dataset which is comprised of 5.6 million children from eight U.S. academic children's health systems. Six test algorithms (four cases, two non-cases) that combined use of specific medications for Crohn's disease plus the presence of Crohn's diagnosis were initially tested against the entire PEDSnet dataset. From these, three were selected for performance assessment using manual chart review (primary case algorithm, n = 360, primary non-case algorithm, n = 360, and alternative case algorithm, n = 80). Non-cases were patients having gastrointestinal diagnoses other than inflammatory bowel disease. Sensitivity, specificity, and positive predictive value (PPV) were assessed for the primary case and primary non-case algorithms. RESULTS: Of the six algorithms tested, the least restrictive algorithm requiring just ≥1 Crohn's diagnosis code yielded 11 950 cases across PEDSnet (prevalence 21/10 000). The most restrictive algorithm requiring ≥3 Crohn's disease diagnoses plus at least one medication yielded 7868 patients (prevalence 14/10 000). The most restrictive algorithm had the highest PPV (95%) and high sensitivity (91%) and specificity (94%). False positives were due primarily to a diagnosis reversal (from Crohn's disease to ulcerative colitis) or having a diagnosis of "indeterminate colitis." False negatives were rare. CONCLUSIONS: Using diagnosis codes and medications available from PEDSnet, we developed a computable phenotype for pediatric Crohn's disease that had high specificity, sensitivity and predictive value. This process will be of use for developing computable phenotypes for other pediatric diseases, to facilitate cohort identification for retrospective and prospective studies, and to optimize clinical care through the PEDSnet Learning Health System.

The Importance of Filter Collection for Accurate Measurement of Cigarette Smoking.

OBJECTIVES: We examined the impact of cigarette filter collection on reports of cigarettes per day (CPD) versus self-reported CPD and to assess the utility of a pre-intervention baseline period in smoking studies. METHODS: Using baseline data from 522 non-treatment seeking smokers, we assessed differences in self-reported CPD via phone screen (CPD PS) and during baseline (CPD BL). We analyzed self-reported cigarette measures to predict carbon monoxide (CO), a measure of smoke exposure. RESULTS: On average, CPD PS was 2.8 CPD more than CPD BL, and reporting multiples of 10 were more often found in CPD PS compared with CPD BL (54.7% vs17.2%, respectively). CPD BL was more strongly associated with CO than self-report CPD. Number of cigarettes smoked today, time since last cigarette, and nicotine dependence were significantly associated with CO. CONCLUSIONS: CPD BL using filter collection is a more accurate measure of cigarette consumption than self-report, which may have implications for assessment of nicotine dependence. When feasible, studies should include a pre-intervention baseline period for comparison data with study outcomes. In addition to CPD BL, studies should assess time since last cigarette and the number of cigarettes smoked today when using CO as a biological measure of smoke exposure.