Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System.

BACKGROUND: The Meaningful Use (MU) program has increased the national emphasis on electronic measurement of hospital quality. OBJECTIVE: To evaluate stroke MU and one VHA stroke electronic clinical quality measure (eCQM) in national VHA data and determine sources of error in using centralized electronic health record (EHR) data. DESIGN: Our study is a retrospective cross-sectional study of stroke quality measure eCQMs vs. chart review in a national EHR. We developed local SQL algorithms to generate the eCQMs, then modified them to run on VHA Central Data Warehouse (CDW) data. eCQM results were generated from CDW data in 2130 ischemic stroke admissions in 11 VHA hospitals. Local and CDW results were compared to chart review. MAIN MEASURES: We calculated the raw proportion of matching cases, sensitivity/specificity, and positive/negative predictive values (PPV/NPV) for the numerators and denominators of each eCQM. To assess overall agreement for each eCQM, we calculated a weighted kappa and prevalence-adjusted bias-adjusted kappa statistic for a three-level outcome: ineligible, eligible-passed, or eligible-failed. KEY RESULTS: In five eCQMs, the proportion of matched cases between CDW and chart ranged from 95.4 %-99.7 % (denominators) and 87.7 %-97.9 % (numerators). PPVs tended to be higher (range 96.8 %-100 % in CDW) with NPVs less stable and lower. Prevalence-adjusted bias-adjusted kappas for overall agreement ranged from 0.73-0.95. Common errors included difficulty in identifying: (1) mechanical VTE prophylaxis devices, (2) hospice and other specific discharge disposition, and (3) contraindications to receiving care processes. CONCLUSIONS: Stroke MU indicators can be relatively accurately generated from existing EHR systems (nearly 90 % match to chart review), but accuracy decreases slightly in central compared to local data sources. To improve stroke MU measure accuracy, EHRs should include standardized data elements for devices, discharge disposition (including hospice and comfort care status), and recording contraindications.

Development and Validation of Electronic Quality Measures to Assess Care for Patients With Transient Ischemic Attack and Minor Ischemic Stroke.

BACKGROUND: Despite interest in using electronic health record (EHR) data to assess quality of care, the accuracy of such data is largely unknown. We sought to develop and validate transient ischemic attack and minor ischemic stroke electronic quality measures (eQMs) using EHR data. METHODS AND RESULTS: A random sample of patients with transient ischemic attack or minor ischemic stroke, cared for in Veterans Health Administration facilities (fiscal year 2011), was identified. We constructed 31 eQMs based on existing quality measures. Chart review was the criterion standard for validating the eQMs. To evaluate eQMs in terms of eligibility, we calculated the proportion of patients who were genuinely not eligible to receive a process (based on chart review) and who were correctly identified as not eligible by the EHR data (specificity). To assess eQMs about classification of whether patients received a process, we calculated the proportion of patients who actually received the process (based on chart review) and who were classified correctly by the EHR data as passing (sensitivity). Seven hundred sixty-three patients were included. About eligibility, specificity varied from 25% (brain imaging; carotid imaging) to 99% (anticoagulation quality). About pass rates, sensitivity varied from 30% (antihypertensive class) to 100% (coronary risk assessment; international normalized ratio measured). The 16 eQMs with ≥70% specificity in eligibility and ≥70% sensitivity in pass rates included coronary risk assessment, international normalized ratio measured, HbA1c measurement, speech language pathology consultation, anticoagulation for atrial fibrillation, discharge on statin, lipid management, neurology consultation, Holter, deep vein thrombosis prophylaxis, oral hypoglycemic intensification, cholesterol medication intensification, antihypertensive intensification, antihypertensive class, carotid stenosis intervention, and substance abuse referral for alcohol. CONCLUSIONS: It is feasible to construct valid eQMs for processes of transient ischemic attack and minor ischemic stroke care. Healthcare systems with EHRs should consider using electronic data to evaluate care for their patients with transient ischemic attack and to complement and expand quality measurement programs currently focused on patients with stroke.