Patient Surveys Are Used Most Often to Assess Health Care Innovations, Rigorous Methods Are Less Common.

The Agency of Healthcare Research and Quality Healthcare Innovations Exchange (IE) was developed to collect and report on innovative approaches to improving health care. The team reviewed 348 IE innovations including patient-reported satisfaction or experience measures. Innovations most often measured overall rating of care (61% of innovations), followed by access (52%) and provider-patient communication (12%). More than half used patient satisfaction surveys (n = 187) rather than patient experience surveys (n = 64). Innovations using patient experience surveys more often measured specific aspects of patient care, for example, access, versus a general overall rating of care. Most innovations using patient experience surveys administered nonvalidated, homegrown surveys, with few using the Agency of Healthcare Research and Quality-endorsed, psychometrically-tested CAHPS (Consumer Assessment of Healthcare Providers and Systems) survey. The most common study design was postimplementation-only (65%), highlighting that methodological rigor used to assess patient-centeredness in the IE is low. Broad use of patient experience surveys and more rigorous evaluation study designs has increased some over time but is still lacking.

AHRQ's Healthcare Extension Service-State-Based Solutions to Health Care Improvement.

This Viewpoint from AHRQ describes the plan to create a national health care extension service to disseminate actionable knowledge, with a goal to reduce the gap from evidence of clinical effectiveness to clinical practice.

What AHRQ Learned While Working to Transform Primary Care.

Building on previous efforts to transform primary care, the Agency for Healthcare Research and Quality (AHRQ) launched EvidenceNOW: Advancing Heart Health in 2015. This 3-year initiative provided external quality improvement support to small and medium-size primary care practices to implement evidence-based cardiovascular care. Despite challenges, results from an independent national evaluation demonstrated that the EvidenceNOW model successfully boosted the capacity of primary care practices to improve quality of care, while helping to advance heart health. Reflecting on AHRQ's own learnings as the funder of this work, 3 key lessons emerged: (1) there will always be surprises that will require flexibility and real-time adaptation; (2) primary care transformation is about more than technology; and (3) it takes time and experience to improve care delivery and health outcomes. EvidenceNOW taught us that lasting practice transformation efforts need to be responsive to anticipated and unanticipated changes, relationship-oriented, and not tied to a specific disease or initiative. We believe these lessons argue for a national primary care extension service that provides ongoing support for practice transformation.

A Critical Appraisal of AHRQ's "Diagnostic Errors" Report.

The Agency for Healthcare Research and Quality (AHRQ) report "Diagnostic Errors in the Emergency Department" generated significant mass media interest with negative implications for the safety of contemporary emergency care. The assumptions and methodology underlying this report are problematic, while multiple ongoing efforts to improve the quality and quantity of diagnostic research are missed, neglected, or ignored. The AHRQ report identifies reasonable target diseases for targeting diagnostic quality improvement efforts, as well as viable methods by which to measure any initiatives impact on diagnostic error. We note additional opportunities to improve the status quo by funding emergency department-based diagnostic research and healthcare system-level patient safety research and highlighting innovative approaches to diagnostic science within emergency medicine.

Update of the agency for healthcare research and quality guidance on using nonrandomized studies in evidence syntheses.

▪.

Feasibility of Capturing Adverse Events From Insurance Claims Data Using International Classification of Diseases, Tenth Revision, Codes Coupled to Present on Admission Indicators.

OBJECTIVE: The aim of the study was to investigate the feasibility of using administrative data to screen adverse events in Korea. METHODS: We used a diagnosis-related groups claims data set and the information of the checklist of healthcare quality improvement (a part of the value incentive program) to verify adverse events in fiscal year 2018. Adverse events were identified using patient safety indicator (PSI) clusters and a present on admission indicator (POA). The PSIs consisted of 19 clusters representing subcategories of adverse events, such as hospital-acquired infection. Among the adverse events identified using PSI clusters, "POA = N," which means not present at the time of admission, was only deemed as the case in the final stage. We compared the agreement on the occurrence of adverse events from claims data with a reference standard data set (i.e., checklist of healthcare quality improvement) and presented them by PSI cluster and institution. RESULTS: The cases of global PSI for any adverse event numbered 27,320 (2.32%) among all diagnostic codes in 2018. In terms of institutional distribution, considerable variation was observed throughout the clusters. For example, only 13.2% of institutions (n = 387) reported any global PSI for any adverse event throughout the whole year. The agreement between the reference standard and the claims data was poor, in the range of 2.2% to 10.8%, in 3 types of adverse events. The current claims data system (i.e., diagnostic codes coupled to POA indicators) failed to capture a large majority of adverse events identified using the reference standard. CONCLUSIONS: Our results imply that the coding status of International Classification of Diseases, Tenth Revision, codes and POA indicators should be refined before using them as quality indicators.

Relationships Between Pediatric Safety Indicators Across a National Sample of Pediatric Hospitals: Dispelling the Myth of the "Safest" Hospital.

OBJECTIVE: There are many measures of healthcare quality, but no obvious summary measures to simplify ranking of hospital performance. With public reporting and accountability for hospital performance, the validity of composite measures for performance rankings has increased importance. This study aimed to explore the covariance of pediatric hospital quality indicators and evaluate the use of a single composite score. METHODS: We performed an observational study of pediatric hospital performance across 13 safety indicators extracted from the Pediatric Health Information System, a comparative database of children's hospitals in the United States. We included patients discharged from 36 hospitals from January 1, 2016, to December 31, 2019. Using principal components analysis, we investigate relationships among patient safety measures from the Agency for Healthcare Research and Quality pediatric quality indicators and Center for Medicare and Medicaid Services hospital-acquired conditions. We compare and summarize rankings based on individual safety indicators and calculate alternative composite scores. RESULTS: We identified 5 orthogonal variance components accounting for 68% of variation in pediatric hospital quality indicators. Rankings demonstrated greater within-hospital variation compared with between-hospital variation. We observed discordant rankings across commonly used summary measures and conclude that these pediatric safety measures demonstrate at least 2 underlying variance components. CONCLUSIONS: This study demonstrates the multifactorial nature of patient safety. This implies no unique ordering of hospitals based on these measures, and thus, no pediatric hospital can claim to be "the safest." This raises further questions about appropriate methods to rank hospitals by safety.

Implementation of an Antibiotic Stewardship Program in Long-term Care Facilities Across the US.

IMPORTANCE: Antibiotic overuse in long-term care (LTC) is common, prompting calls for antibiotic stewardship programs (ASPs) designed for specific use in these settings. The optimal approach to establish robust, sustainable ASPs in LTC facilities is unknown. OBJECTIVES: To determine if the Agency for Healthcare Research and Quality (AHRQ) Safety Program for Improving Antibiotic Use, an educational initiative to establish ASPs focusing on patient safety, is associated with reductions in antibiotic use in LTC settings. DESIGN, SETTING, AND PARTICIPANTS: This quality improvement study including 439 LTC facilities in the US assessed antibiotic therapy data following a pragmatic quality-improvement program, which was implemented to assist facilities in establishing ASPs and with antibiotic decision-making. Training was conducted between December 2018 and November 2019. Data were analyzed from January 2019 to December 2019. INTERVENTIONS: Fifteen webinars occurred over 12 months (December 2018 to November 2019), accompanied by additional tools, activities, posters, and pocket cards. All clinical staff were encouraged to participate. MAIN OUTCOMES AND MEASURES: The primary outcome was antibiotic starts per 1000 resident-days. Secondary outcomes included days of antibiotic therapy (DOT) per 1000 resident-days, the number of urine cultures per 1000 resident-days, and Clostridioides difficile laboratory-identified events per 10 000 resident-days. All outcomes compared data from the baseline (January-February 2019) to the completion of the program (November-December 2019). Generalized linear mixed models with random intercepts at the site level assessed changes over time. RESULTS: Of a total 523 eligible LTC facilities, 439 (83.9%) completed the safety program. The mean difference for antibiotic starts from baseline to study completion per 1000 resident-days was -0.41 (95% CI, -0.76 to -0.07; P = .02), with fluoroquinolones showing the greatest decrease at -0.21 starts per 1000 resident-days (95% CI, -0.35 to -0.08; P = .002). The mean difference for antibiotic DOT per 1000 resident-days was not significant (-3.05; 95% CI, -6.34 to 0.23; P = .07). Reductions in antibiotic starts and use were greater in facilities with greater program engagement (as measured by webinar attendance). While antibiotic starts and DOT in these facilities decreased by 1.12 per 1000 resident-days (95% CI, -1.75 to -0.49; P < .001) and 9.97 per 1000 resident-days (95% CI, -15.4 to -4.6; P < .001), respectively, no significant reductions occurred in low engagement facilities. Urine cultures per 1000 resident-days decreased by 0.38 (95% CI, -0.61 to -0.15; P = .001). There was no significant change in facility-onset C difficile laboratory-identified events. CONCLUSIONS AND RELEVANCE: Participation in the AHRQ safety program was associated with the development of ASPs that actively engaged clinical staff in the decision-making processes around antibiotic prescriptions in participating LTC facilities. The reduction in antibiotic DOT and starts, which was more pronounced in more engaged facilities, indicates that implementation of this multifaceted program may support successful ASPs in LTC settings.