Digital Maturity as a Predictor of Quality and Safety Outcomes in US Hospitals: Cross-Sectional Observational Study.

BACKGROUND: This study demonstrates that digital maturity contributes to strengthened quality and safety performance outcomes in US hospitals. Advanced digital maturity is associated with more digitally enabled work environments with automated flow of data across information systems to enable clinicians and leaders to track quality and safety outcomes. This research illustrates that an advanced digitally enabled workforce is associated with strong safety leadership and culture and better patient health and safety outcomes. OBJECTIVE: This study aimed to examine the relationship between digital maturity and quality and safety outcomes in US hospitals. METHODS: The data sources were hospital safety letter grades as well as quality and safety scores on a continuous scale published by The Leapfrog Group. We used the digital maturity level (measured using the Electronic Medical Record Assessment Model [EMRAM]) of 1026 US hospitals. This was a cross-sectional, observational study. Logistic, linear, and Tweedie regression analyses were used to explore the relationships among The Leapfrog Group's Hospital Safety Grades, individual Leapfrog safety scores, and digital maturity levels classified as advanced or fully developed digital maturity (EMRAM levels 6 and 7) or underdeveloped maturity (EMRAM level 0). Digital maturity was a predictor while controlling for hospital characteristics including teaching status, urban or rural location, hospital size measured by number of beds, whether the hospital was a referral center, and type of hospital ownership as confounding variables. Hospitals were divided into the following 2 groups to compare safety and quality outcomes: hospitals that were digitally advanced and hospitals with underdeveloped digital maturity. Data from The Leapfrog Group's Hospital Safety Grades report published in spring 2019 were matched to the hospitals with completed EMRAM assessments in 2019. Hospital characteristics such as number of hospital beds were obtained from the CMS database. RESULTS: The results revealed that the odds of achieving a higher Leapfrog Group Hospital Safety Grade was statistically significantly higher, by 3.25 times, for hospitals with advanced digital maturity (EMRAM maturity of 6 or 7; odds ratio 3.25, 95% CI 2.33-4.55). CONCLUSIONS: Hospitals with advanced digital maturity had statistically significantly reduced infection rates, reduced adverse events, and improved surgical safety outcomes. The study findings suggest a significant difference in quality and safety outcomes among hospitals with advanced digital maturity compared with hospitals with underdeveloped digital maturity.

Hospital commitments to address diagnostic errors: An assessment of 95 US hospitals.

BACKGROUND: Diagnostic errors are a leading cause of patient harm. In 2022, the Leapfrog Group published a report containing 29 evidence-based practices that hospitals can adopt to reduce diagnostic errors. OBJECTIVES: To understand the extent to which US hospitals have already implemented these practices, we conducted a national pilot survey of Leapfrog-participating hospitals. METHODS: To reduce respondent burden, we divided the 29 practices across two surveys: one focused on organizational culture and structure (Domain 1), and the second focused on the diagnostic process itself (Domain 2). RESULTS: A total of 95 hospitals from 23 states responded to one or both surveys. On average, hospitals reported implementing 9 of the 16 practices (56%) in Domain 1 and 8 of the 13 practices (62%) in Domain 2. The rate of practice implementation varied greatly, with some hospitals implementing as few as three practices in their domain. The most commonly implemented practices were ensuring access to medical interpreters, continuous access to radiologists, ensuring staff and patients can report diagnostic errors and concerns, and having a formal process to identify and notify patients when diagnostic errors occur. The least implemented practices included convening a multidisciplinary team focused on diagnostic safety and quality, a CEO commitment to diagnostic excellence, conducting diagnosis-focused risk assessments, and training clinicians to optimize clinical reasoning in the diagnostic process. CONCLUSIONS: The findings suggest large and important implementation gaps for practices related to diagnostic excellence and can inform new initiatives to promote diagnostic excellence in US hospitals.

Development of the Leapfrog Group's Bar Code Medication Administration Standard to Address Hospital Inpatient Medication Safety.

ABSTRACT: Medication errors are the most common type of error in hospitals and reflect a leading cause of avoidable harm to patients. Bar code medication administration (BCMA) systems are a technology designed to help intercept medication errors at the point of medication administration. This article describes the process of developing, testing, and refining a standard for BCMA adoption and use in U.S. hospitals, as measured through the Leapfrog Hospital Survey. Building on the published literature and an expert panel's collective experience in studying, implementing, and using BCMA systems, the expert panel recommended a standard with 4 key domains. Leapfrog's BCMA standard provides hospitals with a "how-to guide" on what best practice looks like for using BCMA to ensure safe medication administration at the bedside.

The tradeoffs between safety and alert fatigue: Data from a national evaluation of hospital medication-related clinical decision support.

OBJECTIVE: The study sought to evaluate the overall performance of hospitals that used the Computerized Physician Order Entry Evaluation Tool in both 2017 and 2018, along with their performance against fatal orders and nuisance orders. MATERIALS AND METHODS: We evaluated 1599 hospitals that took the test in both 2017 and 2018 by using their overall percentage scores on the test, along with the percentage of fatal orders appropriately alerted on, and the percentage of nuisance orders incorrectly alerted on. RESULTS: Hospitals showed overall improvement; the mean score in 2017 was 58.1%, and this increased to 66.2% in 2018. Fatal order performance improved slightly from 78.8% to 83.0% (P < .001), though there was almost no change in nuisance order performance (89.0% to 89.7%; P = .43). Hospitals alerting on one or more nuisance orders had a 3-percentage-point increase in their overall score. DISCUSSION: Despite the improvement of overall scores in 2017 and 2018, there was little improvement in fatal order performance, suggesting that hospitals are not targeting the deadliest orders first. Nuisance order performance showed almost no improvement, and some hospitals may be achieving higher scores by overalerting, suggesting that the thresholds for which alerts are fired from are too low. CONCLUSIONS: Although hospitals improved overall from 2017 to 2018, there is still important room for improvement for both fatal and nuisance orders. Hospitals that incorrectly alerted on one or more nuisance orders had slightly higher overall performance, suggesting that some hospitals may be achieving higher scores at the cost of overalerting, which has the potential to cause clinician burnout and even worsen safety.

National Trends in the Safety Performance of Electronic Health Record Systems From 2009 to 2018.

Importance: Despite the broad adoption of electronic health record (EHR) systems across the continuum of care, safety problems persist. Objective: To measure the safety performance of operational EHRs in hospitals across the country during a 10-year period. Design, Setting, and Participants: This case series included all US adult hospitals nationwide that used the National Quality Forum Health IT Safety Measure EHR computerized physician order entry safety test administered by the Leapfrog Group between 2009 and 2018. Data were analyzed from July 1, 2018 to December 1, 2019. Exposure: The Health IT Safety Measure test, which uses simulated medication orders that have either injured or killed patients previously to evaluate how well hospital EHRs could identify medication errors with potential for patient harm. Main Outcomes and Measures: Descriptive statistics for performance on the assessment test over time were calculated at the overall test score level, type of decision support category level, and EHR vendor level. Results: Among 8657 hospital-years observed during the study, mean (SD) scores on the overall test increased from 53.9% (18.3%) in 2009 to 65.6% (15.4%) in 2018. Mean (SD) hospital score for the categories representing basic clinical decision support increased from 69.8% (20.8%) in 2009 to 85.6% (14.9%) in 2018. For the categories representing advanced clinical decision support, the mean (SD) score increased from 29.6% (22.4%) in 2009 to 46.1% (21.6%) in 2018. There was considerable variation in test performance by EHR. Conclusions and Relevance: These findings suggest that despite broad adoption and optimization of EHR systems in hospitals, wide variation in the safety performance of operational EHR systems remains across a large sample of hospitals and EHR vendors. Hospitals using some EHR vendors had significantly higher test scores. Overall, substantial safety risk persists in current hospital EHR systems.

Assessing the safety of electronic health records: a national longitudinal study of medication-related decision support.

BACKGROUND: Electronic health records (EHR) can improve safety via computerised physician order entry with clinical decision support, designed in part to alert providers and prevent potential adverse drug events at entry and before they reach the patient. However, early evidence suggested performance at preventing adverse drug events was mixed. METHODS: We used data from a national, longitudinal sample of 1527 hospitals in the USA from 2009 to 2016 who took a safety performance assessment test using simulated medication orders to test how well their EHR prevented medication errors with potential for patient harm. We calculated the descriptive statistics on performance on the assessment over time, by years of hospital experience with the test and across hospital characteristics. Finally, we used ordinary least squares regression to identify hospital characteristics associated with higher test performance. RESULTS: The average hospital EHR system correctly prevented only 54.0% of potential adverse drug events tested on the 44-order safety performance assessment in 2009; this rose to 61.6% in 2016. Hospitals that took the assessment multiple times performed better in subsequent years than those taking the test the first time, from 55.2% in the first year of test experience to 70.3% in the eighth, suggesting efforts to participate in voluntary self-assessment and improvement may be helpful in improving medication safety performance. CONCLUSION: Hospital medication order safety performance has improved over time but is far from perfect. The specifics of EHR medication safety implementation and improvement play a key role in realising the benefits of computerising prescribing, as organisations have substantial latitude in terms of what they implement. Intentional quality improvement efforts appear to be a critical part of high safety performance and may indicate the importance of a culture of safety.

National trends in safety performance of electronic health record systems in children's hospitals.

Objective: To evaluate the safety of computerized physician order entry (CPOE) and associated clinical decision support (CDS) systems in electronic health record (EHR) systems at pediatric inpatient facilities in the US using the Leapfrog Group's pediatric CPOE evaluation tool. Methods: The Leapfrog pediatric CPOE evaluation tool, a previously validated tool to assess the ability of a CPOE system to identify orders that could potentially lead to patient harm, was used to evaluate 41 pediatric hospitals over a 2-year period. Evaluation of the last available test for each institution was performed, assessing performance overall as well as by decision support category (eg, drug-drug, dosing limits). Longitudinal analysis of test performance was also carried out to assess the impact of testing and the overall trend of CPOE performance in pediatric hospitals. Results: Pediatric CPOE systems were able to identify 62% of potential medication errors in the test scenarios, but ranged widely from 23-91% in the institutions tested. The highest scoring categories included drug-allergy interactions, dosing limits (both daily and cumulative), and inappropriate routes of administration. We found that hospitals with longer periods since their CPOE implementation did not have better scores upon initial testing, but after initial testing there was a consistent improvement in testing scores of 4 percentage points per year. Conclusions: Pediatric computerized physician order entry (CPOE) systems on average are able to intercept a majority of potential medication errors, but vary widely among implementations. Prospective and repeated testing using the Leapfrog Group's evaluation tool is associated with improved ability to intercept potential medication errors.

Are health care provider organizations ready to tackle diagnostic error? A survey of Leapfrog-participating hospitals.

BACKGROUND: A 2015 National Academy of Medicine report on improving diagnosis in health care made recommendations for direct action by hospitals and health systems. Little is known about how health care provider organizations are addressing diagnostic safety/quality. METHODS: This study is an anonymous online survey of safety professionals from US hospitals and health systems in July-August 2016. The survey was sent to those attending a Leapfrog Group webinar on misdiagnosis (n=188). The instrument was focused on knowledge, attitudes, and capability to address diagnostic errors at the institutional level. RESULTS: Overall, 61 (32%) responded, including community hospitals (42%), integrated health networks (25%), and academic centers (21%). Awareness was high, but commitment and capability were low (31% of leaders understand the problem; 28% have sufficient safety resources; and 25% have made diagnosis a top institutional safety priority). Ongoing efforts to improve diagnostic safety were sparse and mostly included root cause analysis and peer review feedback around diagnostic errors. The top three barriers to addressing diagnostic error were lack of awareness of the problem, lack of measures of diagnostic accuracy and error, and lack of feedback on diagnostic performance. The top two tools viewed as critically important for locally tackling the problem were routine feedback on diagnostic performance and culture change to emphasize diagnostic safety. CONCLUSIONS: Although hospitals and health systems appear to be aware of diagnostic errors as a major safety imperative, most organizations (even those that appear to be making a strong commitment to patient safety) are not yet doing much to improve diagnosis. Going forward, efforts to activate health care organizations will be essential to improving diagnostic safety.