Maximizing the Ability of Health IT and AI to Improve Patient Safety.

This Viewpoint discusses how health information technology (IT) and artificial intelligence (AI) can be used to transform patient safety.

Automating detection of diagnostic error of infectious diseases using machine learning.

Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.

Building the future of ID: A call to action for quality improvement research and measurement.

Quality is central to value-based care and measurement is essential for assessing performance and understanding improvement over time. Both value-based care and methods for quality measurement are evolving. Infectious Diseases has been less engaged than other specialties in quality measure development, and Infectious Diseases providers must seize the opportunity to engage with quality measure development and research. Antimicrobial stewardship programs are an ideal starting point for Infectious Diseases-related quality measure development; antimicrobial stewardship program interventions and best practices are Infectious Diseases-specific, measurable, and impactful, yet grossly undercompensated. Herein, we provide a scheme for prioritizing research focused on development of Infectious Diseases-specific quality measures. Maturation of quality measurement research in Infectious Diseases, beginning with an initial focus on stewardship-related conditions then expanding to non-stewardship topics, will allow Infectious Diseases to take control of its future in value-based care, and promote the growth of Infectious Diseases through greater recognition of its value.

Patient Safety and Artificial Intelligence in Clinical Care.

This Viewpoint offers 3 recommendations for health care organizations and other stakeholders to consider as part of the Health and Human Services' artificial intelligence safety program.

Healthcare-associated infections and conditions in the era of digital measurement.

As the third edition of the Compendium of Strategies to Prevent Healthcare-Associated Infections in Acute Care Hospitals is released with the latest recommendations for the prevention and management of healthcare-associated infections (HAIs), a new approach to reporting HAIs is just beginning to unfold. This next generation of HAI reporting will be fully electronic and based largely on existing data in electronic health record (EHR) systems and other electronic data sources. It will be a significant change in how hospitals report HAIs and how the Centers for Disease Control and Prevention (CDC) and other agencies receive this information. This paper outlines what that future electronic reporting system will look like and how it will impact HAI reporting.

Improving Sepsis Outcomes in the Era of Pay-for-Performance and Electronic Quality Measures: A Joint IDSA/ACEP/PIDS/SHEA/SHM/SIDP Position Paper.

The Centers for Medicare & Medicaid Services (CMS) introduced the Severe Sepsis/Septic Shock Management Bundle (SEP-1) as a pay-for-reporting measure in 2015 and is now planning to make it a pay-for-performance measure by incorporating it into the Hospital Value-Based Purchasing Program. This joint IDSA/ACEP/PIDS/SHEA/SHM/SIPD position paper highlights concerns with this change. Multiple studies indicate that SEP-1 implementation was associated with increased broad-spectrum antibiotic use, lactate measurements, and aggressive fluid resuscitation for patients with suspected sepsis but not with decreased mortality rates. Increased focus on SEP-1 risks further diverting attention and resources from more effective measures and comprehensive sepsis care. We recommend retiring SEP-1 rather than using it in a payment model and shifting instead to new sepsis metrics that focus on patient outcomes. CMS is developing a community-onset sepsis 30-day mortality electronic clinical quality measure (eCQM) that is an important step in this direction. The eCQM preliminarily identifies sepsis using systemic inflammatory response syndrome (SIRS) criteria, antibiotic administrations or diagnosis codes for infection or sepsis, and clinical indicators of acute organ dysfunction. We support the eCQM but recommend removing SIRS criteria and diagnosis codes to streamline implementation, decrease variability between hospitals, maintain vigilance for patients with sepsis but without SIRS, and avoid promoting antibiotic use in uninfected patients with SIRS. We further advocate for CMS to harmonize the eCQM with the Centers for Disease Control and Prevention's (CDC) Adult Sepsis Event surveillance metric to promote unity in federal measures, decrease reporting burden for hospitals, and facilitate shared prevention initiatives. These steps will result in a more robust measure that will encourage hospitals to pay more attention to the full breadth of sepsis care, stimulate new innovations in diagnosis and treatment, and ultimately bring us closer to our shared goal of improving outcomes for patients.

How Safe are Outpatient Electronic Health Records? An Evaluation of Medication-Related Decision Support using the Ambulatory Electronic Health Record Evaluation Tool.

BACKGROUND: The purpose of the Ambulatory Electronic Health Record (EHR) Evaluation Tool is to provide outpatient clinics with an assessment that they can use to measure the ability of the EHR system to detect and prevent common prescriber errors. The tool consists of a medication safety test and a medication reconciliation module. OBJECTIVES: The goal of this study was to perform a broad evaluation of outpatient medication-related decision support using the Ambulatory EHR Evaluation Tool. METHODS: We performed a cross-sectional study with 10 outpatient clinics using the Ambulatory EHR Evaluation Tool. For the medication safety test, clinics were provided test patients and associated medication test orders to enter in their EHR, where they recorded any advice or information they received. Once finished, clinics received an overall percentage score of unsafe orders detected and individual order category scores. For the medication reconciliation module, clinics were asked to electronically reconcile two medication lists, where modifications were made by adding and removing medications and changing the dosage of select medications. RESULTS: For the medication safety test, the mean overall score was 57%, with the highest score being 70%, and the lowest score being 40%. Clinics performed well in the drug allergy (100%), drug dose daily (85%), and inappropriate medication combinations (74%) order categories. Order categories with the lowest performance were drug laboratory (10%) and drug monitoring (3%). Most clinics (90%) scored a 0% in at least one order category. For the medication reconciliation module, only one clinic (10%) could reconcile medication lists electronically; however, there was no clinical decision support available that checked for drug interactions. CONCLUSION: We evaluated a sample of ambulatory practices around their medication-related decision support and found that advanced capabilities within these systems have yet to be widely implemented. The tool was practical to use and identified substantial opportunities for improvement in outpatient medication safety.

Inpatient EHR User Experience and Hospital EHR Safety Performance.

IMPORTANCE: Despite the broad adoption and optimization of electronic health record (EHR) systems across the continuum of care, serious usability and safety problems persist. OBJECTIVE: To assess whether EHR safety performance is associated with EHR frontline user experience in a national sample of hospitals. DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional study included all US adult hospitals that used the National Quality Forum Leapfrog Health IT Safety Measure and also used the ARCH Collaborative EHR User experience survey from January 1, 2017, to January 1, 2019. Data analysis was performed from September 2020 to November 2022. MAIN OUTCOMES AND MEASURES: The primary outcomes were hospital performance on the Leapfrog Health IT Safety measure (overall and 10 subcomponents) and the ARCH collaborative frontline user experience scores (overall and 8 subcomponents). Ordinary least squares models with survey responses clustered by hospital were used to assess associations between the overall measures and their subcomponents. RESULTS: There were 112 hospitals and 5689 frontline user surveys included in the study. Hospitals scored a mean of 0.673 (range, 0.297-0.973) on the Leapfrog Health IT safety measure; the mean ARCH EHR user experience score was 3.377 (range, 1 [best] to 5 [worst]). The adjusted ß coefficient between the overall safety score and overall user experience score was 0.011 (95% CI, 0.006-0.016). The ARCH overall score was also significantly associated with 10 subcategory scores of the Leapfrog Health IT safety score, and the overall Leapfrog score was associated with the 8 subcategory scores of the ARCH user experience score. CONCLUSIONS AND RELEVANCE: This cross-sectional study found a positive association between frontline user-rated EHR usability and EHR safety performance. This finding suggests that improving EHR usability, which is a current well-known pain point for EHR users, could have direct benefits in terms of improved EHR safety.

Risk Factors for Severe COVID-19 Among Health Care Workers.

OBJECTIVE: Evaluate potential risk factors for severe coronavirus disease 2019 (COVID-19) among health care workers (HCWs) at the University of Virginia Medical Center in Charlottesville, Virginia. METHODS: We conducted a retrospective manual chart review of data from HCWs who were diagnosed with COVID-19 from March 2020 to March 2021. Using data from patient medical histories, we ascertained risk factors for COVID-19-related emergency department encounter, hospitalization, or death. RESULTS: We had 634 patients in total, and 9.8% had a severe COVID-19-related outcome. A history of deep vein thrombosis/pulmonary embolism/stroke (odds ratio, 19.6; 95% confidence interval, 5.11 to 94.7), as well as asthma, chronic lung disease, diabetes, or current immunocompromised status, was associated with increased adjusted odds of COVID-19-related emergency department encounter/hospitalization/death. CONCLUSIONS: A preexisting history of deep vein thrombosis/pulmonary embolism/stroke is a novel risk factor for poor COVID-19 outcomes among a cohort of HCWs.

Examining medication ordering errors using AHRQ network of patient safety databases.

BACKGROUND: Studies examining the effects of computerized order entry (CPOE) on medication ordering errors demonstrate that CPOE does not consistently prevent these errors as intended. We used the Agency for Healthcare Research and Quality (AHRQ) Network of Patient Safety Databases (NPSD) to investigate the frequency and degree of harm of reported events that occurred at the ordering stage, characterized by error type. MATERIALS AND METHODS: This was a retrospective observational study of safety events reported by healthcare systems in participating patient safety organizations from 6/2010 through 12/2020. All medication and other substance ordering errors reported to NPSD via common format v1.2 between 6/2010 through 12/2020 were analyzed. We aggregated and categorized the frequency of reported medication ordering errors by error type, degree of harm, and demographic characteristics. RESULTS: A total of 12 830 errors were reported during the study period. Incorrect dose accounted for 3812 errors (29.7%), followed by incorrect medication 2086 (16.3%), and incorrect duration 765 (6.0%). Of 5282 events that reached the patient and had a known level of severity, 12 resulted in death, 4 resulted in severe harm, 45 resulted in moderate harm, 341 resulted in mild harm, and 4880 resulted in no harm. CONCLUSION: Incorrect dose and incorrect drug orders were the most commonly reported and harmful types of medication ordering errors. Future studies should aim to develop and test interventions focused on CPOE to prevent medication ordering errors, prioritizing wrong-dose and wrong-drug errors.

Electronic Health Record Legal Settlements in the US Since the 2009 Health Information Technology for Economic and Clinical Health Act.

This cross-sectional study describes allegations of misconduct by electronic health record vendors.

The Development and Piloting of the Ambulatory Electronic Health Record Evaluation Tool: Lessons Learned.

BACKGROUND: Substantial research has been performed about the impact of computerized physician order entry on medication safety in the inpatient setting; however, relatively little has been done in ambulatory care, where most medications are prescribed. OBJECTIVE: To outline the development and piloting process of the Ambulatory Electronic Health Record (EHR) Evaluation Tool and to report the quantitative and qualitative results from the pilot. METHODS: The Ambulatory EHR Evaluation Tool closely mirrors the inpatient version of the tool, which is administered by The Leapfrog Group. The tool was piloted with seven clinics in the United States, each using a different EHR. The tool consists of a medication safety test and a medication reconciliation module. For the medication test, clinics entered test patients and associated test orders into their EHR and recorded any decision support they received. An overall percentage score of unsafe orders detected, and order category scores were provided to clinics. For the medication reconciliation module, clinics demonstrated how their EHR electronically detected discrepancies between two medication lists. RESULTS: For the medication safety test, the clinics correctly alerted on 54.6% of unsafe medication orders. Clinics scored highest in the drug allergy (100%) and drug-drug interaction (89.3%) categories. Lower scoring categories included drug age (39.3%) and therapeutic duplication (39.3%). None of the clinics alerted for the drug laboratory or drug monitoring orders. In the medication reconciliation module, three (42.8%) clinics had an EHR-based medication reconciliation function; however, only one of those clinics could demonstrate it during the pilot. CONCLUSION: Clinics struggled in areas of advanced decision support such as drug age, drug laboratory, and drub monitoring. Most clinics did not have an EHR-based medication reconciliation function and this process was dependent on accessing patients' medication lists. Wider use of this tool could improve outpatient medication safety and can inform vendors about areas of improvement.

Measuring Patient Safety: The Medicare Patient Safety Monitoring System (Past, Present, and Future).

ABSTRACT: The explicit declaration in the landmark 1999 Institute of Medicine report "To Err Is Human" that, in the United States, 44,000 to 98,000 patients die each year as a consequence of "medical errors" gave widespread validation to the magnitude of the patient safety problem and catalyzed a number of U.S. federal government programs to measure and improve the safety of the national healthcare system. After more than 10 years, one of those federal programs, the Medicare Patient Safety Monitoring System (MPSMS), has reached a level of maturity and stability that has made it useful for the consistent measurement of the safety of inpatient care. The MPSMS is a chart review-based national patient safety surveillance system that provides rates of 21 specific hospital inpatient adverse event measures, which have been divided into 4 clinical domains (general, hospital-acquired infections, postprocedure adverse events, and adverse drug events) for analysis. The 2014 MPSMS national sample was drawn from 1109 hospitals and includes approximately 20,000 medical records of patients admitted to the hospital (all payors) for at least 1 of the 4 conditions of congestive heart failure, acute myocardial infarction, pneumonia, and major surgical procedures as defined by the Centers for Medicare and Medicaid Services Surgical Care Improvement Project. The MPSMS is now going through a major transformation to capture additional types of adverse events and is being redeveloped as the Quality and Safety Review System (QSRS). As an example of this transformation, QSRS will electronically import electronic data, which are standardized according to the Centers for Medicare and Medicaid Services billing definitions and will be updated and evolve over time to incorporate expanded standardized data available from electronic health records. This article reviews the development of MPSMS, the strengths and limitations of MPSMS, and expected future directions in patient safety measurement, focusing on those issues that are informing the development and implementation of QSRS.

The tradeoffs between safety and alert fatigue: Data from a national evaluation of hospital medication-related clinical decision support.

OBJECTIVE: The study sought to evaluate the overall performance of hospitals that used the Computerized Physician Order Entry Evaluation Tool in both 2017 and 2018, along with their performance against fatal orders and nuisance orders. MATERIALS AND METHODS: We evaluated 1599 hospitals that took the test in both 2017 and 2018 by using their overall percentage scores on the test, along with the percentage of fatal orders appropriately alerted on, and the percentage of nuisance orders incorrectly alerted on. RESULTS: Hospitals showed overall improvement; the mean score in 2017 was 58.1%, and this increased to 66.2% in 2018. Fatal order performance improved slightly from 78.8% to 83.0% (P < .001), though there was almost no change in nuisance order performance (89.0% to 89.7%; P = .43). Hospitals alerting on one or more nuisance orders had a 3-percentage-point increase in their overall score. DISCUSSION: Despite the improvement of overall scores in 2017 and 2018, there was little improvement in fatal order performance, suggesting that hospitals are not targeting the deadliest orders first. Nuisance order performance showed almost no improvement, and some hospitals may be achieving higher scores by overalerting, suggesting that the thresholds for which alerts are fired from are too low. CONCLUSIONS: Although hospitals improved overall from 2017 to 2018, there is still important room for improvement for both fatal and nuisance orders. Hospitals that incorrectly alerted on one or more nuisance orders had slightly higher overall performance, suggesting that some hospitals may be achieving higher scores at the cost of overalerting, which has the potential to cause clinician burnout and even worsen safety.

National Trends in the Safety Performance of Electronic Health Record Systems From 2009 to 2018.

Importance: Despite the broad adoption of electronic health record (EHR) systems across the continuum of care, safety problems persist. Objective: To measure the safety performance of operational EHRs in hospitals across the country during a 10-year period. Design, Setting, and Participants: This case series included all US adult hospitals nationwide that used the National Quality Forum Health IT Safety Measure EHR computerized physician order entry safety test administered by the Leapfrog Group between 2009 and 2018. Data were analyzed from July 1, 2018 to December 1, 2019. Exposure: The Health IT Safety Measure test, which uses simulated medication orders that have either injured or killed patients previously to evaluate how well hospital EHRs could identify medication errors with potential for patient harm. Main Outcomes and Measures: Descriptive statistics for performance on the assessment test over time were calculated at the overall test score level, type of decision support category level, and EHR vendor level. Results: Among 8657 hospital-years observed during the study, mean (SD) scores on the overall test increased from 53.9% (18.3%) in 2009 to 65.6% (15.4%) in 2018. Mean (SD) hospital score for the categories representing basic clinical decision support increased from 69.8% (20.8%) in 2009 to 85.6% (14.9%) in 2018. For the categories representing advanced clinical decision support, the mean (SD) score increased from 29.6% (22.4%) in 2009 to 46.1% (21.6%) in 2018. There was considerable variation in test performance by EHR. Conclusions and Relevance: These findings suggest that despite broad adoption and optimization of EHR systems in hospitals, wide variation in the safety performance of operational EHR systems remains across a large sample of hospitals and EHR vendors. Hospitals using some EHR vendors had significantly higher test scores. Overall, substantial safety risk persists in current hospital EHR systems.

Using a Pediatric Trigger Tool to Estimate Total Harm Burden Hospital-acquired Conditions Represent.

INTRODUCTION: To improve patient safety, the Centers for Medicare & Medicaid Services (CMS) has promoted systematically measuring and reporting harm due to patient care. The CMS's Partnership for Patients program identified 9 hospital-acquired conditions (HACs) for reduction, to make care safer, more reliable, and less costly. However, the proportion of inpatient pediatric harm represented by these HACs is unknown. METHODS: We conducted a retrospective review of 240 harms previously identified using the Pediatric All-Cause Harm Measurement Tool, a trigger tool that is applied to medical records to comprehensively identify harms. The original sample included 600 randomly selected patients from 6 children's hospitals in February 2012. Patients with rehabilitation, obstetric, newborn nursery, and psychiatric admissions were excluded. The 240 identified harms were classified as a HAC if the event description potentially met the definition of 1 of the 9 CMS-defined HACs. HAC assessment was performed independently by 2 coauthors and compared using Cohen's Kappa. RESULTS: Two hundred forty harms across 6 children's hospitals were identified in February 2012 using a pediatric global trigger tool. Agreement between the coauthors on HAC classification was high (Kappa = 0.77). After reconciling differences, of the 240 identified harms, 58 (24.2%; 95% confidence interval: 9.1-31.7%) were classified as a CMS-defined HAC. CONCLUSIONS: One-fourth of all harms detected by a pediatric-specific trigger tool are represented by HACs. Although substantial effort is focused on identifying and minimizing HACs, to better understand and ultimately mitigate harm, more comprehensive harm identification and quantification may be needed to address events unidentified using this approach.

Adverse Events in Hospitalized Pediatric Patients.

: media-1vid110.1542/5789657761001PEDS-VA_2017-3360Video Abstract BACKGROUND: Patient safety concerns over the past 2 decades have prompted widespread efforts to reduce adverse events (AEs). It is unclear whether these efforts have resulted in reductions in hospital-wide AE rates. We used a validated safety surveillance tool, the Global Assessment of Pediatric Patient Safety, to measure temporal trends (2007-2012) in AE rates among hospitalized children. METHODS: We conducted a retrospective surveillance study of randomly selected pediatric inpatient records from 16 teaching and nonteaching hospitals. We constructed Poisson regression models with hospital random intercepts, controlling for patient age, sex, insurance, and chronic conditions, to estimate changes in AE rates over time. RESULTS: Examining 3790 records, reviewers identified 414 AEs (19.1 AEs per 1000 patient days; 95% confidence interval [CI] 17.2-20.9) and 210 preventable AEs (9.5 AEs per 1000 patient days; 95% CI 8.2-10.8). On average, teaching hospitals had higher AE rates than nonteaching hospitals (26.2 [95% CI 23.7-29.0] vs 5.1 [95% CI 3.7-7.1] AEs per 1000 patient days, P < .001). Chronically ill children had higher AE rates than patients without chronic conditions (33.9 [95% CI 24.5-47.0] vs 14.0 [95% CI 11.8-16.5] AEs per 1000 patient days, P < .001). Multivariate analyses revealed no significant changes in AE rates over time. When stratified by hospital type, neither teaching nor nonteaching hospitals experienced significant temporal AE rate variations. CONCLUSIONS: AE rates in pediatric inpatients are high and did not improve from 2007 to 2012. Pediatric AE rates were substantially higher in teaching hospitals as well as in patients with more chronic conditions.