Call me Dr Ishmael: trends in electronic health record notes available at emergency department visits and admissions.

Objectives: Numerous studies have identified information overload as a key issue for electronic health records (EHRs). This study describes the amount of text data across all notes available to emergency physicians in the EHR, trended over the time since EHR establishment. Materials and Methods: We conducted a retrospective analysis of EHR data from a large healthcare system, examining the number of notes and a corresponding number of total words and total tokens across all notes available to physicians during patient encounters in the emergency department (ED). We assessed the change in these metrics over a 17-year period between 2006 and 2023. Results: The study cohort included 730 968 ED visits made by 293 559 unique patients and a total note count of 132 574 964. The median note count for all encounters in 2006 was 5 (IQR 1-16), accounting for 1735 (IQR 447-5521) words. By the last full year of the study period, 2022, the median number of notes had grown to 359 (IQR 84-943), representing 359 (IQR 84-943) words. Note and word counts were higher for admitted patients. Discussion: The volume of notes available for review by providers has increased by over 30-fold in the 17 years since the implementation of the EHR at a large health system. The task of reviewing these notes has become commensurately more difficult. These data point to the critical need for new strategies and tools for filtering, synthesizing, and summarizing information to achieve the promise of the medical record.

Influence of time-to-diagnosis on time-to-percutaneous coronary intervention for emergency department ST-elevation myocardial infarction patients: Time-to-electrocardiogram matters.

Objectives: Earlier electrocardiogram (ECG) acquisition for ST-elevation myocardial infarction (STEMI) is associated with earlier percutaneous coronary intervention (PCI) and better patient outcomes. However, the exact relationship between timely ECG and timely PCI is unclear. Methods: We quantified the influence of door-to-ECG (D2E) time on ECG-to-PCI balloon (E2B) intervention in this three-year retrospective cohort study, including patients from 10 geographically diverse emergency departments (EDs) co-located with a PCI center. The study included 576 STEMI patients excluding those with a screening ECG before ED arrival or non-diagnostic initial ED ECG. We used a linear mixed-effects model to evaluate D2E's influence on E2B with piecewise linear terms for D2E times associated with time intervals designated as ED intake (0-10 min), triage (11-30 min), and main ED (>30 min). We adjusted for demographic and visit characteristics, past medical history, and included ED location as a random effect. Results: The median E2B interval was longer (76 vs 68 min, p < 0.001) in patients with D2E >10 min than in those with timely D2E. The proportion of patients identified at the intake, triage, and main ED intervals was 65.8%, 24.9%, and 9.7%, respectively. The D2E and E2B association was statistically significant in the triage phase, where a 1-minute change in D2E was associated with a 1.24-minute change in E2B (95% confidence interval [CI]: 0.44-2.05, p = 0.003). Conclusion: Reducing D2E is associated with a shorter E2B. Targeting D2E reduction in patients currently diagnosed during triage (11-30 min) may be the greatest opportunity to improve D2B and could enable 24.9% more ED STEMI patients to achieve timely D2E.

Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department-Based Clinical Decision Support Tool to Prevent Future Falls.

BACKGROUND: Clinical decision support (CDS) tools that incorporate machine learning-derived content have the potential to transform clinical care by augmenting clinicians' expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing-personal visits to clinicians by an expert in a specific health IT tool-as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool's implementation. OBJECTIVE: This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department-based CDS tool to prevent future falls and identifying factors impacting clinicians' use of the tool through an analysis of the resultant qualitative data. METHODS: Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians' use of the CDS tool. RESULTS: The following categories of factors that impacted clinicians' use of the CDS were identified: (1) aspects of the CDS tool's design (2) clinicians' understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians' perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to. CONCLUSIONS: Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians' use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool's implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians.

Multicenter Development and Prospective Validation of eCARTv5: A Gradient Boosted Machine Learning Early Warning Score.

Rationale: Early detection of clinical deterioration using early warning scores may improve outcomes. However, most implemented scores were developed using logistic regression, only underwent retrospective internal validation, and were not tested in important patient subgroups. Objectives: To develop a gradient boosted machine model (eCARTv5) for identifying clinical deterioration and then validate externally, test prospectively, and evaluate across patient subgroups. Methods: All adult patients hospitalized on the wards in seven hospitals from 2008- 2022 were used to develop eCARTv5, with demographics, vital signs, clinician documentation, and laboratory values utilized to predict intensive care unit transfer or death in the next 24 hours. The model was externally validated retrospectively in 21 hospitals from 2009-2023 and prospectively in 10 hospitals from February to May 2023. eCARTv5 was compared to the Modified Early Warning Score (MEWS) and the National Early Warning Score (NEWS) using the area under the receiver operating characteristic curve (AUROC). Measurements and Main Results: The development cohort included 901,491 admissions, the retrospective validation cohort included 1,769,461 admissions, and the prospective validation cohort included 46,330 admissions. In retrospective validation, eCART had the highest AUROC (0.835; 95%CI 0.834, 0.835), followed by NEWS (0.766 (95%CI 0.766, 0.767)), and MEWS (0.704 (95%CI 0.703, 0.704)). eCART's performance remained high (AUROC ≥0.80) across a range of patient demographics, clinical conditions, and during prospective validation. Conclusions: We developed eCARTv5, which accurately identifies early clinical deterioration in hospitalized ward patients. Our model performed better than the NEWS and MEWS retrospectively, prospectively, and across a range of subgroups.

Automating risk stratification for geriatric syndromes in the emergency department.

BACKGROUND: Geriatric emergency department (GED) guidelines endorse screening older patients for geriatric syndromes in the ED, but there have been significant barriers to widespread implementation. The majority of screening programs require engagement of a clinician, nurse, or social worker, adding to already significant workloads at a time of record-breaking ED patient volumes, staff shortages, and hospital boarding crises. Automated, electronic health record (EHR)-embedded risk stratification approaches may be an alternate solution for extending the reach of the GED mission by directing human actions to a smaller subset of higher risk patients. METHODS: We define the concept of automated risk stratification and screening using existing EHR data. We discuss progress made in three potential use cases in the ED: falls, cognitive impairment, and end-of-life and palliative care, emphasizing the importance of linking automated screening with systems of healthcare delivery. RESULTS: Research progress and operational deployment vary by use case, ranging from deployed solutions in falls screening to algorithmic validation in cognitive impairment and end-of-life care. CONCLUSIONS: Automated risk stratification offers a potential solution to one of the most pressing problems in geriatric emergency care: identifying high-risk populations of older adults most appropriate for specific GED care. Future work is needed to realize the promise of improved care with less provider burden by creating tools suitable for widespread deployment as well as best practices for their implementation and governance.

Dashboarding to Monitor Machine-Learning-Based Clinical Decision Support Interventions.

BACKGROUND: Existing monitoring of machine-learning-based clinical decision support (ML-CDS) is focused predominantly on the ML outputs and accuracy thereof. Improving patient care requires not only accurate algorithms but also systems of care that enable the output of these algorithms to drive specific actions by care teams, necessitating expanding their monitoring. OBJECTIVES: In this case report, we describe the creation of a dashboard that allows the intervention development team and operational stakeholders to govern and identify potential issues that may require corrective action by bridging the monitoring gap between model outputs and patient outcomes. METHODS: We used an iterative development process to build a dashboard to monitor the performance of our intervention in the broader context of the care system. RESULTS: Our investigation of best practices elsewhere, iterative design, and expert consultation led us to anchor our dashboard on alluvial charts and control charts. Both the development process and the dashboard itself illuminated areas to improve the broader intervention. CONCLUSION: We propose that monitoring ML-CDS algorithms with regular dashboards that allow both a context-level view of the system and a drilled down view of specific components is a critical part of implementing these algorithms to ensure that these tools function appropriately within the broader care system.

Effectiveness of an Emergency Department-Based Machine Learning Clinical Decision Support Tool to Prevent Outpatient Falls Among Older Adults: Protocol for a Quasi-Experimental Study.

BACKGROUND: Emergency department (ED) providers are important collaborators in preventing falls for older adults because they are often the first health care providers to see a patient after a fall and because at-home falls are often preceded by previous ED visits. Previous work has shown that ED referrals to falls interventions can reduce the risk of an at-home fall by 38%. Screening patients at risk for a fall can be time-consuming and difficult to implement in the ED setting. Machine learning (ML) and clinical decision support (CDS) offer the potential of automating the screening process. However, it remains unclear whether automation of screening and referrals can reduce the risk of future falls among older patients. OBJECTIVE: The goal of this paper is to describe a research protocol for evaluating the effectiveness of an automated screening and referral intervention. These findings will inform ongoing discussions about the use of ML and artificial intelligence to augment medical decision-making. METHODS: To assess the effectiveness of our program for patients receiving the falls risk intervention, our primary analysis will be to obtain referral completion rates at 3 different EDs. We will use a quasi-experimental design known as a sharp regression discontinuity with regard to intent-to-treat, since the intervention is administered to patients whose risk score falls above a threshold. A conditional logistic regression model will be built to describe 6-month fall risk at each site as a function of the intervention, patient demographics, and risk score. The odds ratio of a return visit for a fall and the 95% CI will be estimated by comparing those identified as high risk by the ML-based CDS (ML-CDS) and those who were not but had a similar risk profile. RESULTS: The ML-CDS tool under study has been implemented at 2 of the 3 EDs in our study. As of April 2023, a total of 1326 patient encounters have been flagged for providers, and 339 unique patients have been referred to the mobility and falls clinic. To date, 15% (45/339) of patients have scheduled an appointment with the clinic. CONCLUSIONS: This study seeks to quantify the impact of an ML-CDS intervention on patient behavior and outcomes. Our end-to-end data set allows for a more meaningful analysis of patient outcomes than other studies focused on interim outcomes, and our multisite implementation plan will demonstrate applicability to a broad population and the possibility to adapt the intervention to other EDs and achieve similar results. Our statistical methodology, regression discontinuity design, allows for causal inference from observational data and a staggered implementation strategy allows for the identification of secular trends that could affect causal associations and allow mitigation as necessary. TRIAL REGISTRATION: ClinicalTrials.gov NCT05810064; https://www.clinicaltrials.gov/study/NCT05810064. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/48128.

Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults.

BACKGROUND: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. OBJECTIVE: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. METHODS: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. RESULTS: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. CONCLUSIONS: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS. TRIAL REGISTRATION: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.

Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation.

BACKGROUND: Heuristic evaluations, while commonly used, may inadequately capture the severity of identified usability issues. In the domain of health care, usability issues can pose different levels of risk to patients. Incorporating diverse expertise (eg, clinical and patient) in the heuristic evaluation process can help assess and address potential negative impacts on patient safety that may otherwise go unnoticed. One document that should be highly usable for patients-with the potential to prevent adverse outcomes-is the after visit summary (AVS). The AVS is the document given to a patient upon discharge from the emergency department (ED), which contains instructions on how to manage symptoms, medications, and follow-up care. OBJECTIVE: This study aims to assess a multistage method for integrating diverse expertise (ie, clinical, an older adult care partner, and health IT) with human factors engineering (HFE) expertise in the usability evaluation of the patient-facing ED AVS. METHODS: We conducted a three-staged heuristic evaluation of an ED AVS using heuristics developed for use in evaluating patient-facing documentation. In stage 1, HFE experts reviewed the AVS to identify usability issues. In stage 2, 6 experts of varying expertise (ie, emergency medicine physicians, ED nurses, geriatricians, transitional care nurses, and an older adult care partner) rated each previously identified usability issue on its potential impact on patient comprehension and patient safety. Finally, in stage 3, an IT expert reviewed each usability issue to identify the likelihood of successfully addressing the issue. RESULTS: In stage 1, we identified 60 usability issues that violated a total of 108 heuristics. In stage 2, 18 additional usability issues that violated 27 heuristics were identified by the study experts. Impact ratings ranged from all experts rating the issue as "no impact" to 5 out of 6 experts rating the issue as having a "large negative impact." On average, the older adult care partner representative rated usability issues as being more significant more of the time. In stage 3, 31 usability issues were rated by an IT professional as "impossible to address," 21 as "maybe," and 24 as "can be addressed." CONCLUSIONS: Integrating diverse expertise when evaluating usability is important when patient safety is at stake. The non-HFE experts, included in stage 2 of our evaluation, identified 23% (18/78) of all the usability issues and, depending on their expertise, rated those issues as having differing impacts on patient comprehension and safety. Our findings suggest that, to conduct a comprehensive heuristic evaluation, expertise from all the contexts in which the AVS is used must be considered. Combining those findings with ratings from an IT expert, usability issues can be strategically addressed through redesign. Thus, a 3-staged heuristic evaluation method offers a framework for integrating context-specific expertise efficiently, while providing practical insights to guide human-centered design.

Satisfaction of Older Patients With Emergency Department Care: Psychometric Properties and Construct Validity of the Consumer Emergency Care Satisfaction Scale.

BACKGROUND: Patient satisfaction is an important indicator of quality of care, but its measurement remains challenging. The Consumer Emergency Care Satisfaction Scale (CECSS) was developed to measure patient satisfaction in the emergency department (ED). Although this is a valid and reliable tool, several aspects of the CECSS need to be improved, including the definition, dimension, and scoring of scales. PURPOSE: The purpose of this study was to examine the construct validity of the CECSS and make suggestions on how to improve the tool to measure overall satisfaction with ED care. METHODS: We administered 2 surveys to older adults who presented with a fall to the ED and used electronic health record data to examine construct validity of the CECSS and ceiling effects. RESULTS: Using several criteria, we improved construct validity of the CECSS, reduced ceiling effects, and standardized scoring. CONCLUSION: We addressed several methodological issues with the CECSS and provided recommendations for improvement.

Multisite evaluation of prediction models for emergency department crowding before and during the COVID-19 pandemic.

OBJECTIVE: To develop a machine learning framework to forecast emergency department (ED) crowding and to evaluate model performance under spatial and temporal data drift. MATERIALS AND METHODS: We obtained 4 datasets, identified by the location: 1-large academic hospital and 2-rural hospital, and time period: pre-coronavirus disease (COVID) (January 1, 2019-February 1, 2020) and COVID-era (May 15, 2020-February 1, 2021). Our primary target was a binary outcome that is equal to 1 if the number of patients with acute respiratory illness that were ED boarding for more than 4 h was above a prescribed historical percentile. We trained a random forest and used the area under the curve (AUC) to evaluate out-of-sample performance for 2 experiments: (1) we evaluated the impact of sudden temporal drift by training models using pre-COVID data and testing them during the COVID-era, (2) we evaluated the impact of spatial drift by testing models trained at location 1 on data from location 2, and vice versa. RESULTS: The baseline AUC values for ED boarding ranged from 0.54 (pre-COVID at location 2) to 0.81 (COVID-era at location 1). Models trained with pre-COVID data performed similarly to COVID-era models (0.82 vs 0.78 at location 1). Models that were transferred from location 2 to location 1 performed worse than models trained at location 1 (0.51 vs 0.78). DISCUSSION AND CONCLUSION: Our results demonstrate that ED boarding is a predictable metric for ED crowding, models were not significantly impacted by temporal data drift, and any attempts at implementation must consider spatial data drift.

Retrospective analysis of the human-centered design process used to develop a clinical decision support in the emergency department: PE Dx Study Part 2.

With the growing implementation and use of health IT such as Clinical Decision Support (CDS), there is increasing attention on the potential negative impact of these technologies on patients (e.g., medication errors) and clinicians (e.g., increased workload, decreased job satisfaction, burnout). Human-Centered Design (HCD) and Human Factors (HF) principles are recommended to improve the usability of health IT and reduce its negative impact on patients and clinicians; however, challenges persist. The objective of this study is to understand how an HCD process influences the usability of health IT. We conducted a systematic retrospective analysis of the HCD process used in the design of a CDS for pulmonary embolism diagnosis in the emergency department (ED). Guided by the usability outcomes (e.g., barriers and facilitators) of the CDS use "in the wild" (see Part 1 of this research in the accompanying manuscript), we performed deductive content analysis of 17 documents (e.g., design session transcripts) produced during the HCD process. We describe if and how the design team considered the barriers and facilitators during the HCD process. We identified 7 design outcomes of the HCD process, for instance designing a workaround and making a design change to the CDS. We identify gaps in the current HCD process and demonstrate the need for a continuous health IT design process.

Post-implementation usability evaluation of a human factors-based clinical decision support for pulmonary embolism (PE) diagnosis (Dx): PE Dx Study Part 1.

While there is promise for health IT, such as Clinical Decision Support (CDS), to improve patient safety and clinician efficiency, poor usability has hindered widespread use of these tools. Human Factors (HF) principles and methods remain the gold standard for health IT design; however, there is limited information on how HF methods and principles influence CDS usability "in the wild". In this study, we explore the usability of an HF-based CDS used in the clinical environment; the CDS was designed according to a human-centered design process, which is described in Carayon et al. (2020). In this study, we interviewed 12 emergency medicine physicians, identifying 294 excerpts of barriers and facilitators of the CDS. Sixty-eight percent of excerpts related to the HF principles applied in the human-centered design of the CDS. The remaining 32% of excerpts related to 18 inductively-created categories, which highlight gaps in the CDS design process. Several barriers were related to the physical environment and organization work system elements as well as physicians' broader workflow in the emergency department (e.g., teamwork). This study expands our understanding of the usability outcomes of HF-based CDS "in the wild". We demonstrate the value of HF principles in the usability of CDS and identify areas for improvement to future human-centered design of CDS. The relationship between these usability outcomes and the HCD process is explored in an accompanying Part 2 manuscript.

Does What Happens in the ED Stay in the ED? The Effects of Emergency Department Physician Workload on Post-ED Care Use.

Problem definition: Emergency department (ED) crowding has been a pressing concern in healthcare systems in the U.S. and other developed countries. As such, many researchers have studied its effects on outcomes within the ED. In contrast, we study the effects of ED crowding on system performance outside the ED-specifically, on post-ED care utilization. Further, we explore the mediating effects of care intensity in the ED on post-ED care use. Methodology/results: We utilize a dataset assembled from more than four years of microdata from a large U.S. hospital and exhaustive billing data in an integrated health system. By using count models and instrumental variable analyses to answer the proposed research questions, we find that there is an increasing concave relationship between ED physician workload and post-ED care use. When ED workload increases from its 5th percentile to the median, the number of post-discharge care events (i.e., medical services) for patients who are discharged home from the ED increases by 5% and it is stable afterwards. Further, we identify physician test-ordering behavior as a mechanism for this effect: when the physician is busier, she responds by ordering more tests for less severe patients. We document that this "extra" testing generates "extra" post-ED care utilization for these patients. Managerial implications: This paper contributes new insights on how physician and patient behaviors under ED crowding impact a previously unstudied system performance measure: post-ED care utilization. Our findings suggest that prior studies estimating the cost of ED crowding underestimate the true effect, as they do not consider the "extra" post-ED care utilization.

Operationalizing a real-time scoring model to predict fall risk among older adults in the emergency department.

Predictive models are increasingly being developed and implemented to improve patient care across a variety of clinical scenarios. While a body of literature exists on the development of models using existing data, less focus has been placed on practical operationalization of these models for deployment in real-time production environments. This case-study describes challenges and barriers identified and overcome in such an operationalization for a model aimed at predicting risk of outpatient falls after Emergency Department (ED) visits among older adults. Based on our experience, we provide general principles for translating an EHR-based predictive model from research and reporting environments into real-time operation.

Governance of Clinical AI applications to facilitate safe and equitable deployment in a large health system: Key elements and early successes.

One of the key challenges in successful deployment and meaningful adoption of AI in healthcare is health system-level governance of AI applications. Such governance is critical not only for patient safety and accountability by a health system, but to foster clinician trust to improve adoption and facilitate meaningful health outcomes. In this case study, we describe the development of such a governance structure at University of Wisconsin Health (UWH) that provides oversight of AI applications from assessment of validity and user acceptability through safe deployment with continuous monitoring for effectiveness. Our structure leverages a multi-disciplinary steering committee along with project specific sub-committees. Members of the committee formulate a multi-stakeholder perspective spanning informatics, data science, clinical operations, ethics, and equity. Our structure includes guiding principles that provide tangible parameters for endorsement of both initial deployment and ongoing usage of AI applications. The committee is tasked with ensuring principles of interpretability, accuracy, and fairness across all applications. To operationalize these principles, we provide a value stream to apply the principles of AI governance at different stages of clinical implementation. This structure has enabled effective clinical adoption of AI applications. Effective governance has provided several outcomes: (1) a clear and institutional structure for oversight and endorsement; (2) a path towards successful deployment that encompasses technologic, clinical, and operational, considerations; (3) a process for ongoing monitoring to ensure the solution remains acceptable as clinical practice and disease prevalence evolve; (4) incorporation of guidelines for the ethical and equitable use of AI applications.

Implementation and safety evaluation of autoverification for select low-risk, high-volume medications in the emergency department.

PURPOSE: Use of autoverification has decreased in many emergency departments (EDs) with the expansion of emergency medicine (EM) pharmacists. Few studies have evaluated ways to prioritize verification of medications. Here we describe a process to design, implement, and measure the safety of autoverification of low-risk, high-volume medications. SUMMARY: A 3-month retrospective review of medications ordered and administered in the ED generated a list of medications to be considered for autoverification. Concurrently, a novel risk stratification tool was created to identify low-risk medications. Taking these together, medications that were high volume and low risk were considered potentially autoverified medications (PAMs). To evaluate the safety of PAMs, a retrospective review of the ED medication orders placed before implementation of autoverification was performed. A total of 7,433 medication orders were reviewed. Of these, 3,057 orders (41%) were identified as PAMs. EM pharmacists verified 2,982 (97.5%) of the orders without changes. Of the remaining 93 orders that were modified or discontinued and met autoverification criteria, only 2 (0.07%) were identified as potentially inappropriate for autoverification. CONCLUSION: Low-risk, high-volume medications can be safely autoverified in the ED by using a systematic approach to order selection. Using these methods can provide large decreases in verification volume, close to 41%, without compromising patient safety.

Older Adult Patients and Care Partners as Knowledge Brokers in Fragmented Health Care.

OBJECTIVE: To describe older adult patients' and care partners' knowledge broker roles during emergency department (ED) visits. BACKGROUND: Older adult patients are vulnerable to communication and coordination challenges during an ED visit, which can be exacerbated by the time and resource constrained ED environment. Yet, as a constant throughout the patient journey, patients and care partners can act as an information conduit, or knowledge broker, between fragmented care systems to attain high-quality, safe care. METHODS: Participants included 14 older adult patients (≥ 65 years old) and their care partners (e.g., spouse, adult child) who presented to the ED after having experienced a fall. Human factors researchers collected observation data from patients, care partners and clinician interactions during the patient's ED visit. We used an inductive content analysis to determine the role of patients and care partners as knowledge brokers. RESULTS: We found that patients and care partners act as knowledge brokers by providing information about diagnostic testing, medications, the patient's health history, and care accommodations at the disposition location. Patients and care partners filled the role of knowledge broker proactively (i.e. offer information) and reactively (i.e. are asked to provide information by clinicians or staff), within-ED work system and across work systems (e.g., between the ED and hospital), and in anticipation of future knowledge brokering. CONCLUSION: Patients and care partners, acting as knowledge brokers, often fill gaps in communication and participate in care coordination that assists in mitigating health care fragmentation.

Fallacy of Median Door-to-ECG Time: Hidden Opportunities for STEMI Screening Improvement.

Background ST-segment elevation myocardial infarction (STEMI) guidelines recommend screening arriving emergency department (ED) patients for an early ECG in those with symptoms concerning for myocardial ischemia. Process measures target median door-to-ECG (D2E) time of 10 minutes. Methods and Results This 3-year descriptive retrospective cohort study, including 676 ED-diagnosed patients with STEMI from 10 geographically diverse facilities across the United States, examines an alternative approach to quantifying performance: proportion of patients meeting the goal of D2E≤10 minutes. We also identified characteristics associated with D2E>10 minutes and estimated the proportion of patients with screening ECG occurring during intake, triage, and main ED care periods. We found overall median D2E was 7 minutes (IQR:4-16; range: 0-1407 minutes; range of ED medians: 5-11 minutes). Proportion of patients with D2E>10 minutes was 37.9% (ED range: 21.5%-57.1%). Patients with D2E>10 minutes, compared to those with D2E≤10 minutes, were more likely female (32.8% versus 22.6%, P=0.005), Black (23.4% versus 12.4%, P=0.005), non-English speaking (24.6% versus 19.5%, P=0.032), diabetic (40.2% versus 30.2%, P=0.010), and less frequently reported chest pain (63.3% versus 87.4%, P<0.001). ECGs were performed during ED intake in 62.1% of visits, ED triage in 25.3%, and main ED care in 12.6%. Conclusions Examining D2E>10 minutes can identify opportunities to improve care for more ED patients with STEMI. Our findings suggest sex, race, language, and diabetes are associated with STEMI diagnostic delays. Moving the acquisition of ECGs completed during triage to intake could achieve the D2E≤10 minutes goal for 87.4% of ED patients with STEMI. Sophisticated screening, accounting for differential risk and diversity in STEMI presentations, may further improve timely detection.

Comparing emergency department use among individuals with varying levels of cognitive impairment.

INTRODUCTION: As the population ages, Alzheimer's disease and related dementias (ADRD) are becoming increasingly common in patients presenting to the emergency department (ED). This study compares the frequency of ED use among a cohort of individuals with well-defined cognitive performance (cognitively intact, mild cognitive impairment (MCI), and ADRD). METHODS: We performed a retrospective cohort study of English-speaking, community-dwelling individuals evaluated at four health system-based multidisciplinary memory clinics from 2014-2016. We obtained demographic and clinical data, including neuropsychological testing results, through chart review and linkage to electronic health record data. We characterized the frequency and quantity of ED use within one year (6 months before and after) of cognitive evaluation and compared ED use between the three groups using bivariate and multivariate approaches. RESULTS: Of the 779 eligible patients, 89 were diagnosed as cognitively intact, 372 as MCI, and 318 as ADRD. The proportion of subjects with any annual ED use did not increase significantly with greater cognitive impairment: cognitively intact (16.9%), MCI (26.1%), and ADRD (28.9%) (p = 0.072). Average number of ED visits increased similarly: cognitively intact (0.27, SD 0.72), MCI (0.41, SD 0.91), and ADRD (0.55, SD 1.25) (p = 0.059). Multivariate logistic regression results showed that patients with MCI (odds ratio (OR) 1.62; CI = 0.87-3.00) and ADRD (OR 1.84; CI = 0.98-3.46) did not significantly differ from cognitively intact adults in any ED use. Multivariate negative binomial regression found patients with MCI (incidence rate ratio (IRR) 1.38; CI = 0.79-2.41) and ADRD (IRR 1.76, CI = 1.00-3.10) had elevated but non-significant risk of an ED visit compared to cognitively intact individuals. CONCLUSION: Though there was no significant difference in ED use in this small sample from one health system, our estimates are comparable to other published work. Results suggested a trend towards higher utilization among adults with MCI or ADRD compared to those who were cognitively intact. We must confirm our findings in other settings to better understand how to optimize systems of acute illness care for individuals with MCI and ADRD.