Medication Safety Amid Technological Change: Usability Evaluation to Inform Inpatient Nurses' Electronic Health Record System Transition.

BACKGROUND: Electronic health record (EHR) system transitions are challenging for healthcare organizations. High-volume, safety-critical tasks like barcode medication administration (BCMA) should be evaluated, yet standards for ensuring safety during transition have not been established. OBJECTIVE: Identify risks in common and problem-prone medication tasks to inform safe transition between BCMA systems and establish benchmarks for future system changes. DESIGN: Staff nurses completed simulation-based usability testing in the legacy system (R1) and new system pre- (R2) and post-go-live (R3). Tasks included (1) Hold/Administer, (2) IV Fluids, (3) PRN Pain, (4) Insulin, (5) Downtime/PRN, and (6) Messaging. Audiovisual recordings of task performance were systematically analyzed for time, navigation, and errors. The System Usability Scale measured perceived usability and satisfaction. Post-simulation interviews captured nurses' qualitative comments and perceptions of the systems. PARTICIPANTS: Fifteen staff nurses completed 2-3-h simulation sessions. Eleven completed both R1 and R2, and seven completed all three rounds. Clinical experience ranged from novice (< 1 year) to experienced (> 10 years). Practice settings included adult and pediatric patient populations in ICU, stepdown, and acute care departments. MAIN MEASURES: Task completion rates/times, safety and non-safety-related use errors (interaction difficulties), and user satisfaction. KEY RESULTS: Overall success rates remained relatively stable in all tasks except two: IV Fluids task success increased substantially (R1: 17%, R2: 54%, R3: 100%) and Downtime/PRN task success decreased (R1: 92%, R2: 64%, R3: 22%). Among the seven nurses who completed all rounds, overall safety-related errors decreased 53% from R1 to R3 and 50% from R2 to R3, and average task times for successfully completed tasks decreased 22% from R1 to R3 and 38% from R2 to R3. CONCLUSIONS: Usability testing is a reasonable approach to compare different BCMA tasks to anticipate transition problems and establish benchmarks with which to monitor and evaluate system changes going forward.

Decision-making in anesthesiology: will artificial intelligence make intraoperative care safer?

PURPOSE OF REVIEW: This article explores the impact of recent applications of artificial intelligence on clinical anesthesiologists' decision-making. RECENT FINDINGS: Naturalistic decision-making, a rich research field that aims to understand how cognitive work is accomplished in complex environments, provides insight into anesthesiologists' decision processes. Due to the complexity of clinical work and limits of human decision-making (e.g. fatigue, distraction, and cognitive biases), attention on the role of artificial intelligence to support anesthesiologists' decision-making has grown. Artificial intelligence, a computer's ability to perform human-like cognitive functions, is increasingly used in anesthesiology. Examples include aiding in the prediction of intraoperative hypotension and postoperative complications, as well as enhancing structure localization for regional and neuraxial anesthesia through artificial intelligence integration with ultrasound. SUMMARY: To fully realize the benefits of artificial intelligence in anesthesiology, several important considerations must be addressed, including its usability and workflow integration, appropriate level of trust placed on artificial intelligence, its impact on decision-making, the potential de-skilling of practitioners, and issues of accountability. Further research is needed to enhance anesthesiologists' clinical decision-making in collaboration with artificial intelligence.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific.

INTRODUCTION: Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established. METHODS: Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters. RESULTS: The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability. CONCLUSIONS: The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

Incidence and Classification of Nonroutine Events during Anesthesia Care.

BACKGROUND: A nonroutine event is any aspect of clinical care perceived by clinicians or trained observers as a deviation from optimal care based on the context of the clinical situation. The authors sought to delineate the incidence and nature of intraoperative nonroutine events during anesthesia care. METHODS: The authors prospectively collected audio, video, and relevant clinical information on 556 cases at three academic hospitals from 1998 to 2004. In addition to direct observation, anesthesia providers were surveyed for nonroutine event occurrence and details at the end of each study case. For the 511 cases with reviewable video, 400 cases had no reported nonroutine events and 111 cases had at least one nonroutine event reported. Each nonroutine event was analyzed by trained anesthesiologists. Rater reliability assessment, comparisons (nonroutine event vs. no event) of patient and case variables were performed. RESULTS: Of 511 cases, 111 (21.7%) contained 173 nonroutine events; 35.1% of event-containing cases had more than one nonroutine event. Of the 173 events, 69.4% were rated as having patient impact and 12.7% involved patient injury. Longer case duration (25th vs. 75th percentile; odds ratio, 1.83; 95% CI, 1.15 to 2.93; P = 0.032) and presence of a comorbid diagnosis (odds ratio, 2.14; 95% CI, 1.35 to 3.40; P = 0.001) were associated with nonroutine events. Common contributory factors were related to the patient (63.6% [110 of 173]) and anesthesia provider (59.0% [102 of 173]) categories. The most common patient impact events involved the cardiovascular system (37.4% [64 of 171]), airway (33.3% [57 of 171]), and human factors, drugs, or equipment (31.0% [53 of 171]). CONCLUSIONS: This study describes characteristics of intraoperative nonroutine events in a cohort of cases at three academic hospitals. Nonroutine event-containing cases were commonly associated with patient impact and injury. Thus, nonroutine event monitoring in conjunction with traditional error reporting may enhance our understanding of potential intraoperative failure modes to guide prospective safety interventions.

CYP2D6 Genotype-guided Metoprolol Therapy in Cardiac Surgery Patients: Rationale and Design of the Pharmacogenetic-guided Metoprolol Management for Postoperative Atrial Fibrillation in Cardiac Surgery (PREEMPTIVE) Pilot Study.

OBJECTIVES: The Preemptive Pharmacogenetic-guided Metoprolol Management for Atrial Fibrillation in Cardiac Surgery (PREEMPTIVE) pilot trial aims to use existing institutional resources to develop a process for integrating CYP2D6 pharmacogenetic test results into the patient electronic health record, to develop an evidence-based clinical decision support tool to facilitate CYP2D6 genotype-guided metoprolol administration in the cardiac surgery setting, and to determine the impact of implementing this CYP2D6 genotype-guided integrated approach on the incidence of postoperative atrial fibrillation (AF), provider, and cost outcomes. DESIGN: One-arm Bayesian adaptive design clinical trial. SETTING: Single center, university hospital. PARTICIPANTS: The authors will screen (including CYP2D6 genotype) up to 600 (264 ± 144 expected under the adaptive design) cardiac surgery patients, and enroll up to 200 (88 ± 48 expected) poor, intermediate, and ultrarapid CYP2D6 metabolizers over a period of 2 years at a tertiary academic center. INTERVENTIONS: All consented and enrolled patients will receive the intervention of CYP2D6 genotype-guided metoprolol management based on CYP2D6 phenotype classified as a poor, intermediate, extensive (normal), or ultrarapid metabolizer. MEASUREMENTS AND MAIN RESULTS: The primary outcome will be the incidence of postoperative AF. Secondary outcomes relating to rates of CYP2D6 genotype-guided prescription changes, costs, lengths of stay, and implementation metrics also will be investigated. CONCLUSIONS: The PREEMPTIVE pilot study is the first perioperative pilot trial to provide essential information for the design of a future, large-scale trial comparing CYP2D6 genotype-guided metoprolol management with a nontailored strategy in terms of managing AF. In addition, secondary outcomes regarding implementation, clinical benefit, safety, and cost-effectiveness in patients undergoing cardiac surgery will be examined.

Burnout and Resiliency in Perianesthesia Nurses: Findings and Recommendations From a National Study of Members of the American Society of PeriAnesthesia Nurses.

PURPOSE: Describe prevalence of burnout in perianesthesia nurses, explore risks, mitigating factors. DESIGN: Cross-sectional descriptive. METHODS: Survey containing Maslach Burnout Inventory, Short Form-12, and Social Support and Personal Coping was emailed to American Society of PeriAnesthesia Nurses. Regression analysis examined relationships between burnout and health, social support, personal coping, substance use, and demographics. FINDINGS: Of 2,837 respondents, 18% were currently and 35% were formerly burned out, with lower incidence in those >40 years. Currently burned out nurses had worse health and also perceived a lack of advancement opportunities and organizational investment in the individual. Lower burnout was associated with regular participation in physical (P < .001), creative (P = .004), or mindfulness hobbies (P < .001) and ease in discussing work problems with spouse or partner (P = .001). CONCLUSIONS: Despite burnout nurses' empathy for their patients is maintained. Interests outside of work, personal and work support, healthy work environment, and regular physical activities can improve burnout.

Prevalence of Potentially Distracting Noncare Activities and Their Effects on Vigilance, Workload, and Nonroutine Events during Anesthesia Care.

BACKGROUND: When workload is low, anesthesia providers may perform non-patient care activities of a clinical, educational, or personal nature. Data are limited on the incidence or impact of distractions on actual care. We examined the prevalence of self-initiated nonclinical distractions and their effects on anesthesia workload, vigilance, and the occurrence of nonroutine events. METHODS: In 319 qualifying cases in an academic medical center using a Web-based electronic medical chart, a trained observer recorded video and performed behavioral task analysis. Participant workload and response to a vigilance (alarm) light were randomly measured. Postoperatively, participants were interviewed to elicit possible nonroutine events. Two anesthesiologists reviewed each event to evaluate their association with distractions. RESULTS: At least one self-initiated distraction was observed in 171 cases (54%), largely during maintenance. Distractions accounted for 2% of case time and lasted 2.3 s (median). The most common distraction was personal internet use. Distractions were more common in longer cases but were not affected by case type or American Society of Anesthesiologists physical status. Workload ratings were significantly lower during distraction-containing case periods and vigilance latencies were significantly longer in cases without any distractions. Three distractions were temporally associated with, but did not cause, events. CONCLUSIONS: Both nurse anesthetists and residents performed potentially distracting tasks of a personal and/or educational nature in a majority of cases. Self-initiated distractions were rarely associated with events. This study suggests that anesthesia professionals using sound judgment can self-manage nonclinical activities. Future efforts should focus on eliminating more cognitively absorbing and less escapable distractions, as well as training in distraction management.

Acoustic features of auditory medical alarms-An experimental study of alarm volume.

Audible alarms are a ubiquitous feature of all high-paced, high-risk domains such as aviation and nuclear power where operators control complex systems. In such settings, a missed alarm can have disastrous consequences. It is conventional wisdom that for alarms to be heard, "louder is better," so that alarm levels in operational environments routinely exceed ambient noise levels. Through a robust experimental paradigm in an anechoic environment to study human response to audible alerting stimuli in a cognitively demanding setting, akin to high-tempo and high-risk domains, clinician participants responded to patient crises while concurrently completing an auditory speech intelligibility and visual vigilance distracting task as the level of alarms were varied as a signal-to-noise ratio above and below hospital background noise. There was little difference in performance on the primary task when the alarm sound was -11 dB below background noise as compared with +4 dB above background noise-a typical real-world situation. Concurrent presentation of the secondary auditory speech intelligibility task significantly degraded performance. Operator performance can be maintained with alarms that are softer than background noise. These findings have widespread implications for the design and implementation of alarms across all high-consequence settings.

Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists.

BACKGROUND: We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods. METHODS: A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant's technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist. RESULTS: Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance. CONCLUSIONS: Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.

A Survey Evaluating Burnout, Health Status, Depression, Reported Alcohol and Substance Use, and Social Support of Anesthesiologists.

BACKGROUND: Burnout affects all medical specialists, and concern about it has become common in today's health care environment. The gold standard of burnout measurement in health care professionals is the Maslach Burnout Inventory-Human Services Survey (MBI-HSS), which measures emotional exhaustion, depersonalization (DP), and personal accomplishment. Besides affecting work quality, burnout is thought to affect health problems, mental health issues, and substance use negatively, although confirmatory data are lacking. This study evaluates some of these effects. METHODS: In 2011, the American Society of Anesthesiologists and the journal Anesthesiology cosponsored a webinar on burnout. As part of the webinar experience, we included access to a survey using MBI-HSS, 12-item Short Form Health Survey (SF-12), Social Support and Personal Coping (SSPC-14) survey, and substance use questions. Results were summarized using sample statistics, including mean, standard deviation, count, proportion, and 95% confidence intervals. Adjusted linear regression methods examined associations between burnout and substance use, SF-12, SSPC-14, and respondent demographics. RESULTS: Two hundred twenty-one respondents began the survey, and 170 (76.9%) completed all questions. There were 266 registrants total (31 registrants for the live webinar and 235 for the archive event), yielding an 83% response rate. Among respondents providing job titles, 206 (98.6%) were physicians and 2 (0.96%) were registered nurses. The frequency of high-risk responses ranged from 26% to 59% across the 3 MBI-HSS categories, but only about 15% had unfavorable scores in all 3. Mean mental composite score of the SF-12 was 1 standard deviation below normative values and was significantly associated with all MBI-HSS components. With SSPC-14, respondents scored better in work satisfaction and professional support than in personal support and workload. Males scored worse on DP and personal accomplishment and, relative to attending physicians, residents scored worse on DP. There was no significant association between MBI-HSS and substance use. CONCLUSIONS: Many anesthesiologists exhibit some high-risk burnout characteristics, and these are associated with lower mental health scores. Personal and professional support were associated with less emotional exhaustion, but overall burnout scores were associated with work satisfaction and professional support. Respondents were generally economically satisfied but also felt less in control at work and that their job kept them from friends and family. The association between burnout and substance use may not be as strong as previously believed. Additional work, perhaps with other survey instruments, is needed to confirm our results.

Facilitated Nurse Medication-Related Event Reporting to Improve Medication Management Quality and Safety in Intensive Care Units.

BACKGROUND: Medication safety presents an ongoing challenge for nurses working in complex, fast-paced, intensive care unit (ICU) environments. Studying ICU nurse's medication management-especially medication-related events (MREs)-provides an approach to analyze and improve medication safety and quality. OBJECTIVES: The goal of this study was to explore the utility of facilitated MRE reporting in identifying system deficiencies and the relationship between MREs and nurses' work in the ICUs. METHODS: We conducted 124 structured 4-hour observations of nurses in three different ICUs. Each observation included measurement of nurse's moment-to-moment activity and self-reports of workload and negative mood. The observer then obtained MRE reports from the nurse using a structured tool. The MREs were analyzed by three experts. RESULTS: MREs were reported in 35% of observations. The 60 total MREs included four medication errors and seven adverse drug events. Of the 49 remaining MREs, 65% were associated with negative patient impact. Task/process deficiencies were the most common contributory factor for MREs. MRE occurrence was correlated with increased total task volume. MREs also correlated with increased workload, especially during night shifts. DISCUSSION: Most of these MREs would not be captured by traditional event reporting systems. Facilitated MRE reporting provides a robust information source about potential breakdowns in medication management safety and opportunities for system improvement.

A simulation-based curriculum to introduce key teamwork principles to entering medical students.

BACKGROUND: Failures of teamwork and interpersonal communication have been cited as a major patient safety issue. Although healthcare is increasingly being provided in interdisciplinary teams, medical school curricula have traditionally not explicitly included the specific knowledge, skills, attitudes, and behaviors required to function effectively as part of such teams. METHODS: As part of a new "Foundations" core course for beginning medical students that provided a two-week introduction to the most important themes in modern healthcare, a multidisciplinary team, in collaboration with the Center for Experiential Learning and Assessment, was asked to create an experiential introduction to teamwork and interpersonal communication. We designed and implemented a novel, all-day course to teach second-week medical students basic teamwork and interpersonal principles and skills using immersive simulation methods. Students' anonymous comprehensive course evaluations were collected at the end of the day. Through four years of iterative refinement based on students' course evaluations, faculty reflection, and debriefing, the course changed and matured. RESULTS: Four hundred twenty evaluations were collected. Course evaluations were positive with almost all questions having means and medians greater than 5 out of 7 across all 4 years. Sequential year comparisons were of greatest interest for examining the effects of year-to-year curricular improvements. Differences were not detected among any of the course evaluation questions between 2007 and 2008 except that more students in 2008 felt that the course further developed their "Decision Making Abilities" (OR 1.69, 95% CI 1.07-2.67). With extensive changes to the syllabus and debriefer selection/assignment, concomitant improvements were observed in these aspects between 2008 and 2009 (OR = 2.11, 95% CI: 1.28-3.50). Substantive improvements in specific exercises also yielded significant improvements in the evaluations of those exercises. CONCLUSIONS: This curriculum could be valuable to other medical schools seeking to inculcate teamwork foundations in their medical school's preclinical curricula. Moreover, this curriculum can be used to facilitate teamwork principles important to inter-disciplinary, as well as uni-disciplinary, collaboration.

Paper or plastic? Simulation based evaluation of two versions of a cognitive aid for managing pediatric peri-operative critical events by anesthesia trainees: evaluation of the society for pediatric anesthesia emergency checklist.

Cognitive aids (CA), including emergency manuals and checklists, are tools designed to assist users in prioritizing and performing complex tasks during time sensitive, high stress situations (Marshall in Anesth Analgesia 117(5):1162-1171, 2013; Marshall and Mehra in Anaesthesia 69(7):669-677, 2014). The society for pediatric anesthesia (SPA) has developed a series of emergency checklists tailored for use by pediatric perioperative teams that cover a wide range of intraoperative critical events (Shaffner et al. in Anesth Analgesia 117(4):960-979, 2013). In this study, we evaluated user preferences for a CA (SPA checklist) using two different presentation formats, paper and electronic, during management of simulated critical events. Anesthesia trainees managed the simulated critical events under one of three randomized conditions: (1) memory alone, (2) with a paper version of the CA, (3) with an electronic version of the CA. Following participation in the simulated critical events, participants were asked to complete a survey regarding their experience using the different versions of the CA. The percentage of favorable responses for each format of the CA was compared using a mixed effects proportional odds model. There were 143 simulated events managed by 89 anesthesia trainees. Approximately one out of three trainees (electronic 29 %, paper 30 %) assigned to use the CA chose not to use it and completed the scenario from memory alone. The survey was completed by 68 % of participants, 58 % of trainees preferred the paper version and 35 % preferred the electronic version. All survey responses that reached statistical significance favored the paper version. In this study, anesthesia trainees had a favorable opinion of the content and perceived clinical relevance of both versions of the CA. In both quantitative and qualitative analysis, the paper version of the CA was preferred over the electronic version by participants. Despite overall favorable responses to the CA, a sizeable number of participants chose not to use either version the CA during the crisis.

A Multimodal Intervention Improves Postanesthesia Care Unit Handovers.

BACKGROUND: Failures of communication are a major contributor to perioperative adverse events. Transitions of care may be particularly vulnerable. We sought to improve postoperative handovers. METHODS: We introduced a multimodal intervention in an adult and a pediatric postanesthesia care unit (PACU) to improve postoperative handovers between anesthesia providers (APs) and PACU registered nurses (RNs). The intervention included a standardized electronic handover report form, a didactic webinar, mandatory simulation training focused on improving interprofessional communication, and post-training performance feedback. Trained, blinded nurse observers scored PACU handovers during 17 months using a structured tool consisting of 8 subscales and a global score (1-5 scale). Multivariate logistic regression assessed the effect of the intervention on the proportion of observed handovers receiving a global effectiveness rating of ≥3. RESULTS: Four hundred fifty-two clinicians received the simulation-based training, and 981 handovers were observed and rated. In the adult PACU, the estimated percentages of acceptable handovers (global ratings ≥3) among AP-RN pairs, where neither received simulation-based training (untrained dyads), was 3% (95% confidence interval, 1%-11%) at day 0, 10% (5%-19%) at training initiation (day 40), and 57% (33%-78%) at 1-year post-training initiation (day 405). For AP-RN pairs where at least one received the simulation-based training (trained dyads), these percentages were estimated to be 18% (11%-28%) and 68% (57%-76%) on days 40 and 405, respectively. The percentage of acceptable handovers was significantly greater on day 405 than it was on day 40 for both untrained (P < 0.001) and trained dyads (P < 0.001). Similar patterns were observed in the pediatric PACU. Three years later, the unadjusted estimate of the probability of an acceptable handover was 87% (72%-95%) in the adult PACU and 56% (40%-72%) in the pediatric PACU. CONCLUSIONS: A multimodal intervention substantially improved interprofessional PACU handovers, including those by clinicians who had not undergone formal simulation training. An effect appeared to be present >3 years later.

Implementing goal-directed protocols reduces length of stay after cardiac surgery.

OBJECTIVE: To test the effect of a high reliability organization (HRO) intervention on patient lengths of stay in the CVICU and hospital. The authors proposed that (1) higher safety related evidence based protocol (SREBP) team compliance scores and (2) lower SREBP milestone scores are associated with shorter lengths of CVICU and hospital stay. DESIGN: A prospective, longitudinal observational evaluation was used to assess the effects of SREBP-focused rounding processes and a milestone-tracking tool. SETTING: United States, university academic medical center's 27-bed CVICU. PARTICIPANTS: Six hundred sixty-five adult cardiac surgery patients and the CVICU care team (100 registered nurses and 16 clinical providers) participated. MEASUREMENTS AND MAIN RESULTS: Team compliance was the proportion of SREBP-related team behaviors exhibited during daily rounds. Patients' milestone scores were the cumulative difference between actual and expected times for 4 SREBP milestones over 48 hours. Milestones achieved earlier than expected indicated reduced complication risk, and milestones achieved later than expected indicated increased risk. As team compliance increased, CVICU length of stay decreased 0.66 (95% CI: -0.04 to 1.28; p = 0.08) days; hospital stay decreased 0.89 times (95% CI: 0.77-1.03; p = 0.008). As the mean milestone scores increased from -7 to 12, length of ICU stay increased 2.63 (95% CI: 1.66-3.59; p<0.001) days; hospital length of stay increased 1.44 times (95% CI: 1.23-1.7; p = 0.05). CONCLUSIONS: A milestone-driven pathway supported by team rounding was associated with decreased lengths of CVICU and hospital stay. However, tracking patient trajectories by milestones suggests a more complex relationship than anticipated and presents new opportunities for SREBP implementation and research.

Prognostic clinical decision support for pneumonia in the emergency department: A randomized trial.

BACKGROUND: Hospitalization rates for childhood pneumonia vary widely. Risk-based clinical decision support (CDS) interventions may reduce unwarranted variation. METHODS: We conducted a pragmatic randomized trial in two US pediatric emergency departments (EDs) comparing electronic health record (EHR)-integrated prognostic CDS versus usual care for promoting appropriate ED disposition in children (<18 years) with pneumonia. Encounters were randomized 1:1 to usual care versus custom CDS featuring a validated pneumonia severity score predicting risk for severe in-hospital outcomes. Clinicians retained full decision-making authority. The primary outcome was inappropriate ED disposition, defined as early transition to lower- or higher-level care. Safety and implementation outcomes were also evaluated. RESULTS: The study enrolled 536 encounters (269 usual care and 267 CDS). Baseline characteristics were similar across arms. Inappropriate disposition occurred in 3% of usual care encounters and 2% of CDS encounters (adjusted odds ratio: 0.99, 95% confidence interval: [0.32, 2.95]) Length of stay was also similar and adverse safety outcomes were uncommon in both arms. The tool's custom user interface and content were viewed as strengths by surveyed clinicians (>70% satisfied). Implementation barriers include intrinsic (e.g., reaching the right person at the right time) and extrinsic factors (i.e., global pandemic). CONCLUSIONS: EHR-based prognostic CDS did not improve ED disposition decisions for children with pneumonia. Although the intervention's content was favorably received, low subject accrual and workflow integration problems likely limited effectiveness. Clinical Trials Registration: NCT06033079.