Adapting Cognitive Task Analysis Methods for Use in a Large Sample Simulation Study of High-Risk Healthcare Events.

Cognitive task analysis (CTA) methods are traditionally used to conduct small-sample, in-depth studies. In this case study, CTA methods were adapted for a large multi-site study in which 102 anesthesiologists worked through four different high-fidelity simulated high-consequence incidents. Cognitive interviews were used to elicit decision processes following each simulated incident. In this paper, we highlight three practical challenges that arose: (1) standardizing the interview techniques for use across a large, distributed team of diverse backgrounds; (2) developing effective training; and (3) developing a strategy to analyze the resulting large amount of qualitative data. We reflect on how we addressed these challenges by increasing standardization, developing focused training, overcoming social norms that hindered interview effectiveness, and conducting a staged analysis. We share findings from a preliminary analysis that provides early validation of the strategy employed. Analysis of a subset of 64 interview transcripts using a decompositional analysis approach suggests that interviewers successfully elicited descriptions of decision processes that varied due to the different challenges presented by the four simulated incidents. A holistic analysis of the same 64 transcripts revealed individual differences in how anesthesiologists interpreted and managed the same case.

Decision-Making During High-Risk Events: A Systematic Literature Review.

Effective decision-making in crisis events is challenging due to time pressure, uncertainty, and dynamic decisional environments. We conducted a systematic literature review in PubMed and PsycINFO, identifying 32 empiric research papers that examine how trained professionals make naturalistic decisions under pressure. We used structured qualitative analysis methods to extract key themes. The studies explored different aspects of decision-making across multiple domains. The majority (19) focused on healthcare; military, fire and rescue, oil installation, and aviation domains were also represented. We found appreciable variability in research focus, methodology, and decision-making descriptions. We identified five main themes: (1) decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. Recognition-primed decision-making (RPD) strategies were reported in all studies that analyzed this aspect. Analytical strategies were also prominent, appearing more frequently in contexts with less time pressure and explicit training to generate multiple explanations. Practitioner experience, time pressure, stress, and uncertainty were major influencing factors. Professionals must adapt to the time available, types of uncertainty, and individual skills when making decisions in high-risk situations. Improved understanding of these decisional factors can inform evidence-based enhancements to training, technology, and process design.

Collaborative Use of Lung Mechanics Simulation for Testing and Iterative Design for Three Emergency Use Ventilation Device Projects.

SUMMARY STATEMENT: We describe our collaboration with engineering, clinical, and simulation colleagues to use a lung simulator (IngMar Medical ASL 5000) to aid in the development of 3 open-source ventilation devices for patients with COVID-19.Twenty-nine test conditions were created by programming software lung models of varying disease severity in the ASL 5000 to test basic functionality, safety features, and compliance with regulatory requirements for emergency use authorization for the 3 projects' prototypes. More than 200 simulations were performed, with the design team present to enable rapid troubleshooting and design iteration in real time.Working with 3 separate simultaneous ventilation device projects allowed us to rapidly learn from each, improving our ability to successfully collaborate with the different design/build teams.This project illustrates the role of simulation in facilitating collaborative innovation in health care, both in emergency and everyday settings that extend beyond the COVID-19 pandemic.

A Simple Ventilator Designed To Be Used in Shortage Crises: Construction and Verification Testing.

BACKGROUND: The COVID-19 pandemic has demonstrated the possibility of severe ventilator shortages in the near future. OBJECTIVE: We aimed to develop an acute shortage ventilator. METHODS: The ventilator was designed to mechanically compress a self-inflating bag resuscitator, using a modified ventilator patient circuit, which is controlled by a microcontroller and an optional laptop. It was designed to operate in both volume-controlled mode and pressure-controlled assist modes. We tested the ventilator in 4 modes using an artificial lung while measuring the volume, flow, and pressure delivered over time by the ventilator. RESULTS: The ventilator was successful in reaching the desired tidal volume and respiratory rates specified in national emergency use resuscitator system guidelines. The ventilator responded to simulated spontaneous breathing. CONCLUSIONS: The key design goals were achieved. We developed a simple device with high performance for short-term use, made primarily from common hospital parts and generally available nonmedical components to avoid any compatibility or safety issues with the patient, and at low cost, with a unit cost per ventilator is less than $400 US excluding the patient circuit parts, that can be easily manufactured.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific.

INTRODUCTION: Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established. METHODS: Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters. RESULTS: The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability. CONCLUSIONS: The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

Clinical Uses and Impacts of Emergency Manuals During Perioperative Crises.

BACKGROUND: Performing key actions efficiently during crises can determine clinical outcomes, yet even expert clinicians omit key actions. Simulation-based studies of crises show that correct performance of key actions dramatically increases when emergency manuals (EMs) are used. Despite widespread dissemination of EMs, there is a need to understand in clinical contexts, when, how, and how often EMs are used and not used, along with perceived impacts. METHODS: We conducted interviews with the anesthesia professionals involved in perioperative crises, identified with criterion-based sampling, occurring between October 2014 and May 2016 at 2 large academic medical centers with a history of EM training and implementation. Our convergent, mixed-methods study of the interview data extracted quantitative counts and qualitative themes of EM use and nonuse during clinical crises. RESULTS: Interviews with 53 anesthesia professionals yielded 80 descriptions of applicable clinical crises, with varying durations and event types. Of 69 unique patients whose cases involved crises, the EM was used during 37 (54%; 95% confidence interval [CI], 41-66). Impacts on clinician team members included decreased stress for individual anesthesia professionals (95%), enabled teamwork (73%), and calmed atmosphere (46%). Impacts on delivery of patient care included specific action improvements, including catching errors of omission, for example, turning off anesthetic during cardiac arrest, only after EM use (59%); process improvements, for example, double-checking all actions were completed (41%); and impediments (0%). In 8% of crises, EM use was associated with potential distractions, although none were perceived to harm delivery of patient care. For 32 EM nonuses (46%; 95% CI, 34-59), participants self-identified errors of omission or delays in key actions (56%), all key actions performed (13%), and crisis too brief for EM to be used (31%). CONCLUSIONS: This study provides evidence that EMs in operating rooms are being used during many applicable crises and that clinicians perceive EM use to add value. The reported negative effects were minimal and potentially offset by positive effects.

Guidelines for the Responsible Use of Deception in Simulation: Ethical and Educational Considerations.

STATEMENT: Many techniques and modifications commonly used by the simulation community have been identified as deceptive. Deception is an important issue addressed by both the newly adopted Healthcare Simulationist Code of Ethics and the American Psychological Association Code of Conduct. Some view these approaches as essential whereas others question their necessity as well as their untoward psychological effects. In an attempt to offer guidance to simulation-based healthcare educators, we explore educational practices commonly identified as deceptive along with their potential benefits and detriments. We then address important decision points and high-risk situations that should be avoided to uphold ethical boundaries and psychological safety among learners. These are subsequently analyzed in light of the Code of Ethics and used to formulate guidelines for educators that are intended to ensure that deception, when necessary, is implemented in as psychologically safe a manner as possible.

De-escalating Angry Caregivers: A Randomized Controlled Trial of a Novel Communication Curriculum for Pediatric Residents.

OBJECTIVE: Medical providers struggle when communicating with angry patients and their caregivers. Pediatric residents perceive communication competencies as an important priority for learning, yet they lack confidence and desire more training in communicating with angry families. Few curricula exist to support trainees with de-escalation skill development. We developed, implemented, and evaluated the impact of a novel de-escalation curriculum on pediatric resident communication skills. METHODS: We conducted a randomized controlled trial of a 90-minute de-escalation curriculum for pediatric residents from August to September 2016. Trained standardized patient (SP) actors rated residents' communication skills following 2 unique encounters before and after the intervention or control sessions. Residents completed a retrospective pre/post communication skills self-assessment and curriculum evaluation. We used independent and paired t-tests to assess for communication improvements. RESULTS: Eighty-four of 88 (95%) eligible residents participated (43 intervention, 41 control). Residents reported frequent encounters with angry caregivers. At baseline, interns had significantly lower mean SP-rated de-escalation skills than other residents (P = .03). Intervention residents did not improve significantly more than controls on their pre/post change in mean SP-rated de-escalation skills; however, intervention residents improved significantly on their pre/post mean self-assessed de-escalation skills (P ≤ .03). CONCLUSIONS: Despite significant self-assessed improvements, residents' SP-rated de-escalation skills did not improve following a skills-based intervention. Nevertheless, our study illustrates the need for de-escalation curricula focused on strategies and peer discussion, suggests optimal timing of delivery during fall of intern year, and offers an assessment tool for exploration in future studies.

Use of an Emergency Manual During an Intraoperative Cardiac Arrest by an Interprofessional Team: A Positive-Exemplar Case Study of a New Patient Safety Tool.

BACKGROUND: An emergency manual (EM) is a set of evidence-based crisis checklists, or cognitive aids, that can improve team performance. EMs are used in other safety-critical industries, and health care simulation studies have shown their efficacy, but use in clinical settings is nascent. A case study was conducted on the use of an EM during one intraoperative crisis, which entailed the assessment of the impact of the EM's use on teamwork and patient care and the identification of lessons for effectively using EMs during future clinical crises. METHODS: In a case study of a single crisis, an EM was used during a cardiac arrest at a tertiary care hospital that had systematically implemented perioperative EMs. Semistructured interviews were conducted with all six clinicians present, interview transcripts were iteratively coded, and thematic analysis was performed. RESULTS: All clinician participants stated that EM use enabled effective team functioning via reducing stress of individual clinicians, fostering a calm work environment, and improving teamwork and communication. These impacts in turn improved the delivery of patient care during a clinical crisis and influenced participants' intended EM use during future appropriate crises. CONCLUSION: In this positive-exemplar case study, an EM was used to improve delivery of evidence-based patient care through effective clinical team functioning. EM use must complement rather than replace good clinician education, judgment, and teamwork. More broadly, understanding why and how things go well via analyzing positive-exemplar case studies, as a converse of root cause analyses for negative events, can be used to identify effective applications of safety innovations.

Priorities Related to Improving Healthcare Safety Through Simulation.

STATEMENT: Improving healthcare safety is a worthwhile and important endeavor. Simulation-based activities can help with such a goal through research and training. In this manner, it can focus on education and training, assessment and metrics, process improvement, and culture change to help move forward both patient safety and quality of care.This article will address the following three main topics: (1) designing simulation-based activities to promote high reliability in healthcare, (2) developing simulation-based activities to foster resilience in healthcare systems, and (3) evaluating the impact of adverse events in healthcare and how simulation-based activities can be used to determine and potentially to prevent their cause. These topics will be treated sequentially, providing synopses of concepts and giving examples of research currently being undertaken. It will then highlight current priorities for simulation-based research in this domain by drawing from insights obtained and a targeted literature review.

Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists.

BACKGROUND: We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods. METHODS: A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant's technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist. RESULTS: Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance. CONCLUSIONS: Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.

A Taxonomy of Delivery and Documentation Deviations During Delivery of High-Fidelity Simulations.

INTRODUCTION: We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations. METHODS: Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development. RESULTS: The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy. CONCLUSIONS: This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.