Usability Testing Via Simulation: Optimizing the NEAR4PEM Preintubation Checklist With a Human Factors Approach.

OBJECTIVES: To inform development of a preintubation checklist for pediatric emergency departments via multicenter usability testing of a prototype checklist. METHODS: This was a prospective, mixed methods study across 7 sites in the National Emergency Airway Registry for Pediatric Emergency Medicine (NEAR4PEM) collaborative. Pediatric emergency medicine attending physicians and senior fellows at each site were first oriented to a checklist prototype, including content previously identified using a modified Delphi approach. Each site used the checklist in 2 simulated cases: an "easy airway" and a "difficult airway" scenario. Facilitators recorded verbalization, completion, and timing of checklist items. After each simulation, participants completed an anonymous usability survey. Structured debriefings were used to gather additional feedback on checklist usability. Comments from the surveys and debriefing were qualitatively analyzed using a framework approach. Responses informed human factors-based optimization of the checklist. RESULTS: Fifty-five pediatric emergency medicine physicians/fellows (4-13 per site) participated. Participants found the prototype checklist to be helpful, easy to use, clear, and of appropriate length. During the simulations, 93% of checklist items were verbalized and more than 80% were completed. Median time to checklist completion was 6.2 minutes (interquartile range, 4.8-7.1) for the first scenario and 4.2 minutes (interquartile range, 2.7-5.8) for the second. Survey and debriefing data identified the following strengths: facilitating a shared mental model, cognitively offloading the team leader, and prompting contingency planning. Suggestions for checklist improvement included clarifying specific items, providing more detailed prompts, and allowing institution-specific customization. Integration of these data with human factors heuristic inspection resulted in a final checklist. CONCLUSIONS: Simulation-based, human factors usability testing of the National Emergency Airway Registry for Pediatric Emergency Medicine Preintubation Checklist allowed optimization prior to clinical implementation. Next steps involve integration into real-world settings utilizing rigorous implementation science strategies, with concurrent evaluation of the impact on patient outcomes and safety.

Impact of the COVID-19 Pandemic on Low-Value Testing and Treatment of Bronchiolitis.

BACKGROUND: Viral testing and treatments such as systemic steroids and inhaled corticosteroids are low-value care for routine bronchiolitis. We sought to determine the impact of the COVID-19 pandemic on low-value care in young children with bronchiolitis. METHODS: This was a retrospective, cross-sectional study using the Pediatric Health Information Systems database. We included children <2 years seen in a pediatric emergency department for bronchiolitis. We selected a priori 3 study periods: September 2018 to February 2020 (prepandemic), March 2020 to August 2022 (early pandemic), and September 2022 to January 2023 (late pandemic). Low-value care included respiratory syncytial virus testing, chest radiography, albuterol, or corticosteroids and was compared across the 3 time periods. RESULTS: At least 1 element of low-value care was provided in 45%, 47%, and 44% of encounters in the prepandemic, early pandemic, and late pandemic periods, respectively. There was little variation in the use of albuterol and chest radiography across time periods and a slight increase in systemic corticosteroid use from prepandemic to early and late pandemic groups. Viral testing increased from 36% prepandemic to 65% early pandemic and 67% late pandemic, which appeared to be driven by SARS-CoV-2 testing and combination viral testing. CONCLUSIONS: There was no clinically significant change in low-value care for bronchiolitis during the pandemic. Because of SARS-CoV-2 testing, however, overall frequency of viral testing increased dramatically over time. This marked increase in overall viral testing should be taken into consideration for future quality improvement efforts.

Rapid cycle deliberate practice to improve airway skills and performance of trainees in a pediatric emergency department.

Objective: The study objective was to determine the effect of a rapid cycle deliberate practice (RCDP) program on simulated and actual airway skills by pediatric emergency medicine (PEM) fellows. Methods: We designed and implemented a 12-month RCDP airway skills curriculum for PEM fellows at an academic pediatric institution. The curriculum was designed using airway training literature, RCDP principals, and internal quality assurance airway video review program. Simulation training scenarios increased in complexity throughout the curriculum. PEM fellows participated in monthly sessions. Two PEM faculty facilitated the sessions, utilizing a step-by-step objective structured clinical evaluation (OSCE)-style tool for each scenario. Data were collected for all four levels of the Kirkpatrick Model of Training Evaluation-participant response (reaction, pre-post session survey), skills performance in the simulation setting (learning, pre-post OSCE), skills performance for actual patients (behavior, video review), and patient outcomes (results, video review). Results: During the study period (August 2021 to June 2022), 13 PEM fellows participated in 112 sessions (mean nine sessions per fellow). PEM fellows reported improved comfort in all domains of airway management, including intubation performance. Participant OSCE scores improved posttraining (pretraining median score for trainees 57 [IQR 57-59], posttraining median 61 [IQR 61-62], p = 0.0005). Over the 12 months, PEM fellows performed 45 intubation attempts in the pediatric emergency department (median patient age 4 years [IQR 1-9 years]). Compared to a 5-year historical cohort, participants had higher first-pass success (87% vs. 71%, p = 0.028) and shorter attempt duration (22 s vs. 29 s, p = 0.018). There was no significant difference in the frequency of oxyhemoglobin desaturation in the training period versus the historical period (7% vs. 15%, p = 0.231). Conclusions: At multiple levels of educational outcomes, including participant behavior and patient outcomes, an RCDP program was associated with improved airway skills and performance of PEM fellows.

Structured Callouts, the Shared Mental Model, and Teamwork: A Video-Based Study in a Pediatric Emergency Department.

OBJECTIVES: The shared mental model is essential to high-quality resuscitations. A structured callout (SCO) is often performed to establish the shared mental model, but the literature on SCOs is limited. The objectives of this study are to describe performance of SCOs during pediatric medical emergencies and to determine whether a SCO is associated with better teamwork. METHODS: This was a retrospective study in the resuscitation area of an academic pediatric emergency department, where performance of a SCO is a standard expectation. Only medical or nontrauma patients were eligible for inclusion. Data collection was performed by structured video review by 2 observers and verified by a third blinded observer. A SCO was defined as team leader (Pediatric Emergency Medicine fellow or faculty physician) verbalization of at least 1 element of the patient history/examination or an assessment of patient physiology and 1 element of the diagnostic or therapeutic plan. We independently measured teamwork using the Teamwork Emergency Assessment Measure (TEAM) tool. RESULTS: We reviewed 60 patient encounters from the pediatric emergency department resuscitation area between April 2018 and June 2020. Median patient age was 6 years; the team leader was a Pediatric Emergency Medicine fellow in 55% of encounters. The physician team leader performed a SCO in 38 (63%) of patient encounters. The TEAM scores were collected for 46 encounters. Mean TEAM score (SD) was 42.3 (1.7) in patients with a SCO compared with 40.0 (3.0) in those without a SCO ( P = 0.007). CONCLUSIONS: Performance of a SCO was associated with better teamwork, but the difference was of unclear clinical significance.

Coaching the coach: A randomized controlled study of a novel curriculum for procedural coaching during intubation.

Background: Videolaryngoscopy allows real-time procedural coaching during intubation. This study sought to develop and assess an online curriculum to train pediatric emergency medicine attending physicians to deliver procedural coaching during intubation. Methods: Curriculum development consisted of semistructured interviews with 12 pediatric emergency medicine attendings with varying levels of airway expertise analyzed using a constructivist grounded theory approach. Following development, the curriculum was implemented and assessed through a multicenter randomized controlled trial enrolling participants in one of three cohorts: the coaching module, unnarrated video recordings of intubations, and a module on ventilator management. Participants completed identical pre and post assessments asking them to select the correct coaching feedback and provided reactions for qualitative thematic analysis. Results: Content from interviews was synthesized into a video-enhanced 15-min online coaching module illustrating proper technique for intubation and strategies for procedural coaching. Eighty-seven of 104 randomized physicians enrolled in the curriculum; 83 completed the pre and post assessments (80%). The total percentage correct did not differ between pre and post assessments for any cohort. Participants receiving the coaching module demonstrated improved performance on patient preparation, made more suggestions for improvement, and experienced a greater increase in confidence in procedural coaching. Qualitative analysis identified multiple benefits of the module, revealed that exposure to video recordings without narration is insufficient, and identified feedback on suggestions for improvement as an opportunity for deliberate practice. Conclusions: This study leveraged clinical and educational digital technology to develop a curriculum dedicated to the content expertise and coaching skills needed to provide feedback during intubations performed with videolaryngoscopy. This brief curriculum changed behavior in simulated coaching scenarios but would benefit from additional support for deliberate practice.

The Importance of Median Glossoepiglottic Fold Engagement on Laryngeal View and Tracheal Intubation Success in Children.

STUDY OBJECTIVE: Our study objective was to determine if the location of laryngoscope blade tip placement is associated with clinically important tracheal intubation outcomes in a pediatric emergency department. METHODS: We conducted a video-based observational study of pediatric emergency department patients undergoing tracheal intubation with standard geometry Macintosh and Miller video laryngoscope blades (Storz C-MAC, Karl Storz). Our main exposures were direct lifting of the epiglottis versus blade tip placement within the vallecula and median glossoepiglottic fold engagement versus not when the blade tip was placed in the vallecula. Our main outcomes were glottic visualization and procedural success. We compared measures of glottic visualization between successful and unsuccessful attempts using generalized linear mixed models. RESULTS: Proceduralists placed the blade tip in the vallecula (indirectly lifting the epiglottis) during 123 (71.9%) of 171 attempts. When compared with indirectly lifting the epiglottis, directly lifting the epiglottis was associated with improved visualization-by percentage of glottic opening (POGO) (adjusted odds ratio [AOR], 11.0; 95% confidence interval [CI], 5.1 to 23.6) and modified Cormack-Lehane (AOR, 21.5; 95% CI, 6.6 to 69.9). When in the vallecula, engagement of the median glossoepiglottic fold was associated with improved POGO (AOR, 3.6; 95% CI, 1.9 to 6.8), modified Cormack-Lehane (AOR, 3.9; 95% CI, 1.1 to 14.1), and success (AOR, 9.9; 95% CI, 2.3 to 43.7). CONCLUSIONS: Emergency tracheal intubation can be performed in children at a high level by directly or indirectly lifting the epiglottis. If indirectly lifting the epiglottis, median glossoepiglottic fold engagement is helpful in maximizing glottic visualization and procedural success.

The Concise Assessment of Leader Management Tool: Evaluation of Healthcare Provider Leadership During Real-Life Pediatric Emergencies.

INTRODUCTION: Resuscitation events in pediatric critical and emergency care are high risk, and strong leadership is an important component of an effective response. The Concise Assessment of Leadership Management (CALM) tool, designed to assess the strength of leadership skills during pediatric crises, has shown promising validity and reliability in simulated settings. The objective of this study was to generate further validity and reliability evidence for the CALM by applying it to real-life emergency events. METHODS: A prospective, video-based study was conducted in an academic pediatric emergency department. Three reviewers independently applied the CALM tool to the assessment of pediatric emergency department physicians as they led both a cardiac arrest and a sepsis event. Time to critical event (epinephrine, fluid, and antibiotic administration) was collected via video review. Based on Kane's framework, we conducted fully crossed, person × event × rater generalizability (G) and decision (D) studies. Interrater reliability was calculated using Gwet AC 2 and intraclass correlation coefficients. Time to critical events was correlated with CALM scores using Spearman coefficient. RESULTS: Nine team leaders were assessed in their leadership of 2 resuscitations each. The G coefficient was 0.68, with 26% subject variance, 20% rater variance, and no case variance. Thirty-three percent of the variance (33%) was attributed to third-order interactions and unknown factors. Gwet AC 2 was 0.3 and intraclass correlation was 0.58. The CALM score and time to epinephrine correlated at -0.79 ( P = 0.01). The CALM score and time to fluid administration correlated at -0.181 ( P = 0.64). CONCLUSIONS: This study provides additional validity evidence for the CALM tool's use in this context if used with multiple raters, aligning with data from the previous simulation-based CALM validity study. Further development may improve reliability. It also serves as an exemplar of the rigors of conducting validity work within medical simulation.

Video-Assisted Laryngoscopy for Pediatric Tracheal Intubation in the Emergency Department: A Multicenter Study of Clinical Outcomes.

STUDY OBJECTIVE: To explore the association between video-assisted laryngoscopy (use of a videolaryngoscope regardless of where laryngoscopists direct their gaze), first-attempt success, and adverse airway outcomes. METHODS: We conducted an observational study using data from 2 airway consortiums that perform prospective surveillance: the National Emergency Airway Registry for Children (NEAR4KIDS) and a pediatric emergency medicine airway education collaborative. Data collected included patient and procedural characteristics and procedural outcomes. We performed multivariable analyses of the association of video-assisted laryngoscopy with individual patient outcomes and evaluated the association between site-level video-assisted laryngoscopy use and tracheal intubation outcomes. RESULTS: The study cohort included 1,412 tracheal intubation encounters performed from January 2017 to March 2021 across 11 participating sites. Overall, the first-attempt success was 70.0%. Video-assisted laryngoscopy was associated with increased odds of first-attempt success (odds ratio [OR] 2.01; 95% confidence interval [CI], 1.48 to 2.73) and decreased odds of severe adverse airway outcomes (OR 0.70; 95% CI, 0.58 to 0.85) including decreased severe hypoxia (OR 0.69; 95% CI, 0.55 to 0.87). Sites varied substantially in the use of video-assisted laryngoscopy (range from 12.9% to 97.8%), and sites with high use of video-assisted laryngoscopy (> 80%) experienced increased first-attempt success even after adjusting for individual patient laryngoscope use (OR 2.30; 95% CI, 1.79 to 2.95). CONCLUSION: Video-assisted laryngoscopy is associated with increased first-attempt success and fewer adverse airway outcomes for patients intubated in the pediatric emergency department. There is wide variability in the use of video-assisted laryngoscopy, and the high use is associated with increased odds of first-attempt success.

Making a move: Using simulation to identify latent safety threats before the care of injured patients in a new physical space.

BACKGROUND: In today's rapidly changing health care environment, hospitals are expanding into newly built spaces. Preserving patient safety by identifying latent safety threats (LSTs) in advance of opening a new physical space is key to continued excellent care. At our level 1 pediatric trauma center, the hospital undertook a 5-year project to build a critical care tower, including a new emergency department with five trauma bays. To allow for identification and mitigation of LSTs before opening, we performed simulation-based clinical systems testing. METHODS: Eight simulation scenarios were developed, based on actual patient presentations, incorporating a variety of injury patterns. Scenarios included workflow and movement from the helipad and squad entrance as well as to radiology, the operating room, and the pediatric intensive care unit. A multiple resuscitation scenario was also designed to test the use of all five bays simultaneously. Multidisciplinary high-fidelity simulations were conducted in the new tower. Key trauma and emergency department stakeholders facilitated all sessions, using a structured framework for systems integration debriefing framework and failure mode and effect analysis to identify and prioritize LSTs, respectively. RESULTS: Eight sessions were conducted for 2 months. A total of 201 staff participated, including trauma surgeons, respiratory therapists, nurses, emergency physicians, x-ray technicians, pharmacists, emergency medical services, and operating room staff. In total, 118 LSTs (average of 14.8/session) were identified. Latent safety threats were categorized. An action plan for mitigation was developed after applying failure mode and effects analysis prioritization scores (based on severity, probability, and ease of detection). CONCLUSION: Systems-focused trauma simulations identified a large number of LSTs before the opening of a new critical care building. Identification of LSTs is feasible and facilitates mitigation before actual patient care begins, improving patient safety. LEVEL OF EVIDENCE: Therapeutic/Care Management; Level IV.