Using OR Black Box Technology to Determine Quality Improvement Outcomes for In-situ Timeout and Debrief Simulation.

OBJECTIVE: The purpose of this study was to determine quality improvement outcomes following the pilot implementation of an in-situ simulation designed to enhance surgical safety checklist performance. BACKGROUND: OR Black Box (ORBB) technology allows near real-time assessment for surgical safety checklist performance. Before our study, timeout quality was 73.3%, compliance was 99.9%, and engagement was 89.7% (n=1993 cases); Debrief Quality was 76.0%, compliance was 66.9%, and engagement was 66.7% (n=1842 cases). METHODS: This IRB-approved study used prospective convergent multi-methods. During 2 months, a 15-minute in-situ simulation, incorporating rapid cycle deliberate practice, was implemented for OR teams. ORBB analytics generated Timeout and Debrief scores for actual operations performed by surgeons who participated in simulation (Sim-group) versus those who did not (No-sim group) over 6 months, including 2 months pre-intervention, during-intervention, and post-intervention. Inductive content analysis was performed based on simulation discussions to determine team member perspectives. RESULTS: Thirty simulations with 163 interprofessional participants were conducted. ORBB data from 1570 cases were analyzed. Scores were significantly better for the Sim-group compared with the No-sim group for debrief quality (84% vs. 79% P<0.001, during-intervention), compliance (73% vs. 66%, P<0.001, post-intervention), and engagement (80% vs. 73%, P=0.012, during-intervention). There were no between-group differences for Timeout scores. Thematic analysis identified 2 primary categories: "culture of safety" and "policy." CONCLUSIONS: This simulation-based QI intervention created a psychologically safe training environment for OR teams. The novel use of ORBB technology facilitated outcome analysis and showed significantly better Debrief scores for simulation-trained surgeons compared with nontrained surgeons.

Interprofessional staff perspectives on the adoption of or black box technology and simulations to improve patient safety: a multi-methods survey.

INTRODUCTION: Medical errors still plague healthcare. Operating Room Black Box (ORBB) and ORBB-simulation (ORBBSIM) are innovative emerging technologies which continuously capture as well as categorize intraoperative data, team information, and audio-visual files, in effort to improve objective quality measures. ORBB and ORBBSIM have an opportunity to improve patient safety, yet a paucity of implementation literature exists. Overcoming implementation barriers is critical. This study sought to obtain rich insights while identifying facilitators and barriers to adoption of ORBB and ORBBSIM in alignment with Donabedian's model of health services and healthcare quality. Enrichment themes included translational performance improvement and real-world examples to develop sessions. METHODS: Interprofessional OR staff were invited to complete two surveys assessing staff's perceptions using TeamSTEPPS's validated Teamwork Perceptions Questionnaire (T-TPQ) and open-ended questions. Descriptive statistics were calculated for quantitative variables, and inductive phenomenological content analysis was used for qualitative. RESULTS: Survey 1 captured 71 responses from 334 invited (RR 21%) while survey 2 captured 47 responses from 157 (RR 29.9%). The T-TPQ score was 65.2, with Communication (70.4) the highest construct and Leadership (58.0) the lowest. Quality Improvement (QI), Patient Safety, and Objective Case Review were the most common perceived ORBB benefits. Trends suggested a reciprocal benefit of dual ORBB and ORBBSIM adoption. Trends also suggested that dual implementation can promote Psychological Safety, culture, trust, and technology comfort. The need for an implementation plan built on change management principles and a constructive culture were key findings. CONCLUSIONS: Findings supported ORBB implementation themes from previous literature and deepened our understanding through the exploration of team culture. This blueprint provides a model to help organizations adopt ORBB and ORBBSIM. Outcomes can establish an empirical paradigm for future studies.

Simulation-Based Education in US Undergraduate Medical Education: A Descriptive Study.

PURPOSE: Simulation-based education (SBE) provides experiential learning, improvement in quality of care, and reduction in errors. In 2011, the Association of American Medical Colleges described adoption of SBE in 68.0% of medical schools and 25.0% of teaching hospitals. We sought to examine current trends of SBE integration in American undergraduate medical education since previous publications. METHODS: From 2016 to 2019, University of Texas Southwestern Medical Center postgraduate year 1 residents were invited to participate in a survey assessing medical school simulation experience with 26 clinical tasks from three categories: procedural, communication, and other. Deidentified results were analyzed to assess demographics including sex, specialty, residency program type, allopathic versus osteopathic medical school, and medical school region. RESULTS: Nine hundred sixty-seven of 1047 (92.3%) responses were obtained, representing 139 US medical schools, 91% from allopathic training. Of procedural tasks, most simulated was suturing (n = 848, 89.6%) and least simulated was thoracentesis (n = 737, 80.9%). Of communication tasks, most simulated was taking a history (n = 475, 51.1% reporting simulation >30) and least simulated (never or ≤1) were obtaining a consent (n = 669, 73.2%) and disclosing a medical error (n = 666, 72.4%). Of other tasks, most simulated was chest compressions (n = 898, 96.0%) and least simulated was operating a defibrillator (n = 206, 22.1%). Results were similar regardless of procedural or nonprocedural program. There was no significant difference in SBE exposure between allopathic and osteopathic students ( P = 0.89). Two participants (0.002%) reported no simulation exposure. CONCLUSIONS: Our study is the first to describe a high prevalence of SBE adoption in medical schools nationwide since the Association of American Medical Colleges' 2011 publication, with overall equal exposure for students regardless of residency type and allopathic or osteopathic medical school. Despite widespread adoption of simulation, opportunities remain to expand SBE use to teach critically important communication skills.

The role of simulation-based training in healthcare-associated infection (HAI) prevention.

Objectives: To perform a review of the literature on the role of simulation-based training (SBT) in healthcare-associated infection (HAI) prevention and to highlight the importance of SBT as an educational tool in infection prevention. Methods: We reviewed English language publications from PubMed to select original articles that utilized SBT as the primary mode of education for infection prevention efforts in acute-care hospitals. Results: Overall, 27 publications utilized SBT as primary mode of education for HAI prevention in acute-care hospitals. Training included the following: hand hygiene in 3 studies (11%), standard precaution in 1 study (4%), disaster preparedness in 4 studies (15%), central-line-associated blood stream infection (CLABSI) prevention in 14 studies (52%), catheter-associated urinary tract infection (CAUTI) prevention in 2 studies (7%), surgical site infection prevention in 2 studies (7%), and ventilatory associated pneumonia prevention in 1 study (4%). SBT improved learner's sense of competence and confidence, increased knowledge and compliance in infection prevention measures, decreased HAI rates, and reduced healthcare costs. Conclusion: SBT can function as a teaching tool in day-to-day infection prevention efforts as well as in disaster preparedness. SBT is underutilized in infection prevention but can serve as a crucial educational tool.

Coronavirus disease 2019 (COVID-19) personal protective equipment training: Using simulation-based training to prevent healthcare-associated infection.

In this pre- and postintervention study, we demonstrate the feasibility and effectiveness of comprehensive simulation-based personal protective equipment (PPE) training amid the COVID-19 pandemic. With rapid-cycle, deliberate practice feedback, simulation-based training can improve the learners' sense of confidence and security while standardizing PPE protocols.

Advanced age, comorbidity and the risk of mortality in COVID-19 infection.

BACKGROUND: Advanced age and comorbidities have been described to increase the risk of mortality associated with COVID-19 infection. However, the degree to which comorbidities influence mortality among younger and older adults with and without comorbidity in COVID-19 infection has not been clearly elucidated. OBJECTIVE: To examine the impact of comorbidity on mortality among younger and older unvaccinated adults with COVID-19 infection admitted to a safety-net hospital. METHODS: This is a retrospective study in which 638 unvaccinated COVID-19 positive study participants admitted to a safety-net hospital between March 1, 2020, and August 31, 2020 were included. The risk of in-hospital mortality or referral to hospice (adverse outcome) was compared among younger and older participants with and without comorbidity. RESULTS: A total of 62 patients had adverse outcome while in the hospital (10%). Risk factors independently associated with adverse outcome included advanced age (OR(CI) 9.21 (2.29-37.06), p=.002), male sex (OR(CI) 2.6(1.34-5.16), p=.005), living in most disadvantaged area (OR(CI) 2.42(1.8-5.42), p=.03), history of diabetes (OR(CI) 2.35(1.12-4.95), p=.023), and history of heart failure (OR(CI) 4.00(2.09-7.63), p<.001). Further analysis after creating risk groups based on participants age and the presence of diabetes and / or heart failure was performed. The probability of adverse outcome was highest among older male participants with comorbidities (Pr =0.315 (CI: 0.176-0.454)). The probability of adverse outcome among older participants without diabetes and heart failure (Pr =0.081 (CI: .010 -0.152) was less than the probability for younger patients with diabetes and heart failure (Pr: 203 (CI: 0.103 - 0.303) CONCLUSIONS: While older adults with comorbidities were the most vulnerable for adverse outcome, the risk of adverse outcome among older adults without comorbidities was less than that of younger adults with comorbidities.

At-home medical student simulation: achieving knot-tying proficiency using video-based assessment.

Purpose: Due to the pandemic, we restructured our medical student knot-tying simulation to a virtual format. This study evaluated curriculum feasibility and effectiveness. Methods: Over 4 weeks, second-year medical students (n = 229) viewed a video tutorial (task demonstration, errors, scoring) and self-practiced to proficiency (no critical errors, < 2 min) using at-home suture kits (simple interrupted suture, instrument tie, penrose drain model). Optional virtual tutoring sessions were offered. Students submitted video performance for proficiency verification. Two sets of 14 videos were viewed by two surgeons until inter-rater reliability (IRR) was established. Students scoring "needs remediation" attended virtual remediation sessions. Non-parametric statistics were performed using RStudio. Results: All 229 medical students completed the curriculum within 1-4 h; 1.3% attended an optional tutorial. All videos were assessed: 4.8% "exceeds expectations", 60.7% "meets expectations", and 34.5% "needs remediation." All 79 needing remediation due to critical errors achieved proficiency during 1-h group sessions. IRR Cohen's κ was 0.69 (initial) and 1.0 (ultimate). Task completion time was 56 (47-68) s (median [IQR]); p < 0.01 between all pairs. Students rated the overall curriculum (79.2%) and overall curriculum and video tutorial effectiveness (92.7%) as "agree" or "strongly agree". No definitive preference emerged regarding virtual versus in-person formats; however, 80.2% affirmed wanting other at-home skills curricula. Comments supported home practice as lower stress; remediation students valued direct formative feedback. Conclusions: A completely virtual 1-month knot-tying simulation is feasible and effective in achieving proficiency using video-based assessment and as-needed remediation strategies for a large student class.

Does a Written Tool to Guide Structured Debriefing Improve Discourse? Implications for Interprofessional Team Simulation.

PURPOSE: Timely debriefing following a simulated event supports learners in critically reflecting on their performance and areas for improvement. Content of debriefing has been shown to affect learner skill acquisition and retention. The use of good judgment statements from debriefing facilitators is considered superior to judgmental or nonjudgmental statements. Ideally, the majority of the conversation will consist of learner self-reflection and focused facilitation rather than directive performance feedback. We hypothesized that the introduction of a written tool to help facilitate high-quality debriefing techniques could improve the ratio of judgmental, nonjudgmental, and good judgment statements from facilitators, as well as shift the percentage of talk in the debrief away from directive performance feedback and toward self-assessment and focused facilitation. METHODS: The University of Wisconsin Joint Trauma Simulation Program is an interdisciplinary project to improve quality of trauma care through simulation. Simulations use teams of five trauma trainees: two surgery residents, an emergency medicine resident, and two nurses. Three faculty members conducted the scenarios and debriefings. Debriefings were video recorded. Videos were transcribed and dialogue analyzed according to the teaching/learning strategy used in each turn of talk. Discourse was coded into three categories: (1) learner self-assessment; (2) focused facilitation; and (3) directive performance feedback. Each facilitation statement was coded as either (1) judgmental; (2) nonjudgmental, or (3) good judgment. The TEAM Debrief Tool is a written guide designed to help facilitators adhere to best practices, with example structure and phrasing, similar to the Promoting Excellence and Reflective Learning in Simulation tool. Pre- and post-implementation analysis was completed to assess for efficacy of the tool. RESULTS: Seven videos before the implementation of the tool and seven videos after implementation were analyzed. The percentage of learner self-assessment increased significantly with tool use (7.23% vs 24.99%, p = 0.00004), and directive performance feedback decreased significantly (56.13% vs 32.75%, p = 0.0042). There was no significant change in the percentage of talk using focused facilitation. After implementation of the tool, there was a significant decrease in use of the nonjudgmental debriefing style (60.63% vs 37.31%, p = 0.00017), and a significant increase in the use of good judgment debriefing (38.77% vs 59.82%, p = 0.00038). There was also a slight increase in judgmental debriefing (0.60% vs 2.87%, p = 0.0027). CONCLUSIONS: The discourse in our interprofessional trauma simulation debriefings unaided by a written debriefing tool skewed heavily toward direct performance feedback, with a preponderance of nonjudgmental statements. After introduction of the tool, dialogue shifted significantly toward learner self-assessment, and there was a large increase in utilization of debriefing with good judgment. This shift toward higher quality debriefing styles demonstrates the utility of such a tool in the debriefing of interprofessional simulations.

Pre-simulation orientation for medical trainees: An approach to decrease anxiety and improve confidence and performance.

BACKGROUND: We assessed the effect of basic orientation to the simulation environment on anxiety, confidence, and clinical decision making. METHODS: Twenty-four graduating medical students participated in a two-week surgery preparatory curriculum, including three simulations. Baseline anxiety was assessed pre-course. Scenarios were completed on day 2 and day 9. Prior to the first simulation, participants were randomly divided into two groups. Only one group received a pre-simulation orientation. Before the second simulation, all students received the same orientation. Learner anxiety was reported immediately preceding and following each simulation. Confidence was assessed post-simulation. Performance was evaluated by surgical faculty. RESULTS: The oriented group experienced decreased anxiety following the first simulation (p = 0.003); the control group did not. Compared to the control group, the oriented group reported less anxiety and greater confidence and received higher performance scores following all three simulations (all p < 0.05). CONCLUSIONS: Pre-simulation orientation reduces anxiety while increasing confidence and improving performance.

Identifying Nontechnical Skill Deficits in Trainees Through Interdisciplinary Trauma Simulation.

OBJECTIVE: The goal of this study was to investigate nontechnical skills in a simulated trauma setting both before and after a debriefing session in order to better understand areas to target for the development of educational interventions. DESIGN: Wilcoxon signed rank tests were used to compare scores on the 5 domains of the T-NOTECHS pre- and postdebriefings. A qualitative analysis using the PEARLS debriefing framework was performed to provide a rich description of the strategies used by the debriefing facilitators. SETTING: The Joint Trauma Simulation Program is an interdisciplinary project designed to improve the quality of trauma care through simulation exercises emphasizing nontechnical skills development. PARTICIPANTS: Thirteen teams of 5 trauma trainees participated in trauma resuscitation simulations: a surgical chief resident, a surgical junior resident, an emergency medicine resident, and 2 emergency medicine nurses. RESULTS: Teams significantly improved on communication and interaction skills in the simulation scenarios from pre- to postdebriefing. The debrief facilitators spent most of their time engaged in Directive Performance Feedback (56.13%). CONCLUSIONS: Interprofessional team simulation in trauma resuscitation scenarios followed by debriefing differently affected individual nontechnical skills domains. Additional facilitation strategies, such as focused facilitation and encouraging learner self-assessment, may target other nontechnical skills in different ways.