RESUMO
OBJECTIVE: The purpose of this study was to determine quality improvement outcomes following the pilot implementation of an in-situ simulation designed to enhance surgical safety checklist performance. BACKGROUND: OR Black Box (ORBB) technology allows near real-time assessment for surgical safety checklist performance. Before our study, timeout quality was 73.3%, compliance was 99.9%, and engagement was 89.7% (n=1993 cases); Debrief Quality was 76.0%, compliance was 66.9%, and engagement was 66.7% (n=1842 cases). METHODS: This IRB-approved study used prospective convergent multi-methods. During 2 months, a 15-minute in-situ simulation, incorporating rapid cycle deliberate practice, was implemented for OR teams. ORBB analytics generated Timeout and Debrief scores for actual operations performed by surgeons who participated in simulation (Sim-group) versus those who did not (No-sim group) over 6 months, including 2 months pre-intervention, during-intervention, and post-intervention. Inductive content analysis was performed based on simulation discussions to determine team member perspectives. RESULTS: Thirty simulations with 163 interprofessional participants were conducted. ORBB data from 1570 cases were analyzed. Scores were significantly better for the Sim-group compared with the No-sim group for debrief quality (84% vs. 79% P<0.001, during-intervention), compliance (73% vs. 66%, P<0.001, post-intervention), and engagement (80% vs. 73%, P=0.012, during-intervention). There were no between-group differences for Timeout scores. Thematic analysis identified 2 primary categories: "culture of safety" and "policy." CONCLUSIONS: This simulation-based QI intervention created a psychologically safe training environment for OR teams. The novel use of ORBB technology facilitated outcome analysis and showed significantly better Debrief scores for simulation-trained surgeons compared with nontrained surgeons.
RESUMO
Purpose: Due to the pandemic, we restructured our medical student knot-tying simulation to a virtual format. This study evaluated curriculum feasibility and effectiveness. Methods: Over 4 weeks, second-year medical students (n = 229) viewed a video tutorial (task demonstration, errors, scoring) and self-practiced to proficiency (no critical errors, < 2 min) using at-home suture kits (simple interrupted suture, instrument tie, penrose drain model). Optional virtual tutoring sessions were offered. Students submitted video performance for proficiency verification. Two sets of 14 videos were viewed by two surgeons until inter-rater reliability (IRR) was established. Students scoring "needs remediation" attended virtual remediation sessions. Non-parametric statistics were performed using RStudio. Results: All 229 medical students completed the curriculum within 1-4 h; 1.3% attended an optional tutorial. All videos were assessed: 4.8% "exceeds expectations", 60.7% "meets expectations", and 34.5% "needs remediation." All 79 needing remediation due to critical errors achieved proficiency during 1-h group sessions. IRR Cohen's κ was 0.69 (initial) and 1.0 (ultimate). Task completion time was 56 (47-68) s (median [IQR]); p < 0.01 between all pairs. Students rated the overall curriculum (79.2%) and overall curriculum and video tutorial effectiveness (92.7%) as "agree" or "strongly agree". No definitive preference emerged regarding virtual versus in-person formats; however, 80.2% affirmed wanting other at-home skills curricula. Comments supported home practice as lower stress; remediation students valued direct formative feedback. Conclusions: A completely virtual 1-month knot-tying simulation is feasible and effective in achieving proficiency using video-based assessment and as-needed remediation strategies for a large student class.