RESUMEN
OBJECTIVE: The purpose of this study was to determine quality improvement outcomes following the pilot implementation of an in-situ simulation designed to enhance surgical safety checklist performance. BACKGROUND: OR Black Box (ORBB) technology allows near real-time assessment for surgical safety checklist performance. Before our study, timeout quality was 73.3%, compliance was 99.9%, and engagement was 89.7% (n=1993 cases); Debrief Quality was 76.0%, compliance was 66.9%, and engagement was 66.7% (n=1842 cases). METHODS: This IRB-approved study used prospective convergent multi-methods. During 2 months, a 15-minute in-situ simulation, incorporating rapid cycle deliberate practice, was implemented for OR teams. ORBB analytics generated Timeout and Debrief scores for actual operations performed by surgeons who participated in simulation (Sim-group) versus those who did not (No-sim group) over 6 months, including 2 months pre-intervention, during-intervention, and post-intervention. Inductive content analysis was performed based on simulation discussions to determine team member perspectives. RESULTS: Thirty simulations with 163 interprofessional participants were conducted. ORBB data from 1570 cases were analyzed. Scores were significantly better for the Sim-group compared with the No-sim group for debrief quality (84% vs. 79% P<0.001, during-intervention), compliance (73% vs. 66%, P<0.001, post-intervention), and engagement (80% vs. 73%, P=0.012, during-intervention). There were no between-group differences for Timeout scores. Thematic analysis identified 2 primary categories: "culture of safety" and "policy." CONCLUSIONS: This simulation-based QI intervention created a psychologically safe training environment for OR teams. The novel use of ORBB technology facilitated outcome analysis and showed significantly better Debrief scores for simulation-trained surgeons compared with nontrained surgeons.
RESUMEN
OBJECTIVE: We report for the first time the use of the Operating Room Black Box (ORBB) to track checklist compliance, engagement, and quality. BACKGROUND: Implementation of operative checklists is associated with improved outcomes. Compliance is difficult to monitor. Most studies report either no assessment of checklist compliance or deployed in-person short-term assessment. The ORBB a novel artificially intelligence-driven data analytic platform affords the opportunity to assess checklist compliance without disrupting surgical workflow. METHODS: This was a retrospective review of prospectively collected ORBB data. Operative cases included elective surgery at a quaternary referral center. Cases were analyzed as prepolicy change (first 9 months) or as a postpolicy change (last 9 months). Measures of checklist compliance, engagement, and quality were assessed. RESULTS: There were 3879 cases that were performed and monitored for checklist compliance between August 15, 2020, and February 20, 2022. The overall scores for compliance, engagement, and quality were 81%, 84%, and 67% respectively. When broken down by phase, the scores for time-out were compliance 100%, engagement 98%, and quality 61%. Scores for the debrief phase were 81% for compliance, 98% for engagement, and 66% for quality. After a hospital policy change, the debrief scores improved significantly (85%; P <0.001 for compliance, 88%; P <0.001 for engagement and 71%; P <0.001 for quality). CONCLUSIONS: ORBB provides the unprecedented ability to assess not only compliance with surgical safety checklists but also engagement and quality. Utilization of this technology allows the assessment of compliance in near real time and to accurately address safety threats that may arise from noncompliance.