Impact of team performance on the surgical safety checklist on patient outcomes: an operating room black box analysis.

BACKGROUND: Surgical safety checklists reduce adverse events, but monitoring adherence to checklists is confounded by observation bias. The ORBB platform can monitor checklist compliance and correlate compliance with outcomes. This study aims to evaluate the association between checklist compliance and patient outcomes using the ORBB platform. METHODS: This is a retrospective analysis of data from the electronic medical record of cases performed in ORBB-equipped operating rooms at a single quaternary referral center. All patients who did not opt out and underwent surgery at UT Southwestern Medical Center in ORBB-equipped rooms from August 2020 to September 2022 were included. The ORBB platform was set-up in five operating rooms and surgical safety checklist compliance was monitored by way of AI-based video review. RESULTS: Overall, 4581 patients were included in this analysis.. Performance on the checklist was associated with lower mortality (OR, 0.96; 95% CI, 0.94-0.98; P < 0.05), and decreased length of stay (estimate [E]: -0.02 days; 95% CI, -0.03 to -0.005; P < 0.05). Performance during "timeouts" was associated with mortality (OR, 0.97; 95% CI, 0.94-0.99; P < 0.05). "Debriefings" were independently associated with mortality (OR, 0.98; 95% CI, 0.96-0.99; P < 0.05), length of stay (Estimate, -0.0009 days; 95% CI, -0.02 to -0.001; P < 0.05), and ICU admission (OR, 0.99; 95% CI, 0.98-0.99; P < 0.05). CONCLUSION: Procedures performed by surgical teams who performed better on the surgical safety checklist tended to have better outcomes. This innovative technology could substantially enhance our ability to understand and mitigate threats to patients in real-time.

Using OR Black Box Technology to Determine Quality Improvement Outcomes for In-situ Timeout and Debrief Simulation.

OBJECTIVE: The purpose of this study was to determine quality improvement outcomes following the pilot implementation of an in-situ simulation designed to enhance surgical safety checklist performance. BACKGROUND: OR Black Box (ORBB) technology allows near real-time assessment for surgical safety checklist performance. Before our study, timeout quality was 73.3%, compliance was 99.9%, and engagement was 89.7% (n=1993 cases); Debrief Quality was 76.0%, compliance was 66.9%, and engagement was 66.7% (n=1842 cases). METHODS: This IRB-approved study used prospective convergent multi-methods. During 2 months, a 15-minute in-situ simulation, incorporating rapid cycle deliberate practice, was implemented for OR teams. ORBB analytics generated Timeout and Debrief scores for actual operations performed by surgeons who participated in simulation (Sim-group) versus those who did not (No-sim group) over 6 months, including 2 months pre-intervention, during-intervention, and post-intervention. Inductive content analysis was performed based on simulation discussions to determine team member perspectives. RESULTS: Thirty simulations with 163 interprofessional participants were conducted. ORBB data from 1570 cases were analyzed. Scores were significantly better for the Sim-group compared with the No-sim group for debrief quality (84% vs. 79% P<0.001, during-intervention), compliance (73% vs. 66%, P<0.001, post-intervention), and engagement (80% vs. 73%, P=0.012, during-intervention). There were no between-group differences for Timeout scores. Thematic analysis identified 2 primary categories: "culture of safety" and "policy." CONCLUSIONS: This simulation-based QI intervention created a psychologically safe training environment for OR teams. The novel use of ORBB technology facilitated outcome analysis and showed significantly better Debrief scores for simulation-trained surgeons compared with nontrained surgeons.