Using the Operating Room Black Box to Assess Surgical Team Member Adaptation Under Uncertainty: An Observational Study.

OBJECTIVE: Identify how surgical team members uniquely contribute to teamwork and adapt their teamwork skills during instances of uncertainty. BACKGROUND: The importance of surgical teamwork in preventing patient harm is well documented. Yet, little is known about how key roles (nurse, anesthesiologist, surgeon, and medical trainee) uniquely contribute to teamwork during instances of uncertainty, particularly when adapting to and rectifying an intraoperative adverse event (IAE). METHODS: Audiovisual data of 23 laparoscopic cases from a large community teaching hospital were prospectively captured using OR Black Box. Human factors researchers retrospectively coded videos for teamwork skills (backup behavior, coordination, psychological safety, situation assessment, team decision-making, and leadership) by team role under 2 conditions of uncertainty: associated with an IAE versus no IAE. Surgeons identified IAEs. RESULTS: In all, 1015 instances of teamwork skills were observed. Nurses adapted to IAEs by expressing more backup behavior skills (5.3× increase; 13.9 instances/hour during an IAE vs 2.2 instances/hour when no IAE) while surgeons and medical trainees expressed more psychological safety skills (surgeons: 3.6× increase; 30.0 instances/hour vs 6.6 instances/hour and trainees: 6.6× increase; 31.2 instances/hour vs 4.1 instances/hour). All roles expressed fewer situation assessment skills during an IAE versus no IAE. CONCLUSIONS: OR Black Box enabled the assessment of critically important details about how team members uniquely contribute during instances of uncertainty. Some teamwork skills were amplified, while others dampened when dealing with IAEs. The knowledge of how each role contributes to teamwork and adapts to IAEs should be used to inform the design of tailored interventions to strengthen interprofessional teamwork.

Creating a Practical Transformational Change Management Model for Novel Artificial Intelligence-Enabled Technology Implementation in the Operating Room.

Objective: To identify change management (CM) strategies for implementing novel artificial intelligence and similar novel technologies in operating rooms and create a new CM model for future trials and applications inspired by the abovementioned strategies and established models. Methods: Key phases of technology implementation were defined, and strategies for transformational CM were created and applied in a recent CM experience at our institution between October 15, 2020 and October 15, 2021. We appraised existing CM models and propose the newly created model. Results: The key phases of the technology implementation were as follows: (1) team assembly; (2) committee approvals; (3) CM; and (4) system installation and go-live. Key strategies were (1) assemble team with necessary expertise; (2) anticipate potential institutional cultural and regulatory hurdles; (3) add agility to project planning and execution; (4) accommodate institutional culture and regulations; (5) early clinical partner buy-in and stakeholder engagement; and (6) consistent communication, all of which contributed to the new CM model creation. Conclusion: Key CM strategies and a new CM model addressing the unique needs and characteristics of operating room novel technology implementation were identified and created. The new model may be customized and tested for individual institution and project's needs and characteristics.

Analyzing and Discussing Human Factors Affecting Surgical Patient Safety Using Innovative Technology: Creating a Safer Operating Culture.

INTRODUCTION: Surgical errors often occur because of human factor-related issues. A medical data recorder (MDR) may be used to analyze human factors in the operating room. The aims of this study were to assess intraoperative safety threats and resilience support events by using an MDR and to identify frequently discussed safety and quality improvement issues during structured postoperative multidisciplinary debriefings using the MDR outcome report. METHODS: In a cross-sectional study, 35 standard laparoscopic procedures were performed and recorded using the MDR. Outcome data were analyzed using the automated Systems Engineering Initiative for Patient Safety model. The video-assisted MDR outcome report reflects on safety threat and resilience support events (categories: person, tasks, tools and technology, psychical and external environment, and organization). Surgeries were debriefed by the entire team using this report. Qualitative data analysis was used to evaluate the debriefings. RESULTS: A mean (SD) of 52.5 (15.0) relevant events were identified per surgery. Both resilience support and safety threat events were most often related to the interaction between persons (272 of 360 versus 279 of 400). During the debriefings, communication failures (also category person) were the main topic of discussion. CONCLUSIONS: Patient safety threats identified by the MDR and discussed by the operating room team were most frequently related to communication, teamwork, and situational awareness. To create an even safer operating culture, educational and quality improvement initiatives should aim at training the entire operating team, as it contributes to a shared mental model of relevant safety issues.

Eye Tracking Use in Surgical Research: A Systematic Review.

INTRODUCTION: Eye tracking (ET) is a popular tool to study what factors affect the visual behaviour of surgical team members. To our knowledge, there have been no reviews to date that evaluate the broad use of ET in surgical research. This review aims to identify and assess the quality of this evidence, to synthesize how ET can be used to inform surgical practice, and to provide recommendations to improve future ET surgical studies. METHODS: In line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a systematic literature review was conducted. An electronic search was performed in MEDLINE, Cochrane Central, Embase, and Web of Science databases up to September 2020. Included studies used ET to measure the visual behaviour of members of the surgical team during surgery or surgical tasks. The included studies were assessed by two independent reviewers. RESULTS: A total of 7614 studies were identified, and 111 were included for data extraction. Eleven applications were identified; the four most common were skill assessment (41%), visual attention assessment (22%), workload measurement (17%), and skills training (10%). A summary was provided of the various ways ET could be used to inform surgical practice, and three areas were identified for the improvement of future ET studies in surgery. CONCLUSIONS: This review provided a comprehensive summary of the various applications of ET in surgery and how ET could be used to inform surgical practice, including how to use ET to improve surgical education. The information provided in this review can also aid in the design and conduct of future ET surgical studies.

Six Sigma in surgery: how to create a safer culture in the operating theatre using innovative technology.

Safe delivery of patient care in the operating theatre is complex and co-dependent of many individual, organisational, and environmental factors, including patient, task and technology, individual, and human factors. The Six Sigma approach aims to implement a data-driven strategy to reduce variability and consequently improve safety. Analytical data platforms such as a Black Box ought to be embraced to support process optimisation and ultimately create a higher level of Six Sigma safety performance of the operating theatre team.

Development of an International Standardized Curriculum for Laparoscopic Sleeve Gastrectomy Teaching Utilizing Modified Delphi Methodology.

BACKGROUND: The performance of laparoscopic sleeve gastrectomy has increased markedly to become the single-most performed bariatric surgical procedure globally. To date, a means of standardized trainee teaching has not been developed. The aim of this study was to design a laparoscopic curriculum for trainees of bariatric surgery utilizing modified Delphi consensus methodology. METHODS: A panel of surgeons was assembled to devise an academic framework of technical, non-technical and cognitive skills utilized in the performance of laparoscopic sleeve gastrectomy. The panel invited 18 bariatric surgeons experienced in laparoscopic gastrectomy from 11 countries to rate the items for inclusion in the curriculum to a predefined level of agreement. RESULTS: A consensus of experts was achieved for 24 of the 30 proposed elements for inclusion within the first round of the curriculum Delphi panel. All components pertaining to anatomical knowledge, peri-operative considerations and non-technical items were accepted. A second round further examined six statements, of which three were accepted. Agreement of the panel was reached for 27 of the cognitive, technical and non-technical components after two rounds. Three statements found no consensus. CONCLUSIONS: Utilizing modified Delphi methodology, a curriculum outlining the most important components of teaching the procedure of laparoscopic sleeve gastrectomy, has been determined by a consensus of international experts in bariatric surgery. The curriculum is suggested as a standard in proficiency-based training of this procedure. It forms a generic template which facilitates individual jurisdictions to perform content validation, adapting the curriculum to local requirements in teaching the next generation of bariatric surgeons.

Quantifying the "Assistant Effect" in Robotic-Assisted Radical Prostatectomy (RARP): Measures of Technical Performance.

PURPOSE: Surgeons are reliant on the bedside assistant during robotic surgeries. Using a modified global rating scale (GRS), we aim to assess the association between an assistant's technical skill on surgeon performance in Robotic-Assisted Radical Prostatectomy (RARP). METHODS: Prospective, intraoperative video from RARP cases at three centers were collected. Baseline demographic and RARP-experience data were collected from participating surgeons and trainees. The dissection of the prostatic pedicle and neurovascular bundle step (NVB) was analyzed. Expert analysts scored the console surgeon performance using the Global Evaluative Assessment of Robotic Skills (GEARS), and the bedside assistant performance using a modified Objective Structured Assessment of Technical Skills (aOSATS). The primary outcome is the association between console surgeon performance, as measured by GEARS, and assistant skill, as measured by aOSATS. Spearman's rho correlations were used to test the relationship between assistant and surgeon technical performance, and a multivariable linear regression model was created to test this association while controlling for patient factors. RESULTS: 92 RARP cases were available for the analysis, comprising 14 console surgeons and 22 different bedside assistants. In only 5 (5.4%) cases, the neurovascular bundle step was completed by a trainee, and in 13 (14.1%) of cases, a staff-level surgeon acted as the bedside assistant. aOSATS score was significantly associated with robotic console experience (P = 0.011), and prior laparoscopic experience (P < 0.001). Assistant aOSATS score showed a weak but significant correlation with surgeon GEARS score during the neurovascular bundle step (spearman's rho = 0.248, P = 0.028). On linear regression, aOSATS remained a significant predictor of console surgeon performance (P = 0.016), after controlling for patient age and BMI, prostate volume, tumor stage, and presence of nerve-sparing. CONCLUSIONS: This is the first study to assess the association between assistant technical skill and surgeon performance in RARP. Additionally, we have provided validity evidence for a modified OSATS global rating scale for training and assessing bedside assistant performance.

System Factors Affecting Patient Safety in the OR: An Analysis of Safety Threats and Resiliency.

OBJECTIVE: The objective of this study is to determine the characteristics and frequency of intraoperative safety threats and resilience supports using a human factors measurement tool. BACKGROUND: Human factors analysis can provide insight into how system elements contribute to intraoperative adverse events. Empiric evidence on safety threats and resilience in surgical practice is lacking. METHODS: A cross-sectional study of 24 patients undergoing elective laparoscopic general surgery at a single center in the Netherlands from May to November, 2017 was conducted. Video, audio, and patient physiologic data from all included procedures were obtained through a multichannel synchronized recording device. Trained analysts reviewed the recordings and coded safety threats and resilience supports. The codes were categorized into 1 of 6 categories (person, task, tools and technology, physical environment, organization, and external environment). RESULTS: A median of 14 safety threats [interquartile range (IQR) 11-16] and 12 resilience supports (IQR 11-16) were identified per case. Most safety threat codes (median 9, IQR 7-12) and resilience support codes (median 10, IQR 7-12) were classified in the person category. The organization category contained a median of 2 (IQR 1-2) safety threat codes and 2 (IQR 2-3) resilience support codes per case. The tools and technology category contributed a small number of safety threats (median 1 per case, IQR 0-1), but rarely provided resilience support. CONCLUSIONS: Through a detailed human factors analysis of elective laparoscopic general surgery cases, this study provided a quantitative analysis of the existing safety threats and resilience supports in a modern endoscopic operating room.

Development of a Model for Video-Assisted Postoperative Team Debriefing.

BACKGROUND: Video-assisted debriefing may be a powerful tool to improve surgical team performance. Nevertheless, a true operating team debriefing culture is lacking to date. This study aimed to find evidence on how to debrief the surgical team and develop a model suitable for debriefing using a video and medical data recorder (MDR) in the operating room (OR). METHODS: A review of the PubMed and Embase databases and Cochrane Library was performed. The identified literature was studied and combined with a conceptual framework to develop a model for postoperative video-assisted team debriefing. Thirty-five surgical cases were recorded with an MDR and debriefed with the operating team using the proposed debrief model and a standardized video-assisted performance report. A questionnaire was used to assess the participants' satisfaction with this debrief model. RESULTS: Debrief models and methods are extensively described in the current medical literature. An overview was provided. The OR team needs a structured debrief model, minimizing resource, effort, and motivational constraints. A structured six-step team debrief model suitable for video-assisted OR team debriefing was developed. The model was tested in 35 multidisciplinary MDR-assisted debriefing sessions and the debriefing sessions were overall rated with a mean of 7.8 (standard deviation 1.4, 10-point Likert scale) by participants. CONCLUSIONS: Debriefing surgical teams using a video and MDR in the OR requires a model on how to use such recordings optimally. To date, no such model existed. The proposed debrief model was tested using a multisource MDR and may be used to facilitate OR debriefing across various settings.

Design and validation of an inertial measurement unit (IMU)-based sensor for capturing camera movement in the operating room.

Intraoperative surgical video enables better surgical training, continued performance enhancement for surgeons and system-level quality improvement initiatives, however the capture of high-quality intraoperative video of open surgical procedures is difficult. Wearable cameras, typically in the form of a head-mounted action camera are frequently used for this purpose, although the video from these devices often contains significant motion artifact due to movement of the surgeon's head. When trying to compare the performance of various wearable cameras in the surgical setting, we could not find a motion sensor appropriate for this purpose. We therefore describe in this article the design, assembly and validation of a small sensor that can be attached to wearable cameras in the operating room to objectively quantify camera motion. The sensor incorporates an inertial measurement unit coupled to a microcontroller. Concurrent validity is established by comparing the positional sensing of the device to a geared tripod head that allows for fine, measured manipulations of the sensor in three orthogonal axes. The methodology of capturing, processing and reporting camera movement for a surgical procedure is also detailed.

Development of the User Experience (UX) and Video Quality Evaluation (VQE) Instruments for Assessment of Intraoperative Video Capture Technology.

OBJECTIVE: There exists a lack of technology to reliably and routinely capture high-quality video of open surgical procedures. To critically evaluate and compare new and existing technology solutions, we must have widely accepted evaluation criteria for intraoperative camera devices. The objective, therefore, was to develop evaluation criteria for intraoperative camera devices, as well as the video product they produce. DESIGN: A modified Delphi process that included 2 iterative surveys was used to build expert consensus and develop 2 evaluation instruments: one to evaluate the user experience (UX) of using an intraoperative camera device, and the second for video quality evaluation (VQE) of the video product. SETTING: Global, through iterative online surveys. PARTICIPANTS: Surgeons who perform open surgery and have experience with intraoperative video capture. RESULTS: Eighty-six experts participated in the first iteration of the survey and 46 in the second. Ten factors met the a priori cutoff for >80% agreement for the UX survey: (1) ease of setup/integration with current practice, (2) comfort, (3) distracting during case, (4) overall satisfaction with wearing the device, (5) would you use this device again, (6) would you recommend this device to colleagues, (7) the weight of wearing the device, (8) sufficient battery life, (9) ability to control device while operating, and (10) degree to which the device interferes or is incompatible with other surgical accessories. Six factors met the cutoff for the VQE survey: (1) camera stability, (2) brightness/exposure, (3) resolution/sharpness, (4) unobstructed view of the surgical field, (5) appropriate field of view, and (6) overall satisfaction with video quality. CONCLUSIONS: These instruments can be used to critically evaluate camera technologies for intraoperative video capture of open surgery.

Assessment of 3-Dimensional vs 2-Dimensional Imaging and Technical Performance Using a Multiport Intraoperative Data Capture and Analytic System for Patients Undergoing Laparoscopic Roux-en-Y Gastric Bypass Surgery.

IMPORTANCE: Errors and adverse events occur frequently in health care. Three-dimensional (3-D) laparoscopic systems claim to provide more realistic depth perception and better spatial orientation compared with their 2-D counterparts. OBJECTIVE: To compare the association of 3-D vs 2-D systems with technical performance during laparoscopic Roux-en-Y gastric bypass (LRYGB) procedures using a multiport intraoperative data capture system. DESIGN, SETTING, AND PARTICIPANTS: This cohort study was performed between May and December 2018, with a total of 50 LRYGB procedures performed in an academic tertiary care center; recordings of the operations were evaluated with a 30-day follow-up. All procedures were performed by the same surgical team. EXPOSURE: Surgical teams used 2-D or 3-D laparoscopic systems. MAIN OUTCOMES AND MEASURES: Technical performance was evaluated using the Objective Structured Assessment of Technical Skill and surgical errors and events using the Generic Error Rating Tool. RESULTS: Of the 50 patients who underwent LRYGB procedures, 42 (86%) were women, with a median (interquartile range) age of 42 (35-47) years and a median (interquartile range) body mass index of 46 (42-48), with no significant demographic differences between the groups whose operations were performed using the 2-D and 3-D systems. The mean (SD) number of errors per case was significantly lower in procedures using the 3-D laparoscopic system than in those using the 2-D system (17 [6] vs 33 [2]; P < .001). The mean (SD) number of error-related events was significantly lower in procedures using the 3-D system than in those using the 2-D system (6 [2] vs 11 [4]; P < .001). Mean (SD) Objective Structured Assessment of Technical Skill scores were significantly higher when the 3-D system was used than when the 2-D system was used (28 [4] vs 22 [3]; P < .001). CONCLUSIONS AND RELEVANCE: In this limited sample of LRYGB procedures, the use of a 3-D laparoscopic system was associated with a statistically significant reduction in errors and events as well as higher Objective Structured Assessment of Technical Skill scores compared with 2-D systems.

System Factors Affecting Intraoperative Risk and Resilience: Applying a Novel Integrated Approach to Study Surgical Performance and Patient Safety.

OBJECTIVE: To identify and categorize system factors in complex laparoscopic surgery that have the potential to either threaten patient safety or support system resilience. BACKGROUND: The operating room is a uniquely complex sociotechnical work system wherein surgical successes prevail despite pervasive safety threats. Holistically characterizing intraoperative factors that thus support system resilience in addition to those that threaten patient safety using contextual methodologies is critical for optimizing surgical safety overall. METHOD: In this prospective descriptive interdisciplinary study, 19 audio/video recordings of complex laparoscopic general surgical procedures were directly observed and transcribed. Using a qualitative systems-based approach, intraoperative human factors with the potential to impact patient safety, either as a safety threat or as a support for resilience, were identified. Adverse events were further assessed for shared threats and supports. Data collection was guided by the Systems Engineering Initiative for Patient Safety 2.0 work system model. RESULTS: A total of 1083 relevant observations were made over 39.8 hours of operative time, enabling the identification of 79 distinct safety threats and 67 resilience supports within the surgical system. Safety threats associated with the physical environment, tasks, organization, and equipment were prevalent and observed in equal measure, whereas supports for resilience were predominantly attributed to clinician behaviors, including proactive team management and skills coaching. Two subclinical adverse events were identified; shared safety threats included suboptimal technology design, whereas shared resilience supports included calm clinician behavior and redundant intraoperative resourcing. CONCLUSIONS: Safety threats and resilience supports were found to be systematic in the surgical setting. Identified safety threats should be prioritized for remediation, and clinician behaviors that contribute to fostering resilience should be valued and protected.

Automated Methods of Technical Skill Assessment in Surgery: A Systematic Review.

OBJECTIVE: The goal of the current study is to systematically review the literature addressing the use of automated methods to evaluate technical skills in surgery. BACKGROUND: The classic apprenticeship model of surgical training includes subjective assessments of technical skill. However, automated methods to evaluate surgical technical skill have been recently studied. These automated methods are a more objective, versatile, and analytical way to evaluate a surgical trainee's technical skill. STUDY DESIGN: A literature search of the Ovid Medline, Web of Science, and EMBASE Classic databases was performed. Articles evaluating automated methods for surgical technical skill assessment were abstracted. The quality of all included studies was assessed using the Medical Education Research Study Quality Instrument. RESULTS: A total of 1715 articles were identified, 76 of which were selected for final analysis. An automated methods pathway was defined that included kinetics and computer vision data extraction methods. Automated methods included tool motion tracking, hand motion tracking, eye motion tracking, and muscle contraction analysis. Finally, machine learning, deep learning, and performance classification were used to analyse these methods. These methods of surgical skill assessment were used in the operating room and simulated environments. The average Medical Education Research Study Quality Instrument score across all studies was 10.86 (maximum score of 18). CONCLUSIONS: Automated methods for technical skill assessment is a growing field in surgical education. We found quality studies evaluating these techniques across many environments and surgeries. More research must be done to ensure these techniques are further verified and implemented in surgical curricula.

Video Technologies for Recording Open Surgery: A Systematic Review.

Video recording of surgical procedures is an important tool for surgical education, performance enhancement, and error analysis. Technology for video recording open surgery, however, is limited. The objective of this article is to provide an overview of the available literature regarding the various technologies used for intraoperative video recording of open surgery. A systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines using the MEDLINE, Cochrane Central, and EMBASE databases. Two authors independently screened the titles and abstracts of the retrieved articles, and those that satisfied the defined inclusion criteria were selected for a full-text review. A total of 2275 publications were initially identified, and 110 were included in the final review. The included articles were categorized based on type of article, surgical subspecialty, type and positioning of camera, and limitations identified with their use. The most common article type was primary-technical (29%), and the dominant specialties were general surgery (22%) and plastic surgery (18%). The most commonly cited camera used was the GoPro (30%) positioned in a head-mount configuration (60%). Commonly cited limitations included poor video quality, inadequate battery life, light overexposure, obstruction by surgical team members, and excessive motion. Open surgery remains the mainstay of many surgical specialties today, and technological innovation is absolutely critical to fulfill the unmet need for better video capture of open surgery. The findings of this article will be valuable for guiding future development of novel technology for this purpose.

Underreporting of Veress Needle Injuries: Comparing Direct Observation and Chart Review Methods.

BACKGROUND: Adverse events in surgery occur frequently, increase likelihood of postoperative morbidity, and mostly take place in the operating rooms. Several surgeons have advocated for learning from adverse events and near misses to help improve patient safety. To do so, one must first understand how to accurately identify and report intraoperative events. MATERIALS AND METHODS: Consecutive laparoscopic cases performed in a referral center were included in the cohort. Veress needle (VN) injuries were characterized according to a priori established criteria. Two methods were used to identify VN injuries: direct observation and patient chart review. For direct observation, trained surgeon assessors identified the outcomes using a comprehensive data capture platform called the operating room black box. On the other hand, operative reports and patient charts were reviewed by trained assessors to identify reported VN injuries. RESULTS: Hundred thirty-one cases were analyzed. There were 12 (9%) VN injuries identified by direct observation compared to 3 (2%) identified in patient chart review method. Injuries to the liver and stomach were identified by both methods, whereas injuries to the omentum were not reported in patient charts even if they required rectification. There were seven VN injuries that required rectification, lasting up to 12% of the operating time. There were 47 (35%) near misses identified through direct observation, whereas none was reported in patient charts. CONCLUSIONS: Direct observation enables characterization of VN injury and near misses with far greater detail and accuracy than patient chart review.

A Novel Method of Setting Performance Standards in Surgery Using Patient Outcomes.

OBJECTIVE: To describe a novel, outcome-based method of standard setting that differentiates between clinical outcomes rather than arbitrary educational goals. BACKGROUND: Standard setting methods used in assessments of procedural skill are currently not evidence-driven or outcome-based. This represents a potential obstacle for the broad implementation of these evaluations in summative assessments such as certification and credentialing. METHODS: The concept is based on deriving a receiver operating characteristic curve from a regression model that incorporates measures of intraoperative surgeon performance and confounding patient characteristics. This allows the creation of a performance standard that best predicts a clinically significant outcome of interest. The discovery cohort used to create the predictive model was derived from pilot data that used the Global Evaluative Assessment of Robotic Skill assessment tool to predict patient urinary continence 3 months following robotic-assisted radical prostatectomy. RESULTS: A receiver operating characteristic curve with an area under the curve of 0.75 was created from predicted probability statistic generated by the predictive model. We chose a predicted probability of 0.35, based on an optimal tradeoff in sensitivity and specificity (Youden Index). Rearranging the regression equation, we determined the performance score required to predict a 35%, patient-adjusted probability of postoperative urinary incontinence. CONCLUSIONS: This novel methodology is context, patient, and assessment-specific. Current standard setting methods do not account for the heterogeneity of the clinical environment. Workplace-based assessments in competency-based medical education require standards that are credible to the educator and the trainee. High-stakes assessments must ensure that surgeons have been evaluated to a standard that prioritizes satisfactory patient outcomes and safety.