Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room.

OBJECTIVE: To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance. BACKGROUND: Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons. METHODS: A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context. RESULTS: From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making. CONCLUSIONS: The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

Measuring and Improving Emotional Intelligence in Surgery: A Systematic Review.

OBJECTIVE: Evaluate how emotional intelligence (EI) has been measured among surgeons and to investigate interventions implemented for improving EI. SUMMARY BACKGROUND: EI has relevant applications in surgery given its alignment with nontechnical skills. In recent years, EI has been measured in a surgical context to evaluate its relationship with measures such as surgeon burnout and the surgeon-patient relationship. METHODS: A systematic review was conducted by searching MEDLINE, EMBASE, CINAHL, and PSYCINFO databases using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. MeSH terms and keywords included "emotional intelligence," "surgery," and "surgeon." Eligible studies included an EI assessment of surgeons, surgical residents, and/or medical students within a surgical context. RESULTS: The initial search yielded 4627 articles. After duplicate removal, 4435 articles were screened by title and abstract and 49 articles proceeded to a full-text read. Three additional articles were found via hand search. A total of 37 articles were included. Studies varied in surgical specialties, settings, and outcome measurements. Most occurred in general surgery, residency programs, and utilized self-report surveys to estimate EI. Notably, EI improved in all studies utilizing an intervention. CONCLUSIONS: The literature entailing the intersection between EI and surgery is diverse but still limited. Generally, EI has been demonstrated to be beneficial in terms of overall well-being and job satisfaction while also protecting against burnout. EI skills may provide a promising modifiable target to achieve desirable outcomes for both the surgeon and the patient. Future studies may emphasize the relevance of EI in the context of surgical teamwork.

Dissecting Cardiac Surgery: A Video-based Recall Protocol to Elucidate Team Cognitive Processes in the Operating Room.

OBJECTIVE: The aim of this study was to elucidate the cognitive processes involved in surgical procedures from the perspective of different team roles (surgeon, anesthesiologist, and perfusionist) and provide a comprehensive compilation of intraoperative cognitive processes. SUMMARY BACKGROUND DATA: Nontechnical skills play a crucial role in surgical team performance and understanding the cognitive processes underlying the intraoperative phase of surgery is essential to improve patient safety in the operating room (OR). METHODS: A mixed-methods approach encompassing semistructured interviews with 9 subject-matter experts. A cognitive task analysis was built upon a hierarchical segmentation of coronary artery bypass grafting procedures and a cued-recall protocol using video vignettes was used. RESULTS: A total of 137 unique surgical cognitive processes were identified, including 33 decision points, 23 critical communications, 43 pitfalls, and 38 strategies. Self-report cognitive workload varied substantially, depending on team role and surgical step. A web-based dashboard was developed, providing an integrated visualization of team cognitive processes in the OR that allows readers to intuitively interact with the study findings. CONCLUSIONS: This study advances the current body of knowledge by making explicit relevant cognitive processes involved during the intraoperative phase of cardiac surgery from the perspective of multiple OR team members. By displaying the research findings in an interactive dashboard, we provide trainees with new knowledge in an innovative fashion that could be used to enhance learning outcomes. In addition, the approach used in the present study can be used to deeply understand the cognitive factors underlying surgical adverse events and errors in the OR.

Analysis of Dynamic Changes in Cognitive Workload During Cardiac Surgery Perfusionists' Interactions With the Cardiopulmonary Bypass Pump.

OBJECTIVE: This novel preliminary study sought to capture dynamic changes in heart rate variability (HRV) as a proxy for cognitive workload among perfusionists while operating the cardiopulmonary bypass (CPB) pump during real-life cardiac surgery. BACKGROUND: Estimations of operators' cognitive workload states in naturalistic settings have been derived using noninvasive psychophysiological measures. Effective CPB pump operation by perfusionists is critical in maintaining the patient's homeostasis during open-heart surgery. Investigation into dynamic cognitive workload fluctuations, and their relationship with performance, is lacking in the literature. METHOD: HRV and self-reported cognitive workload were collected from three Board-certified cardiac perfusionists (N = 23 cases). Five HRV components were analyzed in consecutive nonoverlapping 1-min windows from skin incision through sternal closure. Cases were annotated according to predetermined phases: prebypass, three phases during bypass, and postbypass. Values from all 1min time windows within each phase were averaged. RESULTS: Cognitive workload was at its highest during the time between initiating bypass and clamping the aorta (preclamp phase during bypass), and decreased over the course of the bypass period. CONCLUSION: We identified dynamic, temporal fluctuations in HRV among perfusionists during cardiac surgery corresponding to subjective reports of cognitive workload. Not only does cognitive workload differ for perfusionists during bypass compared with pre- and postbypass phases, but differences in HRV were also detected within the three bypass phases. APPLICATION: These preliminary findings suggest the preclamp phase of CPB pump interaction corresponds to higher cognitive workload, which may point to an area warranting further exploration using passive measurement.

Surgery Task Load Index in Cardiac Surgery: Measuring Cognitive Load Among Teams.

Background. The most commonly used subjective assessment of perceived cognitive load, the NASA Task Load Index (TLX), has proven valuable in measuring individual load among general populations. The surgery task load index (SURG-TLX) was developed and validated to measure cognitive load specifically among individuals within a surgical team. Notably, the TLX lacks temporal sensitivity in its typical retrospective administration. Objective. This study sought to expand the utility of SURG-TLX by investigating individual measures of cognitive load over time during cardiac surgery, and the relationship between individual and team measures of cognitive load and proxies for surgical complexity. Materials & Methods. SURG-TLX was administered retrospectively in the operating room immediately following each case to approximate cognitive load before, during, and after cardiopulmonary bypass for cardiac surgery team members (surgeon, anesthesiologist, and perfusionist). Correlations were calculated to determine the relationship of individual and team measures of cognitive load over the entire procedure with bypass length and surgery length. Results. Results suggest that perceived cognitive load varies throughout the procedure such that cognitive load during bypass significantly differs compared to before or after bypass, across all 3 roles. While on bypass, results show that anesthesiologists experience significantly lower levels of perceived cognitive load than both surgeons and perfusionists. Correlational analyses reveal that perceived cognitive load of both the surgeon and the team had significant positive associations with bypass length and surgery length. Conclusion. Our findings support the utility of SURG-TLX in real cardiac cases as a measure of cognitive load over time, and on an individual and team-wide basis.

Physiological synchronization and entropy as measures of team cognitive load.

The operating room (OR) is a high-risk and complex environment, where multiple specialized professionals work as a team to effectively care for patients in need of surgical interventions. Surgical tasks impose high cognitive demands on OR staff and cognitive overload may have deleterious effects on team performance and patient safety. The aim of the present study was to investigate the feasibility and describe a novel methodological approach to characterize dynamic changes in team cognitive load by measuring synchronization and entropy of heart rate variability parameters during real-life cardiac surgery. Cognitive load was measured by capturing interbeat intervals (IBI) from three team members (surgeon, anesthesiologist and perfusionist) using an unobtrusive wearable heart rate sensor and transmitted in real-time to a smartphone application. Clinical data and operating room audio/video recordings were also collected to provide behavioral and contextual information. We developed symbolic representations of the transient cognitive state of individual team members (Individual Cognitive State - ICS), and overall team (Team Cognitive State - TCS) by comparing IBI data from each team member with themselves and with others. The distribution of TCS symbols during surgery enabled us to display and analyze temporal states and dynamic changes of team cognitive load. Shannon's entropy was calculated to estimate the changing levels of team organization and to detect fluctuations resulting from a variety of cognitive demands and/or specific situations (e.g. medical error, emergency, flow disruptions). An illustrative example from a real cardiac surgery team shows how cognitive load patterns shifted rapidly after an actual near-miss medication event, leading the team to a more organized and synchronized state. The methodological approach described in this study provides a measurement technique for the assessment of team physiological synchronization, which can be applied to many other team-based environments. Future research should gather additional validity evidence to support the proposed methods for team cognitive load measurement.

Interprofessional simulation to improve safety in the epilepsy monitoring unit.

OBJECTIVE: Patient safety is critical for epilepsy monitoring units (EMUs). Effective training is important for educating all personnel, including residents and nurses who frequently cover these units. We performed a needs assessment and developed a simulation-based team training curriculum employing actual EMU sentinel events to train neurology resident-nurse interprofessional teams to maximize effective responses to high-acuity events. METHODS: A mixed-methods design was used. This included the development of a safe-practice checklist to assess team response to acute events in the EMU using expert review with consensus (a modified Delphi process). All nineteen incoming first-year neurology residents and 2 nurses completed a questionnaire assessing baseline knowledge and attitudes regarding seizure management prior to and following a team training program employing simulation and postscenario debriefing. Four resident-nurse teams were recorded while participating in two simulated scenarios. Employing retrospective video review, four trained raters used the newly developed safe-practice checklist to assess team performance. We calculated the interobserver reliability of the checklist for consistency among the raters. We attempted to ascertain whether the training led to improvement in performance in the actual EMU by comparing 10 videos of resident-nurse team responses to seizures 4-8months into the academic year preceding the curricular training to 10 that included those who received the training within 4-8months of the captured video. RESULTS: Knowledge in seizure management was significantly improved following the program, but confidence in seizure management was not. Interrater agreement was moderate to high for consistency of raters for the majority of individual checklist items. We were unable to demonstrate that the training led to sustainable improvement in performance in the actual EMU by the method we used. CONCLUSIONS: A simulated team training curriculum using a safe-practice checklist to improve the management of acute events in an EMU may be an effective method of training neurology residents. However, translating the results into sustainable benefits and confidence in management in the EMU requires further study.

Exploring Intraoperative Cognitive Biases in Cardiac Surgery Teams.

This study focuses on understanding the influence of cognitive biases in the intra-operative decision-making process within cardiac surgery teams, recognizing the complexity and high-stakes nature of such environments. We aimed to investigate the perceived prevalence and impact of cognitive biases among cardiac surgery teams, and how these biases may affect intraoperative decisions and patient safety and outcomes. A mixed-methods approach was utilized, combining quantitative ratings across 32 different cognitive biases (0 to 100 visual analogue scale), regarding their "likelihood of occurring" and "potential for patient harm" during the intraoperative phase of cardiac surgery. Based on these ratings, we collected qualitative insights on the most-rated cognitive biases from semi-structured interviews with surgeons, anaesthesiologists, and perfusionists who work in a cardiac operating room. A total of 16 participants, including cardiac surgery researchers and clinicians, took part in the study. We found a significant presence of cognitive biases, particularly confirmation bias and overconfidence, which influenced decision-making processes and had the potential for patient harm. Of 32 cognitive biases, 6 were rated above the 75th percentile for both criteria (potential for patient harm, likelihood of occurring). Our preliminary findings provide a first step toward a deeper understanding of the complex cognitive mechanisms that underlie clinical reasoning and decision-making in the operating room. Future studies should further explore this topic, especially the relationship between the occurrence of intraoperative cognitive biases and postoperative surgical outcomes. Additionally, the impact of metacognition strategies (e.g. debiasing training) on reducing the impact of cognitive bias and improving intraoperative performance should also be investigated.

Formalizing the Hidden Curriculum of Performance Enhancing Errors.

Despite its inevitability, error remains an uncomfortable topic for discussion amongst surgeons. There are a range of reasons cited for this; significantly, there is an inextricable link between a surgeon's actions and their patient's outcomes. Attempts to reflect on error are often unstructured and without a defined end point, and modern surgical curricula lack content to guide residents' learning on recognizing and reflecting on sentinel events. There is a need to develop a tool to guide a standardized, safe, and constructive response to error. The current educational paradigm revolves around error avoidance. However, there is an evolving evidence base surrounding the inclusion of error management theory (EMT) into surgical training. This method explores and incorporates positive discussions surrounding errors, and has been demonstrated to improve long-term skill acquisition and training outcomes. We must harness the performance enhancing effects of our errors in the same way we do our successes. Implicated in all surgical performance is human factors science/ergonomics (HFE) - the interface between psychology, engineering, and performance. Developing a national HFE curriculum in the context of EMT would provide a common language to facilitate objective reflections regarding surgeons' operative performance and manage the stigma associated with fallibility.