A novel multimodal, intraoperative cognitive workload assessment of cardiac surgery team members.

OBJECTIVE: To characterize cognitive workload (CWL) of cardiac surgery team members in a real-world setting during coronary artery bypass grafting (CABG) surgery using providers' heart rate variability (HRV) data as a surrogate measure of CWL. METHODS: HRV was collected from the surgeon, anesthesiologist, perfusionist, and scrub nurse, and audio/video recordings were made during isolated, nonemergency CABG surgeries (n = 27). Eight surgical phases were annotated by trained researchers, and HRV was calculated for each phase. RESULTS: Significant differences in CWL were observed within a given role across surgical phases. Results are reported as predicted probability (95% confidence interval [CI]). CWL was significantly higher for anesthesiologists during "preparation and induction" (0.57; 95% CI, 0.42-0.71) and "anastomoses" (0.44; 95% CI, 0.30-0.58) compared to other phases, and the same held for nurses during the "opening" (0.51; 95% CI, 0.37-0.65) and "postoperative" (0.68; 95% CI, 0.42-0.86) phases. Additional significant differences were observed between roles within a given surgical phase. For example, surgeons had significantly higher CWL during "anastomoses" (0.81; 95% CI, 0.69-0.89) compared to all other phases, and the same was true of perfusionists during the "opening" (0.79; 95% CI, 0.66-0.88) and "prebypass preparation" (0.50; 95% CI, 0.36-0.64) phases. CONCLUSIONS: Our innovative analysis demonstrates that CWL fluctuates across surgical procedures by role and phase, which may reflect the distribution of primary tasks. This corroborates earlier findings from self-report measures. The data suggest that team-wide, peak CWL during a phase decreases from early phases of surgery through initiation of cardiopumonary bypass (CPB), rises during anastomosis, and decreases after termination of CPB. Knowledge of these trends could encourage the adoption of behaviors to enhance team dynamics and performance.

Deep Learning Analysis of Surgical Video Recordings to Assess Nontechnical Skills.

Importance: Assessing nontechnical skills in operating rooms (ORs) is crucial for enhancing surgical performance and patient safety. However, automated and real-time evaluation of these skills remains challenging. Objective: To explore the feasibility of using motion features extracted from surgical video recordings to automatically assess nontechnical skills during cardiac surgical procedures. Design, Setting, and Participants: This cross-sectional study used video recordings of cardiac surgical procedures at a tertiary academic US hospital collected from January 2021 through May 2022. The OpenPose library was used to analyze videos to extract body pose estimations of team members and compute various team motion features. The Non-Technical Skills for Surgeons (NOTSS) assessment tool was employed for rating the OR team's nontechnical skills by 3 expert raters. Main Outcomes and Measures: NOTSS overall score, with motion features extracted from surgical videos as measures. Results: A total of 30 complete cardiac surgery procedures were included: 26 (86.6%) were on-pump coronary artery bypass graft procedures and 4 (13.4%) were aortic valve replacement or repair procedures. All patients were male, and the mean (SD) age was 72 (6.3) years. All surgical teams were composed of 4 key roles (attending surgeon, attending anesthesiologist, primary perfusionist, and scrub nurse) with additional supporting roles. NOTSS scores correlated significantly with trajectory (r = 0.51, P = .005), acceleration (r = 0.48, P = .008), and entropy (r = -0.52, P = .004) of team displacement. Multiple linear regression, adjusted for patient factors, showed average team trajectory (adjusted R2 = 0.335; coefficient, 10.51 [95% CI, 8.81-12.21]; P = .004) and team displacement entropy (adjusted R2 = 0.304; coefficient, -12.64 [95% CI, -20.54 to -4.74]; P = .003) were associated with NOTSS scores. Conclusions and Relevance: This study suggests a significant link between OR team movements and nontechnical skills ratings by NOTSS during cardiac surgical procedures, suggesting automated surgical video analysis could enhance nontechnical skills assessment. Further investigation across different hospitals and specialties is necessary to validate these findings.

Behavioral sciences applied to acute care teams: a research agenda for the years ahead by a European research network.

BACKGROUND: Multi-disciplinary behavioral research on acute care teams has focused on understanding how teams work and on identifying behaviors characteristic of efficient and effective team performance. We aimed to define important knowledge gaps and establish a research agenda for the years ahead of prioritized research questions in this field of applied health research. METHODS: In the first step, high-priority research questions were generated by a small highly specialized group of 29 experts in the field, recruited from the multinational and multidisciplinary "Behavioral Sciences applied to Acute care teams and Surgery (BSAS)" research network - a cross-European, interdisciplinary network of researchers from social sciences as well as from the medical field committed to understanding the role of behavioral sciences in the context of acute care teams. A consolidated list of 59 research questions was established. In the second step, 19 experts attending the 2020 BSAS annual conference quantitatively rated the importance of each research question based on four criteria - usefulness, answerability, effectiveness, and translation into practice. In the third step, during half a day of the BSAS conference, the same group of 19 experts discussed the prioritization of the research questions in three online focus group meetings and established recommendations. RESULTS: Research priorities identified were categorized into six topics: (1) interventions to improve team process; (2) dealing with and implementing new technologies; (3) understanding and measuring team processes; (4) organizational aspects impacting teamwork; (5) training and health professions education; and (6) organizational and patient safety culture in the healthcare domain. Experts rated the first three topics as particularly relevant in terms of research priorities; the focus groups identified specific research needs within each topic. CONCLUSIONS: Based on research priorities within the BSAS community and the broader field of applied health sciences identified through this work, we advocate for the prioritization for funding in these areas.

Using Digital Biomarkers for Objective Assessment of Perfusionists' Workload and Acute Stress During Cardiac Surgery.

The cardiac operating room (OR) is a high-risk, high-stakes environment inserted into a complex socio-technical healthcare system. During cardiopulmonary bypass (CPB), the most critical phase of cardiac surgery, the perfusionist has a crucial role within the interprofessional OR team, being responsible for optimizing patient perfusion while coordinating other tasks with the surgeon, anesthesiologist, and nurses. The aim of this study was to investigate objective digital biomarkers of perfusionists' workload and stress derived from heart rate variability (HRV) metrics captured via a wearable physiological sensor in a real cardiac OR. We explored the relationships between several HRV parameters and validated self-report measures of surgical task workload (SURG-TLX) and acute stress (STAI-SF), as well as surgical processes and outcome measures. We found that the frequency-domain HRV parameter HF relative power - FFT (%) presented the strongest association with task workload (correlation coefficient: -0.491, p-value: 0.003). We also found that the time-domain HRV parameter RMSSD (ms) presented the strongest correlation with perfusionists' acute stress (correlation coefficient: -0.489, p-value: 0.005). A few workload and stress biomarkers were also associated with bypass time and patient length of stay in the hospital. The findings from this study will inform future research regarding which HRV-based biomarkers are best suited for the development of cognitive support systems capable of monitoring surgical workload and stress in real time.

An Open-Source, Interoperable Architecture for Generating Real-Time Surgical Team Cognitive Alerts from Heart-Rate Variability Monitoring.

Clinical alarm and decision support systems that lack clinical context may create non-actionable nuisance alarms that are not clinically relevant and can cause distractions during the most difficult moments of a surgery. We present a novel, interoperable, real-time system for adding contextual awareness to clinical systems by monitoring the heart-rate variability (HRV) of clinical team members. We designed an architecture for real-time capture, analysis, and presentation of HRV data from multiple clinicians and implemented this architecture as an application and device interfaces on the open-source OpenICE interoperability platform. In this work, we extend OpenICE with new capabilities to support the needs of the context-aware OR including a modularized data pipeline for simultaneously processing real-time electrocardiographic (ECG) waveforms from multiple clinicians to create estimates of their individual cognitive load. The system is built with standardized interfaces that allow for free interchange of software and hardware components including sensor devices, ECG filtering and beat detection algorithms, HRV metric calculations, and individual and team alerts based on changes in metrics. By integrating contextual cues and team member state into a unified process model, we believe future clinical applications will be able to emulate some of these behaviors to provide context-aware information to improve the safety and quality of surgical interventions.

Association Between Operating Room Noise and Team Cognitive Workload in Cardiac Surgery.

Excessive intra-operative noise in cardiac surgery has the potential to serve as source of distraction and additional cognitive workload for the surgical team, and may interfere with optimal performance. The separation from bypass phase is a technically complex phase of surgery, making it highly susceptible to communication breakdowns due to high cognitive demands and requiring tightly coupled team coordination. The objective of this study was to investigate team cognitive workload levels and communication in relation to intra-operative time periods representative of infrequent vs. frequent peaks in ambient noise. Compared to 5-minute segments with no peaks in noise at all, segments with the highest percentage of noise peaks (≥10%) were significantly associated with higher team members' heart rate before, during, and after noise segments analyzed. These noisier segments were also associated with a significantly higher level of case-irrelevant communication events. These data suggest that case-irrelevant conversations associated with a greater degree of excessive peaks in noise may be associated with team workload levels, warranting further investigation into efforts to standardize communication during critical surgical phases.

A Coding Framework for Usability Evaluation of Digital Health Technologies.

Several studies have reported low adherence and high resistance from clinicians to adopt digital health technologies into clinical practice, particularly the use of computer-based clinical decision support systems. Poor usability and lack of integration with the clinical workflow have been identified as primary issues. Few guidelines exist on how to analyze the collected data associated with the usability of digital health technologies. In this study, we aimed to develop a coding framework for the systematic evaluation of users' feedback generated during focus groups and interview sessions with clinicians, underpinned by fundamental usability principles and design components. This codebook also included a coding category to capture the user's clinical role associated with each specific piece of feedback, providing a better understanding of role-specific challenges and perspectives, as well as the level of shared understanding across the multiple clinical roles. Furthermore, a voting system was created to quantitatively inform modifications of the digital system based on usability data. As a use case, we applied this method to an electronic cognitive aid designed to improve coordination and communication in the cardiac operating room, showing that this framework is feasible and useful not only to better understand suboptimal usability aspects, but also to recommend relevant modifications in the design and development of the system from different perspectives, including clinical, technical, and usability teams. The framework described herein may be applied in other highly complex clinical settings, in which digital health systems may play an important role in improving patient care and enhancing patient safety.

Assessing Team Situational Awareness in the Operating Room via Computer Vision.

Situational awareness (SA) at both individual and team levels, plays a critical role in the operating room (OR). During the pre-incision time-out, the entire OR team comes together to deploy the surgical safety checklist (SSC). Worldwide, the implementation of the SSC has been shown to reduce intraoperative complications and mortality among surgical patients. In this study, we investigated the feasibility of applying computer vision analysis on surgical videos to extract team motion metrics that could differentiate teams with good SA from those with poor SA during the pre-incision time-out. We used a validated observation-based tool to assess SA, and a computer vision software to measure body position and motion patterns in the OR. Our findings showed that it is feasible to extract surgical team motion metrics captured via off-the-shelf OR cameras. Entropy as a measure of the level of team organization was able to distinguish surgical teams with good and poor SA. These findings corroborate existing studies showing that computer vision-based motion metrics have the potential to integrate traditional observation-based performance assessments in the OR.

Measuring and Improving Emotional Intelligence in Surgery: A Systematic Review.

OBJECTIVE: Evaluate how emotional intelligence (EI) has been measured among surgeons and to investigate interventions implemented for improving EI. SUMMARY BACKGROUND: EI has relevant applications in surgery given its alignment with nontechnical skills. In recent years, EI has been measured in a surgical context to evaluate its relationship with measures such as surgeon burnout and the surgeon-patient relationship. METHODS: A systematic review was conducted by searching MEDLINE, EMBASE, CINAHL, and PSYCINFO databases using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. MeSH terms and keywords included "emotional intelligence," "surgery," and "surgeon." Eligible studies included an EI assessment of surgeons, surgical residents, and/or medical students within a surgical context. RESULTS: The initial search yielded 4627 articles. After duplicate removal, 4435 articles were screened by title and abstract and 49 articles proceeded to a full-text read. Three additional articles were found via hand search. A total of 37 articles were included. Studies varied in surgical specialties, settings, and outcome measurements. Most occurred in general surgery, residency programs, and utilized self-report surveys to estimate EI. Notably, EI improved in all studies utilizing an intervention. CONCLUSIONS: The literature entailing the intersection between EI and surgery is diverse but still limited. Generally, EI has been demonstrated to be beneficial in terms of overall well-being and job satisfaction while also protecting against burnout. EI skills may provide a promising modifiable target to achieve desirable outcomes for both the surgeon and the patient. Future studies may emphasize the relevance of EI in the context of surgical teamwork.

The Cognitive Relevance of a Formal Pre-incision Time-out in Surgery.

Surgical time-outs are designed to promote situation awareness, teamwork, and error prevention. The pre-incision time-out in particular aims to facilitate shared mental models prior to incision. Objective, unbiased measures to confirm its effectiveness are lacking. We hypothesized that providers' mental workload would reveal team psychophysiological mirroring during a formal, well-executed pre-incision time-out. Heart rate variability was collected during cardiac surgery cases from the surgeon, anesthesiologist, and perfusionist. Data were analyzed for six cases from patient arrival until sternal closure. Annotation of surgical phases was completed according to previously developed standardized process models of aortic valve replacement and coronary artery bypass graft procedures, producing thirteen total surgical phases. Statistical analysis revealed significant main effects. Tukey HSD post hoc tests revealed significant differences across provider roles within various phases, including Anesthesia Induction, Heparinization, Initiation of Bypass, Aortic Clamp and Cardioplegia, Anastomoses or Aortotomy, Separation from Bypass, and Sternal Closure. Despite these observed differences between providers over various surgical phases, the Pre-incision Time-out phase revealed almost negligible differences across roles. This preliminary work supports the utility of the pre-incision safety checklist to focus the attention of surgical team members and promote shared team mental models, measured via psychophysiological mirroring, using an objective mental workload measure. Future studies should investigate the relationship between psychophysiological mirroring among surgical team members and the effectiveness of the pre-incision time-out checklist.

Computer Vision in the Operating Room: Opportunities and Caveats.

Effectiveness of computer vision techniques has been demonstrated through a number of applications, both within and outside healthcare. The operating room environment specifically is a setting with rich data sources compatible with computational approaches and high potential for direct patient benefit. The aim of this review is to summarize major topics in computer vision for surgical domains. The major capabilities of computer vision are described as an aid to surgical teams to improve performance and contribute to enhanced patient safety. Literature was identified through leading experts in the fields of surgery, computational analysis and modeling in medicine, and computer vision in healthcare. The literature supports the application of computer vision principles to surgery. Potential applications within surgery include operating room vigilance, endoscopic vigilance, and individual and team-wide behavioral analysis. To advance the field, we recommend collecting and publishing carefully annotated datasets. Doing so will enable the surgery community to collectively define well-specified common objectives for automated systems, spur academic research, mobilize industry, and provide benchmarks with which we can track progress. Leveraging computer vision approaches through interdisciplinary collaboration and advanced approaches to data acquisition, modeling, interpretation, and integration promises a powerful impact on patient safety, public health, and financial costs.

Towards an AI Coach to Infer Team Mental Model Alignment in Healthcare.

Shared mental models are critical to team success; however, in practice, team members may have misaligned models due to a variety of factors. In safety-critical domains (e.g., aviation, healthcare), lack of shared mental models can lead to preventable errors and harm. Towards the goal of mitigating such preventable errors, here, we present a Bayesian approach to infer misalignment in team members' mental models during complex healthcare task execution. As an exemplary application, we demonstrate our approach using two simulated team-based scenarios, derived from actual teamwork in cardiac surgery. In these simulated experiments, our approach inferred model misalignment with over 75% recall, thereby providing a building block for enabling computer-assisted interventions to augment human cognition in the operating room and improve teamwork.

Analysis of Mirrored Psychophysiological Change of Cardiac Surgery Team Members During Open Surgery.

OBJECTIVE: Mirrored psychophysiological change in cognitive workload indices may reflect shared mental models and effective healthcare team dynamics. In this exploratory analysis, we investigated the frequency of mirrored changes, defined as concurrent peaks in heart rate variability (HRV) across team members, during cardiac surgery. DESIGN: Objective cognitive workload was evaluated via HRV collected from the primary surgical team during cardiac surgery cases (N = 15). Root mean square of the successive differences (RMSSD) was calculated as the primary HRV measure. Procedures were divided into consecutive nonoverlapping 5-minute segments, and RMSSD along with deviations from RMSSD were calculated for each segment. Segments with positive deflections represent above-average cognitive workload. Positive deflections and peaks across dyads within the same segment were counted. SETTING: Data collection for this study took place in the cardiovascular operating room during live surgeries. PARTICIPANTS: Physiological data were collected and analyzed from the attending surgeon, attending anesthesiologist, and primary perfusionist involved with the recorded cases. RESULTS: Of the 641 five-minute segments analyzed, 325 (50.7%) were positive deflections above average, concurrently across at least 2 team members. Within the 325 positive deflections, 26 (8%) represented concurrent peaks in HRV across at least 2 active team members. Mirrored peaks across team members were observed most commonly during the coronary anastomoses or valve replacement phase (N = 12). CONCLUSIONS: In this pilot study, mirrored physiological responses representing peaks in cognitive workload were observed uncommonly across dyads of cardiac surgery team members (1.73 peaks/case on average). Almost half of these occurred during the most technically demanding phases of cardiac surgery, which may underpin teamwork quality. Future work should investigate interactions between technical and nontechnical performance surrounding times of mirrored peaks and expand the sample size.

Analysis of Dynamic Changes in Cognitive Workload During Cardiac Surgery Perfusionists' Interactions With the Cardiopulmonary Bypass Pump.

OBJECTIVE: This novel preliminary study sought to capture dynamic changes in heart rate variability (HRV) as a proxy for cognitive workload among perfusionists while operating the cardiopulmonary bypass (CPB) pump during real-life cardiac surgery. BACKGROUND: Estimations of operators' cognitive workload states in naturalistic settings have been derived using noninvasive psychophysiological measures. Effective CPB pump operation by perfusionists is critical in maintaining the patient's homeostasis during open-heart surgery. Investigation into dynamic cognitive workload fluctuations, and their relationship with performance, is lacking in the literature. METHOD: HRV and self-reported cognitive workload were collected from three Board-certified cardiac perfusionists (N = 23 cases). Five HRV components were analyzed in consecutive nonoverlapping 1-min windows from skin incision through sternal closure. Cases were annotated according to predetermined phases: prebypass, three phases during bypass, and postbypass. Values from all 1min time windows within each phase were averaged. RESULTS: Cognitive workload was at its highest during the time between initiating bypass and clamping the aorta (preclamp phase during bypass), and decreased over the course of the bypass period. CONCLUSION: We identified dynamic, temporal fluctuations in HRV among perfusionists during cardiac surgery corresponding to subjective reports of cognitive workload. Not only does cognitive workload differ for perfusionists during bypass compared with pre- and postbypass phases, but differences in HRV were also detected within the three bypass phases. APPLICATION: These preliminary findings suggest the preclamp phase of CPB pump interaction corresponds to higher cognitive workload, which may point to an area warranting further exploration using passive measurement.

Sensors for Continuous Monitoring of Surgeon's Cognitive Workload in the Cardiac Operating Room.

Monitoring healthcare providers' cognitive workload during surgical procedures can provide insight into the dynamic changes of mental states that may affect patient clinical outcomes. The role of cognitive factors influencing both technical and non-technical skill are increasingly being recognized, especially as the opportunities to unobtrusively collect accurate and sensitive data are improving. Applying sensors to capture these data in a complex real-world setting such as the cardiac surgery operating room, however, is accompanied by myriad social, physical, and procedural constraints. The goal of this study was to investigate the feasibility of overcoming logistical barriers in order to effectively collect multi-modal psychophysiological inputs via heart rate (HR) and near-infrared spectroscopy (NIRS) acquisition in the real-world setting of the operating room. The surgeon was outfitted with HR and NIRS sensors during aortic valve surgery, and validation analysis was performed to detect the influence of intra-operative events on cardiovascular and prefrontal cortex changes. Signals collected were significantly correlated and noted intra-operative events and subjective self-reports coincided with observable correlations among cardiovascular and cerebral activity across surgical phases. The primary novelty and contribution of this work is in demonstrating the feasibility of collecting continuous sensor data from a surgical team member in a real-world setting.

Surgery Task Load Index in Cardiac Surgery: Measuring Cognitive Load Among Teams.

Background. The most commonly used subjective assessment of perceived cognitive load, the NASA Task Load Index (TLX), has proven valuable in measuring individual load among general populations. The surgery task load index (SURG-TLX) was developed and validated to measure cognitive load specifically among individuals within a surgical team. Notably, the TLX lacks temporal sensitivity in its typical retrospective administration. Objective. This study sought to expand the utility of SURG-TLX by investigating individual measures of cognitive load over time during cardiac surgery, and the relationship between individual and team measures of cognitive load and proxies for surgical complexity. Materials & Methods. SURG-TLX was administered retrospectively in the operating room immediately following each case to approximate cognitive load before, during, and after cardiopulmonary bypass for cardiac surgery team members (surgeon, anesthesiologist, and perfusionist). Correlations were calculated to determine the relationship of individual and team measures of cognitive load over the entire procedure with bypass length and surgery length. Results. Results suggest that perceived cognitive load varies throughout the procedure such that cognitive load during bypass significantly differs compared to before or after bypass, across all 3 roles. While on bypass, results show that anesthesiologists experience significantly lower levels of perceived cognitive load than both surgeons and perfusionists. Correlational analyses reveal that perceived cognitive load of both the surgeon and the team had significant positive associations with bypass length and surgery length. Conclusion. Our findings support the utility of SURG-TLX in real cardiac cases as a measure of cognitive load over time, and on an individual and team-wide basis.

Feasibility of Healthcare Providers' Autonomic Activation Recognition in Real-Life Cardiac Surgery Using Noninvasive Sensors.

Cardiac surgery is one of the most complex specialties in medicine, akin to a complex sociotechnical system. Patient outcomes are vulnerable to surgical flow disruptions (SFDs), a source of preventable harm. Healthcare providers' (HCPs) sympathetic activation secondary to emotional states represent an underappreciated source of SFDs. This study's objective was to demonstrate the feasibility of detecting elevated sympathetic nervous system (SNS) activity as a proxy for emotional distress associated with a medication error using heart rate variability (HRV) analysis. After obtaining informed consent, audio/video and HRV data were captured intraoperatively during cardiac surgery from multiple HCPs. Following a critical medication administration error by the anesthesiologist in-training, the attending anesthesiologists' recorded HRV data was analyzed using pyphysio, an open-source signal analysis package, to identify events precipitating this near-miss event. We considered elevated low-frequency/high-frequency (LF/HF) HRV ratio (normal value <2) as a primary indicator of SNS activity and emotional distress. A heightened SNS response by the attending anesthesiologist, observed as an LF/HF ratio value of 3.39, was detected prior to the near-miss event. The attending anesthesiologist confirmed a state of significant SNS activity/distress induced by task-irrelevant environmental factors, which led to a temporarily ineffective mental model. Qualitative analysis of audio/video recordings revealed that SNS activation coincided with an argument over operating room management causing SFD. This preliminary study confirms the feasibility of recognizing potentially detrimental psychophysiological states during cardiac surgery in the wild using HRV analysis. To our knowledge, this is the first case demonstrating SNS activation coinciding with self-reported and observable emotional distress during live surgery using HRV. Irrespective of the HCP's expertise, transient but intense emotional changes may disrupt attention processes leading to SFDs and preventable errors. This work supports the possibility to detect real-time SNS activation, which could enable interventions to proactively mitigate errors. Additional studies on our large database of surgical cases are underway to confirm this observation.

Digital Cognitive Aids to Support Adaptation of Surgical Processes to COVID-19 Protective Policies.

Surgical processes are rapidly being adapted to address the COVID-19 pandemic, with changes in procedures and responsibilities being made to protect both patients and medical teams. These process changes put new cognitive demands on the medical team and increase the likelihood of miscommunication, lapses in judgment, and medical errors. We describe two process model driven cognitive aids, referred to as the Narrative View and the Smart Checklist View, generated automatically from models of the processes. The immediate perceived utility of these cognitive aids is to support medical simulations, particularly when frequent adaptations are needed to quickly respond to changing operating room guidelines.