A novel multimodal, intraoperative cognitive workload assessment of cardiac surgery team members.

OBJECTIVE: To characterize cognitive workload (CWL) of cardiac surgery team members in a real-world setting during coronary artery bypass grafting (CABG) surgery using providers' heart rate variability (HRV) data as a surrogate measure of CWL. METHODS: HRV was collected from the surgeon, anesthesiologist, perfusionist, and scrub nurse, and audio/video recordings were made during isolated, nonemergency CABG surgeries (n = 27). Eight surgical phases were annotated by trained researchers, and HRV was calculated for each phase. RESULTS: Significant differences in CWL were observed within a given role across surgical phases. Results are reported as predicted probability (95% confidence interval [CI]). CWL was significantly higher for anesthesiologists during "preparation and induction" (0.57; 95% CI, 0.42-0.71) and "anastomoses" (0.44; 95% CI, 0.30-0.58) compared to other phases, and the same held for nurses during the "opening" (0.51; 95% CI, 0.37-0.65) and "postoperative" (0.68; 95% CI, 0.42-0.86) phases. Additional significant differences were observed between roles within a given surgical phase. For example, surgeons had significantly higher CWL during "anastomoses" (0.81; 95% CI, 0.69-0.89) compared to all other phases, and the same was true of perfusionists during the "opening" (0.79; 95% CI, 0.66-0.88) and "prebypass preparation" (0.50; 95% CI, 0.36-0.64) phases. CONCLUSIONS: Our innovative analysis demonstrates that CWL fluctuates across surgical procedures by role and phase, which may reflect the distribution of primary tasks. This corroborates earlier findings from self-report measures. The data suggest that team-wide, peak CWL during a phase decreases from early phases of surgery through initiation of cardiopumonary bypass (CPB), rises during anastomosis, and decreases after termination of CPB. Knowledge of these trends could encourage the adoption of behaviors to enhance team dynamics and performance.

A Coding Framework for Usability Evaluation of Digital Health Technologies.

Several studies have reported low adherence and high resistance from clinicians to adopt digital health technologies into clinical practice, particularly the use of computer-based clinical decision support systems. Poor usability and lack of integration with the clinical workflow have been identified as primary issues. Few guidelines exist on how to analyze the collected data associated with the usability of digital health technologies. In this study, we aimed to develop a coding framework for the systematic evaluation of users' feedback generated during focus groups and interview sessions with clinicians, underpinned by fundamental usability principles and design components. This codebook also included a coding category to capture the user's clinical role associated with each specific piece of feedback, providing a better understanding of role-specific challenges and perspectives, as well as the level of shared understanding across the multiple clinical roles. Furthermore, a voting system was created to quantitatively inform modifications of the digital system based on usability data. As a use case, we applied this method to an electronic cognitive aid designed to improve coordination and communication in the cardiac operating room, showing that this framework is feasible and useful not only to better understand suboptimal usability aspects, but also to recommend relevant modifications in the design and development of the system from different perspectives, including clinical, technical, and usability teams. The framework described herein may be applied in other highly complex clinical settings, in which digital health systems may play an important role in improving patient care and enhancing patient safety.

Dissecting Cardiac Surgery: A Video-based Recall Protocol to Elucidate Team Cognitive Processes in the Operating Room.

OBJECTIVE: The aim of this study was to elucidate the cognitive processes involved in surgical procedures from the perspective of different team roles (surgeon, anesthesiologist, and perfusionist) and provide a comprehensive compilation of intraoperative cognitive processes. SUMMARY BACKGROUND DATA: Nontechnical skills play a crucial role in surgical team performance and understanding the cognitive processes underlying the intraoperative phase of surgery is essential to improve patient safety in the operating room (OR). METHODS: A mixed-methods approach encompassing semistructured interviews with 9 subject-matter experts. A cognitive task analysis was built upon a hierarchical segmentation of coronary artery bypass grafting procedures and a cued-recall protocol using video vignettes was used. RESULTS: A total of 137 unique surgical cognitive processes were identified, including 33 decision points, 23 critical communications, 43 pitfalls, and 38 strategies. Self-report cognitive workload varied substantially, depending on team role and surgical step. A web-based dashboard was developed, providing an integrated visualization of team cognitive processes in the OR that allows readers to intuitively interact with the study findings. CONCLUSIONS: This study advances the current body of knowledge by making explicit relevant cognitive processes involved during the intraoperative phase of cardiac surgery from the perspective of multiple OR team members. By displaying the research findings in an interactive dashboard, we provide trainees with new knowledge in an innovative fashion that could be used to enhance learning outcomes. In addition, the approach used in the present study can be used to deeply understand the cognitive factors underlying surgical adverse events and errors in the OR.

Digital Cognitive Aids to Support Adaptation of Surgical Processes to COVID-19 Protective Policies.

Surgical processes are rapidly being adapted to address the COVID-19 pandemic, with changes in procedures and responsibilities being made to protect both patients and medical teams. These process changes put new cognitive demands on the medical team and increase the likelihood of miscommunication, lapses in judgment, and medical errors. We describe two process model driven cognitive aids, referred to as the Narrative View and the Smart Checklist View, generated automatically from models of the processes. The immediate perceived utility of these cognitive aids is to support medical simulations, particularly when frequent adaptations are needed to quickly respond to changing operating room guidelines.

Process-Model-Driven Guidance to Reduce Surgical Procedure Errors: An Expert Opinion.

This paper explains how a detailed, precise surgical process model can help reduce errors by fostering better understanding, providing guidance during surgery, helping train newcomers, and by supporting process improvement. It describes the features that a process-modeling language should have in order to support the precise specification of such models.

Development of an Interactive Dashboard to Analyze Cognitive Workload of Surgical Teams During Complex Procedural Care.

In the surgical setting, team members constantly deal with a high-demand operative environment that requires simultaneously processing a large amount of information. In certain situations, high demands imposed by surgical tasks and other sources may exceed team member's cognitive capacity, leading to cognitive overload which may place patient safety at risk. In the present study, we describe a novel approach to integrate an objective measure of team member's cognitive load with procedural, behavioral and contextual data from real-life cardiac surgeries. We used heart rate variability analysis, capturing data simultaneously from multiple team members (surgeon, anesthesiologist and perfusionist) in a real-time and unobtrusive manner. Using audio-video recordings, behavioral coding and a hierarchical surgical process model, we integrated multiple data sources to create an interactive surgical dashboard, enabling the analysis of the cognitive load imposed by specific steps, substeps and/or tasks. The described approach enables us to detect cognitive load fluctuations over time, under specific conditions (e.g. emergencies, teaching) and in situations that are prone to errors. This in-depth understanding of the relationship between cognitive load, task demands and error occurrence is essential for the development of cognitive support systems to recognize and mitigate errors during complex surgical care in the operating room.

Intelligent Interruption Management System to Enhance Safety and Performance in Complex Surgical and Robotic Procedures.

Procedural flow disruptions secondary to interruptions play a key role in error occurrence during complex medical procedures, mainly because they increase mental workload among team members, negatively impacting team performance and patient safety. Since certain types of interruptions are unavoidable, and consequently the need for multitasking is inherent to complex procedural care, this field can benefit from an intelligent system capable of identifying in which moment flow interference is appropriate without generating disruptions. In the present study we describe a novel approach for the identification of tasks imposing low cognitive load and tasks that demand high cognitive effort during real-life cardiac surgeries. We used heart rate variability analysis as an objective measure of cognitive load, capturing data in a real-time and unobtrusive manner from multiple team members (surgeon, anesthesiologist and perfusionist) simultaneously. Using audio-video recordings, behavioral coding and a hierarchical surgical process model, we integrated multiple data sources to create an interactive surgical dashboard, enabling the identification of specific steps, substeps and tasks that impose low cognitive load. An interruption management system can use these low demand situations to guide the surgical team in terms of the appropriateness of flow interruptions. The described approach also enables us to detect cognitive load fluctuations over time, under specific conditions (e.g. emergencies) or in situations that are prone to errors. An in-depth understanding of the relationship between cognitive overload states, task demands, and error occurrence will drive the development of cognitive supporting systems that recognize and mitigate errors efficiently and proactively during high complex procedures.

Toward Improving Surgical Outcomes by Incorporating Cognitive Load Measurement into Process-Driven Guidance.

This paper summarizes the accomplishments and recent directions of our medical safety project. Our process-based approach uses a detailed, rigorously-defined, and carefully validated process model to provide a dynamically updated, context-aware and thus, "Smart" Checklist to help process performers understand and manage their pending tasks [7]. This paper focuses on support for teams of performers, working independently as well as in close collaboration, in stressful situations that are life critical. Our recent work has three main thrusts: provide effective real-time guidance for closely collaborating teams; develop and evaluate techniques for measuring cognitive load based on biometric observations and human surveys; and, using these measurements plus analysis and discrete event process simulation, predict cognitive load throughout the process model and propose process modifications to help performers better manage high cognitive load situations. This project is a collaboration among software engineers, surgical team members, human factors researchers, and medical equipment instrumentation experts. Experimental prototype capabilities are being built and evaluated based upon process models of two cardiovascular surgery processes, Aortic Valve Replacement (AVR) and Coronary Artery Bypass Grafting (CABG). In this paper we describe our approach for each of the three research thrusts by illustrating our work for heparinization, a common subprocess of both AVR and CABG. Heparinization is a high-risk error-prone procedure that involves complex team interactions and thus highlights the importance of this work for improving patient outcomes.

Process Driven Guidance for Complex Surgical Procedures.

Surgical team processes are known to be complex and error prone. This paper describes an approach that uses a detailed, validated model of a medical process to provide the clinicians who carry out that complex process with offline and online guidance to help reduce errors. Offline guidance is in the form of a hypertext document describing all the ways the process can be carried out. Online guidance is in the form of a context-sensitive and continually updated electronic "checklist" that lists next steps and needed resources, as well as completed steps. In earlier work, we focused on providing such guidance for single-clinician or single-team processes. This paper describes guiding the collaboration of multiple teams of clinicians through complex processes with significant concurrency, complicated exception handling, and precise and timely communication. We illustrate this approach by applying it to a highly complex, high risk subprocess of cardiac surgery.

Cognitive Support During High-Consequence Episodes of Care in Cardiovascular Surgery.

Despite significant efforts to reduce preventable adverse events in medical processes, such events continue to occur at unacceptable rates. This paper describes a computer science approach that uses formal process modeling to provide situationally aware monitoring and management support to medical professionals performing complex processes. These process models represent both normative and non-normative situations, and are validated by rigorous automated techniques such as model checking and fault tree analysis, in addition to careful review by experts. Context-aware Smart Checklists are then generated from the models, providing cognitive support during high-consequence surgical episodes. The approach is illustrated with a case study in cardiovascular surgery.

Assessing the effectiveness of five process elicitation methods: A case study of chemotherapy treatment plan review.

To reduce the probability of failures and to improve outcomes of safety-critical human-intensive processes, such as health care processes, it is important to be able to rigorously analyze such processes. The quality of that analysis often depends on having an accurate, detailed, and sufficiently complete understanding of the process being analyzed, where this understanding is typically represented as a formal process model that could then drive various rigorous analysis approaches. Developing this understanding and the corresponding formal process model may be difficult and, thus, a variety of process elicitation methods are often used. The work presented in this paper evaluates the effectiveness of five common elicitation methods in terms of their ability to elicit detailed process information necessary to support rigorous process analysis. These methods are employed to elicit typical steps and steps for responding to exceptional situations in a safety-critical health care process, the chemotherapy treatment plan review process. The results indicate strengths and weaknesses of each of the elicitation methods and suggest that it is preferable to apply multiple elicitation methods.

The Grad Cohort Workshop: Evaluating an Intervention to Retain Women Graduate Students in Computing.

Women engaged in computing career tracks are vastly outnumbered by men and often must contend with negative stereotypes about their innate technical aptitude. Research suggests women's marginalized presence in computing may result in women psychologically disengaging, and ultimately dropping out, perpetuating women's underrepresentation in computing. To combat this vicious cycle, the Computing Research Association's Committee on the Status of Women in Computing Research (CRA-W) runs a multi-day mentorship workshop for women graduate students called Grad Cohort, which consists of a speaker series and networking opportunities. We studied the long-term impact of Grad Cohort on women Ph.D. students' (a) dedication to becoming well-known in one's field, and giving back to the community (professional goals), (b) the degree to which one feels computing is an important element of "who they are" (computing identity), and (c) beliefs that computing skills are innate (entity beliefs). Of note, entity beliefs are known to be demoralizing and can lead to disengagement from academic endeavors. We compared a propensity score matched sample of women and men Ph.D. students in computing programs who had never participated in Grad Cohort to a sample of past Grad Cohort participants. Grad Cohort participants reported interest in becoming well-known in their field to a greater degree than women non-participants, and to an equivalent degree as men. Also, Grad Cohort participants reported stronger interest in giving back to the community than their peers. Further, whereas women non-participants identified with computing to a lesser degree than men and held stronger entity beliefs than men, Grad Cohort participants' computing identity and entity beliefs were equivalent to men. Importantly, stronger entity beliefs predicted a weaker computing identity among students, with the exception of Grad Cohort participants. This latter finding suggests Grad Cohort may shield students' computing identity from the damaging nature of entity beliefs. Together, these findings suggest Grad Cohort may fortify women's commitment to pursuing computing research careers and move the needle toward greater gender diversity in computing.

Online deviation detection for medical processes.

Human errors are a major concern in many medical processes. To help address this problem, we are investigating an approach for automatically detecting when performers of a medical process deviate from the acceptable ways of performing that process as specified by a detailed process model. Such deviations could represent errors and, thus, detecting and reporting deviations as they occur could help catch errors before harm is done. In this paper, we identify important issues related to the feasibility of the proposed approach and empirically evaluate the approach for two medical procedures, chemotherapy and blood transfusion. For the evaluation, we use the process models to generate sample process executions that we then seed with synthetic errors. The process models describe the coordination of activities of different process performers in normal, as well as in exceptional situations. The evaluation results suggest that the proposed approach could be applied in clinical settings to help catch errors before harm is done.

Using process elicitation and validation to understand and improve chemotherapy ordering and delivery.

BACKGROUND: Chemotherapy ordering and administration, in which errors have potentially severe consequences, was quantitatively and qualitatively evaluated by employing process formalism (or formal process definition), a technique derived from software engineering, to elicit and rigorously describe the process, after which validation techniques were applied to confirm the accuracy of the described process. METHODS: The chemotherapy ordering and administration process, including exceptional situations and individuals' recognition of and responses to those situations, was elicited through informal, unstructured interviews with members of an interdisciplinary team. The process description (or process definition), written in a notation developed for software quality assessment purposes, guided process validation (which consisted of direct observations and semistructured interviews to confirm the elicited details for the treatment plan portion of the process). RESULTS: The overall process definition yielded 467 steps; 207 steps (44%) were dedicated to handling 59 exceptional situations. Validation yielded 82 unique process events (35 new expected but not yet described steps, 16 new exceptional situations, and 31 new steps in response to exceptional situations). Process participants actively altered the process as ambiguities and conflicts were discovered by the elicitation and validation components of the study. Chemotherapy error rates declined significantly during and after the project, which was conducted from October 2007 through August 2008. DISCUSSION: Each elicitation method and the subsequent validation discussions contributed uniquely to understanding the chemotherapy treatment plan review process, supporting rapid adoption of changes, improved communication regarding the process, and ensuing error reduction.

Designing property specifications to improve the safety of the blood transfusion process.

Computer scientists use a number of well-established techniques that have the potential to improve the safety of patient care processes. One is the formal definition of a process; the other is the formal definition of the properties of a process. Even highly regulated processes, such as laboratory specimen acquisition and transfusion therapy, use guidelines that may be vague, misunderstood, and hence erratically implemented. Examining processes in a systematic way has led us to appreciate the potential variability in routine health care practice and the impact of this variability on patient safety in the clinical setting. The purpose of this article is to discuss the use of innovative computer science techniques as a means of formally defining and specifying certain desirable goals of common, high-risk, patient care processes. Our focus is on describing the specification of process properties, that is, the high-level goals of a process that ultimately dictate why a process should be performed in a given manner.

Increasing patient safety and efficiency in transfusion therapy using formal process definitions.

The administration of blood products is a common, resource-intensive, and potentially problem-prone area that may place patients at elevated risk in the clinical setting. Much of the emphasis in transfusion safety has been targeted toward quality control measures in laboratory settings where blood products are prepared for administration as well as in automation of certain laboratory processes. In contrast, the process of transfusing blood in the clinical setting (ie, at the point of care) has essentially remained unchanged over the past several decades. Many of the currently available methods for improving the quality and safety of blood transfusions in the clinical setting rely on informal process descriptions, such as flow charts and medical algorithms, to describe medical processes. These informal descriptions, although useful in presenting an overview of standard processes, can be ambiguous or incomplete. For example, they often describe only the standard process and leave out how to handle possible failures or exceptions. One alternative to these informal descriptions is to use formal process definitions, which can serve as the basis for a variety of analyses because these formal definitions offer precision in the representation of all possible ways that a process can be carried out in both standard and exceptional situations. Formal process definitions have not previously been used to describe and improve medical processes. The use of such formal definitions to prospectively identify potential error and improve the transfusion process has not previously been reported. The purpose of this article is to introduce the concept of formally defining processes and to describe how formal definitions of blood transfusion processes can be used to detect and correct transfusion process errors in ways not currently possible using existing quality improvement methods.