Prediction of Robotic Anastomosis Competency Evaluation (RACE) metrics during vesico-urethral anastomosis using electroencephalography, eye-tracking, and machine learning.

Residents learn the vesico-urethral anastomosis (VUA), a key step in robot-assisted radical prostatectomy (RARP), early in their training. VUA assessment and training significantly impact patient outcomes and have high educational value. This study aimed to develop objective prediction models for the Robotic Anastomosis Competency Evaluation (RACE) metrics using electroencephalogram (EEG) and eye-tracking data. Data were recorded from 23 participants performing robot-assisted VUA (henceforth 'anastomosis') on plastic models and animal tissue using the da Vinci surgical robot. EEG and eye-tracking features were extracted, and participants' anastomosis subtask performance was assessed by three raters using the RACE tool and operative videos. Random forest regression (RFR) and gradient boosting regression (GBR) models were developed to predict RACE scores using extracted features, while linear mixed models (LMM) identified associations between features and RACE scores. Overall performance scores significantly differed among inexperienced, competent, and experienced skill levels (P value < 0.0001). For plastic anastomoses, R2 values for predicting unseen test scores were: needle positioning (0.79), needle entry (0.74), needle driving and tissue trauma (0.80), suture placement (0.75), and tissue approximation (0.70). For tissue anastomoses, the values were 0.62, 0.76, 0.65, 0.68, and 0.62, respectively. The models could enhance RARP anastomosis training by offering objective performance feedback to trainees.

Mental workload evaluation using weighted phase lag index and coherence features extracted from EEG data.

Electroencephalogram (EEG) represents an effective, non-invasive technology to study mental workload. However, volume conduction, a common EEG artifact, influences functional connectivity analysis of EEG data. EEG coherence has been used traditionally to investigate functional connectivity between brain areas associated with mental workload, while weighted Phase Lag Index (wPLI) is a measure that improves on coherence by reducing susceptibility to volume conduction, a common EEG artifact. The goal of this study was to compare two methods of functional connectivity analysis, wPLI and coherence, in the context of mental workload evaluation. The study involved model development for mental workload domains and comparing their performance using coherence-based features, wPLI-based features, and a combination of both. Generalized linear mixed-effects model (GLMM) with the least absolute shrinkage and selection operator (LASSO) feature selection method was used for model development. Results indicated that the model developed using a combination of both feature types demonstrated improved predictive performance across all mental workload domains, compared to models that used each feature type individually. The R2 values were 0.82 for perceived task complexity, 0.71 for distraction, 0.91 for mental demand, 0.85 for physical demand, 0.74 for situational stress, and 0.74 for temporal demand. Furthermore, task complexity and functional connectivity patterns in different brain areas were identified as significant contributors to perceived mental workload (p-value<0.05). Findings showed the potential of using EEG data for mental workload evaluation which suggests that combination of coherence and wPLI can improve the accuracy of mental workload domains prediction. Future research should aim to validate these results on larger, diverse datasets to confirm their generalizability and refine the predictive models.

Development of performance and learning rate evaluation models in robot-assisted surgery using electroencephalography and eye-tracking.

The existing performance evaluation methods in robot-assisted surgery (RAS) are mainly subjective, costly, and affected by shortcomings such as the inconsistency of results and dependency on the raters' opinions. The aim of this study was to develop models for an objective evaluation of performance and rate of learning RAS skills while practicing surgical simulator tasks. The electroencephalogram (EEG) and eye-tracking data were recorded from 26 subjects while performing Tubes, Suture Sponge, and Dots and Needles tasks. Performance scores were generated by the simulator program. The functional brain networks were extracted using EEG data and coherence analysis. Then these networks, along with community detection analysis, facilitated the extraction of average search information and average temporal flexibility features at 21 Brodmann areas (BA) and four band frequencies. Twelve eye-tracking features were extracted and used to develop linear random intercept models for performance evaluation and multivariate linear regression models for the evaluation of the learning rate. Results showed that subject-wise standardization of features improved the R2 of the models. Average pupil diameter and rate of saccade were associated with performance in the Tubes task (multivariate analysis; p-value = 0.01 and p-value = 0.04, respectively). Entropy of pupil diameter was associated with performance in Dots and Needles task (multivariate analysis; p-value = 0.01). Average temporal flexibility and search information in several BAs and band frequencies were associated with performance and rate of learning. The models may be used to objectify performance and learning rate evaluation in RAS once validated with a broader sample size and tasks.

Neurocognitive Correlates of Clinical Decision Making: A Pilot Study Using Electroencephalography.

The development of sound clinical reasoning, while essential for optimal patient care, can be quite an elusive process. Researchers typically rely on a self-report or observational measures to study decision making, but clinicians' reasoning processes may not be apparent to themselves or outside observers. This study explored electroencephalography (EEG) to examine neurocognitive correlates of clinical decision making during a simulated American Board of Anesthesiology-style standardized oral exam. Eight novice anesthesiology residents and eight fellows who had recently passed their board exams were included in the study. Measures included EEG recordings from each participant, demographic information, self-reported cognitive load, and observed performance. To examine neurocognitive correlates of clinical decision making, power spectral density (PSD) and functional connectivity between pairs of EEG channels were analyzed. Although both groups reported similar cognitive load (p = 0.840), fellows outperformed novices based on performance scores (p < 0.001). PSD showed no significant differences between the groups. Several coherence features showed significant differences between fellows and residents, mostly related to the channels within the frontal, between the frontal and parietal, and between the frontal and temporal areas. The functional connectivity patterns found in this study could provide some clues for future hypothesis-driven studies in examining the underlying cognitive processes that lead to better clinical reasoning.

Surgical skill level classification model development using EEG and eye-gaze data and machine learning algorithms.

The aim of this study was to develop machine learning classification models using electroencephalogram (EEG) and eye-gaze features to predict the level of surgical expertise in robot-assisted surgery (RAS). EEG and eye-gaze data were recorded from 11 participants who performed cystectomy, hysterectomy, and nephrectomy using the da Vinci robot. Skill level was evaluated by an expert RAS surgeon using the modified Global Evaluative Assessment of Robotic Skills (GEARS) tool, and data from three subtasks were extracted to classify skill levels using three classification models-multinomial logistic regression (MLR), random forest (RF), and gradient boosting (GB). The GB algorithm was used with a combination of EEG and eye-gaze data to classify skill levels, and differences between the models were tested using two-sample t tests. The GB model using EEG features showed the best performance for blunt dissection (83% accuracy), retraction (85% accuracy), and burn dissection (81% accuracy). The combination of EEG and eye-gaze features using the GB algorithm improved the accuracy of skill level classification to 88% for blunt dissection, 93% for retraction, and 86% for burn dissection. The implementation of objective skill classification models in clinical settings may enhance the RAS surgical training process by providing objective feedback about performance to surgeons and their teachers.

Developing cognitive workload and performance evaluation models using functional brain network analysis.

Cognition, defined as the ability to learn, remember, sustain attention, make decisions, and solve problems, is essential in daily activities and in learning new skills. The purpose of this study was to develop cognitive workload and performance evaluation models using features that were extracted from Electroencephalogram (EEG) data through functional brain network and spectral analyses. The EEG data were recorded from 124 brain areas of 26 healthy participants conducting two cognitive tasks on a robot simulator. The functional brain network and Power Spectral Density features were extracted from EEG data using coherence and spectral analyses, respectively. Participants reported their perceived cognitive workload using the SURG-TLX questionnaire after each exercise, and the simulator generated actual performance scores. The extracted features, actual performance scores, and subjectively assessed cognitive workload values were used to develop linear models for evaluating performance and cognitive workload. Furthermore, the Pearson correlation was used to find the correlation between participants' age, performance, and cognitive workload. The findings demonstrated that combined EEG features retrieved from spectral analysis and functional brain networks can be used to evaluate cognitive workload and performance. The cognitive workload in conducting only Matchboard level 3, which is more challenging than Matchboard level 2, was correlated with age (0.54, p-value = 0.01). This finding may suggest playing more challenging computer games are more helpful in identifying changes in cognitive workload caused by aging. The findings could open the door for a new era of objective evaluation and monitoring of cognitive workload and performance.

Performance and learning rate prediction models development in FLS and RAS surgical tasks using electroencephalogram and eye gaze data and machine learning.

OBJECTIVE: This study explored the use of electroencephalogram (EEG) and eye gaze features, experience-related features, and machine learning to evaluate performance and learning rates in fundamentals of laparoscopic surgery (FLS) and robotic-assisted surgery (RAS). METHODS: EEG and eye-tracking data were collected from 25 participants performing three FLS and 22 participants performing two RAS tasks. Generalized linear mixed models, using L1-penalized estimation, were developed to objectify performance evaluation using EEG and eye gaze features, and linear models were developed to objectify learning rate evaluation using these features and performance scores at the first attempt. Experience metrics were added to evaluate their role in learning robotic surgery. The differences in performance across experience levels were tested using analysis of variance. RESULTS: EEG and eye gaze features and experience-related features were important for evaluating performance in FLS and RAS tasks with reasonable results. Residents outperformed faculty in FLS peg transfer (p value = 0.04), while faculty and residents both excelled over pre-medical students in the FLS pattern cut (p value = 0.01 and p value < 0.001, respectively). Fellows outperformed pre-medical students in FLS suturing (p value = 0.01). In RAS tasks, both faculty and fellows surpassed pre-medical students (p values for the RAS pattern cut were 0.001 for faculty and 0.003 for fellows, while for RAS tissue dissection, the p value was less than 0.001 for both groups), with residents also showing superior skills in tissue dissection (p value = 0.03). CONCLUSION: Findings could be used to develop training interventions for improving surgical skills and have implications for understanding motor learning and designing interventions to enhance learning outcomes.

Developing surgical skill level classification model using visual metrics and a gradient boosting algorithm.

Objective: Assessment of surgical skills is crucial for improving training standards and ensuring the quality of primary care. This study aimed to develop a gradient boosting classification model (GBM) to classify surgical expertise into inexperienced, competent, and experienced levels in robot-assisted surgery (RAS) using visual metrics. Methods: Eye gaze data were recorded from 11 participants performing four subtasks; blunt dissection, retraction, cold dissection, and hot dissection using live pigs and the da Vinci robot. Eye gaze data were used to extract the visual metrics. One expert RAS surgeon evaluated each participant's performance and expertise level using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The extracted visual metrics were used to classify surgical skill levels and to evaluate individual GEARS metrics. Analysis of Variance (ANOVA) was used to test the differences for each feature across skill levels. Results: Classification accuracies for blunt dissection, retraction, cold dissection, and burn dissection were 95%, 96%, 96%, and 96%, respectively. The time to complete only the retraction was significantly different among the 3 skill levels (p-value = 0.04). Performance was significantly different for 3 categories of surgical skill level for all subtasks (p-values<0.01). The extracted visual metrics were strongly associated with GEARS metrics (R2>0.7 for GEARS metrics evaluation models). Conclusions: Machine learning (ML) algorithms trained by visual metrics of RAS surgeons can classify surgical skill levels and evaluate GEARS measures. The time to complete a surgical subtask may not be considered a stand-alone factor for skill level assessment.

Utilizing deep neural networks and electroencephalogram for objective evaluation of surgeon's distraction during robot-assisted surgery.

OBJECTIVE: To develop an algorithm for objective evaluation of distraction of surgeons during robot-assisted surgery (RAS). MATERIALS AND METHODS: Electroencephalogram (EEG) of 22 medical students was recorded while performing five key tasks on the robotic surgical simulator: Instrument Control, Ball Placement, Spatial Control II, Fourth Arm Tissue Retraction, and Hands-on Surgical Training Tasks. All students completed the Surgery Task Load Index (SURG-TLX), which includes one domain for subjective assessment of distraction (scale: 1-20). Scores were divided into low (score 1-6, subjective label: 1), intermediate (score 7-12, subjective label: 2), and high distraction (score 13-20, subjective label: 3). These cut-off values were arbitrarily considered based on a verbal assessment of participants and experienced surgeons. A Deep Convolutional Neural Network (CNN) algorithm was trained utilizing EEG recordings from the medical students and used to classify their distraction levels. The accuracy of our method was determined by comparing the subjective distraction scores on SURG-TLX and the results from the proposed classification algorithm. Also, Pearson correlation was utilized to assess the relationship between performance scores (generated by the simulator) and distraction (Subjective assessment scores). RESULTS: The proposed end-to-end model classified distraction into low, intermediate, and high with 94%, 89%, and 95% accuracy, respectively. We found a significant negative correlation (r = -0.21; p = 0.003) between performance and SURG-TLX distraction scores. CONCLUSIONS: Herein we report, to our knowledge, the first objective method to assess and quantify distraction while performing robotic surgical tasks on the robotic simulator, which may improve patient safety. Validation in the clinical setting is required.

Surgical Hand Gesture Recognition Utilizing Electroencephalogram as Input to the Machine Learning and Network Neuroscience Algorithms.

Surgical gestures detection can provide targeted, automated surgical skill assessment and feedback during surgical training for robot-assisted surgery (RAS). Several sources including surgical videos, robot tool kinematics, and an electromyogram (EMG) have been proposed to reach this goal. We aimed to extract features from electroencephalogram (EEG) data and use them in machine learning algorithms to classify robot-assisted surgical gestures. EEG was collected from five RAS surgeons with varying experience while performing 34 robot-assisted radical prostatectomies over the course of three years. Eight dominant hand and six non-dominant hand gesture types were extracted and synchronized with associated EEG data. Network neuroscience algorithms were utilized to extract functional brain network and power spectral density features. Sixty extracted features were used as input to machine learning algorithms to classify gesture types. The analysis of variance (ANOVA) F-value statistical method was used for feature selection and 10-fold cross-validation was used to validate the proposed method. The proposed feature set used in the extra trees (ET) algorithm classified eight gesture types performed by the dominant hand of five RAS surgeons with an accuracy of 90%, precision: 90%, sensitivity: 88%, and also classified six gesture types performed by the non-dominant hand with an accuracy of 93%, precision: 94%, sensitivity: 94%.

Association between Functional Brain Network Metrics and Surgeon Performance and Distraction in the Operating Room.

OBJECTIVE: The aim of this work was to examine (electroencephalogram) EEG features that represent dynamic changes in the functional brain network of a surgical trainee and whether these features can be used to evaluate a robot assisted surgeon's (RAS) performance and distraction level in the operating room. MATERIALS AND METHODS: Electroencephalogram (EEG) data were collected from three robotic surgeons in an operating room (OR) via a 128-channel EEG headset with a frequency of 500 samples/second. Signal processing and network neuroscience algorithms were applied to the data to extract EEG features. The SURG-TLX and NASA-TLX metrics were subjectively evaluated by a surgeon and mentor at the end of each task. The scores given to performance and distraction metrics were used in the analyses here. Statistical test data were utilized to select EEG features that have a significant relationship with surgeon performance and distraction while carrying out a RAS surgical task in the OR. RESULTS: RAS surgeon performance and distraction had a relationship with the surgeon's functional brain network metrics as recorded throughout OR surgery. We also found a significant negative Pearson correlation between performance and the distraction level (-0.37, p-value < 0.0001). CONCLUSIONS: The method proposed in this study has potential for evaluating RAS surgeon performance and the level of distraction. This has possible applications in improving patient safety, surgical mentorship, and training.

Identifying mental health status using deep neural network trained by visual metrics.

Mental health is an integral part of the quality of life of cancer patients. It has been found that mental health issues, such as depression and anxiety, are more common in cancer patients. They may result in catastrophic consequences, including suicide. Therefore, monitoring mental health metrics (such as hope, anxiety, and mental well-being) is recommended. Currently, there is lack of objective method for mental health evaluation, and most of the available methods are limited to subjective face-to-face discussions between the patient and psychotherapist. In this study we introduced an objective method for mental health evaluation using a combination of convolutional neural network and long short-term memory (CNN-LSTM) algorithms learned and validated by visual metrics time-series. Data were recorded by the TobiiPro eyeglasses from 16 patients with cancer after major oncologic surgery and nine individuals without cancer while viewing18 artworks in an in-house art gallery. Pre-study and post-study questionnaires of Herth Hope Index (HHI; for evaluation of hope), anxiety State-Trait Anxiety Inventory for Adults (STAI; for evaluation of anxiety) and Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS; for evaluation of mental well-being) were completed by participants. Clinical psychotherapy and statistical suggestions for cutoff scores were used to assign an individual's mental health metrics level during each session into low (class 0), intermediate (class 1), and high (class 2) levels. Our proposed model was used to objectify evaluation and categorize HHI, STAI, and WEMWBS status of individuals. Classification accuracy of the model was 93.81%, 94.76%, and 95.00% for HHI, STAI, and WEMWBS metrics, respectively. The proposed model can be integrated into applications for home-based mental health monitoring to be used by patients after oncologic surgery to identify patients at risk.

Dynamic changes of brain functional states during surgical skill acquisition.

There is lack of a standardized measure of technical proficiency and skill acquisition for robot-assisted surgery (RAS). Learning surgical skills, in addition to the interaction with the machine and the new surgical environment adds to the complexity of the learning process. Moreover, evaluation of surgeon performance in operating room is required to optimize patient safety. In this study, we investigated the dynamic changes of RAS trainee's brain functional states by practice. We also developed brain functional state measurements to find the relationship between RAS skill acquisition (especially human-machine interaction skills) and reconfiguration of brain functional states. This relationship may help in providing trainees with helpful, structured feedback regarding skills requiring improvement and will help in tailoring training activities.

Functional Brain States Measure Mentor-Trainee Trust during Robot-Assisted Surgery.

Mutual trust is important in surgical teams, especially in robot-assisted surgery (RAS) where interaction with robot-assisted interface increases the complexity of relationships within the surgical team. However, evaluation of trust between surgeons is challenging and generally based on subjective measures. Mentor-Trainee trust was defined as assessment of mentor on trainee's performance quality and approving trainee's ability to continue performing the surgery. Here, we proposed a novel method of objectively assessing mentor-trainee trust during RAS based on patterns of brain activity of surgical mentor observing trainees. We monitored the EEG activity of a mentor surgeon while he observed procedures performed by surgical trainees and quantified the mentor's brain activity using functional and cognitive brain state features. We used methods from machine learning classification to identity key features that distinguish trustworthiness from concerning performances. Results showed that during simple surgical task, functional brain features are sufficient to classify trust. While, during more complex tasks, the addition of cognitive features could provide additional accuracy, but functional brain state features drive classification performance. These results indicate that functional brain network interactions hold information that may help objective trainee specific mentorship and aid in laying the foundation of automation in the human-robot shared control environment during RAS.

Cognitive learning and its future in urology: surgical skills teaching and assessment.

PURPOSE OF REVIEW: The aim of this study is to provide an overview of the current status of novel cognitive training approaches in surgery and to investigate the potential role of cognitive training in surgical education. RECENT FINDINGS: Kinematics of end-effector trajectories, as well as cognitive state features of surgeon trainees and mentors have recently been studied as modalities to objectively evaluate the expertise level of trainees and to shorten the learning process. Virtual reality and haptics also have shown promising in research results in improving the surgical learning process by providing feedback to the trainee. SUMMARY: 'Cognitive training' is a novel approach to enhance training and surgical performance. The utility of cognitive training in improving motor skills in other fields, including sports and rehabilitation, is promising enough to justify its utilization to improve surgical performance. However, some surgical procedures, especially ones performed during human-robot interaction in robot-assisted surgery, are much more complicated than sport and rehabilitation. Cognitive training has shown promising results in surgical skills-acquisition in complicated environments such as surgery. However, these methods are mostly developed in research groups using limited individuals. Transferring this research into the clinical applications is a demanding challenge. The aim of this review is to provide an overview of the current status of these novel cognitive training approaches in surgery and to investigate the potential role of cognitive training in surgical education.

Technical mentorship during robot-assisted surgery: a cognitive analysis.

OBJECTIVE: To investigate cognitive and mental workload assessments, which may play a critical role in defining successful mentorship. MATERIALS AND METHODS: The 'Mind Maps' project aimed at evaluating cognitive function with regard to surgeon's expertise and trainee's skills. The study included electroencephalogram (EEG) recordings of a mentor observing trainee surgeons in 20 procedures involving extended lymph node dissection (eLND) or urethrovesical anastomosis (UVA), with simultaneous assessment of trainees using the National Aeronautics and Space Administration Task Load index (NASA-TLX) questionnaire. We also compared the brain activity of the mentor during this study with his own brain activity while actually performing the same surgical steps from previous procedures populated in the 'Mind Maps' project. RESULTS: During eLND and UVA, when the mentor thought the trainee's mental demand and effort were low based on his NASA-TLX questionnaire (not satisfied with his performance), his EEG-based mental workload increased (reflecting more concern and attention). The mentor was mentally engaged and concerned while he was engrossed in observing the surgery. This was further supported by the finding that there was no significant difference in the mental demands and workload between observing and operating for the expert surgeon. CONCLUSIONS: This study objectively evaluated the cognitive engagement of a surgical mentor teaching technical skills during surgery. The study provides a deeper understanding of how surgical teaching actually works and opens new horizons for assessment and teaching of surgery. Further research is needed to study the feasibility of this novel concept in assessment and guidance of surgical performance.

Mentor's brain functional connectivity network during robotic assisted surgery mentorship.

In many complicated cognitive-motor tasks mentoring is inevitable during the learning process. Although mentors are expert in doing the task, trainee's operation might be new for a mentor. This makes mentoring a very difficult task which demands not only the knowledge and experience of a mentor, but also his/her ability to follow trainee's movements and patiently advise him/her during the operation. We hypothesize that information binding throughout the mentor's brain areas, contributed to the task, changes as the expertise level of the trainee improves from novice to intermediate and expert. This can result in the change of mentor's level of satisfaction. The brain functional connectivity network is extracted by using brain activity of a mentor during mentoring novice and intermediate surgeons, watching expert surgeon operation, and doing Urethrovesical Anasthomosis (UVA) procedure by himself. By using the extracted network, we investigate the role of modularity and neural activity efficiency in mentoring. Brain activity is measured by using a 24-channel ABM Neuro-headset with the frequency of 256 Hz. One mentor operates 26 UVA procedures and three trainees with the expertise level of novice, intermediate, and expert perform 26 UVA procedures under the supervision of mentor. Our results indicate that the modularity of functional connectivity network is higher when mentor performs the task or watches the expert operation comparing mentoring the novice and intermediate surgeons. At the end of each operation, mentor subjectively assesses the quality of operation by giving scores to NASA-TLX indexes. Performance score is used to discuss our results. The extracted significant positive correlation between performance level and modularity (r = 0.38, p - value <; 0.005) shows the increase of automaticity and decrease in neural activity cost by improving the performance.

Understanding Cognitive Performance During Robot-Assisted Surgery.

OBJECTIVE: To understand cognitive function of an expert surgeon in various surgical scenarios while performing robot-assisted surgery. MATERIALS AND METHODS: In an Internal Review Board approved study, National Aeronautics and Space Administration-Task Load Index (NASA-TLX) questionnaire with surgical field notes were simultaneously completed. A wireless electroencephalography (EEG) headset was used to monitor brain activity during all procedures. Three key portions were evaluated: lysis of adhesions, extended lymph node dissection, and urethro-vesical anastomosis (UVA). Cognitive metrics extracted were distraction, mental workload, and mental state. RESULTS: In evaluating lysis of adhesions, mental state (EEG) was associated with better performance (NASA-TLX). Utilizing more mental resources resulted in better performance as self-reported. Outcomes of lysis were highly dependent on cognitive function and decision-making skills. In evaluating extended lymph node dissection, there was a negative correlation between distraction level (EEG) and mental demand, physical demand and effort (NASA-TLX). Similar to lysis of adhesion, utilizing more mental resources resulted in better performance (NASA-TLX). Lastly, with UVA, workload (EEG) negatively correlated with mental and temporal demand and was associated with better performance (NASA-TLX). The EEG recorded workload as seen here was a combination of both cognitive performance (finding solution) and motor workload (execution). Majority of workload was contributed by motor workload of an expert surgeon. During UVA, muscle memory and motor skills of expert are keys to completing the UVA. CONCLUSION: Cognitive analysis shows that expert surgeons utilized different mental resources based on their need.