Nontechnical Skills Assessment in Acute Care Trauma Simulations: A Mixed Methods Approach Using Eye Tracking and Behavioral Marker Systems.

INTRODUCTION: The condition of trauma patients and the urgent need for timely resuscitation present unique challenges to trauma teams. These difficulties are exacerbated for military trauma teams in combat environments. Consequently, there is a need for continued improvement of nontechnical skills (NTS) training for trauma teams. However, current approaches to NTS assessment rely on subjective ratings, which can introduce bias. Accordingly, objective methods of NTS evaluation are needed. Eye-tracking (ET) methods have been applied to studying communication, situation awareness, and leadership in many health care settings, and could be applied to studying physicians' NTS during trauma situations. In this study, we aimed to assess the relationship between trauma team leaders' objective gaze patterns and subjective expert NTS ratings during patient care simulations. MATERIALS AND METHODS: After Institutional Review Board approval, 9 trauma teams from first-year post-graduate general surgery and emergency medicine residents were recruited to participate in 1 of 2 trauma simulations (a difficult airway case and a multi-patient trauma). Each scenario lasted approximately 15 minutes. All team leaders wore a mobile ET system to evaluate gaze metrics-time to first fixation (TTFF), average fixation duration (AFD), and total percentage of the scenario (TPS) focused on Areas of Interest (AOI), which included patient, care team, diagnostic equipment, and patient care equipment. Trained faculty raters completed the Non-Technical Skills for Surgeons (NOTSS) assessment tool and the Trauma Non-Technical Skills (T-NOTECHS) scale. One-way analysis of variance, Kruskal-Wallis, and appropriate post-hoc pairwise comparison tests were run to assess differences between ET metrics across AOI groups. Spearman's Rho tests were used to assess correlations between ET and subjective NTS ratings. RESULTS: Compared to other NTS domains, trauma teams scored relatively poorly on communication across both T-NOTECHS (3.29$ \pm $0.61, maximum = 5) and NOTSS (2.87$ \pm $0.66, maximum = 4). We found significant differences in trauma team leaders' TTFF between teammates and the patient (Team: 1.56 vs Patient: 29.82 seconds, P < .001). TTFF on the diagnostic equipment was negatively correlated (P < .05) to multiple measures of subjective NTS assessments. There were no significant differences in AFD between AOIs, and AFD on teammates was positively correlated (P < .05) to communication and teamwork. There were significant differences in TPS across most AOI pairs (P < .05), and the average TPS fixated was highest on the patient (32%). Finally, there were several significant correlations between additional ET and NTS metrics. CONCLUSIONS: This study utilized a mixed methods approach to assess trauma team leaders' NTS in simulated acute care trauma simulations. Our results provide several objective insights into trauma team leaders' NTS behaviors during patient care simulations. Such objective insights provide a more comprehensive understanding of NTS behaviors and can be leveraged to guide NTS training of trauma physicians in the future. More studies are needed to apply these methods to capture NTS from a larger sample of teams in both simulated and real trauma environments.

Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills.

Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.

Development of a Hand Motion-based Assessment System for Endotracheal Intubation Training.

Endotracheal intubation (ETI) is a procedure to manage and secure an unconscious patient's airway. It is one of the most critical skills in emergency or intensive care. Regular training and practice are required for medical providers to maintain proficiency. Currently, ETI training is assessed by human supervisors who may make inconsistent assessments. This study aims at developing an automated assessment system that analyzes ETI skills and classifies a trainee into an experienced or a novice immediately after training. To make the system more available and affordable, we investigate the feasibility of utilizing only hand motion features as determining factors of ETI proficiency. To this end, we extract 18 features from hand motion in time and frequency domains, and also 12 force features for comparison. Subsequently, feature selection algorithms are applied to identify an ideal feature set for developing classification models. Experimental results show that an artificial neural network (ANN) classifier with five hand motion features selected by a correlation-based algorithm achieves the highest accuracy of 91.17% while an ANN with five force features has only 80.06%. This study corroborates that a simple assessment system based on a small number of hand motion features can be effective in assisting ETI training.

Preliminary Experience With Inertial Movement Technology to Characterize Endotracheal Intubation Kinematics.

BACKGROUND: Endotracheal intubation (ETI) is an important emergency intervention. Only limited data describe ETI skill acquisition and often use bulky technology, not easily transitioned to the clinical setting. In this study, we used small, portable inertial detection technology to characterize intubation kinematic differences between experienced and novice intubators. METHODS: We performed a prospective study including novice (<10 prior clinical ETI) and experienced (>100 clinical ETI) emergency providers. We tracked upper extremity motion with roll, pitch, and yaw using inertial measurement units (IMU) placed on the bilateral hands and wrists of the intubator. Subject performed 6 simulated emergency intubations on a mannequin. Using machine learning algorithms, we determined the motions that best discriminated experienced and novice providers. RESULTS: We included data on 12 novice and 5 experienced providers. Four machine learning algorithms (artificial neural network, support vector machine, decision tree, and K-nearest neighbor search) were applied. Artificial neural network had the greatest accuracy (95% confidence interval) for discriminating between novice and experienced providers (91.17%, 90.8%-91.5%) and was the most parsimonious of the tested algorithms. Using artificial neural network, information from 5 movement features (right hand, roll amplitude; right hand, pitch amplitude; right hand, yaw standard deviation; left hand, yaw standard deviation; left hand, pitch frequency of peak amplitude) was able discriminated experienced from novice providers. CONCLUSIONS: Novice and experienced providers have different ETI movement patterns and can be distinguished by 5 specific movements. Inertial detection technology can be used to characterize the kinematics of emergency airway management.