Granular estimation of user cognitive workload using multi-modal physiological sensors.

Mental workload (MWL) is a crucial area of study due to its significant influence on task performance and potential for significant operator error. However, measuring MWL presents challenges, as it is a multi-dimensional construct. Previous research on MWL models has focused on differentiating between two to three levels. Nonetheless, tasks can vary widely in their complexity, and little is known about how subtle variations in task difficulty influence workload indicators. To address this, we conducted an experiment inducing MWL in up to 5 levels, hypothesizing that our multi-modal metrics would be able to distinguish between each MWL stage. We measured the induced workload using task performance, subjective assessment, and physiological metrics. Our simulated task was designed to induce diverse MWL degrees, including five different math and three different verbal tiers. Our findings indicate that all investigated metrics successfully differentiated between various MWL levels induced by different tiers of math problems. Notably, performance metrics emerged as the most effective assessment, being the only metric capable of distinguishing all the levels. Some limitations were observed in the granularity of subjective and physiological metrics. Specifically, the subjective overall mental workload couldn't distinguish lower levels of workload, while all physiological metrics could detect a shift from lower to higher levels, but did not distinguish between workload tiers at the higher or lower ends of the scale (e.g., between the easy and the easy-medium tiers). Despite these limitations, each pair of levels was effectively differentiated by one or more metrics. This suggests a promising avenue for future research, exploring the integration or combination of multiple metrics. The findings suggest that subtle differences in workload levels may be distinguishable using combinations of subjective and physiological metrics.

Evaluation of a passive arm-support exoskeleton for surgical team members: Results from live surgeries.

BACKGROUND: Musculoskeletal symptoms and injuries adversely impact the health of surgical team members and their performance in the operating room (OR). Though ergonomic risks in surgery are well-recognized, mitigating these risks is especially difficult. In this study, we aimed to assess the impacts of an exoskeleton when used by OR team members during live surgeries. METHODS: A commercial passive arm-support exoskeleton was used. One surgical nurse, one attending surgeon, and five surgical trainees participated. Twenty-seven surgeries were completed, 12 with and 15 without the exoskeleton. Upper-body postures and muscle activation levels were measured during the surgeries using inertial measurement units and electromyography sensors, respectively. Postures, muscle activation levels, and self-report metrics were compared between the baseline and exoskeleton conditions using non-parametric tests. RESULTS: Using the exoskeleton significantly decreased the percentage of time in demanding postures (>45° shoulder elevation) for the right shoulder by 7% and decreased peak muscle activation of the left trapezius, right deltoid, and right lumbar erector spinae muscles, by 7%, 8%, and 12%, respectively. No differences were found in perceived effort, and overall scores on usability ranged from "OK" to "excellent." CONCLUSIONS: Arm-support exoskeletons have the potential to assist OR team members in reducing musculoskeletal pain and fatigue indicators. To further increase usability in the OR, however, better methods are needed to identify the surgical tasks for which an exoskeleton is effective.

Eye tracking and audio sensors to evaluate surgeon's non-technical skills: An empirical study.

Non-Technical Skills (NTS) of medical teams are currently measured using subjective and resource-intensive ratings given by experts. This study explores if objective NTS assessment approaches with eye-tracking and audio sensors can measure teamwork and communication skills in surgery. Eight surgeons participated in a simulated two-phase surgical scenario developed to assess their NTS. Sensor-based audio, eye tracking and video data were collected and analyzed along with rating from the NOTSS scale. Different levels of communication were detected by the sensor data during the two phases of the simulated surgery. Sensor data detected leadership qualities among surgeons based on speech metrics, and eye tracking offered additional evidence about gaze patterns related to NTS. This objective approach to NTS measurement captured differences in communication in greater detail as opposed to a single collective rating obtained using current assessment tools.

Nontechnical Skills Assessment in Acute Care Trauma Simulations: A Mixed Methods Approach Using Eye Tracking and Behavioral Marker Systems.

INTRODUCTION: The condition of trauma patients and the urgent need for timely resuscitation present unique challenges to trauma teams. These difficulties are exacerbated for military trauma teams in combat environments. Consequently, there is a need for continued improvement of nontechnical skills (NTS) training for trauma teams. However, current approaches to NTS assessment rely on subjective ratings, which can introduce bias. Accordingly, objective methods of NTS evaluation are needed. Eye-tracking (ET) methods have been applied to studying communication, situation awareness, and leadership in many health care settings, and could be applied to studying physicians' NTS during trauma situations. In this study, we aimed to assess the relationship between trauma team leaders' objective gaze patterns and subjective expert NTS ratings during patient care simulations. MATERIALS AND METHODS: After Institutional Review Board approval, 9 trauma teams from first-year post-graduate general surgery and emergency medicine residents were recruited to participate in 1 of 2 trauma simulations (a difficult airway case and a multi-patient trauma). Each scenario lasted approximately 15 minutes. All team leaders wore a mobile ET system to evaluate gaze metrics-time to first fixation (TTFF), average fixation duration (AFD), and total percentage of the scenario (TPS) focused on Areas of Interest (AOI), which included patient, care team, diagnostic equipment, and patient care equipment. Trained faculty raters completed the Non-Technical Skills for Surgeons (NOTSS) assessment tool and the Trauma Non-Technical Skills (T-NOTECHS) scale. One-way analysis of variance, Kruskal-Wallis, and appropriate post-hoc pairwise comparison tests were run to assess differences between ET metrics across AOI groups. Spearman's Rho tests were used to assess correlations between ET and subjective NTS ratings. RESULTS: Compared to other NTS domains, trauma teams scored relatively poorly on communication across both T-NOTECHS (3.29$ \pm $0.61, maximum = 5) and NOTSS (2.87$ \pm $0.66, maximum = 4). We found significant differences in trauma team leaders' TTFF between teammates and the patient (Team: 1.56 vs Patient: 29.82 seconds, P < .001). TTFF on the diagnostic equipment was negatively correlated (P < .05) to multiple measures of subjective NTS assessments. There were no significant differences in AFD between AOIs, and AFD on teammates was positively correlated (P < .05) to communication and teamwork. There were significant differences in TPS across most AOI pairs (P < .05), and the average TPS fixated was highest on the patient (32%). Finally, there were several significant correlations between additional ET and NTS metrics. CONCLUSIONS: This study utilized a mixed methods approach to assess trauma team leaders' NTS in simulated acute care trauma simulations. Our results provide several objective insights into trauma team leaders' NTS behaviors during patient care simulations. Such objective insights provide a more comprehensive understanding of NTS behaviors and can be leveraged to guide NTS training of trauma physicians in the future. More studies are needed to apply these methods to capture NTS from a larger sample of teams in both simulated and real trauma environments.

Deep learning enables accurate soft tissue tendon deformation estimation in vivo via ultrasound imaging.

Image-based deformation estimation is an important tool used in a variety of engineering problems, including crack propagation, fracture, and fatigue failure. These tools have been important in biomechanics research where measuring in vitro and in vivo tissue deformations are important for evaluating tissue health and disease progression. However, accurately measuring tissue deformation in vivo is particularly challenging due to limited image signal-to-noise ratio. Therefore, we created a novel deep-learning approach for measuring deformation from a sequence of images collected in vivo called StrainNet. Utilizing a training dataset that incorporates image artifacts, StrainNet was designed to maximize performance in challenging, in vivo settings. Artificially generated image sequences of human flexor tendons undergoing known deformations were used to compare benchmark StrainNet against two conventional image-based strain measurement techniques. StrainNet outperformed the traditional techniques by nearly 90%. High-frequency ultrasound imaging was then used to acquire images of the flexor tendons engaged during contraction. Only StrainNet was able to track tissue deformations under the in vivo test conditions. Findings revealed strong correlations between tendon deformation and applied forces, highlighting the potential for StrainNet to be a valuable tool for assessing rehabilitation strategies or disease progression. Additionally, by using real-world data to train our model, StrainNet was able to generalize and reveal important relationships between the effort exerted by the participant and tendon mechanics. Overall, StrainNet demonstrated the effectiveness of using deep learning for image-based strain analysis in vivo.

Utilizing Eye Tracking to Assess Medical Student Non-Technical Performance During Scenario-Based Simulation: Results of a Pilot Study.

Background: Non-technical skills (NTS) are essential for safe surgical patient management. However, assessing NTS involves observer-based ratings, which can introduce bias. Eye tracking (ET) has been proposed as an effective method to capture NTS. The purpose of the current study was to determine if ET metrics are associated with NTS performance. Methods: Participants wore a mobile ET system and participated in two patient care simulations, where they managed a deteriorating patient. The scenarios featured several challenges to leadership, which were evaluated using a 4-point Likert scale. NTS were evaluated by trained raters using the Non-Technical Skills for Surgeons (NOTSS) scale. ET metrics included percentage of fixations and visits on areas of interest. Results: Ten medical students participated. Average visit duration on the patient was negatively correlated with participants' communication and leadership. Average visit duration on the patient's intravenous access was negatively correlated with participants' decision making and situation awareness. Conclusions: Our preliminary data suggests that visual attention on the patient was negatively associated with NTS and may indicate poor comprehension of the patient's status due to heightened cognitive load. In future work, researchers and educators should consider using ET to objectively evaluate and provide feedback on their NTS.