An Eye-Fixation Related Electroencephalography Technique for Predicting Situation Awareness: Implications for Driver State Monitoring Systems.

OBJECTIVE: This study developed a fixation-related electroencephalography band power (FRBP) approach for situation awareness (SA) assessment in automated driving. BACKGROUND: Maintaining good SA in Level 3 automated vehicles is crucial to drivers' takeover performance when the automated system fails. A multimodal fusion approach that enables the analysis of the visual behavioral and cognitive processes of SA can facilitate real-time assessment of SA in future driver state monitoring systems. METHOD: Thirty participants performed three simulated automated driving tasks. After each task, the Situation Awareness Global Assessment Technique (SAGAT) was deployed to capture their SA about key elements that could affect their takeover task performance. Participants eye movements and brain activities were recorded. Data on their brain activity after each eye fixation on the key elements were extracted and labeled according to the correctness of the SAGAT. Mixed-effects models were used to identify brain regions that were indicative of SA, and machine learning models for SA assessment were developed based on the identified brain regions. RESULTS: Participants' alpha and theta oscillation at frontal and temporal areas are indicative of SA. In addition, the FRBP technique can be used to predict drivers' SA with an accuracy of 88% using a neural network model. CONCLUSION: The FRBP technique, which incorporates eye movements and brain activities, can provide more comprehensive evaluation of SA. Findings highlight the potential of utilizing FRBP to monitor drivers' SA in real-time. APPLICATION: The proposed framework can be expanded and applied to driver state monitoring systems to measure human SA in real-world driving.

Physiological Measurements of Situation Awareness: A Systematic Review.

OBJECTIVE: The goal of this systematic literature review is to investigate the relationship between indirect physiological measurements and direct measures of situation awareness (SA). BACKGROUND: Across different environments and tasks, assessments of SA are often performed using techniques designed specifically to directly measure SA, such as SAGAT, SPAM, and/or SART. However, research suggests that indirect physiological sensing methods may also be capable of predicting SA. Currently, it is unclear which particular physiological approaches are sensitive to changes in SA. METHOD: Seven databases were searched using the PRISMA reporting guidelines. Eligibility criteria included human-subject experiments that used at least one direct SA assessment technique, as well as at least one physiological measurement. Information extracted from each article was the physiological metric(s), the direct SA measurement(s), the correlation between these two metrics, and the experimental task(s). All studies underwent a quality assessment. RESULTS: Twenty-five articles were included in this review. Eye tracking techniques were the most commonly used physiological measures, and correlations between conscious aspects of eye movement measures and direct SA scores were observed. Evidence for cardiovascular predictors of SA were mixed. EEG studies were too few to form strong conclusions, but were consistently positive. CONCLUSION: Further investigation is needed to methodically collect more relevant data and comprehensively model the relationships between a wider range of physiological measurements and direct assessments of SA. APPLICATION: This review will guide researchers and practitioners in methods to indirectly assess SA with sensors and highlight opportunities for future research on wearables and SA.

Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving.

Automated driving systems are becoming increasingly prevalent throughout society. In conditionally automated vehicles, drivers may engage in non-driving-related tasks (NDRTs), which can negatively affect their situation awareness (SA) and preparedness to resume control of the vehicle, when necessary. Previous work has investigated engagement in NDRTs, but questions remain unanswered regarding its effect on drivers' SA during a takeover event. The objective of the current study is to use eye-tracking to aid in understanding how visual engagement in NDRTs affects changes in SA of the driving environment after a takeover request (TOR) has been issued. Thirty participants rode in a simulated SAE Level 3 automated driving environment and engaged in three separate pre-TOR tasks (Surrogate Reference Task, Monitoring Task, and Peripheral Detection Task) until presented with a TOR. Situation Awareness Global Assessment Technique (SAGAT) scores and gaze behavior were recorded during the post-TOR segment. Overall, longer times spent viewing the driving scene, and more dispersed visual attention allocation, were observed to be associated with better overall SA. Also, location-based eye tracking metrics show most promise in differentiating between task conditions with significantly different SAGAT scores. Findings from this work can inform the development of real-time SA assessment techniques using eye movements and ultimately contribute to improved operator roadway awareness for next-generation automated transportation.

Mirage events & driver haptic steering alerts in a motion-base driving simulator: A method for selecting an optimal HMI.

This paper describes a new method, a 'mirage scenario,' to support formative evaluation of driver alerting or warning displays for manual and automated driving. This method provides driving contexts (e.g., various Times-To-Collision (TTCs) to a lead vehicle) briefly presented and then removed. In the present study, during each mirage event, a haptic steering display was evaluated. This haptic display indicated a steering response may be initiated to drive around an obstacle ahead. A motion-base simulator was used in a 32-participant study to present vehicle motion cues similar to the actual application. Surprise was neither present nor of concern, as it would be for a summative evaluation of a forward collision warning system. Furthermore, no collision avoidance maneuvers were performed, thereby reducing the risk of simulator sickness. This paper illustrates the mirage scenario procedures, the rating methods and definitions used with the mirage scenario, and analysis of the ratings obtained, together with a multi-attribute utility theory (MAUT) approach to evaluate and select among alternative designs for future summative evaluation.