A Novel EEG-Based Assessment of Distraction in Simulated Driving under Different Road and Traffic Conditions.

The drivers' distraction plays a crucial role in road safety as it is one of the main impacting causes of road accidents. The phenomenon of distraction encompasses both psychological and environmental factors and, therefore, addressing the complex interplay contributing to human distraction in automotive is crucial for developing technologies and interventions for improving road safety. In scientific literature, different works were proposed for the distraction characterization in automotive, but there is still the lack of a univocal measure to assess the degree of distraction, nor a gold-standard tool that allows to "detect" eventual events, road traffic, and additional driving tasks that might contribute to the drivers' distraction. Therefore, the present study aimed at developing an EEG-based "Distraction index" obtained by the combination of the driver's mental workload and attention neurometrics and investigating and validating its reliability by analyzing together subjective and behavioral measures. A total of 25 licensed drivers were involved in this study, where they had to drive in two different scenarios, i.e., City and Highway, while different secondary tasks were alternatively proposed in addition to the main one to modulate the driver's attentional demand. The statistical analysis demonstrated the reliability of the proposed EEG-based distraction index in identifying the drivers' distraction when driving along different roads and traffic conditions (all p < 0.001). More importantly, the proposed index was demonstrated to be reliable in identifying which are the most impacting additional driving tasks on the drivers' distraction (all p < 0.01).

Neurophysiological mental fatigue assessment for developing user-centered Artificial Intelligence as a solution for autonomous driving.

The human factor plays a key role in the automotive field since most accidents are due to drivers' unsafe and risky behaviors. The industry is now pursuing two main solutions to deal with this concern: in the short term, there is the development of systems monitoring drivers' psychophysical states, such as inattention and fatigue, and in the medium-long term, there is the development of fully autonomous driving. This second solution is promoted by recent technological progress in terms of Artificial Intelligence and sensing systems aimed at making vehicles more and more accurately aware of their "surroundings." However, even with an autonomous vehicle, the driver should be able to take control of the vehicle when needed, especially during the current transition from the lower (SAE < 3) to the highest level (SAE = 5) of autonomous driving. In this scenario, the vehicle has to be aware not only of its "surroundings" but also of the driver's psychophysical state, i.e., a user-centered Artificial Intelligence. The neurophysiological approach is one the most effective in detecting improper mental states. This is particularly true if considering that the more automatic the driving will be, the less available the vehicular data related to the driver's driving style. The present study aimed at employing a holistic approach, considering simultaneously several neurophysiological parameters, in particular, electroencephalographic, electrooculographic, photopletismographic, and electrodermal activity data to assess the driver's mental fatigue in real time and to detect the onset of fatigue increasing. This would ideally work as an information/trigger channel for the vehicle AI. In all, 26 professional drivers were engaged in a 45-min-lasting realistic driving task in simulated conditions, during which the previously listed biosignals were recorded. Behavioral (reaction times) and subjective measures were also collected to validate the experimental design and to support the neurophysiological results discussion. Results showed that the most sensitive and timely parameters were those related to brain activity. To a lesser extent, those related to ocular parameters were also sensitive to the onset of mental fatigue, but with a delayed effect. The other investigated parameters did not significantly change during the experimental session.

A Neuroergonomic Approach Fostered by Wearable EEG for the Multimodal Assessment of Drivers Trainees.

When assessing trainees' progresses during a driving training program, instructors can only rely on the evaluation of a trainee's explicit behavior and their performance, without having any insight about the training effects at a cognitive level. However, being able to drive does not imply knowing how to drive safely in a complex scenario such as the road traffic. Indeed, the latter point involves mental aspects, such as the ability to manage and allocate one's mental effort appropriately, which are difficult to assess objectively. In this scenario, this study investigates the validity of deploying an electroencephalographic neurometric of mental effort, obtained through a wearable electroencephalographic device, to improve the assessment of the trainee. The study engaged 22 young people, without or with limited driving experience. They were asked to drive along five different but similar urban routes, while their brain activity was recorded through electroencephalography. Moreover, driving performance, subjective and reaction times measures were collected for a multimodal analysis. In terms of subjective and performance measures, no driving improvement could be detected either through the driver's subjective measures or through their driving performance. On the other side, through the electroencephalographic neurometric of mental effort, it was possible to catch their improvement in terms of mental performance, with a decrease in experienced mental demand after three repetitions of the driving training tasks. These results were confirmed by the analysis of reaction times, that significantly improved from the third repetition as well. Therefore, being able to measure when a task is less mentally demanding, and so more automatic, allows to deduce the degree of users training, becoming capable of handling additional tasks and reacting to unexpected events.

Neurophysiological Assessment of An Innovative Maritime Safety System in Terms of Ship Operators' Mental Workload, Stress, and Attention in the Full Mission Bridge Simulator.

The current industrial environment relies heavily on maritime transportation. Despite the continuous technological advances for the development of innovative safety software and hardware systems, there is a consistent gap in the scientific literature regarding the objective evaluation of the performance of maritime operators. The human factor is profoundly affected by changes in human performance or psychological state. The difficulty lies in the fact that the technology, tools, and protocols for investigating human performance are not fully mature or suitable for experimental investigation. The present research aims to integrate these two concepts by (i) objectively characterizing the psychological state of mariners, i.e., mental workload, stress, and attention, through their electroencephalographic (EEG) signal analysis, and (ii) validating an innovative safety framework countermeasure, defined as Human Risk-Informed Design (HURID), through the aforementioned neurophysiological approach. The proposed study involved 26 mariners within a high-fidelity bridge simulator while encountering collision risk in congested waters with and without the HURID. Subjective, behavioral, and neurophysiological data, i.e., EEG, were collected throughout the experimental activities. The results showed that the participants experienced a statistically significant higher mental workload and stress while performing the maritime activities without the HURID, while their attention level was statistically lower compared to the condition in which they performed the experiments with the HURID (all p < 0.05). Therefore, the presented study confirmed the effectiveness of the HURID during maritime operations in critical scenarios and led the way to extend the neurophysiological evaluation of the HFs of maritime operators during the performance of critical and/or standard shipboard tasks.

Wearable Technologies for Electrodermal and Cardiac Activity Measurements: A Comparison between Fitbit Sense, Empatica E4 and Shimmer GSR3.

The capability of measuring specific neurophysiological and autonomic parameters plays a crucial role in the objective evaluation of a human's mental and emotional states. These human aspects are commonly known in the scientific literature to be involved in a wide range of processes, such as stress and arousal. These aspects represent a relevant factor especially in real and operational environments. Neurophysiological autonomic parameters, such as Electrodermal Activity (EDA) and Photoplethysmographic data (PPG), have been usually investigated through research-graded devices, therefore resulting in a high degree of invasiveness, which could negatively interfere with the monitored user's activity. For such a reason, in the last decade, recent consumer-grade wearable devices, usually designed for fitness-tracking purposes, are receiving increasing attention from the scientific community, and are characterized by a higher comfort, ease of use and, therefore, by a higher compatibility with daily-life environments. The present preliminary study was aimed at assessing the reliability of a consumer wearable device, i.e., the Fitbit Sense, with respect to a research-graded wearable, i.e., the Empatica E4 wristband, and a laboratory device, i.e., the Shimmer GSR3+. EDA and PPG data were collected among 12 participants while they performed multiple resting conditions. The results demonstrated that the EDA- and PPG-derived features computed through the wearable and research devices were positively and significantly correlated, while the reliability of the consumer device was significantly lower.

Neurophysiological Evaluation of Students' Experience during Remote and Face-to-Face Lessons: A Case Study at Driving School.

Nowadays, fostered by technological progress and contextual circumstances such as the economic crisis and pandemic restrictions, remote education is experiencing growing deployment. However, this growth has generated widespread doubts about the actual effectiveness of remote/online learning compared to face-to-face education. The present study was aimed at comparing face-to-face and remote education through a multimodal neurophysiological approach. It involved forty students at a driving school, in a real classroom, experiencing both modalities. Wearable devices to measure brain, ocular, heart and sweating activities were employed in order to analyse the students' neurophysiological signals to obtain insights into the cognitive dimension. In particular, four parameters were considered: the Eye Blink Rate, the Heart Rate and its Variability and the Skin Conductance Level. In addition, the students filled out a questionnaire at the end to obtain an explicit measure of their learning performance. Data analysis showed higher cognitive activity, in terms of attention and mental engagement, in the in-presence setting compared to the remote modality. On the other hand, students in the remote class felt more stressed, particularly during the first part of the lesson. The analysis of questionnaires demonstrated worse performance for the remote group, thus suggesting a common "disengaging" behaviour when attending remote courses, thus undermining their effectiveness. In conclusion, neuroscientific tools could help to obtain insights into mental concerns, often "blind", such as decreasing attention and increasing stress, as well as their dynamics during the lesson itself, thus allowing the definition of proper countermeasures to emerging issues when introducing new practices into daily life.

Validation of an EEG-based Neurometric for online monitoring and detection of mental drowsiness while driving.

The driving drowsiness has been identified as one of the major causes of road traffic accidents, causing fatalities and permanent injuring. Drowsy drivers are prone to suddenly lose control of the car, mostly without any prior behavioral cue. The present study involved 19 participants in a simulated driving protocol, designed to induce mental drowsiness into the drivers. The objective of the study consisted in testing an innovative Electroencephalographic (EEG)-based index, the MDrow index, in detecting the driving drowsiness. Such an index, derived from parietal EEG channels, was already validated in our previous work achieving outstanding performance with respect to more conventional techniques. In this study, the possibility of obtaining a similar index from the frontal sites in order to foster its exploitation in real environments has been investigated. The results demonstrated the capability of the "frontal" MDrow index in evaluating the driving drowsiness experienced by the participants with performance comparable to that one previously validated over parietal sites. Also, the impact of the reduction of the electrodes number on index reliability has been investigated, in order to evaluate its compatibility with current wearable EEG devices.

Real-time Pilot Crew's Mental Workload and Arousal Assessment During Simulated Flights for Training Evaluation: a Case Study.

Training assessment is usually done by evaluating information derived from instructor's supervision related to the pilot's operational performance and behavior. However, this approach lacks objective measures, especially regarding the pilots' mental states while accomplishing the flight training tasks. The study therefore aimed at developing and testing a method for gathering and analyzing in real-time pilots' brain activity and skin conductance to improve the training evaluation. In this regard, Novice pilots' neurophysiological signals were acquired throughout multi-crew training sessions. The results demonstrated how the methodology proposed was able to endow real-time pilots' mental workload and arousal assessment for i) better evaluating training progress and operational behavior during the training session, and ii) for objectively comparing different training sessions.

Evaluation of a New Lightweight EEG Technology for Translational Applications of Passive Brain-Computer Interfaces.

Technologies like passive brain-computer interfaces (BCI) can enhance human-machine interaction. Anyhow, there are still shortcomings in terms of easiness of use, reliability, and generalizability that prevent passive-BCI from entering real-life situations. The current work aimed to technologically and methodologically design a new gel-free passive-BCI system for out-of-the-lab employment. The choice of the water-based electrodes and the design of a new lightweight headset met the need for easy-to-wear, comfortable, and highly acceptable technology. The proposed system showed high reliability in both laboratory and realistic settings, performing not significantly different from the gold standard based on gel electrodes. In both cases, the proposed system allowed effective discrimination (AUC > 0.9) between low and high levels of workload, vigilance, and stress even for high temporal resolution (<10 s). Finally, the generalizability of the proposed system has been tested through a cross-task calibration. The system calibrated with the data recorded during the laboratory tasks was able to discriminate the targeted human factors during the realistic task reaching AUC values higher than 0.8 at 40 s of temporal resolution in case of vigilance and workload, and 20 s of temporal resolution for the stress monitoring. These results pave the way for ecologic use of the system, where calibration data of the realistic task are difficult to obtain.

EEG-Based Index for Timely Detecting User's Drowsiness Occurrence in Automotive Applications.

Human errors are widely considered among the major causes of road accidents. Furthermore, it is estimated that more than 90% of vehicle crashes causing fatal and permanent injuries are directly related to mental tiredness, fatigue, and drowsiness of the drivers. In particular, driving drowsiness is recognized as a crucial aspect in the context of road safety, since drowsy drivers can suddenly lose control of the car. Moreover, the driving drowsiness episodes mostly appear suddenly without any prior behavioral evidence. The present study aimed at characterizing the onset of drowsiness in car drivers by means of a multimodal neurophysiological approach to develop a synthetic electroencephalographic (EEG)-based index, able to detect drowsy events. The study involved 19 participants in a simulated scenario structured in a sequence of driving tasks under different situations and traffic conditions. The experimental conditions were designed to induce prominent mental drowsiness in the final part. The EEG-based index, so-called "MDrow index", was developed and validated to detect the driving drowsiness of the participants. The MDrow index was derived from the Global Field Power calculated in the Alpha EEG frequency band over the parietal brain sites. The results demonstrated the reliability of the proposed MDrow index in detecting the driving drowsiness experienced by the participants, resulting also more sensitive and timely sensible with respect to more conventional autonomic parameters, such as the EyeBlinks Rate and the Heart Rate Variability, and to subjective measurements (self-reports).

Validation of a Light EEG-Based Measure for Real-Time Stress Monitoring during Realistic Driving.

Driver's stress affects decision-making and the probability of risk occurrence, and it is therefore a key factor in road safety. This suggests the need for continuous stress monitoring. This work aims at validating a stress neurophysiological measure-a Neurometric-for out-of-the-lab use obtained from lightweight EEG relying on two wet sensors, in real-time, and without calibration. The Neurometric was tested during a multitasking experiment and validated with a realistic driving simulator. Twenty subjects participated in the experiment, and the resulting stress Neurometric was compared with the Random Forest (RF) model, calibrated by using EEG features and both intra-subject and cross-task approaches. The Neurometric was also compared with a measure based on skin conductance level (SCL), representing one of the physiological parameters investigated in the literature mostly correlated with stress variations. We found that during both multitasking and realistic driving experiments, the Neurometric was able to discriminate between low and high levels of stress with an average Area Under Curve (AUC) value higher than 0.9. Furthermore, the stress Neurometric showed higher AUC and stability than both the SCL measure and the RF calibrated with a cross-task approach. In conclusion, the Neurometric proposed in this work proved to be suitable for out-of-the-lab monitoring of stress levels.

Neuroscientific Methods for Exploring User Perceptions While Dealing With Mobile Advertising: A Novel and Integrated Approach.

Display and native ads represent two of the most widely used digital advertising formats employed by advertisers that aim to grab the attention of online users. In recent years, the native format has become very popular because it relies on deceptive features that make harder the recognition of its advertising nature, reducing avoiding behaviors such as the banner blindness phenomena, traditionally associated to display advertising, and so increasing its advertising effectiveness. The present study, based on a forefront research protocol specifically designed for the advertising research on smartphone devices, aims to investigate through neurophysiological and self-reported measures, the perception of display and native ads placed within article webpages, and to assess the efficacy of an integrated approach. Eye-tracking results showed higher visual attention and longer viewing time associated with native advertisements in comparison to traditional display advertisements, confirming and extending evidence provided by previous research. Despite a significantly higher rate of self-reported advertising intent was detected for articles containing display ads when compared to articles containing native ads, no differences have been found while performing the same comparison for the neurophysiological measures of emotional involvement and approaching motivation of for the self-reported measures of pleasantness and annoyance. Such findings along with the employment of an innovative research protocol, contribute to providing further cues to the current debate related to the effectiveness of two of the most widely used digital advertising formats.

Mental Effort Estimation by Passive BCI: A Cross-Subject Analysis.

Despite the technological advancements, the employment of passive brain computer interface (BCI) out of the laboratory context is still challenging. This is largely due to methodological reasons. On the one hand, machine learning methods have shown their potential in maximizing performance for user mental states classification. On the other hand, the issues related to the necessary and frequent calibration of algorithms and to the temporal resolution of the measurement (i.e. how long it takes to have a reliable state measure) are still unsolved. This work explores the performances of a passive BCI system for mental effort monitoring consisting of three frontal electroencephalographic (EEG) channels. In particular, three calibration approaches have been tested: an intra-subject approach, a cross-subject approach, and a free-calibration procedure based on the simple average of theta activity over the three employed channels. A Random Forest model has been employed in the first two cases. The results obtained during multi-tasking have shown that the cross-subject approach allows the classification of low and high mental effort with an AUC higher than 0.9, with a related time resolution of 45 seconds. Moreover, these performances are not significantly different from the intra-subject approach although they are significantly higher than the calibration-free approach. In conclusion, these results suggest that a light (three EEG channels) passive BCI system based on a Random Forest algorithm and cross-subject calibration could be a simple and reliable tool for out-of-the-lab employment.

Label-Based Alignment Multi-Source Domain Adaptation for Cross-Subject EEG Fatigue Mental State Evaluation.

Accurate detection of driving fatigue is helpful in significantly reducing the rate of road traffic accidents. Electroencephalogram (EEG) based methods are proven to be efficient to evaluate mental fatigue. Due to its high non-linearity, as well as significant individual differences, how to perform EEG fatigue mental state evaluation across different subjects still keeps challenging. In this study, we propose a Label-based Alignment Multi-Source Domain Adaptation (LA-MSDA) for cross-subject EEG fatigue mental state evaluation. Specifically, LA-MSDA considers the local feature distributions of relevant labels between different domains, which efficiently eliminates the negative impact of significant individual differences by aligning label-based feature distributions. In addition, the strategy of global optimization is introduced to address the classifier confusion decision boundary issues and improve the generalization ability of LA-MSDA. Experimental results show LA-MSDA can achieve remarkable results on EEG-based fatigue mental state evaluation across subjects, which is expected to have wide application prospects in practical brain-computer interaction (BCI), such as online monitoring of driver fatigue, or assisting in the development of on-board safety systems.

The Sample Size Matters: To What Extent the Participant Reduction Affects the Outcomes of a Neuroscientific Research. A Case-Study in Neuromarketing Field.

The sample size is a crucial concern in scientific research and even more in behavioural neurosciences, where besides the best practice it is not always possible to reach large experimental samples. In this study we investigated how the outcomes of research change in response to sample size reduction. Three indices computed during a task involving the observations of four videos were considered in the analysis, two related to the brain electroencephalographic (EEG) activity and one to autonomic physiological measures, i.e., heart rate and skin conductance. The modifications of these indices were investigated considering five subgroups of sample size (32, 28, 24, 20, 16), each subgroup consisting of 630 different combinations made by bootstrapping n (n = sample size) out of 36 subjects, with respect to the total population (i.e., 36 subjects). The correlation analysis, the mean squared error (MSE), and the standard deviation (STD) of the indexes were studied at the participant reduction and three factors of influence were considered in the analysis: the type of index, the task, and its duration (time length). The findings showed a significant decrease of the correlation associated to the participant reduction as well as a significant increase of MSE and STD (p < 0.05). A threshold of subjects for which the outcomes remained significant and comparable was pointed out. The effects were to some extents sensitive to all the investigated variables, but the main effect was due to the task length. Therefore, the minimum threshold of subjects for which the outcomes were comparable increased at the reduction of the spot duration.

An EEG-Based Transfer Learning Method for Cross-Subject Fatigue Mental State Prediction.

Fatigued driving is one of the main causes of traffic accidents. The electroencephalogram (EEG)-based mental state analysis method is an effective and objective way of detecting fatigue. However, as EEG shows significant differences across subjects, effectively "transfering" the EEG analysis model of the existing subjects to the EEG signals of other subjects is still a challenge. Domain-Adversarial Neural Network (DANN) has excellent performance in transfer learning, especially in the fields of document analysis and image recognition, but has not been applied directly in EEG-based cross-subject fatigue detection. In this paper, we present a DANN-based model, Generative-DANN (GDANN), which combines Generative Adversarial Networks (GAN) to enhance the ability by addressing the issue of different distribution of EEG across subjects. The comparative results show that in the analysis of cross-subject tasks, GDANN has a higher average accuracy of 91.63% in fatigue detection across subjects than those of traditional classification models, which is expected to have much broader application prospects in practical brain-computer interaction (BCI).

Wearable Technologies for Mental Workload, Stress, and Emotional State Assessment during Working-Like Tasks: A Comparison with Laboratory Technologies.

The capability of monitoring user's performance represents a crucial aspect to improve safety and efficiency of several human-related activities. Human errors are indeed among the major causes of work-related accidents. Assessing human factors (HFs) could prevent these accidents through specific neurophysiological signals' evaluation but laboratory sensors require highly-specialized operators and imply a certain grade of invasiveness which could negatively interfere with the worker's activity. On the contrary, consumer wearables are characterized by their ease of use and their comfortability, other than being cheaper compared to laboratory technologies. Therefore, wearable sensors could represent an ideal substitute for laboratory technologies for a real-time assessment of human performances in ecological settings. The present study aimed at assessing the reliability and capability of consumer wearable devices (i.e., Empatica E4 and Muse 2) in discriminating specific mental states compared to laboratory equipment. The electrooculographic (EOG), electrodermal activity (EDA) and photoplethysmographic (PPG) signals were acquired from a group of 17 volunteers who took part to the experimental protocol in which different working scenarios were simulated to induce different levels of mental workload, stress, and emotional state. The results demonstrated that the parameters computed by the consumer wearable and laboratory sensors were positively and significantly correlated and exhibited the same evidences in terms of mental states discrimination.

Joint Analysis of Eye Blinks and Brain Activity to Investigate Attentional Demand during a Visual Search Task.

In several fields, the need for a joint analysis of brain activity and eye activity to investigate the association between brain mechanisms and manifest behavior has been felt. In this work, two levels of attentional demand, elicited through a conjunction search task, have been modelled in terms of eye blinks, brain activity, and brain network features. Moreover, the association between endogenous neural mechanisms underlying attentional demand and eye blinks, without imposing a time-locked structure to the analysis, has been investigated. The analysis revealed statistically significant spatial and spectral modulations of the recorded brain activity according to the different levels of attentional demand, and a significant reduction in the number of eye blinks when a higher amount of attentional investment was required. Besides, the integration of information coming from high-density electroencephalography (EEG), brain source localization, and connectivity estimation allowed us to merge spectral and causal information between brain areas, characterizing a comprehensive model of neurophysiological processes behind attentional demand. The analysis of the association between eye and brain-related parameters revealed a statistically significant high correlation (R > 0.7) of eye blink rate with anterofrontal brain activity at 8 Hz, centroparietal brain activity at 12 Hz, and a significant moderate correlation with the participation of right Intra Parietal Sulcus in alpha band (R = -0.62). Due to these findings, this work suggests the possibility of using eye blinks measured from one sensor placed on the forehead as an unobtrusive measure correlating with neural mechanisms underpinning attentional demand.

A Video-Based Technique for Heart Rate and Eye Blinks Rate Estimation: A Potential Solution for Telemonitoring and Remote Healthcare.

Current telemedicine and remote healthcare applications foresee different interactions between the doctor and the patient relying on the use of commercial and medical wearable sensors and internet-based video conferencing platforms. Nevertheless, the existing applications necessarily require a contact between the patient and sensors for an objective evaluation of the patient's state. The proposed study explored an innovative video-based solution for monitoring neurophysiological parameters of potential patients and assessing their mental state. In particular, we investigated the possibility to estimate the heart rate (HR) and eye blinks rate (EBR) of participants while performing laboratory tasks by mean of facial-video analysis. The objectives of the study were focused on: (i) assessing the effectiveness of the proposed technique in estimating the HR and EBR by comparing them with laboratory sensor-based measures and (ii) assessing the capability of the video-based technique in discriminating between the participant's resting state (Nominal condition) and their active state (Non-nominal condition). The results demonstrated that the HR and EBR estimated through the facial-video technique or the laboratory equipment did not statistically differ (p > 0.1), and that these neurophysiological parameters allowed to discriminate between the Nominal and Non-nominal states (p < 0.02).