Burst c-VEP Based BCI: Optimizing stimulus design for enhanced classification with minimal calibration data and improved user experience.

The utilization of aperiodic flickering visual stimuli under the form of code-modulated Visual Evoked Potentials (c-VEP) represents a pivotal advancement in the field of reactive Brain-Computer Interface (rBCI). A major advantage of the c-VEP approach is that the training of the model is independent of the number and complexity of targets, which helps reduce calibration time. Nevertheless, the existing designs of c-VEP stimuli can be further improved in terms of visual user experience but also to achieve a higher signal-to-noise ratio, while shortening the selection time and calibration process. In this study, we introduce an innovative variant of code-VEP, referred to as "Burst c-VEP". This original approach involves the presentation of short bursts of aperiodic visual flashes at a deliberately slow rate, typically ranging from two to four flashes per second. The rationale behind this design is to leverage the sensitivity of the primary visual cortex to transient changes in low-level stimuli features to reliably elicit distinctive series of visual evoked potentials. In comparison to other types of faster-paced code sequences, burst c-VEP exhibit favorable properties to achieve high bitwise decoding performance using convolutional neural networks (CNN), which yields potential to attain faster selection time with the need for less calibration data. Furthermore, our investigation focuses on reducing the perceptual saliency of c-VEP through the attenuation of visual stimuli contrast and intensity to significantly improve users' visual comfort. The proposed solutions were tested through an offline 4-classes c-VEP protocol involving 12 participants. Following a factorial design, participants were instructed to focus on c-VEP targets whose pattern (burst and maximum-length sequences) and amplitude (100% or 40% amplitude depth modulations) were manipulated across experimental conditions. Firstly, the full amplitude burst c-VEP sequences exhibited higher accuracy, ranging from 90.5% (with 17.6s of calibration data) to 95.6% (with 52.8s of calibration data), compared to its m-sequence counterpart (71.4% to 85.0%). The mean selection time for both types of codes (1.5 s) compared favorably to reports from previous studies. Secondly, our findings revealed that lowering the intensity of the stimuli only slightly decreased the accuracy of the burst code sequences to 94.2% while leading to substantial improvements in terms of user experience. Taken together, these results demonstrate the high potential of the proposed burst codes to advance reactive BCI both in terms of performance and usability. The collected dataset, along with the proposed CNN architecture implementation, are shared through open-access repositories.

Capturing Cognitive Events Embedded in the Real World Using Mobile Electroencephalography and Eye-Tracking.

The study of cognitive processes underlying natural behaviors implies departing from computerized paradigms and artificial experimental probes. The present study aims to assess the feasibility of capturing neural markers (P300 ERPs) of cognitive processes evoked in response to the identification of task-relevant objects embedded in a real-world environment. To this end, EEG and eye-tracking data were recorded while participants attended stimuli presented on a tablet and while they searched for books in a library. Initial analyses of the library data revealed that P300-like features shifted in time. A Dynamic Time Warping analysis confirmed the presence of P300 ERP in the library condition. Library data were then lag-corrected based on cross-correlation coefficients. Together, these approaches uncovered P300 ERP responses in the library recordings. These findings highlight the relevance of scalable experimental designs, joint brain and body recordings, and template-matching analyses to capture cognitive events during natural behaviors.

Towards Mixed-Initiative Human-Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction.

The design of human-robot interactions is a key challenge to optimize operational performance. A promising approach is to consider mixed-initiative interactions in which the tasks and authority of each human and artificial agents are dynamically defined according to their current abilities. An important issue for the implementation of mixed-initiative systems is to monitor human performance to dynamically drive task allocation between human and artificial agents (i.e., robots). We, therefore, designed an experimental scenario involving missions whereby participants had to cooperate with a robot to fight fires while facing hazards. Two levels of robot automation (manual vs. autonomous) were randomly manipulated to assess their impact on the participants' performance across missions. Cardiac activity, eye-tracking, and participants' actions on the user interface were collected. The participants performed differently to an extent that we could identify high and low score mission groups that also exhibited different behavioral, cardiac and ocular patterns. More specifically, our findings indicated that the higher level of automation could be beneficial to low-scoring participants but detrimental to high-scoring ones, and vice versa. In addition, inter-subject single-trial classification results showed that the studied behavioral and physiological features were relevant to predict mission performance. The highest average balanced accuracy (74%) was reached using the features extracted from all input devices. These results suggest that an adaptive HRI driving system, that would aim at maximizing performance, would be capable of analyzing such physiological and behavior markers online to further change the level of automation when it is relevant for the mission purpose.

Monitoring Pilot's Mental Workload Using ERPs and Spectral Power with a Six-Dry-Electrode EEG System in Real Flight Conditions.

Recent technological progress has allowed the development of low-cost and highly portable brain sensors such as pre-amplified dry-electrodes to measure cognitive activity out of the laboratory. This technology opens promising perspectives to monitor the "brain at work" in complex real-life situations such as while operating aircraft. However, there is a need to benchmark these sensors in real operational conditions. We therefore designed a scenario in which twenty-two pilots equipped with a six-dry-electrode EEG system had to perform one low load and one high load traffic pattern along with a passive auditory oddball. In the low load condition, the participants were monitoring the flight handled by a flight instructor, whereas they were flying the aircraft in the high load condition. At the group level, statistical analyses disclosed higher P300 amplitude for the auditory target (Pz, P4 and Oz electrodes) along with higher alpha band power (Pz electrode), and higher theta band power (Oz electrode) in the low load condition as compared to the high load one. Single trial classification accuracy using both event-related potentials and event-related frequency features at the same time did not exceed chance level to discriminate the two load conditions. However, when considering only the frequency features computed over the continuous signal, classification accuracy reached around 70% on average. This study demonstrates the potential of dry-EEG to monitor cognition in a highly ecological and noisy environment, but also reveals that hardware improvement is still needed before it can be used for everyday flight operations.

Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting.

Individuals often have reduced ability to hear alarms in real world situations (e.g., anesthesia monitoring, flying airplanes) when attention is focused on another task, sometimes with devastating consequences. This phenomenon is called inattentional deafness and usually occurs under critical high workload conditions. It is difficult to simulate the critical nature of these tasks in the laboratory. In this study, dry electroencephalography is used to investigate inattentional deafness in real flight while piloting an airplane. The pilots participating in the experiment responded to audio alarms while experiencing critical high workload situations. It was found that missed relative to detected alarms were marked by reduced stimulus evoked phase synchrony in theta and alpha frequencies (6-14 Hz) from 120 to 230 ms poststimulus onset. Correlation of alarm detection performance with intertrial coherence measures of neural phase synchrony showed different frequency and time ranges for detected and missed alarms. These results are consistent with selective attentional processes actively disrupting oscillatory coherence in sensory networks not involved with the primary task (piloting in this case) under critical high load conditions. This hypothesis is corroborated by analyses of flight parameters showing greater maneuvering associated with difficult phases of flight occurring during missed alarms. Our results suggest modulation of neural oscillation is a general mechanism of attention utilizing enhancement of phase synchrony to sharpen alarm perception during successful divided attention, and disruption of phase synchrony in brain networks when attentional demands of the primary task are great, such as in the case of inattentional deafness.

Assessment of Ocular and Physiological Metrics to Discriminate Flight Phases in Real Light Aircraft.

OBJECTIVE: The purpose of the present study was to find psychophysiological proxies that are straightforward to use and could be implemented in actual flight conditions to accurately discriminate pilots' workload levels. BACKGROUND: Piloting an aircraft is a complex activity where cognitive limitations may jeopardize flight safety. There is a need to implement solutions to monitor pilots' workload level to improve flight safety. There has been recent interest in combining psychophysiological measurements. Most of these studies were conducted in flight simulators at the group level, limiting the interpretation of the results. METHODS: We conducted an experiment with 11 pilots performing two standard traffic patterns in a light aircraft. Five metrics were derived from their ocular and cardiac activities and were evaluated through three flight phases: takeoff, downwind, and landing. RESULTS: Statistical analyses showed that the saccadic rate was the most efficient metric to distinguish between the three flight phases. In addition, a classifier trained on the ocular data collected from the first run predicted the flight phase within a second run with an accuracy of 75%. No gain in the classifier accuracy has been found by combining cardiac and ocular metrics. CONCLUSIONS: Ocular-based metrics may be more suitable than cardiac ones to provide relevant information on pilots' flying activity in operational settings. APPLICATIONS: Electrocardiographic and eye-tracking devices could be implemented in future cockpits as additional flight data for accident analysis, an objective pilot's state evaluation for training, and proxies for human-machine interactions to improve flight safety.

Neural signature of inattentional deafness.

Inattentional deafness is the failure to hear otherwise audible sounds (usually alarms) that may occur under high workload conditions. One potential cause for its occurrence could be an attentional bottleneck that occurs when task demands are high, resulting in lack of resources for processing of additional tasks. In this fMRI experiment, we explore the brain regions active during the occurrence of inattentional deafness using a difficult perceptual-motor task in which the participants fly through a simulated Red Bull air race course and at the same time push a button on the joystick to the presence of audio alarms. Participants were instructed to focus on the difficult piloting task and to press the button on the joystick quickly when they noticed an audio alarm. The fMRI results revealed that audio misses relative to hits had significantly greater activity in the right inferior frontal gyrus IFG and the superior medial frontal cortex. Consistent with an attentional bottleneck, activity in these regions was also present for poor flying performance (contrast of gates missed versus gates passed for the flying task). A psychophysiological interaction analysis from the IFG identified reduced effective connectivity to auditory processing regions in the right superior temporal gyrus for missed audio alarms relative to audio alarms that were heard. This study identifies a neural signature of inattentional deafness in an ecologically valid situation by directly measuring differences in brain activity and effective connectivity between audio alarms that were not heard compared to those that were heard. Hum Brain Mapp 38:5440-5455, 2017. © 2017 Wiley Periodicals, Inc.

Petri net-based modelling of human-automation conflicts in aviation.

Analyses of aviation safety reports reveal that human-machine conflicts induced by poor automation design are remarkable precursors of accidents. A review of different crew-automation conflicting scenarios shows that they have a common denominator: the autopilot behaviour interferes with the pilot's goal regarding the flight guidance via 'hidden' mode transitions. Considering both the human operator and the machine (i.e. the autopilot or the decision functions) as agents, we propose a Petri net model of those conflicting interactions, which allows them to be detected as deadlocks in the Petri net. In order to test our Petri net model, we designed an autoflight system that was formally analysed to detect conflicting situations. We identified three conflicting situations that were integrated in an experimental scenario in a flight simulator with 10 general aviation pilots. The results showed that the conflicts that we had a-priori identified as critical had impacted the pilots' performance. Indeed, the first conflict remained unnoticed by eight participants and led to a potential collision with another aircraft. The second conflict was detected by all the participants but three of them did not manage the situation correctly. The last conflict was also detected by all the participants but provoked typical automation surprise situation as only one declared that he had understood the autopilot behaviour. These behavioural results are discussed in terms of workload and number of fired 'hidden' transitions. Eventually, this study reveals that both formal and experimental approaches are complementary to identify and assess the criticality of human-automation conflicts. Practitioner Summary: We propose a Petri net model of human-automation conflicts. An experiment was conducted with general aviation pilots performing a scenario involving three conflicting situations to test the soundness of our formal approach. This study reveals that both formal and experimental approaches are complementary to identify and assess the criticality conflicts.

Attentional costs and failures in air traffic control notifications.

Large display screens are common in supervisory tasks, meaning that alerts are often perceived in peripheral vision. Five air traffic control notification designs were evaluated in their ability to capture attention during an ongoing supervisory task, as well as their impact on the primary task. A range of performance measures, eye-tracking and subjective reports showed that colour, even animated, was less effective than movement, and notifications sometimes went unnoticed. Designs that drew attention to the notified aircraft by a pulsating box, concentric circles or the opacity of the background resulted in faster perception and no missed notifications. However, the latter two designs were intrusive and impaired primary task performance, while the simpler animated box captured attention without an overhead cognitive cost. These results highlight the need for a holistic approach to evaluation, achieving a balance between the benefits for one aspect of performance against the potential costs for another. Practitioner summary: We performed a holistic examination of air traffic control notification designs regarding their ability to capture attention during an ongoing supervisory task. The combination of performance, eye-tracking and subjective measurements demonstrated that the best design achieved a balance between attentional power and the overhead cognitive cost to primary task performance.

Evaluation of a headphones-fitted EEG system for the recording of auditory evoked potentials and mental workload assessment.

Advancements in portable neuroimaging technologies open up new opportunities to gain insight into the neural dynamics and cognitive processes underlying day-to-day behaviors. In this study, we evaluated the relevance of a headphone- mounted electroencephalogram (EEG) system for monitoring mental workload. The participants (N = 12) were instructed to pay attention to auditory alarms presented sporadically while performing the Multi-Attribute Task Battery (MATB) whose difficulty was staged across three conditions to manipulate mental workload. The P300 Event-Related Potentials (ERP) elicited by the presentation of auditory alarms were used as probes of attentional resources available. The amplitude and latency of P300 ERPs were compared across experimental conditions. Our findings indicate that the P300 ERP component can be captured using a headphone-mounted EEG system. Moreover, neural responses to alarm could be used to classify mental workload with high accuracy (over 80%) at a single-trial level. Our analyses indicated that the signal-to-noise ratio acquired by the sponge-based sensors remained stable throughout the recordings. These results highlight the potential of portable neuroimaging technology for the development of neuroassistive applications while underscoring the current limitations and challenges associated with the integration of EEG sensors in everyday-life wearable technologies. Overall, our study contributes to the growing body of research exploring the feasibility and validity of wearable neuroimaging technologies for the study of human cognition and behavior in real-world settings.

POMDP-BCI: A Benchmark of (Re)Active BCI Using POMDP to Issue Commands.

OBJECTIVE: Past research in Brain-Computer Interfaces (BCI) have presented different decoding algorithms for different modalities. Meanwhile, highly specific decision making processes have been developed for some of these modalities, while others lack such a component in their classic pipeline. The present work proposes a model based on Partially Observable Markov Decission Process (POMDP) that works as a high-level decision making framework for three different active/reactive BCI modalities. METHODS: We tested our approach on three different BCI modalities using publicly available datasets. We compared the general POMDP model as a decision making process with state of the art methods for each BCI modality. Accuracy, false positive (FP) trials, no-action (NA) trials and average decision time are presented as metrics. RESULTS: Our results show how the presented POMDP models achieve comparable or better performance to the presented baseline methods, while being usable for the three proposed experiments without significant changes. Crucially, it offers the possibility of taking no-action (NA) when the decoding does not perform well. CONCLUSION: The present work implements a flexible POMDP model that acts as a sequential decision framework for BCI systems that lack such a component, and perform comparably to those that include it. SIGNIFICANCE: We believe the proposed POMDP framework provides several interesting properties for future BCI developments, mainly the generalizability to any BCI modality and the possible integration of other physiological or brain data pipelines under a unified decision-making framework.

Modeling Complex EEG Data Distribution on the Riemannian Manifold Toward Outlier Detection and Multimodal Classification.

OBJECTIVE: The usage of Riemannian geometry for Brain-computer interfaces (BCIs) has gained momentum in recent years. Most of the machine learning techniques proposed for Riemannian BCIs consider the data distribution on a manifold to be unimodal. However, the distribution is likely to be multimodal rather than unimodal since high-data variability is a crucial limitation of electroencephalography (EEG). In this paper, we propose a novel data modeling method for considering complex data distributions on a Riemannian manifold of EEG covariance matrices, aiming to improve BCI reliability. METHODS: Our method, Riemannian spectral clustering (RiSC), represents EEG covariance matrix distribution on a manifold using a graph with proposed similarity measurement based on geodesic distances, then clusters the graph nodes through spectral clustering. This allows flexibility to model both a unimodal and a multimodal distribution on a manifold. RiSC can be used as a basis to design an outlier detector named outlier detection Riemannian spectral clustering (odenRiSC) and a multimodal classifier named multimodal classifier Riemannian spectral clustering (mcRiSC). All required parameters of odenRiSC/mcRiSC are selected in data-driven manner. Moreover, there is no need to pre-set a threshold for outlier detection and the number of modes for multimodal classification. RESULTS: The experimental evaluation revealed odenRiSC can detect EEG outliers more accurately than existing methods and mcRiSC outperformed the standard unimodal classifier, especially on high-variability datasets. CONCLUSION: odenRiSC/mcRiSC are anticipated to contribute to making real-life BCIs outside labs and neuroergonomics applications more robust. SIGNIFICANCE: RiSC can work as a robust EEG outlier detector and multimodal classifier.

Affective decision making under uncertainty during a plausible aviation task: an fMRI study.

In aeronautics, plan continuation error (PCE) represents failure to revise a flight plan despite emerging evidence suggesting that it is no longer safe. Assuming that PCE may be associated with a shift from cold to hot reasoning, we hypothesized that this transition may result from a large range of strong negative emotional influences linked with the decision to abort a landing and circle for a repeat attempt, referred to as a "go-around". We investigated this hypothesis by combining functional neuroimaging with an ecologically valid aviation task performed under contextual variation in incentive and situational uncertainty. Our goal was to identify regional brain activity related to the sorts of conservative or liberal decision-making strategies engaged when participants were both exposed to a financial payoff matrix constructed to bias responses in favor of landing acceptance, while they were simultaneously experiencing maximum levels of uncertainty related to high levels of stimulus ambiguity. Combined with the observed behavioral outcomes, our neuroimaging results revealed a shift from cold to hot decision making in response to high uncertainty when participants were exposed to the financial incentive. Most notably, while we observed activity increases in response to uncertainty in many frontal regions such as dorsolateral prefrontal cortex (DLPFC) and anterior cingulate cortex (ACC), less overall activity was observed when the reward was combined with uncertainty. Moreover, participants with poor decision making, quantified as a lower discriminability index d', exhibited riskier behavior coupled with lower activity in the right DLPFC. These outcomes suggest a disruptive effect of biased financial incentive and high uncertainty on the rational decision-making neural network, and consequently, on decision relevance.

Evaluation of head-free eye tracking as an input device for air traffic control.

The purpose of this study was to investigate the possibility to integrate a free head motion eye-tracking system as input device in air traffic control (ATC) activity. Sixteen participants used an eye tracker to select targets displayed on a screen as quickly and accurately as possible. We assessed the impact of the presence of visual feedback about gaze position and the method of target selection on selection performance under different difficulty levels induced by variations in target size and target-to-target separation. We tend to consider that the combined use of gaze dwell-time selection and continuous eye-gaze feedback was the best condition as it suits naturally with gaze displacement over the ATC display and free the hands of the controller, despite a small cost in terms of selection speed. In addition, target size had a greater impact on accuracy and selection time than target distance. These findings provide guidelines on possible further implementation of eye tracking in ATC everyday activity. PRACTITIONER SUMMARY: We investigated the possibility to integrate a free head motion eye-tracking system as input device in air traffic control (ATC). We found that the combined use of gaze dwell-time selection and continuous eye-gaze feedback allowed the best performance and that target size had a greater impact on performance than target distance.

The role of brain-localized gamma and alpha oscillations in inattentional deafness: implications for understanding human attention.

Introduction: The processes involved in how the attention system selectively focuses on perceptual and motor aspects related to a specific task, while suppressing features of other tasks and/or objects in the environment, are of considerable interest for cognitive neuroscience. The goal of this experiment was to investigate neural processes involved in selective attention and performance under multi-task situations. Several studies have suggested that attention-related gamma-band activity facilitates processing in task-specific modalities, while alpha-band activity inhibits processing in non-task-related modalities. However, investigations into the phenomenon of inattentional deafness/blindness (inability to observe stimuli in non-dominant task when primary task is demanding) have yet to observe gamma-band activity. Methods: This EEG experiment utilizes an engaging whole-body perceptual motor task while carrying out a secondary auditory detection task to investigate neural correlates of inattentional deafness in natural immersive high workload conditions. Differences between hits and misses on the auditory detection task in the gamma (30-50 Hz) and alpha frequency (8-12 Hz) range were carried out at the cortical source level using LORETA. Results: Participant auditory task performance correlated with an increase in gamma-band activity for hits over misses pre- and post-stimulus in left auditory processing regions. Alpha-band activity was greater for misses relative to hits in right auditory processing regions pre- and post-stimulus onset. These results are consistent with the facilitatory/inhibitory role of gamma/alpha-band activity for neural processing. Additional gamma- and alpha-band activity was found in frontal and parietal brain regions which are thought to reflect various attentional monitoring, selection, and switching processes. Discussion: The results of this study help to elucidate the role of gamma and alpha frequency bands in frontal and modality-specific regions involved with selective attention in multi-task immersive situations.

Effect of time-delay on lunar sampling tele-operations: Evidences from cardiac, ocular and behavioral measures.

The purpose of this study is to quantify performance in human-robot interaction under time-delay conditions in a lunar tele-operations sampling task, by testing the hypothesis that an increase of time-delay would lead to higher perceived workload and lower human performance in human-robotic integrated operations. Tele-operation is key in the exploration of the Moon, and allows for robotic elements to be controlled from orbital infrastructure and other planetary bodies such as the Earth. Considering that future missions aim to control rovers (amongst others for sampling tasks) from Earth (delay: 3s), the Gateway (delay: 0.5s) and the Moon (delay: 0s), control under the time-delay conditions for these locations must be studied. Time-delay can affect performance, and understanding the performance means that mission operations can be planned bottom-up, which benefits both the preparation of the crew and the design of rovers. An experiment was conducted with 18 engineers who were assigned to control a robotic arm under three time-delay conditions, representing the three control locations. Several metrics were derived from cardiac, ocular, subjective and behavioral measures. The analyses disclosed that the large time-delay condition statistically increased the perceived workload, the time to complete the mission and decreased heart rate variability compared to the other conditions. However, no effect of time-delay was found on attentional and executive abilities. The metrics proved to be effective in the study of performance quantification in human-robot interaction for tele-operations in lunar control scenarios. This approach can be implemented for a larger range of robotic activities, such as tele-operated driving.

Open multi-session and multi-task EEG cognitive Dataset for passive brain-computer Interface Applications.

Brain-Computer Interfaces and especially passive Brain-Computer interfaces (pBCI), with their ability to estimate and monitor user mental states, are receiving increasing attention from both the fundamental research and the applied research and development communities. Testing new pipelines and benchmarking classifiers and feature extraction algorithms is central to further research within this domain. Unfortunately, data sharing in pBCI research is still scarce. The COG-BCI database encompasses the recordings of 29 participants over 3 separate sessions with 4 different tasks (MATB, N-Back, PVT, Flanker) designed to elicit different mental states, for a total of over 100 hours of open EEG data. This dataset was validated on a subjective, behavioral and physiological level, to ensure its usefulness to the pBCI community. Furthermore, a proof of concept is given with an example of mental workload estimation pipeline and results, to ensure that the data can be used for the design and evaluation of pBCI pipelines. This body of work presents a large effort to promote the use of pBCIs in an open science framework.

Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation.

As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.

Dual Passive Reactive Brain-Computer Interface: A Novel Approach to Human-Machine Symbiosis.

The present study proposes a novel concept of neuroadaptive technology, namely a dual passive-reactive Brain-Computer Interface (BCI), that enables bi-directional interaction between humans and machines. We have implemented such a system in a realistic flight simulator using the NextMind classification algorithms and framework to decode pilots' intention (reactive BCI) and to infer their level of attention (passive BCI). Twelve pilots used the reactive BCI to perform checklists along with an anti-collision radar monitoring task that was supervised by the passive BCI. The latter simulated an automatic avoidance maneuver when it detected that pilots missed an incoming collision. The reactive BCI reached 100% classification accuracy with a mean reaction time of 1.6 s when exclusively performing the checklist task. Accuracy was up to 98.5% with a mean reaction time of 2.5 s when pilots also had to fly the aircraft and monitor the anti-collision radar. The passive BCI achieved a F1-score of 0.94. This first demonstration shows the potential of a dual BCI to improve human-machine teaming which could be applied to a variety of applications.

Reward and uncertainty favor risky decision-making in pilots: evidence from cardiovascular and oculometric measurements.

In this paper we examined plan continuation error (PCE), a well known error made by pilots consisting in continuing the flight plan despite adverse meteorological conditions. Our hypothesis is that a large range of strong negative emotional consequences, including those induced by economic pressure, are associated with the decision to revise the flight plan and favor PCE. We investigated the economic hypothesis with a simplified landing task (reproduction of a real aircraft instrument) in which uncertainty and reward were manipulated. Heart rate (HR), heart rate variability (HRV) and eye tracking measurements were performed to get objective clues both on the cognitive and emotional state of the volunteers. Results showed that volunteers made more risky decisions under the influence of the financial incentive, in particular when uncertainty was high. Psychophysiological examination showed that HR increased and total HRV decreased in response to the cognitive load generated by the task. In addition, HR also increased in response to the financially motivated condition. Eventually, fixation times increased when uncertainty was high, confirming the difficulty in obtaining/interpreting information from the instrument in this condition. These results support the assumption that risky-decision making observed in pilots can be, at least partially, explained by a shift from cold to hot (emotional) decision-making in response to economic constraints and uncertainty.