Mobile brain/body imaging of landmark-based navigation with high-density EEG.

Coupling behavioral measures and brain imaging in naturalistic, ecological conditions is key to comprehend the neural bases of spatial navigation. This highly integrative function encompasses sensorimotor, cognitive, and executive processes that jointly mediate active exploration and spatial learning. However, most neuroimaging approaches in humans are based on static, motion-constrained paradigms and they do not account for all these processes, in particular multisensory integration. Following the Mobile Brain/Body Imaging approach, we aimed to explore the cortical correlates of landmark-based navigation in actively behaving young adults, solving a Y-maze task in immersive virtual reality. EEG analysis identified a set of brain areas matching state-of-the-art brain imaging literature of landmark-based navigation. Spatial behavior in mobile conditions additionally involved sensorimotor areas related to motor execution and proprioception usually overlooked in static fMRI paradigms. Expectedly, we located a cortical source in or near the posterior cingulate, in line with the engagement of the retrosplenial complex in spatial reorientation. Consistent with its role in visuo-spatial processing and coding, we observed an alpha-power desynchronization while participants gathered visual information. We also hypothesized behavior-dependent modulations of the cortical signal during navigation. Despite finding few differences between the encoding and retrieval phases of the task, we identified transient time-frequency patterns attributed, for instance, to attentional demand, as reflected in the alpha/gamma range, or memory workload in the delta/theta range. We confirmed that combining mobile high-density EEG and biometric measures can help unravel the brain structures and the neural modulations subtending ecological landmark-based navigation.

The CLAIRE COVID-19 initiative: approach, experiences and recommendations.

A volunteer effort by Artificial Intelligence (AI) researchers has shown it can deliver significant research outcomes rapidly to help tackle COVID-19. Within two months, CLAIRE's self-organising volunteers delivered the World's first comprehensive curated repository of COVID-19-related datasets useful for drug-repurposing, drafted review papers on the role CT/X-ray scan analysis and robotics could play, and progressed research in other areas. Given the pace required and nature of voluntary efforts, the teams faced a number of challenges. These offer insights in how better to prepare for future volunteer scientific efforts and large scale, data-dependent AI collaborations in general. We offer seven recommendations on how to best leverage such efforts and collaborations in the context of managing future crises.

Interactions of spatial strategies producing generalization gradient and blocking: A computational approach.

We present a computational model of spatial navigation comprising different learning mechanisms in mammals, i.e., associative, cognitive mapping and parallel systems. This model is able to reproduce a large number of experimental results in different variants of the Morris water maze task, including standard associative phenomena (spatial generalization gradient and blocking), as well as navigation based on cognitive mapping. Furthermore, we show that competitive and cooperative patterns between different navigation strategies in the model allow to explain previous apparently contradictory results supporting either associative or cognitive mechanisms for spatial learning. The key computational mechanism to reconcile experimental results showing different influences of distal and proximal cues on the behavior, different learning times, and different abilities of individuals to alternatively perform spatial and response strategies, relies in the dynamic coordination of navigation strategies, whose performance is evaluated online with a common currency through a modular approach. We provide a set of concrete experimental predictions to further test the computational model. Overall, this computational work sheds new light on inter-individual differences in navigation learning, and provides a formal and mechanistic approach to test various theories of spatial cognition in mammals.

Human EEG reveals distinct neural correlates of power and precision grasping types.

Hand grasping is a sophisticated motor task that has received much attention by the neuroscientific community, which demonstrated how grasping activates a network involving parietal, pre-motor and motor cortices using fMRI, ECoG, LFPs and spiking activity. Yet, there is a need for a more precise spatio-temporal analysis as it is still unclear how these brain activations over large cortical areas evolve at the sub-second level. In this study, we recorded ten human participants (1 female) performing visually-guided, self-paced reaching and grasping with precision or power grips. Following the results, we demonstrate the existence of neural correlates of grasping from broadband EEG in self-paced conditions and show how neural correlates of precision and power grasps differentially evolve as grasps unfold. 100 ms before the grasp is secured, bilateral parietal regions showed increasingly differential patterns. Afterwards, sustained differences between both grasps occurred over the bilateral motor and parietal regions, and medial pre-frontal cortex. Furthermore, these differences were sufficiently discriminable to allow single-trial decoding with 70% decoding performance. Functional connectivity revealed differences at the network level between grasps in fronto-parietal networks, in terms of upper-alpha cortical oscillatory power with a strong involvement of ipsilateral hemisphere. Our results supported the existence of fronto-parietal recurrent feedback loops, with stronger interactions for precision grips due to the finer motor control required for this grasping type.

Classification of upper limb center-out reaching tasks by means of EEG-based continuous decoding techniques.

BACKGROUND: One of the current challenges in brain-machine interfacing is to characterize and decode upper limb kinematics from brain signals, e.g. to control a prosthetic device. Recent research work states that it is possible to do so based on low frequency EEG components. However, the validity of these results is still a matter of discussion. In this paper, we assess the feasibility of decoding upper limb kinematics from EEG signals in center-out reaching tasks during passive and active movements. METHODS: The decoding of arm movement was performed using a multidimensional linear regression. Passive movements were analyzed using the same methodology to study the influence of proprioceptive sensory feedback in the decoding. Finally, we evaluated the possible advantages of classifying reaching targets, instead of continuous trajectories. RESULTS: The results showed that arm movement decoding was significantly above chance levels. The results also indicated that EEG slow cortical potentials carry significant information to decode active center-out movements. The classification of reached targets allowed obtaining the same conclusions with a very high accuracy. Additionally, the low decoding performance obtained from passive movements suggests that discriminant modulations of low-frequency neural activity are mainly related to the execution of movement while proprioceptive feedback is not sufficient to decode upper limb kinematics. CONCLUSIONS: This paper contributes to the assessment of feasibility of using linear regression methods to decode upper limb kinematics from EEG signals. From our findings, it can be concluded that low frequency bands concentrate most of the information extracted from upper limb kinematics decoding and that decoding performance of active movements is above chance levels and mainly related to the activation of cortical motor areas. We also show that the classification of reached targets from decoding approaches may be a more suitable real-time methodology than a direct decoding of hand position.

Discriminant brain connectivity patterns of performance monitoring at average and single-trial levels.

Electrophysiological and neuroimaging evidence suggest the existence of common mechanisms for monitoring erroneous events, independent of the source of errors. Previous works have described modulations of theta activity in the medial frontal cortex elicited by either self-generated errors or erroneous feedback. In turn, similar patterns have recently been reported to appear after the observation of external errors. We report cross-regional interactions after observation of errors at both average and single-trial levels. We recorded scalp electroencephalography (EEG) signals from 15 subjects while monitoring the movement of a cursor on a computer screen. Connectivity patterns, estimated using multivariate auto-regressive models, show increased error-related modulations of the information transfer in the theta and alpha bands between frontocentral and frontolateral areas. Conversely, a decrease of connectivity in the beta band is also observed. These network patterns are similar to those elicited by self-generated errors. However, since no motor response is required, they appear to be related to intrinsic mechanisms of error processing, instead of being linked to co-activation of motor areas. Noticeably, we demonstrate that cross-regional interaction patterns can be estimated on a trial-by-trial basis. These trial-specific patterns, consistent with the multi-trial analysis, convey discriminant information on whether a trial was elicited by observation of an erroneous action. Overall, our study supports the role of frequency-specific modulations in the medial frontal cortex in coordinating cross-regional activity during cognitive monitoring at a single-trial basis.

Customizing the human-avatar mapping based on EEG error related potentials.

Objective.A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption.Approach.We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion.Main results.Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling 'Plug-and-Play' ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports.Significance.We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.

EEG error-related potentials encode magnitude of errors and individual perceptual thresholds.

Error-related potentials (ErrPs) are a prominent electroencephalogram (EEG) correlate of performance monitoring, and so crucial for learning and adapting our behavior. It is poorly understood whether ErrPs encode further information beyond error awareness. We report an experiment with sixteen participants over three sessions in which occasional visual rotations of varying magnitude occurred during a cursor reaching task. We designed a brain-computer interface (BCI) to detect ErrPs that provided real-time feedback. The individual ErrP-BCI decoders exhibited good transfer across sessions and scalability over the magnitude of errors. A non-linear relationship between the ErrP-BCI output and the magnitude of errors predicts individual perceptual thresholds to detect errors. We also reveal theta-gamma oscillatory coupling that co-varied with the magnitude of the required adjustment. Our findings open new avenues to probe and extend current theories of performance monitoring by incorporating continuous human interaction tasks and analysis of the ErrP complex rather than individual peaks.

EEG signature of breaks in embodiment in VR.

The brain mechanism of embodiment in a virtual body has grown a scientific interest recently, with a particular focus on providing optimal virtual reality (VR) experiences. Disruptions from an embodied state to a less- or non-embodied state, denominated Breaks in Embodiment (BiE), are however rarely studied despite their importance for designing interactions in VR. Here we use electroencephalography (EEG) to monitor the brain's reaction to a BiE, and investigate how this reaction depends on previous embodiment conditions. The experimental protocol consisted of two sequential steps; an induction step where participants were either embodied or non-embodied in an avatar, and a monitoring step where, in some cases, participants saw the avatar's hand move while their hand remained still. Our results show the occurrence of error-related potentials linked to observation of the BiE event in the monitoring step. Importantly, this EEG signature shows amplified potentials following the non-embodied condition, which is indicative of an accumulation of errors across steps. These results provide neurophysiological indications on how progressive disruptions impact the expectation of embodiment for a virtual body.

EEG-Based Brain-Computer Interfaces are Vulnerable to Backdoor Attacks.

Research and development of electroencephalogram (EEG) based brain-computer interfaces (BCIs) have advanced rapidly, partly due to deeper understanding of the brain and wide adoption of sophisticated machine learning approaches for decoding the EEG signals. However, recent studies have shown that machine learning algorithms are vulnerable to adversarial attacks. This paper proposes to use narrow period pulse for poisoning attack of EEG-based BCIs, which makes adversarial attacks much easier to implement. One can create dangerous backdoors in the machine learning model by injecting poisoning samples into the training set. Test samples with the backdoor key will then be classified into the target class specified by the attacker. What most distinguishes our approach from previous ones is that the backdoor key does not need to be synchronized with the EEG trials, making it very easy to implement. The effectiveness and robustness of the backdoor attack approach is demonstrated, highlighting a critical security concern for EEG-based BCIs and calling for urgent attention to address it.

The timing of exploratory decision-making revealed by single-trial topographic EEGanalyses.

Decision-making in an uncertain environment is driven by two major needs: exploring the environment to gather information or exploiting acquired knowledge to maximize reward. The neural processes underlying exploratory decision-making have been mainly studied by means of functional magnetic resonance imaging, overlooking any information about the time when decisions are made. Here, we carried out an electroencephalography (EEG) experiment, in order to detect the time when the brain generators responsible for these decisions have been sufficiently activated to lead to the next decision. Our analyses, based on a classification scheme, extract time-unlocked voltage topographies during reward presentation and use them to predict the type of decisions made on the subsequent trial. Classification accuracy, measured as the area under the Receiver Operator's Characteristic curve was on average 0.65 across 7 subjects. Classification accuracy was above chance levels already after 516 ms on average, across subjects. We speculate that decisions were already made before this critical period, as confirmed by a positive correlation with reaction times across subjects. On an individual subject basis, distributed source estimations were performed on the extracted topographies to statistically evaluate the neural correlates of decision-making. For trials leading to exploration, there was significantly higher activity in dorsolateral prefrontal cortex and the right supramarginal gyrus; areas responsible for modulating behavior under risk and deduction. No area was more active during exploitation. We show for the first time the temporal evolution of differential patterns of brain activation in an exploratory decision-making task on a single-trial basis.

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions With Drones.

In search and rescue missions, drone operations are challenging and cognitively demanding. High levels of cognitive workload can affect rescuers' performance, leading to failure with catastrophic outcomes. To face this problem, we propose a machine learning algorithm for real-time cognitive workload monitoring to understand if a search and rescue operator has to be replaced or if more resources are required. Our multimodal cognitive workload monitoring model combines the information of 25 features extracted from physiological signals, such as respiration, electrocardiogram, photoplethysmogram, and skin temperature, acquired in a noninvasive way. To reduce both subject and day inter-variability of the signals, we explore different feature normalization techniques, and introduce a novel weighted-learning method based on support vector machines suitable for subject-specific optimizations. On an unseen test set acquired from 34 volunteers, our proposed subject-specific model is able to distinguish between low and high cognitive workloads with an average accuracy of 87.3% and 91.2% while controlling a drone simulator using both a traditional controller and a new-generation controller, respectively.

International data governance for neuroscience.

As neuroscience projects increase in scale and cross international borders, different ethical principles, national and international laws, regulations, and policies for data sharing must be considered. These concerns are part of what is collectively called data governance. Whereas neuroscience data transcend borders, data governance is typically constrained within geopolitical boundaries. An international data governance framework and accompanying infrastructure can assist investigators, institutions, data repositories, and funders with navigating disparate policies. Here, we propose principles and operational considerations for how data governance in neuroscience can be navigated at an international scale and highlight gaps, challenges, and opportunities in a global brain data ecosystem. We consider how to approach data governance in a way that balances data protection requirements and the need for open science, so as to promote international collaboration through federated constructs such as the International Brain Initiative (IBI).

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier.

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Invariability of EEG error-related potentials during continuous feedback protocols elicited by erroneous actions at predicted or unpredicted states.

Objective.When humans perceive an erroneous action, an EEG error-related potential (ErrP) is elicited as a neural response. ErrPs have been largely investigated in discrete feedback protocols, where actions are executed at discrete steps, to enable seamless brain-computer interaction. However, there are only a few studies that investigate ErrPs in continuous feedback protocols. The objective of the present study is to better understand the differences between two types of ErrPs elicited during continuous feedback protocols, where errors may occur either at predicted or unpredicted states. We hypothesize that ErrPs of the unpredicted state is associated with longer latency as it requires higher cognitive workload to evaluate actions compared to the predicted states.Approach.Participants monitored the trajectory of an autonomous cursor that occasionally made erroneous actions on its way to the target in two conditions, namely, predicted or unpredicted states. After characterizing the ErrP waveform elicited by erroneous actions in the two conditions, we performed single-trial decoding of ErrPs in both synchronous (i.e. time-locked to the onset of the erroneous action) and asynchronous manner. Furthermore, we explored the possibility to transfer decoders built with data of one of the conditions to the other condition.Main results.As hypothesized, erroneous actions at unpredicted states gave rise to ErrPs with higher latency than erroneous actions at predicted states, a correlate of higher cognitive effort in the former condition. Moreover, ErrP decoders trained in a given condition successfully transferred to the other condition with a slight loss of classification performance. This was the case for synchronous as well as asynchronous ErrP decoding, showing the invariability of ErrPs across conditions.Significance.These results advance the characterization of ErrPs during continuous feedback protocols, enlarging the potential use of ErrPs during natural operation of brain-controlled devices as it is not necessary to have different decoders for each kind of erroneous conditions.

Context-Aware Learning for Generative Models.

This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum-likelihood estimation (MLE) of parameters through expectation-maximization (EM) improves over the regular unsupervised case and can approach the performances of supervised learning, despite the absence of any explicit ground-truth data labeling. By direct application of the missing information principle (MIP), the algorithms' performances are proven to range between the conventional supervised and unsupervised MLE extremities proportionally to the information content of the contextual assistance provided. The acquired benefits regard higher estimation precision, smaller standard errors, faster convergence rates, and improved classification accuracy or regression fitness shown in various scenarios while also highlighting important properties and differences among the outlined situations. Applicability is showcased with three real-world unsupervised classification scenarios employing Gaussian mixture models. Importantly, we exemplify the natural extension of this methodology to any type of generative model by deriving an equivalent context-aware algorithm for variational autoencoders (VAs), thus broadening the spectrum of applicability to unsupervised deep learning with artificial neural networks. The latter is contrasted with a neural-symbolic algorithm exploiting side information.

User Adaptation to Closed-Loop Decoding of Motor Imagery Termination.

One of the most popular methods in non-invasive brain machine interfaces (BMI) relies on the decoding of sensorimotor rhythms associated to sustained motor imagery. Although motor imagery has been intensively studied, its termination is mostly neglected. OBJECTIVE: Here, we provide insights in the decoding of motor imagery termination and investigate the use of such decoder in closed-loop BMI. METHODS: Participants (N = 9) were asked to perform kinesthetic motor imagery of both hands simultaneously cued with a clock indicating the initiation and termination of the action. Using electroencephalogram (EEG) signals, we built a decoder to detect the transition between event-related desynchronization and event-related synchronization. Features for this decoder were correlates of motor termination in the upper µ and ß bands. RESULTS: The decoder reached an accuracy of 76.2% (N = 9), revealing the high robustness of our approach. More importantly, this paper shows that the decoding of motor termination has an intrinsic latency mainly due to the delayed appearance of its correlates. Because the latency was consistent and thus predictable, users were able to compensate it after training. CONCLUSION: Using our decoding system, BMI users were able to adapt their behavior and modulate their sensorimotor rhythm to stop the device (clock) accurately on time. SIGNIFICANCE: These results show the importance of closed-loop evaluations of BMI decoders and open new possibilities for BMI control using decoding of movement termination.

Closed-loop EEG study on visual recognition during driving.

Objective.In contrast to the classical visual brain-computer interface (BCI) paradigms, which adhere to a rigid trial structure and restricted user behavior, electroencephalogram (EEG)-based visual recognition decoding during our daily activities remains challenging. The objective of this study is to explore the feasibility of decoding the EEG signature of visual recognition in experimental conditions promoting our natural ocular behavior when interacting with our dynamic environment.Approach.In our experiment, subjects visually search for a target object among suddenly appearing objects in the environment while driving a car-simulator. Given that subjects exhibit an unconstrained overt visual behavior, we based our study on eye fixation-related potentials (EFRPs). We report on gaze behavior and single-trial EFRP decoding performance (fixations on visually similar target vs. non-target objects). In addition, we demonstrate the application of our approach in a closed-loop BCI setup.Main results.To identify the target out of four symbol types along a road segment, the BCI system integrated decoding probabilities of multiple EFRP and achieved the average online accuracy of 0.37 ± 0.06 (12 subjects), statistically significantly above the chance level. Using the acquired data, we performed a comparative study of classification algorithms (discriminating target vs. non-target) and feature spaces in a simulated online scenario. The EEG approaches yielded similar moderate performances of at most 0.6 AUC, yet statistically significantly above the chance level. In addition, the gaze duration (dwell time) appears to be an additional informative feature in this context.Significance.These results show that visual recognition of sudden events can be decoded during active driving. Therefore, this study lays a foundation for assistive and recommender systems based on the driver's brain signals.

Tiny noise, big mistakes: adversarial perturbations induce errors in brain-computer interface spellers.

An electroencephalogram (EEG)-based brain-computer interface (BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g. amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a computer. Most studies so far focused on making EEG-based BCI spellers faster and more reliable; however, few have considered their security. This study, for the first time, shows that P300 and steady-state visual evoked potential BCI spellers are very vulnerable, i.e. they can be severely attacked by adversarial perturbations, which are too tiny to be noticed when added to EEG signals, but can mislead the spellers to spell anything the attacker wants. The consequence could range from merely user frustration to severe misdiagnosis in clinical applications. We hope our research can attract more attention to the security of EEG-based BCI spellers, and more broadly, EEG-based BCIs, which has received little attention before.

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials.

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user's error expectation of the robot's current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user's preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user's preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.