Customizing the human-avatar mapping based on EEG error related potentials.

Objective.A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption.Approach.We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion.Main results.Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling 'Plug-and-Play' ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports.Significance.We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.

Procedural Motor Memory Deficits in Patients With Long-COVID.

BACKGROUND AND OBJECTIVES: At least 15% of patients who recover from acute severe acute respiratory syndrome coronavirus 2 infection experience lasting symptoms ("Long-COVID") including "brain fog" and deficits in declarative memory. It is not known if Long-COVID affects patients' ability to form and retain procedural motor skill memories. The objective was to determine the ability of patients with Long-COVID to acquire and consolidate a new procedural motor skill over 2 training days. The primary outcome was to determine difference in early learning, measured as the increase in correct sequence typing speed over the initial 11 practice trials of a new skill. The secondary outcomes were initial and final typing speed on days 1 and 2, learning rate, overnight consolidation, and typing accuracy. METHODS: In this prospective, cross-sectional, online, case-control study, participants learned a sequential motor skill over 2 consecutive days (NCT05746624). Patients with Long-COVID (reporting persistent post-coronavirus disease 2019 [COVID-19] symptoms for more than 4 weeks) were recruited at the NIH. Patients were matched one-to-one by age and sex to controls recruited during the pandemic using a crowd-sourcing platform. Selection criteria included age 18-90 years, English speaking, right-handed, able to type with the left hand, denied active fever or respiratory infection, and no previous task exposure. Data were also compared with an age-matched and sex-matched control group who performed the task online before the COVID-19 pandemic (prepandemic controls). RESULTS: In total, 105 of 236 patients contacted agreed to participate and completed the experiment (mean ± SD age 46 ± 12.8 years, 82% female). Both healthy control groups had 105 participants (mean age 46 ± 13.1 and 46 ± 11.9 years, 82% female). Early learning was comparable across groups (Long-COVID: 0.36 ± 0.24 correct sequences/second, pandemic controls: 0.36 ± 0.53 prepandemic controls: 0.38 ± 0.57, patients vs pandemic controls [CI -0.068 to 0.067], vs prepandemic controls [CI -0.084 to 0.052], and between controls [CI -0.083 to 0.053], p = 0.82). Initial and final typing speeds on days 1 and 2 were slower in patients than controls. Patients with Long-COVID showed a significantly reduced overnight consolidation and a nonsignificant trend to reduced learning rates. DISCUSSION: Early learning was comparable in patients with Long-COVID and controls. Anomalous initial performance is consistent with executive dysfunction. Reduction in overnight consolidation may relate to deficits in procedural memory formation.

Inferring individual evaluation criteria for reaching trajectories with obstacle avoidance from EEG signals.

During reaching actions, the human central nerve system (CNS) generates the trajectories that optimize effort and time. When there is an obstacle in the path, we make sure that our arm passes the obstacle with a sufficient margin. This comfort margin varies between individuals. When passing a fragile object, risk-averse individuals may adopt a larger margin by following the longer path than risk-prone people do. However, it is not known whether this variation is associated with a personalized cost function used for the individual optimal control policies and how it is represented in our brain activity. This study investigates whether such individual variations in evaluation criteria during reaching results from differentiated weighting given to energy minimization versus comfort, and monitors brain error-related potentials (ErrPs) evoked when subjects observe a robot moving dangerously close to a fragile object. Seventeen healthy participants monitored a robot performing safe, daring and unsafe trajectories around a wine glass. Each participant displayed distinct evaluation criteria on the energy efficiency and comfort of robot trajectories. The ErrP-BCI outputs successfully inferred such individual variation. This study suggests that ErrPs could be used in conjunction with an optimal control approach to identify the personalized cost used by CNS. It further opens new avenues for the use of brain-evoked potential to train assistive robotic devices through the use of neuroprosthetic interfaces.

EEG error-related potentials encode magnitude of errors and individual perceptual thresholds.

Error-related potentials (ErrPs) are a prominent electroencephalogram (EEG) correlate of performance monitoring, and so crucial for learning and adapting our behavior. It is poorly understood whether ErrPs encode further information beyond error awareness. We report an experiment with sixteen participants over three sessions in which occasional visual rotations of varying magnitude occurred during a cursor reaching task. We designed a brain-computer interface (BCI) to detect ErrPs that provided real-time feedback. The individual ErrP-BCI decoders exhibited good transfer across sessions and scalability over the magnitude of errors. A non-linear relationship between the ErrP-BCI output and the magnitude of errors predicts individual perceptual thresholds to detect errors. We also reveal theta-gamma oscillatory coupling that co-varied with the magnitude of the required adjustment. Our findings open new avenues to probe and extend current theories of performance monitoring by incorporating continuous human interaction tasks and analysis of the ErrP complex rather than individual peaks.

Combined low-frequency brain oscillatory activity and behavior predict future errors in human motor skill.

Human skills are composed of sequences of individual actions performed with utmost precision. When occasional errors occur, they may have serious consequences, for example, when pilots are manually landing a plane. In such cases, the ability to predict an error before it occurs would clearly be advantageous. Here, we asked whether it is possible to predict future errors in a keyboard procedural human motor skill. We report that prolonged keypress transition times (KTTs), reflecting slower speed, and anomalous delta-band oscillatory activity in cingulate-entorhinal-precuneus brain regions precede upcoming errors in skill. Combined anomalous low-frequency activity and prolonged KTTs predicted up to 70% of future errors. Decoding strength (posterior probability of error) increased progressively approaching the errors. We conclude that it is possible to predict future individual errors in skill sequential performance.

EEG signature of breaks in embodiment in VR.

The brain mechanism of embodiment in a virtual body has grown a scientific interest recently, with a particular focus on providing optimal virtual reality (VR) experiences. Disruptions from an embodied state to a less- or non-embodied state, denominated Breaks in Embodiment (BiE), are however rarely studied despite their importance for designing interactions in VR. Here we use electroencephalography (EEG) to monitor the brain's reaction to a BiE, and investigate how this reaction depends on previous embodiment conditions. The experimental protocol consisted of two sequential steps; an induction step where participants were either embodied or non-embodied in an avatar, and a monitoring step where, in some cases, participants saw the avatar's hand move while their hand remained still. Our results show the occurrence of error-related potentials linked to observation of the BiE event in the monitoring step. Importantly, this EEG signature shows amplified potentials following the non-embodied condition, which is indicative of an accumulation of errors across steps. These results provide neurophysiological indications on how progressive disruptions impact the expectation of embodiment for a virtual body.

A highly stable electrode with low electrode-skin impedance for wearable brain-computer interface.

To date, brain-computer interfaces (BCIs) have proved to play a key role in many medical applications, for example, the rehabilitation of stroke patients. For post-stroke rehabilitation, the BCIs require the EEG electrodes to precisely translate the brain signals of patients into intended movements of the paralyzed limb for months. However, the gold standard silver/silver-chloride electrodes cannot satisfy the requirements for long-term stability and preparation-free recording capability in wearable EEG devices, thus limiting the versatility of EEG in wearable BCI applications over time outside the rehabilitation center. Here, we design a long-term stable and low electrode-skin interfacial impedance conductive polymer-hydrogel EEG electrode that maintains a lower impedance value than gel-based electrodes for 29 days. With this technology, EEG-based long-term and wearable BCIs could be realized in the near future. To demonstrate this, our designed electrode is applied for a wireless single-channel EEG device that detects changes in alpha rhythms in eye-open/eye-close conditions. In addition, we validate that the designed electrodes could capture oscillatory rhythms in motor imagery protocols as well as low-frequency time-locked event-related potentials from healthy subjects, with similar or better performance than gel-based electrodes. Finally, we demonstrate the use of the designed electrode in online BCI-based functional electrical stimulation, which could be used for post-stroke rehabilitation.

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials.

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user's error expectation of the robot's current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user's preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user's preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.

Invariability of EEG error-related potentials during continuous feedback protocols elicited by erroneous actions at predicted or unpredicted states.

Objective.When humans perceive an erroneous action, an EEG error-related potential (ErrP) is elicited as a neural response. ErrPs have been largely investigated in discrete feedback protocols, where actions are executed at discrete steps, to enable seamless brain-computer interaction. However, there are only a few studies that investigate ErrPs in continuous feedback protocols. The objective of the present study is to better understand the differences between two types of ErrPs elicited during continuous feedback protocols, where errors may occur either at predicted or unpredicted states. We hypothesize that ErrPs of the unpredicted state is associated with longer latency as it requires higher cognitive workload to evaluate actions compared to the predicted states.Approach.Participants monitored the trajectory of an autonomous cursor that occasionally made erroneous actions on its way to the target in two conditions, namely, predicted or unpredicted states. After characterizing the ErrP waveform elicited by erroneous actions in the two conditions, we performed single-trial decoding of ErrPs in both synchronous (i.e. time-locked to the onset of the erroneous action) and asynchronous manner. Furthermore, we explored the possibility to transfer decoders built with data of one of the conditions to the other condition.Main results.As hypothesized, erroneous actions at unpredicted states gave rise to ErrPs with higher latency than erroneous actions at predicted states, a correlate of higher cognitive effort in the former condition. Moreover, ErrP decoders trained in a given condition successfully transferred to the other condition with a slight loss of classification performance. This was the case for synchronous as well as asynchronous ErrP decoding, showing the invariability of ErrPs across conditions.Significance.These results advance the characterization of ErrPs during continuous feedback protocols, enlarging the potential use of ErrPs during natural operation of brain-controlled devices as it is not necessary to have different decoders for each kind of erroneous conditions.

EEG Sensorimotor Correlates of Speed During Forearm Passive Movements.

Although passive movement therapy has been widely adopted to recover lost motor functions of impaired body parts, the underlying neural mechanisms are still unclear. In this context, fully understanding how the proprioceptive input modulates the brain activity may provide valuable insights. Specifically, it has not been investigated how the speed of motions, passively guided by a haptic device, affects the sensorimotor rhythms (SMR). On the grounds that faster passive motions elicit larger quantity of afferent input, we hypothesize a proportional relationship between localized SMR features and passive movement speed. To address this hypothesis, we conducted an experiment where healthy subjects received passive forearm oscillations at different speed levels while their electroencephalogram was recorded. The mu and beta event related desynchronization (ERD) and beta rebound of both left and right sensorimotor areas are analyzed by linear mixed-effects models. Results indicate that passive movement speed is correlated with the contralateral beta rebound and ipsilateral mu ERD. The former has been previously linked with the processing of proprioceptive afferent input quantity, while the latter with speed-dependent inhibitory processes. This suggests the existence of functionally-distinct frequency-specific neuronal populations associated with passive movements. In future, our findings may guide the design of novel rehabilitation paradigms.

EEG-Based Lower-Limb Movement Onset Decoding: Continuous Classification and Asynchronous Detection.

Brain-machine interfaces have been used to incorporate the user intention to trigger robotic devices by decoding movement onset from electroencephalography. Active neural participation is crucial to promote brain plasticity thus to enhance the opportunity of motor recovery. This paper presents the decoding of lower-limb movement-related cortical potentials with continuous classification and asynchronous detection. We executed experiments in a customized gait trainer, where 10 healthy subjects performed self-initiated ankle plantar flexion. We further analyzed the features, evaluated the impact of the limb side, and compared the proposed framework with other typical decoding methods. No significant differences were observed between the left and right legs in terms of neural signatures of movement and classification performance. We obtained a higher true positive rate, lower false positives, and comparable latencies with respect to the existing online detection methods. This paper demonstrates the feasibility of the proposed framework to build a closed-loop gait trainer. Potential applications include gait training neurorehabilitation in clinical trials.