Getting off track: Cortical feedback processing network modulated by continuous error signal during target-feedback mismatch.

Performance monitoring and feedback processing - especially in the wake of erroneous outcomes - represent a crucial aspect of everyday life, allowing us to deal with imminent threats in the short term but also promoting necessary behavioral adjustments in the long term to avoid future conflicts. Over the last thirty years, research extensively analyzed the neural correlates of processing discrete error stimuli, unveiling the error-related negativity (ERN) and error positivity (Pe) as two main components of the cognitive response. However, the connection between the ERN/Pe and distinct stages of error processing, ranging from action monitoring to subsequent corrective behavior, remains ambiguous. Furthermore, mundane actions such as steering a vehicle already transgress the scope of discrete erroneous events and demand fine-tuned feedback control, and thus, the processing of continuous error signals - a topic scarcely researched at present. We analyzed two electroencephalography datasets to investigate the processing of continuous erroneous signals during a target tracking task, employing feedback in various levels and modalities. We observed significant differences between correct (slightly delayed) and erroneous feedback conditions in the larger one of the two datasets that we analyzed, both in sensor and source space. Furthermore, we found strong error-induced modulations that appeared consistent across datasets and error conditions, indicating a clear order of engagement of specific brain regions that correspond to individual components of error processing.

Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome.

Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.

Transfer Learning in Trajectory Decoding: Sensor or Source Space?

In this study, across-participant and across-session transfer learning was investigated to minimize the calibration time of the brain-computer interface (BCI) system in the context of continuous hand trajectory decoding. We reanalyzed data from a study with 10 able-bodied participants across three sessions. A leave-one-participant-out (LOPO) model was utilized as a starting model. Recursive exponentially weighted partial least squares regression (REW-PLS) was employed to overcome the memory limitation due to the large pool of training data. We considered four scenarios: generalized with no update (Gen), generalized with cumulative update (GenC), and individual models with cumulative (IndC) and non-cumulative (Ind) updates, with each one trained with sensor-space features or source-space features. The decoding performance in generalized models (Gen and GenC) was lower than the chance level. In individual models, the cumulative update (IndC) showed no significant improvement over the non-cumulative model (Ind). The performance showed the decoder's incapability to generalize across participants and sessions in this task. The results suggested that the best correlation could be achieved with the sensor-space individual model, despite additional anatomical information in the source-space features. The decoding pattern showed a more localized pattern around the precuneus over three sessions in Ind models.

Distinct cortical networks for hand movement initiation and directional processing: An EEG study.

Movement preparation and initiation have been shown to involve large scale brain networks. Recent findings suggest that movement preparation and initiation are represented in functionally distinct cortical networks. In electroencephalographic (EEG) recordings, movement initiation is reflected as a strong negative potential at medial central channels that is phase-locked to the movement onset - the movement-related cortical potential (MRCP). Movement preparation describes the process of transforming high level movement goals to low level commands. An integral part of this transformation process is directional processing (i.e., where to move). The processing of movement direction during visuomotor and oculomotor tasks is associated with medial parieto-occipital cortex (PO) activity, phase-locked to the presentation of potential movement goals. We surmised that the network generating the MRCP (movement initiation) would encode less information about movement direction than the parieto-occipital network processing movement direction. Here, we studied delta band EEG activity during center-out reaching movements (2D; 4 directions) in visuomotor and oculomotor tasks. In 15 healthy participants, we found a consistent representation of movement direction in PO 300-400 â€‹ms after the direction cue irrespective of the task. Despite generating the MRCP, sensorimotor areas (SM) encoded less information about the movement direction than PO. Moreover, the encoded directional information in SM was less consistent across participants and specific to the visuomotor task. In a classification approach, we could infer the four movement directions from the delta band EEG activity with moderate accuracies up to 55.9%. The accuracies for cue-aligned data were significantly higher than for movement onset-aligned data in either task, which also suggests a stronger representation of movement direction during movement preparation. Therefore, we present direct evidence that EEG delta band amplitude modulations carry information about both arm movement initiation and movement direction, and that they are represented in two distinct cortical networks.

Corneo-retinal-dipole and eyelid-related eye artifacts can be corrected offline and online in electroencephalographic and magnetoencephalographic signals.

Eye movements and blinks contaminate electroencephalographic (EEG) and magnetoencephalographic (MEG) activity. As the eye moves, the corneo-retinal dipole (CRD) and eyelid introduce potential/field changes in the M/EEG activity. These eye artifacts can affect a brain-computer interface and thereby impinge on neurofeedback quality. Here, we introduce the sparse generalized eye artifact subspace subtraction (SGEYESUB) algorithm that can correct these eye artifacts offline and in real time. We provide an open source reference implementation of the algorithm and the paradigm to obtain calibration data. Once the algorithm is fitted to calibration data (approx. 5 â€‹min), the eye artifact correction reduces to a matrix multiplication. We compared SGEYESUB with 4 state-of-the-art algorithms using M/EEG activity of 69 participants. SGEYESUB achieved the best trade-off between correcting the eye artifacts and preserving brain activity. Residual correlations between the corrected M/EEG channels and the eye artifacts were below 0.1. Error-related and movement-related cortical potentials were attenuated by less than 0.5 â€‹µV. Our results furthermore demonstrate that CRD and eyelid-related artifacts can be assumed to be stationary for at least 1-1.5 â€‹h, validating the feasibility of our approach in offline and online eye artifact correction.

Online Reduction of Artifacts in EEG of Simultaneous EEG-fMRI Using Reference Layer Adaptive Filtering (RLAF).

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) allow us to study the active human brain from two perspectives concurrently. Signal processing based artifact reduction techniques are mandatory for this, however, to obtain reasonable EEG quality in simultaneous EEG-fMRI. Current artifact reduction techniques like average artifact subtraction (AAS), typically become less effective when artifact reduction has to be performed on-the-fly. We thus present and evaluate a new technique to improve EEG quality online. This technique adds up with online AAS and combines a prototype EEG-cap for reference recordings of artifacts, with online adaptive filtering and is named reference layer adaptive filtering (RLAF). We found online AAS + RLAF to be highly effective in improving EEG quality. Online AAS + RLAF outperformed online AAS and did so in particular online in terms of the chosen performance metrics, these being specifically alpha rhythm amplitude ratio between closed and opened eyes (3-45% improvement), signal-to-noise-ratio of visual evoked potentials (VEP) (25-63% improvement), and VEPs variability (16-44% improvement). Further, we found that EEG quality after online AAS + RLAF is occasionally even comparable with the offline variant of AAS at a 3T MRI scanner. In conclusion RLAF is a very effective add-on tool to enable high quality EEG in simultaneous EEG-fMRI experiments, even when online artifact reduction is necessary.

Imagine squeezing a cactus: Cortical activation during affective motor imagery measured by functional near-infrared spectroscopy.

The activation of different brain areas during kinaesthetic and visual motor imagery has been extensively studied, whereas little is known about affective motor imagery, i.e. the imagery of pleasant/unpleasant movements. In the present neuroimaging study we investigated cortical activation of kinaesthetic motor imagery (KMI) based on emotional stimulus content by means of functional near infrared spectroscopy (fNIRS). Twenty healthy adult participants were instructed to imagine affective, and neutral motor tasks while multichannel fNIRS was recorded simultaneously. In the affective MI condition they had to imagine e.g. squeezing a cactus with their right hand several times, eliciting an unpleasant emotion. In the neutral condition their task was to imagine squeezing a ball. Significant differences in oxy-hemoglobin [oxy-Hb] concentration changes during KMI including affective objects in different brain regions were found. Specifically activation in left parietal and frontal regions was increased during the imagery of squeezing a cactus which induced a painful feeling. Both areas are also involved in the perception of pain and commonly labelled as parts of the "pain matrix". Our study provides novel insights in cortical activation patterns during affective motor imagery and its psychological and cognitive mechanisms underlying pain experience.

Action affordances and visuo-spatial complexity in motor imagery: An fMRI study.

Imagining a complex action requires not only motor-related processing but also visuo-spatial imagery. In the current study, we examined visuo-spatial complexity and action affordances in motor imagery (MI). Using functional magnetic resonance imaging, we investigated the neural activity in MI of reach-to-grasp movements of the right hand in five conditions. Thirty participants were scanned while imagining grasping an everyday object, grasping a geometrical shape, grasping next to an everyday object, grasping next to a geometrical shape, and grasping at nothing (no object involved). We found that MI of grasping next to an object recruited the visuo-spatial cognition network including posterior parietal and premotor regions more strongly than MI of grasping an object. This indicates that grasping next to an object requires additional processing resources rendering MI more complex. MI of a grasping movement involving a familiar everyday object compared to a geometrical shape yielded stronger activation in motor-related regions, including the bilateral supplementary motor area. This activation might be due to inhibitory processes preventing motor execution of motor scripts evoked by everyday objects (action affordances). Our results indicate that visuo-spatial cognition plays a significant role in MI.

EEG Oscillations Are Modulated in Different Behavior-Related Networks during Rhythmic Finger Movements.

Sequencing and timing of body movements are essential to perform motoric tasks. In this study, we investigate the temporal relation between cortical oscillations and human motor behavior (i.e., rhythmic finger movements). High-density EEG recordings were used for source imaging based on individual anatomy. We separated sustained and movement phase-related EEG source amplitudes based on the actual finger movements recorded by a data glove. Sustained amplitude modulations in the contralateral hand area show decrease for α (10-12 Hz) and ß (18-24 Hz), but increase for high γ (60-80 Hz) frequencies during the entire movement period. Additionally, we found movement phase-related amplitudes, which resembled the flexion and extension sequence of the fingers. Especially for faster movement cadences, movement phase-related amplitudes included high ß (24-30 Hz) frequencies in prefrontal areas. Interestingly, the spectral profiles and source patterns of movement phase-related amplitudes differed from sustained activities, suggesting that they represent different frequency-specific large-scale networks. First, networks were signified by the sustained element, which statically modulate their synchrony levels during continuous movements. These networks may upregulate neuronal excitability in brain regions specific to the limb, in this study the right hand area. Second, movement phase-related networks, which modulate their synchrony in relation to the movement sequence. We suggest that these frequency-specific networks are associated with distinct functions, including top-down control, sensorimotor prediction, and integration. The separation of different large-scale networks, we applied in this work, improves the interpretation of EEG sources in relation to human motor behavior. SIGNIFICANCE STATEMENT: EEG recordings provide high temporal resolution suitable to relate cortical oscillations to actual movements. Investigating EEG sources during rhythmic finger movements, we distinguish sustained from movement phase-related amplitude modulations. We separate these two EEG source elements motivated by our previous findings in gait. Here, we found two types of large-scale networks, representing the right fingers in distinction from the time sequence of the movements. These findings suggest that EEG source amplitudes reconstructed in a cortical patch are the superposition of these simultaneously present network activities. Separating these frequency-specific networks is relevant for studying function and possible dysfunction of the cortical sensorimotor system in humans as well as to provide more advanced features for brain-computer interfaces.

EEG neural correlates of goal-directed movement intention.

Using low-frequency time-domain electroencephalographic (EEG) signals we show, for the same type of upper limb movement, that goal-directed movements have different neural correlates than movements without a particular goal. In a reach-and-touch task, we explored the differences in the movement-related cortical potentials (MRCPs) between goal-directed and non-goal-directed movements. We evaluated if the detection of movement intention was influenced by the goal-directedness of the movement. In a single-trial classification procedure we found that classification accuracies are enhanced if there is a goal-directed movement in mind. Furthermore, by using the classifier patterns and estimating the corresponding brain sources, we show the importance of motor areas and the additional involvement of the posterior parietal lobule in the discrimination between goal-directed movements and non-goal-directed movements. We discuss next the potential contribution of our results on goal-directed movements to a more reliable brain-computer interface (BCI) control that facilitates recovery in spinal-cord injured or stroke end-users.

Force related hemodynamic responses during execution and imagery of a hand grip task: A functional near infrared spectroscopy study.

We examined force related hemodynamic changes during the performance of a motor execution (ME) and motor imagery (MI) task by means of multichannel functional near infrared spectroscopy (fNIRS). The hemodynamic responses of fourteen healthy participants were measured while they performed a hand grip execution or imagery task with low and high grip forces. We found an overall higher increase of [oxy-Hb] concentration changes during ME for both grip forces but with a delayed peak maximum for the lower grip force. During the MI task with lower grip force, the [oxy-Hb] level increases are stronger compared to the MI with higher grip force. The facilitation in performing MI with higher grip strength might thus indicate less inhibition of the actual motor act which could also explain the later increase onset of [oxy-Hb] in the ME task with the lower grip force. Our results suggest that execution and imagery of a hand grip task with high and low grip forces, leads to different cortical activation patterns. Since impaired control of grip forces during object manipulation in particular is one aspect of fine motor control deficits after stroke, our study will contribute to future rehabilitation programs enhancing patient's grip force control.

High and low gamma EEG oscillations in central sensorimotor areas are conversely modulated during the human gait cycle.

Investigating human brain function is essential to develop models of cortical involvement during walking. Such models could advance the analysis of motor impairments following brain injuries (e.g., stroke) and may lead to novel rehabilitation approaches. In this work, we applied high-density EEG source imaging based on individual anatomy to enable neuroimaging during walking. To minimize the impact of muscular influence on EEG recordings we introduce a novel artifact correction method based on spectral decomposition. High γ oscillations (>60Hz) were previously reported to play an important role in motor control. Here, we investigate high γ amplitudes while focusing on two different aspects of a walking experiment, namely the fact that a person walks and the rhythmicity of walking. We found that high γ amplitudes (60-80Hz), located focally in central sensorimotor areas, were significantly increased during walking compared to standing. Moreover, high γ (70-90Hz) amplitudes in the same areas are modulated in relation to the gait cycle. Since the spectral peaks of high γ amplitude increase and modulation do not match, it is plausible that these two high γ elements represent different frequency-specific network interactions. Interestingly, we found high γ (70-90Hz) amplitudes to be coupled to low γ (24-40Hz) amplitudes, which both are modulated in relation to the gait cycle but conversely to each other. In summary, our work is a further step towards modeling cortical involvement during human upright walking.

Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG.

The present study aims to gain insights into the effects of training with a motor imagery (MI)-based brain-computer interface (BCI) on activation patterns of the sensorimotor cortex. We used functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) to investigate long-term training effects across 10 sessions using a 2-class (right hand and feet) MI-based BCI in fifteen subjects. In the course of the training a significant enhancement of activation pattern emerges, represented by an [oxy-Hb] increase in fNIRS and a stronger event-related desynchronization in the upper ß-frequency band in the EEG. These effects were only visible in participants with relatively low BCI performance (mean accuracy ≤ 70%). We found that training with an MI-based BCI affects cortical activation patterns especially in users with low BCI performance. Our results may serve as a valuable contribution to the field of BCI research and provide information about the effects that training with an MI-based BCI has on cortical activation patterns. This might be useful for clinical applications of BCI which aim at promoting and guiding neuroplasticity.

Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing.

Brain-computer interfaces (BCIs) can translate brain signals directly into commands for external devices. Electroencephalography (EEG)-based BCIs mostly rely on the classification of discrete mental states, leading to unintuitive control. The ERC-funded project "Feel Your Reach" aimed to establish a novel framework based on continuous decoding of hand/arm movement intention, for a more natural and intuitive control. Over the years, we investigated various aspects of natural control, however, the individual components had not yet been integrated. Here, we present a first implementation of the framework in a comprehensive online study, combining (i) goal-directed movement intention, (ii) trajectory decoding, and (iii) error processing in a unique closed-loop control paradigm. Testing involved twelve able-bodied volunteers, performing attempted movements, and one spinal cord injured (SCI) participant. Similar movement-related cortical potentials and error potentials to previous studies were revealed, and the attempted movement trajectories were overall reconstructed. Source analysis confirmed the involvement of sensorimotor and posterior parietal areas for goal-directed movement intention and trajectory decoding. The increased experiment complexity and duration led to a decreased performance than each single BCI. Nevertheless, the study contributes to understanding natural motor control, providing insights for more intuitive strategies for individuals with motor impairments.

Multimodal decoding of error processing in a virtual reality flight simulation.

Technological advances in head-mounted displays (HMDs) facilitate the acquisition of physiological data of the user, such as gaze, pupil size, or heart rate. Still, interactions with such systems can be prone to errors, including unintended behavior or unexpected changes in the presented virtual environments. In this study, we investigated if multimodal physiological data can be used to decode error processing, which has been studied, to date, with brain signals only. We examined the feasibility of decoding errors solely with pupil size data and proposed a hybrid decoding approach combining electroencephalographic (EEG) and pupillometric signals. Moreover, we analyzed if hybrid approaches can improve existing EEG-based classification approaches and focused on setups that offer increased usability for practical applications, such as the presented game-like virtual reality flight simulation. Our results indicate that classifiers trained with pupil size data can decode errors above chance. Moreover, hybrid approaches yielded improved performance compared to EEG-based decoders in setups with a reduced number of channels, which is crucial for many out-of-the-lab scenarios. These findings contribute to the development of hybrid brain-computer interfaces, particularly in combination with wearable devices, which allow for easy acquisition of additional physiological data.

Methods for motion artifact reduction in online brain-computer interface experiments: a systematic review.

Brain-computer interfaces (BCIs) have emerged as a promising technology for enhancing communication between the human brain and external devices. Electroencephalography (EEG) is particularly promising in this regard because it has high temporal resolution and can be easily worn on the head in everyday life. However, motion artifacts caused by muscle activity, fasciculation, cable swings, or magnetic induction pose significant challenges in real-world BCI applications. In this paper, we present a systematic review of methods for motion artifact reduction in online BCI experiments. Using the PRISMA filter method, we conducted a comprehensive literature search on PubMed, focusing on open access publications from 1966 to 2022. We evaluated 2,333 publications based on predefined filtering rules to identify existing methods and pipelines for motion artifact reduction in EEG data. We present a lookup table of all papers that passed the defined filters, all used methods, and pipelines and compare their overall performance and suitability for online BCI experiments. We summarize suitable methods, algorithms, and concepts for motion artifact reduction in online BCI applications, highlight potential research gaps, and discuss existing community consensus. This review aims to provide a comprehensive overview of the current state of the field and guide researchers in selecting appropriate methods for motion artifact reduction in online BCI experiments.

Decoding movement kinematics from EEG using an interpretable convolutional neural network.

Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.

Neural and Pupillometric Correlates of Error Perception in an Immersive VR Flight Simulation.

Algorithms detecting erroneous events, as used in brain-computer interfaces, usually rely solely on neural correlates of error perception. The increasing availability of wearable displays with built-in pupillometric sensors enables access to additional physiological data, potentially improving error detection. Hence, we measured both electroencephalographic (EEG) and pupillometric signals of 19 participants while performing a navigation task in an immersive virtual reality (VR) setting. We found EEG and pupillometric correlates of error perception and significant differences between distinct error types. Further, we found that actively performing tasks delays error perception. We believe that the results of this work could contribute to improving error detection, which has rarely been studied in the context of immersive VR.

Videoconference fatigue from a neurophysiological perspective: experimental evidence based on electroencephalography (EEG) and electrocardiography (ECG).

In the recent past, many organizations and people have substituted face-to-face meetings with videoconferences. Among others, tools like Zoom, Teams, and Webex have become the "new normal" of human social interaction in many domains (e.g., business, education). However, this radical adoption and extensive use of videoconferencing tools also has a dark side, referred to as videoconference fatigue (VCF). To date only self-report evidence has shown that VCF is a serious issue. However, based on self-reports alone it is hardly possible to provide a comprehensive understanding of a cognitive phenomenon like VCF. Against this background, we examined VCF also from a neurophysiological perspective. Specifically, we collected and analyzed electroencephalography (continuous and event-related) and electrocardiography (heart rate and heart rate variability) data to investigate whether VCF can also be proven on a neurophysiological level. We conducted a laboratory experiment based on a within-subjects design (N = 35). The study context was a university lecture, which was given in a face-to-face and videoconferencing format. In essence, the neurophysiological data-together with questionnaire data that we also collected-show that 50 min videoconferencing, if compared to a face-to-face condition, results in changes in the human nervous system which, based on existing literature, can undoubtedly be interpreted as fatigue. Thus, individuals and organizations must not ignore the fatigue potential of videoconferencing. A major implication of our study is that videoconferencing should be considered as a possible complement to face-to-face interaction, but not as a substitute.

Relationship between electrical brain responses to motor imagery and motor impairment in stroke.

BACKGROUND AND PURPOSE: New strategies like motor imagery based brain-computer interfaces, which use brain signals such as event-related desynchronization (ERD) or event-related synchronization (ERS) for motor rehabilitation after a stroke, are undergoing investigation. However, little is known about the relationship between ERD and ERS patterns and the degree of stroke impairment. The aim of this work was to clarify this relationship. METHODS: EEG during motor imagery and execution were measured in 29 patients with first-ever monolateral stroke causing any degree of motor deficit in the upper limb. The strength and laterality of the ERD or ERS patterns were correlated with the scores of the European Stroke Scale, the Medical Research Council, and the Modified Ashworth Scale. RESULTS: Mean age of the patients was 58 ± 15 years; mean time from the incident was 4 ± 4 months. Stroke lesions were cortical (n=8), subcortical (n=11), or mixed (n=10), attributable to either an ischemic event (n=26) or a hemorrhage (n=3), affecting the right (n=16) or left (n=13) hemisphere. Higher impairment was related to stronger ERD in the unaffected hemisphere and higher spasticity was related to stronger ERD in the affected hemisphere. Both were related to a relatively stronger ERS in the affected hemisphere. CONCLUSIONS: The results of this study may have implications for the design of potential poststroke rehabilitation interventions based on brain-computer interface technologies that use neurophysiological signals like ERD or ERS as neural substrates for the mutual interaction between brain and machine and, ultimately, help stroke patients to regain motor control.