EEG decoding with spatiotemporal convolutional neural network for visualization and closed-loop control of sensorimotor activities: A simultaneous EEG-fMRI study.

Closed-loop neurofeedback training utilizes neural signals such as scalp electroencephalograms (EEG) to manipulate specific neural activities and the associated behavioral performance. A spatiotemporal filter for high-density whole-head scalp EEG using a convolutional neural network can overcome the ambiguity of the signaling source because each EEG signal includes information on the remote regions. We simultaneously acquired EEG and functional magnetic resonance images in humans during the brain-computer interface (BCI) based neurofeedback training and compared the reconstructed and modeled hemodynamic responses of the sensorimotor network. Filters constructed with a convolutional neural network captured activities in the targeted network with spatial precision and specificity superior to those of the EEG signals preprocessed with standard pipelines used in BCI-based neurofeedback paradigms. The middle layers of the trained model were examined to characterize the neuronal oscillatory features that contributed to the reconstruction. Analysis of the layers for spatial convolution revealed the contribution of distributed cortical circuitries to reconstruction, including the frontoparietal and sensorimotor areas, and those of temporal convolution layers that successfully reconstructed the hemodynamic response function. Employing a spatiotemporal filter and leveraging the electrophysiological signatures of the sensorimotor excitability identified in our middle layer analysis would contribute to the development of a further effective neurofeedback intervention.

Rapid-IAF: Rapid Identification of Individual Alpha Frequency in EEG Data Using Sequential Bayesian Estimation.

Rapid and robust identification of the individual alpha frequency (IAF) in electroencephalogram (EEG) is an essential factor for successful brain-computer interface (BCI) use. Here we demonstrate an algorithm to determine the IAF from short-term resting-state scalp EEG data. First, we outlined the algorithm to determine IAF from short-term resting scalp EEG data and evaluated its reliability using a large-scale dataset of scalp EEG during motor imagery-based BCI use and independent dataset for generalizability confirmation (N = 147). Next, we characterized the relationship between IAF and responsive frequency band of sensorimotor rhythm, which exhibits prominent event-related desynchronization (SMR-ERD) while attempting unilateral and movement. The proposed sequential Bayesian estimation algorithm (Rapid-IAF) determined IAF from less than 26-second resting EEG data among 95% of participants, indicating a clear advance over the conventional methods, which uses 2-15 minutes of data in previous literatures. We confirmed that the determined IAF corresponded to the frequency of SMR, which exhibits the most prominent event-related desynchronization during BCI use (individual SMR-ERD frequency, ISF). Moreover, intraclass correlation revealed that the estimated IAF was more stable than ISF across sessions, suggesting its reliability and utility for robust BCI use without intermittent recalibration. In summary, our method rapidly and reliably determined IAF compared to the conventional method using the spectral power change based on task-related response. The method can be utilized to quick BCI initialization. The demonstration of rapid, task-free parametrization of individual variability of neural responses would be of importance for future BCI systems including neural communication via a cursor, an avatar or robots, and closed-loop neurofeedback training.

Two common issues in synchronized multimodal recordings with EEG: Jitter and latency.

Multimodal recording using electroencephalogram (EEG) and other biological signals (e.g., muscle activities, eye movement, pupil diameters, or body kinematics data) is ubiquitous in human neuroscience research. However, the precise time alignment of multiple data from heterogeneous sources (i.e., devices) is often arduous due to variable recording parameters of commercially available research devices and complex experimental setups. In this review, we introduced the versatility of a Lab Streaming Layer (LSL)-based application that can overcome two common issues in measuring multimodal data: jitter and latency. We discussed the issues of jitter and latency in multimodal recordings and the benefits of time-synchronization when recording with multiple devices. In addition, a computer simulation was performed to highlight how the millisecond-order jitter readily affects the signal-to-noise ratio of the electrophysiological outcome. Together, we argue that the LSL-based system can be used for research requiring precise time-alignment of datasets. Studies that detect stimulus-induced transient neural responses or test hypotheses regarding temporal relationships of different functional aspects with multimodal data would benefit most from LSL-based systems.

High-density scalp electroencephalogram dataset during sensorimotor rhythm-based brain-computer interfacing.

Real-time functional imaging of human neural activity and its closed-loop feedback enable voluntary control of targeted brain regions. In particular, a brain-computer interface (BCI), a direct bridge of neural activities and machine actuation is one promising clinical application of neurofeedback. Although a variety of studies reported successful self-regulation of motor cortical activities probed by scalp electroencephalogram (EEG), it remains unclear how neurophysiological, experimental conditions or BCI designs influence variability in BCI learning. Here, we provide the EEG data during using BCIs based on sensorimotor rhythm (SMR), consisting of 4 separate datasets. All EEG data were acquired with a high-density scalp EEG setup containing 128 channels covering the whole head. All participants were instructed to perform motor imagery of right-hand movement as the strategy to control BCIs based on the task-related power attenuation of SMR magnitude, that is event-related desynchronization. This dataset would allow researchers to explore the potential source of variability in BCI learning efficiency and facilitate follow-up studies to test the explicit hypotheses explored by the dataset.

Thirty-minute motor imagery exercise aided by EEG sensorimotor rhythm neurofeedback enhances morphing of sensorimotor cortices: a double-blind sham-controlled study.

Neurofeedback training using electroencephalogram (EEG)-based brain-computer interfaces (BCIs) combined with mental rehearsals of motor behavior has demonstrated successful self-regulation of motor cortical excitability. However, it remains unclear whether the acquisition of skills to voluntarily control neural excitability is accompanied by structural plasticity boosted by neurofeedback. Here, we sought short-term changes in cortical structures induced by 30 min of BCI-based neurofeedback training, which aimed at the regulation of sensorimotor rhythm (SMR) in scalp EEG. When participants performed kinesthetic motor imagery of right finger movement with online feedback of either event-related desynchronisation (ERD) of SMR magnitude from the contralateral sensorimotor cortex (SM1) or those from other participants (i.e. placebo), the learning rate of SMR-ERD control was significantly different. Although overlapped structural changes in gray matter volumes were found in both groups, significant differences revealed by group-by-group comparison were spatially different; whereas the veritable neurofeedback group exhibited sensorimotor area-specific changes, the placebo exhibited spatially distributed changes. The white matter change indicated a significant decrease in the corpus callosum in the verum group. Furthermore, the learning rate of SMR regulation was correlated with the volume changes in the ipsilateral SM1, suggesting the involvement of interhemispheric motor control circuitries in BCI control tasks.

Beta rhythmicity in human motor cortex reflects neural population coupling that modulates subsequent finger coordination stability.

Human behavior is not performed completely as desired, but is influenced by the inherent rhythmicity of the brain. Here we show that anti-phase bimanual coordination stability is regulated by the dynamics of pre-movement neural oscillations in bi-hemispheric primary motor cortices (M1) and supplementary motor area (SMA). In experiment 1, pre-movement bi-hemispheric M1 phase synchrony in beta-band (M1-M1 phase synchrony) was online estimated from 129-channel scalp electroencephalograms. Anti-phase bimanual tapping preceded by lower M1-M1 phase synchrony exhibited significantly longer duration than tapping preceded by higher M1-M1 phase synchrony. Further, the inter-individual variability of duration was explained by the interaction of pre-movement activities within the motor network; lower M1-M1 phase synchrony and spectral power at SMA were associated with longer duration. The necessity of cortical interaction for anti-phase maintenance was revealed by sham-controlled repetitive transcranial magnetic stimulation over SMA in another experiment. Our results demonstrate that pre-movement cortical oscillatory coupling within the motor network unknowingly influences bimanual coordination performance in humans after consolidation, suggesting the feasibility of augmenting human motor ability by covertly monitoring preparatory neural dynamics.

De Novo Brain-Computer Interfacing Deforms Manifold of Populational Neural Activity Patterns in Human Cerebral Cortex.

Human brains are capable of modulating innate activities to adapt to novel environments and tasks; for sensorimotor neural system this means acquisition of a rich repertoire of activity patterns that improve behavioral performance. To directly map the process of acquiring the neural repertoire during tasks onto performance improvement, we analyzed net neural populational activity during the learning of its voluntary modulation by brain-computer interface (BCI) operation in female and male humans. The recorded whole-head high-density scalp electroencephalograms (EEGs) were subjected to dimensionality reduction algorithm to capture changes in cortical activity patterns represented by the synchronization of neuronal oscillations during adaptation. Although the preserved variance of targeted features in the reduced dimensions was 20%, we found systematic interactions between the activity patterns and BCI classifiers that detected motor attempt; the neural manifold derived in the embedded space was stretched along with motor-related features of EEG by model-based fixed classifiers but not with adaptive classifiers that were constantly recalibrated to user activity. Moreover, the manifold was deformed to be orthogonal to the boundary by de novo classifiers with a fixed decision boundary based on biologically unnatural features. Collectively, the flexibility of human cortical signaling patterns (i.e., neural plasticity) is only induced by operation of a BCI whose classifier required fixed activities, and the adaptation could be induced even the requirement is not consistent with biologically natural responses. These principles of neural adaptation at a macroscopic level may underlie the ability of humans to learn wide-ranging behavioral repertoires and adapt to novel environments.

Use of common average reference and large-Laplacian spatial-filters enhances EEG signal-to-noise ratios in intrinsic sensorimotor activity.

BACKGROUND: Oscillations in the resting-state scalp electroencephalogram (EEG) represent various intrinsic brain activities. One of the characteristic EEG oscillations is the sensorimotor rhythm (SMR)-with its arch-shaped waveform in alpha- and betabands-that reflect sensorimotor activity. The representation of sensorimotor activity by the SMR depends on the signal-to-noise ratio achieved by EEG spatial filters. NEW METHOD: We employed simultaneous recording of EEG and functional magnetic resonance imaging, and 10-min resting-state brain activities were recorded in 19 healthy volunteers. To compare the EEG spatial-filtering methods commonly used for extracting sensorimotor cortical activities, we assessed nine different spatial-filters: a default reference of EEG amplifier system, a common average reference (CAR), small-, middle- and large-Laplacian filters, and four types of bipolar manners (C3-Cz, C3-F3, C3-P3, and C3-T7). We identified the brain region that correlated with the EEG-SMR power obtained after each spatial-filtering method was applied. Subsequently, we calculated the proportion of the significant voxels in the sensorimotor cortex as well as the sensorimotor occupancy in all significant regions to examine the sensitivity and specificity of each spatial-filter. RESULTS: The CAR and large-Laplacian spatial-filters were superior at improving the signal-to-noise ratios for extracting sensorimotor activity from the EEG-SMR signal. COMPARISON WITH EXISTING METHODS: Our results are consistent with the spatial-filter selection to extract task-dependent activation for better control of EEG-SMR-based interventions. Our approach has the potential to identify the optimal spatial-filter for EEG-SMR. CONCLUSIONS: Evaluating spatial-filters for extracting spontaneous sensorimotor activity from the EEG is a useful procedure for constructing more effective EEG-SMR-based interventions.

Scalp electroencephalograms over ipsilateral sensorimotor cortex reflect contraction patterns of unilateral finger muscles.

A variety of neural substrates are implicated in the initiation, coordination, and stabilization of voluntary movements underpinned by adaptive contraction and relaxation of agonist and antagonist muscles. To achieve such flexible and purposeful control of the human body, brain systems exhibit extensive modulation during the transition from resting state to motor execution and to maintain proper joint impedance. However, the neural structures contributing to such sensorimotor control under unconstrained and naturalistic conditions are not fully characterized. To elucidate which brain regions are implicated in generating and coordinating voluntary movements, we employed a physiologically inspired, two-stage method to decode relaxation and three patterns of contraction in unilateral finger muscles (i.e., extension, flexion, and co-contraction) from high-density scalp electroencephalograms (EEG). The decoder consisted of two parts employed in series. The first discriminated between relaxation and contraction. If the EEG data were discriminated as contraction, the second stage then discriminated among the three contraction patterns. Despite the difficulty in dissociating detailed contraction patterns of muscles within a limb from scalp EEG signals, the decoder performance was higher than chance-level by 2-fold in the four-class classification. Moreover, weighted features in the trained decoders revealed EEG features differentially contributing to decoding performance. During the first stage, consistent with previous reports, weighted features were localized around sensorimotor cortex (SM1) contralateral to the activated fingers, while those during the second stage were localized around ipsilateral SM1. The loci of these weighted features suggested that the coordination of unilateral finger muscles induced different signaling patterns in ipsilateral SM1 contributing to motor control. Weighted EEG features enabled a deeper understanding of human sensorimotor processing as well as of a more naturalistic control of brain-computer interfaces.