EEG-VTTCNet: A loss joint training model based on the vision transformer and the temporal convolution network for EEG-based motor imagery classification.

Brain-computer interface (BCI) is a technology that directly connects signals between the human brain and a computer or other external device. Motor imagery electroencephalographic (MI-EEG) signals are considered a promising paradigm for BCI systems, with a wide range of potential applications in medical rehabilitation, human-computer interaction, and virtual reality. Accurate decoding of MI-EEG signals poses a significant challenge due to issues related to the quality of the collected EEG data and subject variability. Therefore, developing an efficient MI-EEG decoding network is crucial and warrants research. This paper proposes a loss joint training model based on the vision transformer (VIT) and the temporal convolutional network (EEG-VTTCNet) to classify MI-EEG signals. To take advantage of multiple modules together, the EEG-VTTCNet adopts a shared convolution strategy and a dual-branching strategy. The dual-branching modules perform complementary learning and jointly train shared convolutional modules with better performance. We conducted experiments on the BCI Competition IV-2a and IV-2b datasets, and the proposed network outperformed the current state-of-the-art techniques with an accuracy of 84.58% and 90.94%, respectively, for the subject-dependent mode. In addition, we used t-SNE to visualize the features extracted by the proposed network, further demonstrating the effectiveness of the feature extraction framework. We also conducted extensive ablation and hyperparameter tuning experiments to construct a robust network architecture that can be well generalized.

Dual-Alpha: a large EEG study for dual-frequency SSVEP brain-computer interface.

BACKGROUND: The domain of brain-computer interface (BCI) technology has experienced significant expansion in recent years. However, the field continues to face a pivotal challenge due to the dearth of high-quality datasets. This lack of robust datasets serves as a bottleneck, constraining the progression of algorithmic innovations and, by extension, the maturation of the BCI field. FINDINGS: This study details the acquisition and compilation of electroencephalogram data across 3 distinct dual-frequency steady-state visual evoked potential (SSVEP) paradigms, encompassing over 100 participants. Each experimental condition featured 40 individual targets with 5 repetitions per target, culminating in a comprehensive dataset consisting of 21,000 trials of dual-frequency SSVEP recordings. We performed an exhaustive validation of the dataset through signal-to-noise ratio analyses and task-related component analysis, thereby substantiating its reliability and effectiveness for classification tasks. CONCLUSIONS: The extensive dataset presented is set to be a catalyst for the accelerated development of BCI technologies. Its significance extends beyond the BCI sphere and holds considerable promise for propelling research in psychology and neuroscience. The dataset is particularly invaluable for discerning the complex dynamics of binocular visual resource distribution.

EEG Analyses of visual cue effects on executed movements.

BACKGROUND: In electroencephalographic (EEG) or electrocorticographic (ECoG) experiments, visual cues are commonly used for timing synchronization but may inadvertently induce neural activity and cognitive processing, posing challenges when decoding self-initiated tasks. NEW METHOD: To address this concern, we introduced four new visual cues (Fade, Rotation, Reference, and Star) and investigated their impact on brain signals. Our objective was to identify a cue that minimizes its influence on brain activity, facilitating cue-effect free classifier training for asynchronous applications, particularly aiding individuals with severe paralysis. RESULTS: 22 able-bodied, right-handed participants aged 18-30 performed hand movements upon presentation of the visual cues. Analysis of time-variability between movement onset and cue-aligned data, grand average MRCP, and classification outcomes revealed significant differences among cues. Rotation and Reference cue exhibited favorable results in minimizing temporal variability, maintaining MRCP patterns, and achieving comparable accuracy to self-paced signals in classification. COMPARISON WITH EXISTING METHODS: Our study contrasts with traditional cue-based paradigms by introducing novel visual cues designed to mitigate unintended neural activity. We demonstrate the effectiveness of Rotation and Reference cue in eliciting consistent and accurate MRCPs during motor tasks, surpassing previous methods in achieving precise timing and high discriminability for classifier training. CONCLUSIONS: Precision in cue timing is crucial for training classifiers, where both Rotation and Reference cue demonstrate minimal variability and high discriminability, highlighting their potential for accurate classifications in online scenarios. These findings offer promising avenues for refining brain-computer interface systems, particularly for individuals with motor impairments, by enabling more reliable and intuitive control mechanisms.

Human-Robot Interaction in Motor Imagery: A System Based on the STFCN for Unilateral Upper Limb Rehabilitation Assistance.

BACKGROUND: Rehabilitation training based on the brain-computer interface of motor imagery (MI-BCI) can help restore the connection between the brain and movement. However, the performance of most popular MI-BCI system is coarse-level, which means that they are good at guiding the rehabilitation exercises of different parts of the body, but not for the individual component. New methods In this paper, we designed a fine-level MI-BCI system for unilateral upper limb rehabilitation assistance. Besides, due to the low discrimination of different sample classes in a single part, a classification algorithm called spatial-temporal filtering convolutional network (STFCN) was proposed that used spatial filtering and deep learning. COMPARISON WITH EXISTING METHODS: Our STFCN outperforms popular methods in recent years using BCI IV 2a and 2b data sets. RESULTS: To verify the effectiveness of our system, we recruited 6 volunteers and collected their data for a four-classification online experiments, resulting in an average accuracy of 62.7%. CONCLUSION: This fine-level MI-BCI system has good appli-cation prospects, and inspires more exploration of rehabilitation in a single part of the human body.

Neural subspaces of imagined movements in parietal cortex remain stable over several years in humans.

Objective A crucial goal in brain-machine interfacing is the long-term stability of neural decoding performance, ideally without regular retraining. Long-term stability has only been previously demonstrated in non-human primate experiments and only in primary sensorimotor cortices. Here we extend previous methods to determine long-term stability in humans by identifying and aligning low-dimensional structures in neural data. Approach Over a period of 1106 and 871 days respectively, two participants completed an imagined center-out reaching task. The longitudinal accuracy between all day pairs was assessed by latent subspace alignment using principal components analysis and canonical correlations analysis of multi-unit intracortical recordings in different brain regions (Brodmann Area 5, Anterior Intraparietal Area and the junction of the postcentral and intraparietal sulcus). Main Results We show the long-term stable representation of neural activity in subspaces of intracortical recordings from higher-order association areas in humans. Significance These results can be practically applied to significantly expand the longevity and generalizability of brain-computer interfaces. Clinical Trials NCT01849822, NCT01958086, NCT01964261.

Exploring Feature Selection and Classification Techniques to Improve the Performance of an Electroencephalography-Based Motor Imagery Brain-Computer Interface System.

The accuracy of classifying motor imagery (MI) activities is a significant challenge when using brain-computer interfaces (BCIs). BCIs allow people with motor impairments to control external devices directly with their brains using electroencephalogram (EEG) patterns that translate brain activity into control signals. Many researchers have been working to develop MI-based BCI recognition systems using various time-frequency feature extraction and classification approaches. However, the existing systems still face challenges in achieving satisfactory performance due to large amount of non-discriminative and ineffective features. To get around these problems, we suggested a multiband decomposition-based feature extraction and classification method that works well, along with a strong feature selection method for MI tasks. Our method starts by splitting the preprocessed EEG signal into four sub-bands. In each sub-band, we then used a common spatial pattern (CSP) technique to pull out narrowband-oriented useful features, which gives us a high-dimensional feature vector. Subsequently, we utilized an effective feature selection method, Relief-F, which reduces the dimensionality of the final features. Finally, incorporating advanced classification techniques, we classified the final reduced feature vector. To evaluate the proposed model, we used the three different EEG-based MI benchmark datasets, and our proposed model achieved better performance accuracy than existing systems. Our model's strong points include its ability to effectively reduce feature dimensionality and improve classification accuracy through advanced feature extraction and selection methods.

Hebbian plasticity induced by temporally coincident BCI enhances post-stroke motor recovery.

Functional electrical stimulation (FES) can support functional restoration of a paretic limb post-stroke. Hebbian plasticity depends on temporally coinciding pre- and post-synaptic activity. A tight temporal relationship between motor cortical (MC) activity associated with attempted movement and FES-generated visuo-proprioceptive feedback is hypothesized to enhance motor recovery. Using a brain-computer interface (BCI) to classify MC spectral power in electroencephalographic (EEG) signals to trigger FES-delivery with detection of movement attempts improved motor outcomes in chronic stroke patients. We hypothesized that heightened neural plasticity earlier post-stroke would further enhance corticomuscular functional connectivity and motor recovery. We compared subcortical non-dominant hemisphere stroke patients in BCI-FES and Random-FES (FES temporally independent of MC movement attempt detection) groups. The primary outcome measure was the Fugl-Meyer Assessment, Upper Extremity (FMA-UE). We recorded high-density EEG and transcranial magnetic stimulation-induced motor evoked potentials before and after treatment. The BCI group showed greater: FMA-UE improvement; motor evoked potential amplitude; beta oscillatory power and long-range temporal correlation reduction over contralateral MC; and corticomuscular coherence with contralateral MC. These changes are consistent with enhanced post-stroke motor improvement when movement is synchronized with MC activity reflecting attempted movement.

Wasserstein generative adversarial network with gradient penalty and convolutional neural network based motor imagery EEG classification.

Objective.Due to the difficulty in acquiring motor imagery electroencephalography (MI-EEG) data and ensuring its quality, insufficient training data often leads to overfitting and inadequate generalization capabilities of deep learning-based classification networks. Therefore, we propose a novel data augmentation method and deep learning classification model to enhance the decoding performance of MI-EEG further.Approach.The raw EEG signals were transformed into the time-frequency maps as the input to the model by continuous wavelet transform. An improved Wasserstein generative adversarial network with gradient penalty data augmentation method was proposed, effectively expanding the dataset used for model training. Additionally, a concise and efficient deep learning model was designed to improve decoding performance further.Main results.It has been demonstrated through validation by multiple data evaluation methods that the proposed generative network can generate more realistic data. Experimental results on the BCI Competition IV 2a and 2b datasets and the actual collected dataset show that classification accuracies are 83.4%, 89.1% and 73.3%, and Kappa values are 0.779, 0.782 and 0.644, respectively. The results indicate that the proposed model outperforms state-of-the-art methods.Significance.Experimental results demonstrate that this method effectively enhances MI-EEG data, mitigates overfitting in classification networks, improves MI classification accuracy, and holds positive implications for MI tasks.

A review on the performance of brain-computer interface systems used for patients with locked-in and completely locked-in syndrome.

Patients with locked-in syndrome (LIS) and complete locked-in syndrome (CLIS) own a fully functional brain restricted within a non-functional body. In order to help LIS patients stay connected with their surroundings, brain-computer interfaces (BCIs) and related technologies have emerged. BCIs translate brain activity into actions that can be performed by external devices enabling LIS patients to communicate, leading to an increase in their quality of life. The past decade has seen the rapid development of BCIs that have the potential to be used for patients with locked-in syndrome, from which a great deal is tested only on healthy subjects and not on actual patients. This study aims to (1) provide the readers with a comprehensive study that contributes to this growing area of research by exploring the performance of BCIs tested specifically on LIS and CLIS patients, (2) give an overview of different modalities and paradigms used in different stages of the locked-in syndrome, and (3) discuss the contributions and limitations of BCIs introduced for the LIS and CLIS patients in the state-of-the-art and lay a groundwork for researchers interested in this field.

A delayed matching task-based study on action sequence of motor imagery.

The way people imagine greatly affects performance of brain-computer interface (BCI) based on motion imagery (MI). Action sequence is a basic unit of imitation, learning, and memory for motor behavior. Whether it influences the MI-BCI is unknown, and how to manifest this influence is difficult since the MI is a spontaneous brain activity. To investigate the influence of the action sequence, this study proposes a novel paradigm named action sequences observing and delayed matching task to use images and videos to guide people to observe, match and reinforce the memory of sequence. Seven subjects' ERPs and MI performance are analyzed under four different levels of complexities or orders of the sequence. Results demonstrated that the action sequence in terms of complexity and sequence order significantly affects the MI. The complex action in positive order obtains stronger ERD/ERS and more pronounced MI feature distributions, and yields an MI classification accuracy that is 12.3% higher than complex action in negative order (p < 0.05). In addition, the ERP amplitudes derived from the supplementary motor area show a positive correlation to the MI. This study demonstrates a new perspective of improving imagery in the MI-BCI by considering the complexity and order of the action sequences, and provides a novel index for manifesting the MI performance by ERP.

Brain-computer interface on wrist training with or without neurofeedback in subacute stroke: a study protocol for a double-blinded, randomized control pilot trial.

Background: After a stroke, damage to the part of the brain that controls movement results in the loss of motor function. Brain-computer interface (BCI)-based stroke rehabilitation involves patients imagining movement without physically moving while the system measures the perceptual-motor rhythm in the motor cortex. Visual feedback through virtual reality and functional electrical stimulation is provided simultaneously. The superiority of real BCI over sham BCI in the subacute phase of stroke remains unclear. Therefore, we aim to compare the effects of real and sham BCI on motor function and brain activity among patients with subacute stroke with weak wrist extensor strength. Methods: This is a double-blinded randomized controlled trial. Patients with stroke will be categorized into real BCI and sham BCI groups. The BCI task involves wrist extension for 60 min/day, 5 times/week for 4 weeks. Twenty sessions will be conducted. The evaluation will be conducted four times, as follows: before the intervention, 2 weeks after the start of the intervention, immediately after the intervention, and 4 weeks after the intervention. The assessments include a clinical evaluation, electroencephalography, and electromyography using motor-evoked potentials. Discussion: Patients will be categorized into two groups, as follows: those who will be receiving neurofeedback and those who will not receive this feedback during the BCI rehabilitation training. We will examine the importance of motor imaging feedback, and the effect of patients' continuous participation in the training rather than their being passive.Clinical Trial Registration: KCT0008589.

Developing a tablet-based brain-computer interface and robotic prototype for upper limb rehabilitation.

Background: The current study explores the integration of a motor imagery (MI)-based BCI system with robotic rehabilitation designed for upper limb function recovery in stroke patients. Methods: We developed a tablet deployable BCI control of the virtual iTbot for ease of use. Twelve right-handed healthy adults participated in this study, which involved a novel BCI training approach incorporating tactile vibration stimulation during MI tasks. The experiment utilized EEG signals captured via a gel-free cap, processed through various stages including signal verification, training, and testing. The training involved MI tasks with concurrent vibrotactile stimulation, utilizing common spatial pattern (CSP) training and linear discriminant analysis (LDA) for signal classification. The testing stage introduced a real-time feedback system and a virtual game environment where participants controlled a virtual iTbot robot. Results: Results showed varying accuracies in motor intention detection across participants, with an average true positive rate of 63.33% in classifying MI signals. Discussion: The study highlights the potential of MI-based BCI in robotic rehabilitation, particularly in terms of engagement and personalization. The findings underscore the feasibility of BCI technology in rehabilitation and its potential use for stroke survivors with upper limb dysfunctions.

Benchmarking brain-computer interface algorithms: Riemannian approaches vs convolutional neural networks.

OBJECTIVE: To date, a comprehensive comparison of Riemannian decoding methods with deep convolutional neural networks for EEG-based Brain-Computer Interfaces remains absent from published work. We address this research gap by using MOABB, The Mother Of All BCI Benchmarks, to compare novel convolutional neural networks to state-of-the-art Riemannian approaches across a broad range of EEG datasets, including motor imagery, P300, and steady-state visual evoked potentials paradigms. Approach. We systematically evaluated the performance of convolutional neural networks, specifically EEGNet, shallow ConvNet, and deep ConvNet, against well-established Riemannian decoding methods using MOABB processing pipelines. This evaluation included within- session, cross-session, and cross-subject methods, to provide a practical analysis of model effectiveness and to find an overall solution that performs well across different experimental settings. Main results. We find no significant differences in decoding performance between convolutional neural networks and Riemannian methods for within-session, cross-session, and cross-subject analyses. Significance. The results show that, when using traditional Brain-Computer Interface paradigms, the choice between CNNs and Riemannian methods may not heavily impact decoding performances in many experimental settings. These findings provide researchers with flexibility in choosing decoding approaches based on factors such as ease of implementation, computational efficiency or individual preferences.

Lightweight attention mechanisms for EEG emotion recognition for brain computer interface.

BACKGROUND: In the realm of brain-computer interfaces (BCI), identifying emotions from electroencephalogram (EEG) data is a difficult endeavor because of the volume of data, the intricacy of the signals, and the several channels that make up the signals. NEW METHODS: Using dual-stream structure scaling and multiple attention mechanisms (LDMGEEG), a lightweight network is provided to maximize the accuracy and performance of EEG-based emotion identification. Reducing the number of computational parameters while maintaining the current level of classification accuracy is the aim. This network employs a symmetric dual-stream architecture to assess separately time-domain and frequency-domain spatio-temporal maps constructed using differential entropy features of EEG signals as inputs. RESULT: The experimental results show that after significantly lowering the number of parameters, the model achieved the best possible performance in the field, with a 95.18 % accuracy on the SEED dataset. COMPARISON WITH EXISTING METHODS: Moreover, it reduced the number of parameters by 98 % when compared to existing models. CONCLUSION: The proposed method distinct channel-time/frequency-space multiple attention and post-attention methods enhance the model's ability to aggregate features and result in lightweight performance.

Closed-Loop Infraslow Brain-Computer Interface can Modulate Cortical Activity and Connectivity in Individuals With Chronic Painful Knee Osteoarthritis: A Secondary Analysis of a Randomized Placebo-Controlled Clinical Trial.

Introduction. Chronic pain is a percept due to an imbalance in the activity between sensory-discriminative, motivational-affective, and descending pain-inhibitory brain regions. Evidence suggests that electroencephalography (EEG) infraslow fluctuation neurofeedback (ISF-NF) training can improve clinical outcomes. It is unknown whether such training can induce EEG activity and functional connectivity (FC) changes. A secondary data analysis of a feasibility clinical trial was conducted to determine whether EEG ISF-NF training can significantly alter EEG activity and FC between the targeted cortical regions in people with chronic painful knee osteoarthritis (OA). Methods. A parallel, two-arm, double-blind, randomized, sham-controlled clinical trial was conducted. People with chronic knee pain associated with OA were randomized to receive sham NF training or source-localized ratio ISF-NF training protocol to down-train ISF bands at the somatosensory (SSC), dorsal anterior cingulate (dACC), and uptrain pregenual anterior cingulate cortices (pgACC). Resting state EEG was recorded at baseline and immediate post-training. Results. The source localization mapping demonstrated a reduction (P = .04) in the ISF band activity at the left dorsolateral prefrontal cortex (LdlPFC) in the active NF group. Region of interest analysis yielded significant differences for ISF (P = .008), slow (P = .007), beta (P = .043), and gamma (P = .012) band activities at LdlPFC, dACC, and bilateral SSC. The FC between pgACC and left SSC in the delta band was negatively correlated with pain bothersomeness in the ISF-NF group. Conclusion. The EEG ISF-NF training can modulate EEG activity and connectivity in individuals with chronic painful knee osteoarthritis, and the observed EEG changes correlate with clinical pain measures.

Correcting for ERP latency jitter improves gaze-independent BCI decoding.

Objective.Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze.Approach.ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment.Main results.WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session.Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.

Rehabilitation Based on BCI: An Innovative Enhancement for Sensorimotor Cortex Rhythms Systemization.

The research proposes a novel strategy for categorizing electroencephalograms (EEG) in real-time brain-computer interfaces that have rehabilitation applications. The methodology utilizes Five Cross-Common Spatial Patterns (FCCSP) to develop a motor movement/imagery systemization model that extracts multi-domain characteristics with excellent performance. The goal is to eliminate the impact caused by EEG's nonstationarity. The article highlights the findings of a real-time technique that is incorporated into a comprehensive prediction system, and it offers an innovative method to boost accuracy in real-time Sensory-Motor cortex Rhythms (SMR). The accuracy increased from 57.14% using raw EEG to 85.71% after preprocessing, and from 58.08% to 97.94% in public domain SMR. The proposed Butterworth bandpass filter is optimized using the FCCSP to determine the ideal bandwidth that incorporates the whole EEG features in beta waves. The Hybrid Systemization of the Correlated Feature Removal classifier is then integrated with the FCCSP method to create improved predictive models. As a consequence, while applied to real-time and PhysioNet datasets, the outcome system achieved outstanding accuracy values of 85.71% and 97.94%, respectively. This demonstrates the robustness of the strategy to increase SMR prediction efficiency.

Single-trial movement intention detection estimation in patients with Parkinson's disease: a movement-related cortical potential study.

Objectives. Parkinson patients often suffer from motor impairments such as tremor and freezing of movement that can be difficult to treat. To unfreeze movement, it has been suggested to provide sensory stimuli. To avoid constant stimulation, episodes with freezing of movement needs to be detected which is a challenge. This can potentially be obtained using a brain-computer interface (BCI) based on movement-related cortical potentials (MRCPs) that are observed in association with the intention to move. The objective in this study was to detect MRCPs from single-trial EEG.Approach. Nine Parkinson patients executed 100 wrist movements and 100 ankle movements while continuous EEG and EMG were recorded. The experiment was repeated in two sessions on separate days. Using temporal, spectral and template matching features, a random forest (RF), linear discriminant analysis, and k-nearest neighbours (kNN) classifier were constructed in offline analysis to discriminate between epochs containing movement-related or idle brain activity to provide an estimation of the performance of a BCI. Three classification scenarios were tested: 1) within-session (using training and testing data from the same session and participant), between-session (using data from the same participant from session one for training and session two for testing), and across-participant (using data from all participants except one for training and testing on the remaining participant).Main results. The within-session classification scenario was associated with the highest classification accuracies which were in the range of 88%-89% with a similar performance across sessions. The performance dropped to 69%-75% and 70%-75% for the between-session and across-participant classification scenario, respectively. The highest classification accuracies were obtained for the RF and kNN classifiers.Significance. The results indicate that it is possible to detect movement intentions in individuals with Parkinson's disease such that they can operate a BCI which may control the delivery of sensory stimuli to unfreeze movement.