Cross-dataset transfer learning for motor imagery signal classification via multi-task learning and pre-training.

Objective.Deep learning (DL) models have been proven to be effective in decoding motor imagery (MI) signals in Electroencephalogram (EEG) data. However, DL models' success relies heavily on large amounts of training data, whereas EEG data collection is laborious and time-consuming. Recently, cross-dataset transfer learning has emerged as a promising approach to meet the data requirements of DL models. Nevertheless, transferring knowledge across datasets involving different MI tasks remains a significant challenge in cross-dataset transfer learning, limiting the full utilization of valuable data resources. APPROACH: This study proposes a pre-training-based cross-dataset transfer learning method inspired by Hard Parameter Sharing in multi-task learning. Different datasets with distinct MI paradigms are considered as different tasks, classified with shared feature extraction layers and individual task-specific layers to allow cross-dataset classification with one unified model. Then, Pre-training and fine-tuning are employed to transfer knowledge across datasets. We also designed four fine-tuning schemes and conducted extensive experiments on them. MAIN RESULTS: The results showed that compared to models without pre-training, models with pre-training achieved a maximum increase in accuracy of 7.76%. Moreover, when limited training data were available, the pre-training method significantly improved DL model's accuracy by 27.34% at most. The experiments also revealed that pre-trained models exhibit faster convergence and remarkable robustness. The training time per subject could be reduced by up to 102.83 s, and the variance of classification accuracy decreased by 75.22% at best. SIGNIFICANCE: This study represents the first comprehensive investigation of the cross-dataset transfer learning method between two datasets with different MI tasks. The proposed pre-training method requires only minimal fine-tuning data when applying DL models to new MI paradigms, making MI-Brain-computer interface more practical and user-friendly.

Filter bank sinc-convolutional network with channel self-attention for high performance motor imagery decoding.

Objective.Motor Imagery Brain-Computer Interface (MI-BCI) is an active Brain-Computer Interface (BCI) paradigm focusing on the identification of motor intention, which is one of the most important non-invasive BCI paradigms. In MI-BCI studies, deep learning-based methods (especially lightweight networks) have attracted more attention in recent years, but the decoding performance still needs further improving.Approach.To solve this problem, we designed a filter bank structure with sinc-convolutional layers for spatio-temporal feature extraction of MI-electroencephalography in four motor rhythms. The Channel Self-Attention method was introduced for feature selection based on both global and local information, so as to build a model called Filter Bank Sinc-convolutional Network with Channel Self-Attention for high performance MI-decoding. Also, we proposed a data augmentation method based on multivariate empirical mode decomposition to improve the generalization capability of the model.Main results.We performed an intra-subject evaluation experiment on unseen data of three open MI datasets. The proposed method achieved mean accuracy of 78.20% (4-class scenario) on BCI Competition IV IIa, 87.34% (2-class scenario) on BCI Competition IV IIb, and 72.03% (2-class scenario) on Open Brain Machine Interface (OpenBMI) dataset, which are significantly higher than those of compared deep learning-based methods by at least 3.05% (p= 0.0469), 3.18% (p= 0.0371), and 2.27% (p= 0.0024) respectively.Significance.This work provides a new option for deep learning-based MI decoding, which can be employed for building BCI systems for motor rehabilitation.

MCFHNet: Multi-Channel Fusion Hybrid Network for Efficient EEG-fNIRS Multi-modal Motor Imagery Decoding.

Motor Imagery-based Brain Computer Interface (MI-BCI) is a typical active BCI with a main focus on motor intention identification. Hybrid motor imagery (MI) decoding methods that based on multi-modal fusion of Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), especially deep learning-based methods, become popular in recent MI-BCI studies. However, the fusion strategy and network design in deep learning-based methods are complex. To solve this problem, we proposed the multi-channel fusion method (MCF) to simplify current fusion methods, and we designed a multi-channel fusion hybrid network (MCFHNet) based on MCF. MCFHNet combines depthwise convolutional layers, channel attention mechanism, and Bidirectional Long Short Term Memory (Bi-LSTM) layers, which enables strong capability of feature extraction in spatial and temporal domain. The comparison between MCFHNet and representative deep learning-based methods was performed on an open EEG-fNIRS dataset. We found the proposed method can yield superior performance (mean accuracy of 99.641 % in 5-fold cross validation of an intra-subject experiment). This work provides a new option for multi-modal MI decoding, which can be applied in the rehabilitation field based on hybrid BCI systems.

[Progress of classification algorithms for motor imagery electroencephalogram signals].

Motor imagery (MI), motion intention of the specific body without actual movements, has attracted wide attention in fields as neuroscience. Classification algorithms for motor imagery electroencephalogram (MI-EEG) signals are able to distinguish different MI tasks based on the physiological information contained by the EEG signals, especially the features extracted from them. In recent years, there have been some new advances in classification algorithms for MI-EEG signals in terms of classifiers versus machine learning strategies. In terms of classifiers, traditional machine learning classifiers have been improved by some researchers, deep learning and Riemannian geometry classifiers have been widely applied as well. In terms of machine learning strategies, ensemble learning, adaptive learning, and transfer learning strategies have been utilized to improve classification accuracies or reach other targets. This paper reviewed the progress of classification algorithms for MI-EEG signals, summarized and evaluated the existing classifiers and machine learning strategies, to provide new ideas for developing classification algorithms with higher performance.

BCI Monitor Enhances Electroencephalographic and Cerebral Hemodynamic Activations During Motor Training.

Motor imagery-based brain-computer interface (MI-BCI) controlling functional electrical stimulation (FES) is promising for disabled patients to restore their motor functions. However, it remains unclear how much the BCI part can contribute to the functional coupling between the brain and muscle. Specifically, whether it can enhance the cerebral activation for motor training? Here, we investigate the electroencephalographic and cerebral hemodynamic responses for MI-BCI-FES training and MI-FES training, respectively. Twelve healthy subjects were recruited in the motor training study when concurrent electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were recorded. Compared with the MI-FES training conditions, the MI-BCI-FES could induce significantly stronger event-related desynchronization (ERD) and blood oxygen response, which demonstrates that BCI indeed plays a functional role in the closed-loop motor training. Therefore, this paper verifies the feasibility of using BCI to train motor functions in a closed-loop manner.

EEG oscillatory patterns and classification of sequential compound limb motor imagery.

BACKGROUND: A number of studies have been done on movement imagination of motor sequences with a single limb. However, brain oscillatory patterns induced by movement imagination of motor sequences involving multiple limbs have not been reported in recent years. The goal of the present study was to verify the feasibility of application of motor sequences involving multiple limbs to brain-computer interface (BCI) systems based on motor imagery (MI). The changes of EEG patterns and the inter-influence between movements associated with the imagination of motor sequences were also investigated. METHODS: The experiment, where 12 healthy subjects participated, involved one motor sequence with a single limb and three kinds of motor sequences with two or three limbs. The activity involved mental simulation, imagining playing drums with two conditions (60 and 30 beats per minute for the first and second conditions, respectively). RESULTS: Movement imagination of different limbs in the sequence contributed to time-variant event-related desynchronization (ERD) patterns within both mu and beta rhythms, which was more obvious for the second condition compared with the first condition. The ERD values of left/right hand imagery with prior hand imagery were significantly larger than those with prior foot imagery, while the phase locking values (PLVs) between central electrodes and the mesial frontocentral electrode of non-initial movement were significantly larger than those of the initial movement during imagination of motor sequences for both conditions. Classification results showed that the power spectral density (PSD) based method outperformed the multi-class common spatial patterns (multi-CSP) based method: The highest accuracies were 82.86 % and 91.43 %, and the mean values were 65 % and 74.14 % for the first and second conditions, respectively. CONCLUSIONS: This work implies that motor sequences involving multiple limbs can be utilized to build a multimodal classification paradigm in MI-based BCI systems, and that prior movement imagination can result in the changes of neural activities in motor areas during subsequent movement imagination in the process of limb switching.

Evaluation and comparison of effective connectivity during simple and compound limb motor imagery.

Motor imagery (MI) has been demonstrated beneficial in motor rehabilitation in patients with movement disorders. In contrast with simple limb motor imagery, less work was reported about the effective connectivity networks of compound limb motor imagery which involves several parts of limbs. This work aimed to investigate the differences of information flow patterns between simple limb motor imagery and compound limb motor imagery. Ten subjects participated in the experiment involving three tasks of simple limb motor imagery (left hand, right hand, feet) and three tasks of compound limb motor imagery (both hands, left hand combined with right foot, right hand combined with left foot). The causal interactions among different neural regions were evaluated by Short-time Directed Transfer Function (SDTF). Quite different from the networks of simple limb motor imagery, more effective interactions overlying larger brain regions were observed during compound limb motor imagery. These results imply that there exist significant differences in the patterns of EEG activity flow between simple limb motor imagery and compound limb motor imagery, which present more complex networks and could be utilized in motor rehabilitation for more benefit in patients with movement disorders.

EEG feature comparison and classification of simple and compound limb motor imagery.

BACKGROUND: Motor imagery can elicit brain oscillations in Rolandic mu rhythm and central beta rhythm, both originating in the sensorimotor cortex. In contrast with simple limb motor imagery, less work was reported about compound limb motor imagery which involves several parts of limbs. The goal of this study was to investigate the differences of the EEG patterns between simple limb motor imagery and compound limb motor imagery, and discuss the separability of multiple types of mental tasks. METHODS: Ten subjects participated in the experiment involving three tasks of simple limb motor imagery (left hand, right hand, feet), three tasks of compound limb motor imagery (both hands, left hand combined with right foot, right hand combined with left foot) and rest state. Event-related spectral perturbation (ERSP), power spectral entropy (PSE) and spatial distribution coefficient were adopted to analyze these seven EEG patterns. Then three algorithms of modified multi-class common spatial patterns (CSP) were used for feature extraction and classification was implemented by support vector machine (SVM). RESULTS: The induced event-related desynchronization (ERD) affects more components within both alpha and beta bands resulting in more broad ERD bands at electrode positions C3, Cz and C4 during left/right hand combined with contralateral foot imagery, whose PSE values are significant higher than that of simple limb motor imagery. From the topographical distribution, simultaneous imagination of upper limb and contralateral lower limb certainly contributes to the activation of more areas on cerebral cortex. Classification result shows that multi-class stationary Tikhonov regularized CSP (Multi-sTRCSP) outperforms other two multi-class CSP methods, with the highest accuracy of 84% and mean accuracy of 70%. CONCLUSIONS: The work implies that there exist the separable differences between simple limb motor imagery and compound limb motor imagery, which can be utilized to build a multimodal classification paradigm in motor imagery based brain-computer interface (BCI) systems.