Transferring a deep learning model from healthy subjects to stroke patients in a motor imagery brain-computer interface.

Objective.Motor imagery (MI) brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have been developed primarily for stroke rehabilitation, however, due to limited stroke data, current deep learning methods for cross-subject classification rely on healthy data. This study aims to assess the feasibility of applying MI-BCI models pre-trained using data from healthy individuals to detect MI in stroke patients.Approach.We introduce a new transfer learning approach where features from two-class MI data of healthy individuals are used to detect MI in stroke patients. We compare the results of the proposed method with those obtained from analyses within stroke data. Experiments were conducted using Deep ConvNet and state-of-the-art subject-specific machine learning MI classifiers, evaluated on OpenBMI two-class MI-EEG data from healthy subjects and two-class MI versus rest data from stroke patients.Main results.Results of our study indicate that through domain adaptation of a model pre-trained using healthy subjects' data, an average MI detection accuracy of 71.15% (±12.46%) can be achieved across 71 stroke patients. We demonstrate that the accuracy of the pre-trained model increased by 18.15% after transfer learning (p<0.001). Additionally, the proposed transfer learning method outperforms the subject-specific results achieved by Deep ConvNet and FBCSP, with significant enhancements of 7.64% (p<0.001) and 5.55% (p<0.001) in performance, respectively. Notably, the healthy-to-stroke transfer learning approach achieved similar performance to stroke-to-stroke transfer learning, with no significant difference (p>0.05). Explainable AI analyses using transfer models determined channel relevance patterns that indicate contributions from the bilateral motor, frontal, and parietal regions of the cortex towards MI detection in stroke patients.Significance.Transfer learning from healthy to stroke can enhance the clinical use of BCI algorithms by overcoming the challenge of insufficient clinical data for optimal training.

LGGNet: Learning From Local-Global-Graph Representations for Brain-Computer Interface.

Neuropsychological studies suggest that co-operative activities among different brain functional areas drive high-level cognitive processes. To learn the brain activities within and among different functional areas of the brain, we propose local-global-graph network (LGGNet), a novel neurologically inspired graph neural network (GNN), to learn local-global-graph (LGG) representations of electroencephalography (EEG) for brain-computer interface (BCI). The input layer of LGGNet comprises a series of temporal convolutions with multiscale 1-D convolutional kernels and kernel-level attentive fusion. It captures temporal dynamics of EEG which then serves as input to the proposed local-and global-graph-filtering layers. Using a defined neurophysiologically meaningful set of local and global graphs, LGGNet models the complex relations within and among functional areas of the brain. Under the robust nested cross-validation settings, the proposed method is evaluated on three publicly available datasets for four types of cognitive classification tasks, namely the attention, fatigue, emotion, and preference classification tasks. LGGNet is compared with state-of-the-art (SOTA) methods, such as DeepConvNet, EEGNet, R2G-STNN, TSception, regularized graph neural network (RGNN), attention-based multiscale convolutional neural network-dynamical graph convolutional network (AMCNN-DGCN), hierarchical recurrent neural network (HRNN), and GraphNet. The results show that LGGNet outperforms these methods, and the improvements are statistically significant ( ) in most cases. The results show that bringing neuroscience prior knowledge into neural network design yields an improvement of classification performance. The source code can be found at https://github.com/yi-ding-cs/LGG.

Relevance-based channel selection in motor imagery brain-computer interface.

Objective.Channel selection in the electroencephalogram (EEG)-based brain-computer interface (BCI) has been extensively studied for over two decades, with the goal being to select optimal subject-specific channels that can enhance the overall decoding efficacy of the BCI. With the emergence of deep learning (DL)-based BCI models, there arises a need for fresh perspectives and novel techniques to conduct channel selection. In this regard, subject-independent channel selection is relevant, since DL models trained using cross-subject data offer superior performance, and the impact of inherent inter-subject variability of EEG characteristics on subject-independent DL training is not yet fully understood.Approach.Here, we propose a novel methodology for implementing subject-independent channel selection in DL-based motor imagery (MI)-BCI, using layer-wise relevance propagation (LRP) and neural network pruning. Experiments were conducted using Deep ConvNet and 62-channel MI data from the Korea University EEG dataset.Main Results.Using our proposed methodology, we achieved a 61% reduction in the number of channels without any significant drop (p = 0.09) in subject-independent classification accuracy, due to the selection of highly relevant channels by LRP. LRP relevance-based channel selections provide significantly better accuracies compared to conventional weight-based selections while using less than 40% of the total number of channels, with differences in accuracies ranging from 5.96% to 1.72%. The performance of the adapted sparse-LRP model using only 16% of the total number of channels is similar to that of the adapted baseline model (p = 0.13). Furthermore, the accuracy of the adapted sparse-LRP model using only 35% of the total number of channels exceeded that of the adapted baseline model by 0.53% (p = 0.81). Analyses of channels chosen by LRP confirm the neurophysiological plausibility of selection, and emphasize the influence of motor, parietal, and occipital channels in MI-EEG classification.Significance.The proposed method addresses a traditional issue in EEG-BCI decoding, while being relevant and applicable to the latest developments in the field of BCI. We believe that our work brings forth an interesting and important application of model interpretability as a problem-solving technique.

Performance Evaluation of Compressed Deep CNN for Motor Imagery Classification using EEG.

Recently, deep learning and convolutional neural networks (CNNs) have reported several promising results in the classification of Motor Imagery (MI) using Electroencephalography (EEG). With the gaining popularity of CNN-based BCI, the challenges in deploying it in a real-world mobile and embedded device with limited computational and memory resources need to be explored. Towards this objective, we investigate the impact of the magnitude-based weight pruning technique to reduce the number of parameters of the pre-trained CNN-based classifier while maintaining its performance. We evaluated the proposed method on an open-source Korea University dataset which consists of 54 healthy subjects' EEG, recorded while performing right-and left-hand MI. Experimental results demonstrate that the subject-independent model can be maximumly pruned to 90% sparsity, with a compression ratio of 4.77× while retaining classification accuracy at 84.44% with minimal loss of 0.02% when compared to the baseline model's performance. Therefore, the proposed method can be used to design more compact deep CNN- based BCIs without compromising on their performance.

Investigation on Robustness of EEG-based Brain-Computer Interfaces.

Electroencephalogram (EEG)-based brain-computer interface (BCI) systems tend to suffer from performance degradation due to the presence of noise and artifacts in EEG data. This study is aimed at systematically investigating the robustness of state-of-the-art machine learning and deep learning based EEG-BCI models for motor imagery classification against simulated channel-specific noise in EEG data, at various low values of signal-to-noise ratio (SNR). Our results illustrate higher robustness of deep learning based MI classification models compared to the traditional machine learning based model, while identifying a set of channels with large sensitivity to simulated channel-specific noise. The EEGNet is relatively more robust towards channel-specific noise than Shallow ConvNet and FBCSP. We propose a preliminary solution, based on activation function, to improve the robustness of the deep learning models. By using saturating nonlinearities, the percentage drop in classification accuracy for SNR of -18 dB had reduced from 10.99% to 6.53% for EEGNet and 14.05% to 3.57% for Shallow ConvNet. Through this study, we emphasize the need for a more precise solution for enhancing the robustness, and thereby usability of EEG-BCI systems.

Design Considerations for Long Term Non-invasive Brain Computer Interface Training With Tetraplegic CYBATHLON Pilot.

Several studies in the recent past have demonstrated how Brain Computer Interface (BCI) technology can uncover the neural mechanisms underlying various tasks and translate them into control commands. While a multitude of studies have demonstrated the theoretic potential of BCI, a point of concern is that the studies are still confined to lab settings and mostly limited to healthy, able-bodied subjects. The CYBATHLON 2020 BCI race represents an opportunity to further develop BCI design strategies for use in real-time applications with a tetraplegic end user. In this study, as part of the preparation to participate in CYBATHLON 2020 BCI race, we investigate the design aspects of BCI in relation to the choice of its components, in particular, the type of calibration paradigm and its relevance for long-term use. The end goal was to develop a user-friendly and engaging interface suited for long-term use, especially for a spinal-cord injured (SCI) patient. We compared the efficacy of conventional open-loop calibration paradigms with real-time closed-loop paradigms, using pre-trained BCI decoders. Various indicators of performance were analyzed for this study, including the resulting classification performance, game completion time, brain activation maps, and also subjective feedback from the pilot. Our results show that the closed-loop calibration paradigms with real-time feedback is more engaging for the pilot. They also show an indication of achieving better online median classification performance as compared to conventional calibration paradigms (p = 0.0008). We also observe that stronger and more localized brain activation patterns are elicited in the closed-loop paradigm in which the experiment interface closely resembled the end application. Thus, based on this longitudinal evaluation of single-subject data, we demonstrate that BCI-based calibration paradigms with active user-engagement, such as with real-time feedback, could help in achieving better user acceptability and performance.

Adaptive transfer learning for EEG motor imagery classification with deep Convolutional Neural Network.

In recent years, deep learning has emerged as a powerful tool for developing Brain-Computer Interface (BCI) systems. However, for deep learning models trained entirely on the data from a specific individual, the performance increase has only been marginal owing to the limited availability of subject-specific data. To overcome this, many transfer-based approaches have been proposed, in which deep networks are trained using pre-existing data from other subjects and evaluated on new target subjects. This mode of transfer learning however faces the challenge of substantial inter-subject variability in brain data. Addressing this, in this paper, we propose 5 schemes for adaptation of a deep convolutional neural network (CNN) based electroencephalography (EEG)-BCI system for decoding hand motor imagery (MI). Each scheme fine-tunes an extensively trained, pre-trained model and adapt it to enhance the evaluation performance on a target subject. We report the highest subject-independent performance with an average (N=54) accuracy of 84.19% (±9.98%) for two-class motor imagery, while the best accuracy on this dataset is 74.15% (±15.83%) in the literature. Further, we obtain a statistically significant improvement (p=0.005) in classification using the proposed adaptation schemes compared to the baseline subject-independent model.

A Multi-view CNN with Novel Variance Layer for Motor Imagery Brain Computer Interface.

Accurate and robust classification of Motor Imagery (MI) from Electroencephalography (EEG) signals is among the most challenging tasks in Brain-Computer Interface (BCI) field. To address this challenge, this paper proposes a novel, neuro-physiologically inspired convolutional neural network (CNN) named Filter-Bank Convolutional Network (FBCNet) for MI classification. Capturing neurophysiological signatures of MI, FBCNet first creates a multi-view representation of the data by bandpass-filtering the EEG into multiple frequency bands. Next, spatially discriminative patterns for each view are learned using a CNN layer. Finally, the temporal information is aggregated using a new variance layer and a fully connected layer classifies the resultant features into MI classes. We evaluate the performance of FBCNet on a publicly available dataset from Korea University for classification of left vs right hand MI in a subject-specific 10-fold cross-validation setting. Results show that FBCNet achieves more than 6.7% higher accuracy compared to other state-of-the-art deep learning architectures while requiring less than 1% of the learning parameters. We explain the higher classification accuracy achieved by FBCNet using feature visualization where we show the superiority of FBCNet in learning interpretable and highly generalizable discriminative features. We provide the source code of FBCNet for reproducibility of results.

Prognostic and Monitory EEG-Biomarkers for BCI Upper-Limb Stroke Rehabilitation.

With the availability of multiple rehabilitative interventions, identifying the one that elicits the best motor outcome based on the unique neuro-clinical profile of the stroke survivor is a challenging task. Predicting the potential of recovery using biomarkers specific to an intervention hence becomes important. To address this, we investigate intervention-specific prognostic and monitory biomarkers of motor function improvements using quantitative electroencephalography (QEEG) features in 19 chronic stroke patients following two different upper extremity rehabilitative interventions viz. Brain-computer interface (BCI) and transcranial direct current stimulation coupled BCI (tDCS-BCI). Brain symmetry index was found to be the best prognostic QEEG for clinical gains following BCI intervention ( r = -0.80 , p = 0.02 ), whereas power ratio index (PRI) was observed to be the best predictor for tDCS-BCI ( r = -0.96 , p = 0.004 ) intervention. Importantly, statistically significant between-intervention differences observed in the predictive capabilities of these features suggest that intervention-specific biomarkers can be identified. This approach can be further pursued to distinctly predict the expected response of a patient to available interventions. The intervention with the highest predicted gains may then be recommended to the patient, thereby enabling a personalized rehabilitation regime.

Neurophysiological predictors and spectro-spatial discriminative features for enhancing SMR-BCI.

Neural engineering research is actively engaged in optimizing the robustness of sensorimotor rhythms (SMR)-brain-computer interface (BCI) to boost its potential real-world use. OBJECTIVE: This paper investigates two vital factors in efficient and robust SMR-BCI design-algorithms that address subject-variability of optimal features and neurophysiological factors that correlate with BCI performance. Existing SMR-BCI research using electroencephalogram (EEG) to classify bilateral motor imagery (MI) focus on identifying subject-specific frequency bands with most discriminative motor patterns localized to sensorimotor region. APPROACH: A novel strategy to further optimize BCI performance by taking into account the variability of discriminative spectral regions across various EEG channels is proposed in this paper. MAIN RESULTS: The proposed technique results in a significant ([Formula: see text]) increase in average ([Formula: see text]) classification accuracy by [Formula: see text] accompanied by a considerable reduction in number of channels and bands. The session-to-session transfer variation in spectro-spatial patterns using proposed algorithm is investigated offline and classification performance of the optimized BCI model is successfully evaluated in an online SMR-BCI. Further, the effective prediction of SMR-BCI performance with physiological indicators derived from multi-channel resting-state EEG is demonstrated. The results indicate that the resting state activation patterns such as entropy and gamma power from pre-motor (fronto-central) and posterior (parietal and centro-parietal) areas, and beta power from posterior (centro-parietal) areas estimate BCI performance with minimum error. These patterns, strongly related to BCI performance, may represent certain cognitive states during rest. SIGNIFICANCE: Findings reported in this paper imply the need for subject-specific modelling of BCI and the prediction of BCI performance using multi-channel rest-state parameters, to ensure enhanced BCI performance.

Wavlet phase-locking based binary classification of hand movement directions from EEG.

OBJECTIVE: Brain signals can be used to extract relevant features to decode various limb movement parameters such as the direction of upper limb movements. Amplitude based feature extraction techniques have been used to study such motor activity of upper limbs whereas phase synchrony, used to estimate functional relationship between signals, has rarely been used to study single hand movements in different directions. APPROACH: In this paper, a novel phase-locking-based feature extraction method, called wavelet phase-locking value (W-PLV) is proposed to analyse synchronous EEG channel-pairs and classify hand movement directions. EEG data collected from seven subjects performing right hand movements in four orthogonal directions in the horizontal plane is used for this analysis. MAIN RESULTS: Our proposed W-PLV based method achieves a mean binary classification accuracy of 76.85% over seven subjects using wavelet levels corresponding to ⩽12 Hz EEG. The results also show direction-dependent information in various wavelet levels and indicate the presence of relevant information in slow cortical potentials (<1 Hz) as well as higher wavelet levels (⩽12 Hz). SIGNIFICANCE: This study presents a thorough analysis of the phase-locking patterns extracted from EEG corresponding to hand movements in different directions using W-PLV across various wavelet levels and verifies their discriminative ability in the single trial binary classification of hand movement directions.

Correction: Real-Time Subject-Independent Pattern Classification of Overt and Covert Movements from fNIRS Signals.

[This corrects the article DOI: 10.1371/journal.pone.0159959.].

Real-Time Subject-Independent Pattern Classification of Overt and Covert Movements from fNIRS Signals.

Recently, studies have reported the use of Near Infrared Spectroscopy (NIRS) for developing Brain-Computer Interface (BCI) by applying online pattern classification of brain states from subject-specific fNIRS signals. The purpose of the present study was to develop and test a real-time method for subject-specific and subject-independent classification of multi-channel fNIRS signals using support-vector machines (SVM), so as to determine its feasibility as an online neurofeedback system. Towards this goal, we used left versus right hand movement execution and movement imagery as study paradigms in a series of experiments. In the first two experiments, activations in the motor cortex during movement execution and movement imagery were used to develop subject-dependent models that obtained high classification accuracies thereby indicating the robustness of our classification method. In the third experiment, a generalized classifier-model was developed from the first two experimental data, which was then applied for subject-independent neurofeedback training. Application of this method in new participants showed mean classification accuracy of 63% for movement imagery tasks and 80% for movement execution tasks. These results, and their corresponding offline analysis reported in this study demonstrate that SVM based real-time subject-independent classification of fNIRS signals is feasible. This method has important applications in the field of hemodynamic BCIs, and neuro-rehabilitation where patients can be trained to learn spatio-temporal patterns of healthy brain activity.

Adaptive estimation of hand movement trajectory in an EEG based brain-computer interface system.

OBJECTIVE: The various parameters that define a hand movement such as its trajectory, speed, etc, are encoded in distinct brain activities. Decoding this information from neurophysiological recordings is a less explored area of brain-computer interface (BCI) research. Applying non-invasive recordings such as electroencephalography (EEG) for decoding makes the problem more challenging, as the encoding is assumed to be deep within the brain and not easily accessible by scalp recordings. APPROACH: EEG based BCI systems can be developed to identify the neural features underlying movement parameters that can be further utilized to provide a detailed and well defined control command set to a BCI output device. A real-time continuous control is better suited for practical BCI systems, and can be achieved by continuous adaptive reconstruction of movement trajectory than discrete brain activity classifications. In this work, we adaptively reconstruct/estimate the parameters of two-dimensional hand movement trajectory, namely movement speed and position, from multi-channel EEG recordings. The data for analysis is collected by performing an experiment that involved center-out right-hand movement tasks in four different directions at two different speeds in random order. We estimate movement trajectory using a Kalman filter that models the relation between brain activity and recorded parameters based on a set of defined predictors. We propose a method to define these predictor variables that includes spatial, spectral and temporally localized neural information and to select optimally informative variables. MAIN RESULTS: The proposed method yielded correlation of (0.60 ± 0.07) between recorded and estimated data. Further, incorporating the proposed predictor subset selection, the correlation achieved is (0.57 ± 0.07, p < 0.004) with significant gain in stability of the system, as well as dramatic reduction in number of predictors (76%) for the savings of computational time. SIGNIFICANCE: The proposed system provides a real time movement control system using EEG-BCI with control over movement speed and position. These results are higher and statistically significant compared to existing techniques in EEG based systems and thus promise the applicability of the proposed method for efficient estimation of movement parameters and for continuous motor control.

Multi-class EEG classification of voluntary hand movement directions.

OBJECTIVE: Studies have shown that low frequency components of brain recordings provide information on voluntary hand movement directions. However, non-invasive techniques face more challenges compared to invasive techniques. APPROACH: This study presents a novel signal processing technique to extract features from non-invasive electroencephalography (EEG) recordings for classifying voluntary hand movement directions. The proposed technique comprises the regularized wavelet-common spatial pattern algorithm to extract the features, mutual information-based feature selection, and multi-class classification using the Fisher linear discriminant. EEG data from seven healthy human subjects were collected while they performed voluntary right hand center-out movement in four orthogonal directions. In this study, the movement direction dependent signal-to-noise ratio is used as a parameter to denote the effectiveness of each temporal frequency bin in the classification of movement directions. MAIN RESULTS: Significant (p < 0.005) movement direction dependent modulation in the EEG data was identified largely towards the end of movement at low frequencies (≤6 Hz) from the midline parietal and contralateral motor areas. Experimental results on single trial classification of the EEG data collected yielded an average accuracy of (80.24 ± 9.41)% in discriminating the four different directions using the proposed technique on features extracted from low frequency components. SIGNIFICANCE: The proposed feature extraction strategy provides very high multi-class classification accuracies, and the results are proven to be more statistically significant than existing methods. The results obtained suggest the possibility of multi-directional movement classification from single-trial EEG recordings using the proposed technique in low frequency components.

EEG-based classification of fast and slow hand movements using Wavelet-CSP algorithm.

A brain-computer interface (BCI) acquires brain signals, extracts informative features, and translates these features to commands to control an external device. This paper investigates the application of a noninvasive electroencephalography (EEG)-based BCI to identify brain signal features in regard to actual hand movement speed. This provides a more refined control for a BCI system in terms of movement parameters. An experiment was performed to collect EEG data from subjects while they performed right-hand movement at two different speeds, namely fast and slow, in four different directions. The informative features from the data were obtained using the Wavelet-Common Spatial Pattern (W-CSP) algorithm that provided high-temporal-spatial-spectral resolution. The applicability of these features to classify the two speeds and to reconstruct the speed profile was studied. The results for classifying speed across seven subjects yielded a mean accuracy of 83.71% using a Fisher Linear Discriminant (FLD) classifier. The speed components were reconstructed using multiple linear regression and significant correlation of 0.52 (Pearson's linear correlation coefficient) was obtained between recorded and reconstructed velocities on an average. The spatial patterns of the W-CSP features obtained showed activations in parietal and motor areas of the brain. The results achieved promises to provide a more refined control in BCI by including control of movement speed.

Differences in hemodynamic activations between motor imagery and upper limb FES with NIRS.

A brain-computer interface (BCI) based on near-infrared spectroscopy (NIRS) could act as a tool for rehabilitation of stroke patients due to the neural activity induced by motor imagery aided by real-time feedback of hemodynamic activation. When combined with functional electrical stimulation (FES) of the affected limb, BCI is expected to have an even greater benefit due to the contingency established between motor imagery and afferent, haptic feedback from stimulation. Yet, few studies have explored such an approach, presumably due to the difficulty in dissociating and thus decoding the hemodynamic response (HDR) between motor imagery and peripheral stimulation. Here, for the first time, we demonstrate that NIRS signals elicited by motor imagery can be reliably discriminated from those due to FES, by first performing a univariate analysis of the NIRS signals, and subsequently by multivariate pattern classification. Our results showing that robust classification of motor imagery from the rest condition is possible support previous findings that imagery could be used to drive a BCI based on NIRS. More importantly, we demonstrate for the first time the successful classification of motor imagery and FES, indicating that it is technically feasible to implement a contingent NIRS-BCI with FES.