A BCI based visual-haptic neurofeedback training improves cortical activations and classification performance during motor imagery.

OBJECTIVE: We proposed a brain-computer interface (BCI) based visual-haptic neurofeedback training (NFT) by incorporating synchronous visual scene and proprioceptive electrical stimulation feedback. The goal of this work was to improve sensorimotor cortical activations and classification performance during motor imagery (MI). In addition, their correlations and brain network patterns were also investigated respectively. APPROACH: 64-channel electroencephalographic (EEG) data were recorded in nineteen healthy subjects during MI before and after NFT. During NFT sessions, the synchronous visual-haptic feedbacks were driven by real-time lateralized relative event-related desynchronization (lrERD). MAIN RESULTS: By comparison between previous and posterior control sessions, the cortical activations measured by multi-band (i.e. alpha_1: 8-10 Hz, alpha_2: 11-13 Hz, beta_1: 15-20 Hz and beta_2: 22-28 Hz) absolute ERD powers and lrERD patterns were significantly enhanced after the NFT. The classification performance was also significantly improved, achieving a ~9% improvement and reaching ~85% in mean classification accuracy from a relatively poor performance. Additionally, there were significant correlations between lrERD patterns and classification accuracies. The partial directed coherence based functional connectivity (FC) networks covering the sensorimotor area also showed an increase after the NFT. SIGNIFICANCE: These findings validate the feasibility of our proposed NFT to improve sensorimotor cortical activations and BCI performance during motor imagery. And it is promising to optimize conventional NFT manner and evaluate the effectiveness of motor training.

Deep learning for electroencephalogram (EEG) classification tasks: a review.

OBJECTIVE: Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks? APPROACH: A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture. MAIN RESULTS: For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review. SIGNIFICANCE: This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

Classification of different reaching movements from the same limb using EEG.

OBJECTIVE: Brain-computer-interfaces (BCIs) have been proposed not only as assistive technologies but also as rehabilitation tools for lost functions. However, due to the stochastic nature, poor spatial resolution and signal to noise ratio from electroencephalography (EEG), multidimensional decoding has been the main obstacle to implement non-invasive BCIs in real-live rehabilitation scenarios. This study explores the classification of several functional reaching movements from the same limb using EEG oscillations in order to create a more versatile BCI for rehabilitation. APPROACH: Nine healthy participants performed four 3D center-out reaching tasks in four different sessions while wearing a passive robotic exoskeleton at their right upper limb. Kinematics data were acquired from the robotic exoskeleton. Multiclass extensions of Filter Bank Common Spatial Patterns (FBCSP) and a linear discriminant analysis (LDA) classifier were used to classify the EEG activity into four forward reaching movements (from a starting position towards four target positions), a backward movement (from any of the targets to the starting position and rest). Recalibrating the classifier using data from previous or the same session was also investigated and compared. MAIN RESULTS: Average EEG decoding accuracy were significantly above chance with 67%, 62.75%, and 50.3% when decoding three, four and six tasks from the same limb, respectively. Furthermore, classification accuracy could be increased when using data from the beginning of each session as training data to recalibrate the classifier. SIGNIFICANCE: Our results demonstrate that classification from several functional movements performed by the same limb is possible with acceptable accuracy using EEG oscillations, especially if data from the same session are used to recalibrate the classifier. Therefore, an ecologically valid decoding could be used to control assistive or rehabilitation mutli-degrees of freedom (DoF) robotic devices using EEG data. These results have important implications towards assistive and rehabilitative neuroprostheses control in paralyzed patients.

Pure visual imagery as a potential approach to achieve three classes of control for implementation of BCI in non-motor disorders.

OBJECTIVE: The achievement of multiple instances of control with the same type of mental strategy represents a way to improve flexibility of brain-computer interface (BCI) systems. Here we test the hypothesis that pure visual motion imagery of an external actuator can be used as a tool to achieve three classes of electroencephalographic (EEG) based control, which might be useful in attention disorders. APPROACH: We hypothesize that different numbers of imagined motion alternations lead to distinctive signals, as predicted by distinct motion patterns. Accordingly, a distinct number of alternating sensory/perceptual signals would lead to distinct neural responses as previously demonstrated using functional magnetic resonance imaging (fMRI). We anticipate that differential modulations should also be observed in the EEG domain. EEG recordings were obtained from twelve participants using three imagery tasks: imagery of a static dot, imagery of a dot with two opposing motions in the vertical axis (two motion directions) and imagery of a dot with four opposing motions in vertical or horizontal axes (four directions). The data were analysed offline. MAIN RESULTS: An increase of alpha-band power was found in frontal and central channels as a result of visual motion imagery tasks when compared with static dot imagery, in contrast with the expected posterior alpha decreases found during simple visual stimulation. The successful classification and discrimination between the three imagery tasks confirmed that three different classes of control based on visual motion imagery can be achieved. The classification approach was based on a support vector machine (SVM) and on the alpha-band relative spectral power of a small group of six frontal and central channels. Patterns of alpha activity, as captured by single-trial SVM closely reflected imagery properties, in particular the number of imagined motion alternations. SIGNIFICANCE: We found a new mental task based on visual motion imagery with potential for the implementation of multiclass (3) BCIs. Our results are consistent with the notion that frontal alpha synchronization is related with high internal processing demands, changing with the number of alternation levels during imagery. Together, these findings suggest the feasibility of pure visual motion imagery tasks as a strategy to achieve multiclass control systems with potential for BCI and in particular, neurofeedback applications in non-motor (attentional) disorders.

Weighted spatial based geometric scheme as an efficient algorithm for analyzing single-trial EEGS to improve cue-based BCI classification.

There is a growing interest in analyzing the geometrical behavior of electroencephalogram (EEG) covariance matrix in the context of brain computer interface (BCI). The bottleneck of the current Riemannian framework is the bias of the mean vector of EEG signals to the noisy trials, which deteriorates the covariance matrix in the manifold space. This study presents a spatial weighting scheme to reduce the effect of noisy trials on the mean vector. To assess the proposed method, dataset IIa from BCI competition IV, containing the EEG trials of 9 subjects performing four mental tasks, was utilized. The performance of the proposed method is compared to the classical Riemannian method along with Common Spatial Pattern (CSP) on the dataset. The results show that when considering just two imagery classes, the proposed method performs on par with CSP method, whereas in the multi class scenario, the proposed algorithm outperforms the CSP approach on seven out of nine subjects. Incidentally, the proposed method obtains better accuracy for the majority of subjects compared to the classical Riemannian method.

Employing an active mental task to enhance the performance of auditory attention-based brain-computer interfaces.

OBJECTIVE: A majority of auditory brain-computer interfaces (BCIs) use the attentional modulation of auditory event-related potentials (ERPs) for communication and control. This study investigated whether the performance of an ERP-based auditory BCI can be further improved by increasing the mental efforts associated with the execution of the attention-related task. METHODS: Subjects mentally selected a target among a random sequence of spoken digits. Upon the detection of the target digit, the subjects were required to perform an active mental task (AMT) - mentally discriminating the gender property of the target voice. The total number of presented digits was manipulated to investigate possible influences of the number of choices. The subjects also participated in two control experiments, in which they were asked to (1) press a button to report their discrimination results or (2) simply count the appearance of the target digit without performing the AMT. RESULTS: Two ERP components, that is, a negative shift around 200 ms (Nd) over the fronto-central area and a positive deflection during 500-600 ms (late positive component, LPC) over the central-parietal area, were modulated by execution of the AMT. Compared to a counting task, the AMT resulted in paradigm-specific enhanced LPC responses. The latency of the LPC was significantly correlated with the behavioural reaction time, indicating that the LPC could originate from a response-related brain network similar to P3b. The AMT paradigm resulted in an increase of 4-6% in BCI classification accuracies, compared to a counting paradigm that was considered to represent the traditional auditory attention BCI paradigms (p < 0.05). In addition, the BCI classification accuracies were not significantly affected by the number of BCI choices in the AMT paradigm. CONCLUSIONS: (1) LPC was identified as the AMT-specific ERP component and (2) the performance of auditory BCIs can be improved from the human response side by introducing additional mental efforts when executing attention-related tasks. SIGNIFICANCE: The neurophysiological characteristics of the recently proposed auditory BCI paradigm using an AMT were explored. The results suggest the proposed paradigm as a candidate for improving the performance of auditory BCIs.