Magnetoencephalogram-based brain-computer interface for hand-gesture decoding using deep learning.

Advancements in deep learning algorithms over the past decade have led to extensive developments in brain-computer interfaces (BCI). A promising imaging modality for BCI is magnetoencephalography (MEG), which is a non-invasive functional imaging technique. The present study developed a MEG sensor-based BCI neural network to decode Rock-Paper-scissors gestures (MEG-RPSnet). Unique preprocessing pipelines in tandem with convolutional neural network deep-learning models accurately classified gestures. On a single-trial basis, we found an average of 85.56% classification accuracy in 12 subjects. Our MEG-RPSnet model outperformed two state-of-the-art neural network architectures for electroencephalogram-based BCI as well as a traditional machine learning method, and demonstrated equivalent and/or better performance than machine learning methods that have employed invasive, electrocorticography-based BCI using the same task. In addition, MEG-RPSnet classification performance using an intra-subject approach outperformed a model that used a cross-subject approach. Remarkably, we also found that when using only central-parietal-occipital regional sensors or occipitotemporal regional sensors, the deep learning model achieved classification performances that were similar to the whole-brain sensor model. The MEG-RSPnet model also distinguished neuronal features of individual hand gestures with very good accuracy. Altogether, these results show that noninvasive MEG-based BCI applications hold promise for future BCI developments in hand-gesture decoding.

Elements Influencing sEMG-Based Gesture Decoding: Muscle Fatigue, Forearm Angle and Acquisition Time.

The surface Electromyography (sEMG) signal contains information about movement intention generated by the human brain, and it is the most intuitive and common solution to control robots, orthotics, prosthetics and rehabilitation equipment. In recent years, gesture decoding based on sEMG signals has received a lot of research attention. In this paper, the effects of muscle fatigue, forearm angle and acquisition time on the accuracy of gesture decoding were researched. Taking 11 static gestures as samples, four specific muscles (i.e., superficial flexor digitorum (SFD), flexor carpi ulnaris (FCU), extensor carpi radialis longus (ECRL) and finger extensor (FE)) were selected to sample sEMG signals. Root Mean Square (RMS), Waveform Length (WL), Zero Crossing (ZC) and Slope Sign Change (SSC) were chosen as signal eigenvalues; Linear Discriminant Analysis (LDA) and Probabilistic Neural Network (PNN) were used to construct classification models, and finally, the decoding accuracies of the classification models were obtained under different influencing elements. The experimental results showed that the decoding accuracy of the classification model decreased by an average of 7%, 10%, and 13% considering muscle fatigue, forearm angle and acquisition time, respectively. Furthermore, the acquisition time had the biggest impact on decoding accuracy, with a maximum reduction of nearly 20%.

Investigating Data Cleaning Methods to Improve Performance of Brain-Computer Interfaces Based on Stereo-Electroencephalography.

Stereo-electroencephalography (SEEG) utilizes localized and penetrating depth electrodes to directly measure electrophysiological brain activity. The implanted electrodes generally provide a sparse sampling of multiple brain regions, including both cortical and subcortical structures, making the SEEG neural recordings a potential source for the brain-computer interface (BCI) purpose in recent years. For SEEG signals, data cleaning is an essential preprocessing step in removing excessive noises for further analysis. However, little is known about what kinds of effect that different data cleaning methods may exert on BCI decoding performance and, moreover, what are the reasons causing the differentiated effects. To address these questions, we adopted five different data cleaning methods, including common average reference, gray-white matter reference, electrode shaft reference, bipolar reference, and Laplacian reference, to process the SEEG data and evaluated the effect of these methods on improving BCI decoding performance. Additionally, we also comparatively investigated the changes of SEEG signals induced by these different methods from multiple-domain (e.g., spatial, spectral, and temporal domain). The results showed that data cleaning methods could improve the accuracy of gesture decoding, where the Laplacian reference produced the best performance. Further analysis revealed that the superiority of the data cleaning method with excellent performance might be attributed to the increased distinguishability in the low-frequency band. The findings of this work highlighted the importance of applying proper data clean methods for SEEG signals and proposed the application of Laplacian reference for SEEG-based BCI.

Chinese characters are read using not only visual but also writing motor information.

It has been suggested that mature reading systems consist of both a visual analysis decoding system and a motor gesture decoding system that facilitates reading by processing writing motor information. However, there is still uncertainty about the mechanisms and effectiveness of the latter system when reading Chinese characters. This study therefore aimed to provide empirical neural evidence for this phenomenon using a writing video as prompt in a delayed sequential same/different judgment task. We investigated whether and how the orthographic processing of target characters was modulated by the writing direction of the prompt (forward vs. backward) and the character repetition (repeat vs. nonrepeat) between the prompt and target characters. The results indicated that (a) the N170 component was more negative under the forward condition than under the backward condition; and (b) both writing direction and character repetition modulated the centro-parietal N200 component. These writing-direction effects were specific to Chinese characters--they did not apply to Korean characters, which are visually similar to Chinese characters but unfamiliar to the participants. These results suggest that the experience of learning Chinese might establish a motor gesture decoding system for reading, which begins to perform general orthographic representation at an early stage and works together with the visual analysis decoding system to achieve deep orthographic processing.