A novel temporal-frequency combination pattern optimization approach based on information fusion for motor imagery BCIs.

Motor imagery (MI) stands as a powerful paradigm within Brain-Computer Interface (BCI) research due to its ability to induce changes in brain rhythms detectable through common spatial patterns (CSP). However, the raw feature sets captured often contain redundant and invalid information, potentially hindering CSP performance. Methodology-wise, we propose the Information Fusion for Optimizing Temporal-Frequency Combination Pattern (IFTFCP) algorithm to enhance raw feature optimization. Initially, preprocessed data undergoes simultaneous processing in both time and frequency domains via sliding overlapping time windows and filter banks. Subsequently, we introduce the Pearson-Fisher combinational method along with Discriminant Correlation Analysis (DCA) for joint feature selection and fusion. These steps aim to refine raw electroencephalogram (EEG) features. For precise classification of binary MI problems, an Radial Basis Function (RBF)-kernel Support Vector Machine classifier is trained. To validate the efficacy of IFTFCP and evaluate it against other techniques, we conducted experimental investigations using two EEG datasets. Results indicate a notably superior classification performance, boasting an average accuracy of 78.14% and 85.98% on dataset 1 and dataset 2, which is better than other methods outlined in this article. The study's findings suggest potential benefits for the advancement of MI-based BCI strategies, particularly in the domain of feature fusion.

Filter bank temporally local multivariate synchronization index for SSVEP-based BCI.

BACKGROUND: Multivariate synchronization index (MSI) has been successfully applied for frequency detection in steady state visual evoked potential (SSVEP) based brain-computer interface (BCI) systems. However, the standard MSI algorithm and its variants cannot simultaneously take full advantage of the time-local structure and the harmonic components in SSVEP signals, which are both crucial for frequency detection performance. To overcome the limitation, we propose a novel filter bank temporally local MSI (FBTMSI) algorithm to further improve SSVEP frequency detection accuracy. The method explicitly utilizes the temporal information of signal for covariance matrix estimation and employs filter bank decomposition to exploits SSVEP-related harmonic components. RESULTS: We employed the cross-validation strategy on the public Benchmark dataset to optimize the parameters and evaluate the performance of the FBTMSI algorithm. Experimental results show that FBTMSI outperforms the standard MSI, temporally local MSI (TMSI) and filter bank driven MSI (FBMSI) algorithms across multiple experimental settings. In the case of data length of one second, the average accuracy of FBTMSI is 9.85% and 3.15% higher than that of the FBMSI and the TMSI, respectively. CONCLUSIONS: The promising results demonstrate the effectiveness of the FBTMSI algorithm for frequency recognition and show its potential in SSVEP-based BCI applications.

Correcting for ERP latency jitter improves gaze-independent BCI decoding.

OBJECTIVE: Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze. METHODS: ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment. RESULTS & DISCUSSION: WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.

Recording the tactile P300 with the cEEGrid for potential use in a brain-computer interface.

Brain-computer interfaces (BCIs) are scientifically well established, but they rarely arrive in the daily lives of potential end-users. This could be in part because electroencephalography (EEG), a prevalent method to acquire brain activity for BCI operation, is considered too impractical to be applied in daily life of end-users with physical impairment as an assistive device. Hence, miniaturized EEG systems such as the cEEGrid have been developed. While they promise to be a step toward bridging the gap between BCI development, lab demonstrations, and home use, they still require further validation. Encouragingly, the cEEGrid has already demonstrated its ability to record visually and auditorily evoked event-related potentials (ERP), which are important as input signal for many BCIs. With this study, we aimed at evaluating the cEEGrid in the context of a BCI based on tactually evoked ERPs. To compare the cEEGrid with a conventional scalp EEG, we recorded brain activity with both systems simultaneously. Forty healthy participants were recruited to perform a P300 oddball task based on vibrotactile stimulation at four different positions. This tactile paradigm has been shown to be feasible for BCI repeatedly but has never been tested with the cEEGrid. We found distinct P300 deflections in the cEEGrid data, particularly at vertical bipolar channels. With an average of 63%, the cEEGrid classification accuracy was significantly above the chance level (25%) but significantly lower than the 81% reached with the EEG cap. Likewise, the P300 amplitude was significantly lower (cEEGrid R2-R7: 1.87 µV, Cap Cz: 3.53 µV). These results indicate that a tactile BCI using the cEEGrid could potentially be operated, albeit with lower efficiency. Additionally, participants' somatosensory sensitivity was assessed, but no correlation to the accuracy of either EEG system was shown. Our research contributes to the growing amount of literature comparing the cEEGrid to conventional EEG systems and provides first evidence that the tactile P300 can be recorded behind the ear. A BCI based on a thus simplified EEG system might be more readily accepted by potential end-users, provided the accuracy can be substantially increased, e.g., by training and improved classification.

Musical Neurofeedback Advancements, Feedback Modalities, and Applications: A Systematic Review.

The field of EEG-Neurofeedback (EEG-NF) training has showcased significant promise in treating various mental disorders, while also emerging as a cognitive enhancer across diverse applications. The core principle of EEG-NF involves consciously guiding the brain in desired directions, necessitating active engagement in neurofeedback (NF) tasks over an extended period. Music listening tasks have proven to be effective stimuli for such training, influencing emotions, mood, and brainwave patterns. This has spurred the development of musical NF systems and training protocols. Despite these advancements, there exists a gap in systematic literature that comprehensively explores and discusses the various modalities of feedback mechanisms, its benefits, and the emerging applications. Addressing this gap, our review article presents a thorough literature survey encompassing studies on musical NF conducted over the past decade. This review highlights the several benefits and applications ranging from neurorehabilitation to therapeutic interventions, stress management, diagnostics of neurological disorders, and sports performance enhancement. While acknowledged for advantages and popularity of musical NF, there is an opportunity for growth in the literature in terms of the need for systematic randomized controlled trials to compare its effectiveness with other modalities across different tasks. Addressing this gap will involve developing standardized methodologies for studying protocols and optimizing parameters, presenting an exciting prospect for advancing the field.

Narrow band-pass filtered canonical correlation analysis for frequency identification in SSVEP signals.

Steady-state visual evoked potentials (SSVEP) are generated in the parieto-occipital regions, accompanied by background noise and artifacts. A strong pre-processing method is required to reduce this background noise and artifacts. This study proposed a narrow band-pass filtered canonical correlation analysis (NBPFCCA) to recognize frequency components in SSVEP signals. The proposed method is tested on the publicly available 40 stimulus frequencies dataset recorded from 35 subjects and 4 class in-house dataset acquired from 10 subjects. The performance of the proposed NBPFCCA method is compared with the standard canonical correlation analysis (CCA) and the filter bank CCA (FBCCA). The mean frequency detection accuracy of the standard CCA is 86.21% for the benchmark dataset, and it is improved to 95.58% in the proposed method. Results indicate that the proposed method significantly outperforms the standard canonical correlation analysis with an increase of 9.37% and 17% in frequency recognition accuracy of the benchmark and in-house datasets, respectively.

Evaluation of Different Types of Stimuli in an Event-Related Potential-Based Brain-Computer Interface Speller under Rapid Serial Visual Presentation.

Rapid serial visual presentation (RSVP) is currently a suitable gaze-independent paradigm for controlling visual brain-computer interfaces (BCIs) based on event-related potentials (ERPs), especially for users with limited eye movement control. However, unlike gaze-dependent paradigms, gaze-independent ones have received less attention concerning the specific choice of visual stimuli that are used. In gaze-dependent BCIs, images of faces-particularly those tinted red-have been shown to be effective stimuli. This study aims to evaluate whether the colour of faces used as visual stimuli influences ERP-BCI performance under RSVP. Fifteen participants tested four conditions that varied only in the visual stimulus used: grey letters (GL), red famous faces with letters (RFF), green famous faces with letters (GFF), and blue famous faces with letters (BFF). The results indicated significant accuracy differences only between the GL and GFF conditions, unlike prior gaze-dependent studies. Additionally, GL achieved higher comfort ratings compared with other face-related conditions. This study highlights that the choice of stimulus type impacts both performance and user comfort, suggesting implications for future ERP-BCI designs for users requiring gaze-independent systems.

High-Density Electroencephalogram Facilitates the Detection of Small Stimuli in Code-Modulated Visual Evoked Potential Brain-Computer Interfaces.

In recent years, there has been a considerable amount of research on visual evoked potential (VEP)-based brain-computer interfaces (BCIs). However, it remains a big challenge to detect VEPs elicited by small visual stimuli. To address this challenge, this study employed a 256-electrode high-density electroencephalogram (EEG) cap with 66 electrodes in the parietal and occipital lobes to record EEG signals. An online BCI system based on code-modulated VEP (C-VEP) was designed and implemented with thirty targets modulated by a time-shifted binary pseudo-random sequence. A task-discriminant component analysis (TDCA) algorithm was employed for feature extraction and classification. The offline and online experiments were designed to assess EEG responses and classification performance for comparison across four different stimulus sizes at visual angles of 0.5°, 1°, 2°, and 3°. By optimizing the data length for each subject in the online experiment, information transfer rates (ITRs) of 126.48 ± 14.14 bits/min, 221.73 ± 15.69 bits/min, 258.39 ± 9.28 bits/min, and 266.40 ± 6.52 bits/min were achieved for 0.5°, 1°, 2°, and 3°, respectively. This study further compared the EEG features and classification performance of the 66-electrode layout from the 256-electrode EEG cap, the 32-electrode layout from the 128-electrode EEG cap, and the 21-electrode layout from the 64-electrode EEG cap, elucidating the pivotal importance of a higher electrode density in enhancing the performance of C-VEP BCI systems using small stimuli.

Exploring the Relationship between Behavioral and Neurological Impairments Due to Mild Cognitive Impairment: Correlation Study between Virtual Kiosk Test and EEG-SSVEP.

Amnestic mild cognitive impairment (aMCI) is a transitional stage between normal aging and Alzheimer's disease, making early screening imperative for potential intervention and prevention of progression to Alzheimer's disease (AD). Therefore, there is a demand for research to identify effective and easy-to-use tools for aMCI screening. While behavioral tests in virtual reality environments have successfully captured behavioral features related to instrumental activities of daily living for aMCI screening, further investigations are necessary to establish connections between cognitive decline and neurological changes. Utilizing electroencephalography with steady-state visual evoked potentials, this study delved into the correlation between behavioral features recorded during virtual reality tests and neurological features obtained by measuring neural activity in the dorsal stream. As a result, this multimodal approach achieved an impressive screening accuracy of 98.38%.

An end-to-end multi-task motor imagery EEG classification neural network based on dynamic fusion of spectral-temporal features.

Electroencephalograph (EEG) brain-computer interfaces (BCI) have potential to provide new paradigms for controlling computers and devices. The accuracy of brain pattern classification in EEG BCI is directly affected by the quality of features extracted from EEG signals. Currently, feature extraction heavily relies on prior knowledge to engineer features (for example from specific frequency bands); therefore, better extraction of EEG features is an important research direction. In this work, we propose an end-to-end deep neural network that automatically finds and combines features for motor imagery (MI) based EEG BCI with 4 or more imagery classes (multi-task). First, spectral domain features of EEG signals are learned by compact convolutional neural network (CCNN) layers. Then, gated recurrent unit (GRU) neural network layers automatically learn temporal patterns. Lastly, an attention mechanism dynamically combines (across EEG channels) the extracted spectral-temporal features, reducing redundancy. We test our method using BCI Competition IV-2a and a data set we collected. The average classification accuracy on 4-class BCI Competition IV-2a was 85.1 % ± 6.19 %, comparable to recent work in the field and showing low variability among participants; average classification accuracy on our 6-class data was 64.4 % ± 8.35 %. Our dynamic fusion of spectral-temporal features is end-to-end and has relatively few network parameters, and the experimental results show its effectiveness and potential.

Testing the Conjecture That Quantum Processes Create Conscious Experience.

The question of what generates conscious experience has mesmerized thinkers since the dawn of humanity, yet its origins remain a mystery. The topic of consciousness has gained traction in recent years, thanks to the development of large language models that now arguably pass the Turing test, an operational test for intelligence. However, intelligence and consciousness are not related in obvious ways, as anyone who suffers from a bad toothache can attest-pain generates intense feelings and absorbs all our conscious awareness, yet nothing particularly intelligent is going on. In the hard sciences, this topic is frequently met with skepticism because, to date, no protocol to measure the content or intensity of conscious experiences in an observer-independent manner has been agreed upon. Here, we present a novel proposal: Conscious experience arises whenever a quantum mechanical superposition forms. Our proposal has several implications: First, it suggests that the structure of the superposition determines the qualia of the experience. Second, quantum entanglement naturally solves the binding problem, ensuring the unity of phenomenal experience. Finally, a moment of agency may coincide with the formation of a superposition state. We outline a research program to experimentally test our conjecture via a sequence of quantum biology experiments. Applying these ideas opens up the possibility of expanding human conscious experience through brain-quantum computer interfaces.

From lab to life: assessing the impact of real-world interactions on the operation of rapid serial visual presentation-based brain-computer interfaces.

OBJECTIVE: Brain-computer interfaces (BCI) have been extensively researched in controlled lab settings where the P300 Event-related Potential (ERP), elicited in the Rapid Serial Visual Presentation (RSVP) paradigm, has shown promising potential. However, deploying BCIs outside of laboratory settings is challenging due to the presence of contaminating artifacts that often occur as a result of activities such as talking, head movements, and body movements. These artifacts can severely contaminate the measured EEG signals and consequently impede detection of the P300 ERP. Our goal is to assess the impact of these real-world noise factors on the performance of a RSVP-BCI, specifically focusing on single-trial P300 detection. Approach. In this study, we examine the impact of movement activity on the performance of a P300-based RSVP-BCI application designed to allow users to search images at high speed. Using machine learning, we assessed P300 detection performance using both EEG data captured in optimal recording conditions (e.g. where participants were instructed to refrain from moving) and a variety of conditions where the participant intentionally produced movements to contaminate the EEG recording. Main results. The results, presented as Area Under the Receiver Operating Characteristic Curve (ROC-AUC) scores, provide insight into the significant impact of noise on single-trial P300 detection. Notably, there is a reduction in classifier detection accuracy when intentionally contaminated RSVP trials are used for training and testing, when compared to using non-intentionally contaminated RSVP trials. Significance. Our findings underscore the necessity of addressing and mitigating noise in EEG recordings to facilitate the use of BCIs in real-world settings, thus extending the reach of EEG technology beyond the confines of the laboratory.

A deep neural network and transfer learning combined method for cross-task classification of error-related potentials.

Background: Error-related potentials (ErrPs) are electrophysiological responses that naturally occur when humans perceive wrongdoing or encounter unexpected events. It offers a distinctive means of comprehending the error-processing mechanisms within the brain. A method for detecting ErrPs with high accuracy holds significant importance for various ErrPs-based applications, such as human-in-the-loop Brain-Computer Interface (BCI) systems. Nevertheless, current methods fail to fulfill the generalization requirements for detecting such ErrPs due to the high non-stationarity of EEG signals across different tasks and the limited availability of ErrPs datasets. Methods: This study introduces a deep learning-based model that integrates convolutional layers and transformer encoders for the classification of ErrPs. Subsequently, a model training strategy, grounded in transfer learning, is proposed for the effective training of the model. The datasets utilized in this study are available for download from the publicly accessible databases. Results: In cross-task classification, an average accuracy of about 78% was achieved, exceeding the baseline. Furthermore, in the leave-one-subject-out, within-session, and cross-session classification scenarios, the proposed model outperformed the existing techniques with an average accuracy of 71.81, 78.74, and 77.01%, respectively. Conclusions: Our approach contributes to mitigating the challenge posed by limited datasets in the ErrPs field, achieving this by reducing the requirement for extensive training data for specific target tasks. This may serve as inspiration for future studies that concentrate on ErrPs and their applications.

STaRNet: A spatio-temporal and Riemannian network for high-performance motor imagery decoding.

Brain-computer interfaces (BCIs), representing a transformative form of human-computer interaction, empower users to interact directly with external environments through brain signals. In response to the demands for high accuracy, robustness, and end-to-end capabilities within BCIs based on motor imagery (MI), this paper introduces STaRNet, a novel model that integrates multi-scale spatio-temporal convolutional neural networks (CNNs) with Riemannian geometry. Initially, STaRNet integrates a multi-scale spatio-temporal feature extraction module that captures both global and local features, facilitating the construction of Riemannian manifolds from these comprehensive spatio-temporal features. Subsequently, a matrix logarithm operation transforms the manifold-based features into the tangent space, followed by a dense layer for classification. Without preprocessing, STaRNet surpasses state-of-the-art (SOTA) models by achieving an average decoding accuracy of 83.29% and a kappa value of 0.777 on the BCI Competition IV 2a dataset, and 95.45% accuracy with a kappa value of 0.939 on the High Gamma Dataset. Additionally, a comparative analysis between STaRNet and several SOTA models, focusing on the most challenging subjects from both datasets, highlights exceptional robustness of STaRNet. Finally, the visualizations of learned frequency bands demonstrate that temporal convolutions have learned MI-related frequency bands, and the t-SNE analyses of features across multiple layers of STaRNet exhibit strong feature extraction capabilities. We believe that the accurate, robust, and end-to-end capabilities of the STaRNet will facilitate the advancement of BCIs.

A Randomized Trial Utilizing EEG Brain Computer Interface to Improve Facial Emotion Recognition in Autistic Adults.

PURPOSE: Many individuals with autism spectrum disorder (ASD) experience challenges with facial emotion recognition (FER), which may exacerbate social difficulties in ASD. Few studies have examined whether FER can be experimentally manipulated and improved for autistic people. This study utilized a randomized controlled trial design to examine acceptability and preliminary clinical impact of a novel mixed reality-based neurofeedback program, FER Assistant, using EEG brain computer interface (BCI)-assisted technology to improve FER for autistic adolescents and adults. METHODS: Twenty-seven autistic male participants (M age: 21.12 years; M IQ: 105.78; 85% white) were randomized to the active condition to receive FER Assistant (n = 17) or waitlist control (n = 10). FER Assistant participants received ten sessions utilizing BCI-assisted neurofeedback training in FER. All participants, regardless of randomization, completed a computerized FER task at baseline and endpoint. RESULTS: Results partially indicated that FER Assistant was acceptable to participants. Regression analyses demonstrated that participation in FER Assistant led to group differences in FER at endpoint, compared to a waitlist control. However, analyses examining reliable change in FER indicated no reliable improvement or decline for FER Assistant participants, whereas two waitlist participants demonstrated reliable decline. CONCLUSION: Given the preliminary nature of this work, results collectively suggest that FER Assistant may be an acceptable intervention. Results also suggest that FER may be a potential mechanism that is amenable to intervention for autistic individuals, although additional trials using larger sample sizes are warranted.

Motor Imagery EEG Signal Classification Using Distinctive Feature Fusion with Adaptive Structural LASSO.

A motor imagery brain-computer interface connects the human brain and computers via electroencephalography (EEG). However, individual differences in the frequency ranges of brain activity during motor imagery tasks pose a challenge, limiting the manual feature extraction for motor imagery classification. To extract features that match specific subjects, we proposed a novel motor imagery classification model using distinctive feature fusion with adaptive structural LASSO. Specifically, we extracted spatial domain features from overlapping and multi-scale sub-bands of EEG signals and mined discriminative features by fusing the task relevance of features with spatial information into the adaptive LASSO-based feature selection. We evaluated the proposed model on public motor imagery EEG datasets, demonstrating that the model has excellent performance. Meanwhile, ablation studies and feature selection visualization of the proposed model further verified the great potential of EEG analysis.

The Effect of Caffeine on Movement-Related Cortical Potential Morphology and Detection.

Movement-related cortical potential (MRCP) is observed in EEG recordings prior to a voluntary movement. It has been used for e.g., quantifying motor learning and for brain-computer interfacing (BCIs). The MRCP amplitude is affected by various factors, but the effect of caffeine is underexplored. The aim of this study was to investigate if a cup of coffee with 85 mg caffeine modulated the MRCP amplitude and the classification of MRCPs versus idle activity, which estimates BCI performance. Twenty-six healthy participants performed 2 × 100 ankle dorsiflexion separated by a 10-min break before a cup of coffee was consumed, followed by another 100 movements. EEG was recorded during the movements and divided into epochs, which were averaged to extract three average MRCPs that were compared. Also, idle activity epochs were extracted. Features were extracted from the epochs and classified using random forest analysis. The MRCP amplitude did not change after consuming caffeine. There was a slight increase of two percentage points in the classification accuracy after consuming caffeine. In conclusion, a cup of coffee with 85 mg caffeine does not affect the MRCP amplitude, and improves MRCP-based BCI performance slightly. The findings suggest that drinking coffee is only a minor confounder in MRCP-related studies.

Dual model transfer learning to compensate for individual variability in brain-computer interface.

BACKGROUND AND OBJECTIVE: Recent advancements in brain-computer interface (BCI) technology have seen a significant shift towards incorporating complex decoding models such as deep neural networks (DNNs) to enhance performance. These models are particularly crucial for sophisticated tasks such as regression for decoding arbitrary movements. However, these BCI models trained and tested on individual data often face challenges with limited performance and generalizability across different subjects. This limitation is primarily due to a tremendous number of parameters of DNN models. Training complex models demands extensive datasets. Nevertheless, group data from many subjects may not produce sufficient decoding performance because of inherent variability in neural signals both across individuals and over time METHODS: To address these challenges, this study proposed a transfer learning approach that could effectively adapt to subject-specific variability in cortical regions. Our method involved training two separate movement decoding models: one on individual data and another on pooled group data. We then created a salience map for each cortical region from the individual model, which helped us identify the input's contribution variance across subjects. Based on the contribution variance, we combined individual and group models using a modified knowledge distillation framework. This approach allowed the group model to be universally applicable by assigning greater weights to input data, while the individual model was fine-tuned to focus on areas with significant individual variance RESULTS: Our combined model effectively encapsulated individual variability. We validated this approach with nine subjects performing arm-reaching tasks, with our method outperforming (mean correlation coefficient, r = 0.75) both individual (r = 0.70) and group models (r = 0.40) in decoding performance. In particular, there were notable improvements in cases where individual models showed low performances (e.g., r = 0.50 in the individual decoder to r = 0.61 in the proposed decoder) CONCLUSIONS: These results not only demonstrate the potential of our method for robust BCI, but also underscore its ability to generalize individual data for broader applicability.

A Convolutional Neural Network for SSVEP Identification by Using a Few-Channel EEG.

The application of wearable electroencephalogram (EEG) devices is growing in brain-computer interfaces (BCI) owing to their good wearability and portability. Compared with conventional devices, wearable devices typically support fewer EEG channels. Devices with few-channel EEGs have been proven to be available for steady-state visual evoked potential (SSVEP)-based BCI. However, fewer-channel EEGs can cause the BCI performance to decrease. To address this issue, an attention-based complex spectrum-convolutional neural network (atten-CCNN) is proposed in this study, which combines a CNN with a squeeze-and-excitation block and uses the spectrum of the EEG signal as the input. The proposed model was assessed on a wearable 40-class dataset and a public 12-class dataset under subject-independent and subject-dependent conditions. The results show that whether using a three-channel EEG or single-channel EEG for SSVEP identification, atten-CCNN outperformed the baseline models, indicating that the new model can effectively enhance the performance of SSVEP-BCI with few-channel EEGs. Therefore, this SSVEP identification algorithm based on a few-channel EEG is particularly suitable for use with wearable EEG devices.