Immediate brain plasticity after one hour of brain-computer interface (BCI).

KEY POINTS: Two groups of inexperienced brain-computer interface users underwent a purely mental EEG-BCI session that rapidly impacted on their brain. Modulations in structural and functional MRI were found after only 1 h of BCI training. Two different types of BCI (based on motor imagery or visually evoked potentials) were employed and analyses showed that the brain plastic changes are spatially specific for the respective neurofeedback. This spatial specificity promises tailored therapeutic interventions (e.g. for stroke patients). ABSTRACT: A brain-computer-interface (BCI) allows humans to control computational devices using only neural signals. However, it is still an open question, whether performing BCI also impacts on the brain itself, i.e. whether brain plasticity is induced. Here, we show rapid and spatially specific signs of brain plasticity measured with functional and structural MRI after only 1 h of purely mental BCI training in BCI-naive subjects. We employed two BCI approaches with neurofeedback based on (i) modulations of EEG rhythms by motor imagery (MI-BCI) or (ii) event-related potentials elicited by visually targeting flashing letters (ERP-BCI). Before and after the BCI session we performed structural and functional MRI. For both BCI approaches we found increased T1-weighted MR signal in the grey matter of the respective target brain regions, such as occipital/parietal areas after ERP-BCI and precuneus and sensorimotor regions after MI-BCI. The latter also showed increased functional connectivity and higher task-evoked BOLD activity in the same areas. Our results demonstrate for the first time that BCI by means of targeted neurofeedback rapidly impacts on MRI measures of brain structure and function. The spatial specificity of BCI-induced brain plasticity promises therapeutic interventions tailored to individual functional deficits, for example in patients after stroke.

A large scale screening study with a SMR-based BCI: Categorization of BCI users and differences in their SMR activity.

Brain-Computer Interfaces (BCIs) are inefficient for a non-negligible part of the population, estimated around 25%. To understand this phenomenon in Sensorimotor Rhythm (SMR) based BCIs, data from a large-scale screening study conducted on 80 novice participants with the Berlin BCI system and its standard machine-learning approach were investigated. Each participant performed one BCI session with resting state Encephalography, Motor Observation, Motor Execution and Motor Imagery recordings and 128 electrodes. A significant portion of the participants (40%) could not achieve BCI control (feedback performance > 70%). Based on the performance of the calibration and feedback runs, BCI users were stratified in three groups. Analyses directed to detect and elucidate the differences in the SMR activity of these groups were performed. Statistics on reactive frequencies, task prevalence and classification results are reported. Based on their SMR activity, also a systematic list of potential reasons leading to performance drops and thus hints for possible improvements of BCI experimental design are given. The categorization of BCI users has several advantages, allowing researchers 1) to select subjects for further analyses as well as for testing new BCI paradigms or algorithms, 2) to adopt a better subject-dependent training strategy and 3) easier comparisons between different studies.

Ensembles of adaptive spatial filters increase BCI performance: an online evaluation.

OBJECTIVE: In electroencephalographic (EEG) data, signals from distinct sources within the brain are widely spread by volume conduction and superimposed such that sensors receive mixtures of a multitude of signals. This reduction of spatial information strongly hampers single-trial analysis of EEG data as, for example, required for brain-computer interfacing (BCI) when using features from spontaneous brain rhythms. Spatial filtering techniques are therefore greatly needed to extract meaningful information from EEG. Our goal is to show, in online operation, that common spatial pattern patches (CSPP) are valuable to counteract this problem. APPROACH: Even though the effect of spatial mixing can be encountered by spatial filters, there is a trade-off between performance and the requirement of calibration data. Laplacian derivations do not require calibration data at all, but their performance for single-trial classification is limited. Conversely, data-driven spatial filters, such as common spatial patterns (CSP), can lead to highly distinctive features; however they require a considerable amount of training data. Recently, we showed in an offline analysis that CSPP can establish a valuable compromise. In this paper, we confirm these results in an online BCI study. In order to demonstrate the paramount feature that CSPP requires little training data, we used them in an adaptive setting with 20 participants and focused on users who did not have success with previous BCI approaches. MAIN RESULTS: The results of the study show that CSPP adapts faster and thereby allows users to achieve better feedback within a shorter time than previous approaches performed with Laplacian derivations and CSP filters. The success of the experiment highlights that CSPP has the potential to further reduce BCI inefficiency. SIGNIFICANCE: CSPP are a valuable compromise between CSP and Laplacian filters. They allow users to attain better feedback within a shorter time and thus reduce BCI inefficiency to one-fourth in comparison to previous non-adaptive paradigms.

Extracting latent brain states--Towards true labels in cognitive neuroscience experiments.

Neuroscientific data is typically analyzed based on the behavioral response of the participant. However, the errors made may or may not be in line with the neural processing. In particular in experiments with time pressure or studies where the threshold of perception is measured, the error distribution deviates from uniformity due to the structure in the underlying experimental set-up. When we base our analysis on the behavioral labels as usually done, then we ignore this problem of systematic and structured (non-uniform) label noise and are likely to arrive at wrong conclusions in our data analysis. This paper contributes a remedy to this important scenario: we present a novel approach for a) measuring label noise and b) removing structured label noise. We demonstrate its usefulness for EEG data analysis using a standard d2 test for visual attention (N=20 participants).

Pre-stimulus sensorimotor rhythms influence brain-computer interface classification performance.

The influence of pre-stimulus ongoing brain activity on post-stimulus task performance has recently been analyzed in several studies. While pre-stimulus activity in the parieto-occipital area has been exhaustively investigated with congruent results, less is known about the sensorimotor areas, for which studies reported inconsistent findings. In this work, the topic is addressed in a brain-computer interface (BCI) setting based on modulations of sensorimotor rhythms (SMR). The goal is to assess whether and how pre-stimulus SMR activity influences the successive task execution quality and consequently the classification performance. Grand average data of 23 participants performing right and left hand motor imagery were analyzed. Trials were separated into two groups depending on the SMR amplitude in the 1000 ms interval preceding the cue, and classification by common spatial patterns (CSPs) preprocessing and linear discriminant analysis (LDA) was carried out in the post-stimulus time interval, i.e., during the task execution. The correlation between trial group and classification performance was assessed by an analysis of variance. As a result of this analysis, trials with higher SMR amplitude in the 1000 ms interval preceding the cue yielded significantly better classification performance than trials with lower amplitude. A further investigation of brain activity patterns revealed that this increase in accuracy is mainly due to the persistence of a higher SMR amplitude over the ipsilateral hemisphere. Our findings support the idea that exploiting information about the ongoing SMR might be the key to boosting performance in future SMR-BCI experiments and motor related tasks in general.

Psychological predictors of SMR-BCI performance.

BACKGROUND: After about 30 years of research on Brain-Computer Interfaces (BCIs) there is little knowledge about the phenomenon, that some people - healthy as well as individuals with disease - are not able to learn BCI-control. To elucidate this "BCI-inefficiency" phenomenon, the current study investigated whether psychological parameters, such as attention span, personality or motivation, could predict performance in a single session with a BCI controlled by modulation of sensorimotor rhythms (SMR) with motor imagery. METHODS: A total of N=83 healthy BCI novices took part in the session. Psychological parameters were measured with an electronic test-battery including clinical, personality and performance tests. Predictors were determined by binary logistic regression analyses. RESULTS: The output variable of the Two-Hand Coordination Test (2HAND) "overall mean error duration" which is a measure for the accuracy of fine motor skills accounted for 11% of the variance in BCI-inefficiency. The Attitudes Towards Work (AHA) test variable "performance level" which can be interpreted as a degree of concentration and a neurophysiological SMR predictor were also identified as significant predictors of SMR BCI performance. CONCLUSION: Psychological parameters as measured in this study play a moderate role for one-session performance in a BCI controlled by modulation of SMR.

Common Spatial Pattern Patches: online evaluation on BCI-naive users.

Brain-Computer Interfaces (BCI) based on the voluntary modulation of sensorimotor rhythms (SMRs) induced by motor imagery are very prominent because allow a continuous control of the external device. Nevertheless, the design of a SMR-based BCI system that provides every user with a reliable BCI control from the first session, i.e., without extensive training, is still a big challenge. Considerable advances in this direction have been made by the machine learning co-adaptive calibration approach, which combines online adaptation techniques with subject learning in order to offer the user a feedback from the beginning of the experiment. Recently, based on offline analyses, we proposed the novel Common Spatial Patterns Patches (CSPP) technique as a good candidate to improve the co-adaptive calibration. CSPP is an ensemble of localized spatial filters, each of them optimized on subject-specific data by CSP analysis. Here, the evaluation of CSPP in online operation is presented for the first time. Results on three BCI-naive participants show indeed promising results. All three users reach the threshold criterion of 70% accuracy within one session, even one candidate for whom the weak SMR at rest predicted deficient BCI control. Concurrent recordings of the SMR during a relax condition as well as the course of BCI performance indicate a clear learning effect.

Co-adaptive calibration to improve BCI efficiency.

All brain-computer interface (BCI) groups that have published results of studies involving a large number of users performing BCI control based on the voluntary modulation of sensorimotor rhythms (SMR) report that BCI control could not be achieved by a non-negligible number of subjects (estimated 20% to 25%). This failure of the BCI system to read the intention of the user is one of the greatest problems and challenges in BCI research. There are two main causes for this problem in SMR-based BCI systems: either no idle SMR is observed over motor areas of the user, or this idle rhythm is not modulated during motor imagery, resulting in a classification performance lower than 70% (criterion level) that renders the control of a BCI application (like a speller) difficult or impossible. Previously, we introduced the concept of machine learning based co-adaptive calibration, which provided substantially improved performance for a variety of users. Here, we use a similar approach and investigate to what extent co-adaptive learning enables significant BCI control for completely novice users, as well as for those who could not achieve control with a conventional SMR-based BCI.

CSP patches: an ensemble of optimized spatial filters. An evaluation study.

Laplacian filters are widely used in neuroscience. In the context of brain-computer interfacing, they might be preferred to data-driven approaches such as common spatial patterns (CSP) in a variety of scenarios such as, e.g., when no or few user data are available or a calibration session with a multi-channel recording is not possible, which is the case in various applications. In this paper we propose the use of an ensemble of local CSP patches (CSPP) which can be considered as a compromise between Laplacian filters and CSP. Our CSPP only needs a very small number of trials to be optimized and significantly outperforms Laplacian filters in all settings studied. Additionally, CSPP also outperforms multi-channel CSP and a regularized version of CSP even when only very few calibration data are available, acting as a CSP regularizer without the need of additional hyperparameters and at a very low cost: 2-5 min of data recording, i.e. ten times less than CSP.

Machine-learning-based coadaptive calibration for brain-computer interfaces.

Brain-computer interfaces (BCIs) allow users to control a computer application by brain activity as acquired (e.g., by EEG). In our classic machine learning approach to BCIs, the participants undertake a calibration measurement without feedback to acquire data to train the BCI system. After the training, the user can control a BCI and improve the operation through some type of feedback. However, not all BCI users are able to perform sufficiently well during feedback operation. In fact, a nonnegligible portion of participants (estimated 15%-30%) cannot control the system (a BCI illiteracy problem, generic to all motor-imagery-based BCIs). We hypothesize that one main difficulty for a BCI user is the transition from offline calibration to online feedback. In this work, we investigate adaptive machine learning methods to eliminate offline calibration and analyze the performance of 11 volunteers in a BCI based on the modulation of sensorimotor rhythms. We present an adaptation scheme that individually guides the user. It starts with a subject-independent classifier that evolves to a subject-optimized state-of-the-art classifier within one session while the user interacts continuously. These initial runs use supervised techniques for robust coadaptive learning of user and machine. Subsequent runs use unsupervised adaptation to track the features' drift during the session and provide an unbiased measure of BCI performance. Using this approach, without any offline calibration, six users, including one novice, obtained good performance after 3 to 6 minutes of adaptation. More important, this novel guided learning also allows participants with BCI illiteracy to gain significant control with the BCI in less than 60 minutes. In addition, one volunteer without sensorimotor idle rhythm peak at the beginning of the BCI experiment developed it during the course of the session and used voluntary modulation of its amplitude to control the feedback application.

Large-scale EEG/MEG source localization with spatial flexibility.

We propose a novel approach to solving the electro-/magnetoencephalographic (EEG/MEG) inverse problem which is based upon a decomposition of the current density into a small number of spatial basis fields. It is designed to recover multiple sources of possibly different extent and depth, while being invariant with respect to phase angles and rotations of the coordinate system. We demonstrate the method's ability to reconstruct simulated sources of random shape and show that the accuracy of the recovered sources can be increased, when interrelated field patterns are co-localized. Technically, this leads to large-scale mathematical problems, which are solved using recent advances in convex optimization. We apply our method for localizing brain areas involved in different types of motor imagery using real data from Brain-Computer Interface (BCI) sessions. Our approach based on single-trial localization of complex Fourier coefficients yields class-specific focal sources in the sensorimotor cortices.

The Berlin Brain-Computer Interface: Non-Medical Uses of BCI Technology.

Brain-computer interfacing (BCI) is a steadily growing area of research. While initially BCI research was focused on applications for paralyzed patients, increasingly more alternative applications in healthy human subjects are proposed and investigated. In particular, monitoring of mental states and decoding of covert user states have seen a strong rise of interest. Here, we present some examples of such novel applications which provide evidence for the promising potential of BCI technology for non-medical uses. Furthermore, we discuss distinct methodological improvements required to bring non-medical applications of BCI technology to a diversity of layperson target groups, e.g., ease of use, minimal training, general usability, short control latencies.

Localization of class-related mu-rhythm desynchronization in motor imagery based brain-computer interface sessions.

We localize the sources of class-dependent event-related desynchronisation (ERD) of the mu-rhythm related to different types of motor imagery in Brain-Computer Interfacing (BCI) sessions. Our approach is based on localization of single-trial Fourier coefficients using sparse basis field expansions (S-FLEX). The analysis reveals focal sources in the sensorimotor cortices, a finding which can be regarded as a proof for the expected neurophysiological origin of the BCI control signal. As a technical contribution, we extend S-FLEX to the multiple measurement case in a way that the activity of different frequency bins within the mu-band is coherently localized.

Common spatial pattern patches - an optimized filter ensemble for adaptive brain-computer interfaces.

Laplacian filters are commonly used in Brain Computer Interfacing (BCI). When only data from few channels are available, or when, like at the beginning of an experiment, no previous data from the same user is available complex features cannot be used. In this case band power features calculated from Laplacian filtered channels represents an easy, robust and general feature to control a BCI, since its calculation does not involve any class information. For the same reason, the performance obtained with Laplacian features is poor in comparison to subject-specific optimized spatial filters, such as Common Spatial Patterns (CSP) analysis, which, on the other hand, can be used just in a later phase of the experiment, since they require a considerable amount of training data in order to enroll a stable and good performance. This drawback is particularly evident in case of poor performing BCI users, whose data is highly non-stationary and contains little class relevant information. Therefore, Laplacian filtering is preferred to CSP, e.g., in the initial period of co-adaptive calibration, a novel BCI paradigm designed to alleviate the problem of BCI illiteracy. In fact, in the co-adaptive calibration design the experiment starts with a subject-independent classifier and simple features are needed in order to obtain a fast adaptation of the classifier to the newly acquired user's data. Here, the use of an ensemble of local CSP patches (CSPP) is proposed, which can be considered as a compromise between Laplacians and CSP: CSPP needs less data and channels than CSP, while being superior to Laplacian filtering. This property is shown to be particularly useful for the co-adaptive calibration design and is demonstrated on off-line data from a previous co-adaptive BCI study.

Neurophysiological predictor of SMR-based BCI performance.

Brain-computer interfaces (BCIs) allow a user to control a computer application by brain activity as measured, e.g., by electroencephalography (EEG). After about 30years of BCI research, the success of control that is achieved by means of a BCI system still greatly varies between subjects. For about 20% of potential users the obtained accuracy does not reach the level criterion, meaning that BCI control is not accurate enough to control an application. The determination of factors that may serve to predict BCI performance, and the development of methods to quantify a predictor value from psychological and/or physiological data serve two purposes: a better understanding of the 'BCI-illiteracy phenomenon', and avoidance of a costly and eventually frustrating training procedure for participants who might not obtain BCI control. Furthermore, such predictors may lead to approaches to antagonize BCI illiteracy. Here, we propose a neurophysiological predictor of BCI performance which can be determined from a two minute recording of a 'relax with eyes open' condition using two Laplacian EEG channels. A correlation of r=0.53 between the proposed predictor and BCI feedback performance was obtained on a large data base with N=80 BCI-naive participants in their first session with the Berlin brain-computer interface (BBCI) system which operates on modulations of sensory motor rhythms (SMRs).

On optimal channel configurations for SMR-based brain-computer interfaces.

One crucial question in the design of electroencephalogram (EEG)-based brain-computer interface (BCI) experiments is the selection of EEG channels. While a setup with few channels is more convenient and requires less preparation time, a dense placement of electrodes provides more detailed information and henceforth could lead to a better classification performance. Here, we investigate this question for a specific setting: a BCI that uses the popular CSP algorithm in order to classify voluntary modulations of sensorimotor rhythms (SMR). In a first approach 13 different fixed channel configurations are compared to the full one consisting of 119 channels. The configuration with 48 channels results to be the best one, while configurations with less channels, from 32 to 8, performed not significantly worse than the best configuration in cases where only few training trials are available. In a second approach an optimal channel configuration is obtained by an iterative procedure in the spirit of stepwise variable selection with nonparametric multiple comparisons. As a surprising result, in the second approach a setting with 22 channels centered over the motor areas was selected. Thanks to the acquisition of a large data set recorded from 80 novice participants using 119 EEG channels, the results of this study can be expected to have a high degree of generalizability.

Improving BCI performance by task-related trial pruning.

Noise in electroencephalography data (EEG) is an ubiquitous problem that limits the performance of brain computer interfaces (BCI). While typical EEG artifacts are usually removed by trial rejection or by filtering, noise induced in the data by the subject's failure to produce the required mental state is very harmful. Such "noise" effects are rather common, especially for naive subjects in their training phase and, thus, standard artifact removal methods would inevitably fail. In this paper, we present a novel method which aims to detect such defected trials taking into account the intended task by use of Relevant Dimensionality Estimation (RDE), a new machine learning method for denoising in feature space. In this manner, our method effectively "cleans" the training data and thus allows better BCI classification. Preliminary results conducted on a data set of 43 naive subjects show a significant improvement for 74% of the subjects.