A visual brain-computer interface as communication aid for patients with amyotrophic lateral sclerosis.

OBJECTIVE: Brain-Computer Interface (BCI) spellers that make use of code-modulated Visual Evoked Potentials (cVEP) may provide a fast and more accurate alternative to existing visual BCI spellers for patients with Amyotrophic Lateral Sclerosis (ALS). However, so far the cVEP speller has only been tested on healthy participants. METHODS: We assess the brain responses, BCI performance and user experience of the cVEP speller in 20 healthy participants and 10 ALS patients. All participants performed a cued and free spelling task, and a free selection of Yes/No answers. RESULTS: 27 out of 30 participants could perform the cued spelling task with an average accuracy of 79% for ALS patients, 88% for healthy older participants and 94% for healthy young participants. All 30 participants could answer Yes/No questions freely, with an average accuracy of around 90%. CONCLUSIONS: With ALS patients typing on average 10 characters per minute, the cVEP speller presented in this paper outperforms other visual BCI spellers. SIGNIFICANCE: These results support a general usability of cVEP signals for ALS patients, which may extend far beyond the tested speller to control e.g. an alarm, automatic door, or TV within a smart home.

Electrophysiological responses of relatedness to consecutive word stimuli in relation to an actively recollected target word.

In this paper, we investigate the robustness of electrophysiological responses of relatedness to multiple consecutive word stimuli (probes), in relation to an actively recollected target word. Such relatedness information could be used by a Brain Computer Interface to infer the active semantic concept on a user's mind, by integrating the knowledge of the relationship between the multiple probe words and the 'unknown' target. Such a BCI can take advantage of the N400: an event related potential that is sensitive to semantic content of a stimulus in relation to an established semantic context. However, it is unknown whether the N400 is suited for the multiple probing paradigm we propose, as other intervening words might distract from the established context (i.e., the target word). We perform an experiment in which we present up to ten words after an initial target word, and find no attenuation of the strength of the N400 in grand average ERPs and no decrease in classification accuracy for probes occurring later in the sequences. These results are groundwork for developing a BCI that infers the concept on a user's mind through repeated probing, however, low single trial decoding accuracy, and high subject variability may limit practical applicability.

Free Wally: Where motor intentions meet reason and consequence.

To investigate the neural preparation and awareness of an intention to act, neuroscientists typically examine spontaneous movements: self-paced flexions of the hand or foot. However, these movements may not present a straightforward case of intended action as they are performed in absence of reasons to act and without the evaluation of action consequences. Therefore, a common criticism of these studies is that they lack ecological validity, because the results do not generalize to the more societally relevant deliberate actions that we perform in daily life. We agree that research on intended action should include reason-based deliberate actions in order to be more relevant for debates about human agency and free will. Therefore, we have developed a computer game called "Free Wally", which invites players to perform deliberate actions to achieve a goal. Free Wally provides a controlled environment for studying deliberate intended action, by presenting information for deciding whether or not to act, what action to perform and when to perform it, incorporating all basic components of an ecologically valid intended act. As a first step to validate our setup, we compare this game to a second computer game that measures spontaneous actions in a traditional way. While playing either game, the timing of the experienced intentions to act is measured using a real-time probing method. Moreover, the neural preparation for action is measured in terms of the (lateralized) readiness potential and alpha/beta event-related desynchronization across the motor cortex. No differences were found between the games in these last stages of action preparation, suggesting that the Free Wally game can be used to study intended action. However, differences in earlier stages during intention formation are to be expected. With Free Wally as a tool, we hope to encourage further research into the formation and content of ecologically valid motor intentions.

Probing for Intentions: Why Clocks Do Not Provide the Only Measurement of Time.

Having an intention to act is commonly operationalized as the moment at which awareness of an urge or decision to act arises. Measuring this moment has been challenging due to the dependence on first-person reports of subjective experience rather than objective behavioral or neural measurements. Commonly, this challenge is met using (variants of) Libet's clock method. In 2008, Matsuhashi and Hallett published a novel probing strategy as an alternative to the clock method. We believe their probe method could provide a valuable addition to the clock method because: it measures the timing of an intention in real-time, it can be combined with additional (tactile, visual or auditory) stimuli to create a more ecologically valid experimental context, and it allows the measurement of the point of no return. Yet to this date, the probe method has not been applied widely - possibly due to concerns about the effects that the probes might have on the intention and/or action preparation processes. To address these concerns, a 2 × 2 within-subject design is tested. In this design, two variables are manipulated: (1) the requirement of an introspection report and (2) the presence of an auditory probe. Three observables are measured that provide information about the timing of an intention to act: (1) awareness reports of the subjective experience of having an intention, (2) neural preparatory activity for action, and (3) behavioral data of the performed actions. The presence of probes was found to speed up mean action times by roughly 300 ms, but did not alter the neural preparation for action. The requirement of an introspection report did influence brain signals: reducing the amplitude of the readiness potential and increasing the desynchronization in the alpha and beta bands over the motor cortex prior to action onset. By discussing the strengths and weaknesses of the probe method compared to the clock method, we hope to demonstrate its added value and promote its use in future research.

Monitoring of language selection errors in switching: Not all about conflict.

Although bilingual speakers are very good at selectively using one language rather than another, sometimes language selection errors occur. To investigate how bilinguals monitor their speech errors and control their languages in use, we recorded event-related potentials (ERPs) in unbalanced Dutch-English bilingual speakers in a cued language-switching task. We tested the conflict-based monitoring model of Nozari and colleagues by investigating the error-related negativity (ERN) and comparing the effects of the two switching directions (i.e., to the first language, L1 vs. to the second language, L2). Results show that the speakers made more language selection errors when switching from their L2 to the L1 than vice versa. In the EEG, we observed a robust ERN effect following language selection errors compared to correct responses, reflecting monitoring of speech errors. Most interestingly, the ERN effect was enlarged when the speakers were switching to their L2 (less conflict) compared to switching to the L1 (more conflict). Our findings do not support the conflict-based monitoring model. We discuss an alternative account in terms of error prediction and reinforcement learning.

Preserved and attenuated electrophysiological correlates of visual spatial attention in elderly subjects.

Healthy aging is associated with changes in many neurocognitive functions. While on the behavioral level, visual spatial attention capacities are relatively stable with increasing age, the underlying neural processes change. In this study, we investigated attention-related modulations of the stimulus-locked event-related potential (ERP) and occipital oscillations in the alpha band (8-14Hz) in young and elderly participants. Both groups performed a visual attention task equally well and the ERP showed comparable attention-related modulations in both age groups. However, in elderly subjects, oscillations in the alpha band were massively reduced both during the task and in the resting state and the typical task-related lateralized pattern of alpha activity was not observed. These differences between young and elderly participants were observed on the group level as well as on the single trial level. The results indicate that younger and older adults use different neural strategies to reach the same performance in a covert visual spatial attention task.

Detecting traces of consciousness in the process of intending to act.

An intention to act has different onsets when it is measured in different ways. When participants provide a self-initiated report on the onset of their awareness of intending to act, the report occurs around 150 ms prior to action. However, when the same participants are repeatedly asked about their awareness of intending at different points in time, the onset of intending is found up to 2 s prior to action. This 'probed' awareness has its onset around the same time as the brain starts preparing the act, as measured using EEG. First of all, this undermines straightforward interpretations about the temporal relation between unconscious brain states and conscious intentions and actions. Secondly, we suggest that these results present a problem for the view that intentions are mental states occurring at a single point in time. Instead, we suggest the results to support the interpretation of an intention to act as a multistage process developing over time. This process of intending seems to develop during the process of acting, leaving reportable traces in consciousness at certain points along the road.

Decoding motor responses from the EEG during altered states of consciousness induced by propofol.

OBJECTIVE: Patients undergoing general anesthesia may awaken and become aware of the surgical procedure. Due to neuromuscular blocking agents, patients could be conscious yet unable to move. Using brain-computer interface (BCI) technology, it may be possible to detect movement attempts from the EEG. However, it is unknown how an anesthetic influences the brain response to motor tasks. APPROACH: We tested the offline classification performance of a movement-based BCI in 12 healthy subjects at two effect-site concentrations of propofol. For each subject a second classifier was trained on the subject's data obtained before sedation, then tested on the data obtained during sedation ('transfer classification'). MAIN RESULTS: At concentration 0.5 µg ml(-1), despite an overall propofol EEG effect, the mean single trial classification accuracy was 85% (95% CI 81%-89%), and 83% (79%-88%) for the transfer classification. At 1.0 µg ml(-1), the accuracies were 81% (76%-86%), and 72% (66%-79%), respectively. At the highest propofol concentration for four subjects, unlike the remaining subjects, the movement-related brain response had been largely diminished, and the transfer classification accuracy was not significantly above chance. These subjects showed a slower and more erratic task response, indicating an altered state of consciousness distinct from that of the other subjects. SIGNIFICANCE: The results show the potential of using a BCI to detect intra-operative awareness and justify further development of this paradigm. At the same time, the relationship between motor responses and consciousness and its clinical relevance for intraoperative awareness requires further investigation.

Optimal Multitrial Prediction Combination and Subject-Specific Adaptation for Minimal Training Brain Switch Designs.

Brain-Computer Interface (BCI) systems are traditionally designed by taking into account user-specific data to enable practical use. More recently, subject independent (SI) classification algorithms have been developed which bypass the subject specific adaptation and enable rapid use of the system. A brain switch is a particular BCI system where the system is required to distinguish from two separate mental tasks corresponding to the on-off commands of a switch. Such applications require a low false positive rate (FPR) while having an acceptable response time (RT) until the switch is activated. In this work, we develop a methodology that produces optimal brain switch behavior through subject specific (SS) adaptation of: a) a multitrial prediction combination model and b) an SI classification model. We propose a statistical model of combining classifier predictions that enables optimal FPR calibration through a short calibration session. We trained an SI classifier on a training synchronous dataset and tested our method on separate holdout synchronous and asynchronous brain switch experiments. Although our SI model obtained similar performance between training and holdout datasets, 86% and 85% for the synchronous and 69% and 66% for the asynchronous the between subject FPR and TPR variability was high (up to 62%). The short calibration session was then employed to alleviate that problem and provide decision thresholds that achieve when possible a target FPR=1% with good accuracy for both datasets.

Classifying Regularized Sensor Covariance Matrices: An Alternative to CSP.

Common spatial patterns (CSP) is a commonly used technique for classifying imagined movement type brain-computer interface (BCI) datasets. It has been very successful with many extensions and improvements on the basic technique. However, a drawback of CSP is that the signal processing pipeline contains two supervised learning stages: the first in which class- relevant spatial filters are learned and a second in which a classifier is used to classify the filtered variances. This may lead to potential overfitting issues, which are generally avoided by limiting CSP to only a few filters.

Decoding Sensorimotor Rhythms during Robotic-Assisted Treadmill Walking for Brain Computer Interface (BCI) Applications.

Locomotor malfunction represents a major problem in some neurological disorders like stroke and spinal cord injury. Robot-assisted walking devices have been used during rehabilitation of patients with these ailments for regaining and improving walking ability. Previous studies showed the advantage of brain-computer interface (BCI) based robot-assisted training combined with physical therapy in the rehabilitation of the upper limb after stroke. Therefore, stroke patients with walking disorders might also benefit from using BCI robot-assisted training protocols. In order to develop such BCI, it is necessary to evaluate the feasibility to decode walking intention from cortical patterns during robot-assisted gait training. Spectral patterns in the electroencephalogram (EEG) related to robot-assisted active and passive walking were investigated in 10 healthy volunteers (mean age 32.3±10.8, six female) and in three acute stroke patients (all male, mean age 46.7±16.9, Berg Balance Scale 20±12.8). A logistic regression classifier was used to distinguish walking from baseline in these spectral EEG patterns. Mean classification accuracies of 94.0±5.4% and 93.1±7.9%, respectively, were reached when active and passive walking were compared against baseline. The classification performance between passive and active walking was 83.4±7.4%. A classification accuracy of 89.9±5.7% was achieved in the stroke patients when comparing walking and baseline. Furthermore, in the healthy volunteers modulation of low gamma activity in central midline areas was found to be associated with the gait cycle phases, but not in the stroke patients. Our results demonstrate the feasibility of BCI-based robotic-assisted training devices for gait rehabilitation.

Using Actual and Imagined Walking Related Desynchronization Features in a BCI.

Recently, brain-computer interface (BCI) research has extended to investigate its possible use in motor rehabilitation. Most of these investigations have focused on the upper body. Only few studies consider gait because of the difficulty of recording EEG during gross movements. However, for stroke patients the rehabilitation of gait is of crucial importance. Therefore, this study investigates if a BCI can be based on walking related desynchronization features. Furthermore, the influence of complexity of the walking movements on the classification performance is investigated. Two BCI experiments were conducted in which healthy subjects performed a cued walking task, a more complex walking task (backward or adaptive walking), and imagination of the same tasks. EEG data during these tasks was classified into walking and no-walking. The results from both experiments show that despite the automaticity of walking and recording difficulties, brain signals related to walking could be classified rapidly and reliably. Classification performance was higher for actual walking movements than for imagined walking movements. There was no significant increase in classification performance for both the backward and adaptive walking tasks compared with the cued walking tasks. These results are promising for developing a BCI for the rehabilitation of gait.

Detection of attempted movement from the EEG during neuromuscular block: proof of principle study in awake volunteers.

Brain-Computer Interfaces (BCIs) have the potential to detect intraoperative awareness during general anaesthesia. Traditionally, BCI research is aimed at establishing or improving communication and control for patients with permanent paralysis. Patients experiencing intraoperative awareness also lack the means to communicate after administration of a neuromuscular blocker, but may attempt to move. This study evaluates the principle of detecting attempted movements from the electroencephalogram (EEG) during local temporary neuromuscular blockade. EEG was obtained from four healthy volunteers making 3-second hand movements, both before and after local administration of rocuronium in one isolated forearm. Using offline classification analysis we investigated whether the attempted movements the participants made during paralysis could be distinguished from the periods when they did not move or attempt to move. Attempted movement trials were correctly identified in 81 (68-94)% (mean (95% CI)) and 84 (74-93)% of the cases using 30 and 9 EEG channels, respectively. Similar accuracies were obtained when training the classifier on the participants' actual movements. These results provide proof of the principle that a BCI can detect movement attempts during neuromuscular blockade. Based on this, in the future a BCI may serve as a communication channel between a patient under general anaesthesia and the anaesthesiologist.

Broad-Band Visually Evoked Potentials: Re(con)volution in Brain-Computer Interfacing.

Brain-Computer Interfaces (BCIs) allow users to control devices and communicate by using brain activity only. BCIs based on broad-band visual stimulation can outperform BCIs using other stimulation paradigms. Visual stimulation with pseudo-random bit-sequences evokes specific Broad-Band Visually Evoked Potentials (BBVEPs) that can be reliably used in BCI for high-speed communication in speller applications. In this study, we report a novel paradigm for a BBVEP-based BCI that utilizes a generative framework to predict responses to broad-band stimulation sequences. In this study we designed a BBVEP-based BCI using modulated Gold codes to mark cells in a visual speller BCI. We defined a linear generative model that decomposes full responses into overlapping single-flash responses. These single-flash responses are used to predict responses to novel stimulation sequences, which in turn serve as templates for classification. The linear generative model explains on average 50% and up to 66% of the variance of responses to both seen and unseen sequences. In an online experiment, 12 participants tested a 6 × 6 matrix speller BCI. On average, an online accuracy of 86% was reached with trial lengths of 3.21 seconds. This corresponds to an Information Transfer Rate of 48 bits per minute (approximately 9 symbols per minute). This study indicates the potential to model and predict responses to broad-band stimulation. These predicted responses are proven to be well-suited as templates for a BBVEP-based BCI, thereby enabling communication and control by brain activity only.

Lost in time...: The search for intentions and Readiness Potentials.

In 1983 Libet et al. found that the Readiness Potential (RP) precedes the intention to act by 350ms and the actual movement by 500ms on average. Using our own replication study, we illustrate how seemingly innocuous technical details are actually crucially relevant to the debate surrounding the interpretation of Libet-style experiments. For instance, using one specific method for determining the RP onset actually led to a reversal of Libet's results (i.e., the intention preceded the RP onset) for one of the participants. Claims regarding the causal relation between RP and intention cannot be based on averages, but require individual, case by case analyses, which show no exceptions in the temporal relation between RP and intention. We conclude that, properly speaking, Libet-style results in themselves cannot yet be taken as proof for the type of conclusions that are often formulated regarding the non-existence of free will.

Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia.

Combining electrophysiological and hemodynamic features is a novel approach for improving current performance of brain switches based on sensorimotor rhythms (SMR). This study was conducted with a dual purpose: to test the feasibility of using a combined electroencephalogram/functional near-infrared spectroscopy (EEG-fNIRS) SMR-based brain switch in patients with tetraplegia, and to examine the performance difference between motor imagery and motor attempt for this user group. A general improvement was found when using both EEG and fNIRS features for classification as compared to using the single-modality EEG classifier, with average classification rates of 79% for attempted movement and 70% for imagined movement. For the control group, rates of 87% and 79% were obtained, respectively, where the "attempted movement" condition was replaced with "actual movement." A combined EEG-fNIRS system might be especially beneficial for users who lack sufficient control of current EEG-based brain switches. The average classification performance in the patient group for attempted movement was significantly higher than for imagined movement using the EEG-only as well as the combined classifier, arguing for the case of a paradigm shift in current brain switch research.

Towards a communication brain computer interface based on semantic relations.

This article investigates a possible Brain Computer Interface (BCI) based on semantic relations. The BCI determines which prime word a subject has in mind by presenting probe words using an intelligent algorithm. Subjects indicate when a presented probe word is related to the prime word by a single finger tap. The detection of the neural signal associated with this movement is used by the BCI to decode the prime word. The movement detector combined both the evoked (ERP) and induced (ERD) responses elicited with the movement. Single trial movement detection had an average accuracy of 67%. The decoding of the prime word had an average accuracy of 38% when using 100 probes and 150 possible targets, and 41% after applying a dynamic stopping criterium, reducing the average number of probes to 47. The article shows that the intelligent algorithm used to present the probe words has a significantly higher performance than a random selection of probes. Simulations demonstrate that the BCI also works with larger vocabulary sizes, and the performance scales logarithmically with vocabulary size.

Exploring the impact of target eccentricity and task difficulty on covert visual spatial attention and its implications for brain computer interfacing.

OBJECTIVE: Covert visual spatial attention is a relatively new task used in brain computer interfaces (BCIs) and little is known about the characteristics which may affect performance in BCI tasks. We investigated whether eccentricity and task difficulty affect alpha lateralization and BCI performance. APPROACH: We conducted a magnetoencephalography study with 14 participants who performed a covert orientation discrimination task at an easy or difficult stimulus contrast at either a near (3.5°) or far (7°) eccentricity. Task difficulty was manipulated block wise and subjects were aware of the difficulty level of each block. MAIN RESULTS: Grand average analyses revealed a significantly larger hemispheric lateralization of posterior alpha power in the difficult condition than in the easy condition, while surprisingly no difference was found for eccentricity. The difference between task difficulty levels was significant in the interval between 1.85 s and 2.25 s after cue onset and originated from a stronger decrease in the contralateral hemisphere. No significant effect of eccentricity was found. Additionally, single-trial classification analysis revealed a higher classification rate in the difficult (65.9%) than in the easy task condition (61.1%). No effect of eccentricity was found in classification rate. SIGNIFICANCE: Our results indicate that manipulating the difficulty of a task gives rise to variations in alpha lateralization and that using a more difficult task improves covert visual spatial attention BCI performance. The variations in the alpha lateralization could be caused by different factors such as an increased mental effort or a higher visual attentional demand. Further research is necessary to discriminate between them. We did not discover any effect of eccentricity in contrast to results of previous research.

Decoding speech perception by native and non-native speakers using single-trial electrophysiological data.

Brain-computer interfaces (BCIs) are systems that use real-time analysis of neuroimaging data to determine the mental state of their user for purposes such as providing neurofeedback. Here, we investigate the feasibility of a BCI based on speech perception. Multivariate pattern classification methods were applied to single-trial EEG data collected during speech perception by native and non-native speakers. Two principal questions were asked: 1) Can differences in the perceived categories of pairs of phonemes be decoded at the single-trial level? 2) Can these same categorical differences be decoded across participants, within or between native-language groups? Results indicated that classification performance progressively increased with respect to the categorical status (within, boundary or across) of the stimulus contrast, and was also influenced by the native language of individual participants. Classifier performance showed strong relationships with traditional event-related potential measures and behavioral responses. The results of the cross-participant analysis indicated an overall increase in average classifier performance when trained on data from all participants (native and non-native). A second cross-participant classifier trained only on data from native speakers led to an overall improvement in performance for native speakers, but a reduction in performance for non-native speakers. We also found that the native language of a given participant could be decoded on the basis of EEG data with accuracy above 80%. These results indicate that electrophysiological responses underlying speech perception can be decoded at the single-trial level, and that decoding performance systematically reflects graded changes in the responses related to the phonological status of the stimuli. This approach could be used in extensions of the BCI paradigm to support perceptual learning during second language acquisition.

Detecting semantic priming at the single-trial level.

Semantic priming is usually studied by examining ERPs over many trials and subjects. This article aims at detecting semantic priming at the single-trial level. By using machine learning techniques it is possible to analyse and classify short traces of brain activity, which could, for example, be used to build a Brain Computer Interface (BCI). This article describes an experiment where subjects were presented with word pairs and asked to decide whether the words were related or not. A classifier was trained to determine whether the subjects judged words as related or unrelated based on one second of EEG data. The results show that the classifier accuracy when training per subject varies between 54% and 67%, and is significantly above chance level for all subjects (N  = 12) and the accuracy when training over subjects varies between 51% and 63%, and is significantly above chance level for 11 subjects, pointing to a general effect.