Cooperation in mind: Motor imagery of joint and single actions is represented in different brain areas.

In this study brain activity during motor imagery (MI) of joint actions, compared to single actions and rest conditions, was investigated using functional magnetic resonance imaging (fMRI). To the best of our knowledge, this is the first neuroimaging study which directly investigated the neural correlates of joint action motor imagery. Twenty-one healthy participants imagined three different motor tasks (dancing, carrying a box, wiping). Each imagery task was performed at two kinds: alone (single action MI) or with a partner (joint action MI). We hypothesized that to imagine a cooperative task would lead to a stronger cortical activation in motor related areas due to a higher vividness and intensification of the imagery. This would be elicited by the integration of the action simulation of the virtual partner to one's own action. Comparing the joint action and the single action condition with the rest condition, we found significant activation in the precentral gyrus and precuneus respectively. Furthermore the joint action MI showed higher activation patterns in the premotor cortex (inferior and middle frontal gyrus) compared to the single action MI. The imagery of a more vivid and engaging task, like our joint action imagery, could improve rehabilitation processes since a more distributed brain activity is found. Furthermore, the joint action imagery compared to single action imagery might be an appropriate BCI task due to its clear spatial distinction of activation.

Continuous Erroneous Feedback Processing during Deviation from the Road within a 2D Steering Task.

Recent research of our group indicated that erroneous feedback processing can not only be detected via established correlates in the electroencephalogram (EEG) for discrete stimuli, but also arises as modulations of the brain signals when faced with a continuous and periodic error signal. However, limitations in our previous paradigm prevented a definitive statement on the error signal as the exclusive source of the modulations, as well as on the connection between the observed error-related negativity (ERN)-like and error positivity (Pe)-like continuous correlates. Within a new paradigm involving EEG recordings of 10 participants, we disentangled modulation sources, substantiating our hypothesis that the observed periodicity arises primarily due to feedback processing. Further, we provide evidence that the continuous ERN- and Pe-like potentials are locked to separate phases in the error signal, rather than time-locked to a shared event, indicating that both potentials arise independently of one another.

Short-lived brain state after cued motor imagery in naive subjects.

Multi-channel electroencephalography recordings have shown that a visual cue, indicating right hand, left hand or foot motor imagery, can induce a short-lived brain state in the order of about 500 ms. In the present study, 10 able-bodied subjects without any motor imagery experience (naive subjects) were asked to imagine the indicated limb movement for some seconds. Common spatial filtering and linear single-trial classification was applied to discriminate between two conditions (two brain states: right hand vs. left hand, left hand vs. foot and right hand vs. foot). The corresponding classification accuracies (mean +/- SD) were 80.0 +/- 10.6%, 83.3 +/- 10.2% and 83.6 +/- 8.8%, respectively. Inspection of central mu and beta rhythms revealed a short-lasting somatotopically specific event-related desynchronization (ERD) in the upper mu and/or beta bands starting approximately 300 ms after the cue onset and lasting for less than 1 s.

Self-initiation of EEG-based brain-computer communication using the heart rate response.

Self-initiation, that is the ability of a brain-computer interface (BCI) user to autonomously switch on and off the system, is a very important issue. In this work we analyze whether the respiratory heart rate response, induced by brisk inspiration, can be used as an additional communication channel. After only 20 min of feedback training, ten healthy subjects were able to self-initiate and operate a 4-class steady-state visual evoked potential-based (SSVEP) BCI by using one bipolar ECG and one bipolar EEG channel only. Threshold detection was used to measure a beat-to-beat heart rate increase. Despite this simple method, during a 30 min evaluation period on average only 2.9 non-intentional switches (heart rate changes) were detected.

15 years of BCI research at Graz University of Technology: current projects.

Over the last 15 years, the Graz Brain-Computer Interface (BCI) has been developed and all components such as feature extraction and classification, mode of operation, mental strategy, and type of feedback have been investigated. Recent projects deal with the development of asynchronous BCIs, the presentation of feedback and applications for communication and control.

From classic motor imagery to complex movement intention decoding: The noninvasive Graz-BCI approach.

In this chapter, we give an overview of the Graz-BCI research, from the classic motor imagery detection to complex movement intentions decoding. We start by describing the classic motor imagery approach, its application in tetraplegic end users, and the significant improvements achieved using coadaptive brain-computer interfaces (BCIs). These strategies have the drawback of not mirroring the way one plans a movement. To achieve a more natural control-and to reduce the training time-the movements decoded by the BCI need to be closely related to the user's intention. Within this natural control, we focus on the kinematic level, where movement direction and hand position or velocity can be decoded from noninvasive recordings. First, we review movement execution decoding studies, where we describe the decoding algorithms, their performance, and associated features. Second, we describe the major findings in movement imagination decoding, where we emphasize the importance of estimating the sources of the discriminative features. Third, we introduce movement target decoding, which could allow the determination of the target without knowing the exact movement-by-movement details. Aside from the kinematic level, we also address the goal level, which contains relevant information on the upcoming action. Focusing on hand-object interaction and action context dependency, we discuss the possible impact of some recent neurophysiological findings in the future of BCI control. Ideally, the goal and the kinematic decoding would allow an appropriate matching of the BCI to the end users' needs, overcoming the limitations of the classic motor imagery approach.

Evaluation of induced and evoked changes in EEG during selective attention to verbal stimuli.

BACKGROUND: Two challenges need to be addressed before bringing non-motor mental tasks for brain-computer interface (BCI) control to persons in a minimally conscious state (MCS), who can be behaviorally unresponsive even when proven to be consciously aware: first, keeping the cognitive demands as low as possible so that they could be fulfilled by persons with MCS. Second, increasing the control of experimental protocol (i.e. type and timing of the task performance). NEW METHOD: The goal of this study is twofold: first goal is to develop an experimental paradigm that can facilitate the performance of brain-teasers (e.g. mental subtraction and word generation) on the one hand, and can increase the control of experimental protocol on the other hand. The second goal of this study is to exploit the similar findings for mentally attending to someone else's verbal performance of brain-teaser tasks and self-performing the same tasks to setup an online BCI, and to compare it in healthy participants to the current "state-of-the-art" motor imagery (MI, sports). RESULTS: The response accuracies for the best performing healthy participants indicate that selective attention to verbal performance of mental subtraction (SUB) is a viable alternative to the MI. Time-frequency analysis of the SUB task in one participant with MCS did not reveal any significant (p<0.05) EEG changes, whereas imagined performance of one sport of participants' choice (SPORT) revealed task-related EEG changes over neurophysiological plausible cortical areas. COMPARISON WITH EXISTING METHODS: We found that mentally attending to someone else's verbal performance of brain-teaser tasks leads to similar results as in self-performing the same tasks. CONCLUSIONS: In this work we demonstrated that a single auditory selective attention task (i.e. mentally attending to someone else's verbal performance of mental subtraction) can modulate both induced and evoked changes in EEG, and be used for yes/no communication in an auditory scanning paradigm.

Classification of unconscious like/dislike decisions: First results towards a novel application for BCI technology.

More and more applications for BCI technology emerge that are not restricted to communication or control, like gaming, rehabilitation, Neuro-IS research, neuro-economics or security. In this context a so called passive BCI, a system that derives its outputs from arbitrary brain activity for enriching a human-machine interaction with implicit information on the actual user state will be used. Concretely EEG-based BCI technology enables the use of signals related to attention, intentions and mental state, without relying on indirect measures based on overt behavior or other physiological signals which is an important point e.g. in Neuromarketing research. The scope of this pilot EEG-study was to detect like/dislike decisions on car stimuli just by means of ERP analysis. Concretely to define user preferences concerning different car designs by implementing an offline BCI based on shrinkage LDA classification. Although classification failed in the majority of participants the elicited early (sub) conscious ERP components reflect user preferences for cars. In a broader sense this study should pave the way towards a "product design BCI" suitable for neuromarketing research.

Motor imagery-induced EEG patterns in individuals with spinal cord injury and their impact on brain-computer interface accuracy.

OBJECTIVE: Assimilating the diagnosis complete spinal cord injury (SCI) takes time and is not easy, as patients know that there is no 'cure' at the present time. Brain-computer interfaces (BCIs) can facilitate daily living. However, inter-subject variability demands measurements with potential user groups and an understanding of how they differ to healthy users BCIs are more commonly tested with. Thus, a three-class motor imagery (MI) screening (left hand, right hand, feet) was performed with a group of 10 able-bodied and 16 complete spinal-cord-injured people (paraplegics, tetraplegics) with the objective of determining what differences were present between the user groups and how they would impact upon the ability of these user groups to interact with a BCI. APPROACH: Electrophysiological differences between patient groups and healthy users are measured in terms of sensorimotor rhythm deflections from baseline during MI, electroencephalogram microstate scalp maps and strengths of inter-channel phase synchronization. Additionally, using a common spatial pattern algorithm and a linear discriminant analysis classifier, the classification accuracy was calculated and compared between groups. MAIN RESULTS: It is seen that both patient groups (tetraplegic and paraplegic) have some significant differences in event-related desynchronization strengths, exhibit significant increases in synchronization and reach significantly lower accuracies (mean (M) = 66.1%) than the group of healthy subjects (M = 85.1%). SIGNIFICANCE: The results demonstrate significant differences in electrophysiological correlates of motor control between healthy individuals and those individuals who stand to benefit most from BCI technology (individuals with SCI). They highlight the difficulty in directly translating results from healthy subjects to participants with SCI and the challenges that, therefore, arise in providing BCIs to such individuals.

Separating heart and brain: on the reduction of physiological noise from multichannel functional near-infrared spectroscopy (fNIRS) signals.

OBJECTIVE: Functional near-infrared spectroscopy (fNIRS) is an emerging technique for the in vivo assessment of functional activity of the cerebral cortex as well as in the field of brain-computer interface (BCI) research. A common challenge for the utilization of fNIRS in these areas is a stable and reliable investigation of the spatio-temporal hemodynamic patterns. However, the recorded patterns may be influenced and superimposed by signals generated from physiological processes, resulting in an inaccurate estimation of the cortical activity. Up to now only a few studies have investigated these influences, and still less has been attempted to remove/reduce these influences. The present study aims to gain insights into the reduction of physiological rhythms in hemodynamic signals (oxygenated hemoglobin (oxy-Hb), deoxygenated hemoglobin (deoxy-Hb)). APPROACH: We introduce the use of three different signal processing approaches (spatial filtering, a common average reference (CAR) method; independent component analysis (ICA); and transfer function (TF) models) to reduce the influence of respiratory and blood pressure (BP) rhythms on the hemodynamic responses. MAIN RESULTS: All approaches produce large reductions in BP and respiration influences on the oxy-Hb signals and, therefore, improve the contrast-to-noise ratio (CNR). In contrast, for deoxy-Hb signals CAR and ICA did not improve the CNR. However, for the TF approach, a CNR-improvement in deoxy-Hb can also be found. SIGNIFICANCE: The present study investigates the application of different signal processing approaches to reduce the influences of physiological rhythms on the hemodynamic responses. In addition to the identification of the best signal processing method, we also show the importance of noise reduction in fNIRS data.

Watching object related movements modulates mirror-like activity in parietal brain regions.

OBJECTIVE: We studied the activation of cortical motor and parietal areas during the observation of object related grasping movements. By manipulating the type of an object (realistic versus abstract) and the type of grasping (correct versus incorrect), we addressed the question how observing such object related movements influences cortical rhythmicity, especially the mu-rhythm, in the context of an "extended" human mirror neuron system (MNS). METHODS: Multichannel electroencephalogram (EEG) was recorded during the observation of different object-related grasping actions in twenty healthy subjects. Different movies were presented, showing sequences of correct or incorrect hand grasping actions related to an abstract or realistic (daily life) object. RESULTS: Event-related de/synchronization (ERD/ERS) analyses revealed a larger ERD in the upper alpha (10-12 Hz), beta (16-20 Hz) and gamma (36-40 Hz) frequency bands over parietal brain regions depending on the type of grasping. The type of object only influenced ERD patterns in the gamma band range (36-40 Hz) at parietal sites suggesting a strong relation of gamma band activity and cortical object representation. Abstract and realistic objects produced lower beta band synchronization at central sites only, whereas depending on the type of grasping an ERS in the upper alpha band (10-12 Hz) was observed. CONCLUSION: Depending on the type of the grasped object and the type of grasping stronger parietal cortical activation occurred during movement observation. SIGNIFICANCE: Discussing the results in terms of an "extended" human mirror neuron system (MNS), it could be concluded that beside sensorimotor areas a stronger involvement of parietal brain regions was found depending on the type of object and grasping movement observed.

Prototype of an auto-calibrating, context-aware, hybrid brain-computer interface.

We present the prototype of a context-aware framework that allows users to control smart home devices and to access internet services via a Hybrid BCI system of an auto-calibrating sensorimotor rhythm (SMR) based BCI and another assistive device (Integra Mouse mouth joystick). While there is extensive literature that describes the merit of Hybrid BCIs, auto-calibrating and co-adaptive ERD BCI training paradigms, specialized BCI user interfaces, context-awareness and smart home control, there is up to now, no system that includes all these concepts in one integrated easy-to-use framework that can truly benefit individuals with severe functional disabilities by increasing independence and social inclusion. Here we integrate all these technologies in a prototype framework that does not require expert knowledge or excess time for calibration. In a first pilot-study, 3 healthy volunteers successfully operated the system using input signals from an ERD BCI and an Integra Mouse and reached average positive predictive values (PPV) of 72 and 98% respectively. Based on what we learned here we are planning to improve the system for a test with a larger number of healthy volunteers so we can soon bring the system to benefit individuals with severe functional disability.

Toward smarter BCIs: extending BCIs through hybridization and intelligent control.

This paper summarizes two novel ways to extend brain-computer interface (BCI) systems. One way involves hybrid BCIs. A hybrid BCI is a system that combines a BCI with another device to help people send information. Different types of hybrid BCIs are discussed, along with challenges and issues. BCIs are also being extended through intelligent systems. Software that allows high-level control, incorporates context and the environment and/or uses virtual reality can substantially improve BCI systems. Throughout the paper, we critically address the real benefits of these improvements relative to existing technology and practices. We also present new challenges that are likely to emerge as these novel BCI directions become more widespread.

The auditory p300-based SSBCI: a door to minimally conscious patients?

In this study we report on the evaluation of a novel auditory single-switch BCI in nine patients diagnosed with MCS. The task included a simple and a complex oddball paradigm, the latter uses the tone stream segregation phenomenon. In all patients a significant difference between deviant and frequent tones could be observed in EEG. However, in some cases the deviant tones produce a significant negative peak and in some a very late positive peak. These preliminary findings are relevant in order to address future customization of this auditory ssBCI-based paradigm for unresponsive patients.

Toward a hybrid brain-computer interface based on imagined movement and visual attention.

Brain-computer interface (BCI) systems do not work for all users. This article introduces a novel combination of tasks that could inspire BCI systems that are more accurate than conventional BCIs, especially for users who cannot attain accuracy adequate for effective communication. Subjects performed tasks typically used in two BCI approaches, namely event-related desynchronization (ERD) and steady state visual evoked potential (SSVEP), both individually and in a 'hybrid' condition that combines both tasks. Electroencephalographic (EEG) data were recorded across three conditions. Subjects imagined moving the left or right hand (ERD), focused on one of the two oscillating visual stimuli (SSVEP), and then simultaneously performed both tasks. Accuracy and subjective measures were assessed. Offline analyses suggested that half of the subjects did not produce brain patterns that could be accurately discriminated in response to at least one of the two tasks. If these subjects produced comparable EEG patterns when trying to use a BCI, these subjects would not be able to communicate effectively because the BCI would make too many errors. Results also showed that switching to a different task used in BCIs could improve accuracy in some of these users. Switching to a hybrid approach eliminated this problem completely, and subjects generally did not consider the hybrid condition more difficult. Results validate this hybrid approach and suggest that subjects who cannot use a BCI should consider switching to a different BCI approach, especially a hybrid BCI. Subjects proficient with both approaches might combine them to increase information throughput by improving accuracy, reducing selection time, and/or increasing the number of possible commands.

Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges.

In recent years, new research has brought the field of electroencephalogram (EEG)-based brain-computer interfacing (BCI) out of its infancy and into a phase of relative maturity through many demonstrated prototypes such as brain-controlled wheelchairs, keyboards, and computer games. With this proof-of-concept phase in the past, the time is now ripe to focus on the development of practical BCI technologies that can be brought out of the lab and into real-world applications. In particular, we focus on the prospect of improving the lives of countless disabled individuals through a combination of BCI technology with existing assistive technologies (AT). In pursuit of more practical BCIs for use outside of the lab, in this paper, we identify four application areas where disabled individuals could greatly benefit from advancements in BCI technology, namely, "Communication and Control", "Motor Substitution", "Entertainment", and "Motor Recovery". We review the current state of the art and possible future developments, while discussing the main research issues in these four areas. In particular, we expect the most progress in the development of technologies such as hybrid BCI architectures, user-machine adaptation algorithms, the exploitation of users' mental states for BCI reliability and confidence measures, the incorporation of principles in human-computer interaction (HCI) to improve BCI usability, and the development of novel BCI technology including better EEG devices.