Estimating person-specific neural correlates of mental rotation: A machine learning approach.

Using neurophysiological measures to model how the brain performs complex cognitive tasks such as mental rotation is a promising way towards precise predictions of behavioural responses. The mental rotation task requires objects to be mentally rotated in space. It has been used to monitor progressive neurological disorders. Up until now, research on neural correlates of mental rotation have largely focused on group analyses yielding models with features common across individuals. Here, we propose an individually tailored machine learning approach to identify person-specific patterns of neural activity during mental rotation. We trained ridge regressions to predict the reaction time of correct responses in a mental rotation task using task-related, electroencephalographic (EEG) activity of the same person. When tested on independent data of the same person, the regression model predicted the reaction times significantly more accurately than when only the average reaction time was used for prediction (bootstrap mean difference of 0.02, 95% CI: 0.01-0.03, p < .001). When tested on another person's data, the predictions were significantly less accurate compared to within-person predictions. Further analyses revealed that considering person-specific reaction times and topographical activity patterns substantially improved a model's generalizability. Our results indicate that a more individualized approach towards neural correlates can improve their predictive performance of behavioural responses, particularly when combined with machine learning.

Perspectives of Implementation of Closed-Loop Deep Brain Stimulation: From Neurological to Psychiatric Disorders.

BACKGROUND: Deep brain stimulation (DBS) is a highly efficient, evidence-based therapy to alleviate symptoms and improve quality of life in movement disorders such as Parkinson's disease, essential tremor, and dystonia, which is also being applied in several psychiatric disorders, such as obsessive-compulsive disorder and depression, when they are otherwise resistant to therapy. SUMMARY: At present, DBS is clinically applied in the so-called open-loop approach, with fixed stimulation parameters, irrespective of the patients' clinical state(s). This approach ignores the brain states or feedback from the central nervous system or peripheral recordings, thus potentially limiting its efficacy and inducing side effects by stimulation of the targeted networks below or above the therapeutic level. KEY MESSAGES: The currently emerging closed-loop (CL) approaches are designed to adapt stimulation parameters to the electrophysiological surrogates of disease symptoms and states. CL-DBS paves the way for adaptive personalized DBS protocols. This review elaborates on the perspectives of the CL technology and discusses its opportunities as well as its potential pitfalls for both clinical and research use in neuropsychiatric disorders.

Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain-computer interfaces.

Objective.Covariance matrices of noisy multichannel electroencephalogram (EEG) time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain-computer interfaces (BCI) based on event-related potentials (ERP) and a linear discriminant analysis (LDA) classifier, the state of the art covariance estimation uses shrinkage regularization. As this is a general covariance regularization approach, we aim at improving LDA further by better exploiting the domain-specific characteristics of the EEG to regularize the covariance estimates.Approach.We propose to enforce a block-Toeplitz structure for the covariance matrix of the LDA, which implements an assumption of signal stationarity in short time windows for each channel.Main results.An offline re-analysis of data collected from 213 subjects under 13 different event-related potential BCI protocols showed a significantly increased binary classification performance of this 'ToeplitzLDA' compared to shrinkage regularized LDA (up to 6 AUC points,p < 0.001) and Riemannian classification approaches (up to 2 AUC points,p < 0.001). In an unsupervised visual speller application, this improvement would translate to a relative reduction of spelling errors by 81% on average for 25 subjects. Additionally, aside from lower memory and reduced time complexity for LDA training, ToeplitzLDA proves to be robust against drastic increases of the number of temporal features.Significance.The proposed covariance estimation allows BCI researchers to improve classification rates and reduce calibration times of BCI protocols using event-related potentials and thus support the usability of corresponding applications. Its lower computational and memory needs could make it a valuable algorithm especially for mobile BCIs.

Aphasia recovery by language training using a brain-computer interface: a proof-of-concept study.

Aphasia, the impairment to understand or produce language, is a frequent disorder after stroke with devastating effects. Conventional speech and language therapy include each formal intervention for improving language and communication abilities. In the chronic stage after stroke, it is effective compared with no treatment, but its effect size is small. We present a new language training approach for the rehabilitation of patients with aphasia based on a brain-computer interface system. The approach exploits its capacity to provide feedback time-locked to a brain state. Thus, it implements the idea that reinforcing an appropriate language processing strategy may induce beneficial brain plasticity. In our approach, patients perform a simple auditory target word detection task whilst their EEG was recorded. The constant decoding of these signals by machine learning models generates an individual and immediate brain-state-dependent feedback. It indicates to patients how well they accomplish the task during a training session, even if they are unable to speak. Results obtained from a proof-of-concept study with 10 stroke patients with mild to severe chronic aphasia (age range: 38-76 years) are remarkable. First, we found that the high-intensity training (30 h, 4 days per week) was feasible, despite a high-word presentation speed and unfavourable stroke-induced EEG signal characteristics. Second, the training induced a sustained recovery of aphasia, which generalized to multiple language aspects beyond the trained task. Specifically, all tested language assessments (Aachen Aphasia Test, Snodgrass & Vanderwart, Communicative Activity Log) showed significant medium to large improvements between pre- and post-training, with a standardized mean difference of 0.63 obtained for the Aachen Aphasia Test, and five patients categorized as non-aphasic at post-training assessment. Third, our data show that these language improvements were accompanied neither by significant changes in attention skills nor non-linguistic skills. Investigating possible modes of action of this brain-computer interface-based language training, neuroimaging data (EEG and resting-state functional MRI) indicates a training-induced faster word processing, a strengthened language network and a rebalancing between the language- and default mode networks.

Manipulating Single-Trial Motor Performance in Chronic Stroke Patients by Closed-Loop Brain State Interaction.

Motor impaired patients performing repetitive motor tasks often reveal large single-trial performance variations. Based on a data-driven framework, we extracted robust oscillatory brain states from pre-trial intervals, which are predictive for the upcoming motor performance on the level of single trials. Based on the brain state estimate, i.e. whether the brain state predicts a good or bad upcoming performance, we implemented a novel gating strategy for the start of trials by selecting specifically suitable or unsuitable trial starting time points. In a pilot study with four chronic stroke patients with hand motor impairments, we conducted a total of 41 sessions. After few initial calibration sessions, patients completed approximately 15 hours of effective hand motor training during eight online sessions using the gating strategy. Patients' reaction times were significantly reduced for suitable trials compared to unsuitable trials and shorter overall trial durations under suitable states were found in two patients. Overall, this successful proof-of-concept pilot study motivates to transfer this closed-loop training framework to a clinical study and to other application fields, such as cognitive rehabilitation, sport sciences or systems neuroscience.

Improving Covariance Matrices Derived from Tiny Training Datasets for the Classification of Event-Related Potentials with Linear Discriminant Analysis.

Electroencephalogram data used in the domain of brain-computer interfaces typically has subpar signal-to-noise ratio and data acquisition is expensive. An effective and commonly used classifier to discriminate event-related potentials is the linear discriminant analysis which, however, requires an estimate of the feature distribution. While this information is provided by the feature covariance matrix its large number of free parameters calls for regularization approaches like Ledoit-Wolf shrinkage. Assuming that the noise of event-related potential recordings is not time-locked, we propose to decouple the time component from the covariance matrix of event-related potential data in order to further improve the estimates of the covariance matrix for linear discriminant analysis. We compare three regularized variants thereof and a feature representation based on Riemannian geometry against our proposed novel linear discriminant analysis with time-decoupled covariance estimates. Extensive evaluations on 14 electroencephalogram datasets reveal, that the novel approach increases the classification performance by up to four percentage points for small training datasets, and gracefully converges to the performance of standard shrinkage-regularized LDA for large training datasets. Given these results, practitioners in this field should consider using our proposed time-decoupled covariance estimation when they apply linear discriminant analysis to classify event-related potentials, especially when few training data points are available.

A Pilot Study on Data-Driven Adaptive Deep Brain Stimulation in Chronically Implanted Essential Tremor Patients.

Deep brain stimulation (DBS) is an established therapy for Parkinson's disease (PD) and essential-tremor (ET). In adaptive DBS (aDBS) systems, online tuning of stimulation parameters as a function of neural signals may improve treatment efficacy and reduce side-effects. State-of-the-art aDBS systems use symptom surrogates derived from neural signals-so-called neural markers (NMs)-defined on the patient-group level, and control strategies assuming stationarity of symptoms and NMs. We aim at improving these aDBS systems with (1) a data-driven approach for identifying patient- and session-specific NMs and (2) a control strategy coping with short-term non-stationary dynamics. The two building blocks are implemented as follows: (1) The data-driven NMs are based on a machine learning model estimating tremor intensity from electrocorticographic signals. (2) The control strategy accounts for local variability of tremor statistics. Our study with three chronically implanted ET patients amounted to five online sessions. Tremor quantified from accelerometer data shows that symptom suppression is at least equivalent to that of a continuous DBS strategy in 3 out-of 4 online tests, while considerably reducing net stimulation (at least 24%). In the remaining online test, symptom suppression was not significantly different from either the continuous strategy or the no treatment condition. We introduce a novel aDBS system for ET. It is the first aDBS system based on (1) a machine learning model to identify session-specific NMs, and (2) a control strategy coping with short-term non-stationary dynamics. We show the suitability of our aDBS approach for ET, which opens the door to its further study in a larger patient population.

Identifying controllable cortical neural markers with machine learning for adaptive deep brain stimulation in Parkinson's disease.

The identification of oscillatory neural markers of Parkinson's disease (PD) can contribute not only to the understanding of functional mechanisms of the disorder, but may also serve in adaptive deep brain stimulation (DBS) systems. These systems seek online adaptation of stimulation parameters in closed-loop as a function of neural markers, aiming at improving treatment's efficacy and reducing side effects. Typically, the identification of PD neural markers is based on group-level studies. Due to the heterogeneity of symptoms across patients, however, such group-level neural markers, like the beta band power of the subthalamic nucleus, are not present in every patient or not informative about every patient's motor state. Instead, individual neural markers may be preferable for providing a personalized solution for the adaptation of stimulation parameters. Fortunately, data-driven bottom-up approaches based on machine learning may be utilized. These approaches have been developed and applied successfully in the field of brain-computer interfaces with the goal of providing individuals with means of communication and control. In our contribution, we present results obtained with a novel supervised data-driven identification of neural markers of hand motor performance based on a supervised machine learning model. Data of 16 experimental sessions obtained from seven PD patients undergoing DBS therapy show that the supervised patient-specific neural markers provide improved decoding accuracy of hand motor performance, compared to group-level neural markers reported in the literature. We observed that the individual markers are sensitive to DBS therapy and thus, may represent controllable variables in an adaptive DBS system.

Real-time detection of neural oscillation bursts allows behaviourally relevant neurofeedback.

Neural oscillations as important information carrier in the brain, are increasingly interpreted as transient bursts rather than as sustained oscillations. Short (<150 ms) bursts of beta-waves (15-30 Hz) have been documented in humans, monkeys and mice. These events were correlated with memory, movement and perception, and were even suggested as the primary ingredient of all beta-band activity. However, a method to measure these short-lived events in real-time and to investigate their impact on behaviour is missing. Here we present a real-time data analysis system, capable to detect short narrowband bursts, and demonstrate its usefulness to increase the beta-band burst-rate in rats. This neurofeedback training induced changes in overall oscillatory power, and bursts could be decoded from the movement of the rats, thus enabling future investigation of the role of oscillatory bursts.

A Robust Screen-Free Brain-Computer Interface for Robotic Object Selection.

Brain signals represent a communication modality that can allow users of assistive robots to specify high-level goals, such as the object to fetch and deliver. In this paper, we consider a screen-free Brain-Computer Interface (BCI), where the robot highlights candidate objects in the environment using a laser pointer, and the user goal is decoded from the evoked responses in the electroencephalogram (EEG). Having the robot present stimuli in the environment allows for more direct commands than traditional BCIs that require the use of graphical user interfaces. Yet bypassing a screen entails less control over stimulus appearances. In realistic environments, this leads to heterogeneous brain responses for dissimilar objects-posing a challenge for reliable EEG classification. We model object instances as subclasses to train specialized classifiers in the Riemannian tangent space, each of which is regularized by incorporating data from other objects. In multiple experiments with a total of 19 healthy participants, we show that our approach not only increases classification performance but is also robust to both heterogeneous and homogeneous objects. While especially useful in the case of a screen-free BCI, our approach can naturally be applied to other experimental paradigms with potential subclass structure.

An Easy-to-Use and Fast Assessment of Patient-Specific DBS-Induced Changes in Hand Motor Control in Parkinson's Disease.

For Parkinson's disease (PD), efficient and fast monitoring of fine motor function is fundamental for capturing transient phenomena induced by deep brain stimulation (DBS), thus, enabling a fast and accurate tuning of stimulation parameters. Tuning of DBS parameters is important for obtaining a patient-specific optimal clinical effect and to regularly compensate for disease progress. We propose a fine motor function assessment framework for capturing transient DBS-induced changes. The main goals are to obtain a fast, repeatable, objective, robust, and DBS-sensitive motor-score, in addition to a high-dimensional characterization of motor components by means of an interpretable data-driven model. To achieve this, we combine a hand motor-task, termed the copy-draw test, with a linear model for analyzing features extracted from the proposed task. The approach was tested with four patients totaling eight sessions analyzed. Our approach delivers a motor-score that is sensitive to DBS-induced changes in motor function. It can be applied repeatedly within seconds. The interpretability of the underlying machine learning model provides a direct overview of the feature relevance. This analysis allows to detect and characterize single movement components that are sensitive to DBS. The proposed assessment framework is an useful tool to push forward the data-driven identification of PD-relevant neural markers. Consequent to this end, the source code of the paradigm is made publicly available.

Post-hoc Labeling of Arbitrary M/EEG Recordings for Data-Efficient Evaluation of Neural Decoding Methods.

Many cognitive, sensory and motor processes have correlates in oscillatory neural source activity, which is embedded as a subspace in the recorded brain signals. Decoding such processes from noisy magnetoencephalogram/electroencephalogram (M/EEG) signals usually requires data-driven analysis methods. The objective evaluation of such decoding algorithms on experimental raw signals, however, is a challenge: the amount of available M/EEG data typically is limited, labels can be unreliable, and raw signals often are contaminated with artifacts. To overcome some of these problems, simulation frameworks have been introduced which support the development of data-driven decoding algorithms and their benchmarking. For generating artificial brain signals, however, most of the existing frameworks make strong and partially unrealistic assumptions about brain activity. This limits the generalization of results observed in the simulation to real-world scenarios. In the present contribution, we show how to overcome several shortcomings of existing simulation frameworks. We propose a versatile alternative, which allows for an objective evaluation and benchmarking of novel decoding algorithms using real neural signals. It allows to generate comparatively large datasets with labels being deterministically recoverable from the arbitrary M/EEG recordings. A novel idea to generate these labels is central to this framework: we determine a subspace of the true M/EEG recordings and utilize it to derive novel labels. These labels contain realistic information about the oscillatory activity of some underlying neural sources. For two categories of subspace-defining methods, we showcase how such labels can be obtained-either by an exclusively data-driven approach (independent component analysis-ICA), or by a method exploiting additional anatomical constraints (minimum norm estimates-MNE). We term our framework post-hoc labeling of M/EEG recordings. To support the adoption of the framework by practitioners, we have exemplified its use by benchmarking three standard decoding methods-i.e., common spatial patterns (CSP), source power-comodulation (SPoC), and convolutional neural networks (ConvNets)-wrt. Varied dataset sizes, label noise, and label variability. Source code and data are made available to the reader for facilitating the application of our post-hoc labeling framework.

Mining Within-Trial Oscillatory Brain Dynamics to Address the Variability of Optimized Spatial Filters.

Data-driven spatial filtering algorithms optimize scores, such as the contrast between two conditions to extract oscillatory brain signal components. Most machine learning approaches for the filter estimation, however, disregard within-trial temporal dynamics and are extremely sensitive to changes in training data and involved hyperparameters. This leads to highly variable solutions and impedes the selection of a suitable candidate for, e.g., neurotechnological applications. Fostering component introspection, we propose to embrace this variability by condensing the functional signatures of a large set of oscillatory components into homogeneous clusters, each representing specific within-trial envelope dynamics. The proposed method is exemplified by and evaluated on a complex hand force task with a rich within-trial structure. Based on electroencephalography data of 18 healthy subjects, we found that the components' distinct temporal envelope dynamics are highly subject-specific. On average, we obtained seven clusters per subject, which were strictly confined regarding their underlying frequency bands. As the analysis method is not limited to a specific spatial filtering algorithm, it could be utilized for a wide range of neurotechnological applications, e.g., to select and monitor functionally relevant features for brain-computer interface protocols in stroke rehabilitation.

Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems : Guidelines Derived from Simulation and Real-World Data.

We report on novel supervised algorithms for single-trial brain state decoding. Their reliability and robustness are essential to efficiently perform neurotechnological applications in closed-loop. When brain activity is assessed by multichannel recordings, spatial filters computed by the source power comodulation (SPoC) algorithm allow identifying oscillatory subspaces. They regress to a known continuous trial-wise variable reflecting, e.g. stimulus characteristics, cognitive processing or behavior. In small dataset scenarios, this supervised method tends to overfit to its training data as the involved recordings via electroencephalogram (EEG), magnetoencephalogram or local field potentials generally provide a low signal-to-noise ratio. To improve upon this, we propose and characterize three types of regularization techniques for SPoC: approaches using Tikhonov regularization (which requires model selection via cross-validation), combinations of Tikhonov regularization and covariance matrix normalization as well as strategies exploiting analytical covariance matrix shrinkage. All proposed techniques were evaluated both in a novel simulation framework and on real-world data. Based on the simulation findings, we saw our expectations fulfilled, that SPoC regularization generally reveals the largest benefit for small training sets and under severe label noise conditions. Relevant for practitioners, we derived operating ranges of regularization hyperparameters for cross-validation based approaches and offer open source code. Evaluating all methods additionally on real-world data, we observed an improved regression performance mainly for datasets from subjects with initially poor performance. With this proof-of-concept paper, we provided a generalizable regularization framework for SPoC which may serve as a starting point for implementing advanced techniques in the future.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation.

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

Extremely Reduced Data Sets Indicate Optimal Stimulation Parameters for Classification in Brain-Computer Interfaces.

The time between the onset of subsequent auditory or visual stimuli - also known as stimulus onset asynchrony (SOA) - determines many of the event-related potential characteristics of the resulting evoked brain signals. Specifically, the SOA value influences the performance of an individual subject in brain-computer interface (BCI) applications like spellers. In the past, subject-specific optimization of the SOA was rarely considered in BCI studies. Our research strives to reduce the time requirements of individual BCI stimulus parameter optimization. This work contributes to this goal in two ways. First, we show that even the classification performance on extremely reduced training data subsets reveals the influence of SOA. Second, we show, that these noisy estimates are sufficient to make decisions for individual choices of the SOA that transfer to good classification performance on large training data sets. Thus our work contributes to individually tailored SOA selection procedures for BCI users.

A simulated environment for early development stages of reinforcement learning algorithms for closed-loop deep brain stimulation.

In recent years, closed-loop adaptive deep brain stimulation (aDBS) for Parkinson's disease (PD) has gained focus in the research community, due to promising proof-of-concept studies showing its suitability for improving DBS therapy and ameliorating related side effects.The main challenges faced in the aDBS control problem is the presence of non-stationary/non-linear dynamics and the heterogeneity of PD's phenotype, making the exploration of data-driven dynamics-aware control algorithms a promising research direction. However, due to the severe safety constraints related to working with patients, aDBS is a sensitive research field that requires surrogate development platforms with growing complexity, as novel control algorithms are validated.With our current contribution, we propose the characterization and categorization of non-stationary dynamics found in the aDBS problem. We show how knowledge about these dynamics can be embedded in a surrogate simulation environment, which has been designed to support early development stages of aDBS control strategies, specifically those based on reinforcement learning (RL) algorithms. Finally, we present a comparison of representative RL methods designed to cope with the type of non-stationary dynamics found in aDBS.To allow reproducibility and encourage adoption of our approach, the source code of the developed methods and simulation environment are made available online.

Two Player Online Brain-Controlled Chess.

Brain-computer interfaces (BCIs) allow for translating brain signals into control commands, e.g. to control games. One of the biggest quests of the BCI community is to realize new exciting applications. In this paper, we present a two player online chess application where both players control their pieces solely with their brain activity. Control is realized in a two-step process where players first select the desired chess piece and then the field they want to move it to. A selection is accomplished with visual event-related potentials that are elicited by highlighting each possible piece or field one by one. Six healthy volunteers participated in our study by playing a game against each other in pairs over a free chess server. They successfully completed at least one match per pair. Our results show that even BCI-naive players obtain almost perfect control over the application. On average, 96% of the moves were correct. Most of the errors came from technical problems that can be corrected in future versions of the application. Given the high popularity of chess, this intuitive BCI game is an appealing new application for patients and for healthy users that want to explore BCIs.

Influence of User Tasks on EEG-based Classification Performance in a Hazard Detection Paradigm.

Attention-based brain-computer interface (BCI) paradigms offer a way to exert control, but also to provide insight into a user's perception and judgment of the environment. For a sufficient classification performance, user engagement and motivation are critical aspects. Consequently, many paradigms require the user to perform an auxiliary task, such as mentally counting subsets of stimuli or pressing a button when encountering them. In this work, we compare two user tasks, mental counting and button-presses, in a hazard detection paradigm in driving videos. We find that binary classification performance of events based on the electroencephalogram as well as user preference are higher for button presses. Amplitudes of evoked responses are higher for the counting task-an observation which holds even after projecting out motor-related potentials during the data preprocessing. Our results indicate that the choice of button-presses can be a preferable choice in such BCIs based on prediction performance as well as user preference.