Practical real-time MEG-based neural interfacing with optically pumped magnetometers.

BACKGROUND: Brain-computer interfaces decode intentions directly from the human brain with the aim to restore lost functionality, control external devices or augment daily experiences. To combine optimal performance with wide applicability, high-quality brain signals should be captured non-invasively. Magnetoencephalography (MEG) is a potent candidate but currently requires costly and confining recording hardware. The recently developed optically pumped magnetometers (OPMs) promise to overcome this limitation, but are currently untested in the context of neural interfacing. RESULTS: In this work, we show that OPM-MEG allows robust single-trial analysis which we exploited in a real-time 'mind-spelling' application yielding an average accuracy of 97.7%. CONCLUSIONS: This shows that OPM-MEG can be used to exploit neuro-magnetic brain responses in a practical and flexible manner, and opens up new avenues for a wide range of new neural interface applications in the future.

Localization of deep brain activity with scalp and subdural EEG.

To what extent electrocorticography (ECoG) and electroencephalography (scalp EEG) differ in their capability to locate sources of deep brain activity is far from evident. Compared to EEG, the spatial resolution and signal-to-noise ratio of ECoG is superior but its spatial coverage is more restricted, as is arguably the volume of tissue activity effectively measured from. Moreover, scalp EEG studies are providing evidence of locating activity from deep sources such as the hippocampus using high-density setups during quiet wakefulness. To address this question, we recorded a multimodal dataset from 4 patients with refractory epilepsy during quiet wakefulness. This data comprises simultaneous scalp, subdural and depth EEG electrode recordings. The latter was located in the hippocampus or insula and provided us with our "ground truth" for source localization of deep activity. We applied independent component analysis (ICA) for the purpose of separating the independent sources in theta, alpha and beta frequency band activity. In all patients subdural- and scalp EEG components were observed which had a significant zero-lag correlation with one or more contacts of the depth electrodes. Subsequent dipole modeling of the correlating components revealed dipole locations that were significantly closer to the depth electrodes compared to the dipole location of non-correlating components. These findings support the idea that components found in both recording modalities originate from neural activity in close proximity to the depth electrodes. Sources localized with subdural electrodes were ~70% closer to the depth electrode than sources localized with EEG with an absolute improvement of around ~2cm. In our opinion, this is not a considerable improvement in source localization accuracy given that, for clinical purposes, ECoG electrodes were implanted in close proximity to the depth electrodes. Furthermore, the ECoG grid attenuates the scalp EEG, due to the electrically isolating silastic sheets in which the ECoG electrodes are embedded. Our results on dipole modeling show that the deep source localization accuracy of scalp EEG is comparable to that of ECoG. SIGNIFICANCE STATEMENT: Deep and subcortical regions play an important role in brain function. However, as joint recordings at multiple spatial scales to study brain function in humans are still scarce, it is still unresolved to what extent ECoG and EEG differ in their capability to locate sources of deep brain activity. To the best of our knowledge, this is the first study presenting a dataset of simultaneously recorded EEG, ECoG and depth electrodes in the hippocampus or insula, with a focus on non-epileptiform activity (quiet wakefulness). Furthermore, we are the first study to provide experimental findings on the comparison of source localization of deep cortical structures between invasive and non-invasive brain activity measured from the cortical surface.

High-gamma oscillations precede visual steady-state responses: A human electrocorticography study.

The robust steady-state cortical activation elicited by flickering visual stimulation has been exploited by a wide range of scientific studies. As the fundamental neural response inherits the spectral properties of the gazed flickering, the paradigm has been used to chart cortical characteristics and their relation to pathologies. However, despite its widespread adoption, the underlying neural mechanisms are not well understood. Here, we show that the fundamental response is preceded by high-gamma (55-125 Hz) oscillations which are also synchronised to the gazed frequency. Using a subdural recording of the primary and associative visual cortices of one human subject, we demonstrate that the latencies of the high-gamma and fundamental components are highly correlated on a single-trial basis albeit that the latter is consistently delayed by approximately 55 ms. These results corroborate previous reports that top-down feedback projections are involved in the generation of the fundamental response, but, in addition, we show that trial-to-trial variability in fundamental latency is paralleled by a highly similar variability in high-gamma latency. Pathology- or paradigm-induced alterations in steady-state responses could thus originate either from deviating visual gamma responses or from aberrations in the neural feedback mechanism. Experiments designed to tease apart the two processes are expected to provide deeper insights into the studied paradigm.

Semantic and perceptual priming activate partially overlapping brain networks as revealed by direct cortical recordings in humans.

Facilitation of object processing in the brain due to a related context (priming) can be influenced by both semantic connections and perceptual similarity. It is thus important to discern these two when evaluating the spatio-temporal dynamics of primed object processing. The repetition-priming paradigm frequently used to study perceptual priming is, however, unable to differentiate between the mentioned priming effects, possibly leading to confounded results. In the current study, we recorded brain signals from the scalp and cerebral convexity of nine patients with refractory epilepsy in response to related and unrelated image-pairs, all of which shared perceptual features while only related ones had a semantic connection. While previous studies employing a repetition-priming paradigm observed largely overlapping networks between semantic and perceptual priming effects, our results suggest that this overlap is only partial (both temporally and spatially). These findings stress the importance of controlling for perceptual features when studying semantic priming.

Representation of steady-state visual evoked potentials elicited by luminance flicker in human occipital cortex: An electrocorticography study.

Despite the widespread use of steady-state visual evoked potentials (SSVEPs) elicited by luminance flicker in clinical and research settings, their spatial and temporal representation in the occipital cortex largely remain elusive. We performed intracranial-EEG recordings in response to targets flickering at frequencies from 11 to 15 Hz using a subdural electrode grid covering the entire right occipital cortex of a human subject, and we were able to consistently locate the gazed stimulus frequency at the posterior side of the primary visual cortex (V1). Peripheral flickering, undetectable in scalp-EEG, elicited activations in the interhemispheric fissure at locations consistent with retinotopic maps. Both foveal and peripheral activations spatially coincided with activations in the high gamma band. We detected localized alpha synchronization at the lateral edge of V2 during stimulation and transient post-stimulation theta band activations at the posterior part of the occipital cortex. Scalp-EEG exhibited only a minor occipital post-stimulation theta activation, but a strong transient frontal activation.

Accurate Decoding of Short, Phase-Encoded SSVEPs.

Four novel EEG signal features for discriminating phase-coded steady-state visual evoked potentials (SSVEPs) are presented, and their performance in view of target selection in an SSVEP-based brain-computer interfacing (BCI) is assessed. The novel features are based on phase estimation and correlations between target responses. The targets are decoded from the feature scores using the least squares support vector machine (LS-SVM) classifier, and it is shown that some of the proposed features compete with state-of-the-art classifiers when using short (0.5 s) EEG recordings in a binary classification setting.

Imagined speech event detection from electrocorticography and its transfer between speech modes and subjects.

Speech brain-computer interfaces aim to support communication-impaired patients by translating neural signals into speech. While impressive progress was achieved in decoding performed, perceived and attempted speech, imagined speech remains elusive, mainly due to the absence of behavioral output. Nevertheless, imagined speech is advantageous since it does not depend on any articulator movements that might become impaired or even lost throughout the stages of a neurodegenerative disease. In this study, we analyzed electrocortigraphy data recorded from 16 participants in response to 3 speech modes: performed, perceived (listening), and imagined speech. We used a linear model to detect speech events and examined the contributions of each frequency band, from delta to high gamma, given the speech mode and electrode location. For imagined speech detection, we observed a strong contribution of gamma bands in the motor cortex, whereas lower frequencies were more prominent in the temporal lobe, in particular of the left hemisphere. Based on the similarities in frequency patterns, we were able to transfer models between speech modes and participants with similar electrode locations.

Phase-Spatial Beamforming Renders a Visual Brain Computer Interface Capable of Exploiting EEG Electrode Phase Shifts in Motion-Onset Target Responses.

OBJECTIVE: in this work, we aim to develop a more efficient visual motion-onset based Brain-computer interface (BCI). Brain-computer interfaces provide communication facilities that do not rely on the brain's usual pathways. Visual BCIs are based on changes in EEG activity in response to attended flashing or flickering targets. A less taxing way to encode such targets is with briefly moving stimuli, the onset of which elicits a lateralized EEG potential over the parieto-occipital scalp area called the motion-onset visual evoked potential (mVEP). METHODS: We recruited 21 healthy subjects for an experiment in which motion-onset stimulations translating leftwards (LT) or rightwards (RT) were encoding 9 displayed targets. We propose a novel algorithm that exploits the phase-shift between EEG electrodes to improve target decoding performance. We hereto extend the spatiotemporal beamformer (stBF) with a phase extracting procedure, leading to the phase-spatial beamformer (psBF). RESULTS: we show that psBF performs significantly better than the stBF (p < 0.001 for 1 and 2 stimulus repetitions and p < 0.01 for 3 to 5 stimulus repetitions), as well as the previously validated linear support-vector machines (p < 0.001 for 5 stimulus repetitions and p < 0.01 for 1,2 and 6 stimulus repetitions) and stepwise linear discriminant analysis decoders (p < 0.001 for all repetitions) when simultaneously addressing timing and translation direction. CONCLUSION: We provide evidence of decodability of joint direction and target in mVEP responses. SIGNIFICANCE: the described methods can aid in the development of a faster and more comfortable BCI based on mVEPs.

Analytic beamformer transformation for transfer learning in motion-onset visual evoked potential decoding.

Objective.While decoders of electroencephalography-based event-related potentials (ERPs) are routinely tailored to the individual user to maximize performance, developing them on populations for individual usage has proven much more challenging. We propose the analytic beamformer transformation (ABT) to extract phase and/or magnitude information from spatiotemporal ERPs in response to motion-onset stimulation.Approach.We have tested ABT on 52 motion-onset visual evoked potential (mVEP) datasets from 26 healthy subjects and compared the classification accuracy of support vector machine (SVM), spatiotemporal beamformer (stBF) and stepwise linear discriminant analysis (SWLDA) when trained on individual subjects and on a population thereof.Main results.When using phase- and combined phase/magnitude information extracted by ABT, we show significant improvements in accuracy of population-trained classifiers applied to individual users (p< 0.001). We also show that 450 epochs are needed for a correct functioning of ABT, which corresponds to 2 min of paradigm stimulation.Significance.We have shown that ABT can be used to create population-trained mVEP classifiers using a limited number of epochs. We expect this to pertain to other ERPs or synchronous stimulation paradigms, allowing for a more effective, population-based training of visual BCIs. Finally, as ABT renders recordings across subjects more structurally invariant, it could be used for transfer learning purposes in view of plug-and-play BCI applications.

Cognitive tasks propagate the neural entrainment in response to a visual 40 Hz stimulation in humans.

Introduction: Alzheimer's disease is one of the great challenges in the coming decades, and despite great efforts, a widely effective disease-modifying therapy in humans remains elusive. One particular promising non-pharmacological therapy that has received increased attention in recent years is based on the Gamma ENtrainment Using Sensory stimulation (GENUS), a high-frequency neural response elicited by a visual and/or auditory stimulus at 40 Hz. While this has shown to be effective in animal models, studies on human participants have reported varying success. The current work hypothesizes that the varying success in humans is due to differences in cognitive workload during the GENUS sessions. Methods: We recruited a cohort of 15 participants who underwent a scalp-EEG recording as well as one epilepsy patient who was implanted with 50 subdural surface electrodes over temporo-occipital and temporo-basal cortex and 14 depth contacts that targeted the hippocampus and insula. All participants completed several GENUS sessions, in each of which a different cognitive task was performed. Results: We found that the inclusion of a cognitive task during the GENUS session not only has a positive effect on the strength and extent of the gamma entrainment, but also promotes the propagation of gamma entrainment to additional neural areas including deep ones such as hippocampus which were not recruited when no cognitive task was required from the participants. The latter is of particular interest given that the hippocampal complex is considered to be one of the primary targets for AD therapies. Discussion: This work introduces a possible improvement strategy for GENUS therapy that might contribute to increasing the efficacy of the therapy or shortening the time needed for the positive outcome.

Electrophysiological Proxy of Cognitive Reserve Index.

Cognitive reserve (CR) postulates that individual differences in task performance can be attributed to differences in the brain's ability to recruit additional networks or adopt alternative cognitive strategies. Variables that are descriptive of lifetime experience such as socioeconomic status, educational attainment, and leisure activity are common proxies of CR. CR is mostly studied using neuroimaging techniques such as functional MRI (fMRI) in which case individuals with a higher CR were observed to activate a smaller brain network compared to individuals with a lower CR, when performing a task equally effectively (higher efficiency), and electroencephalography (EEG) where a particular EEG component (P300) that reflects the attention and working memory load, has been targeted. Despite the contribution of multiple factors such as age, education (formal and informal), working, leisure, and household activities in CR formation, most neuroimaging studies, and those using EEG in particular, focus on formal education level only. The aim of the current EEG study is to investigate how the P300 component, evoked in response to an oddball paradigm, is associated with other components of CR besides education, such as working and leisure activity in older adults. We have used hereto a recently introduced CR index questionnaire (CRIq) that quantifies both professional and leisure activities in terms of their cognitive demand and number of years practiced, as well as a data-driven approach for EEG analysis. We observed complex relationships between CRIq subcomponents and P300 characteristics. These results are especially important given that, unlike previous studies, our measurements (P300 and CRIq) do not require active use of the same executive function and, thus, render our results free of a collinearity bias.

Mental workload of young and older adults gauged with ERPs and spectral power during N-Back task performance.

Mental workload has been shown to correlate with alpha and theta band power but only few EEG studies focused on the relation between these bands and Event Related Potentials (ERPs), more specifically the P300 component. We report on an EEG study on mental workload where not only young but also older adults performed an N-Back task. Participants watched a sequence of visual pictures and indicated whether the current picture was the same as the one shown N pictures before. We considered N = 4 difficulty levels and analyzed the relation between these and P300 amplitude and theta and alpha band power, and also examined the effect of age, level of education, work activities, and task accuracy. Our results revealed a decrease in P300 amplitude and alpha band activity for higher difficulty levels for young adults in the parietal region. However, for older adults, fatigue played a more important role than we could anticipate as the alpha band power increased for the highest task difficulty level, and since performance accuracy also decreased, it could even be a sign of task disengagement. Beside alpha band, theta band activity showed a positive correlation with task difficulty level for both young and older adults. Additionally, we found higher P300 amplitudes for young adults compared to older adults, in line with their higher performance accuracies and lower reaction times. In conclusion, we showed that P300 amplitude and alpha and theta bands power provide complementary information for judging mental workload during N-Back performance for young and older subjects and for detecting mental fatigue and task disengagement.

N-back training and transfer effects revealed by behavioral responses and EEG.

INTRODUCTION: Cognitive function performance decreases in older individuals compared to young adults. To curb this decline, cognitive training is applied, but it is not clear whether it improves only the trained task or also other cognitive functions. To investigate this, we considered an N-back working memory (WM) training task and verified whether it improves both trained WM and untrained cognitive functions. METHODS: As EEG studies have noted task difficulty and age-related changes in time-locked EEG responses, called event-related potentials (ERPs), we focused on the relation between the P300 ERP component, task difficulty level, and behavior response accuracy and reaction time (RT) in young and older healthy adults. We used two groups of young and older healthy participants to assess the effect of N-back training: cognitive training group (CTG) and passive control group (PCG). Before and after training, cognitive tests were administered to both groups to evaluate transfer effects. RESULTS: Despite the observed age-related differences in the P300 ERP component and in terms of RT and accuracy, our findings demonstrate a stronger improvement in the trained task for older CTGs compared to younger CTGs, larger near- and far-transfer effect to WM and fluid intelligence for both younger and older CTGs, and a far-transfer effect to attention but only for older adults. Significant differences in response accuracy were shown between young and older subjects in spatial memory and attention tests. CONCLUSION: The application of a WM training is a promising tool for both healthy adults, and in particular for older subjects, as it showed physiological and behavioral differences in cognitive plasticity across life span and evidence of benefits in the trained task and near-/far-transfer effects to other cognitive functions.

Event Related Potential Study of Language Interaction in Bilingual Aphasia Patients.

Half of the global population can be considered bilingual. Nevertheless when faced with patients with aphasia, clinicians and therapists usually ignore the patient's second language (L2) albeit its interference in first language (L1) processing has been shown. The excellent temporal resolution by which each individual linguistic component can be gaged during word-processing, promoted the event-related potential (ERP) technique for studying language processing in healthy bilinguals and monolingual aphasia patients. However, this technique has not yet been applied in the context of bilingual aphasia. In the current study, we report on L2 interference in L1 processing using the ERP technique in bilingual aphasia. We tested four bilingual- and one trilingual patients with aphasia, as well as several young and older (age-matched with patients) healthy subjects as controls. We recorded ERPs when subjects were engaged in a semantic association judgment task on 122 related and 122 unrelated Dutch word-pairs (prime and target words). In 61 related and 61 unrelated word-pairs, an inter-lingual homograph was used as prime. In these word-pairs, when the target was unrelated to the prime in Dutch (L1), it was associated to the English (L2) meaning of the homograph. Results showed a significant effect of homograph use as a prime on early and/or late ERPs in response to word-pairs related in Dutch or English. Each patient presented a unique pattern of L2 interference in L1 processing as reflected by his/her ERP image. These interferences depended on the patient's pre- and post-morbid L2 proficiency. When the proficiency was high, the L2 interference in L1 processing was higher. Furthermore, the mechanism of interference in patients that were pre-morbidly highly proficient in L2 additionally depended on the frequency of pre-morbid L2 exposure. In summary, we showed that the mechanism behind L2 interference in L1 processing in bilingual patients with aphasia depends on a complex interaction between pre- and post-morbid L2 proficiency, pre- and post-morbid L2 exposure, impairment and the presented stimulus (inter-lingual homographs). Our ERP study complements the usually adopted behavioral approach by providing new insights into language interactions on the level of individual linguistic components in bilingual patients with aphasia.

Decoding Steady-State Visual Evoked Potentials From Electrocorticography.

We report on a unique electrocorticography (ECoG) experiment in which Steady-State Visual Evoked Potentials (SSVEPs) to frequency- and phase-tagged stimuli were recorded from a large subdural grid covering the entire right occipital cortex of a human subject. The paradigm is popular in EEG-based Brain Computer Interfacing where selectable targets are encoded by different frequency- and/or phase-tagged stimuli. We compare the performance of two state-of-the-art SSVEP decoders on both ECoG- and scalp-recorded EEG signals, and show that ECoG-based decoding is more accurate for very short stimulation lengths (i.e., less than 1 s). Furthermore, whereas the accuracy of scalp-EEG decoding benefits from a multi-electrode approach, to address interfering EEG responses and noise, ECoG decoding enjoys only a marginal improvement as even a single electrode, placed over the posterior part of the primary visual cortex, seems to suffice. This study shows, for the first time, that EEG-based SSVEP decoders can in principle be applied to ECoG, and can be expected to yield faster decoding speeds using less electrodes.

Spatiotemporal Beamforming: A Transparent and Unified Decoding Approach to Synchronous Visual Brain-Computer Interfacing.

Brain-Computer Interfaces (BCIs) decode brain activity with the aim to establish a direct communication channel with an external device. Albeit they have been hailed to (re-)establish communication in persons suffering from severe motor- and/or communication disabilities, only recently BCI applications have been challenging other assistive technologies. Owing to their considerably increased performance and the advent of affordable technological solutions, BCI technology is expected to trigger a paradigm shift not only in assistive technology but also in the way we will interface with technology. However, the flipside of the quest for accuracy and speed is most evident in EEG-based visual BCI where it has led to a gamut of increasingly complex classifiers, tailored to the needs of specific stimulation paradigms and use contexts. In this contribution, we argue that spatiotemporal beamforming can serve several synchronous visual BCI paradigms. We demonstrate this for three popular visual paradigms even without attempting to optimizing their electrode sets. For each selectable target, a spatiotemporal beamformer is applied to assess whether the corresponding signal-of-interest is present in the preprocessed multichannel EEG signals. The target with the highest beamformer output is then selected by the decoder (maximum selection). In addition to this simple selection rule, we also investigated whether interactions between beamformer outputs could be employed to increase accuracy by combining the outputs for all targets into a feature vector and applying three common classification algorithms. The results show that the accuracy of spatiotemporal beamforming with maximum selection is at par with that of the classification algorithms and interactions between beamformer outputs do not further improve that accuracy.

Code-modulated visual evoked potentials using fast stimulus presentation and spatiotemporal beamformer decoding.

When encoding visual targets using various lagged versions of a pseudorandom binary sequence of luminance changes, the EEG signal recorded over the viewer's occipital pole exhibits so-called code-modulated visual evoked potentials (cVEPs), the phase lags of which can be tied to these targets. The cVEP paradigm has enjoyed interest in the brain-computer interfacing (BCI) community for the reported high information transfer rates (ITR, in bits/min). In this study, we introduce a novel decoding algorithm based on spatiotemporal beamforming, and show that this algorithm is able to accurately identify the gazed target. Especially for a small number of repetitions of the coding sequence, our beamforming approach significantly outperforms an optimised support vector machine (SVM)-based classifier, which is considered state-of-the-art in cVEP-based BCI. In addition to the traditional 60 Hz stimulus presentation rate for the coding sequence, we also explore the 120 Hz rate, and show that the latter enables faster communication, with a maximal median ITR of 172.87 bits/min. Finally, we also report on a transition effect in the EEG signal following the onset of the stimulus sequence, and recommend to exclude the first 150 ms of the trials from decoding when relying on a single presentation of the stimulus sequence.

Faster P300 Classifier Training Using Spatiotemporal Beamforming.

The linearly-constrained minimum-variance (LCMV) beamformer is traditionally used as a spatial filter for source localization, but here we consider its spatiotemporal extension for P300 classification. We compare two variants and show that the spatiotemporal LCMV beamformer is at par with state-of-the-art P300 classifiers, but several orders of magnitude faster in training the classifier.

Frequency- and Phase Encoded SSVEP Using Spatiotemporal Beamforming.

In brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) the number of selectable targets is rather limited when each target has its own stimulation frequency. One way to remedy this is by combining frequency- with phase encoding. We introduce a new multivariate spatiotemporal filter, based on Linearly Constrained Minimum Variance (LCMV) beamforming, for discriminating between frequency-phase encoded targets more accurately, even when using short signal lengths than with (extended) Canonical Correlation Analysis (CCA), which is traditionally posited for this stimulation paradigm.