Real-Time Navigation in Google Street View® Using a Motor Imagery-Based BCI.

Navigation in virtual worlds is ubiquitous in games and other virtual reality (VR) applications and mainly relies on external controllers. As brain-computer interfaces (BCI)s rely on mental control, bypassing traditional neural pathways, they provide to paralyzed users an alternative way to navigate. However, the majority of BCI-based navigation studies adopt cue-based visual paradigms, and the evoked brain responses are encoded into navigation commands. Although robust and accurate, these paradigms are less intuitive and comfortable for navigation compared to imagining limb movements (motor imagery, MI). However, decoding motor imagery from EEG activity is notoriously challenging. Typically, wet electrodes are used to improve EEG signal quality, including a large number of them to discriminate between movements of different limbs, and a cuedbased paradigm is used instead of a self-paced one to maximize decoding performance. Motor BCI applications primarily focus on typing applications or on navigating a wheelchair-the latter raises safety concerns-thereby calling for sensors scanning the environment for obstacles and potentially hazardous scenarios. With the help of new technologies such as virtual reality (VR), vivid graphics can be rendered, providing the user with a safe and immersive experience; and they could be used for navigation purposes, a topic that has yet to be fully explored in the BCI community. In this study, we propose a novel MI-BCI application based on an 8-dry-electrode EEG setup, with which users can explore and navigate in Google Street View®. We pay attention to system design to address the lower performance of the MI decoder due to the dry electrodes' lower signal quality and the small number of electrodes. Specifically, we restricted the number of navigation commands by using a novel middle-level control scheme and avoided decoder mistakes by introducing eye blinks as a control signal in different navigation stages. Both offline and online experiments were conducted with 20 healthy subjects. The results showed acceptable performance, even given the limitations of the EEG set-up, which we attribute to the design of the BCI application. The study suggests the use of MI-BCI in future games and VR applications for consumers and patients temporarily or permanently devoid of muscle control.

Effects of risk factors on longitudinal changes in brain structure and function in the progression of AD.

INTRODUCTION: Past research on Alzheimer's disease (AD) has focused on biomarkers, cognition, and neuroimaging as primary predictors of its progression, albeit additional ones have recently gained attention. When turning to the prediction of the progression from one stage to another, one could benefit from the joint assessment of imaging-based biomarkers and risk/protective factors. METHODS: We included 86 studies that fulfilled our inclusion criteria. RESULTS: Our review summarizes and discusses the results of 30 years of longitudinal research on brain changes assessed with neuroimaging and the risk/protective factors and their effect on AD progression. We group results into four sections: genetic, demographic, cognitive and cardiovascular, and lifestyle factors. DISCUSSION: Given the complex nature of AD, including risk factors could prove invaluable for a better understanding of AD progression. Some of these risk factors are modifiable and could be targeted by potential future treatments.

Practical real-time MEG-based neural interfacing with optically pumped magnetometers.

BACKGROUND: Brain-computer interfaces decode intentions directly from the human brain with the aim to restore lost functionality, control external devices or augment daily experiences. To combine optimal performance with wide applicability, high-quality brain signals should be captured non-invasively. Magnetoencephalography (MEG) is a potent candidate but currently requires costly and confining recording hardware. The recently developed optically pumped magnetometers (OPMs) promise to overcome this limitation, but are currently untested in the context of neural interfacing. RESULTS: In this work, we show that OPM-MEG allows robust single-trial analysis which we exploited in a real-time 'mind-spelling' application yielding an average accuracy of 97.7%. CONCLUSIONS: This shows that OPM-MEG can be used to exploit neuro-magnetic brain responses in a practical and flexible manner, and opens up new avenues for a wide range of new neural interface applications in the future.

Localization of deep brain activity with scalp and subdural EEG.

To what extent electrocorticography (ECoG) and electroencephalography (scalp EEG) differ in their capability to locate sources of deep brain activity is far from evident. Compared to EEG, the spatial resolution and signal-to-noise ratio of ECoG is superior but its spatial coverage is more restricted, as is arguably the volume of tissue activity effectively measured from. Moreover, scalp EEG studies are providing evidence of locating activity from deep sources such as the hippocampus using high-density setups during quiet wakefulness. To address this question, we recorded a multimodal dataset from 4 patients with refractory epilepsy during quiet wakefulness. This data comprises simultaneous scalp, subdural and depth EEG electrode recordings. The latter was located in the hippocampus or insula and provided us with our "ground truth" for source localization of deep activity. We applied independent component analysis (ICA) for the purpose of separating the independent sources in theta, alpha and beta frequency band activity. In all patients subdural- and scalp EEG components were observed which had a significant zero-lag correlation with one or more contacts of the depth electrodes. Subsequent dipole modeling of the correlating components revealed dipole locations that were significantly closer to the depth electrodes compared to the dipole location of non-correlating components. These findings support the idea that components found in both recording modalities originate from neural activity in close proximity to the depth electrodes. Sources localized with subdural electrodes were ~70% closer to the depth electrode than sources localized with EEG with an absolute improvement of around ~2cm. In our opinion, this is not a considerable improvement in source localization accuracy given that, for clinical purposes, ECoG electrodes were implanted in close proximity to the depth electrodes. Furthermore, the ECoG grid attenuates the scalp EEG, due to the electrically isolating silastic sheets in which the ECoG electrodes are embedded. Our results on dipole modeling show that the deep source localization accuracy of scalp EEG is comparable to that of ECoG. SIGNIFICANCE STATEMENT: Deep and subcortical regions play an important role in brain function. However, as joint recordings at multiple spatial scales to study brain function in humans are still scarce, it is still unresolved to what extent ECoG and EEG differ in their capability to locate sources of deep brain activity. To the best of our knowledge, this is the first study presenting a dataset of simultaneously recorded EEG, ECoG and depth electrodes in the hippocampus or insula, with a focus on non-epileptiform activity (quiet wakefulness). Furthermore, we are the first study to provide experimental findings on the comparison of source localization of deep cortical structures between invasive and non-invasive brain activity measured from the cortical surface.

High-gamma oscillations precede visual steady-state responses: A human electrocorticography study.

The robust steady-state cortical activation elicited by flickering visual stimulation has been exploited by a wide range of scientific studies. As the fundamental neural response inherits the spectral properties of the gazed flickering, the paradigm has been used to chart cortical characteristics and their relation to pathologies. However, despite its widespread adoption, the underlying neural mechanisms are not well understood. Here, we show that the fundamental response is preceded by high-gamma (55-125 Hz) oscillations which are also synchronised to the gazed frequency. Using a subdural recording of the primary and associative visual cortices of one human subject, we demonstrate that the latencies of the high-gamma and fundamental components are highly correlated on a single-trial basis albeit that the latter is consistently delayed by approximately 55 ms. These results corroborate previous reports that top-down feedback projections are involved in the generation of the fundamental response, but, in addition, we show that trial-to-trial variability in fundamental latency is paralleled by a highly similar variability in high-gamma latency. Pathology- or paradigm-induced alterations in steady-state responses could thus originate either from deviating visual gamma responses or from aberrations in the neural feedback mechanism. Experiments designed to tease apart the two processes are expected to provide deeper insights into the studied paradigm.

Semantic and perceptual priming activate partially overlapping brain networks as revealed by direct cortical recordings in humans.

Facilitation of object processing in the brain due to a related context (priming) can be influenced by both semantic connections and perceptual similarity. It is thus important to discern these two when evaluating the spatio-temporal dynamics of primed object processing. The repetition-priming paradigm frequently used to study perceptual priming is, however, unable to differentiate between the mentioned priming effects, possibly leading to confounded results. In the current study, we recorded brain signals from the scalp and cerebral convexity of nine patients with refractory epilepsy in response to related and unrelated image-pairs, all of which shared perceptual features while only related ones had a semantic connection. While previous studies employing a repetition-priming paradigm observed largely overlapping networks between semantic and perceptual priming effects, our results suggest that this overlap is only partial (both temporally and spatially). These findings stress the importance of controlling for perceptual features when studying semantic priming.

Exploring the Organization of Semantic Memory through Unsupervised Analysis of Event-related Potentials.

Modern multivariate methods have enabled the application of unsupervised techniques to analyze neurophysiological data without strict adherence to predefined experimental conditions. We demonstrate a multivariate method that leverages priming effects on the evoked potential to perform hierarchical clustering on a set of word stimuli. The current study focuses on the semantic relationships that play a key role in the organization of our mental lexicon of words and concepts. The N400 component of the event-related potential is considered a reliable neurophysiological response that is indicative of whether accessing one concept facilitates subsequent access to another (i.e., one "primes" the other). To further our understanding of the organization of the human mental lexicon, we propose to utilize the N400 component to drive a clustering algorithm that can uncover, given a set of words, which particular subsets of words show mutual priming. Such a scheme requires a reliable measurement of the amplitude of the N400 component without averaging across many trials, which was here achieved using a recently developed multivariate analysis method based on beamforming. We validated our method by demonstrating that it can reliably detect, without any prior information about the nature of the stimuli, a well-known feature of the organization of our semantic memory: the distinction between animate and inanimate concepts. These results motivate further application of our method to data-driven exploration of disputed or unknown relationships between stimuli.

Representation of steady-state visual evoked potentials elicited by luminance flicker in human occipital cortex: An electrocorticography study.

Despite the widespread use of steady-state visual evoked potentials (SSVEPs) elicited by luminance flicker in clinical and research settings, their spatial and temporal representation in the occipital cortex largely remain elusive. We performed intracranial-EEG recordings in response to targets flickering at frequencies from 11 to 15 Hz using a subdural electrode grid covering the entire right occipital cortex of a human subject, and we were able to consistently locate the gazed stimulus frequency at the posterior side of the primary visual cortex (V1). Peripheral flickering, undetectable in scalp-EEG, elicited activations in the interhemispheric fissure at locations consistent with retinotopic maps. Both foveal and peripheral activations spatially coincided with activations in the high gamma band. We detected localized alpha synchronization at the lateral edge of V2 during stimulation and transient post-stimulation theta band activations at the posterior part of the occipital cortex. Scalp-EEG exhibited only a minor occipital post-stimulation theta activation, but a strong transient frontal activation.

Accurate Decoding of Short, Phase-Encoded SSVEPs.

Four novel EEG signal features for discriminating phase-coded steady-state visual evoked potentials (SSVEPs) are presented, and their performance in view of target selection in an SSVEP-based brain-computer interfacing (BCI) is assessed. The novel features are based on phase estimation and correlations between target responses. The targets are decoded from the feature scores using the least squares support vector machine (LS-SVM) classifier, and it is shown that some of the proposed features compete with state-of-the-art classifiers when using short (0.5 s) EEG recordings in a binary classification setting.

Stable Voice Clusters Identified When Using the Maximum versus Minimum Intensity Curve in the Phonetogram.

OBJECTIVE: To examine whether the minimum as well as the maximum voice intensity (i.e. sound pressure level, SPL) curves of a voice range profile (VRP) are required when discovering different voice groups based on a clustering analysis. In this approach, no a priori labeling of voice types is used. PATIENTS AND METHODS: VRPs of 194 (84 male and 110 female) professional singers were registered and processed. Cluster analysis was performed with the use of features related to (1) both the maximum and minimum SPL curves and (2) the maximum SPL curve only. RESULTS: Features related to the maximum as well as the minimum SPL curves showed three clusters in both male and female voices. These clusters, or voice groups, are based on voice types with similar VRP features. However, when using features related only to the maximum SPL curve, the clusters became less obvious. CONCLUSION: Features related to the maximum and minimum SPL curves of a VRP are both needed in order to identify the three voice clusters.

Language model applications to spelling with Brain-Computer Interfaces.

Within the Ambient Assisted Living (AAL) community, Brain-Computer Interfaces (BCIs) have raised great hopes as they provide alternative communication means for persons with disabilities bypassing the need for speech and other motor activities. Although significant advancements have been realized in the last decade, applications of language models (e.g., word prediction, completion) have only recently started to appear in BCI systems. The main goal of this article is to review the language model applications that supplement non-invasive BCI-based communication systems by discussing their potential and limitations, and to discern future trends. First, a brief overview of the most prominent BCI spelling systems is given, followed by an in-depth discussion of the language models applied to them. These language models are classified according to their functionality in the context of BCI-based spelling: the static/dynamic nature of the user interface, the use of error correction and predictive spelling, and the potential to improve their classification performance by using language models. To conclude, the review offers an overview of the advantages and challenges when implementing language models in BCI-based communication systems when implemented in conjunction with other AAL technologies.

Imagined speech event detection from electrocorticography and its transfer between speech modes and subjects.

Speech brain-computer interfaces aim to support communication-impaired patients by translating neural signals into speech. While impressive progress was achieved in decoding performed, perceived and attempted speech, imagined speech remains elusive, mainly due to the absence of behavioral output. Nevertheless, imagined speech is advantageous since it does not depend on any articulator movements that might become impaired or even lost throughout the stages of a neurodegenerative disease. In this study, we analyzed electrocortigraphy data recorded from 16 participants in response to 3 speech modes: performed, perceived (listening), and imagined speech. We used a linear model to detect speech events and examined the contributions of each frequency band, from delta to high gamma, given the speech mode and electrode location. For imagined speech detection, we observed a strong contribution of gamma bands in the motor cortex, whereas lower frequencies were more prominent in the temporal lobe, in particular of the left hemisphere. Based on the similarities in frequency patterns, we were able to transfer models between speech modes and participants with similar electrode locations.

Spiking image processing unit based on neural analog of Boolean logic operations.

McCulloch and Pitts hypothesized in 1943 that the brain is entirely composed of logic gates, akin to current computers' IP cores, which led to several neural analogs of Boolean logic. The current study proposes a spiking image processing unit (SIPU) based on spiking frequency gates and coordinate logic operations, as a dynamical model of synapses and spiking neurons. SIPU can imitate DSP functions like edge recognition, picture magnification, noise reduction, etc. but can be extended to cater for more advanced computing tasks. The proposed spiking Boolean logic platform can be used to develop advanced applications without relying on learning or specialized datasets. It could aid in gaining a deeper understanding of complex brain functions and spur new forms of neural analogs.

Distinct mechanisms for coding of visual actions in macaque temporal cortex.

Temporal cortical neurons are known to respond to visual dynamic-action displays. Many human psychophysical and functional imaging studies examining biological motion perception have used treadmill walking, in contrast to previous macaque single-cell studies. We assessed the coding of locomotion in rhesus monkey (Macaca mulatta) temporal cortex using movies of stationary walkers, varying both form and motion (i.e., different facing directions) or varying only the frame sequence (i.e., forward vs backward walking). The majority of superior temporal sulcus and inferior temporal neurons were selective for facing direction, whereas a minority distinguished forward from backward walking. Support vector machines using the temporal cortical population responses as input classified facing direction well, but forward and backward walking less so. Classification performance for the latter improved markedly when the within-action response modulation was considered, reflecting differences in momentary body poses within the locomotion sequences. Responses to static pose presentations predicted the responses during the course of the action. Analyses of the responses to walking sequences wherein the start frame was varied across trials showed that some neurons also carried a snapshot sequence signal. Such sequence information was present in neurons that responded to static snapshot presentations and in neurons that required motion. Our data suggest that actions are analyzed by temporal cortical neurons using distinct mechanisms. Most neurons predominantly signal momentary pose. In addition, temporal cortical neurons, including those responding to static pose, are sensitive to pose sequence, which can contribute to the signaling of learned action sequences.

Phase-Spatial Beamforming Renders a Visual Brain Computer Interface Capable of Exploiting EEG Electrode Phase Shifts in Motion-Onset Target Responses.

OBJECTIVE: in this work, we aim to develop a more efficient visual motion-onset based Brain-computer interface (BCI). Brain-computer interfaces provide communication facilities that do not rely on the brain's usual pathways. Visual BCIs are based on changes in EEG activity in response to attended flashing or flickering targets. A less taxing way to encode such targets is with briefly moving stimuli, the onset of which elicits a lateralized EEG potential over the parieto-occipital scalp area called the motion-onset visual evoked potential (mVEP). METHODS: We recruited 21 healthy subjects for an experiment in which motion-onset stimulations translating leftwards (LT) or rightwards (RT) were encoding 9 displayed targets. We propose a novel algorithm that exploits the phase-shift between EEG electrodes to improve target decoding performance. We hereto extend the spatiotemporal beamformer (stBF) with a phase extracting procedure, leading to the phase-spatial beamformer (psBF). RESULTS: we show that psBF performs significantly better than the stBF (p < 0.001 for 1 and 2 stimulus repetitions and p < 0.01 for 3 to 5 stimulus repetitions), as well as the previously validated linear support-vector machines (p < 0.001 for 5 stimulus repetitions and p < 0.01 for 1,2 and 6 stimulus repetitions) and stepwise linear discriminant analysis decoders (p < 0.001 for all repetitions) when simultaneously addressing timing and translation direction. CONCLUSION: We provide evidence of decodability of joint direction and target in mVEP responses. SIGNIFICANCE: the described methods can aid in the development of a faster and more comfortable BCI based on mVEPs.

Analytic beamformer transformation for transfer learning in motion-onset visual evoked potential decoding.

Objective.While decoders of electroencephalography-based event-related potentials (ERPs) are routinely tailored to the individual user to maximize performance, developing them on populations for individual usage has proven much more challenging. We propose the analytic beamformer transformation (ABT) to extract phase and/or magnitude information from spatiotemporal ERPs in response to motion-onset stimulation.Approach.We have tested ABT on 52 motion-onset visual evoked potential (mVEP) datasets from 26 healthy subjects and compared the classification accuracy of support vector machine (SVM), spatiotemporal beamformer (stBF) and stepwise linear discriminant analysis (SWLDA) when trained on individual subjects and on a population thereof.Main results.When using phase- and combined phase/magnitude information extracted by ABT, we show significant improvements in accuracy of population-trained classifiers applied to individual users (p< 0.001). We also show that 450 epochs are needed for a correct functioning of ABT, which corresponds to 2 min of paradigm stimulation.Significance.We have shown that ABT can be used to create population-trained mVEP classifiers using a limited number of epochs. We expect this to pertain to other ERPs or synchronous stimulation paradigms, allowing for a more effective, population-based training of visual BCIs. Finally, as ABT renders recordings across subjects more structurally invariant, it could be used for transfer learning purposes in view of plug-and-play BCI applications.

Cognitive tasks propagate the neural entrainment in response to a visual 40 Hz stimulation in humans.

Introduction: Alzheimer's disease is one of the great challenges in the coming decades, and despite great efforts, a widely effective disease-modifying therapy in humans remains elusive. One particular promising non-pharmacological therapy that has received increased attention in recent years is based on the Gamma ENtrainment Using Sensory stimulation (GENUS), a high-frequency neural response elicited by a visual and/or auditory stimulus at 40 Hz. While this has shown to be effective in animal models, studies on human participants have reported varying success. The current work hypothesizes that the varying success in humans is due to differences in cognitive workload during the GENUS sessions. Methods: We recruited a cohort of 15 participants who underwent a scalp-EEG recording as well as one epilepsy patient who was implanted with 50 subdural surface electrodes over temporo-occipital and temporo-basal cortex and 14 depth contacts that targeted the hippocampus and insula. All participants completed several GENUS sessions, in each of which a different cognitive task was performed. Results: We found that the inclusion of a cognitive task during the GENUS session not only has a positive effect on the strength and extent of the gamma entrainment, but also promotes the propagation of gamma entrainment to additional neural areas including deep ones such as hippocampus which were not recruited when no cognitive task was required from the participants. The latter is of particular interest given that the hippocampal complex is considered to be one of the primary targets for AD therapies. Discussion: This work introduces a possible improvement strategy for GENUS therapy that might contribute to increasing the efficacy of the therapy or shortening the time needed for the positive outcome.

Towards predicting ECoG-BCI performance: assessing the potential of scalp-EEG.

Objective. Implanted brain-computer interfaces (BCIs) employ neural signals to control a computer and may offer an alternative communication channel for people with locked-in syndrome (LIS). Promising results have been obtained using signals from the sensorimotor (SM) area. However, in earlier work on home-use of an electrocorticography (ECoG)-based BCI by people with LIS, we detected differences in ECoG-BCI performance, which were related to differences in the modulation of low frequency band (LFB) power in the SM area. For future clinical implementation of ECoG-BCIs, it will be crucial to determine whether reliable performance can be predicted before electrode implantation. To assess if non-invasive scalp-electroencephalography (EEG) could serve such prediction, we here investigated if EEG can detect the characteristics observed in the LFB modulation of ECoG signals.Approach. We included three participants with LIS of the earlier study, and a control group of 20 healthy participants. All participants performed a Rest task, and a Movement task involving actual (healthy) or attempted (LIS) hand movements, while their EEG signals were recorded.Main results.Data of the Rest task was used to determine signal-to-noise ratio, which showed a similar range for LIS and healthy participants. Using data of the Movement task, we selected seven EEG electrodes that showed a consistent movement-related decrease in beta power (13-30 Hz) across healthy participants. Within the EEG recordings of this subset of electrodes of two LIS participants, we recognized the phenomena reported earlier for the LFB in their ECoG recordings. Specifically, strong movement-related beta band suppression was observed in one, but not the other, LIS participant, and movement-related alpha band (8-12 Hz) suppression was practically absent in both. Results of the third LIS participant were inconclusive due to technical issues with the EEG recordings.Significance. Together, these findings support a potential role for scalp EEG in the presurgical assessment of ECoG-BCI candidates.

Information theory filters for wavelet packet coefficient selection with application to corrosion type identification from acoustic emission signals.

The damage caused by corrosion in chemical process installations can lead to unexpected plant shutdowns and the leakage of potentially toxic chemicals into the environment. When subjected to corrosion, structural changes in the material occur, leading to energy releases as acoustic waves. This acoustic activity can in turn be used for corrosion monitoring, and even for predicting the type of corrosion. Here we apply wavelet packet decomposition to extract features from acoustic emission signals. We then use the extracted wavelet packet coefficients for distinguishing between the most important types of corrosion processes in the chemical process industry: uniform corrosion, pitting and stress corrosion cracking. The local discriminant basis selection algorithm can be considered as a standard for the selection of the most discriminative wavelet coefficients. However, it does not take the statistical dependencies between wavelet coefficients into account. We show that, when these dependencies are ignored, a lower accuracy is obtained in predicting the corrosion type. We compare several mutual information filters to take these dependencies into account in order to arrive at a more accurate prediction.

Chronic Study on Brainwave Authentication in a Real-Life Setting: An LSTM-Based Bagging Approach.

With the advent of the digital age, concern about how to secure authorized access to sensitive data is increasing. Besides traditional authentication methods, there is an interest in biometric traits such as fingerprints, the iris, facial characteristics, and, recently, brainwaves, primarily based on electroencephalography (EEG). Current work on EEG-based authentication focuses on acute recordings in laboratory settings using high-end equipment, typically equipped with 64 channels and operating at a high sampling rate. In this work, we validated the feasibility of EEG-based authentication in a real-world, out-of-laboratory setting using a commercial dry-electrode EEG headset and chronic recordings on a population of 15 healthy people. We used an LSTM-based network with bootstrap aggregating (bagging) to decode our recordings in response to a multitask scheme consisting of performed and imagined motor tasks, and showed that it improved the performance of the standard LSTM approach. We achieved an authentication accuracy, false acceptance rate (FAR), and false rejection rate (FRR) of 92.6%, 2.5%, and 5.0% for the performed motor task; 92.5%, 2.6%, and 4.9% for the imagined motor task; and 93.0%, 1.9%, and 5.1% for the combined tasks, respectively. We recommend the proposed method for time- and data-limited scenarios.