Electrophysiological model of human temporal contrast sensitivity based on SSVEP.

The present study aims to connect the psychophysical research on the human visual perception of flicker with the neurophysiological research on steady-state visual evoked potentials (SSVEPs) in the context of their application needs and current technological developments. In four experiments, we investigated whether a temporal contrast sensitivity model could be established based on the electrophysiological responses to repetitive visual stimulation and, if so, how this model compares to the psychophysical models of flicker visibility. We used data from 62 observers viewing periodic flicker at a range of frequencies and modulation depths sampled around the perceptual visibility thresholds. The resulting temporal contrast sensitivity curve (TCSC) was similar in shape to its psychophysical counterpart, confirming that the human visual system is most sensitive to repetitive visual stimulation at frequencies between 10 and 20 Hz. The electrophysiological TCSC, however, was below the psychophysical TCSC measured in our experiments for lower frequencies (1-50 Hz), crossed it when the frequency was 50 Hz, and stayed above while decreasing at a slower rate for frequencies in the gamma range (40-60 Hz). This finding provides evidence that SSVEPs could be measured even without the conscious perception of flicker, particularly at frequencies above 50 Hz. The cortical and perceptual mechanisms that apply at higher temporal frequencies, however, do not seem to directly translate to lower frequencies. The presence of harmonics, which show better response for many frequencies, suggests non-linear processing in the visual system. These findings are important for the potential applications of SSVEPs in studying, assisting, or augmenting human cognitive and sensorimotor functions.

Inducing a mental context for associative memory formation with real-time fMRI neurofeedback.

Memory, one of the hallmarks of human cognition, can be modified when humans voluntarily modulate neural population activity using neurofeedback. However, it is currently unknown whether neurofeedback can influence the integration of memories, and whether memory is facilitated or impaired after such neural perturbation. In this study, participants memorized objects while we provided them with abstract neurofeedback based on their brain activity patterns in the ventral visual stream. This neurofeedback created an implicit face or house context in the brain while memorizing the objects. The results revealed that participants created associations between each memorized object and its implicit context solely due to the neurofeedback manipulation. Our findings shed light onto how memory formation can be influenced by synthetic memory tags with neurofeedback and advance our understanding of mnemonic processing.

Brain-computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review.

Objective.Code-modulated visual evoked potentials (c-VEP) have been consolidated in recent years as robust control signals capable of providing non-invasive brain-computer interfaces (BCIs) for reliable, high-speed communication. Their usefulness for communication and control purposes has been reflected in an exponential increase of related articles in the last decade. The aim of this review is to provide a comprehensive overview of the literature to gain understanding of the existing research on c-VEP-based BCIs, since its inception (1984) until today (2021), as well as to identify promising future research lines.Approach.The literature review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. After assessing the eligibility of journal manuscripts, conferences, book chapters and non-indexed documents, a total of 70 studies were included. A comprehensive analysis of the main characteristics and design choices of c-VEP-based BCIs was discussed, including stimulation paradigms, signal processing, modeling responses, applications, etc.Main results.The literature review showed that state-of-the-art c-VEP-based BCIs are able to provide an accurate control of the system with a large number of commands, high selection speeds and even without calibration. In general, a lack of validation in real setups was observed, especially regarding the validation with disabled populations. Future work should be focused toward developing self-paced c-VEP-based portable BCIs applied in real-world environments that could exploit the unique benefits of c-VEP paradigms. Some aspects such as asynchrony, unsupervised training, or code optimization still require further research and development.Significance.Despite the growing popularity of c-VEP-based BCIs, to the best of our knowledge, this is the first literature review on the topic. In addition to providing a joint discussion of the advances in the field, some future lines of research are suggested to contribute to the development of reliable plug-and-play c-VEP-based BCIs.

A visual brain-computer interface as communication aid for patients with amyotrophic lateral sclerosis.

OBJECTIVE: Brain-Computer Interface (BCI) spellers that make use of code-modulated Visual Evoked Potentials (cVEP) may provide a fast and more accurate alternative to existing visual BCI spellers for patients with Amyotrophic Lateral Sclerosis (ALS). However, so far the cVEP speller has only been tested on healthy participants. METHODS: We assess the brain responses, BCI performance and user experience of the cVEP speller in 20 healthy participants and 10 ALS patients. All participants performed a cued and free spelling task, and a free selection of Yes/No answers. RESULTS: 27 out of 30 participants could perform the cued spelling task with an average accuracy of 79% for ALS patients, 88% for healthy older participants and 94% for healthy young participants. All 30 participants could answer Yes/No questions freely, with an average accuracy of around 90%. CONCLUSIONS: With ALS patients typing on average 10 characters per minute, the cVEP speller presented in this paper outperforms other visual BCI spellers. SIGNIFICANCE: These results support a general usability of cVEP signals for ALS patients, which may extend far beyond the tested speller to control e.g. an alarm, automatic door, or TV within a smart home.

SSVEP phase synchronies and propagation during repetitive visual stimulation at high frequencies.

Steady-state visual evoked potentials (SSVEPs), the brain response to visual flicker stimulation, have proven beneficial in both research and clinical applications. Despite the practical advantages of stimulation at high frequencies in terms of visual comfort and safety, high frequency-SSVEPs have not received enough attention and little is known about the mechanisms behind their generation and propagation in time and space. In this study, we investigated the origin and propagation of SSVEPs in the gamma frequency band (40-60 Hz) by studying the dynamic properties of EEG in 32 subjects. Using low-resolution brain electromagnetic tomography (sLORETA) we identified the cortical sources involved in SSVEP generation in that frequency range to be in the primary visual cortex, Brodmann areas 17, 18 and 19 with minor contribution from sources in central and frontal sites. We investigated the SSVEP propagation as measured on the scalp in the framework of the existing theories regarding the neurophysiological mechanism through which the SSVEP spreads through the cortex. We found a progressive phase shift from posterior parieto-occipital sites over the cortex with a phase velocity of approx. 8-14 m/s and wavelength of about 21 and 24 cm. The SSVEP spatial properties appear sensitive to input frequency with higher stimulation frequencies showing a faster propagation speed.

Electrophysiological responses of relatedness to consecutive word stimuli in relation to an actively recollected target word.

In this paper, we investigate the robustness of electrophysiological responses of relatedness to multiple consecutive word stimuli (probes), in relation to an actively recollected target word. Such relatedness information could be used by a Brain Computer Interface to infer the active semantic concept on a user's mind, by integrating the knowledge of the relationship between the multiple probe words and the 'unknown' target. Such a BCI can take advantage of the N400: an event related potential that is sensitive to semantic content of a stimulus in relation to an established semantic context. However, it is unknown whether the N400 is suited for the multiple probing paradigm we propose, as other intervening words might distract from the established context (i.e., the target word). We perform an experiment in which we present up to ten words after an initial target word, and find no attenuation of the strength of the N400 in grand average ERPs and no decrease in classification accuracy for probes occurring later in the sequences. These results are groundwork for developing a BCI that infers the concept on a user's mind through repeated probing, however, low single trial decoding accuracy, and high subject variability may limit practical applicability.

Perception and production in interaction during non-native speech category learning.

Establishing non-native phoneme categories can be a notoriously difficult endeavour-in both speech perception and speech production. This study asks how these two domains interact in the course of this learning process. It investigates the effect of perceptual learning and related production practice of a challenging non-native category on the perception and/or production of that category. A four-day perceptual training protocol on the British English /æ/-/ɛ/ vowel contrast was combined with either related or unrelated production practice. After feedback on perceptual categorisation of the contrast, native Dutch participants in the related production group (N = 19) pronounced the trial's correct answer, while participants in the unrelated production group (N = 19) pronounced similar but phonologically unrelated words. Comparison of pre- and post-tests showed significant improvement over the course of training in both perception and production, but no differences between the groups were found. The lack of an effect of production practice is discussed in the light of previous, competing results and models of second-language speech perception and production. This study confirms that, even in the context of related production practice, perceptual training boosts production learning.

Preserved and attenuated electrophysiological correlates of visual spatial attention in elderly subjects.

Healthy aging is associated with changes in many neurocognitive functions. While on the behavioral level, visual spatial attention capacities are relatively stable with increasing age, the underlying neural processes change. In this study, we investigated attention-related modulations of the stimulus-locked event-related potential (ERP) and occipital oscillations in the alpha band (8-14Hz) in young and elderly participants. Both groups performed a visual attention task equally well and the ERP showed comparable attention-related modulations in both age groups. However, in elderly subjects, oscillations in the alpha band were massively reduced both during the task and in the resting state and the typical task-related lateralized pattern of alpha activity was not observed. These differences between young and elderly participants were observed on the group level as well as on the single trial level. The results indicate that younger and older adults use different neural strategies to reach the same performance in a covert visual spatial attention task.

Decoding Sensorimotor Rhythms during Robotic-Assisted Treadmill Walking for Brain Computer Interface (BCI) Applications.

Locomotor malfunction represents a major problem in some neurological disorders like stroke and spinal cord injury. Robot-assisted walking devices have been used during rehabilitation of patients with these ailments for regaining and improving walking ability. Previous studies showed the advantage of brain-computer interface (BCI) based robot-assisted training combined with physical therapy in the rehabilitation of the upper limb after stroke. Therefore, stroke patients with walking disorders might also benefit from using BCI robot-assisted training protocols. In order to develop such BCI, it is necessary to evaluate the feasibility to decode walking intention from cortical patterns during robot-assisted gait training. Spectral patterns in the electroencephalogram (EEG) related to robot-assisted active and passive walking were investigated in 10 healthy volunteers (mean age 32.3±10.8, six female) and in three acute stroke patients (all male, mean age 46.7±16.9, Berg Balance Scale 20±12.8). A logistic regression classifier was used to distinguish walking from baseline in these spectral EEG patterns. Mean classification accuracies of 94.0±5.4% and 93.1±7.9%, respectively, were reached when active and passive walking were compared against baseline. The classification performance between passive and active walking was 83.4±7.4%. A classification accuracy of 89.9±5.7% was achieved in the stroke patients when comparing walking and baseline. Furthermore, in the healthy volunteers modulation of low gamma activity in central midline areas was found to be associated with the gait cycle phases, but not in the stroke patients. Our results demonstrate the feasibility of BCI-based robotic-assisted training devices for gait rehabilitation.

Neural dynamics during repetitive visual stimulation.

OBJECTIVE: Steady-state visual evoked potentials (SSVEPs), the brain responses to repetitive visual stimulation (RVS), are widely utilized in neuroscience. Their high signal-to-noise ratio and ability to entrain oscillatory brain activity are beneficial for their applications in brain-computer interfaces, investigation of neural processes underlying brain rhythmic activity (steady-state topography) and probing the causal role of brain rhythms in cognition and emotion. This paper aims at analyzing the space and time EEG dynamics in response to RVS at the frequency of stimulation and ongoing rhythms in the delta, theta, alpha, beta, and gamma bands. APPROACH: We used electroencephalography (EEG) to study the oscillatory brain dynamics during RVS at 10 frequencies in the gamma band (40-60 Hz). We collected an extensive EEG data set from 32 participants and analyzed the RVS evoked and induced responses in the time-frequency domain. MAIN RESULTS: Stable SSVEP over parieto-occipital sites was observed at each of the fundamental frequencies and their harmonics and sub-harmonics. Both the strength and the spatial propagation of the SSVEP response seem sensitive to stimulus frequency. The SSVEP was more localized around the parieto-occipital sites for higher frequencies (>54 Hz) and spread to fronto-central locations for lower frequencies. We observed a strong negative correlation between stimulation frequency and relative power change at that frequency, the first harmonic and the sub-harmonic components over occipital sites. Interestingly, over parietal sites for sub-harmonics a positive correlation of relative power change and stimulation frequency was found. A number of distinct patterns in delta (1-4 Hz), theta (4-8 Hz), alpha (8-12 Hz) and beta (15-30 Hz) bands were also observed. The transient response, from 0 to about 300 ms after stimulation onset, was accompanied by increase in delta and theta power over fronto-central and occipital sites, which returned to baseline after approx. 500 ms. During the steady-state response, we observed alpha band desynchronization over occipital sites and after 500 ms also over frontal sites, while neighboring areas synchronized. The power in beta band over occipital sites increased during the stimulation period, possibly caused by increase in power at sub-harmonic frequencies of stimulation. Gamma power was also enhanced by the stimulation. SIGNIFICANCE: These findings have direct implications on the use of RVS and SSVEPs for neural process investigation through steady-state topography, controlled entrainment of brain oscillations and BCIs. A deep understanding of SSVEP propagation in time and space and the link with ongoing brain rhythms is crucial for optimizing the typical SSVEP applications for studying, assisting, or augmenting human cognitive and sensorimotor function.

Broad-Band Visually Evoked Potentials: Re(con)volution in Brain-Computer Interfacing.

Brain-Computer Interfaces (BCIs) allow users to control devices and communicate by using brain activity only. BCIs based on broad-band visual stimulation can outperform BCIs using other stimulation paradigms. Visual stimulation with pseudo-random bit-sequences evokes specific Broad-Band Visually Evoked Potentials (BBVEPs) that can be reliably used in BCI for high-speed communication in speller applications. In this study, we report a novel paradigm for a BBVEP-based BCI that utilizes a generative framework to predict responses to broad-band stimulation sequences. In this study we designed a BBVEP-based BCI using modulated Gold codes to mark cells in a visual speller BCI. We defined a linear generative model that decomposes full responses into overlapping single-flash responses. These single-flash responses are used to predict responses to novel stimulation sequences, which in turn serve as templates for classification. The linear generative model explains on average 50% and up to 66% of the variance of responses to both seen and unseen sequences. In an online experiment, 12 participants tested a 6 × 6 matrix speller BCI. On average, an online accuracy of 86% was reached with trial lengths of 3.21 seconds. This corresponds to an Information Transfer Rate of 48 bits per minute (approximately 9 symbols per minute). This study indicates the potential to model and predict responses to broad-band stimulation. These predicted responses are proven to be well-suited as templates for a BBVEP-based BCI, thereby enabling communication and control by brain activity only.

Towards a communication brain computer interface based on semantic relations.

This article investigates a possible Brain Computer Interface (BCI) based on semantic relations. The BCI determines which prime word a subject has in mind by presenting probe words using an intelligent algorithm. Subjects indicate when a presented probe word is related to the prime word by a single finger tap. The detection of the neural signal associated with this movement is used by the BCI to decode the prime word. The movement detector combined both the evoked (ERP) and induced (ERD) responses elicited with the movement. Single trial movement detection had an average accuracy of 67%. The decoding of the prime word had an average accuracy of 38% when using 100 probes and 150 possible targets, and 41% after applying a dynamic stopping criterium, reducing the average number of probes to 47. The article shows that the intelligent algorithm used to present the probe words has a significantly higher performance than a random selection of probes. Simulations demonstrate that the BCI also works with larger vocabulary sizes, and the performance scales logarithmically with vocabulary size.

Online decoding of object-based attention using real-time fMRI.

Visual attention is used to selectively filter relevant information depending on current task demands and goals. Visual attention is called object-based attention when it is directed to coherent forms or objects in the visual field. This study used real-time functional magnetic resonance imaging for moment-to-moment decoding of attention to spatially overlapped objects belonging to two different object categories. First, a whole-brain classifier was trained on pictures of faces and places. Subjects then saw transparently overlapped pictures of a face and a place, and attended to only one of them while ignoring the other. The category of the attended object, face or place, was decoded on a scan-by-scan basis using the previously trained decoder. The decoder performed at 77.6% accuracy indicating that despite competing bottom-up sensory input, object-based visual attention biased neural patterns towards that of the attended object. Furthermore, a comparison between different classification approaches indicated that the representation of faces and places is distributed rather than focal. This implies that real-time decoding of object-based attention requires a multivariate decoding approach that can detect these distributed patterns of cortical activity.

Decoding speech perception by native and non-native speakers using single-trial electrophysiological data.

Brain-computer interfaces (BCIs) are systems that use real-time analysis of neuroimaging data to determine the mental state of their user for purposes such as providing neurofeedback. Here, we investigate the feasibility of a BCI based on speech perception. Multivariate pattern classification methods were applied to single-trial EEG data collected during speech perception by native and non-native speakers. Two principal questions were asked: 1) Can differences in the perceived categories of pairs of phonemes be decoded at the single-trial level? 2) Can these same categorical differences be decoded across participants, within or between native-language groups? Results indicated that classification performance progressively increased with respect to the categorical status (within, boundary or across) of the stimulus contrast, and was also influenced by the native language of individual participants. Classifier performance showed strong relationships with traditional event-related potential measures and behavioral responses. The results of the cross-participant analysis indicated an overall increase in average classifier performance when trained on data from all participants (native and non-native). A second cross-participant classifier trained only on data from native speakers led to an overall improvement in performance for native speakers, but a reduction in performance for non-native speakers. We also found that the native language of a given participant could be decoded on the basis of EEG data with accuracy above 80%. These results indicate that electrophysiological responses underlying speech perception can be decoded at the single-trial level, and that decoding performance systematically reflects graded changes in the responses related to the phonological status of the stimuli. This approach could be used in extensions of the BCI paradigm to support perceptual learning during second language acquisition.

Detecting semantic priming at the single-trial level.

Semantic priming is usually studied by examining ERPs over many trials and subjects. This article aims at detecting semantic priming at the single-trial level. By using machine learning techniques it is possible to analyse and classify short traces of brain activity, which could, for example, be used to build a Brain Computer Interface (BCI). This article describes an experiment where subjects were presented with word pairs and asked to decide whether the words were related or not. A classifier was trained to determine whether the subjects judged words as related or unrelated based on one second of EEG data. The results show that the classifier accuracy when training per subject varies between 54% and 67%, and is significantly above chance level for all subjects (N  = 12) and the accuracy when training over subjects varies between 51% and 63%, and is significantly above chance level for 11 subjects, pointing to a general effect.

A multi-signature brain-computer interface: use of transient and steady-state responses.

OBJECTIVE: The aim of this paper was to increase the information transfer in brain-computer interfaces (BCI). Therefore, a multi-signature BCI was developed and investigated. Stimuli were designed to simultaneously evoke transient somatosensory event-related potentials (ERPs) and steady-state somatosensory potentials (SSSEPs) and the ERPs and SSSEPs in isolation. APPROACH: Twelve subjects participated in two sessions. In the first session, the single and combined stimulation conditions were compared on these somatosensory responses and on the classification performance. In the second session the on-line performance with the combined stimulation was evaluated while subjects received feedback. Furthermore, in both sessions, the performance based on ERP and SSSEP features was compared. MAIN RESULTS: No difference was found in the ERPs and SSSEPs between stimulation conditions. The combination of ERP and SSSEP features did not perform better than with ERP features only. In both sessions, the classification performances based on ERP and combined features were higher than the classification based on SSSEP features. SIGNIFICANCE: Although the multi-signature BCI did not increase performance, it also did not negatively impact it. Therefore, such stimuli could be used and the best performing feature set could then be chosen individually.

Shared processing of perception and imagery of music in decomposed EEG.

The current work investigates the brain activation shared between perception and imagery of music as measured with electroencephalography (EEG). Meta-analyses of four separate EEG experiments are presented, each focusing on perception and imagination of musical sound, with differing levels of stimulus complexity. Imagination and perception of simple accented metronome trains, as manifested in the clock illusion, as well as monophonic melodies are discussed, as well as more complex rhythmic patterns and ecologically natural music stimuli. By decomposing the data with principal component analysis (PCA), similar component distributions are found to explain most of the variance in each experiment. All data sets show a fronto-central and a more central component as the largest sources of variance, fitting with projections seen for the network of areas contributing to the N1/P2 complex. We expanded on these results using tensor decomposition. This allows us to add in the tasks to find shared activation, but does not make assumptions of independence or orthogonality and calculates the relative strengths of these components for each task. The components found in the PCA were shown to be further decomposable into parts that load primarily on to the perception or imagery task, or both, thereby adding more detail. It is shown that the frontal and central components have multiple parts that are differentially active during perception and imagination. A number of possible interpretations of these results are discussed, taking into account the different stimulus materials and measurement conditions.

Decoding of single-trial auditory mismatch responses for online perceptual monitoring and neurofeedback.

Multivariate pattern classification methods are increasingly applied to neuroimaging data in the context of both fundamental research and in brain-computer interfacing approaches. Such methods provide a framework for interpreting measurements made at the single-trial level with respect to a set of two or more distinct mental states. Here, we define an approach in which the output of a binary classifier trained on data from an auditory mismatch paradigm can be used for online tracking of perception and as a neurofeedback signal. The auditory mismatch paradigm is known to induce distinct perceptual states related to the presentation of high- and low-probability stimuli, which are reflected in event-related potential (ERP) components such as the mismatch negativity (MMN). The first part of this paper illustrates how pattern classification methods can be applied to data collected in an MMN paradigm, including discussion of the optimization of preprocessing steps, the interpretation of features and how the performance of these methods generalizes across individual participants and measurement sessions. We then go on to show that the output of these decoding methods can be used in online settings as a continuous index of single-trial brain activation underlying perceptual discrimination. We conclude by discussing several potential domains of application, including neurofeedback, cognitive monitoring and passive brain-computer interfaces.

Introducing the tactile speller: an ERP-based brain-computer interface for communication.

In this study, a tactile speller was developed and compared with existing visual speller paradigms in terms of classification performance and elicited event-related potentials (ERPs). The fingertips of healthy participants were stimulated with short mechanical taps while electroencephalographic activity was measured. The letters of the alphabet were allocated to different fingers and subjects could select one of the fingers by silently counting the number of taps on that finger. The offline and online performance of the tactile speller was compared to the overt and covert attention visual matrix speller and the covert attention Hex-o-Spell speller. For the tactile speller, binary target versus non-target classification accuracy was 67% on average. Classification and decoding accuracies of the tactile speller were lower than the overt matrix speller, but higher than the covert matrix speller, and similar to Hex-o-Spell. The average maximum information transfer rate of the tactile speller was 7.8 bits min(-1) (1.51 char min(-1)), with the best subject reaching a bit-rate of 27 bits min(-1) (5.22 char min(-1)). An increased amplitude of the P300 ERP component was found in response to attended stimuli versus unattended stimuli in all speller types. In addition, the tactile and overt matrix spellers also used the N2 component for discriminating between targets and non-targets. Overall, this study shows that it is possible to use a tactile speller for communication. The tactile speller provides a useful alternative to the visual speller, especially for people whose eye gaze is impaired.

Effects of native language on perceptual sensitivity to phonetic cues.

The present study used electrophysiological and behavioral measures to investigate the perception of an English stop consonant contrast by native English listeners and by native Dutch listeners who were highly proficient in English. A /ba/-/pa/ continuum was created from a naturally produced /pa/ token by removing successive periods of aspiration, thus reducing the voice onset time. Although aspiration is a relevant cue for distinguishing voiced and unvoiced labial stop consonants (/b/ and /p/) in English, prevoicing is the primary cue used to distinguish between these categories in Dutch. In the electrophysiological experiment, participants listened to oddball sequences containing the standard /pa/ stimulus and one of three deviant stimuli while the mismatch-negativity response was measured. Participants then completed an identification task on the same stimuli. The results showed that native English participants were more sensitive to reductions in aspiration than native Dutch participants, as indicated by shifts in the category boundary, by differing within-group patterns of mismatch-negativity responses, and by larger mean evoked potential amplitudes in the native English group for two of the three deviant stimuli. This between-group difference in the sensorineural processing of aspiration cues indicates that native language experience alters the way in which the acoustic features of speech are processed in the auditory brain, even following extensive second-language training.