Don't put words in my mouth: Speech perception can generate False Positive activation of a speech BCI.

Recent studies have demonstrated that speech can be decoded from brain activity and used for brain-computer interface (BCI)-based communication. It is however also known that the area often used as a signal source for speech decoding BCIs, the sensorimotor cortex (SMC), is also engaged when people perceive speech, thus making speech perception a potential source of false positive activation of the BCI. The current study investigated if and how speech perception may interfere with reliable speech BCI control. We recorded high-density electrocorticography (HD-ECoG) data from five subjects while they performed a speech perception and speech production task and trained a support-vector machine (SVM) on the produced speech data. Our results show that decoders that are highly reliable at detecting self-produced speech from brain signals also generate false positives during the perception of speech. We conclude that speech perception interferes with reliable BCI control, and that efforts to limit the occurrence of false positives during daily-life BCI use should be implemented in BCI design to increase the likelihood of successful adaptation by end users.

Reliability and validity of DTI-based indirect disconnection measures.

White matter connections enable the interaction within and between brain networks. Brain lesions can cause structural disconnections that disrupt networks and thereby cognitive functions supported by them. In recent years, novel methods have been developed to quantify the extent of structural disconnection after focal lesions, using tractography data from healthy controls. These methods, however, are indirect and their reliability and validity have yet to be fully established. In this study, we present our implementation of this approach, in a tool supplemented by uncertainty metrics for the predictions overall and at voxel-level. These metrics give an indication of the reliability and are used to compare predictions with direct measures from patients' diffusion tensor imaging (DTI) data in a sample of 95 first-ever stroke patients. Results show that, except for small lesions, the tool can predict fiber loss with high reliability and compares well to direct patient DTI estimates. Clinical utility of the method was demonstrated using lesion data from a subset of patients suffering from hemianopia. Both tract-based measures outperformed lesion localization in mapping visual field defects and showed a network consistent with the known anatomy of the visual system. This study offers an important contribution to the validation of structural disconnection mapping. We show that indirect measures of structural disconnection can be a reliable and valid substitute for direct estimations of fiber loss after focal lesions. Moreover, based on these results, we argue that indirect structural disconnection measures may even be preferable to lower-quality single subject diffusion MRI when based on high-quality healthy control datasets.

The impact of etiology in lesion-symptom mapping - A direct comparison between tumor and stroke.

INTRODUCTION: Lesion-symptom mapping is a key tool in understanding the relationship between brain structures and behavior. However, the behavioral consequences of lesions from different etiologies may vary because of how they affect brain tissue and how they are distributed. The inclusion of different etiologies would increase the statistical power but has been critically debated. Meanwhile, findings from lesion studies are a valuable resource for clinicians and used across different etiologies. Therefore, the main objective of the present study was to directly compare lesion-symptom maps for memory and language functions from two populations, a tumor versus a stroke population. METHODS: Data from two different studies were combined. Both the brain tumor (N = 196) and stroke (N = 147) patient populations underwent neuropsychological testing and an MRI, pre-operatively for the tumor population and within three months after stroke. For this study, we selected two internationally widely used standardized cognitive tasks, the Rey Auditory Verbal Learning Test and the Verbal Fluency Test. We used a state-of-the-art machine learning-based, multivariate voxel-wise approach to produce lesion-symptom maps for these cognitive tasks for both populations separately and combined. RESULTS: Our lesion-symptom mapping results for the separate patient populations largely followed the expected neuroanatomical pattern based on previous literature. Substantial differences in lesion distribution hindered direct comparison. Still, in brain areas with adequate coverage in both groups, considerable LSM differences between the two populations were present for both memory and fluency tasks. Post-hoc analyses of these locations confirmed that the cognitive consequences of focal brain damage varied between etiologies. CONCLUSION: The differences in the lesion-symptom maps between the stroke and tumor population could partly be explained by differences in lesion volume and topography. Despite these methodological limitations, both the lesion-symptom mapping results and the post-hoc analyses confirmed that etiology matters when investigating the cognitive consequences of lesions with lesion-symptom mapping. Therefore, caution is advised with generalizing lesion-symptom results across etiologies.

Decoding attempted phantom hand movements from ipsilateral sensorimotor areas after amputation.

Objective.The sensorimotor cortex is often selected as target in the development of a Brain-Computer Interface, as activation patterns from this region can be robustly decoded to discriminate between different movements the user executes. Up until recently, such BCIs were primarily based on activity in the contralateral hemisphere, where decoding movements still works even years after denervation. However, there is increasing evidence for a role of the sensorimotor cortex in controlling the ipsilateral body. The aim of this study is to investigate the effects of denervation on the movement representation on the ipsilateral sensorimotor cortex.Approach.Eight subjects with acquired above-elbow arm amputation and nine controls performed a task in which they made (or attempted to make with their phantom hand) six different gestures from the American Manual Alphabet. Brain activity was measured using 7T functional MRI, and a classifier was trained to discriminate between activation patterns on four different regions of interest (ROIs) on the ipsilateral sensorimotor cortex.Main results.Classification scores showed that decoding was possible and significantly better than chance level for both the phantom and intact hands from all ROIs. Decoding both the left (intact) and right (phantom) hand from the same hemisphere was also possible with above-chance level classification score.Significance.The possibility to decode both hands from the same hemisphere, even years after denervation, indicates that implantation of motor-electrodes for BCI control possibly need only cover a single hemisphere, making surgery less invasive, and increasing options for people with lateralized damage to motor cortex like after stroke.

When to include ECoG electrode properties in volume conduction models.

OBJECTIVE: Implantable electrodes, such as electrocorticography (ECoG) grids, are used to record brain activity in applications like brain computer interfaces. To improve the spatial sensitivity of ECoG grid recordings, electrode properties need to be better understood. Therefore, the goal of this study is to analyze the importance of including electrodes explicitly in volume conduction calculations. APPROACH: We investigated the influence of ECoG electrode properties on potentials in three geometries with three different electrode models. We performed our simulations with FEMfuns, a volume conduction modeling software toolbox based on the finite element method. MAIN RESULTS: The presence of the electrode alters the potential distribution by an amount that depends on its surface impedance, its distance from the source and the strength of the source. Our modeling results show that when ECoG electrodes are near the sources the potentials in the underlying tissue are more uniform than without electrodes. We show that the recorded potential can change up to a factor of 3, if no extended electrode model is used. In conclusion, when the distance between an electrode and the source is equal to or smaller than the size of the electrode, electrode effects cannot be disregarded. Furthermore, the potential distribution of the tissue under the electrode is affected up to depths equal to the radius of the electrode. SIGNIFICANCE: This paper shows the importance of explicitly including electrode properties in volume conduction models for accurately interpreting ECoG measurements.

Fatigue in brain tumor patients, towards a neuronal biomarker.

BACKGROUND: Many brain tumor patients suffer from fatigue which severely affects their quality of life. There is a lack of objective measurements for fatigue in brain tumor patients. We aimed to find a neurophysiological correlate for fatigue in brain tumor patients. For this purpose, we correlated brain activity associated with phasic alertness with self-reported ratings of fatigue. METHODS: Patients with a meningioma, a low-grade glioma or a high-grade glioma (N = 63) participated in this fMRI study. Brain activity in the central executive network (CEN) and default mode network (DMN) associated with phasic alertness was correlated with self-reported fatigue measured with the multidimensional fatigue inventory (MFI-20). Follow-up analyses were performed for MFI-20 domain scores, individual regions within CEN and DMN, and the tumor sub-groups separately. RESULTS: MFI-20 scores correlated significantly with DMN activity associated with phasic alertness, but not with CEN activity. These results were consistent for each tumor sub-group. Within the DMN, the correlations were strongest in left and right lingual cortex, left and right cuneus, and right precuneus. DISCUSSION: Self-reported fatigue in brain tumor patients was associated with objective measurements of brain activity, specifically the DMN activity related to phasic alertness. This association represents an important step in the development of a biomarker for fatigue in brain tumor patients, and possibly for other patients that suffer from fatigue.

FEMfuns: A Volume Conduction Modeling Pipeline that Includes Resistive, Capacitive or Dispersive Tissue and Electrodes.

Applications such as brain computer interfaces require recordings of relevant neuronal population activity with high precision, for example, with electrocorticography (ECoG) grids. In order to achieve this, both the placement of the electrode grid on the cortex and the electrode properties, such as the electrode size and material, need to be optimized. For this purpose, it is essential to have a reliable tool that is able to simulate the extracellular potential, i.e., to solve the so-called ECoG forward problem, and to incorporate the properties of the electrodes explicitly in the model. In this study, this need is addressed by introducing the first open-source pipeline, FEMfuns (finite element method for useful neuroscience simulations), that allows neuroscientists to solve the forward problem in a variety of different geometrical domains, including different types of source models and electrode properties, such as resistive and capacitive materials. FEMfuns is based on the finite element method (FEM) implemented in FEniCS and includes the geometry tessellation, several electrode-electrolyte implementations and adaptive refinement options. The Python code of the pipeline is available under the GNU General Public License version 3 at https://github.com/meronvermaas/FEMfuns . We tested our pipeline with several geometries and source configurations such as a dipolar source in a multi-layer sphere model and a five-compartment realistically-shaped head model. Furthermore, we describe the main scripts in the pipeline, illustrating its flexible and versatile use. Provided with a sufficiently fine tessellation, the numerical solution of the forward problem approximates the analytical solution. Furthermore, we show dispersive material and interface effects in line with previous literature. Our results indicate substantial capacitive and dispersive effects due to the electrode-electrolyte interface when using stimulating electrodes. The results demonstrate that the pipeline presented in this paper is an accurate and flexible tool to simulate signals generated on electrode grids by the spatiotemporal electrical activity patterns produced by sources and thereby allows the user to optimize grids for brain computer interfaces including exploration of alternative electrode materials/properties.

Processing of Targets and Non-targets in Verbal Working Memory.

In this study we used functional MRI (fMRI) to examine whether defining a stimulus as a target affects brain activation associated with a verbal working memory (WM) task. Seventeen healthy right-handed volunteers performed a Sternberg task with three consonants as memory set. We performed a region of interest based fMRI analysis to examine differences in brain activity patterns between targets and non-targets. Non-target brain activity was subtracted from target activity and hemispheric and fronto-parietal differences were tested by conducting a MANOVA. Participants responded correctly to 97.5% of the stimuli. The fMRI results showed a hemisphere by fronto-parietal location interaction, where targets evoked increased activity in the right frontal regions compared to non-targets, whereas the left frontal task activation did not differ between targets and non-targets. In the parietal regions, targets evoked increased activity compared to non-targets in the lateral anterior, but not the medial posterior part. Our study revealed that defining a stimulus as a target within a verbal WM task evokes an increase in brain activity in right frontal brain regions, compared to non-targets. Our results suggest an important hemispheric differentiation in target processing, in which the right frontal cortex is predominantly involved in processes associated with target stimuli. The left frontal cortex does not differentiate between processing target and non-target stimuli, suggesting involvement in WM processes that are independent of stimulus type. Parietal, the lateral anterior part is predominantly involved in target processing, while the medial posterior part does not differentiate between target and non-target processing.

Classification of Articulator Movements and Movement Direction from Sensorimotor Cortex Activity.

For people suffering from severe paralysis, communication can be difficult or nearly impossible. Technology systems called brain-computer interfaces (BCIs) are being developed to assist these people with communication by using their brain activity to control a computer without any muscle activity. To benefit the development of BCIs that employ neural activity related to speech, we investigated if neural activity patterns related to different articulator movements can be distinguished from each other. We recorded with electrocorticography (ECoG), the neural activity related to different articulator movements in 4 epilepsy patients and classified which articulator participants moved based on the sensorimotor cortex activity patterns. The same was done for different movement directions of a single articulator, the tongue. In both experiments highly accurate classification was obtained, on average 92% for different articulators and 85% for different tongue directions. Furthermore, the data show that only a small part of the sensorimotor cortex is needed for classification (ca. 1 cm2). We show that recordings from small parts of the sensorimotor cortex contain information about different articulator movements which might be used for BCI control. Our results are of interest for BCI systems that aim to decode neural activity related to (actual or attempted) movements from a contained cortical area.

Repeated Vowel Production Affects Features of Neural Activity in Sensorimotor Cortex.

The sensorimotor cortex is responsible for the generation of movements and interest in the ability to use this area for decoding speech by brain-computer interfaces has increased recently. Speech decoding is challenging however, since the relationship between neural activity and motor actions is not completely understood. Non-linearity between neural activity and movement has been found for instance for simple finger movements. Despite equal motor output, neural activity amplitudes are affected by preceding movements and the time between movements. It is unknown if neural activity is also affected by preceding motor actions during speech. We addressed this issue, using electrocorticographic high frequency band (HFB; 75-135 Hz) power changes in the sensorimotor cortex during discrete vowel generation. Three subjects with temporarily implanted electrode grids produced the /i/ vowel at repetition rates of 1, 1.33 and 1.66 Hz. For every repetition, the HFB power amplitude was determined. During the first utterance, most electrodes showed a large HFB power peak, which decreased for subsequent utterances. This result could not be explained by differences in performance. With increasing duration between utterances, more electrodes showed an equal response to all repetitions, suggesting that the duration between vowel productions influences the effect of previous productions on sensorimotor cortex activity. Our findings correspond with previous studies for finger movements and bear relevance for the development of brain-computer interfaces that employ speech decoding based on brain signals, in that past utterances will need to be taken into account for these systems to work accurately.

The influence of prior pronunciations on sensorimotor cortex activity patterns during vowel production.

OBJECTIVE: In recent years, brain-computer interface (BCI) systems have been investigated for their potential as a communication device to assist people with severe paralysis. Decoding speech sensorimotor cortex activity is a promising avenue for the generation of BCI control signals, but is complicated by variability in neural patterns, leading to suboptimal decoding. We investigated whether neural pattern variability associated with sound pronunciation can be explained by prior pronunciations and determined to what extent prior speech affects BCI decoding accuracy. APPROACH: Neural patterns in speech motor areas were evaluated with electrocorticography in five epilepsy patients, who performed a simple speech task that involved pronunciation of the /i/ sound, preceded by either silence, the /a/ sound or the /u/ sound. MAIN RESULTS: The neural pattern related to the /i/ sound depends on previous sounds and is therefore associated with multiple distinct sensorimotor patterns, which is likely to reflect differences in the movements towards this sound. We also show that these patterns still contain a commonality that is distinct from the other vowel sounds (/a/ and /u/). Classification accuracies for the decoding of different sounds do increase, however, when the multiple patterns for the /i/ sound are taken into account. Simply including multiple forms of the /i/ vowel in the training set for the creation of a single /i/ model performs as well as training individual models for each /i/ variation. SIGNIFICANCE: Our results are of interest for the development of BCIs that aim to decode speech sounds from the sensorimotor cortex, since they argue that a multitude of cortical activity patterns associated with speech movements can be reduced to a basis set of models which reflect meaningful language units (vowels), yet it is important to account for the variety of neural patterns associated with a single sound in the training process.

Spatial-Temporal Dynamics of the Sensorimotor Cortex: Sustained and Transient Activity.

How the sensorimotor cortex is organized with respect to controlling different features of movement is unclear. One unresolved question concerns the relation between the duration of an action and the duration of the associated neuronal activity change in the sensorimotor cortex. Using subdural electrocorticography electrodes, we investigated in five subjects, whether high frequency band (HFB; 75-135 Hz) power changes have a transient or sustained relation to speech duration, during pronunciation of the Dutch /i/ vowel with different durations. We showed that the neuronal activity patterns recorded from the sensorimotor cortex can be directly related to action duration in some locations, whereas in other locations, during the same action, neuronal activity is transient, with a peak in HFB activity at movement onset and/or offset. This data sheds light on the neural underpinnings of motor actions and we discuss the possible mechanisms underlying these different response types.

Detailed view on slow sinusoidal, hemodynamic oscillations on the human brain cortex by Fourier transforming oxy/deoxy hyperspectral images.

Slow sinusoidal, hemodynamic oscillations (SSHOs) around 0.1 Hz are frequently seen in mammalian and human brains. In four patients undergoing epilepsy surgery, subtle but robust fluctuations in oxy- and deoxyhemoglobin were detected using hyperspectral imaging of the cortex. These SSHOs were stationary during the entire 4 to 10 min acquisition time. By Fourier filtering the oxy- and deoxyhemoglobin time signals with a small bandwidth, SSHOs became visible within localized regions of the brain, with distinctive frequencies and a continuous phase variation within that region. SSHOs of deoxyhemoglobin appeared to have an opposite phase and 11% smaller amplitude with respect to the oxyhemoglobin SSHOs. Although the origin of SSHOs remains unclear, we find indications that the observed SSHOs may embody a local propagating hemodynamic wave with velocities in line with capillary blood velocities, and conceivably related to vasomotion and maintenance of adequate tissue perfusion. Hyperspectral imaging of the human cortex during surgery allow in-depth characterization of SSHOs, and may give further insight in the nature and potential (clinical) use of SSHOs.

Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids.

For people who cannot communicate due to severe paralysis or involuntary movements, technology that decodes intended speech from the brain may offer an alternative means of communication. If decoding proves to be feasible, intracranial Brain-Computer Interface systems can be developed which are designed to translate decoded speech into computer generated speech or to instructions for controlling assistive devices. Recent advances suggest that such decoding may be feasible from sensorimotor cortex, but it is not clear how this challenge can be approached best. One approach is to identify and discriminate elements of spoken language, such as phonemes. We investigated feasibility of decoding four spoken phonemes from the sensorimotor face area, using electrocorticographic signals obtained with high-density electrode grids. Several decoding algorithms including spatiotemporal matched filters, spatial matched filters and support vector machines were compared. Phonemes could be classified correctly at a level of over 75% with spatiotemporal matched filters. Support Vector machine analysis reached a similar level, but spatial matched filters yielded significantly lower scores. The most informative electrodes were clustered along the central sulcus. Highest scores were achieved from time windows centered around voice onset time, but a 500 ms window before onset time could also be classified significantly. The results suggest that phoneme production involves a sequence of robust and reproducible activity patterns on the cortical surface. Importantly, decoding requires inclusion of temporal information to capture the rapid shifts of robust patterns associated with articulator muscle group contraction during production of a phoneme. The high classification scores are likely to be enabled by the use of high density grids, and by the use of discrete phonemes. Implications for use in Brain-Computer Interfaces are discussed.

Changes in fMRI BOLD dynamics reflect anticipation to moving objects.

The human brain is thought to respond differently to novel versus predictable neural input. In human visual cortex, neural response amplitude to visual input might be determined by the degree of predictability. We investigated how fMRI BOLD responses in human early visual cortex reflect the anticipation of a single moving bar's trajectory. We found that BOLD signals decreased linearly from onset to offset of the stimulus trajectory. Moreover, decreased amplitudes of BOLD responses coincided with an increased initial dip as the stimulus moved along its trajectory. Importantly, motion anticipation effects were absent, when motion coherence was disrupted by means of stimulus contrast reversals. These results show that human early visual cortex anticipates the trajectory of a coherently moving object at the initial stages of visual motion processing. The results can be explained by suppression of predictable input, plausibly underlying the formation of stable visual percepts.

Give me a sign: decoding four complex hand gestures based on high-density ECoG.

The increasing understanding of human brain functions makes it possible to directly interact with the brain for therapeutic purposes. Implantable brain computer interfaces promise to replace or restore motor functions in patients with partial or complete paralysis. We postulate that neuronal states associated with gestures, as they are used in the finger spelling alphabet of sign languages, provide an excellent signal for implantable brain computer interfaces to restore communication. To test this, we evaluated decodability of four gestures using high-density electrocorticography in two participants. The electrode grids were located subdurally on the hand knob area of the sensorimotor cortex covering a surface of 2.5-5.2 cm(2). Using a pattern-matching classification approach four types of hand gestures were classified based on their pattern of neuronal activity. In the two participants the gestures were classified with 97 and 74% accuracy. The high frequencies (>65 Hz) allowed for the best classification results. This proof-of-principle study indicates that the four gestures are associated with a reliable and discriminable spatial representation on a confined area of the sensorimotor cortex. This robust representation on a small area makes hand gestures an interesting control feature for an implantable BCI to restore communication for severely paralyzed people.

Classification of mouth movements using 7 T fMRI.

OBJECTIVE: A brain-computer interface (BCI) is an interface that uses signals from the brain to control a computer. BCIs will likely become important tools for severely paralyzed patients to restore interaction with the environment. The sensorimotor cortex is a promising target brain region for a BCI due to the detailed topography and minimal functional interference with other important brain processes. Previous studies have shown that attempted movements in paralyzed people generate neural activity that strongly resembles actual movements. Hence decodability for BCI applications can be studied in able-bodied volunteers with actual movements. APPROACH: In this study we tested whether mouth movements provide adequate signals in the sensorimotor cortex for a BCI. The study was executed using fMRI at 7 T to ensure relevance for BCI with cortical electrodes, as 7 T measurements have been shown to correlate well with electrocortical measurements. Twelve healthy volunteers executed four mouth movements (lip protrusion, tongue movement, teeth clenching, and the production of a larynx activating sound) while in the scanner. Subjects performed a training and a test run. Single trials were classified based on the Pearson correlation values between the activation patterns per trial type in the training run and single trials in the test run in a 'winner-takes-all' design. MAIN RESULTS: Single trial mouth movements could be classified with 90% accuracy. The classification was based on an area with a volume of about 0.5 cc, located on the sensorimotor cortex. If voxels were limited to the surface, which is accessible for electrode grids, classification accuracy was still very high (82%). Voxels located on the precentral cortex performed better (87%) than the postcentral cortex (72%). SIGNIFICANCE: The high reliability of decoding mouth movements suggests that attempted mouth movements are a promising candidate for BCI in paralyzed people.

Predictions to motion stimuli in human early visual cortex: Effects of motion displacement on motion predictability.

Recently, several studies showed that fMRI BOLD responses to moving random dot stimuli are enhanced at the location of dot appearance, i.e., the motion trailing edge. Possibly, BOLD activity in human visual cortex reflects predictability of visual motion input. In the current study, we investigate to what extent fMRI BOLD responses reflect estimated predictions to visual motion. We varied motion displacement parameters (duration and velocity), while measuring BOLD amplitudes as a function of distance from the trailing edge. We have found that for all stimulus configurations, BOLD signals decrease with increasing distance from the trailing edge. This finding indicates that neural activity directly reflects the predictability of moving dots, rather than their appearance within classical receptive fields. However, different motion displacement parameters exerted only marginal effects on predictability, suggesting that early visual cortex does not literally predict motion trajectories. Rather, the results reveal a heuristic mechanism of motion suppression from trailing to leading edge, plausibly mediated through short-range horizontal connections. Simple heuristic suppression allows the visual system to recognize novel input among many motion signals, while being most energy efficient.

Brain implants for substituting lost motor function: state of the art and potential impact on the lives of motor-impaired seniors.

Recent scientific achievements bring the concept of neural prosthetics for reinstating lost motor function closer to medical application. Current research involves severely paralyzed people under the age of 65, but implications for seniors with stroke or trauma-induced impairments are clearly on the horizon. Demographic changes will lead to a shortage of personnel to care for an increasing population of senior citizens, threatening maintenance of an acceptable level of care and urging ways for people to live longer at their home independent from personal assistance. This is particularly challenging when people suffer from disabilities such as partial paralysis after stroke or trauma, where daily personal assistance is required. For some of these people, neural prosthetics can reinstate some lost motor function and/or lost communication, thereby increasing independence and possibly quality of life. In this viewpoint article, we present the state of the art in decoding brain activity in the service of brain-computer interfacing. Although some noninvasive applications produce good results, we focus on brain implants that benefit from better quality brain signals. Fully implantable neural prostheses for home use are not available yet, but clinical trials are being prepared. More sophisticated systems are expected to follow in the years to come, with capabilities of interest for less severe paralysis. Eventually the combination of smart robotics and brain implants is expected to enable people to interact well enough with their environment to live an independent life in spite of motor disabilities.