Longevity of a Brain-Computer Interface for Amyotrophic Lateral Sclerosis.

The durability of communication with the use of brain-computer interfaces in persons with progressive neurodegenerative disease has not been extensively examined. We report on 7 years of independent at-home use of an implanted brain-computer interface for communication by a person with advanced amyotrophic lateral sclerosis (ALS), the inception of which was reported in 2016. The frequency of at-home use increased over time to compensate for gradual loss of control of an eye-gaze-tracking device, followed by a progressive decrease in use starting 6 years after implantation. At-home use ended when control of the brain-computer interface became unreliable. No signs of technical malfunction were found. Instead, the amplitude of neural signals declined, and computed tomographic imaging revealed progressive atrophy, which suggested that ALS-related neurodegeneration ultimately rendered the brain-computer interface ineffective after years of successful use, although alternative explanations are plausible. (Funded by the National Institute on Deafness and Other Communication Disorders and others; ClinicalTrials.gov number, NCT02224469.).

Nine decades of electrocorticography: A comparison between epidural and subdural recordings.

In recent years, electrocorticography (ECoG) has arisen as a neural signal recording tool in the development of clinically viable neural interfaces. ECoG electrodes are generally placed below the dura mater (subdural) but can also be placed on top of the dura (epidural). In deciding which of these modalities best suits long-term implants, complications and signal quality are important considerations. Conceptually, epidural placement may present a lower risk of complications as the dura is left intact but also a lower signal quality due to the dura acting as a signal attenuator. The extent to which complications and signal quality are affected by the dura, however, has been a matter of debate. To improve our understanding of the effects of the dura on complications and signal quality, we conducted a literature review. We inventorized the effect of the dura on signal quality, decodability and longevity of acute and chronic ECoG recordings in humans and non-human primates. Also, we compared the incidence and nature of serious complications in studies that employed epidural and subdural ECoG. Overall, we found that, even though epidural recordings exhibit attenuated signal amplitude over subdural recordings, particularly for high-density grids, the decodability of epidural recorded signals does not seem to be markedly affected. Additionally, we found that the nature of serious complications was comparable between epidural and subdural recordings. These results indicate that both epidural and subdural ECoG may be suited for long-term neural signal recordings, at least for current generations of clinical and high-density ECoG grids.

Fully Implanted Brain-Computer Interface in a Locked-In Patient with ALS.

Options for people with severe paralysis who have lost the ability to communicate orally are limited. We describe a method for communication in a patient with late-stage amyotrophic lateral sclerosis (ALS), involving a fully implanted brain-computer interface that consists of subdural electrodes placed over the motor cortex and a transmitter placed subcutaneously in the left side of the thorax. By attempting to move the hand on the side opposite the implanted electrodes, the patient accurately and independently controlled a computer typing program 28 weeks after electrode placement, at the equivalent of two letters per minute. The brain-computer interface offered autonomous communication that supplemented and at times supplanted the patient's eye-tracking device. (Funded by the Government of the Netherlands and the European Union; ClinicalTrials.gov number, NCT02224469 .).

Decoding hand gestures from primary somatosensory cortex using high-density ECoG.

Electrocorticography (ECoG) based Brain-Computer Interfaces (BCIs) have been proposed as a way to restore and replace motor function or communication in severely paralyzed people. To date, most motor-based BCIs have either focused on the sensorimotor cortex as a whole or on the primary motor cortex (M1) as a source of signals for this purpose. Still, target areas for BCI are not confined to M1, and more brain regions may provide suitable BCI control signals. A logical candidate is the primary somatosensory cortex (S1), which not only shares similar somatotopic organization to M1, but also has been suggested to have a role beyond sensory feedback during movement execution. Here, we investigated whether four complex hand gestures, taken from the American sign language alphabet, can be decoded exclusively from S1 using both spatial and temporal information. For decoding, we used the signal recorded from a small patch of cortex with subdural high-density (HD) grids in five patients with intractable epilepsy. Notably, we introduce a new method of trial alignment based on the increase of the electrophysiological response, which virtually eliminates the confounding effects of systematic and non-systematic temporal differences within and between gestures execution. Results show that S1 classification scores are high (76%), similar to those obtained from M1 (74%) and sensorimotor cortex as a whole (85%), and significantly above chance level (25%). We conclude that S1 offers characteristic spatiotemporal neuronal activation patterns that are discriminative between gestures, and that it is possible to decode gestures with high accuracy from a very small patch of cortex using subdurally implanted HD grids. The feasibility of decoding hand gestures using HD-ECoG grids encourages further investigation of implantable BCI systems for direct interaction between the brain and external devices with multiple degrees of freedom.

Action preparation shapes processing in early visual cortex.

Preparation for an action, such as grasping an object, is accompanied by an enhanced perception of the object's action-relevant features, such as orientation and size. Cortical feedback from motor planning areas to early visual areas may drive this enhanced perception. To examine whether action preparation modulates activity in early human visual cortex, subjects grasped or pointed to oriented objects while high-resolution fMRI data were acquired. Using multivoxel pattern analysis techniques, we could decode with >70% accuracy whether a grasping or pointing action was prepared from signals in visual cortex as early as V1. These signals in early visual cortex were observed even when actions were only prepared but not executed. Anterior parietal cortex, on the other hand, showed clearest modulation for actual movements. This demonstrates that preparation of actions, even without execution, modulates relevant neuronal populations in early visual areas.

Propagating neocortical gamma bursts are coordinated by traveling alpha waves.

Neocortical neuronal activity is characterized by complex spatiotemporal dynamics. Although slow oscillations have been shown to travel over space in terms of consistent phase advances, it is unknown how this phenomenon relates to neuronal activity in other frequency bands. We here present electrocorticographic data from three male and one female human subject and demonstrate that gamma power is phase locked to traveling alpha waves. Given that alpha activity has been proposed to coordinate neuronal processing reflected in the gamma band, we suggest that alpha waves are involved in coordinating neuronal processing in both space and time.

Cortical theta wanes for language.

The role of low frequency oscillations in language areas is not yet understood. Using ECoG in six human subjects, we studied whether different language regions show prominent power changes in a specific rhythm, in similar manner as the alpha rhythm shows the most prominent power changes in visual areas. Broca's area and temporal language areas were localized in individual subjects using fMRI. In these areas, the theta rhythm showed the most pronounced power changes and theta power decreased significantly during verb generation. To better understand the role of this language-related theta decrease, we then studied the interaction between low frequencies and local neuronal activity reflected in high frequencies. Amplitude-amplitude correlations showed that theta power correlated negatively with high frequency activity, specifically across verb generation trials. Phase-amplitude coupling showed that during control trials, high frequency power was coupled to theta phase, but this coupling decreased significantly during verb generation trials. These results suggest a dynamic interaction between the neuronal mechanisms underlying the theta rhythm and local neuronal activity in language areas. As visual areas show a pronounced alpha rhythm that may reflect pulsed inhibition, language regions show a pronounced theta rhythm with highly similar features.

Spatiotemporal characteristics of electrocortical brain activity during mental calculation.

Mental calculation is a complex mental procedure involving a frontoparietal network of brain regions. Functional MRI (fMRI) studies have revealed interesting characteristics of these regions, but the precise function of some areas remains elusive. In the present study, we used electrocorticographic (ECoG) recordings to chronometrically assess the neuronal processes during mental arithmetic. A calculation task was performed during presurgical 3T fMRI scanning and subsequent ECoG monitoring. Mental calculation induced an increase in fMRI blood oxygen level dependent signal in prefrontal, parietal and lower temporo-occipital regions. The group-fMRI result was subsequently used to cluster the implanted electrodes into anatomically defined regions of interest (ROIs). We observed remarkable differences in high frequency power profiles between ROIs, some of which were closely associated with stimulus presentation and others with the response. Upon stimulus presentation, occipital areas were the first to respond, followed by parietal and frontal areas, and finally by motor areas. Notably, we demonstrate that the fMRI activation in the middle frontal gyrus/precentral gyrus is associated with two subfunctions during mental calculation. This finding reveals the significance of the temporal dynamics of neural ensembles within regions with an apparent uniform function. In conclusion, our results shed more light on the spatiotemporal aspects of brain activation during a mental calculation task, and demonstrate that the use of fMRI data to cluster ECoG electrodes is a useful approach for ECoG group analysis.

Robust compression and detection of epileptiform patterns in ECoG using a real-time spiking neural network hardware framework.

Interictal Epileptiform Discharges (IED) and High Frequency Oscillations (HFO) in intraoperative electrocorticography (ECoG) may guide the surgeon by delineating the epileptogenic zone. We designed a modular spiking neural network (SNN) in a mixed-signal neuromorphic device to process the ECoG in real-time. We exploit the variability of the inhomogeneous silicon neurons to achieve efficient sparse and decorrelated temporal signal encoding. We interface the full-custom SNN device to the BCI2000 real-time framework and configure the setup to detect HFO and IED co-occurring with HFO (IED-HFO). We validate the setup on pre-recorded data and obtain HFO rates that are concordant with a previously validated offline algorithm (Spearman's ρ = 0.75, p = 1e-4), achieving the same postsurgical seizure freedom predictions for all patients. In a remote on-line analysis, intraoperative ECoG recorded in Utrecht was compressed and transferred to Zurich for SNN processing and successful IED-HFO detection in real-time. These results further demonstrate how automated remote real-time detection may enable the use of HFO in clinical practice.

Dissociation between neuronal activity in sensorimotor cortex and hand movement revealed as a function of movement rate.

It is often assumed that similar behavior is generated by the same brain activity. However, this does not take into account the brain state or recent behavioral history and movement initiation or continuation may not be similarly generated in the brain. To study whether similar movements are generated by the same brain activity, we measured neuronal population activity during repeated movements. Three human subjects performed a motor repetition task in which they moved their hand at four different rates (0.3, 0.5, 1, and 2 Hz). From high-resolution electrocorticography arrays implanted on motor and sensory cortex, high-frequency power (65-95 Hz) was extracted as a measure of neuronal population activity. During the two faster movement rates, high-frequency power was significantly suppressed, whereas movement parameters remained highly similar. This suppression was nonlinear: after the initial movement, neuronal population activity was reduced most strongly, and the data fit a model in which a fast decline rapidly converged to saturation. The amplitude of the beta-band suppression did not change with different rates. However, at the faster rates, beta power did not return to baseline between movements but remained suppressed. We take these findings to indicate that the extended beta suppression at the faster rates, which may suggest a release of inhibition in motor cortex, facilitates movement initiation. These results show that the relationship between behavior and neuronal activity is not consistent: recent movement influences the state of motor cortex and facilitates next movements by reducing the required level of neuronal activity.

Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models.

Objective.Development of brain-computer interface (BCI) technology is key for enabling communication in individuals who have lost the faculty of speech due to severe motor paralysis. A BCI control strategy that is gaining attention employs speech decoding from neural data. Recent studies have shown that a combination of direct neural recordings and advanced computational models can provide promising results. Understanding which decoding strategies deliver best and directly applicable results is crucial for advancing the field.Approach.In this paper, we optimized and validated a decoding approach based on speech reconstruction directly from high-density electrocorticography recordings from sensorimotor cortex during a speech production task.Main results.We show that (1) dedicated machine learning optimization of reconstruction models is key for achieving the best reconstruction performance; (2) individual word decoding in reconstructed speech achieves 92%-100% accuracy (chance level is 8%); (3) direct reconstruction from sensorimotor brain activity produces intelligible speech.Significance.These results underline the need for model optimization in achieving best speech decoding results and highlight the potential that reconstruction-based speech decoding from sensorimotor cortex can offer for development of next-generation BCI technology for communication.

Using fMRI to localize target regions for implanted brain-computer interfaces in locked-in syndrome.

OBJECTIVE: Electrocorticography (ECoG)-based brain-computer interface (BCI) systems have the potential to improve quality of life of people with locked-in syndrome (LIS) by restoring their ability to communicate independently. Before implantation of such a system, it is important to localize ECoG electrode target regions. Here, we assessed the predictive value of functional magnetic resonance imaging (fMRI) for the localization of suitable target regions on the sensorimotor cortex for ECoG-based BCI in people with locked-in syndrome. METHODS: Three people with locked-in syndrome were implanted with a chronic, fully implantable ECoG-BCI system. We compared pre-surgical fMRI activity with post-implantation ECoG activity from areas known to be active and inactive during attempted hand movement (sensorimotor hand region and dorsolateral prefrontal cortex, respectively). RESULTS: Results showed a spatial match between fMRI activity and changes in ECoG low and high frequency band power (10 - 30 and 65 - 95 Hz, respectively) during attempted movement. Also, we found that fMRI can be used to select a sub-set of electrodes that show strong task-related signal changes that are therefore likely to generate adequate BCI control. CONCLUSIONS: Our findings indicate that fMRI is a useful non-invasive tool for the pre-surgical workup of BCI implant candidates. SIGNIFICANCE: If these results are confirmed in more BCI studies, fMRI might be used for more efficient surgical BCI procedures with focused cortical coverage and lower participant burden.

Neurophysiologic correlates of fMRI in human motor cortex.

The neurophysiological underpinnings of functional magnetic resonance imaging (fMRI) are not well understood. To understand the relationship between the fMRI blood oxygen level dependent (BOLD) signal and neurophysiology across large areas of cortex, we compared task related BOLD change during simple finger movement to brain surface electric potentials measured on a similar spatial scale using electrocorticography (ECoG). We found that spectral power increases in high frequencies (65-95 Hz), which have been related to local neuronal activity, colocalized with spatially focal BOLD peaks on primary sensorimotor areas. Independent of high frequencies, decreases in low frequency rhythms (<30 Hz), thought to reflect an aspect of cortical-subcortical interaction, colocalized with weaker BOLD signal increase. A spatial regression analysis showed that there was a direct correlation between the amplitude of the task induced BOLD change on different areas of primary sensorimotor cortex and the amplitude of the high frequency change. Low frequency change explained an additional, different part of the spatial BOLD variance. Together, these spectral power changes explained a significant 36% of the spatial variance in the BOLD signal change (R(2) = 0.36). These results suggest that BOLD signal change is largely induced by two separate neurophysiological mechanisms, one being spatially focal neuronal processing and the other spatially distributed low frequency rhythms.

Open multimodal iEEG-fMRI dataset from naturalistic stimulation with a short audiovisual film.

Intracranial human recordings are a valuable and rare resource of information about the brain. Making such data publicly available not only helps tackle reproducibility issues in science, it helps make more use of these valuable data. This is especially true for data collected using naturalistic tasks. Here, we describe a dataset collected from a large group of human subjects while they watched a short audiovisual film. The dataset has several unique features. First, it includes a large amount of intracranial electroencephalography (iEEG) data (51 participants, age range of 5-55 years, who all performed the same task). Second, it includes functional magnetic resonance imaging (fMRI) recordings (30 participants, age range of 7-47) during the same task. Eighteen participants performed both iEEG and fMRI versions of the task, non-simultaneously. Third, the data were acquired using a rich audiovisual stimulus, for which we provide detailed speech and video annotations. This dataset can be used to study neural mechanisms of multimodal perception and language comprehension, and similarity of neural signals across brain recording modalities.

FMRI and intra-cranial electrocorticography recordings in the same human subjects reveals negative BOLD signal coupled with silenced neuronal activity.

Positive blood oxygenation level-dependent (BOLD) responses (PBR), as measured by functional Magnetic Resonance Imaging (fMRI), are the most utilized measurements to non-invasively map activity in the brain. Recent studies have consistently shown that BOLD responses are not exclusively positive. Negative BOLD responses (NBR) have been reported in response to specific sensory stimulations and tasks. However, the exact relationship between NBR and the underlying metabolic and neuronal demand is still under debate. In this study, we investigated the neurophysiological basis of negative BOLD using fMRI and intra-cranial electrophysiology (electrocorticography, ECoG) measurements from the same human participants. We show that, for those electrodes that responded to visual stimulation, PBR are correlated with high-frequency band (HFB) responses. Crucially, NBR were associated with an absence of HFB power responses and an unpredicted decrease in the alpha power responses.

Towards predicting ECoG-BCI performance: assessing the potential of scalp-EEG.

Objective. Implanted brain-computer interfaces (BCIs) employ neural signals to control a computer and may offer an alternative communication channel for people with locked-in syndrome (LIS). Promising results have been obtained using signals from the sensorimotor (SM) area. However, in earlier work on home-use of an electrocorticography (ECoG)-based BCI by people with LIS, we detected differences in ECoG-BCI performance, which were related to differences in the modulation of low frequency band (LFB) power in the SM area. For future clinical implementation of ECoG-BCIs, it will be crucial to determine whether reliable performance can be predicted before electrode implantation. To assess if non-invasive scalp-electroencephalography (EEG) could serve such prediction, we here investigated if EEG can detect the characteristics observed in the LFB modulation of ECoG signals.Approach. We included three participants with LIS of the earlier study, and a control group of 20 healthy participants. All participants performed a Rest task, and a Movement task involving actual (healthy) or attempted (LIS) hand movements, while their EEG signals were recorded.Main results.Data of the Rest task was used to determine signal-to-noise ratio, which showed a similar range for LIS and healthy participants. Using data of the Movement task, we selected seven EEG electrodes that showed a consistent movement-related decrease in beta power (13-30 Hz) across healthy participants. Within the EEG recordings of this subset of electrodes of two LIS participants, we recognized the phenomena reported earlier for the LFB in their ECoG recordings. Specifically, strong movement-related beta band suppression was observed in one, but not the other, LIS participant, and movement-related alpha band (8-12 Hz) suppression was practically absent in both. Results of the third LIS participant were inconclusive due to technical issues with the EEG recordings.Significance. Together, these findings support a potential role for scalp EEG in the presurgical assessment of ECoG-BCI candidates.

Methodological Recommendations for Studies on the Daily Life Implementation of Implantable Communication-Brain-Computer Interfaces for Individuals With Locked-in Syndrome.

Implantable brain-computer interfaces (BCIs) promise to be a viable means to restore communication in individuals with locked-in syndrome (LIS). In 2016, we presented the world-first fully implantable BCI system that uses subdural electrocorticography electrodes to record brain signals and a subcutaneous amplifier to transmit the signals to the outside world, and that enabled an individual with LIS to communicate via a tablet computer by selecting icons in spelling software. For future clinical implementation of implantable communication-BCIs, however, much work is still needed, for example, to validate these systems in daily life settings with more participants, and to improve the speed of communication. We believe the design and execution of future studies on these and other topics may benefit from the experience we have gained. Therefore, based on relevant literature and our own experiences, we here provide an overview of procedures, as well as recommendations, for recruitment, screening, inclusion, imaging, hospital admission, implantation, training, and support of participants with LIS, for studies on daily life implementation of implantable communication-BCIs. With this article, we not only aim to inform the BCI community about important topics of concern, but also hope to contribute to improved methodological standardization of implantable BCI research.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier.

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Brain-computer interfacing based on cognitive control.

OBJECTIVE: Brain-computer interfaces (BCIs) translate deliberate intentions and associated changes in brain activity into action, thereby offering patients with severe paralysis an alternative means of communication with and control over their environment. Such systems are not available yet, partly due to the high performance standard that is required. A major challenge in the development of implantable BCIs is to identify cortical regions and related functions that an individual can reliably and consciously manipulate. Research predominantly focuses on the sensorimotor cortex, which can be activated by imagining motor actions. However, because this region may not provide an optimal solution to all patients, other neuronal networks need to be examined. Therefore, we investigated whether the cognitive control network can be used for BCI purposes. We also determined the feasibility of using functional magnetic resonance imaging (fMRI) for noninvasive localization of the cognitive control network. METHODS: Three patients with intractable epilepsy, who were temporarily implanted with subdural grid electrodes for diagnostic purposes, attempted to gain BCI control using the electrocorticographic (ECoG) signal of the left dorsolateral prefrontal cortex (DLPFC). RESULTS: All subjects quickly gained accurate BCI control by modulation of gamma-power of the left DLPFC. Prelocalization of the relevant region was performed with fMRI and was confirmed using the ECoG signals obtained during mental calculation localizer tasks. INTERPRETATION: The results indicate that the cognitive control network is a suitable source of signals for BCI applications. They also demonstrate the feasibility of translating understanding about cognitive networks derived from functional neuroimaging into clinical applications.

Decoding four hand gestures with a single bipolar pair of electrocorticography electrodes.

Objective.Electrocorticography (ECoG) based brain-computer interfaces (BCIs) can be used to restore communication in individuals with locked-in syndrome. In motor-based BCIs, the number of degrees-of-freedom, and thus the speed of the BCI, directly depends on the number of classes that can be discriminated from the neural activity in the sensorimotor cortex. When considering minimally invasive BCI implants, the size of the subdural ECoG implant must be minimized without compromising the number of degrees-of-freedom.Approach.Here we investigated if four hand gestures could be decoded using a single ECoG strip of four consecutive electrodes spaced 1 cm apart and compared the performance between a unipolar and a bipolar montage. For that we collected data of seven individuals with intractable epilepsy implanted with ECoG grids, covering the hand region of the sensorimotor cortex. Based on the implanted grids, we generated virtual ECoG strips and compared the decoding accuracy between (a) a single unipolar electrode (Unipolar Electrode), (b) a combination of four unipolar electrodes (Unipolar Strip), (c) a single bipolar pair (Bipolar Pair) and (d) a combination of six bipolar pairs (Bipolar Strip).Main results.We show that four hand gestures can be equally well decoded using 'Unipolar Strips' (mean 67.4 ± 11.7%), 'Bipolar Strips' (mean 66.6 ± 12.1%) and 'Bipolar Pairs' (mean 67.6 ± 9.4%), while 'Unipolar Electrodes' (61.6 ± 5.9%) performed significantly worse compared to 'Unipolar Strips' and 'Bipolar Pairs'.Significance.We conclude that a single bipolar pair is a potential candidate for minimally invasive motor-based BCIs and encourage the use of ECoG as a robust and reliable BCI platform for multi-class movement decoding.