Semantic encoding during language comprehension at single-cell resolution.

From sequences of speech sounds1,2 or letters3, humans can extract rich and nuanced meaning through language. This capacity is essential for human communication. Yet, despite a growing understanding of the brain areas that support linguistic and semantic processing4-12, the derivation of linguistic meaning in neural tissue at the cellular level and over the timescale of action potentials remains largely unknown. Here we recorded from single cells in the left language-dominant prefrontal cortex as participants listened to semantically diverse sentences and naturalistic stories. By tracking their activities during natural speech processing, we discover a fine-scale cortical representation of semantic information by individual neurons. These neurons responded selectively to specific word meanings and reliably distinguished words from nonwords. Moreover, rather than responding to the words as fixed memory representations, their activities were highly dynamic, reflecting the words' meanings based on their specific sentence contexts and independent of their phonetic form. Collectively, we show how these cell ensembles accurately predicted the broad semantic categories of the words as they were heard in real time during speech and how they tracked the sentences in which they appeared. We also show how they encoded the hierarchical structure of these meaning representations and how these representations mapped onto the cell population. Together, these findings reveal a finely detailed cortical organization of semantic representations at the neuron scale in humans and begin to illuminate the cellular-level processing of meaning during language comprehension.

Single-neuronal elements of speech production in humans.

Humans are capable of generating extraordinarily diverse articulatory movement combinations to produce meaningful speech. This ability to orchestrate specific phonetic sequences, and their syllabification and inflection over subsecond timescales allows us to produce thousands of word sounds and is a core component of language1,2. The fundamental cellular units and constructs by which we plan and produce words during speech, however, remain largely unknown. Here, using acute ultrahigh-density Neuropixels recordings capable of sampling across the cortical column in humans, we discover neurons in the language-dominant prefrontal cortex that encoded detailed information about the phonetic arrangement and composition of planned words during the production of natural speech. These neurons represented the specific order and structure of articulatory events before utterance and reflected the segmentation of phonetic sequences into distinct syllables. They also accurately predicted the phonetic, syllabic and morphological components of upcoming words and showed a temporally ordered dynamic. Collectively, we show how these mixtures of cells are broadly organized along the cortical column and how their activity patterns transition from articulation planning to production. We also demonstrate how these cells reliably track the detailed composition of consonant and vowel sounds during perception and how they distinguish processes specifically related to speaking from those related to listening. Together, these findings reveal a remarkably structured organization and encoding cascade of phonetic representations by prefrontal neurons in humans and demonstrate a cellular process that can support the production of speech.

Electrochemical and electrophysiological considerations for clinical high channel count neural interfaces.

Electrophysiological recording and stimulation are the gold standard for functional mapping during surgical and therapeutic interventions as well as capturing cellular activity in the intact human brain. A critical component probing human brain activity is the interface material at the electrode contact that electrochemically transduces brain signals to and from free charge carriers in the measurement system. Here, we summarize state-of-the-art electrode array systems in the context of translation for use in recording and stimulating human brain activity. We leverage parametric studies with multiple electrode materials to shed light on the varied levels of suitability to enable high signal-to-noise electrophysiological recordings as well as safe electrophysiological stimulation delivery. We discuss the effects of electrode scaling for recording and stimulation in pursuit of high spatial resolution, channel count electrode interfaces, delineating the electrode-tissue circuit components that dictate the electrode performance. Finally, we summarize recent efforts in the connectorization and packaging for high channel count electrode arrays and provide a brief account of efforts toward wireless neuronal monitoring systems.

Microscale Physiological Events on the Human Cortical Surface.

Despite ongoing advances in our understanding of local single-cellular and network-level activity of neuronal populations in the human brain, extraordinarily little is known about their "intermediate" microscale local circuit dynamics. Here, we utilized ultra-high-density microelectrode arrays and a rare opportunity to perform intracranial recordings across multiple cortical areas in human participants to discover three distinct classes of cortical activity that are not locked to ongoing natural brain rhythmic activity. The first included fast waveforms similar to extracellular single-unit activity. The other two types were discrete events with slower waveform dynamics and were found preferentially in upper cortical layers. These second and third types were also observed in rodents, nonhuman primates, and semi-chronic recordings from humans via laminar and Utah array microelectrodes. The rates of all three events were selectively modulated by auditory and electrical stimuli, pharmacological manipulation, and cold saline application and had small causal co-occurrences. These results suggest that the proper combination of high-resolution microelectrodes and analytic techniques can capture neuronal dynamics that lay between somatic action potentials and aggregate population activity. Understanding intermediate microscale dynamics in relation to single-cell and network dynamics may reveal important details about activity in the full cortical circuit.

Local and distant responses to single pulse electrical stimulation reflect different forms of connectivity.

Measuring connectivity in the human brain involves innumerable approaches using both noninvasive (fMRI, EEG) and invasive (intracranial EEG or iEEG) recording modalities, including the use of external probing stimuli, such as direct electrical stimulation. To examine how different measures of connectivity correlate with one another, we compared 'passive' measures of connectivity during resting state conditions to the more 'active' probing measures of connectivity with single pulse electrical stimulation (SPES). We measured the network engagement and spread of the cortico-cortico evoked potential (CCEP) induced by SPES at 53 out of 104 total sites across the brain, including cortical and subcortical regions, in patients with intractable epilepsy (N=11) who were undergoing intracranial recordings as a part of their clinical care for identifying seizure onset zones. We compared the CCEP network to functional, effective, and structural measures of connectivity during a resting state in each patient. Functional and effective connectivity measures included correlation or Granger causality measures applied to stereoEEG (sEEGs) recordings. Structural connectivity was derived from diffusion tensor imaging (DTI) acquired before intracranial electrode implant and monitoring (N=8). The CCEP network was most similar to the resting state voltage correlation network in channels near to the stimulation location. In contrast, the distant CCEP network was most similar to the DTI network. Other connectivity measures were not as similar to the CCEP network. These results demonstrate that different connectivity measures, including those derived from active stimulation-based probing, measure different, complementary aspects of regional interrelationships in the brain.

CLoSES: A platform for closed-loop intracranial stimulation in humans.

Targeted interrogation of brain networks through invasive brain stimulation has become an increasingly important research tool as well as therapeutic modality. The majority of work with this emerging capability has been focused on open-loop approaches. Closed-loop techniques, however, could improve neuromodulatory therapies and research investigations by optimizing stimulation approaches using neurally informed, personalized targets. Implementing closed-loop systems is challenging particularly with regard to applying consistent strategies considering inter-individual variability. In particular, during intracranial epilepsy monitoring, where much of this research is currently progressing, electrodes are implanted exclusively for clinical reasons. Thus, detection and stimulation sites must be participant- and task-specific. The system must run in parallel with clinical systems, integrate seamlessly with existing setups, and ensure safety features are in place. In other words, a robust, yet flexible platform is required to perform different tests with a single participant and to comply with clinical requirements. In order to investigate closed-loop stimulation for research and therapeutic use, we developed a Closed-Loop System for Electrical Stimulation (CLoSES) that computes neural features which are then used in a decision algorithm to trigger stimulation in near real-time. To summarize CLoSES, intracranial electroencephalography (iEEG) signals are acquired, band-pass filtered, and local and network features are continuously computed. If target features are detected (e.g. above a preset threshold for a certain duration), stimulation is triggered. Not only could the system trigger stimulation while detecting real-time neural features, but we incorporated a pipeline wherein we used an encoder/decoder model to estimate a hidden cognitive state from the neural features. CLoSES provides a flexible platform to implement a variety of closed-loop experimental paradigms in humans. CLoSES has been successfully used with twelve patients implanted with depth electrodes in the epilepsy monitoring unit. During cognitive tasks (N=5), stimulation in closed loop modified a cognitive hidden state on a trial by trial basis. Sleep spindle oscillations (N=6) and sharp transient epileptic activity (N=9) were detected in near real-time, and stimulation was applied during the event or at specified delays (N=3). In addition, we measured the capabilities of the CLoSES system. Total latency was related to the characteristics of the event being detected, with tens of milliseconds for epileptic activity and hundreds of milliseconds for spindle detection. Stepwise latency, the actual duration of each continuous step, was within the specified fixed-step duration and increased linearly with the number of channels and features. We anticipate that probing neural dynamics and interaction between brain states and stimulation responses with CLoSES will lead to novel insights into the mechanism of normal and pathological brain activity, the discovery and evaluation of potential electrographic biomarkers of neurological and psychiatric disorders, and the development and testing of patient-specific stimulation targets and control signals before implanting a therapeutic device.

Selective Formation of Porous Pt Nanorods for Highly Electrochemically Efficient Neural Electrode Interfaces.

The enhanced electrochemical activity of nanostructured materials is readily exploited in energy devices, but their utility in scalable and human-compatible implantable neural interfaces can significantly advance the performance of clinical and research electrodes. We utilize low-temperature selective dealloying to develop scalable and biocompatible one-dimensional platinum nanorod (PtNR) arrays that exhibit superb electrochemical properties at various length scales, stability, and biocompatibility for high performance neurotechnologies. PtNR arrays record brain activity with cellular resolution from the cortical surfaces in birds and nonhuman primates. Significantly, strong modulation of surface recorded single unit activity by auditory stimuli is demonstrated in European Starling birds as well as the modulation of local field potentials in the visual cortex by light stimuli in a nonhuman primate and responses to electrical stimulation in mice. PtNRs record behaviorally and physiologically relevant neuronal dynamics from the surface of the brain with high spatiotemporal resolution, which paves the way for less invasive brain-machine interfaces.

Multimodal Encoding of Novelty, Reward, and Learning in the Primate Nucleus Basalis of Meynert.

Associative learning is crucial for daily function, involving a complex network of brain regions. One region, the nucleus basalis of Meynert (NBM), is a highly interconnected, largely cholinergic structure implicated in multiple aspects of learning. We show that single neurons in the NBM of nonhuman primates (NHPs; n = 2 males; Macaca mulatta) encode learning a new association through spike rate modulation. However, the power of low-frequency local field potential (LFP) oscillations decreases in response to novel, not-yet-learned stimuli but then increase as learning progresses. Both NBM and the dorsolateral prefrontal cortex encode confidence in novel associations by increasing low- and high-frequency LFP power in anticipation of expected rewards. Finally, NBM high-frequency power dynamics are anticorrelated with spike rate modulations. Therefore, novelty, learning, and reward anticipation are separately encoded through differentiable NBM signals. By signaling both the need to learn and confidence in newly acquired associations, NBM may play a key role in coordinating cortical activity throughout the learning process.SIGNIFICANCE STATEMENT Degradation of cells in a key brain region, the nucleus basalis of Meynert (NBM), correlates with Alzheimer's disease and Parkinson's disease progression. To better understand the role of this brain structure in learning and memory, we examined neural activity in the NBM in behaving nonhuman primates while they performed a learning and memory task. We found that single neurons in NBM encoded both salience and an early learning, or cognitive state, whereas populations of neurons in the NBM and prefrontal cortex encode learned state and reward anticipation. The NBM may thus encode multiple stages of learning. These multimodal signals might be leveraged in future studies to develop neural stimulation to facilitate different stages of learning and memory.

Decoding Hidden Cognitive States From Behavior and Physiology Using a Bayesian Approach.

Cognitive processes, such as learning and cognitive flexibility, are both difficult to measure and to sample continuously using objective tools because cognitive processes arise from distributed, high-dimensional neural activity. For both research and clinical applications, that dimensionality must be reduced. To reduce dimensionality and measure underlying cognitive processes, we propose a modeling framework in which a cognitive process is defined as a low-dimensional dynamical latent variable-called a cognitive state, which links high-dimensional neural recordings and multidimensional behavioral readouts. This framework allows us to decompose the hard problem of modeling the relationship between neural and behavioral data into separable encoding-decoding approaches. We first use a state-space modeling framework, the behavioral decoder, to articulate the relationship between an objective behavioral readout (e.g., response times) and cognitive state. The second step, the neural encoder, involves using a generalized linear model (GLM) to identify the relationship between the cognitive state and neural signals, such as local field potential (LFP). We then use the neural encoder model and a Bayesian filter to estimate cognitive state using neural data (LFP power) to generate the neural decoder. We provide goodness-of-fit analysis and model selection criteria in support of the encoding-decoding result. We apply this framework to estimate an underlying cognitive state from neural data in human participants (N=8) performing a cognitive conflict task. We successfully estimated the cognitive state within the 95% confidence intervals of that estimated using behavior readout for an average of 90% of task trials across participants. In contrast to previous encoder-decoder models, our proposed modeling framework incorporates LFP spectral power to encode and decode a cognitive state. The framework allowed us to capture the temporal evolution of the underlying cognitive processes, which could be key to the development of closed-loop experiments and treatments.

Selective attention in the honeybee optic lobes precedes behavioral choices.

Attention allows animals to respond selectively to competing stimuli, enabling some stimuli to evoke a behavioral response while others are ignored. How the brain does this remains mysterious, although it is increasingly evident that even animals with the smallest brains display this capacity. For example, insects respond selectively to salient visual stimuli, but it is unknown where such selectivity occurs in the insect brain, or whether neural correlates of attention might predict the visual choices made by an insect. Here, we investigate neural correlates of visual attention in behaving honeybees (Apis mellifera). Using a closed-loop paradigm that allows tethered, walking bees to actively control visual objects in a virtual reality arena, we show that behavioral fixation increases neuronal responses to flickering, frequency-tagged stimuli. Attention-like effects were reduced in the optic lobes during replay of the same visual sequences, when bees were not able to control the visual displays. When bees were presented with competing frequency-tagged visual stimuli, selectivity in the medulla (an optic ganglion) preceded behavioral selection of a stimulus, suggesting that modulation of early visual processing centers precedes eventual behavioral choices made by these insects.

Closed-Loop Behavioral Control Increases Coherence in the Fly Brain.

A crucial function of the brain is to be able to distinguish whether or not changes in the environment are caused by one's own actions. Even the smallest brains appear to be capable of making this distinction, as has been shown by closed-loop behavioral experiments in flies controlling visual stimuli in virtual reality paradigms. We questioned whether activity in the fruit fly brain is different during such closed-loop behavior, compared with passive viewing of a stimulus. To address this question, we used a procedure to record local field potential (LFP) activity across the fly brain while flies were controlling a virtual object through their movement on an air-supported ball. The virtual object was flickered at a precise frequency (7 Hz), creating a frequency tag that allowed us to track brain responses to the object while animals were behaving. Following experiments under closed-loop control, we replayed the same stimulus to the fly in open loop, such that it could no longer control the stimulus. We found identical receptive fields and similar strength of frequency tags across the brain for the virtual object under closed loop and replay. However, when comparing central versus peripheral brain regions, we found that brain responses were differentially modulated depending on whether flies were in control or not. Additionally, coherence of LFP activity in the brain increased when flies were in control, compared with replay, even if motor behavior was similar. This suggests that processes associated with closed-loop control promote temporal coordination in the insect brain. SIGNIFICANCE STATEMENT: We show that closed-loop control of a visual stimulus promotes temporal coordination across the Drosophila brain, compared with open-loop replay of the same visual sequences. This is significant because it suggests that, to understand goal-directed behavior or visual attention in flies, it may be most informative to sample neural activity from multiple regions across the brain simultaneously, and to examine temporal relationships (e.g., coherence) between these regions.

Insects modify their behaviour depending on the feedback sensor used when walking on a trackball in virtual reality.

When using virtual-reality paradigms to study animal behaviour, careful attention must be paid to how the animal's actions are detected. This is particularly relevant in closed-loop experiments where the animal interacts with a stimulus. Many different sensor types have been used to measure aspects of behaviour, and although some sensors may be more accurate than others, few studies have examined whether, and how, such differences affect an animal's behaviour in a closed-loop experiment. To investigate this issue, we conducted experiments with tethered honeybees walking on an air-supported trackball and fixating a visual object in closed-loop. Bees walked faster and along straighter paths when the motion of the trackball was measured in the classical fashion - using optical motion sensors repurposed from computer mice - than when measured more accurately using a computer vision algorithm called 'FicTrac'. When computer mouse sensors were used to measure bees' behaviour, the bees modified their behaviour and achieved improved control of the stimulus. This behavioural change appears to be a response to a systematic error in the computer mouse sensor that reduces the sensitivity of this sensor system under certain conditions. Although the large perceived inertia and mass of the trackball relative to the honeybee is a limitation of tethered walking paradigms, observing differences depending on the sensor system used to measure bee behaviour was not expected. This study suggests that bees are capable of fine-tuning their motor control to improve the outcome of the task they are performing. Further, our findings show that caution is required when designing virtual-reality experiments, as animals can potentially respond to the artificial scenario in unexpected and unintended ways.

Differential cortical layer engagement during seizure initiation and spread in humans.

Despite decades of research, we still do not understand how spontaneous human seizures start and spread - especially at the level of neuronal microcircuits. In this study, we used laminar arrays of micro-electrodes to simultaneously record the local field potentials and multi-unit neural activities across the six layers of the neocortex during focal seizures in humans. We found that, within the ictal onset zone, the discharges generated during a seizure consisted of current sinks and sources only within the infra-granular and granular layers. Outside of the seizure onset zone, ictal discharges reflected current flow in the supra-granular layers. Interestingly, these patterns of current flow evolved during the course of the seizure - especially outside the seizure onset zone where superficial sinks and sources extended into the deeper layers. Based on these observations, a framework describing cortical-cortical dynamics of seizures is proposed with implications for seizure localization, surgical targeting, and neuromodulation techniques to block the generation and propagation of seizures.

Simultaneous invasive and non-invasive recordings in humans: A novel Rosetta stone for deciphering brain activity.

Simultaneous noninvasive and invasive electrophysiological recordings provide a unique opportunity to achieve a comprehensive understanding of human brain activity, much like a Rosetta stone for human neuroscience. In this review we focus on the increasingly-used powerful combination of intracranial electroencephalography (iEEG) with scalp electroencephalography (EEG) or magnetoencephalography (MEG). We first provide practical insight on how to achieve these technically challenging recordings. We then provide examples from clinical research on how simultaneous recordings are advancing our understanding of epilepsy. This is followed by the illustration of how human neuroscience and methodological advances could benefit from these simultaneous recordings. We conclude with a call for open data sharing and collaboration, while ensuring neuroethical approaches and argue that only with a true collaborative approach the promises of simultaneous recordings will be fulfilled.

An electroencephalogram microdisplay to visualize neuronal activity on the brain surface.

Functional mapping during brain surgery is applied to define brain areas that control critical functions and cannot be removed. Currently, these procedures rely on verbal interactions between the neurosurgeon and electrophysiologist, which can be time-consuming. In addition, the electrode grids that are used to measure brain activity and to identify the boundaries of pathological versus functional brain regions have low resolution and limited conformity to the brain surface. Here, we present the development of an intracranial electroencephalogram (iEEG)-microdisplay that consists of freestanding arrays of 2048 GaN light-emitting diodes laminated on the back of micro-electrocorticography electrode grids. With a series of proof-of-concept experiments in rats and pigs, we demonstrate that these iEEG-microdisplays allowed us to perform real-time iEEG recordings and display cortical activities by spatially corresponding light patterns on the surface of the brain in the surgical field. Furthermore, iEEG-microdisplays allowed us to identify and display cortical landmarks and pathological activities from rat and pig models. Using a dual-color iEEG-microdisplay, we demonstrated coregistration of the functional cortical boundaries with one color and displayed the evolution of electrical potentials associated with epileptiform activity with another color. The iEEG-microdisplay holds promise to facilitate monitoring of pathological brain activity in clinical settings.

Intraoperative application and early experience with novel high-resolution, high-channel-count thin-film electrodes for human microelectrocorticography.

OBJECTIVE: The study objective was to evaluate intraoperative experience with newly developed high-spatial-resolution microelectrode grids composed of poly(3,4-ethylenedioxythiophene) with polystyrene sulfonate (PEDOT:PSS), and those composed of platinum nanorods (PtNRs). METHODS: A cohort of patients who underwent craniotomy for pathological tissue resection and who had high-spatial-resolution microelectrode grids placed intraoperatively were evaluated. Patient demographic and baseline clinical variables as well as relevant microelectrode grid characteristic data were collected. The primary and secondary outcome measures of interest were successful microelectrode grid utilization with usable resting-state or task-related data, and grid-related adverse intraoperative events and/or grid dysfunction. RESULTS: Included in the analysis were 89 cases of patients who underwent a craniotomy for resection of neoplasms (n = 58) or epileptogenic tissue (n = 31). These cases accounted for 94 grids: 58 PEDOT:PSS and 36 PtNR grids. Of these 94 grids, 86 were functional and used successfully to obtain cortical recordings from 82 patients. The mean cortical grid recording duration was 15.3 ± 1.15 minutes. Most recordings in patients were obtained during experimental tasks (n = 52, 58.4%), involving language and sensorimotor testing paradigms, or were obtained passively during resting state (n = 32, 36.0%). There were no intraoperative adverse events related to grid placement. However, there were instances of PtNR grid dysfunction (n = 8) related to damage incurred by suboptimal preoperative sterilization (n = 7) and improper handling (n = 1); intraoperative recordings were not performed. Vaporized peroxide sterilization was the most optimal sterilization method for PtNR grids, providing a significantly greater number of usable channels poststerilization than did steam-based sterilization techniques (median 905.0 [IQR 650.8-935.5] vs 356.0 [IQR 18.0-597.8], p = 0.0031). CONCLUSIONS: High-spatial-resolution microelectrode grids can be readily incorporated into appropriately selected craniotomy cases for clinical and research purposes. Grids are reliable when preoperative handling and sterilization considerations are accounted for. Future investigations should compare the diagnostic utility of these high-resolution grids to commercially available counterparts and assess whether diagnostic discrepancies relate to clinical outcomes.

Flexible, scalable, high channel count stereo-electrode for recording in the human brain.

Over the past decade, stereotactically placed electrodes have become the gold standard for deep brain recording and stimulation for a wide variety of neurological and psychiatric diseases. Current electrodes, however, are limited in their spatial resolution and ability to record from small populations of neurons, let alone individual neurons. Here, we report on an innovative, customizable, monolithically integrated human-grade flexible depth electrode capable of recording from up to 128 channels and able to record at a depth of 10 cm in brain tissue. This thin, stylet-guided depth electrode is capable of recording local field potentials and single unit neuronal activity (action potentials), validated across species. This device represents an advance in manufacturing and design approaches which extends the capabilities of a mainstay technology in clinical neurology.

Modular Phoneme Processing in Human Superior Temporal Gyrus.

Modular organization is fundamental to cortical processing, but its presence is human association cortex is unknown. We characterized phoneme processing with 128-1024 channel micro-arrays at 50-200µm pitch on superior temporal gyrus of 7 patients. High gamma responses were highly correlated within ~1.7mm diameter modules, sharply delineated from adjacent modules with distinct time-courses and phoneme-selectivity. We suggest that receptive language cortex may be organized in discrete processing modules.

Multichannel brain recordings in behaving Drosophila reveal oscillatory activity and local coherence in response to sensory stimulation and circuit activation.

Neural networks in vertebrates exhibit endogenous oscillations that have been associated with functions ranging from sensory processing to locomotion. It remains unclear whether oscillations may play a similar role in the insect brain. We describe a novel "whole brain" readout for Drosophila melanogaster using a simple multichannel recording preparation to study electrical activity across the brain of flies exposed to different sensory stimuli. We recorded local field potential (LFP) activity from >2,000 registered recording sites across the fly brain in >200 wild-type and transgenic animals to uncover specific LFP frequency bands that correlate with: 1) brain region; 2) sensory modality (olfactory, visual, or mechanosensory); and 3) activity in specific neural circuits. We found endogenous and stimulus-specific oscillations throughout the fly brain. Central (higher-order) brain regions exhibited sensory modality-specific increases in power within narrow frequency bands. Conversely, in sensory brain regions such as the optic or antennal lobes, LFP coherence, rather than power, best defined sensory responses across modalities. By transiently activating specific circuits via expression of TrpA1, we found that several circuits in the fly brain modulate LFP power and coherence across brain regions and frequency domains. However, activation of a neuromodulatory octopaminergic circuit specifically increased neuronal coherence in the optic lobes during visual stimulation while decreasing coherence in central brain regions. Our multichannel recording and brain registration approach provides an effective way to track activity simultaneously across the fly brain in vivo, allowing investigation of functional roles for oscillations in processing sensory stimuli and modulating behavior.