ABSTRACT
Speech perception in noise is a challenging everyday task with which many listeners have difficulty. Here, we report a case in which electrical brain stimulation of implanted intracranial electrodes in the left planum temporale (PT) of a neurosurgical patient significantly and reliably improved subjective quality (up to 50%) and objective intelligibility (up to 97%) of speech in noise perception. Stimulation resulted in a selective enhancement of speech sounds compared with the background noises. The receptive fields of the PT sites whose stimulation improved speech perception were tuned to spectrally broad and rapidly changing sounds. Corticocortical evoked potential analysis revealed that the PT sites were located between the sites in Heschl's gyrus and the superior temporal gyrus. Moreover, the discriminability of speech from nonspeech sounds increased in population neural responses from Heschl's gyrus to the PT to the superior temporal gyrus sites. These findings causally implicate the PT in background noise suppression and may point to a novel potential neuroprosthetic solution to assist in the challenging task of speech perception in noise.SIGNIFICANCE STATEMENT Speech perception in noise remains a challenging task for many individuals. Here, we present a case in which the electrical brain stimulation of intracranially implanted electrodes in the planum temporale of a neurosurgical patient significantly improved both the subjective quality (up to 50%) and objective intelligibility (up to 97%) of speech perception in noise. Stimulation resulted in a selective enhancement of speech sounds compared with the background noises. Our local and network-level functional analyses placed the planum temporale sites in between the sites in the primary auditory areas in Heschl's gyrus and nonprimary auditory areas in the superior temporal gyrus. These findings causally implicate planum temporale in acoustic scene analysis and suggest potential neuroprosthetic applications to assist hearing in noise.
Subject(s)
Auditory Cortex , Speech Perception , Acoustic Stimulation , Auditory Cortex/physiology , Brain , Brain Mapping/methods , Hearing , Humans , Magnetic Resonance Imaging/methods , Speech/physiology , Speech Perception/physiologyABSTRACT
Heschl's gyrus (HG) is a brain area that includes the primary auditory cortex in humans. Due to the limitations in obtaining direct neural measurements from this region during naturalistic speech listening, the functional organization and the role of HG in speech perception remain uncertain. Here, we used intracranial EEG to directly record neural activity in HG in eight neurosurgical patients as they listened to continuous speech stories. We studied the spatial distribution of acoustic tuning and the organization of linguistic feature encoding. We found a main gradient of change from posteromedial to anterolateral parts of HG. We also observed a decrease in frequency and temporal modulation tuning and an increase in phonemic representation, speaker normalization, speech sensitivity, and response latency. We did not observe a difference between the two brain hemispheres. These findings reveal a functional role for HG in processing and transforming simple to complex acoustic features and inform neurophysiological models of speech processing in the human auditory cortex.
Subject(s)
Auditory Cortex/physiology , Brain Mapping , Speech Perception/physiology , Adult , Electrocorticography , Epilepsy/diagnosis , Epilepsy/surgery , Female , Humans , Male , Middle Aged , Neurosurgical ProceduresABSTRACT
The progress of therapeutic neuromodulation greatly depends on improving stimulation parameters to most efficiently induce neuroplasticity effects. Intermittent θ-burst stimulation (iTBS), a form of electrical stimulation that mimics natural brain activity patterns, has proved to efficiently induce such effects in animal studies and rhythmic transcranial magnetic stimulation studies in humans. However, little is known about the potential neuroplasticity effects of iTBS applied through intracranial electrodes in humans. This study characterizes the physiological effects of intracranial iTBS in humans and compare them with α-frequency stimulation, another frequently used neuromodulatory pattern. We applied these two stimulation patterns to well-defined regions in the sensorimotor cortex, which elicited contralateral hand muscle contractions during clinical mapping, in patients with epilepsy implanted with intracranial electrodes. Treatment effects were evaluated using oscillatory coherence across areas connected to the treatment site, as defined with corticocortical-evoked potentials. Our results show that iTBS increases coherence in the ß-frequency band within the sensorimotor network indicating a potential neuroplasticity effect. The effect is specific to the sensorimotor system, the ß band, and the stimulation pattern and outlasted the stimulation period by â¼3 min. The effect occurred in four out of seven subjects depending on the buildup of the effect during iTBS treatment and other patterns of oscillatory activity related to ceiling effects within the ß band and to preexistent coherence within the α band. By characterizing the neurophysiological effects of iTBS within well-defined cortical networks, we hope to provide an electrophysiological framework that allows clinicians/researchers to optimize brain stimulation protocols which may have translational value.NEW & NOTEWORTHY θ-Burst stimulation (TBS) protocols in transcranial magnetic stimulation studies have shown improved treatment efficacy in a variety of neuropsychiatric disorders. The optimal protocol to induce neuroplasticity in invasive direct electrical stimulation approaches is not known. We report that intracranial TBS applied in human sensorimotor cortex increases local coherence of preexistent ß rhythms. The effect is specific to the stimulation frequency and the stimulated network and outlasts the stimulation period by â¼3 min.
Subject(s)
Beta Rhythm/physiology , Electric Stimulation Therapy , Electric Stimulation , Electrocorticography , Nerve Net/physiology , Neuronal Plasticity/physiology , Sensorimotor Cortex/physiology , Adult , Female , Humans , Male , Young AdultABSTRACT
Targeted stimulation can be used to modulate the activity of brain networks. Previously we demonstrated that direct electrical stimulation produces predictable poststimulation changes in brain excitability. However, understanding the neural dynamics during stimulation and its relationship to poststimulation effects is limited but critical for treatment optimization. Here, we applied 10 Hz direct electrical stimulation across several cortical regions in 14 human subjects (6 males) implanted with intracranial electrodes for seizure monitoring. The stimulation train was characterized by a consistent increase in high gamma (70-170 Hz) power. Immediately post-train, low-frequency (1-8 Hz) power increased, resulting in an evoked response that was highly correlated with the neural response during stimulation. Using two measures of network connectivity, corticocortical evoked potentials (indexing effective connectivity), and theta coherence (indexing functional connectivity), we found a stronger response to stimulation in regions that were highly connected to the stimulation site. In these regions, repeated cycles of stimulation trains and rest progressively altered the stimulation response. Finally, after just 2 min (â¼10%) of repetitive stimulation, we were able to predict poststimulation connectivity changes with high discriminability. Together, this work reveals a relationship between stimulation dynamics and poststimulation connectivity changes in humans. Thus, measuring neural activity during stimulation can inform future plasticity-inducing protocols.SIGNIFICANCE STATEMENT Brain stimulation tools have the potential to revolutionize the treatment of neuropsychiatric disorders. Despite the widespread use of brain stimulation techniques such as transcranial magnetic stimulation, the therapeutic efficacy of these technologies remains suboptimal. This is in part because of a lack of understanding of the dynamic neural changes that occur during stimulation. In this study, we provide the first detailed characterization of neural activity during plasticity induction through intracranial electrode stimulation and recording in 14 medication-resistant epilepsy patients. These results fill a missing gap in our understanding of stimulation-induced plasticity in humans. In the longer-term, these data will also guide our translational efforts toward non-invasive, personalized, closed-loop neuromodulation therapy for neurological and psychiatric disorders in humans.
Subject(s)
Brain/physiology , Electric Stimulation Therapy , Nerve Net/physiology , Neuronal Plasticity/physiology , Adult , Drug Resistant Epilepsy/physiopathology , Drug Resistant Epilepsy/therapy , Evoked Potentials/physiology , Female , Humans , MaleABSTRACT
How does human brain stimulation result in lasting changes in cortical excitability? Uncertainty on this question hinders the development of personalized brain stimulation therapies. To characterize how cortical excitability is altered by stimulation, we applied repetitive direct electrical stimulation in eight human subjects (male and female) undergoing intracranial monitoring. We evaluated single-pulse corticocortical-evoked potentials (CCEPs) before and after repetitive stimulation across prefrontal (n = 4), temporal (n = 1), and motor (n = 3) cortices. We asked whether a single session of repetitive stimulation was sufficient to induce excitability changes across distributed cortical sites. We found a subset of regions at which 10 Hz prefrontal repetitive stimulation resulted in both potentiation and suppression of excitability that persisted for at least 10 min. We then asked whether these dynamics could be modeled by the prestimulation connectivity profile of each subject. We found that cortical regions (1) anatomically close to the stimulated site and (2) exhibiting high-amplitude CCEPs underwent changes in excitability following repetitive stimulation. We demonstrate high accuracy (72-95%) and discriminability (81-99%) in predicting regions exhibiting changes using individual subjects' prestimulation connectivity profile, and show that adding prestimulation connectivity features significantly improved model performance. The same features predicted regions of modulation following motor and temporal cortices stimulation in an independent dataset. Together, baseline connectivity profile can be used to predict regions susceptible to brain changes and provides a basis for personalizing brain stimulation.SIGNIFICANCE STATEMENT Brain stimulation is increasingly used to treat neuropsychiatric disorders by inducing excitability changes at specific brain regions. However, our understanding of how, when, and where these changes are induced is critically lacking. We inferred plasticity in the human brain after applying electrical stimulation to the brain's surface and measuring changes in excitability. We observed excitability changes in regions anatomically and functionally closer to the stimulation site. Those in responsive regions were accurately predicted using a classifier trained on baseline brain network characteristics. Finally, we showed that the excitability changes can potentially be monitored in real-time. These results begin to fill basic gaps in our understanding of stimulation-induced brain dynamics in humans and offer pathways to optimize stimulation protocols.
Subject(s)
Cerebral Cortex/physiology , Evoked Potentials/physiology , Nerve Net/physiology , Neuronal Plasticity/physiology , Adult , Brain Mapping/methods , Electric Stimulation , Female , Humans , Male , Middle Aged , Young AdultABSTRACT
Whereas the neurophysiology of respiration has traditionally focused on automatic brain stem processes, higher brain mechanisms underlying the cognitive aspects of breathing are gaining increasing interest. Therapeutic techniques have used conscious control and awareness of breathing for millennia with little understanding of the mechanisms underlying their efficacy. Using direct intracranial recordings in humans, we correlated cortical and limbic neuronal activity as measured by the intracranial electroencephalogram (iEEG) with the breathing cycle. We show this to be the direct result of neuronal activity, as demonstrated by both the specificity of the finding to the cortical gray matter and the tracking of breath by the gamma-band (40-150 Hz) envelope in these structures. We extend prior observations by showing the iEEG signal to track the breathing cycle across a widespread network of cortical and limbic structures. We further demonstrate a sensitivity of this tracking to cognitive factors by using tasks adapted from cognitive behavioral therapy and meditative practice. Specifically, volitional control and awareness of breathing engage distinct but overlapping brain circuits. During volitionally paced breathing, iEEG-breath coherence increases in a frontotemporal-insular network, and during attention to breathing, we demonstrate increased coherence in the anterior cingulate, premotor, insular, and hippocampal cortices. Our findings suggest that breathing can act as an organizing hierarchical principle for neuronal oscillations throughout the brain and detail mechanisms of how cognitive factors impact otherwise automatic neuronal processes during interoceptive attention. NEW & NOTEWORTHY Whereas the link between breathing and brain activity has a long history of application to therapy, its neurophysiology remains unexplored. Using intracranial recordings in humans, we show neuronal activity to track the breathing cycle throughout widespread cortical/limbic sites. Volitional pacing of the breath engages frontotemporal-insular cortices, whereas attention to automatic breathing modulates the cingulate cortex. Our findings imply a fundamental role of breathing-related oscillations in driving neuronal activity and provide insight into the neuronal mechanisms of interoceptive attention.
Subject(s)
Attention , Brain Stem/physiology , Cerebral Cortex/physiology , Respiration , Adult , Female , Gamma Rhythm , Humans , MaleABSTRACT
Listeners readily extract multi-dimensional auditory objects such as a 'localized talker' from complex acoustic scenes with multiple talkers. Yet, the neural mechanisms underlying simultaneous encoding and linking of different sound features - for example, a talker's voice and location - are poorly understood. We analyzed invasive intracranial recordings in neurosurgical patients attending to a localized talker in real-life cocktail party scenarios. We found that sensitivity to an individual talker's voice and location features was distributed throughout auditory cortex and that neural sites exhibited a gradient from sensitivity to a single feature to joint sensitivity to both features. On a population level, cortical response patterns of both dual-feature sensitive sites but also single-feature sensitive sites revealed simultaneous encoding of an attended talker's voice and location features. However, for single-feature sensitive sites, the representation of the primary feature was more precise. Further, sites which selective tracked an attended speech stream concurrently encoded an attended talker's voice and location features, indicating that such sites combine selective tracking of an attended auditory object with encoding of the object's features. Finally, we found that attending a localized talker selectively enhanced temporal coherence between single-feature voice sensitive sites and single-feature location sensitive sites, providing an additional mechanism for linking voice and location in multi-talker scenes. These results demonstrate that a talker's voice and location features are linked during multi-dimensional object formation in naturalistic multi-talker scenes by joint population coding as well as by temporal coherence between neural sites. SIGNIFICANCE STATEMENT: Listeners effortlessly extract auditory objects from complex acoustic scenes consisting of multiple sound sources in naturalistic, spatial sound scenes. Yet, how the brain links different sound features to form a multi-dimensional auditory object is poorly understood. We investigated how neural responses encode and integrate an attended talker's voice and location features in spatial multi-talker sound scenes to elucidate which neural mechanisms underlie simultaneous encoding and linking of different auditory features. Our results show that joint population coding as well as temporal coherence mechanisms contribute to distributed multi-dimensional auditory object encoding. These findings shed new light on cortical functional specialization and multidimensional auditory object formation in complex, naturalistic listening scenes. HIGHLIGHTS: Cortical responses to an single talker exhibit a distributed gradient, ranging from sites that are sensitive to both a talker's voice and location (dual-feature sensitive sites) to sites that are sensitive to either voice or location (single-feature sensitive sites).Population response patterns of dual-feature sensitive sites encode voice and location features of the attended talker in multi-talker scenes jointly and with equal precision.Despite their sensitivity to a single feature at the level of individual cortical sites, population response patterns of single-feature sensitive sites also encode location and voice features of a talker jointly, but with higher precision for the feature they are primarily sensitive to.Neural sites which selectively track an attended speech stream concurrently encode the attended talker's voice and location features.Attention selectively enhances temporal coherence between voice and location selective sites over time.Joint population coding as well as temporal coherence mechanisms underlie distributed multi-dimensional auditory object encoding in auditory cortex.
ABSTRACT
Previous studies have investigated the effects of acetylcholine (ACh) on neuronal tuning, coding, and attention in primary visual cortex, but its contribution to coding in extrastriate cortex is unexplored. Here we investigate the effects of ACh on tuning properties of macaque middle temporal area MT neurons and contrast them with effects of gabazine, a GABA(A) receptor blocker. ACh increased neuronal activity, it had no effect on tuning width, but it significantly increased the direction discriminability of a neuron. Gabazine equally increased neuronal activity, but it widened tuning curves and decreased the direction discriminability of a neuron. Although gabazine significantly reduced response reliability, ACh application had little effect on response reliability. Finally, gabazine increased noise correlation of simultaneously recorded neurons, whereas ACh reduced it. Thus, both drugs increased firing rates, but only ACh application improved neuronal tuning and coding in line with effects seen in studies in which attention was selectively manipulated.
Subject(s)
Acetylcholine/pharmacology , Discrimination, Psychological/physiology , Neurons/physiology , Orientation/physiology , Parasympathetic Nervous System/physiology , Temporal Lobe/physiology , gamma-Aminobutyric Acid/physiology , Acetylcholine/administration & dosage , Algorithms , Animals , Attention/drug effects , Discrimination, Psychological/drug effects , Electrophysiological Phenomena , Female , GABA Antagonists/pharmacology , Macaca mulatta , Male , Microinjections , Motion , Neurons/drug effects , Orientation/drug effects , Photic Stimulation , Pyridazines/pharmacology , Receptors, GABA-A/drug effects , Reproducibility of Results , Temporal Lobe/drug effectsABSTRACT
How the human auditory cortex represents spatially separated simultaneous talkers and how talkers' locations and voices modulate the neural representations of attended and unattended speech are unclear. Here, we measured the neural responses from electrodes implanted in neurosurgical patients as they performed single-talker and multi-talker speech perception tasks. We found that spatial separation between talkers caused a preferential encoding of the contralateral speech in Heschl's gyrus (HG), planum temporale (PT), and superior temporal gyrus (STG). Location and spectrotemporal features were encoded in different aspects of the neural response. Specifically, the talker's location changed the mean response level, whereas the talker's spectrotemporal features altered the variation of response around response's baseline. These components were differentially modulated by the attended talker's voice or location, which improved the population decoding of attended speech features. Attentional modulation due to the talker's voice only appeared in the auditory areas with longer latencies, but attentional modulation due to location was present throughout. Our results show that spatial multi-talker speech perception relies upon a separable pre-attentive neural representation, which could be further tuned by top-down attention to the location and voice of the talker.
Subject(s)
Auditory Cortex , Speech Perception , Voice , Auditory Cortex/physiology , Humans , Speech , Speech Perception/physiology , Temporal LobeABSTRACT
The perceptual salience and visibility of image elements is influenced by other elements in their vicinity. The perceptual effect of image elements on an adjacent target element depends on their relative orientation. Collinear flanking elements usually improve sensitivity for the target element, whereas orthogonal elements have a weaker effect. It is believed that the collinear flankers exert these effects through lateral interactions between neurons in the primary visual cortex (area V1), but the precise mechanisms underlying these contextual interactions remain unknown. Here, we directly examined this question by recording the effects of flankers on the responses of V1 neurons at parafoveal representations while monkeys performed a fixation task or a contrast detection task. We found, unexpectedly, that collinear flankers reduce the monkeys' perceptual sensitivity for a central target element. This behavioral effect was explained by a flanker-induced increase in the activity of V1 neurons in the absence of the central target stimulus, which reduced the amplitude of the target response. Our results indicate that the dominant effect of collinear flankers in parafoveal vision is suppression and suggest that these suppressive effects are caused by a decrease in the dynamic range of neurons coding the central target.
Subject(s)
Fovea Centralis/physiology , Neurons/physiology , Visual Cortex/physiology , Analysis of Variance , Animals , Electrophysiology , Macaca mulatta , Orientation/physiology , Photic Stimulation , Visual Perception/physiologyABSTRACT
Context affects the salience and visibility of image elements in visual scenes. Collinear flankers can enhance or decrease the perceptual and neuronal sensitivity to flanked stimuli. These effects are mediated through lateral interactions between neurons in the primary visual cortex (area V1), in conjunction with feedback from higher visual areas. The strength of lateral interactions is affected by cholinergic neuromodulation. Blockade of muscarinic receptors should increase the strength of lateral intracortical interactions, while nicotinic blockade should reduce thalamocortical feed-forward drive. Here we test this proposal through local iontophoretic application of the muscarinic receptor antagonist scopolamine and the nicotinic receptor antagonist mecamylamine, while recording single cells in parafoveal representations in awake fixating macaque V1. Collinear flankers generally reduced neuronal contrast sensitivity. Muscarinic and nicotinic receptor blockade equally reduced neuronal contrast sensitivity. Contrary to our hypothesis, flanker interactions were not systematically affected by either receptor blockade.
Subject(s)
Contrast Sensitivity/physiology , Muscarinic Antagonists/pharmacology , Neurons/physiology , Nicotinic Antagonists/pharmacology , Receptors, Muscarinic/metabolism , Receptors, Nicotinic/metabolism , Visual Cortex/physiology , Animals , Contrast Sensitivity/drug effects , Macaca mulatta , Male , Neurons/drug effects , Photic Stimulation , Receptors, Muscarinic/chemistry , Receptors, Nicotinic/chemistry , Visual Cortex/drug effectsABSTRACT
BACKGROUND: Paralysis and neuropathy, affecting millions of people worldwide, can be accompanied by significant loss of somatosensation. With tactile sensation being central to achieving dexterous movement, brain-computer interface (BCI) researchers have used intracortical and cortical surface electrical stimulation to restore somatotopically-relevant sensation to the hand. However, these approaches are restricted to stimulating the gyral areas of the brain. Since representation of distal regions of the hand extends into the sulcal regions of human primary somatosensory cortex (S1), it has been challenging to evoke sensory percepts localized to the fingertips. OBJECTIVE/HYPOTHESIS: Targeted stimulation of sulcal regions of S1, using stereoelectroencephalography (SEEG) depth electrodes, can evoke focal sensory percepts in the fingertips. METHODS: Two participants with intractable epilepsy received cortical stimulation both at the gyri via high-density electrocorticography (HD-ECoG) grids and in the sulci via SEEG depth electrode leads. We characterized the evoked sensory percepts localized to the hand. RESULTS: We show that highly focal percepts can be evoked in the fingertips of the hand through sulcal stimulation. fMRI, myelin content, and cortical thickness maps from the Human Connectome Project elucidated specific cortical areas and sub-regions within S1 that evoked these focal percepts. Within-participant comparisons showed that percepts evoked by sulcal stimulation via SEEG electrodes were significantly more focal (80% less area; p = 0.02) and localized to the fingertips more often, than by gyral stimulation via HD-ECoG electrodes. Finally, sulcal locations with consistent modulation of high-frequency neural activity during mechanical tactile stimulation of the fingertips showed the same somatotopic correspondence as cortical stimulation. CONCLUSIONS: Our findings indicate minimally invasive sulcal stimulation via SEEG electrodes could be a clinically viable approach to restoring sensation.
Subject(s)
Hand , Somatosensory Cortex , Electric Stimulation , Electrocorticography , Electrodes, Implanted , Humans , TouchABSTRACT
Previous studies have proposed a variety of mechanisms by which attention influences neuronal activity. Here we investigated the mechanisms of attention in the striate cortex of monkeys performing a spatial or an object-based attention task at various stimulus contrasts and compared neuronal contrast response functions with and without attention. Our data are best described by an "additive" interaction: The influence of attention on the neuronal response is relatively independent of the stimulus contrast, at least when the stimulus has enough contrast to become visible. This shows that attention adds to the neuronal responses in a largely contrast invariant manner. These data support recent functional magnetic resonance imaging studies and suggest that feedback from higher areas exerts a constant attentional drive that is mostly task not stimulus driven.
Subject(s)
Attention/physiology , Evoked Potentials, Visual/physiology , Photic Stimulation/methods , Visual Cortex/physiology , Visual Perception/physiology , Animals , Brain Mapping , MacacaABSTRACT
Our understanding of nonlinear stimulus transformations by neural circuits is hindered by the lack of comprehensive yet interpretable computational modeling frameworks. Here, we propose a data-driven approach based on deep neural networks to directly model arbitrarily nonlinear stimulus-response mappings. Reformulating the exact function of a trained neural network as a collection of stimulus-dependent linear functions enables a locally linear receptive field interpretation of the neural network. Predicting the neural responses recorded invasively from the auditory cortex of neurosurgical patients as they listened to speech, this approach significantly improves the prediction accuracy of auditory cortical responses, particularly in nonprimary areas. Moreover, interpreting the functions learned by neural networks uncovered three distinct types of nonlinear transformations of speech that varied considerably from primary to nonprimary auditory regions. The ability of this framework to capture arbitrary stimulus-response mappings while maintaining model interpretability leads to a better understanding of cortical processing of sensory signals.
Subject(s)
Auditory Cortex/physiology , Auditory Perception/physiology , Sensory Receptor Cells/physiology , Acoustic Stimulation , Electrocorticography , Humans , Models, Neurological , Neural Networks, Computer , Nonlinear Dynamics , SpeechABSTRACT
Humans engagement in music rests on underlying elements such as the listeners' cultural background and interest in music. These factors modulate how listeners anticipate musical events, a process inducing instantaneous neural responses as the music confronts these expectations. Measuring such neural correlates would represent a direct window into high-level brain processing. Here we recorded cortical signals as participants listened to Bach melodies. We assessed the relative contributions of acoustic versus melodic components of the music to the neural signal. Melodic features included information on pitch progressions and their tempo, which were extracted from a predictive model of musical structure based on Markov chains. We related the music to brain activity with temporal response functions demonstrating, for the first time, distinct cortical encoding of pitch and note-onset expectations during naturalistic music listening. This encoding was most pronounced at response latencies up to 350 ms, and in both planum temporale and Heschl's gyrus.
Subject(s)
Auditory Perception/physiology , Music , Temporal Lobe/physiology , Acoustic Stimulation , Electroencephalography , Evoked Potentials, Auditory/physiology , Humans , Reaction TimeABSTRACT
Speech communication in real-world environments requires adaptation to changing acoustic conditions. How the human auditory cortex adapts as a new noise source appears in or disappears from the acoustic scene remain unclear. Here, we directly measured neural activity in the auditory cortex of six human subjects as they listened to speech with abruptly changing background noises. We report rapid and selective suppression of acoustic features of noise in the neural responses. This suppression results in enhanced representation and perception of speech acoustic features. The degree of adaptation to different background noises varies across neural sites and is predictable from the tuning properties and speech specificity of the sites. Moreover, adaptation to background noise is unaffected by the attentional focus of the listener. The convergence of these neural and perceptual effects reveals the intrinsic dynamic mechanisms that enable a listener to filter out irrelevant sound sources in a changing acoustic scene.
Subject(s)
Auditory Cortex/physiology , Noise , Speech Perception/physiology , Adaptation, Physiological , Adult , Attention/physiology , Drug Resistant Epilepsy/physiopathology , Drug Resistant Epilepsy/surgery , Electrocorticography , Female , Humans , Male , Speech AcousticsABSTRACT
Auditory stimulus reconstruction is a technique that finds the best approximation of the acoustic stimulus from the population of evoked neural activity. Reconstructing speech from the human auditory cortex creates the possibility of a speech neuroprosthetic to establish a direct communication with the brain and has been shown to be possible in both overt and covert conditions. However, the low quality of the reconstructed speech has severely limited the utility of this method for brain-computer interface (BCI) applications. To advance the state-of-the-art in speech neuroprosthesis, we combined the recent advances in deep learning with the latest innovations in speech synthesis technologies to reconstruct closed-set intelligible speech from the human auditory cortex. We investigated the dependence of reconstruction accuracy on linear and nonlinear (deep neural network) regression methods and the acoustic representation that is used as the target of reconstruction, including auditory spectrogram and speech synthesis parameters. In addition, we compared the reconstruction accuracy from low and high neural frequency ranges. Our results show that a deep neural network model that directly estimates the parameters of a speech synthesizer from all neural frequencies achieves the highest subjective and objective scores on a digit recognition task, improving the intelligibility by 65% over the baseline method which used linear regression to reconstruct the auditory spectrogram. These results demonstrate the efficacy of deep learning and speech synthesis algorithms for designing the next generation of speech BCI systems, which not only can restore communications for paralyzed patients but also have the potential to transform human-computer interaction technologies.
Subject(s)
Speech Intelligibility/physiology , Speech Perception/physiology , Speech/physiology , Acoustic Stimulation/methods , Algorithms , Auditory Cortex/physiology , Brain Mapping , Deep Learning , Evoked Potentials, Auditory/physiology , Humans , Neural Networks, Computer , Neural ProsthesesABSTRACT
The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception.
Subject(s)
Cerebral Cortex/physiology , Facial Recognition/physiology , Image Interpretation, Computer-Assisted , Neural Networks, Computer , Neurons/physiology , Adult , Electroencephalography , Female , Humans , Male , Middle Aged , Young AdultABSTRACT
The human auditory cortex simultaneously processes speech and determines the location of a speaker in space. Neuroimaging studies in humans have implicated core auditory areas in processing the spectrotemporal and the spatial content of sound; however, how these features are represented together is unclear. We recorded directly from human subjects implanted bilaterally with depth electrodes in core auditory areas as they listened to speech from different directions. We found local and joint selectivity to spatial and spectrotemporal speech features, where the spatial and spectrotemporal features are organized independently of each other. This representation enables successful decoding of both spatial and phonetic information. Furthermore, we found that the location of the speaker does not change the spectrotemporal tuning of the electrodes but, rather, modulates their mean response level. Our findings contribute to defining the functional organization of responses in the human auditory cortex, with implications for more accurate neurophysiological models of speech processing.
Subject(s)
Acoustic Stimulation/methods , Auditory Cortex/physiology , Auditory Perception/physiology , Humans , PhoneticsABSTRACT
Acetylcholine is a neuromodulator that shapes information processing in different cortical and subcortical areas. Cell type and location specific cholinergic receptor distributions suggest that acetylcholine in macaque striate cortex should boost feed-forward driven activity, while also reducing population excitability by increasing inhibitory tone. Studies using cholinergic agonists in anesthetized primate V1 have yielded conflicting evidence for such a proposal. Here we investigated how muscarinic or nicotinic receptor blockade affect neuronal excitability and contrast response functions in awake macaque area V1. Muscarinic or nicotinic receptor blockade caused reduced activity for all contrasts tested, without affecting the contrast where neurons reach their half maximal response (c50). The activity reduction upon muscarinic and nicotinic blockade resulted in reduced neuronal contrast sensitivity, as assessed through neurometric functions. In the majority of cells receptor blockade was best described by a response gain model (a multiplicative scaling of responses), indicating that ACh is involved in signal enhancement, not saliency filtering in macaque V1.