Your browser doesn't support javascript.
loading
Mostrar: 20 | 50 | 100
Resultados 1 - 20 de 40
Filtrar
1.
Proc Natl Acad Sci U S A ; 120(42): e2300255120, 2023 10 17.
Artigo em Inglês | MEDLINE | ID: mdl-37819985

RESUMO

Speech production is a complex human function requiring continuous feedforward commands together with reafferent feedback processing. These processes are carried out by distinct frontal and temporal cortical networks, but the degree and timing of their recruitment and dynamics remain poorly understood. We present a deep learning architecture that translates neural signals recorded directly from the cortex to an interpretable representational space that can reconstruct speech. We leverage learned decoding networks to disentangle feedforward vs. feedback processing. Unlike prevailing models, we find a mixed cortical architecture in which frontal and temporal networks each process both feedforward and feedback information in tandem. We elucidate the timing of feedforward and feedback-related processing by quantifying the derived receptive fields. Our approach provides evidence for a surprisingly mixed cortical architecture of speech circuitry together with decoding advances that have important implications for neural prosthetics.


Assuntos
Fala , Lobo Temporal , Humanos , Retroalimentação , Estimulação Acústica
2.
PLoS Biol ; 20(2): e3001493, 2022 02.
Artigo em Inglês | MEDLINE | ID: mdl-35113857

RESUMO

Hearing one's own voice is critical for fluent speech production as it allows for the detection and correction of vocalization errors in real time. This behavior known as the auditory feedback control of speech is impaired in various neurological disorders ranging from stuttering to aphasia; however, the underlying neural mechanisms are still poorly understood. Computational models of speech motor control suggest that, during speech production, the brain uses an efference copy of the motor command to generate an internal estimate of the speech output. When actual feedback differs from this internal estimate, an error signal is generated to correct the internal estimate and update necessary motor commands to produce intended speech. We were able to localize the auditory error signal using electrocorticographic recordings from neurosurgical participants during a delayed auditory feedback (DAF) paradigm. In this task, participants hear their voice with a time delay as they produced words and sentences (similar to an echo on a conference call), which is well known to disrupt fluency by causing slow and stutter-like speech in humans. We observed a significant response enhancement in auditory cortex that scaled with the duration of feedback delay, indicating an auditory speech error signal. Immediately following auditory cortex, dorsal precentral gyrus (dPreCG), a region that has not been implicated in auditory feedback processing before, exhibited a markedly similar response enhancement, suggesting a tight coupling between the 2 regions. Critically, response enhancement in dPreCG occurred only during articulation of long utterances due to a continuous mismatch between produced speech and reafferent feedback. These results suggest that dPreCG plays an essential role in processing auditory error signals during speech production to maintain fluency.


Assuntos
Córtex Auditivo/fisiologia , Percepção Auditiva/fisiologia , Retroalimentação Sensorial/fisiologia , Percepção da Fala/fisiologia , Adulto , Eletrocorticografia , Epilepsia/cirurgia , Feminino , Humanos , Masculino , Córtex Motor/fisiologia , Fala/fisiologia
3.
PLoS Comput Biol ; 20(5): e1012161, 2024 May.
Artigo em Inglês | MEDLINE | ID: mdl-38815000

RESUMO

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses between lower and higher visual brain areas and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.


Assuntos
Adaptação Fisiológica , Modelos Neurológicos , Córtex Visual , Humanos , Córtex Visual/fisiologia , Adaptação Fisiológica/fisiologia , Adulto , Masculino , Feminino , Estimulação Luminosa , Biologia Computacional , Adulto Jovem , Eletroencefalografia , Percepção Visual/fisiologia , Eletrocorticografia
4.
J Neurosci ; 42(40): 7562-7580, 2022 10 05.
Artigo em Inglês | MEDLINE | ID: mdl-35999054

RESUMO

Neural responses to visual stimuli exhibit complex temporal dynamics, including subadditive temporal summation, response reduction with repeated or sustained stimuli (adaptation), and slower dynamics at low contrast. These phenomena are often studied independently. Here, we demonstrate these phenomena within the same experiment and model the underlying neural computations with a single computational model. We extracted time-varying responses from electrocorticographic recordings from patients presented with stimuli that varied in duration, interstimulus interval (ISI) and contrast. Aggregating data across patients from both sexes yielded 98 electrodes with robust visual responses, covering both earlier (V1-V3) and higher-order (V3a/b, LO, TO, IPS) retinotopic maps. In all regions, the temporal dynamics of neural responses exhibit several nonlinear features. Peak response amplitude saturates with high contrast and longer stimulus durations, the response to a second stimulus is suppressed for short ISIs and recovers for longer ISIs, and response latency decreases with increasing contrast. These features are accurately captured by a computational model composed of a small set of canonical neuronal operations, that is, linear filtering, rectification, exponentiation, and a delayed divisive normalization. We find that an increased normalization term captures both contrast- and adaptation-related response reductions, suggesting potentially shared underlying mechanisms. We additionally demonstrate both changes and invariance in temporal response dynamics between earlier and higher-order visual areas. Together, our results reveal the presence of a wide range of temporal and contrast-dependent neuronal dynamics in the human visual cortex and demonstrate that a simple model captures these dynamics at millisecond resolution.SIGNIFICANCE STATEMENT Sensory inputs and neural responses change continuously over time. It is especially challenging to understand a system that has both dynamic inputs and outputs. Here, we use a computational modeling approach that specifies computations to convert a time-varying input stimulus to a neural response time course, and we use this to predict neural activity measured in the human visual cortex. We show that this computational model predicts a wide variety of complex neural response shapes, which we induced experimentally by manipulating the duration, repetition, and contrast of visual stimuli. By comparing data and model predictions, we uncover systematic properties of temporal dynamics of neural signals, allowing us to better understand how the brain processes dynamic sensory information.


Assuntos
Encéfalo , Córtex Visual , Masculino , Feminino , Humanos , Estimulação Luminosa/métodos , Encéfalo/fisiologia , Mapeamento Encefálico/métodos , Fatores de Tempo , Córtex Visual/fisiologia
5.
Brain ; 144(5): 1590-1602, 2021 06 22.
Artigo em Inglês | MEDLINE | ID: mdl-33889945

RESUMO

We describe the spatiotemporal course of cortical high-gamma activity, hippocampal ripple activity and interictal epileptiform discharges during an associative memory task in 15 epilepsy patients undergoing invasive EEG. Successful encoding trials manifested significantly greater high-gamma activity in hippocampus and frontal regions. Successful cued recall trials manifested sustained high-gamma activity in hippocampus compared to failed responses. Hippocampal ripple rates were greater during successful encoding and retrieval trials. Interictal epileptiform discharges during encoding were associated with 15% decreased odds of remembering in hippocampus (95% confidence interval 6-23%). Hippocampal interictal epileptiform discharges during retrieval predicted 25% decreased odds of remembering (15-33%). Odds of remembering were reduced by 25-52% if interictal epileptiform discharges occurred during the 500-2000 ms window of encoding or by 41% during retrieval. During encoding and retrieval, hippocampal interictal epileptiform discharges were followed by a transient decrease in ripple rate. We hypothesize that interictal epileptiform discharges impair associative memory in a regionally and temporally specific manner by decreasing physiological hippocampal ripples necessary for effective encoding and recall. Because dynamic memory impairment arises from pathological interictal epileptiform discharge events competing with physiological ripples, interictal epileptiform discharges represent a promising therapeutic target for memory remediation in patients with epilepsy.


Assuntos
Epilepsia/fisiopatologia , Hipocampo/fisiopatologia , Rememoração Mental/fisiologia , Adolescente , Adulto , Eletrocorticografia , Epilepsia/complicações , Feminino , Humanos , Masculino , Transtornos da Memória/etiologia , Transtornos da Memória/fisiopatologia , Pessoa de Meia-Idade , Adulto Jovem
6.
Brain ; 142(11): 3502-3513, 2019 Nov 01.
Artigo em Inglês | MEDLINE | ID: mdl-31501850

RESUMO

Dynamic interactions between remote but functionally specialized brain regions enable complex information processing. This intercortical communication is disrupted in the neural networks of patients with focal epilepsy, and epileptic activity can exert widespread effects within the brain. Using large-scale human intracranial electroencephalography recordings, we show that interictal epileptiform discharges (IEDs) are significantly coupled with spindles in discrete, individualized brain regions outside of the epileptic network. We found that a substantial proportion of these localized spindles travel across the cortical surface. Brain regions that participate in this IED-driven oscillatory coupling express spindles that have a broader spatial extent and higher tendency to propagate than spindles occurring in uncoupled regions. These altered spatiotemporal oscillatory properties identify areas that are shaped by epileptic activity independent of IED or seizure detection. Our findings suggest that IED-spindle coupling may be an important mechanism of interictal global network dysfunction that could be targeted to prevent disruption of normal neural activity.


Assuntos
Epilepsia/fisiopatologia , Convulsões/fisiopatologia , Adulto , Algoritmos , Encéfalo , Mapeamento Encefálico , Córtex Cerebral/fisiopatologia , Eletrocorticografia , Eletrodos , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Rede Nervosa/fisiopatologia
7.
Proc Natl Acad Sci U S A ; 112(9): 2871-5, 2015 Mar 03.
Artigo em Inglês | MEDLINE | ID: mdl-25730850

RESUMO

For over a century neuroscientists have debated the dynamics by which human cortical language networks allow words to be spoken. Although it is widely accepted that Broca's area in the left inferior frontal gyrus plays an important role in this process, it was not possible, until recently, to detail the timing of its recruitment relative to other language areas, nor how it interacts with these areas during word production. Using direct cortical surface recordings in neurosurgical patients, we studied the evolution of activity in cortical neuronal populations, as well as the Granger causal interactions between them. We found that, during the cued production of words, a temporal cascade of neural activity proceeds from sensory representations of words in temporal cortex to their corresponding articulatory gestures in motor cortex. Broca's area mediates this cascade through reciprocal interactions with temporal and frontal motor regions. Contrary to classic notions of the role of Broca's area in speech, while motor cortex is activated during spoken responses, Broca's area is surprisingly silent. Moreover, when novel strings of articulatory gestures must be produced in response to nonword stimuli, neural activity is enhanced in Broca's area, but not in motor cortex. These unique data provide evidence that Broca's area coordinates the transformation of information across large-scale cortical networks involved in spoken word production. In this role, Broca's area formulates an appropriate articulatory code to be implemented by motor cortex.


Assuntos
Área de Broca/fisiologia , Rede Nervosa/fisiologia , Fala/fisiologia , Adolescente , Adulto , Feminino , Humanos , Masculino , Córtex Motor/fisiologia
9.
PLoS Biol ; 10(1): e1001251, 2012 Jan.
Artigo em Inglês | MEDLINE | ID: mdl-22303281

RESUMO

How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.


Assuntos
Córtex Auditivo/fisiologia , Mapeamento Encefálico , Acústica da Fala , Algoritmos , Simulação por Computador , Eletrodos Implantados , Eletroencefalografia , Feminino , Humanos , Modelos Lineares , Masculino , Modelos Biológicos
10.
bioRxiv ; 2024 May 15.
Artigo em Inglês | MEDLINE | ID: mdl-38798614

RESUMO

The ability to connect the form and meaning of a concept, known as word retrieval, is fundamental to human communication. While various input modalities could lead to identical word retrieval, the exact neural dynamics supporting this convergence relevant to daily auditory discourse remain poorly understood. Here, we leveraged neurosurgical electrocorticographic (ECoG) recordings from 48 patients and dissociated two key language networks that highly overlap in time and space integral to word retrieval. Using unsupervised temporal clustering techniques, we found a semantic processing network located in the middle and inferior frontal gyri. This network was distinct from an articulatory planning network in the inferior frontal and precentral gyri, which was agnostic to input modalities. Functionally, we confirmed that the semantic processing network encodes word surprisal during sentence perception. Our findings characterize how humans integrate ongoing auditory semantic information over time, a critical linguistic function from passive comprehension to daily discourse.

11.
Clin Park Relat Disord ; 10: 100251, 2024.
Artigo em Inglês | MEDLINE | ID: mdl-38645305

RESUMO

Introduction: Given the unique natural history of GBA-related Parkinson's disease (GBA-PD) and the potential for novel treatments in this population, genetic testing prioritization for the identification of GBA-PD patients is crucial for prognostication, individualizing treatment, and stratification for clinical trials. Assessing the predictive value of certain clinical traits for the GBA-variant carrier status will help target genetic testing in clinical settings where cost and access limit its availability. Methods: In-depth clinical characterization through standardized rating scales for motor and non-motor symptoms and self-reported binomial information of a cohort of subjects with PD (n = 100) from our center and from the larger cohort of the Parkinson's Progression Marker Initiative (PPMI) was utilized to evaluate the predictive values of clinical traits for GBA variant carrier status. The model was cross-validated across the two cohorts. Results: Leveraging non-motor symptoms of PD, we established successful discrimination of GBA variants in the PPMI cohort and study cohort (AUC 0.897 and 0.738, respectively). The PPMI cohort model successfully generalized to the study cohort data using both MDS-UPDRS scores and binomial data (AUC 0.740 and 0.734, respectively) while the study cohort model did not. Conclusions: We assessed the predictive value of non-motor symptoms of PD for identifying GBA carrier status in the general PD population. These data can be used to determine a simple, clinically oriented model using either the MDS-UPDRS or subjective symptom reporting from patients. Our results can inform patient counseling about the expected carrier risk and test prioritization for the expected identification of GBA variants.

12.
bioRxiv ; 2024 Jan 17.
Artigo em Inglês | MEDLINE | ID: mdl-37745363

RESUMO

Cortical regions supporting speech production are commonly established using neuroimaging techniques in both research and clinical settings. However, for neurosurgical purposes, structural function is routinely mapped peri-operatively using direct electrocortical stimulation. While this method is the gold standard for identification of eloquent cortical regions to preserve in neurosurgical patients, there is lack of specificity of the actual underlying cognitive processes being interrupted. To address this, we propose mapping the temporal dynamics of speech arrest across peri-sylvian cortices by quantifying the latency between stimulation and speech deficits. In doing so, we are able to substantiate hypotheses about distinct region-specific functional roles (e.g., planning versus motor execution). In this retrospective observational study, we analyzed 20 patients (12 female; age range 14-43) with refractory epilepsy who underwent continuous extra-operative intracranial EEG monitoring of an automatic speech task during clinical bedside language mapping. Latency to speech arrest was calculated as time from stimulation onset to speech arrest onset, controlling for individual speech rate. Most instances of motor-based arrest (87.5% of 96 instances) were in sensorimotor cortex with mid-range latencies to speech arrest with a distributional peak at 0.47 seconds. Speech arrest occurred in numerous regions, with relatively short latencies in supramarginal gyrus (0.46 seconds), superior temporal gyrus (0.51 seconds), and middle temporal gyrus (0.54 seconds), followed by relatively long latencies in sensorimotor cortex (0.72 seconds) and especially long latencies in inferior frontal gyrus (0.95 seconds). Nonparametric testing for speech arrest revealed that region predicted latency; latencies in supramarginal gyrus and in superior temporal gyrus were shorter than in sensorimotor cortex and in inferior frontal gyrus. Sensorimotor cortex is primarily responsible for motor-based arrest. Latencies to speech arrest in supramarginal gyrus and superior temporal gyrus (and to a lesser extent middle temporal gyrus) align with latencies to motor-based arrest in sensorimotor cortex. This pattern of relatively quick cessation of speech suggests that stimulating these regions interferes with the outgoing motor execution. In contrast, the latencies to speech arrest in inferior frontal gyrus and in ventral regions of sensorimotor cortex were significantly longer than those in temporoparietal regions. Longer latencies in the more frontal areas (including inferior frontal gyrus and ventral areas of precentral gyrus and postcentral gyrus) suggest that stimulating these areas interrupts a higher-level speech production process involved in planning. These results implicate the ventral specialization of sensorimotor cortex (including both precentral and postcentral gyri) for speech planning above and beyond motor execution.

13.
bioRxiv ; 2024 Jun 20.
Artigo em Inglês | MEDLINE | ID: mdl-38948730

RESUMO

Syntax, the abstract structure of language, is a hallmark of human cognition. Despite its importance, its neural underpinnings remain obscured by inherent limitations of non-invasive brain measures and a near total focus on comprehension paradigms. Here, we address these limitations with high-resolution neurosurgical recordings (electrocorticography) and a controlled sentence production experiment. We uncover three syntactic networks that are broadly distributed across traditional language regions, but with focal concentrations in middle and inferior frontal gyri. In contrast to previous findings from comprehension studies, these networks process syntax mostly to the exclusion of words and meaning, supporting a cognitive architecture with a distinct syntactic system. Most strikingly, our data reveal an unexpected property of syntax: it is encoded independent of neural activity levels. We propose that this "low-activity coding" scheme represents a novel mechanism for encoding information, reserved for higher-order cognition more broadly.

14.
bioRxiv ; 2024 Jun 21.
Artigo em Inglês | MEDLINE | ID: mdl-38370843

RESUMO

Across the animal kingdom, neural responses in the auditory cortex are suppressed during vocalization, and humans are no exception. A common hypothesis is that suppression increases sensitivity to auditory feedback, enabling the detection of vocalization errors. This hypothesis has been previously confirmed in non-human primates, however a direct link between auditory suppression and sensitivity in human speech monitoring remains elusive. To address this issue, we obtained intracranial electroencephalography (iEEG) recordings from 35 neurosurgical participants during speech production. We first characterized the detailed topography of auditory suppression, which varied across superior temporal gyrus (STG). Next, we performed a delayed auditory feedback (DAF) task to determine whether the suppressed sites were also sensitive to auditory feedback alterations. Indeed, overlapping sites showed enhanced responses to feedback, indicating sensitivity. Importantly, there was a strong correlation between the degree of auditory suppression and feedback sensitivity, suggesting suppression might be a key mechanism that underlies speech monitoring. Further, we found that when participants produced speech with simultaneous auditory feedback, posterior STG was selectively activated if participants were engaged in a DAF paradigm, suggesting that increased attentional load can modulate auditory feedback sensitivity.

15.
Brain Commun ; 6(2): fcae053, 2024.
Artigo em Inglês | MEDLINE | ID: mdl-38505231

RESUMO

Cortical regions supporting speech production are commonly established using neuroimaging techniques in both research and clinical settings. However, for neurosurgical purposes, structural function is routinely mapped peri-operatively using direct electrocortical stimulation. While this method is the gold standard for identification of eloquent cortical regions to preserve in neurosurgical patients, there is lack of specificity of the actual underlying cognitive processes being interrupted. To address this, we propose mapping the temporal dynamics of speech arrest across peri-sylvian cortices by quantifying the latency between stimulation and speech deficits. In doing so, we are able to substantiate hypotheses about distinct region-specific functional roles (e.g. planning versus motor execution). In this retrospective observational study, we analysed 20 patients (12 female; age range 14-43) with refractory epilepsy who underwent continuous extra-operative intracranial EEG monitoring of an automatic speech task during clinical bedside language mapping. Latency to speech arrest was calculated as time from stimulation onset to speech arrest onset, controlling for individual speech rate. Most instances of motor-based arrest (87.5% of 96 instances) were in sensorimotor cortex with mid-range latencies to speech arrest with a distributional peak at 0.47 s. Speech arrest occurred in numerous regions, with relatively short latencies in supramarginal gyrus (0.46 s), superior temporal gyrus (0.51 s) and middle temporal gyrus (0.54 s), followed by relatively long latencies in sensorimotor cortex (0.72 s) and especially long latencies in inferior frontal gyrus (0.95 s). Non-parametric testing for speech arrest revealed that region predicted latency; latencies in supramarginal gyrus and in superior temporal gyrus were shorter than in sensorimotor cortex and in inferior frontal gyrus. Sensorimotor cortex is primarily responsible for motor-based arrest. Latencies to speech arrest in supramarginal gyrus and superior temporal gyrus (and to a lesser extent middle temporal gyrus) align with latencies to motor-based arrest in sensorimotor cortex. This pattern of relatively quick cessation of speech suggests that stimulating these regions interferes with the outgoing motor execution. In contrast, the latencies to speech arrest in inferior frontal gyrus and in ventral regions of sensorimotor cortex were significantly longer than those in temporoparietal regions. Longer latencies in the more frontal areas (including inferior frontal gyrus and ventral areas of precentral gyrus and postcentral gyrus) suggest that stimulating these areas interrupts a higher-level speech production process involved in planning. These results implicate the ventral specialization of sensorimotor cortex (including both precentral and postcentral gyri) for speech planning above and beyond motor execution.

16.
bioRxiv ; 2024 Feb 27.
Artigo em Inglês | MEDLINE | ID: mdl-37745548

RESUMO

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses across the human visual hierarchy and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

17.
bioRxiv ; 2024 Mar 14.
Artigo em Inglês | MEDLINE | ID: mdl-38559163

RESUMO

Objective: This study investigates speech decoding from neural signals captured by intracranial electrodes. Most prior works can only work with electrodes on a 2D grid (i.e., Electrocorticographic or ECoG array) and data from a single patient. We aim to design a deep-learning model architecture that can accommodate both surface (ECoG) and depth (stereotactic EEG or sEEG) electrodes. The architecture should allow training on data from multiple participants with large variability in electrode placements and the trained model should perform well on participants unseen during training. Approach: We propose a novel transformer-based model architecture named SwinTW that can work with arbitrarily positioned electrodes, by leveraging their 3D locations on the cortex rather than their positions on a 2D grid. We train both subject-specific models using data from a single participant as well as multi-patient models exploiting data from multiple participants. Main Results: The subject-specific models using only low-density 8x8 ECoG data achieved high decoding Pearson Correlation Coefficient with ground truth spectrogram (PCC=0.817), over N=43 participants, outperforming our prior convolutional ResNet model and the 3D Swin transformer model. Incorporating additional strip, depth, and grid electrodes available in each participant (N=39) led to further improvement (PCC=0.838). For participants with only sEEG electrodes (N=9), subject-specific models still enjoy comparable performance with an average PCC=0.798. The multi-subject models achieved high performance on unseen participants, with an average PCC=0.765 in leave-one-out cross-validation. Significance: The proposed SwinTW decoder enables future speech neuroprostheses to utilize any electrode placement that is clinically optimal or feasible for a particular participant, including using only depth electrodes, which are more routinely implanted in chronic neurosurgical procedures. Importantly, the generalizability of the multi-patient models suggests the exciting possibility of developing speech neuroprostheses for people with speech disability without relying on their own neural data for training, which is not always feasible.

18.
Nat Commun ; 15(1): 2768, 2024 Mar 30.
Artigo em Inglês | MEDLINE | ID: mdl-38553456

RESUMO

Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. We hypothesize that language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language. To test this hypothesis, we densely record the neural activity patterns in the inferior frontal gyrus (IFG) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derive a continuous vectorial representation for each word (i.e., a brain embedding) in each patient. Using stringent zero-shot mapping we demonstrate that brain embeddings in the IFG and the DLM contextual embedding space have common geometric patterns. The common geometric patterns allow us to predict the brain embedding in IFG of a given left-out word based solely on its geometrical relationship to other non-overlapping words in the podcast. Furthermore, we show that contextual embeddings capture the geometry of IFG embeddings better than static word embeddings. The continuous brain embedding space exposes a vector-based neural code for natural language processing in the human brain.


Assuntos
Encéfalo , Idioma , Humanos , Córtex Pré-Frontal , Processamento de Linguagem Natural
19.
bioRxiv ; 2023 Sep 17.
Artigo em Inglês | MEDLINE | ID: mdl-37745380

RESUMO

Decoding human speech from neural signals is essential for brain-computer interface (BCI) technologies restoring speech function in populations with neurological deficits. However, it remains a highly challenging task, compounded by the scarce availability of neural signals with corresponding speech, data complexity, and high dimensionality, and the limited publicly available source code. Here, we present a novel deep learning-based neural speech decoding framework that includes an ECoG Decoder that translates electrocorticographic (ECoG) signals from the cortex into interpretable speech parameters and a novel differentiable Speech Synthesizer that maps speech parameters to spectrograms. We develop a companion audio-to-audio auto-encoder consisting of a Speech Encoder and the same Speech Synthesizer to generate reference speech parameters to facilitate the ECoG Decoder training. This framework generates natural-sounding speech and is highly reproducible across a cohort of 48 participants. Among three neural network architectures for the ECoG Decoder, the 3D ResNet model has the best decoding performance (PCC=0.804) in predicting the original speech spectrogram, closely followed by the SWIN model (PCC=0.796). Our experimental results show that our models can decode speech with high correlation even when limited to only causal operations, which is necessary for adoption by real-time neural prostheses. We successfully decode speech in participants with either left or right hemisphere coverage, which could lead to speech prostheses in patients with speech deficits resulting from left hemisphere damage. Further, we use an occlusion analysis to identify cortical regions contributing to speech decoding across our models. Finally, we provide open-source code for our two-stage training pipeline along with associated preprocessing and visualization tools to enable reproducible research and drive research across the speech science and prostheses communities.

20.
bioRxiv ; 2023 Jul 12.
Artigo em Inglês | MEDLINE | ID: mdl-36865223

RESUMO

Neuronal oscillations at about 10 Hz, called alpha oscillations, are often thought to arise from synchronous activity across occipital cortex, reflecting general cognitive states such as arousal and alertness. However, there is also evidence that modulation of alpha oscillations in visual cortex can be spatially specific. Here, we used intracranial electrodes in human patients to measure alpha oscillations in response to visual stimuli whose location varied systematically across the visual field. We separated the alpha oscillatory power from broadband power changes. The variation in alpha oscillatory power with stimulus position was then fit by a population receptive field (pRF) model. We find that the alpha pRFs have similar center locations to pRFs estimated from broadband power (70-180 Hz), but are several times larger. The results demonstrate that alpha suppression in human visual cortex can be precisely tuned. Finally, we show how the pattern of alpha responses can explain several features of exogenous visual attention. Significance Statement: The alpha oscillation is the largest electrical signal generated by the human brain. An important question in systems neuroscience is the degree to which this oscillation reflects system-wide states and behaviors such as arousal, alertness, and attention, versus much more specific functions in the routing and processing of information. We examined alpha oscillations at high spatial precision in human patients with intracranial electrodes implanted over visual cortex. We discovered a surprisingly high spatial specificity of visually driven alpha oscillations, which we quantified with receptive field models. We further use our discoveries about properties of the alpha response to show a link between these oscillations and the spread of visual attention.

SELEÇÃO DE REFERÊNCIAS
DETALHE DA PESQUISA