A hierarchy of processing complexity and timescales for natural sounds in human auditory cortex.

Efficient behavior is supported by humans' ability to rapidly recognize acoustically distinct sounds as members of a common category. Within auditory cortex, there are critical unanswered questions regarding the organization and dynamics of sound categorization. Here, we performed intracerebral recordings in the context of epilepsy surgery as 20 patient-participants listened to natural sounds. We built encoding models to predict neural responses using features of these sounds extracted from different layers within a sound-categorization deep neural network (DNN). This approach yielded highly accurate models of neural responses throughout auditory cortex. The complexity of a cortical site's representation (measured by the depth of the DNN layer that produced the best model) was closely related to its anatomical location, with shallow, middle, and deep layers of the DNN associated with core (primary auditory cortex), lateral belt, and parabelt regions, respectively. Smoothly varying gradients of representational complexity also existed within these regions, with complexity increasing along a posteromedial-to-anterolateral direction in core and lateral belt, and along posterior-to-anterior and dorsal-to-ventral dimensions in parabelt. When we estimated the time window over which each recording site integrates information, we found shorter integration windows in core relative to lateral belt and parabelt. Lastly, we found a relationship between the length of the integration window and the complexity of information processing within core (but not lateral belt or parabelt). These findings suggest hierarchies of timescales and processing complexity, and their interrelationship, represent a functional organizational principle of the auditory stream that underlies our perception of complex, abstract auditory information.

Interacting ventral temporal gradients of timescales and functional connectivity and their relationships to visual behavior.

Cortical gradients in endogenous and stimulus-evoked neurodynamic timescales, and long-range cortical interactions, provide organizational constraints to the brain and influence neural populations' roles in cognition. It is unclear how these functional gradients interrelate and which influence behavior. Here, intracranial recordings from 4,090 electrode contacts in 35 individuals map gradients of neural timescales and functional connectivity to assess their interactions along category-selective ventral temporal cortex. Endogenous and stimulus-evoked information processing timescales were not significantly correlated with one another suggesting that local neural timescales are context dependent and may arise through distinct neurophysiological mechanisms. Endogenous neural timescales correlated with functional connectivity even after removing the effects of shared anatomical gradients. Neural timescales and functional connectivity correlated with how strongly a population's activity predicted behavior in a simple visual task. These results suggest both interrelated and distinct neurophysiological processes give rise to different functional connectivity and neural timescale gradients, which together influence behavior.

A new paradigm for investigating real-world social behavior and its neural underpinnings.

Eye tracking and other behavioral measurements collected from patient-participants in their hospital rooms afford a unique opportunity to study natural behavior for basic and clinical translational research. We describe an immersive social and behavioral paradigm implemented in patients undergoing evaluation for surgical treatment of epilepsy, with electrodes implanted in the brain to determine the source of their seizures. Our studies entail collecting eye tracking with other behavioral and psychophysiological measurements from patient-participants during unscripted behavior, including social interactions with clinical staff, friends, and family in the hospital room. This approach affords a unique opportunity to study the neurobiology of natural social behavior, though it requires carefully addressing distinct logistical, technical, and ethical challenges. Collecting neurophysiological data synchronized to behavioral and psychophysiological measures helps us to study the relationship between behavior and physiology. Combining across these rich data sources while participants eat, read, converse with friends and family, etc., enables clinical-translational research aimed at understanding the participants' disorders and clinician-patient interactions, as well as basic research into natural, real-world behavior. We discuss data acquisition, quality control, annotation, and analysis pipelines that are required for our studies. We also discuss the clinical, logistical, and ethical and privacy considerations critical to working in the hospital setting.

Deep brain stimulation for parkinson's disease induces spontaneous cortical hypersynchrony in extended motor and cognitive networks.

The mechanism of action of deep brain stimulation (DBS) to the basal ganglia for Parkinson's disease remains unclear. Studies have shown that DBS decreases pathological beta hypersynchrony between the basal ganglia and motor cortex. However, little is known about DBS's effects on long range corticocortical synchronization. Here, we use machine learning combined with graph theory to compare resting-state cortical connectivity between the off and on-stimulation states and to healthy controls. We found that turning DBS on increased high beta and gamma band synchrony (26 to 50 Hz) in a cortical circuit spanning the motor, occipitoparietal, middle temporal, and prefrontal cortices. The synchrony in this network was greater in DBS on relative to both DBS off and controls, with no significant difference between DBS off and controls. Turning DBS on also increased network efficiency and strength and subnetwork modularity relative to both DBS off and controls in the beta and gamma band. Thus, unlike DBS's subcortical normalization of pathological basal ganglia activity, it introduces greater synchrony relative to healthy controls in cortical circuitry that includes both motor and non-motor systems. This increased high beta/gamma synchronization may reflect compensatory mechanisms related to DBS's clinical benefits, as well as undesirable non-motor side effects.

Multiple adjoining word- and face-selective regions in ventral temporal cortex exhibit distinct dynamics.

The map of category-selectivity in human ventral temporal cortex (VTC) provides organizational constraints to models of object recognition. One important principle is lateral-medial response biases to stimuli that are typically viewed in the center or periphery of the visual field. However, little is known about the relative temporal dynamics and location of regions that respond preferentially to stimulus classes that are centrally viewed, like the face- and word-processing networks. Here, word- and face-selective regions within VTC were mapped using intracranial recordings from 36 patients. Partially overlapping, but also anatomically dissociable patches of face- and word-selectivity were found in VTC. In addition to canonical word-selective regions along the left posterior occipitotemporal sulcus, selectivity was also located medial and anterior to face-selective regions on the fusiform gyrus at the group level and within individual male and female subjects. These regions were replicated using 7 Tesla fMRI in healthy subjects. Left hemisphere word-selective regions preceded right hemisphere responses by 125 ms, potentially reflecting the left hemisphere bias for language; with no hemispheric difference in face-selective response latency. Word-selective regions along the posterior fusiform responded first, then spread medially and laterally, then anteriorally. Face-selective responses were first seen in posterior fusiform regions bilaterally, then proceeded anteriorally from there. For both words and faces, the relative delay between regions was longer than would be predicted by purely feedforward models of visual processing. The distinct time-courses of responses across these regions, and between hemispheres, suggest a complex and dynamic functional circuit supports face and word perception.SIGNIFICANCE STATEMENT:Representations of visual objects in the human brain have been shown to be organized by several principles, including whether those objects tend to be viewed centrally or peripherally in the visual field. However, it remains unclear how regions that process objects that are viewed centrally, like words and faces, are organized relative to one another. Here, invasive and non-invasive neuroimaging suggests there is a mosaic of regions in ventral temporal cortex that respond selectively to either words or faces. These regions display differences in the strength and timing of their responses, both within and between brain hemispheres, suggesting they play different roles in perception. These results illuminate extended, bilateral, and dynamic brain pathways that support face perception and reading.

Lateralized evoked responses in parietal cortex demonstrate visual short-term memory deficits in first-episode schizophrenia.

Working memory dysfunction may be central to neurocognitive deficits in schizophrenia. Maintenance of visual information in working memory, or visual short-term memory (vSTM), is linked to general cognitive dysfunction and predicts functional outcome. Lateralized change-detection tasks afford investigation of the contralateral delay activity (CDA), a useful tool for investigating vSTM dysfunction. Previous work suggests "hyperfocusing" of attention in schizophrenia, such that CDA is increased when a single item is maintained in vSTM but reduced for multiple items. If observed early in the disease, vSTM dysfunction may be a key feature of schizophrenia or target for intervention. We investigated CDA during lateralized vSTM of one versus three items using sensor-level electroencephalography and source-level magnetoencephalography in 26 individuals at their first episode of schizophrenia-spectrum psychosis (FESz) and 26 matched healthy controls. FESz were unable to modulate CDA with increased memory load - high-load CDA was reduced and low-load CDA was increased compared to controls. Further, sources of CDA in posterior parietal cortex were reduced in FESz and indices of working memory were correlated with neurocognitive deficits and symptom severity. These results support working memory maintenance dysfunction as a central and early component to the disorder. Targeted intervention focusing on vSTM deficits may be warranted to alleviate downstream effects of this disability.

Endogenous activity modulates stimulus and circuit-specific neural tuning and predicts perceptual behavior.

Perception reflects not only sensory inputs, but also the endogenous state when these inputs enter the brain. Prior studies show that endogenous neural states influence stimulus processing through non-specific, global mechanisms, such as spontaneous fluctuations of arousal. It is unclear if endogenous activity influences circuit and stimulus-specific processing and behavior as well. Here we use intracranial recordings from 30 pre-surgical epilepsy patients to show that patterns of endogenous activity are related to the strength of trial-by-trial neural tuning in different visual category-selective neural circuits. The same aspects of the endogenous activity that relate to tuning in a particular neural circuit also correlate to behavioral reaction times only for stimuli from the category that circuit is selective for. These results suggest that endogenous activity can modulate neural tuning and influence behavior in a circuit- and stimulus-specific manner, reflecting a potential mechanism by which endogenous neural states facilitate and bias perception.

Reduced Dorsal Visual Oscillatory Activity During Working Memory Maintenance in the First-Episode Schizophrenia Spectrum.

Cognitive deficits in people with schizophrenia are among the hardest to treat and strongly predict functional outcome. The ability to maintain sensory precepts in memory over a short delay is impacted early in the progression of schizophrenia and has been linked to reliable neurophysiological markers. Yet, little is known about the mechanisms of these deficits. Here, we investigated possible neurophysiological mechanisms of impaired visual short-term memory (vSTM, aka working memory maintenance) in the first-episode schizophrenia spectrum (FESz) using magnetoencephalography (MEG). Twenty-eight FESz and 25 matched controls performed a lateralized change detection task where they were cued to selectively attend and remember colors of circles presented in either the left or right peripheral visual field over a 1 s delay. Contralateral alpha suppression (CAS) during the delay period was used to assess selective attention to cued visual hemifields held in vSTM. Delay-period CAS was compared between FESz and controls and between trials presenting one vs three items per visual hemifield. CAS in dorsal visual cortex was reduced in FESz compared to controls in high-load trials, but not low-load trials. Group differences in CAS were found beginning 100 ms after the disappearance of the memory set, suggesting deficits were not due to the initial deployment of attention to the cued visual hemifield prior to stimulus presentation. CAS was not greater for high-load vs low-load trials in FESz subjects, although this effect was prominent in controls. Further, lateralized gamma (34-40 Hz) power emerged in dorsal visual cortex prior to the onset of CAS in controls but not FESz. Gamma power in this cluster differed between groups at both high and low load. CAS deficits observed in FESz were correlated with change detection accuracy, working memory function, estimated IQ, and negative symptoms. Our results implicate deficits in CAS in trials requiring broad, but not narrow, focus of attention to spatially distributed objects maintained in vSTM in FESz, possibly due to reduced ability to broadly distribute visuospatial attention (alpha) or disruption of object-location binding (gamma) during encoding/consolidation. This early pathophysiology may shed light upon mechanisms of emerging working memory deficits that are intrinsic to schizophrenia.

Association between increased theta cordance and early response to ECT in late-life depression.

OBJECTIVES: More than half of patients with major depression who do not respond to initial antidepressants become treatment resistant (TRD), and while electroconvulsive therapy (ECT) is effective, it involves anesthesia and other medical risks that are of concern in geriatric patients. Past studies have suggested that theta cordance (TC), a correlate of cerebral metabolism measured by electroencephalography, could guide treatment decisions related to patient selection and engagement of the therapeutic target. METHODS/DESIGN: Eight patients with late-life treatment resistant depression (LL-TRD) underwent magnetoencephalography (MEG) at baseline and following seven sessions of ECT. We tested whether the mean and regional frontal cortex TC were able to differentiate early responders from nonresponders. RESULTS: Five patients whose depression severity decreased by >30% after seven sessions were considered early responders. We found no baseline differences in mean frontal TC between early responders compared with nonresponders, but early responders exhibited a significant increase in TC following ECT. Further, we found that compared with nonresponders, early responders exhibited a greater change in TC specifically within the right prefrontal cortex. CONCLUSIONS: These results support the hypothesis that increases in frontal TC are associated with antidepressant response. We expand on previous findings by showing that this change is specific to the right prefrontal cortex. Validation of this neural marker could contribute to improved ECT outcomes, by informing early clinical decisions about the acute efficacy of this treatment.

Quantitatively validating the efficacy of artifact suppression techniques to study the cortical consequences of deep brain stimulation with magnetoencephalography.

Deep brain stimulation (DBS) is an established and effective treatment for several movement disorders and is being developed to treat a host of neuropsychiatric disorders including epilepsy, chronic pain, obsessive compulsive disorder, and depression. However, the neural mechanisms through which DBS produces therapeutic benefits, and in some cases unwanted side effects, in these disorders are only partially understood. Non-invasive neuroimaging techniques that can assess the neural effects of active stimulation are important for advancing our understanding of the neural basis of DBS therapy. Magnetoencephalography (MEG) is a safe, passive imaging modality with relatively high spatiotemporal resolution, which makes it a potentially powerful method for examining the cortical network effects of DBS. However, the degree to which magnetic artifacts produced by stimulation and the associated hardware can be suppressed from MEG data, and the comparability between signals measured during DBS-on and DBS-off conditions, have not been fully quantified. The present study used machine learning methods in conjunction with a visual perception task, which should be relatively unaffected by DBS, to quantify how well neural data can be salvaged from artifact contamination introduced by DBS and how comparable DBS-on and DBS-off data are after artifact removal. Machine learning also allowed us to determine whether the spatiotemporal pattern of neural activity recorded during stimulation are comparable to those recorded when stimulation is off. The spatiotemporal patterns of visually evoked neural fields could be accurately classified in all 8 patients with DBS implants during both DBS-on and DBS-off conditions and performed comparably across those two conditions. Further, the classification accuracy for classifiers trained on the spatiotemporal patterns evoked during DBS-on trials and applied to DBS-off trials, and vice versa, were similar to that of the classifiers trained and tested on either trial type, demonstrating the comparability of these patterns across conditions. Together, these results demonstrate the ability of MEG preprocessing techniques, like temporal signal space separation, to salvage neural data from recordings contaminated with DBS artifacts and validate MEG as a powerful tool to study the cortical consequences of DBS.

Dynamic Neural Representations: An Inferential Challenge for fMRI.

Measures of brain activity with high temporal resolution have shown that the information represented in a single brain region undergoes dynamic changes on the scale of milliseconds. This dynamic process presents a unique inferential challenge to low temporal resolution neural measures, such as BOLD fMRI. Potential solutions for fMRI requiring further investigation and development are discussed.

Posterior Fusiform and Midfusiform Contribute to Distinct Stages of Facial Expression Processing.

Though the fusiform is well-established as a key node in the face perception network, its role in facial expression processing remains unclear, due to competing models and discrepant findings. To help resolve this debate, we recorded from 17 subjects with intracranial electrodes implanted in face sensitive patches of the fusiform. Multivariate classification analysis showed that facial expression information is represented in fusiform activity and in the same regions that represent identity, though with a smaller effect size. Examination of the spatiotemporal dynamics revealed a functional distinction between posterior fusiform and midfusiform expression coding, with posterior fusiform showing an early peak of facial expression sensitivity at around 180 ms after subjects viewed a face and midfusiform showing a later and extended peak between 230 and 460 ms. These results support the hypothesis that the fusiform plays a role in facial expression perception and highlight a qualitative functional distinction between processing in posterior fusiform and midfusiform, with each contributing to temporally segregated stages of expression perception.

Adolescent development of cortical oscillations: Power, phase, and support of cognitive maturation.

During adolescence, the integration of specialized functional brain networks related to cognitive control continues to increase. Slow frequency oscillations (4-10 Hz) have been shown to support cognitive control processes, especially within prefrontal regions. However, it is unclear how neural oscillations contribute to functional brain network development and improvements in cognitive control during adolescence. To bridge this gap, we employed magnetoencephalography (MEG) to explore changes in oscillatory power and phase coupling across cortical networks in a sample of 68 adolescents and young adults. We found a redistribution of power from lower to higher frequencies throughout adolescence, such that delta band (1-3 Hz) power decreased, whereas beta band power (14-16 and 22-26 Hz) increased. Delta band power decreased with age most strongly in association networks within the frontal lobe and operculum. Conversely, beta band power increased throughout development, most strongly in processing networks and the posterior cingulate cortex, a hub of the default mode (DM) network. In terms of phase, theta band (5-9 Hz) phase-locking robustly decreased with development, following an anterior-to-posterior gradient, with the greatest decoupling occurring between association networks. Additionally, decreased slow frequency phase-locking between frontolimbic regions was related to decreased impulsivity with age. Thus, greater decoupling of slow frequency oscillations may afford functional networks greater flexibility during the resting state to instantiate control when required.

The Common Time Course of Memory Processes Revealed.

Magnetoencephalography (MEG) was used to compare memory processes in two experiments, one involving recognition of word pairs and the other involving recall of newly learned arithmetic facts. A combination of hidden semi-Markov models and multivariate pattern analysis was used to locate brief "bumps" in the sensor data that marked the onset of different stages of cognitive processing. These bumps identified a separation between a retrieval stage that identified relevant information in memory and a decision stage that determined what response was implied by that information. The encoding, retrieval, decision, and response stages displayed striking similarities across the two experiments in their duration and brain activation patterns. Retrieval and decision processes involve distinct brain activation patterns. We conclude that memory processes for two different tasks, associative recognition versus arithmetic retrieval, follow a common spatiotemporal neural pattern and that both tasks have distinct retrieval and decision stages.

Intrinsic frequency biases and profiles across human cortex.

Recent findings in monkeys suggest that intrinsic periodic spiking activity in selective cortical areas occurs at timescales that follow a sensory or lower order-to-higher order processing hierarchy (Murray JD, Bernacchia A, Freedman DJ, Romo R, Wallis JD, Cai X, Padoa-Schioppa C, Pasternak T, Seo H, Lee D, Wang XJ. Nat Neurosci 17: 1661-1663, 2014). It has not yet been fully explored if a similar timescale hierarchy is present in humans. Additionally, these measures in the monkey studies have not addressed findings that rhythmic activity within a brain area can occur at multiple frequencies. In this study we investigate in humans if regions may be biased toward particular frequencies of intrinsic activity and if a full cortical mapping still reveals an organization that follows this hierarchy. We examined the spectral power in multiple frequency bands (0.5-150 Hz) from task-independent data using magnetoencephalography (MEG). We compared standardized power across bands to find regional frequency biases. Our results demonstrate a mix of lower and higher frequency biases across sensory and higher order regions. Thus they suggest a more complex cortical organization that does not simply follow this hierarchy. Additionally, some regions do not display a bias for a single band, and a data-driven clustering analysis reveals a regional organization with high standardized power in multiple bands. Specifically, theta and beta are both high in dorsal frontal cortex, whereas delta and gamma are high in ventral frontal cortex and temporal cortex. Occipital and parietal regions are biased more narrowly toward alpha power, and ventral temporal lobe displays specific biases toward gamma. Thus intrinsic rhythmic neural activity displays a regional organization but one that is not necessarily hierarchical.NEW & NOTEWORTHY The organization of rhythmic neural activity is not well understood. Whereas it has been postulated that rhythms are organized in a hierarchical manner across brain regions, our novel analysis allows comparison of full cortical maps across different frequency bands, which demonstrate that the rhythmic organization is more complex. Additionally, data-driven methods show that rhythms of multiple frequencies or timescales occur within a particular region and that this nonhierarchical organization is widespread.

Multi-Connection Pattern Analysis: Decoding the representational content of neural communication.

The lack of multivariate methods for decoding the representational content of interregional neural communication has left it difficult to know what information is represented in distributed brain circuit interactions. Here we present Multi-Connection Pattern Analysis (MCPA), which works by learning mappings between the activity patterns of the populations as a factor of the information being processed. These maps are used to predict the activity from one neural population based on the activity from the other population. Successful MCPA-based decoding indicates the involvement of distributed computational processing and provides a framework for probing the representational structure of the interaction. Simulations demonstrate the efficacy of MCPA in realistic circumstances. In addition, we demonstrate that MCPA can be applied to different signal modalities to evaluate a variety of hypothesis associated with information coding in neural communications. We apply MCPA to fMRI and human intracranial electrophysiological data to provide a proof-of-concept of the utility of this method for decoding individual natural images and faces in functional connectivity data. We further use a MCPA-based representational similarity analysis to illustrate how MCPA may be used to test computational models of information transfer among regions of the visual processing stream. Thus, MCPA can be used to assess the information represented in the coupled activity of interacting neural circuits and probe the underlying principles of information transformation between regions.

Aberrant Oscillatory Synchrony Is Biased Toward Specific Frequencies and Processing Domains in the Autistic Brain.

BACKGROUND: Prevailing theories suggest that autism spectrum disorder (ASD) results from impaired brain communication, causing aberrant synchrony among neuronal populations. However, it remains debated whether synchrony abnormalities are among local or long-range circuits, are circuit specific or are generalized, reflect hypersynchrony or reflect hyposynchrony, and are frequency band-specific or are distributed across the frequency spectrum. METHODS: To help clarify these unresolved questions, we recorded spontaneous magnetoencephalography data and used a data-driven, whole-brain analysis of frequency-specific interregional synchrony in higher-functioning adolescents and adults, with 17 ASD and 18 control subjects matched on age, IQ, and sex, and equal for motion. RESULTS: Individuals with ASD showed local hypersynchrony in the theta band (4-7 Hz) in the lateral occipitotemporal cortex. Long-range hyposynchrony was seen in the alpha band (10-13 Hz), which was most prominent in neural circuitry underpinning social processing. The magnitude of this alpha band hyposynchrony was correlated with social symptom severity. CONCLUSIONS: These results suggest that although ASD is associated with both decreased long-range synchrony and increased posterior local synchrony, with each effect limited to a specific frequency band, impairments in social functioning may be most related to decreased alpha band synchronization between critical nodes of the social processing network.

Decoding and disrupting left midfusiform gyrus activity during word reading.

The nature of the visual representation for words has been fiercely debated for over 150 y. We used direct brain stimulation, pre- and postsurgical behavioral measures, and intracranial electroencephalography to provide support for, and elaborate upon, the visual word form hypothesis. This hypothesis states that activity in the left midfusiform gyrus (lmFG) reflects visually organized information about words and word parts. In patients with electrodes placed directly in their lmFG, we found that disrupting lmFG activity through stimulation, and later surgical resection in one of the patients, led to impaired perception of whole words and letters. Furthermore, using machine-learning methods to analyze the electrophysiological data from these electrodes, we found that information contained in early lmFG activity was consistent with an orthographic similarity space. Finally, the lmFG contributed to at least two distinguishable stages of word processing, an early stage that reflects gist-level visual representation sensitive to orthographic statistics, and a later stage that reflects more precise representation sufficient for the individuation of orthographic word forms. These results provide strong support for the visual word form hypothesis and demonstrate that across time the lmFG is involved in multiple stages of orthographic representation.

Altered Gesture and Speech Production in ASD Detract from In-Person Communicative Quality.

This study disentangled the influences of language and social processing on communication in autism spectrum disorder (ASD) by examining whether gesture and speech production differs as a function of social context. The results indicate that, unlike other adolescents, adolescents with ASD did not increase their coherency and engagement in the presence of a visible listener, and that greater coherency and engagement were related to lesser social and communicative impairments. Additionally, the results indicated that adolescents with ASD produced sparser speech and fewer gestures conveying supplementary information, and that both of these effects increased in the presence of a visible listener. Together, these findings suggest that interpersonal communication deficits in ASD are driven more strongly by social processing than language processing.

Dynamic encoding of face information in the human fusiform gyrus.

Humans' ability to rapidly and accurately detect, identify and classify faces under variable conditions derives from a network of brain regions highly tuned to face information. The fusiform face area (FFA) is thought to be a computational hub for face processing; however, temporal dynamics of face information processing in FFA remains unclear. Here we use multivariate pattern classification to decode the temporal dynamics of expression-invariant face information processing using electrodes placed directly on FFA in humans. Early FFA activity (50-75 ms) contained information regarding whether participants were viewing a face. Activity between 200 and 500 ms contained expression-invariant information about which of 70 faces participants were viewing along with the individual differences in facial features and their configurations. Long-lasting (500+ms) broadband gamma frequency activity predicted task performance. These results elucidate the dynamic computational role FFA plays in multiple face processing stages and indicate what information is used in performing these visual analyses.