Electrocorticography reveals continuous auditory and visual speech tracking in temporal and occipital cortex.

During natural speech perception, humans must parse temporally continuous auditory and visual speech signals into sequences of words. However, most studies of speech perception present only single words or syllables. We used electrocorticography (subdural electrodes implanted on the brains of epileptic patients) to investigate the neural mechanisms for processing continuous audiovisual speech signals consisting of individual sentences. Using partial correlation analysis, we found that posterior superior temporal gyrus (pSTG) and medial occipital cortex tracked both the auditory and the visual speech envelopes. These same regions, as well as inferior temporal cortex, responded more strongly to a dynamic video of a talking face compared to auditory speech paired with a static face. Occipital cortex and pSTG carry temporal information about both auditory and visual speech dynamics. Visual speech tracking in pSTG may be a mechanism for enhancing perception of degraded auditory speech.

Sentence processing is modulated by the current linguistic environment and a priori information: An fMRI study.

INTRODUCTION: Words are not processed in isolation but in rich contexts that are used to modulate and facilitate language comprehension. Here, we investigate distinct neural networks underlying two types of contexts, the current linguistic environment and verb-based syntactic preferences. METHODS: We had two main manipulations. The first was the current linguistic environment, where the relative frequencies of two syntactic structures (prepositional object [PO] and double-object [DO]) would either follow everyday linguistic experience or not. The second concerned the preference toward one or the other structure depending on the verb; learned in everyday language use and stored in memory. German participants were reading PO and DO sentences in German while brain activity was measured with functional magnetic resonance imaging. RESULTS: First, the anterior cingulate cortex (ACC) showed a pattern of activation that integrated the current linguistic environment with everyday linguistic experience. When the input did not match everyday experience, the unexpected frequent structure showed higher activation in the ACC than the other conditions and more connectivity from the ACC to posterior parts of the language network. Second, verb-based surprisal of seeing a structure given a verb (PO verb preference but DO structure presentation) resulted, within the language network (left inferior frontal and left middle/superior temporal gyrus) and the precuneus, in increased activation compared to a predictable verb-structure pairing. CONCLUSION: In conclusion, (1) beyond the canonical language network, brain areas engaged in prediction and error signaling, such as the ACC, might use the statistics of syntactic structures to modulate language processing, (2) the language network is directly engaged in processing verb preferences. These two networks show distinct influences on sentence processing.

Encoding and Decoding Models in Cognitive Electrophysiology.

Cognitive neuroscience has seen rapid growth in the size and complexity of data recorded from the human brain as well as in the computational tools available to analyze this data. This data explosion has resulted in an increased use of multivariate, model-based methods for asking neuroscience questions, allowing scientists to investigate multiple hypotheses with a single dataset, to use complex, time-varying stimuli, and to study the human brain under more naturalistic conditions. These tools come in the form of "Encoding" models, in which stimulus features are used to model brain activity, and "Decoding" models, in which neural features are used to generated a stimulus output. Here we review the current state of encoding and decoding models in cognitive electrophysiology and provide a practical guide toward conducting experiments and analyses in this emerging field. Our examples focus on using linear models in the study of human language and audition. We show how to calculate auditory receptive fields from natural sounds as well as how to decode neural recordings to predict speech. The paper aims to be a useful tutorial to these approaches, and a practical introduction to using machine learning and applied statistics to build models of neural activity. The data analytic approaches we discuss may also be applied to other sensory modalities, motor systems, and cognitive systems, and we cover some examples in these areas. In addition, a collection of Jupyter notebooks is publicly available as a complement to the material covered in this paper, providing code examples and tutorials for predictive modeling in python. The aim is to provide a practical understanding of predictive modeling of human brain data and to propose best-practices in conducting these analyses.

Inferior-frontal cortex phase synchronizes with the temporal-parietal junction prior to successful change detection.

The inferior frontal gyrus (IFG) and the temporo-parietal junction (TPJ) are believed to be core structures of human brain networks that activate when sensory top-down expectancies guide goal directed behavior and attentive perception. But it is unclear how activity in IFG and TPJ coordinates during attention demanding tasks and whether functional interactions between both structures are related to successful attentional performance. Here, we tested these questions in electrocorticographic (ECoG) recordings in human subjects using a visual detection task that required sustained attentional expectancy in order to detect non-salient, near-threshold visual events. We found that during sustained attention the successful visual detection was predicted by increased phase synchronization of band-limited 15-30 Hz beta band activity that was absent prior to misses. Increased beta-band phase alignment during attentional engagement early during the task was restricted to inferior and lateral prefrontal cortex, but with sustained attention it extended to long-range IFG-TPJ phase synchronization and included superior prefrontal areas. In addition to beta, a widely distributed network of brain areas comprising the occipital cortex showed enhanced and reduced alpha band phase synchronization before correct detections. These findings identify long-range phase synchrony in the 15-30 Hz beta band as the mesoscale brain signal that predicts the successful deployment of attentional expectancy of sensory events. We speculate that localized beta coherent states in prefrontal cortex index 'top-down' sensory expectancy whose coupling with TPJ subregions facilitates the gating of relevant visual information.

Saccades during visual exploration align hippocampal 3-8 Hz rhythms in human and non-human primates.

Visual exploration in primates depends on saccadic eye movements (SEMs) that cause alternations of neural suppression and enhancement. This modulation extends beyond retinotopic areas, and is thought to facilitate perception; yet saccades may also influence brain regions critical for forming memories of these exploratory episodes. The hippocampus, for example, shows oscillatory activity that is generally associated with encoding of information. Whether or how hippocampal oscillations are influenced by eye movements is unknown. We recorded the neural activity in the human and macaque hippocampus during visual scene search. Across species, SEMs were associated with a time-limited alignment of a low-frequency (3-8 Hz) rhythm. The phase alignment depended on the task and not only on eye movements per se, and the frequency band was not a direct consequence of saccade rate. Hippocampal theta-frequency oscillations are produced by other mammals during repetitive exploratory behaviors, including whisking, sniffing, echolocation, and locomotion. The present results may reflect a similar yet distinct primate homologue supporting active perception during exploration.

Enhancing the signal of corticomuscular coherence.

The availability of multichannel neuroimaging techniques, such as MEG and EEG, provides us with detailed topographical information of the recorded magnetic and electric signals and therefore gives us a good overview on the concomitant signals generated in the brain. To assess the location and the temporal dynamics of neuronal sources with noninvasive recordings, reconstruction tools such as beamformers have been shown to be useful. In the current study, we are in particular interested in cortical motor control involved in the isometric contraction of finger muscles. To this end we are measuring the interaction between the dynamics of brain signals and the electrical activity of hand muscles. We were interested to find out whether in addition to the well-known correlated activity between contralateral primary motor cortex and the hand muscles, additional functional connections can be demonstrated. We adopted coherence as a functional index and propose a so-called nulling beamformer method which is computationally efficient and addresses the localization of multiple correlated sources. In simulations of cortico-motor coherence, the proposed method was able to correctly localize secondary sources. The application of the approach on real electromyographic and magnetoencephalographic data collected during an isometric contraction and rest revealed an additional activity in the hemisphere ipsilateral to the hand involved in the task.