Predicting "When" in Discourse Engages the Human Dorsal Auditory Stream: An fMRI Study Using Naturalistic Stories.

The hierarchical organization of human cortical circuits integrates information across different timescales via temporal receptive windows, which increase in length from lower to higher levels of the cortical hierarchy (Hasson et al., 2015). A recent neurobiological model of higher-order language processing (Bornkessel-Schlesewsky et al., 2015) posits that temporal receptive windows in the dorsal auditory stream provide the basis for a hierarchically organized predictive coding architecture (Friston and Kiebel, 2009). In this stream, a nested set of internal models generates time-based ("when") predictions for upcoming input at different linguistic levels (sounds, words, sentences, discourse). Here, we used naturalistic stories to test the hypothesis that multi-sentence, discourse-level predictions are processed in the dorsal auditory stream, yielding attenuated BOLD responses for highly predicted versus less strongly predicted language input. The results were as hypothesized: discourse-related cues, such as passive voice, which effect a higher predictability of remention for a character at a later point within a story, led to attenuated BOLD responses for auditory input of high versus low predictability within the dorsal auditory stream, specifically in the inferior parietal lobule, middle frontal gyrus, and dorsal parts of the inferior frontal gyrus, among other areas. Additionally, we found effects of content-related ("what") predictions in ventral regions. These findings provide novel evidence that hierarchical predictive coding extends to discourse-level processing in natural language. Importantly, they ground language processing on a hierarchically organized predictive network, as a common underlying neurobiological basis shared with other brain functions. SIGNIFICANCE STATEMENT: Language is the most powerful communicative medium available to humans. Nevertheless, we lack an understanding of the neurobiological basis of language processing in natural contexts: it is not clear how the human brain processes linguistic input within the rich contextual environments of our everyday language experience. This fMRI study provides the first demonstration that, in natural stories, predictions concerning the probability of remention of a protagonist at a later point are processed in the dorsal auditory stream. Results are congruent with a hierarchical predictive coding architecture assuming temporal receptive windows of increasing length from auditory to higher-order cortices. Accordingly, language processing in rich contextual settings can be explained via domain-general, neurobiological mechanisms of information processing in the human brain.

Where Is the Beat? The Neural Correlates of Lexical Stress and Rhythmical Well-formedness in Auditory Story Comprehension.

While listening to continuous speech, humans process beat information to correctly identify word boundaries. The beats of language are stress patterns that are created by combining lexical (word-specific) stress patterns and the rhythm of a specific language. Sometimes, the lexical stress pattern needs to be altered to obey the rhythm of the language. This study investigated the interplay of lexical stress patterns and rhythmical well-formedness in natural speech with fMRI. Previous electrophysiological studies on cases in which a regular lexical stress pattern may be altered to obtain rhythmical well-formedness showed that even subtle rhythmic deviations are detected by the brain if attention is directed toward prosody. Here, we present a new approach to this phenomenon by having participants listen to contextually rich stories in the absence of a task targeting the manipulation. For the interaction of lexical stress and rhythmical well-formedness, we found one suprathreshold cluster localized between the cerebellum and the brain stem. For the main effect of lexical stress, we found higher BOLD responses to the retained lexical stress pattern in the bilateral SMA, bilateral postcentral gyrus, bilateral middle fontal gyrus, bilateral inferior and right superior parietal lobule, and right precuneus. These results support the view that lexical stress is processed as part of a sensorimotor network of speech comprehension. Moreover, our results connect beat processing in language to domain-independent timing perception.

Processing of false belief passages during natural story comprehension: An fMRI study.

The neural correlates of theory of mind (ToM) are typically studied using paradigms which require participants to draw explicit, task-related inferences (e.g., in the false belief task). In a natural setup, such as listening to stories, false belief mentalizing occurs incidentally as part of narrative processing. In our experiment, participants listened to auditorily presented stories with false belief passages (implicit false belief processing) and immediately after each story answered comprehension questions (explicit false belief processing), while neural responses were measured with functional magnetic resonance imaging (fMRI). All stories included (among other situations) one false belief condition and one closely matched control condition. For the implicit ToM processing, we modeled the hemodynamic response during the false belief passages in the story and compared it to the hemodynamic response during the closely matched control passages. For implicit mentalizing, we found activation in typical ToM processing regions, that is the angular gyrus (AG), superior medial frontal gyrus (SmFG), precuneus (PCUN), middle temporal gyrus (MTG) as well as in the inferior frontal gyrus (IFG) billaterally. For explicit ToM, we only found AG activation. The conjunction analysis highlighted the left AG and MTG as well as the bilateral IFG as overlapping ToM processing regions for both implicit and explicit modes. Implicit ToM processing during listening to false belief passages, recruits the left SmFG and billateral PCUN in addition to the "mentalizing network" known form explicit processing tasks.

Electrophysiological evidence for the continuous processing of linguistic categories of regular and irregular verb inflection in German.

A central question concerning word recognition is whether linguistic categories are processed in continuous or categorical ways, in particular, whether regular and irregular inflection is stored and processed by the same or by distinct systems. Here, we contribute to this issue by contrasting regular (regular stem, regular suffix) with semi-irregular (regular stem, irregular suffix) and irregular (irregular stem, irregular suffix) participle formation in a visual priming experiment on German verb inflection. We measured ERPs and RTs and manipulated the inflectional and meaning relatedness between primes and targets. Inflected verb targets (e.g., leite, "head") were preceded either by themselves, by their participle (geleitet, "headed"), by a semantically related verb in the same inflection as the target (führe, "guide") or in the participle form (geführt, "guided"), or by an unrelated verb in the same inflection (nenne, "name"). Results showed that behavioral and ERP priming effects were gradually affected by verb regularity. Regular participles produced a widely distributed frontal and parietal effect, irregular participles produced a small left parietal effect, and semi-irregular participles yielded an effect in-between these two in terms of amplitude and topography. The behavioral and ERP effects further showed that the priming because of participles differs from that because of semantic associates for all verb types. These findings argue for a single processing system that generates participle priming effects for regular, semi-irregular, and irregular verb inflection. Together, the findings provide evidence that the linguistic categories of verb inflection are processed continuously. We present a single-system model that can adequately account for such graded effects.

Where the mass counts: common cortical activation for different kinds of nonsingularity.

Typically, plural nouns are morphosyntactically marked for the number feature, whereas mass nouns are morphosyntactically singular. However, both plural count nouns and mass nouns can be semantically interpreted as nonsingular. In this study, we investigated the hypothesis that their commonality in semantic interpretation may lead to common cortical activation for these different kinds of nonsingularity. To this end, we examined brain activation patterns related to three types of nouns while participants were listening to a narrative. Processing of plural compared with singular nouns was related to increased activation in the left angular gyrus. Processing of mass nouns compared with singular count nouns was related to increased activity bilaterally in the superior temporal cortex and also in the left angular gyrus. No significant activation was observed in the direct comparison between plural and mass nouns. We conclude that the left angular gyrus, also known to be relevant for numerical cognition, is involved in the semantic interpretation of different kinds of nonsingularity.

Effects of ketamine-induced psychopathological symptoms on continuous overt rhyme fluency.

The N-methyl-D-aspartate receptor (NMDAR) has been implicated in the pathophysiology of schizophrenia. Administered to healthy individuals, a subanesthetic dose of the noncompetitive NMDAR antagonist ketamine reproduces several psychopathological symptoms commonly observed in patients with schizophrenia. In a counterbalanced, placebo-controlled, double-blind, within-participants study, fifteen healthy subjects were administered a continuous subanesthetic S-ketamine infusion while cortical activation was measured using functional magnetic resonance imaging. While being scanned, subjects performed an overt word generation task. Ketamine-induced psychopathological symptoms were assessed with the Positive and Negative Syndrome Scale (PANSS). Ketamine administration elicited effects on psychopathology, including difficulties in abstract thinking, lack of spontaneity and flow of conversation as well as formal thought disorder. On a behavioral level, verbal fluency performance was unaffected. The PANSS score for formal thought disorder positively correlated with activation measures encompassing the left superior temporal gyrus, the right middle and inferior frontal gyrus and the precuneus. Difficulty in abstract thinking was correlated with pronounced activations in prefrontal as well as in anterior cingulate regions, whereas hyperactivations in the left superior temporal gyrus were found in association with a lack of spontaneity and flow of conversation. In the absence of behavioral impairments during verbal fluency, NMDAR blocking evoked psychopathological symptoms and cortical activations in regions previously reported in schizophrenia patients. The results provide further support for the hypothesis of an NMDAR dysfunction in the pathophysiology of schizophrenia.

Prosodic Parallelism-Comparing Spoken and Written Language.

The Prosodic Parallelism hypothesis claims adjacent prosodic categories to prefer identical branching of internal adjacent constituents. According to Wiese and Speyer (2015), this preference implies feet contained in the same phonological phrase to display either binary or unary branching, but not different types of branching. The seemingly free schwa-zero alternations at the end of some words in German make it possible to test this hypothesis. The hypothesis was successfully tested by conducting a corpus study which used large-scale bodies of written German. As some open questions remain, and as it is unclear whether Prosodic Parallelism is valid for the spoken modality as well, the present study extends this inquiry to spoken German. As in the previous study, the results of a corpus analysis recruiting a variety of linguistic constructions are presented. The Prosodic Parallelism hypothesis can be demonstrated to be valid for spoken German as well as for written German. The paper thus contributes to the question whether prosodic preferences are similar between the spoken and written modes of a language. Some consequences of the results for the production of language are discussed.

Structural Principles or Frequency of Use? An ERP Experiment on the Learnability of Consonant Clusters.

Phonological knowledge of a language involves knowledge about which segments can be combined under what conditions. Languages vary in the quantity and quality of licensed combinations, in particular sequences of consonants, with Polish being a language with a large inventory of such combinations. The present paper reports on a two-session experiment in which Polish-speaking adult participants learned nonce words with final consonant clusters. The aim was to study the role of two factors which potentially play a role in the learning of phonotactic structures: the phonological principle of sonority (ordering sound segments within the syllable according to their inherent loudness) and the (non-) existence as a usage-based phenomenon. EEG responses in two different time windows (adversely to behavioral responses) show linguistic processing by native speakers of Polish to be sensitive to both distinctions, in spite of the fact that Polish is rich in sonority-violating clusters. In particular, a general learning effect in terms of an N400 effect was found which was demonstrated to be different for sonority-obeying clusters than for sonority-violating clusters. Furthermore, significant interactions of formedness and session, and of existence and session, demonstrate that both factors, the sonority principle and the frequency pattern, play a role in the learning process.

How information structure influences the processing of rhythmic irregularities: ERP evidence from German phrases.

This study explores the influence of focus and givenness on the cognitive processing of rhythmic irregularities occurring in natural speech. Previous ERP studies showed that even subtle rhythmic deviations are detected by the brain if attention is directed towards the rhythmic structure. By using question-answer pairs, it was investigated whether subtle rhythmic irregularities in form of stress clashes (two adjacent stressed syllables) and stress lapses (two adjacent unstressed syllables) are still perceived when presented in post-focus position in an answer sentence and attention is directed away from them, towards the meaning of the element in narrow focus position by the preceding wh-question. Moreover, by visually presenting the lexical-semantic input of the deviating structure in the question, the influence of rhythmical and lexical properties in these two forms of rhythmic deviations are disentangled. While words in the present stress clash condition do not deviate from lexical stress, stress lapses contain deviations from metrical and lexical stress. The data reveal an early negativity effect for stress clashes but not for stress lapses, supporting the assumption that they are processed differently. The absence of a negative component for stress lapses indicates that the metrical deviation alone is not salient enough to be registered in non-focus position. Moreover, the lack of a late positive component suggests that subtle rhythmic deviations are less perceivable and hence more acceptable when presented in non-focus position. Thus, these results show that attentional shift induced by information structure influences the degree of the processing of rhythm.

The relevance of rhythmical alternation in language processing: an ERP study on English compounds.

This study investigates the influence of rhythmic expectancies on language processing. It is assumed that language rhythm involves an alternation of strong and weak beats within a linguistic domain. Hence, in some contexts rhythmically induced stress shifts occur in order to comply with the Rhythm Rule. In English, this rule operates to prevent clashes of stressed adjacent syllables or lapses of adjacent unstressed syllables. While previous studies investigated effects on speech production and perception, this study focuses on brain responses to structures either obeying or deviating from this rule. Event-related potentials show that rhythmic regularity is relevant for language processing: rhythmic deviations evoked different ERP components reflecting the deviance from rhythmic expectancies. An N400 effect found for shifted items reflects higher costs in lexical processing due to stress deviation. The overall results disentangle lexical and rhythmical influences on language processing and complement the findings of previous studies on rhythmical processing.

The role of predictability and structure in word stress processing: an ERP study on Cairene Arabic and a cross-linguistic comparison.

This article presents neurolinguistic data on word stress perception in Cairene Arabic, in comparison to previous results on German and Turkish. The main goal is to investigate how central properties of stress systems such as predictability of stress and metrical structure are reflected in the prosodic processing of words. Cairene Arabic is a language with a regular foot-based word stress system, leading to highly predictable placement of word stress. An ERP study on Cairene Arabic is reported, in which a stress violation paradigm is used to investigate the factors predictability of stress and foot structure. The results of the experiment show that for Cairene Arabic the internal structure of prosodic words in terms of feet determines prosodic processing. This structure effect is complemented by a frequency effect for stress patterns.

The influence of rhythmic (ir)regularities on speech processing: Evidence from an ERP study on German phrases.

The present study investigates the status of rhythmic irregularities occurring in natural speech and the importance of rhythmic alternations in cognitive processing. Previous studies showed the relevance of rhythm for language processing, but there has been only little research using the method of event-related potentials to investigate this phenomenon in a natural metrical context. To this end, an experiment was conducted in which the so-called Rhythm Rule (alternation of stressed and unstressed syllables) was either met or violated by stress clashes or stress lapses which are known to occur in German. The comparison of rhythmic well-formed conditions with the conditions including rhythmic irregularities revealed biphasic EEG-patterns for rhythmically marked structures, i.e., stress clashes and lapses. The present results show that irregular but possible rhythmic variants are costly in language processing, reflected by an early negativity and an N400 in contrast to the well-formed control conditions. Supposedly, the early negativity reflects error detection in rhythmical structure and supports the view that the brain is sensitive to subtle violations of rhythmical structure. A late positive component reflects the evaluation process related to the task requirements. The study shows that subtle rhythmical deviations from the Rhythm Rule are perceived and treated differently from well-formed structures during processing, even if the deviation in question is permitted and can therefore occur in language production.

Stress "deafness" in a Language with Fixed Word Stress: An ERP Study on Polish.

The aim of the present contribution was to examine the factors influencing the prosodic processing in a language with predictable word stress. For Polish, a language with fixed penultimate stress but several well-defined exceptions, difficulties in the processing and representation of prosodic information have been reported (e.g., Peperkamp and Dupoux, 2002). The present study utilized event-related potentials (ERPs) to investigate the factors influencing prosodic processing in Polish. These factors are (i) the predictability of stress and (ii) the prosodic structure in terms of metrical feet. Polish native speakers were presented with correctly and incorrectly stressed Polish words and instructed to judge the correctness of the perceived stress patterns. For some stress violations, an early negativity was found which was interpreted as a reflection of an error-detection mechanism. In addition, exceptional stress patterns (=antepenultimate stress) and post-lexical (=initial) stress evoked a task-related positivity effect (P300) whose amplitude and latency is correlated with the degree of anomaly and deviation from an expectation. In contrast, violations involving the default (=penultimate stress) did not produce such an effect. This asymmetrical result is interpreted to reflect that Polish native speakers are less sensitive to the default pattern than to the exceptional or post-lexical patterns. Behavioral results are orthogonal to the electrophysiological results showing that Polish speakers had difficulties to reject any kind of stress violation. Thus, on a meta-linguistic level Polish speakers appeared to be stress-"deaf" for any kind of stress manipulation, whereas the neural reactions differentiate between the default and lexicalized patterns.

The role of the posterior superior temporal sulcus in the processing of unmarked transitivity.

Is it living or not? The ability to differentiate between animate and inanimate entities is of considerable value in everyday life, since it allows for the dissociation of individuals that may willfully cause an action from objects that cannot. The present fMRI study aimed to shed light on the neural correlates of animacy at a relational-interpretive level, i.e. on the role of animacy in the establishment of relations between entities that are more or less likely to cause an event and differ in their potential to act volitionally. To this end, we investigated the processing of visually presented transitive German sentences (nominative-accusative structures) in which the factors animacy and argument order were manipulated. The relations between the arguments differed in that the animate subject either acted on an inanimate object (a very natural construction in terms of transitivity) or on an animate object (resulting in a sentence deviating from an unmarked transitive structure). Participants performed an acceptability judgment task. Violations of unmarked transitivity yielded a significant activation increase within the posterior left superior temporal sulcus (pSTS), thus suggesting a specific role of this cortical region in the relational use of animacy information. This result indicates that the influence of animacy as a relational feature differs from the impact of this parameter on the word level and is in line with other neuroimaging studies showing an engagement of the pSTS when a matching between syntax and semantics is required. A comparison between object- and subject-initial conditions further revealed a robust effect of argument order in the pars opercularis of the left inferior frontal gyrus (a subregion of Broca's area), thereby replicating previous findings demonstrating a sensitivity of this region to fine-grained language-specific linearization rules.

Linguistic prominence and Broca's area: the influence of animacy as a linearization principle.

Previous neuroimaging findings suggest a sensitivity of the pars opercularis of the left inferior frontal gyrus (i.e. a core subregion of Broca's area) to a number of linguistic dependencies governing the linear sequencing of information in a sentence (e.g. subjects should precede objects; the participant role hierarchy should be respected). The present study used event-related fMRI to examine the hitherto untested hypothesis that the violation of a linearization principle that is purely semantic in nature (animate arguments should precede inanimate arguments) would also lead to increased pars opercularis activation. To this end, we manipulated the features animacy and argument order in German sentences and found a significant increase of activation in the pars opercularis for a violation of the animacy principle even when the other factors mentioned above were controlled for. This result therefore calls for a "supra-syntactic" account of pars opercularis function in the real-time understanding of sentences.

The emergence of the unmarked: a new perspective on the language-specific function of Broca's area.

A number of neuroimaging studies have implicated an involvement of Broca's area, particularly of the pars opercularis of the left inferior frontal gyrus (IFG), in the processing of complex (permuted) sentences. However, functional interpretations of this region's role range from very general (e.g., in terms of working memory) to highly specific (e.g., as supporting particular types of syntactic operations). A dissociation of these competing accounts is often impossible because in most cases, the language internal complexity of permuted sentence structures is accompanied invariably by increasing costs of a more general cognitive nature (e.g., working memory, task difficulty, and acceptability). We used functional magnetic resonance imaging to explore the precise nature of the pars opercularis activation in the processing of permuted sentences by examining the permutation of pronouns in German. Although clearly involving a permutation operation, sentences with an initial object pronoun behave like simple, subject-initial sentences (e.g., in terms of acceptability) because of a rule stating that pronouns should generally precede non-pro-nominal arguments. The results of the experiment show that in contrast to non-pro-nominal permutations, sentences with a permuted pronoun do not engender enhanced pars opercularis activation. Our findings therefore speak against both language-related working memory and transformation-based accounts of this region's role in sentence comprehension. Rather, we argue that the pars opercularis of the left IFG supports the language-specific linearization of hierarchical linguistic dependencies.