The human language system, including its inferior frontal component in "Broca's area," does not support music perception.

Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.

Cross-clause planning in Nungon (Papua New Guinea): Eye-tracking evidence.

Hundreds of languages worldwide use a sentence structure known as the "clause chain," in which 20 or more clauses can be stacked to form a sentence. The Papuan language Nungon is among a subset of clause chaining languages that require "switch-reference" suffixes on nonfinal verbs in chains. These suffixes announce whether the subject of each upcoming clause will differ from the subject of the previous clause. We examine two major issues in psycholinguistics: predictive processing in comprehension, and advance planning in production. Whereas previous work on other languages has demonstrated that sentence planning can be incremental, switch-reference marking would seem to prohibit strictly incremental planning, as it requires speakers to plan the next clause before they can finish producing the current one. This suggests an intriguing possibility: planning strategies may be fundamentally different in Nungon. We used a mobile eye-tracker and solar-powered laptops in a remote village in Papua, New Guinea, to track Nungon speakers' gaze in two experiments: comprehension and production. Curiously, during comprehension, fixation data failed to find evidence that switch-reference marking is used for predictive processing. However, during production, we found evidence for advance planning of switch-reference markers, and, by extension, the subjects they presage. We propose that this degree of advance syntactic planning pushes the boundaries of what is known about sentence planning, drawing on data from a novel morpheme type in an understudied language.

Syntactic and Semantic Processing in Japanese Sentence Reading: A Research Using Functional Near-Infrared Spectroscopy (fNIRS).

In order to examine whether syntactic processing is a necessary prerequisite for semantic integration in Japanese, cortical activation was monitored while participants engaged in silent reading task. Congruous sentences (CON), semantic violation sentences (V-SEM), and syntactic violation sentences (V-SYN) were presented in the experiment. The participants' oxygenated hemoglobin concentration changes during the reading task were measured using functional near-infrared spectroscopy. The results suggest that the CON sentences did not require additional cognitive load on syntactic processing or semantic processing. The V-SEM sentences demanded great cognitive load on semantic processing. Besides, it also elicited great cognitive load on syntactic processing. The V-SYN sentences induced great cognitive load on syntactic processing, but it did not induce additional load on semantic processing. These evidence demonstrates that, in Japanese language processing, the difficultness of semantic processing could influence the difficultness of syntactic processing, while the difficultness of syntactic processing would not influence the difficultness of semantic processing. Our findings are suggestive of the possibility that in Japanese language reading, semantic processing precedes syntactic processing, or semantic processing and syntactic processing are in parallel.

The neurobiology of language beyond single words.

A hallmark of human language is that we combine lexical building blocks retrieved from memory in endless new ways. This combinatorial aspect of language is referred to as unification. Here we focus on the neurobiological infrastructure for syntactic and semantic unification. Unification is characterized by a high-speed temporal profile including both prediction and integration of retrieved lexical elements. A meta-analysis of numerous neuroimaging studies reveals a clear dorsal/ventral gradient in both left inferior frontal cortex and left posterior temporal cortex, with dorsal foci for syntactic processing and ventral foci for semantic processing. In addition to core areas for unification, further networks need to be recruited to realize language-driven communication to its full extent. One example is the theory of mind network, which allows listeners and readers to infer the intended message (speaker meaning) from the coded meaning of the linguistic utterance. This indicates that sensorimotor simulation cannot handle all of language processing.

Syntax, morphosyntax, and serial recall: How language supports short-term memory.

In the classic view of verbal short-term memory, immediate recall is achieved by maintaining phonological representations, while the influence of other linguistic information is negligible. According to language-based accounts, short-term retention of verbal material is inherently bound to language production and comprehension, thus also influenced by semantic or syntactic factors. In line with this, serial recall is better when lists are presented in a canonical word order for English rather than in a noncanonical order (e.g., when adjectives precede nouns rather than vice versa; Perham et al., 2009, Quarterly Journal of Experimental Psychology, 62[7], 1285-1293). However, in many languages, grammaticality is not exclusively determined by word order. In German, an adjective-noun sequence is grammatical only if the adjective is inflected in congruence with the noun's person, number, and grammatical gender. Therefore, we investigated whether similar effects of syntactic word order occur in German. In two modified replications of Perham et al.'s study, we presented lists of three pairs of adjectives and nouns, presented in adjective-noun or in noun-adjective order. In addition, we manipulated morphosyntactic congruence between nouns and adjectives within pairs (Exp. 1: congruently inflected vs. uninflected adjectives; Exp. 2: congruently inflected vs. incongruently inflected adjectives). Both experiments show an interaction: Word order affected recall performance only when adjectives were inflected in congruence with the corresponding noun. These findings are in line with language-based models and indicate that, in a language that determines grammaticality in an interplay of syntactic and morphosyntactic factors, word order alone is not sufficient to improve verbal short-term memory.

Effects of Syntactic Distance and Word Order on Language Processing: An Investigation Based on a Psycholinguistic Treebank of English.

We conducted a broad-coverage investigation of the effects of syntactic distance and word order on language processing against a dependency-annotated reading time corpus of English. A combined method of quantitative syntax and psycholinguistic analyses was adopted to yield converging evidence. It was found that (i) head-initial structures allow greater structural complexity, i.e., larger head-dependent distance, than head-final structures in both language comprehension and production; (ii) within the capacity limit of working memory, syntactic distance is a positive predictor of reading time for a word with a preceding head, whereas a negative predictor of reading time for a word with a following head; and (iii) at the sentence level, syntactic distance is a significant predictor of sentence reading time. These results suggest that (i) different word orders may enjoy different processing mechanisms in terms of cognitive difficulty and processes, which can be explained by an incremental language parser; and (ii) in addition to distance, word order should also be considered as a factor affecting language processing, which is an important extension to distance-based language processing models. Taken as a whole, our study paves the way for corpus-based integration of quantitative linguistic and psycholinguistic methods into understanding language processing and its underlying cognitive mechanisms.

Hierarchical syntactic processing is beyond mere associating: Functional magnetic resonance imaging evidence from a novel artificial grammar.

Grammar is central to any natural language. In the past decades, the artificial grammar of the An Bn type in which a pair of associated elements can be nested in the other pair was considered as a desirable model to mimic human language syntax without semantic interference. However, such a grammar relies on mere associating mechanisms, thus insufficient to reflect the hierarchical nature of human syntax. Here, we test how the brain imposes syntactic hierarchies according to the category relations on linearized sequences by designing a novel artificial "Hierarchical syntactic structure-building Grammar" (HG), and compare this to the An Bn grammar as a "Nested associating Grammar" (NG) based on multilevel associations. Thirty-six healthy German native speakers were randomly assigned to one of the two grammars. Both groups performed a grammaticality judgment task on auditorily presented word sequences generated by the corresponding grammar in the scanner after a successful explicit behavioral learning session. Compared to the NG group, we found that the HG group showed a (a) significantly higher involvement of Brodmann area (BA) 44 in Broca's area and the posterior superior temporal gyrus (pSTG); and (b) qualitatively distinct connectivity between the two regions. Thus, the present study demonstrates that the build-up process of syntactic hierarchies on the basis of category relations critically relies on a distinctive left-hemispheric syntactic network involving BA 44 and pSTG. This indicates that our novel artificial grammar can constitute a suitable experimental tool to investigate syntax-specific processes in the human brain.

Filling the gap in gap-filling: Long-distance dependency formation in sentence production.

In a sentence like Who does the artist think chased the chef?, the who at the beginning depends on the last bit of the sentence, chased the chef. This is an instance of a long-distance dependency. What is the nature of the cognitive process that allows speakers to produce sentences that include distant elements that form dependencies? In four experiments, speakers described drawings that elicited long-distance dependencies. Critically, speakers were sometimes primed to produce a that in sentences where that was ungrammatical due to a grammatical constraint known as the that-trace constraint (e.g.,*Who does the artist think that chased the chef). Results showed that, when primed to say an ungrammatical that, speakers were slower to start to speak. Because the that-trace constraint applies selectively to certain configurations of long-distance dependencies, this suggests that the grammatical details of the long-distance dependency are already planned before speakers start to speak the sentences involving long-distance dependencies. I propose a formal model that explains how speakers plan long-distance dependencies in advance of speaking them while also managing the cognitive pressure to speak sentences incrementally.

The sentence superiority effect in young readers.

The sentence superiority effect observed with skilled adult readers has been taken to reflect parallel processing of word identities and the rapid construction of a preliminary syntactic structure. Here we examined if such processing is already present in primary school children in Grade 3 (average age 8.9 years). Children saw sequences of four horizontally aligned words presented simultaneously for 500 ms and followed by a post-mask and post-cue indicating the position for report of one of the four words. Word identification was more accurate in grammatically correct sequences compared with ungrammatical scrambled sequences of the same words, and this sentence superiority effect did not interact with position. This replicates the pattern found in prior research with adults and suggests that parallel word processing and the associated efficiency in syntactic processing arealready in place in Grade 3. We also found that accuracy in identifying words, independently of the surrounding context, correlated with reading age. This points to efficient word-in-sequence identification as one key ingredient of the process of becoming a skilled reader.

Processing cataphors: Active antecedent search is persistent.

Cataphors precede their antecedents, so they cannot be fully interpreted until those antecedents are encountered. Some researchers propose that cataphors trigger an active search during incremental processing in which the parser predictively posits potential antecedents in upcoming syntactic positions (Kazanina et al., Journal of Memory and Language, 56[3], 384-409, 2007). One characteristic of active search is that it is persistent: If a prediction is disconfirmed in an earlier position, the parser should iteratively search later positions until the predicted element is found. Previous research has assumed, but not established, that antecedent search is persistent. In four experiments in English and Norwegian, we test this hypothesis. Two sentence completion experiments show a strong off-line preference for coreference between a fronted cataphor and the first available argument position (the main subject). When the main subject cannot be the antecedent, participants posit the antecedent in the next closest position: object position. Two self-paced reading studies demonstrate that comprehenders actively expect the antecedent of a fronted cataphor to appear in the main clause subject position, and then successively in object position if the subject does not match the cataphor in gender. Our results therefore support the claim that antecedent search is active and persistent.

Sentence production in Parkinson's disease.

While growing evidence reports changes in language use in non-demented individuals with Parkinson's disease (PD), the presence and nature of the deficits remain largely unclear. Researchers have proposed that dysfunctioning fronto-basal ganglia circuit results in impaired grammatical processes, predicting qualitatively similar language impairments between individuals with PD and agrammatic Broca's aphasia, whereas others suggest that PD is not associated with language-specific grammatical impairment. In addition, there is a paucity of research examining syntactic production in PD at the sentence-level. This study examined sentence production of individuals with PD, healthy older adults, and individuals with agrammatic Broca's aphasia. In Experiment 1, using a Cinderella story-telling task, proportion of grammatical sentences, number of embedded clauses and production of verb arguments in sentences were examined. In Experiment 2, a structured sentence elicitation task was used in which syntactic complexity of sentences (canonical vs. non-canonical word order) was systematically manipulated while minimizing demands for non-syntactic processing. Only the participants with agrammatic Broca's aphasia showed significantly impaired syntactic production in both experiments. Participants with PD did not show impaired syntactic production in either task, despite impairments in lexical retrieval, repetition of words and sentences, and speech production. These findings suggest that impaired syntactic processing may not be a core deficit underlying the changes in language use in non-demented PD. Changes in language use in PD are qualitatively different from language deficits in aphasia.

In dialogue with an avatar, language behavior is identical to dialogue with a human partner.

The use of virtual reality (VR) as a methodological tool is becoming increasingly popular in behavioral research as its flexibility allows for a wide range of applications. This new method has not been as widely accepted in the field of psycholinguistics, however, possibly due to the assumption that language processing during human-computer interactions does not accurately reflect human-human interactions. Yet at the same time there is a growing need to study human-human language interactions in a tightly controlled context, which has not been possible using existing methods. VR, however, offers experimental control over parameters that cannot be (as finely) controlled in the real world. As such, in this study we aim to show that human-computer language interaction is comparable to human-human language interaction in virtual reality. In the current study we compare participants' language behavior in a syntactic priming task with human versus computer partners: we used a human partner, a human-like avatar with human-like facial expressions and verbal behavior, and a computer-like avatar which had this humanness removed. As predicted, our study shows comparable priming effects between the human and human-like avatar suggesting that participants attributed human-like agency to the human-like avatar. Indeed, when interacting with the computer-like avatar, the priming effect was significantly decreased. This suggests that when interacting with a human-like avatar, sentence processing is comparable to interacting with a human partner. Our study therefore shows that VR is a valid platform for conducting language research and studying dialogue interactions in an ecologically valid manner.

Syntactic processing is distributed across the language system.

Language comprehension recruits an extended set of regions in the human brain. Is syntactic processing localized to a particular region or regions within this system, or is it distributed across the entire ensemble of brain regions that support high-level linguistic processing? Evidence from aphasic patients is more consistent with the latter possibility: damage to many different language regions and to white-matter tracts connecting them has been shown to lead to similar syntactic comprehension deficits. However, brain imaging investigations of syntactic processing continue to focus on particular regions within the language system, often parts of Broca's area and regions in the posterior temporal cortex. We hypothesized that, whereas the entire language system is in fact sensitive to syntactic complexity, the effects in some regions may be difficult to detect because of the overall lower response to language stimuli. Using an individual-subjects approach to localizing the language system, shown in prior work to be more sensitive than traditional group analyses, we indeed find responses to syntactic complexity throughout this system, consistent with the findings from the neuropsychological patient literature. We speculate that such distributed nature of syntactic processing could perhaps imply that syntax is inseparable from other aspects of language comprehension (e.g., lexico-semantic processing), in line with current linguistic and psycholinguistic theories and evidence. Neuroimaging investigations of syntactic processing thus need to expand their scope to include the entire system of high-level language processing regions in order to fully understand how syntax is instantiated in the human brain.

Delta, theta, beta, and gamma brain oscillations index levels of auditory sentence processing.

A growing number of studies indicate that multiple ranges of brain oscillations, especially the delta (δ, <4Hz), theta (θ, 4-8Hz), beta (ß, 13-30Hz), and gamma (γ, 30-50Hz) bands, are engaged in speech and language processing. It is not clear, however, how these oscillations relate to functional processing at different linguistic hierarchical levels. Using scalp electroencephalography (EEG), the current study tested the hypothesis that phonological and the higher-level linguistic (semantic/syntactic) organizations during auditory sentence processing are indexed by distinct EEG signatures derived from the δ, θ, ß, and γ oscillations. We analyzed specific EEG signatures while subjects listened to Mandarin speech stimuli in three different conditions in order to dissociate phonological and semantic/syntactic processing: (1) sentences comprising valid disyllabic words assembled in a valid syntactic structure (real-word condition); (2) utterances with morphologically valid syllables, but not constituting valid disyllabic words (pseudo-word condition); and (3) backward versions of the real-word and pseudo-word conditions. We tested four signatures: band power, EEG-acoustic entrainment (EAE), cross-frequency coupling (CFC), and inter-electrode renormalized partial directed coherence (rPDC). The results show significant effects of band power and EAE of δ and θ oscillations for phonological, rather than semantic/syntactic processing, indicating the importance of tracking δ- and θ-rate phonetic patterns during phonological analysis. We also found significant ß-related effects, suggesting tracking of EEG to the acoustic stimulus (high-ß EAE), memory processing (θ-low-ß CFC), and auditory-motor interactions (20-Hz rPDC) during phonological analysis. For semantic/syntactic processing, we obtained a significant effect of γ power, suggesting lexical memory retrieval or processing grammatical word categories. Based on these findings, we confirm that scalp EEG signatures relevant to δ, θ, ß, and γ oscillations can index phonological and semantic/syntactic organizations separately in auditory sentence processing, compatible with the view that phonological and higher-level linguistic processing engage distinct neural networks.

Self-Guided Reading: Touch-Based Measures of Syntactic Processing.

A novel online reading methodology termed Self-Guided Reading (SGR) is examined to determine if it can successfully detect well-studied syntactic processing behaviours. In SGR, a participant runs their finger under masked text in order to reveal a sentence. It is therefore similar to self-paced reading in presentation of stimuli, but different in the motion that the participant makes to interact with the stimuli. The phenomena of relative clause, adverb and noun phrase/sentential attachment are utilised to allow comparison to previous research that employed self-paced reading and eye-tracking. SGR was able to detect the predicted processing behaviours in all sentence types. Moreover, once design choices and task effects are accounted for, SGR was the most consistent in triggering a motor movement change at the predicted point in the sentence. Able to provide a semi-continuous reading measure at low cost, SGR should be investigated further to uncover the full potential of the method for psycholinguistic research.

It's All in the Family: Brain Asymmetry and Syntactic Processing of Word Class.

Although left-hemisphere (LH) specialization for language is often viewed as a key example of functional lateralization, there is increasing evidence that the right hemisphere (RH) can also extract meaning from words and sentences. However, the right hemisphere's ability to appreciate syntactic aspects of language remains poorly understood. In the current study, we used separable, functionally well-characterized electrophysiological indices of lexico-semantic and syntactic processes to demonstrate RH sensitivity to syntactic violations among right-handers with a strong manual preference. Critically, however, the nature of this RH sensitivity to structural information was modulated by a genetically determined factor--familial sinistrality. The right hemisphere in right-handers without left-handed family members processed syntactic violations via the words' accompanying lexico-semantic unexpectedness. In contrast, the right hemisphere in right-handers with left-handed family members could process syntactic information in a manner qualitatively similar to that of the left hemisphere.

Training-induced improvement of noncanonical sentence production does not generalize to comprehension: evidence for modality-specific processes.

The presence or absence of generalization after treatment can provide important insights into the functional relationship between cognitive processes. The aim of the present study was to investigate the relationship between the cognitive processes that underlie sentence comprehension and production in aphasia. Using data from seven participants who took part in a case-series intervention study that focused on noncanonical sentence production [Stadie et al. (2008). Unambiguous generalization effects after treatment of noncanonical sentence production in German agrammatism. Brain and Language, 104, 211-229], we identified patterns of impairments and generalization effects for the two modalities. Results showed (a) dissociations between sentence structures and modalities before treatment, (b) an absence of cross-modal generalization from production to comprehension after treatment, and (c), a co-occurrence of spared comprehension before treatment and generalization across sentence structures within production after treatment. These findings are in line with the assumption of modality-specific, but interacting, cognitive processes in sentence comprehension and production. More specifically, this interaction is assumed to be unidirectional, allowing treatment-induced improvements in production to be supported by preserved comprehension.

The association between aerobic fitness and language processing in children: implications for academic achievement.

Event-related brain potentials (ERPs) have been instrumental for discerning the relationship between children's aerobic fitness and aspects of cognition, yet language processing remains unexplored. ERPs linked to the processing of semantic information (the N400) and the analysis of language structure (the P600) were recorded from higher and lower aerobically fit children as they read normal sentences and those containing semantic or syntactic violations. Results revealed that higher fit children exhibited greater N400 amplitude and shorter latency across all sentence types, and a larger P600 effect for syntactic violations. Such findings suggest that higher fitness may be associated with a richer network of words and their meanings, and a greater ability to detect and/or repair syntactic errors. The current findings extend previous ERP research explicating the cognitive benefits associated with greater aerobic fitness in children and may have important implications for learning and academic performance.

Text Comprehension Mediates Morphological Awareness, Syntactic Processing, and Working Memory in Predicting Chinese Written Composition Performance.

The goal of the present study was to test opposing views about four issues concerning predictors of individual differences in Chinese written composition: (a) Whether morphological awareness, syntactic processing, and working memory represent distinct and measureable constructs in Chinese or are just manifestations of general language ability; (b) whether they are important predictors of Chinese written composition, and if so, the relative magnitudes and independence of their predictive relations; (c) whether observed predictive relations are mediated by text comprehension; and (d) whether these relations vary or are developmentally invariant across three years of writing development. Based on analyses of the performance of students in grades 4 (n = 246), 5 (n = 242) and 6 (n = 261), the results supported morphological awareness, syntactic processing, and working memory as distinct yet correlated abilities that made independent contributions to predicting Chinese written composition, with working memory as the strongest predictor. However, predictive relations were mediated by text comprehension. The final model accounted for approximately 75 percent of the variance in Chinese written composition. The results were largely developmentally invariant across the three grades from which participants were drawn.

Semantic and syntactic processing of emojis in sentential intermediate positions.

The current study investigated the neuro mechanisms of emoji processing as sentence predicate in written context. In the hybrid textuality which is more cognitively engaging, emojis in sentential intermediate positions were designed as either congruent or incongruent to the context. The results showed that incongruent words led to a robust N400 effect, while incongruent emojis only elicited the P600 effect. It implies that semantics and syntax of words can be separated while those of emojis seem to be integrated together. That is, when the meaning of the emoji is violated to the sentential context, its grammatical role cannot be well interpreted, especially when it is used as a key grammatical component in a sentence, such as the predicate. Thus, it shows that even though the meaning of emojis can be interpreted by readers, their syntactic and semantic functions cannot be clearly separated. In comparison with word processing, the larger amplitude with emojis in the time window of 350-500 ms shows more cognitive efforts in emoji semantic processing, possibly arising from the switch of modalities within the visual channel, that is, the multimodal cognitive load.