Diverging neural dynamics for syntactic structure building in naturalistic speaking and listening.

The neural correlates of sentence production are typically studied using task paradigms that differ considerably from the experience of speaking outside of an experimental setting. In this fMRI study, we aimed to gain a better understanding of syntactic processing in spontaneous production versus naturalistic comprehension in three regions of interest (BA44, BA45, and left posterior middle temporal gyrus). A group of participants (n = 16) was asked to speak about the events of an episode of a TV series in the scanner. Another group of participants (n = 36) listened to the spoken recall of a participant from the first group. To model syntactic processing, we extracted word-by-word metrics of phrase-structure building with a top-down and a bottom-up parser that make different hypotheses about the timing of structure building. While the top-down parser anticipates syntactic structure, sometimes before it is obvious to the listener, the bottom-up parser builds syntactic structure in an integratory way after all of the evidence has been presented. In comprehension, neural activity was found to be better modeled by the bottom-up parser, while in production, it was better modeled by the top-down parser. We additionally modeled structure building in production with two strategies that were developed here to make different predictions about the incrementality of structure building during speaking. We found evidence for highly incremental and anticipatory structure building in production, which was confirmed by a converging analysis of the pausing patterns in speech. Overall, this study shows the feasibility of studying the neural dynamics of spontaneous language production.

Speech Prosody Serves Temporal Prediction of Language via Contextual Entrainment.

Temporal prediction assists language comprehension. In a series of recent behavioral studies, we have shown that listeners specifically employ rhythmic modulations of prosody to estimate the duration of upcoming sentences, thereby speeding up comprehension. In the current human magnetoencephalography (MEG) study on participants of either sex, we show that the human brain achieves this function through a mechanism termed entrainment. Through entrainment, electrophysiological brain activity maintains and continues contextual rhythms beyond their offset. Our experiment combined exposure to repetitive prosodic contours with the subsequent presentation of visual sentences that either matched or mismatched the duration of the preceding contour. During exposure to prosodic contours, we observed MEG coherence with the contours, which was source-localized to right-hemispheric auditory areas. During the processing of the visual targets, activity at the frequency of the preceding contour was still detectable in the MEG; yet sources shifted to the (left) frontal cortex, in line with a functional inheritance of the rhythmic acoustic context for prediction. Strikingly, when the target sentence was shorter than expected from the preceding contour, an omission response appeared in the evoked potential record. We conclude that prosodic entrainment is a functional mechanism of temporal prediction in language comprehension. In general, acoustic rhythms appear to endow language for employing the brain's electrophysiological mechanisms of temporal prediction.

Predictive Brain Activity Shows Congruent Semantic Specificity in Language Comprehension and Production.

Sentence fragments strongly predicting a specific subsequent meaningful word elicit larger preword slow waves, prediction potentials (PPs), than unpredictive contexts. To test the current predictive processing models, 128-channel EEG data were collected from both sexes to examine whether (1) different semantic PPs are elicited in language comprehension and production and (2) whether these PPs originate from the same specific "prediction area(s)" or rather from widely distributed category-specific neuronal circuits reflecting the meaning of the predicted item. Slow waves larger after predictable than unpredictable contexts were present both before subjects heard the sentence-final word in the comprehension experiment and before they pronounced the sentence-final word in the production experiment. Crucially, cortical sources underlying the semantic PP were distributed across several cortical areas and differed between the semantic categories of the expected words. In both production and comprehension, the anticipation of animal words was reflected by sources in posterior visual areas, whereas predictable tool words were preceded by sources in the frontocentral sensorimotor cortex. For both modalities, PP size increased with higher cloze probability, thus further confirming that it reflects semantic prediction, and with shorter latencies with which participants completed sentence fragments. These results sit well with theories viewing distributed semantic category-specific circuits as the mechanistic basis of semantic prediction in the two modalities.

Naturalistic Spoken Language Comprehension Is Supported by Alpha and Beta Oscillations.

Brain oscillations are prevalent in all species and are involved in numerous perceptual operations. α oscillations are thought to facilitate processing through the inhibition of task-irrelevant networks, while ß oscillations are linked to the putative reactivation of content representations. Can the proposed functional role of α and ß oscillations be generalized from low-level operations to higher-level cognitive processes? Here we address this question focusing on naturalistic spoken language comprehension. Twenty-two (18 female) Dutch native speakers listened to stories in Dutch and French while MEG was recorded. We used dependency parsing to identify three dependency states at each word: the number of (1) newly opened dependencies, (2) dependencies that remained open, and (3) resolved dependencies. We then constructed forward models to predict α and ß power from the dependency features. Results showed that dependency features predict α and ß power in language-related regions beyond low-level linguistic features. Left temporal, fundamental language regions are involved in language comprehension in α, while frontal and parietal, higher-order language regions, and motor regions are involved in ß. Critically, α- and ß-band dynamics seem to subserve language comprehension tapping into syntactic structure building and semantic composition by providing low-level mechanistic operations for inhibition and reactivation processes. Because of the temporal similarity of the α-ß responses, their potential functional dissociation remains to be elucidated. Overall, this study sheds light on the role of α and ß oscillations during naturalistic spoken language comprehension, providing evidence for the generalizability of these dynamics from perceptual to complex linguistic processes.SIGNIFICANCE STATEMENT It remains unclear whether the proposed functional role of α and ß oscillations in perceptual and motor function is generalizable to higher-level cognitive processes, such as spoken language comprehension. We found that syntactic features predict α and ß power in language-related regions beyond low-level linguistic features when listening to naturalistic speech in a known language. We offer experimental findings that integrate a neuroscientific framework on the role of brain oscillations as "building blocks" with spoken language comprehension. This supports the view of a domain-general role of oscillations across the hierarchy of cognitive functions, from low-level sensory operations to abstract linguistic processes.

The volume and the distribution of premorbid white matter hyperintensities: Impact on post-stroke aphasia.

White matter hyperintensities (WMH) are a radiological manifestation of progressive white matter integrity loss. The total volume and distribution of WMH within the corpus callosum have been associated with pathological cognitive ageing processes but have not been considered in relation to post-stroke aphasia outcomes. We investigated the contribution of both the total volume of WMH, and the extent of WMH lesion load in the corpus callosum to the recovery of language after first-ever stroke. Behavioural and neuroimaging data from individuals (N = 37) with a left-hemisphere stroke were included at the early subacute stage of recovery. Spoken language comprehension and production abilities were assessed using word and sentence-level tasks. Neuroimaging data was used to derive stroke lesion variables (volume and lesion load to language critical regions) and WMH variables (WMH volume and lesion load to three callosal segments). WMH volume did not predict variance in language measures, when considered together with stroke lesion and demographic variables. However, WMH lesion load in the forceps minor segment of the corpus callosum explained variance in early subacute comprehension abilities (t = -2.59, p = .01) together with corrected stroke lesion volume and socio-demographic variables. Premorbid WMH lesions in the forceps minor were negatively associated with early subacute language comprehension after aphasic stroke. This negative impact of callosal WMH on language is consistent with converging evidence from pathological ageing suggesting that callosal WMH disrupt the neural networks supporting a range of cognitive functions.

The role of variation in phonological and semantic working memory capacities in sentence comprehension: neural evidence from healthy and brain-damaged individuals.

Research on the role of working memory (WM) in language processing has typically focused on WM for phonological information. However, considerable behavioral evidence supports the existence of a separate semantic WM system that plays a greater role in language processing. We review the neural evidence that supports the distinction between phonological and semantic WM capacities and discuss how individual differences in these capacities relate to sentence processing. In terms of neural substrates, findings from multivariate functional MRI for healthy participants and voxel-based lesion-symptom mapping for brain-damaged participants imply that the left supramarginal gyrus supports phonological WM, whereas the left inferior frontal gyrus (LIFG) and angular gyrus support semantic WM. In sentence comprehension, individual variation in semantic but not phonological WM related to performance in resolving semantic information and the LIFG region implicated in semantic WM showed fMRI activation during the resolution of semantic interference. Moreover, variation for brain-damaged participants in the integrity of a fiber tract supporting semantic WM had a greater relation to the processing of complex sentences than did the integrity of fiber tracts supporting phonological WM. Overall, the neural findings provide converging evidence regarding the distinction of these two capacities and the greater contribution of individual differences in semantic than phonological WM capacity to sentence processing.

Sustained pacifier use is associated with smaller vocabulary sizes at 1 and 2 years of age: A cross-sectional study.

Pacifier use during childhood has been hypothesized to interfere with language processing, but, to date, there is limited evidence revealing detrimental effects of prolonged pacifier use on infant vocabulary learning. In the present study, parents of 12- and 24-month-old infants were recruited in Oslo (Norway). The sample included 1187 monolingual full-term born (without visual, auditory, or cognitive impairments) infants: 452 (230 girls; 222 boys) 12-month-olds and 735 (345 girls; 390 boys) 24-month-olds. Parents filled out an online Norwegian Communicative Development Inventory (CDI), which assesses the vocabulary in comprehension and production for 12-month-old infants and in production only for 24-month-old infants. CDI scores were transformed into age- and sex-adjusted percentiles using Norwegian norms. Additionally, parents retrospectively reported their child's daytime pacifier use, in hours, at 2-month intervals, from birth to the assessment date. Maternal education was used to control, in the analyses, for the socio-economic status. We found that greater pacifier use in an infant's lifespan was associated with lower vocabulary size. Pacifier use later in life was more negatively associated with vocabulary size than precocious use, and increased the odds of being a low language scorer. In sum, our study moves beyond the findings of momentary effects of experimentally induced "impairment" in articulators' movement on speech perception and suggests that, from 12 months of age, constraints on the infant's speech articulators (pacifier use) may be negatively associated with word comprehension and production. RESEARCH HIGHLIGHT: We examined the relationship between pacifier use and vocabulary sizes in production at 24 months of age and comprehension and production at 12 months of age. Lifespan Pacifier Use (LPU) was negatively correlated with vocabulary sizes in comprehension and production among 12-month-old infants and negatively correlated with production for 24-month-olds. Later pacifier use was found to be more negatively correlated with vocabulary size in infants, as compared to more precocious use. The amount of pacifier use in the 2 months prior to a child's second birthday was predictive of a higher prevalence of low vocabulary scores in 24-month-olds.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production.

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

The relationship between vocabulary depth knowledge, word reading, and reading comprehension in Chinese.

Although vocabulary depth (VD) is recognized as a crucial factor in reading comprehension, the investigation of its role in reading comprehension remains insufficient. This study aimed to address two significant research gaps in this domain. First, empirical evidence is needed to explore the construct of VD knowledge, particularly within the Chinese language. Second, the underlying mechanism that connects VD and reading comprehension requires further clarification. In this study, a sample of 326 native Chinese students from Grade 4 participated in a comprehensive battery of tests assessing VD knowledge, word reading, and reading comprehension. Based on theoretical frameworks of VD knowledge, we measured six subtypes of VD knowledge: polysemy, collocation, word register, part-of-speech, semanticassociations, and homonyms. The results of factor analysis revealed that Chinese VD knowledge can be conceptualized as a two-factor construct, encompassing in-depth semantic knowledge (VD-meaning) and knowledge of word usage (VD-usage). Both VD-meaning and VD-usage demonstrated significant direct effects on reading comprehension, highlighting the critical role of VD in determining reading comprehension outcomes in Chinese. Furthermore, our findings indicated an indirect contribution of VD to reading comprehension, specifically through the mediating effect of word reading on the relationship between VD-meaning and reading comprehension. This study represents a pioneering empirical investigation that delved into the construct of VD in Chinese. In addition, we discuss the role of VD knowledge and its interaction with word reading in the context of Chinese reading comprehension, which could significantly enhance our understanding of the underlying mechanism that links vocabulary knowledge to reading comprehension.

Experiential traces first: Does holding a location in visuospatial working memory affect the processing of space-associated words?

This study aimed to systematically examine whether actively maintaining a visual location in working memory can influence the processing of spatially related words. In five experiments, we asked participants to maintain either the location or the shape of a visually presented stimulus in working memory so that it could later be compared with a test stimulus concerning the relevant target features. In between, we presented participants with words that refer to objects typically encountered in the upper or lower vertical space (roof vs. root, respectively). The task participants performed as a response to these words differed between experiments. In Experiments 1-3, participants performed a lexical decision task, in Experiment 4 they performed a semantic task (deciding whether the word refers to an occupation), and in Experiment 5 they performed a spatial task (deciding whether the word refers to something in the upper or lower visual field.) Only in Experiment 5 did we observe an interaction between the position of the visual stimulus held in working memory (up vs. down) and the meaning of the spatial words (associated with up vs. down). Our results therefore suggest that actively maintaining a stimulus location in working memory does not automatically affect the processing of spatially related words, but does so if the relevant spatial dimension is made highly salient by the task. The results are thus in line with studies showing a strong context-dependency of embodiment effects and thus allow the conclusion that language processing proper is not operating on a sensorimotor representational format.

Online revision process in clause-boundary garden-path sentences.

A long-standing question in sentence processing research concerns the online parsing process in clause-boundary garden-path sentences, such as After Mary dressed John bathed. In this sentence, "John" must be parsed as the matrix subject DP but can be locally analysed as the object of the embedded verb. There is considerable evidence that the parser misanalyses these garden-path sentences. However, the controversy lies in whether the parser revises them during the online parsing process. The present study investigated this revision process through two self-paced reading experiments utilising grammatical constraints on reflexives and subject or object relative clauses embedded within the locally ambiguous DP. The results provided evidence of revision when a subject relative clause was embedded but not when an object relative clause was embedded. These findings suggest that the parser assigns grammatical structures that correspond to input strings during the revision of clause-boundary ambiguities but that object relative clauses affect the online revision process.

Automatic mental simulation in native and non-native speakers.

Pictures of objects are verified faster when they match the implied orientation, shape, and color in a sentence-picture verification task, suggesting that people mentally simulate these features during language comprehension. Previous studies had an unintended correlation between match status and the required response, which may have influenced participants' responses by eliciting strategic use of this correlation. We removed this correlation by including color-matching filler trials and investigated if the color-match effect was still obtained. In both a native sample (Experiment 1) and a non-native sample (Experiment 2), we found strong evidence for a color-match advantage on median reaction time and error rates. Our results are consistent with the view that color is automatically simulated during language comprehension as predicted by the grounded cognition framework.

The neural architecture of language: Integrative modeling converges on predictive processing.

The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a systematic study taking this approach to higher-level cognition: human language processing, our species' signature cognitive skill. We find that the most powerful "transformer" models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (functional MRI and electrocorticography). Models' neural fits ("brain score") and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

Mental simulation of colour properties during language comprehension: influence of context and comprehension stages.

Many studies have shown that mental simulation may occur during language comprehension. Supporting evidence is derived from the matching effects in the sentence-picture verification (SPV) task often used to assess mental simulations of object properties, such as size, orientation, and shape. However, mixed results have been obtained regarding object colour, with researchers reporting matching or mismatching effects. This study investigated the impact of colour information clarity within sentences on the process of mental simulation during language comprehension. Employing the SPV task and using novel objects, we examined whether there is a mental simulation of colour after excluding typical/atypical colour bias and how varying levels of colour information clarity in sentences influence the emergence of matching effects at different stages of comprehension. To address these issues, we conducted two experiments. In Experiment 1, the participants read normal sentences and subsequently engaged in picture verification with a novel object after a 500 ms delay. In Experiment 2, the participants encountered sentences containing both clear and unclear colour information and, after either a 0 ms or 1500 ms interval, completed picture verification tasks with a novel object. Null effects were found in the 500 ms condition for normal sentences and the 0 ms condition for unclear colour information sentences. A mismatching effect appeared in the 0 ms condition after clear colour information sentences, and a matching effect appeared in the 1500 ms condition for all sentences. The results indicated that after excluding colour bias, the participants still formed mental simulations of colour during language comprehension. Our results also indicated that ongoing colour simulation with time pressure impacted the participant responses. The participants ignored unclear colour information under time pressure, but without time pressure, they constructed simulations that were as detailed as possible, regardless of whether the implicit colour information in the sentence was clear.

Assessing two methods of webcam-based eye-tracking for child language research.

We assess the feasibility of conducting web-based eye-tracking experiments with children using two methods of webcam-based eye-tracking: automatic gaze estimation with the WebGazer.js algorithm and hand annotation of gaze direction from recorded webcam videos. Experiment 1 directly compares the two methods in a visual-world language task with five to six year-old children. Experiment 2 more precisely investigates WebGazer.js' spatiotemporal resolution with four to twelve year-old children in a visual-fixation task. We find that it is possible to conduct web-based eye-tracking experiments with children in both supervised (Experiment 1) and unsupervised (Experiment 2) settings - however, the webcam eye-tracking methods differ in their sensitivity and accuracy. Webcam video annotation is well-suited to detecting fine-grained looking effects relevant to child language researchers. In contrast, WebGazer.js gaze estimates appear noisier and less temporally precise. We discuss the advantages and disadvantages of each method and provide recommendations for researchers conducting child eye-tracking studies online.

An ERP Study on the Processing of Subject-Verb and Object-Verb Gender Agreement in Punjabi.

This study was conducted with the aim of exploring the general parsing mechanisms involved in processing different kinds of dependency relations, namely verb agreement with subjects versus objects in Punjabi, an SOV Indo-Aryan language. Event related brain potentials (ERPs) were recorded as twenty-five native Punjabi speakers read transitive sentences. Critical stimuli were either fully acceptable as regards verb agreement, or alternatively violated gender agreement with the subject or object. A linear mixed-models analysis confirmed a P600 effect at the position of the verb for all violations, regardless of whether subject or object agreement was violated. These results thus suggest that an identical mechanism is involved in gender agreement computation in Punjabi regardless of whether the agreement is with the subject or the object argument.

Neural synchrony reflects closure of jabberwocky noun phrases but not predictable pseudoword sequences.

Successful language comprehension requires the combination of individual words into larger linguistic units. In the present minimal-phrase study, we used electroencephalography (EEG) to investigate whether syntactic combination is indexed by changes in neural synchrony, while testing for both token-based and type-based effects. To do this, we analysed intertrial phase coherence (ITPC) elicited by reading two item (words or pseudowords) phrases that were either unifiable or nonunifiable. Results indicated that type-based unifiable phrases elicited increased ITPC relative to all other conditions in the frequency band corresponding to the rate of phrases (0.5 Hz) but not the rate of words (1 Hz). Conversely, we observed a complementary pattern for the N400, which was more sensitive to token-based effects. These findings provide evidence that the combination of single words into larger syntactic structures may be indexed by the synchronous firing of assemblies of neurons oscillating at the rate of phrases during reading.

Electrophysiological evidence for the enhancement of gesture-speech integration by linguistic predictability during multimodal discourse comprehension.

In face-to-face discourse, listeners exploit cues in the input to generate predictions about upcoming words. Moreover, in addition to speech, speakers produce a multitude of visual signals, such as iconic gestures, which listeners readily integrate with incoming words. Previous studies have shown that processing of target words is facilitated when these are embedded in predictable compared to non-predictable discourses and when accompanied by iconic compared to meaningless gestures. In the present study, we investigated the interaction of both factors. We recorded electroencephalogram from 60 Dutch adults while they were watching videos of an actress producing short discourses. The stimuli consisted of an introductory and a target sentence; the latter contained a target noun. Depending on the preceding discourse, the target noun was either predictable or not. Each target noun was paired with an iconic gesture and a gesture that did not convey meaning. In both conditions, gesture presentation in the video was timed such that the gesture stroke slightly preceded the onset of the spoken target by 130 ms. Our ERP analyses revealed independent facilitatory effects for predictable discourses and iconic gestures. However, the interactive effect of both factors demonstrated that target processing (i.e., gesture-speech integration) was facilitated most when targets were part of predictable discourses and accompanied by an iconic gesture. Our results thus suggest a strong intertwinement of linguistic predictability and non-verbal gesture processing where listeners exploit predictive discourse cues to pre-activate verbal and non-verbal representations of upcoming target words.

Two sides of the same coin? Comparing structural priming between production and comprehension in choice data and in reaction times.

Although structural priming seems to rely on the same mechanisms in production and comprehension, effects are not always consistent between modalities. Methodological differences often result in different data types, namely choice data in production and reaction time data in comprehension. In a structural priming experiment with English ditransitives, we collected choice data and reaction time data in both modalities. The choice data showed priming of the DO and PO dative. The reaction times revealed priming of the PO dative. In production, PO targets were chosen faster after a PO prime than after a baseline prime. In comprehension, DO targets were read slower after a PO prime than after a baseline prime. This result can be explained from competition between alternatives during structure selection. Priming leads to facilitation of the primed structure or inhibition of the opposite structure depending on the relative frequency of structures, which may differ across modalities.

Preferential looking to eyes versus mouth in early infancy: heritability and link to concurrent and later development.

BACKGROUND: From birth, infants orient preferentially to faces, and when looking at the face, they attend primarily to eyes and mouth. These areas convey different types of information, and earlier research suggests that genetic factors influence the preference for one or the other in young children. METHODS: In a sample of 535 5-month-old infant twins, we assessed eye (relative to mouth) preference in early infancy, i.e., before neural systems for social communication and language are fully developed. We investigated the contribution of genetic and environmental factors to the preference for looking at eyes, and the association with concurrent traits and follow-up measures. RESULTS: Eye preference was independent from all other concurrent traits measured, and had a moderate-to-high contribution from genetic influences (A = 0.57; 95% CI: 0.45, 0.66). Preference for eyes at 5 months was associated with higher parent ratings of receptive vocabulary at 14 months. No statistically significant association with later autistic traits was found. Preference for eyes was strikingly stable across different stimulus types (e.g., dynamic vs. still), suggesting that infants' preference at this age does not reflect sensitivity to low-level visual cues. CONCLUSIONS: These results suggest that individual differences in infants' preferential looking to eyes versus mouth to a substantial degree reflect genetic variation. The findings provide new leads on both the perceptual basis and the developmental consequences of these attentional biases.