The unique contribution of uncertainty reduction during naturalistic language comprehension.

Language comprehension is an incremental process with prediction. Delineating various mental states during such a process is critical to understanding the relationship between human cognition and the properties of language. Entropy reduction, which indicates the dynamic decrease of uncertainty as language input unfolds, has been recognized as effective in predicting neural responses during comprehension. According to the entropy reduction hypothesis (Hale, 2006), entropy reduction is related to the processing difficulty of a word, the effect of which may overlap with other well-documented information-theoretical metrics such as surprisal or next-word entropy. However, the processing difficulty was often confused with the information conveyed by a word, especially lacking neural differentiation. We propose that entropy reduction represents the cognitive neural process of information gain that can be dissociated from processing difficulty. This study characterized various information-theoretical metrics using GPT-2 and identified the unique effects of entropy reduction in predicting fMRI time series acquired during language comprehension. In addition to the effects of surprisal and entropy, entropy reduction was associated with activations in the left inferior frontal gyrus, bilateral ventromedial prefrontal cortex, insula, thalamus, basal ganglia, and middle cingulate cortex. The reduction of uncertainty, rather than its fluctuation, proved to be an effective factor in modeling neural responses. The neural substrates underlying the reduction in uncertainty might imply the brain's desire for information regardless of processing difficulty.

Non-native language comprehenders encode implied shapes of objects in memory.

Barsalou (1999) proposes that conceptual knowledge is represented by mental simulations containing perceptual information derived from actual experiences. Although a substantial number of studies have provided evidence consistent with this view in native language comprehension, it remains unclear whether the non-native language comprehension processes also include mental simulations. The current study successfully replicates the shape match effect in sentence-picture verification (Zwaan et al., 2002) for non-native English language comprehenders, indicating native-like visual simulations. In addition, participants displayed better delayed recognition memory when the shape of the depicted objects matched the shape that was implied by the sentence than when it did not, suggesting that visual simulations were generated spontaneously in naturalistic non-native language comprehension. Additional correlational analyses revealed no relationship between English proficiency and the size of the match effect.

Online Eye Tracking for Aphasia: A Feasibility Study Comparing Web and Lab Tracking and Implications for Clinical Use.

BACKGROUND & AIMS: Studies using eye-tracking methodology have made important contributions to the study of language disorders such as aphasia. Nevertheless, in clinical groups especially, eye-tracking studies often include small sample sizes, limiting the generalizability of reported findings. Online, webcam-based tracking offers a potential solution to this issue, but web-based tracking has not been compared with in-lab tracking in past studies and has never been attempted in groups with language impairments. MATERIALS & METHODS: Patients with post-stroke aphasia (n = 16) and age-matched controls (n = 16) completed identical sentence-picture matching tasks in the lab (using an EyeLink system) and on the web (using WebGazer.js), with the order of sessions counterbalanced. We examined whether web-based eye tracking is as sensitive as in-lab eye tracking in detecting group differences in sentence processing. RESULTS: Patients were less accurate and slower to respond to all sentence types than controls. Proportions of gazes to the target and foil picture were computed in 100 ms increments, which showed that the two modes of tracking were comparably sensitive to overall group differences across different sentence types. Web tracking showed comparable fluctuations in gaze proportions to target pictures to lab tracking in most analyses, whereas a delay of approximately 500-800 ms appeared in web compared to lab data. DISCUSSION & CONCLUSIONS: Web-based eye tracking is feasible to study impaired language processing in aphasia and is sensitive enough to detect most group differences between controls and patients. Given that validations of webcam-based tracking are in their infancy and how transformative this method could be to several disciplines, much more testing is warranted.

From Sound to Meaning: Navigating Wernicke's Area in Language Processing.

Wernicke's area, a critical brain region associated with language comprehension, was first identified by Carl Wernicke in the late 19th century. Situated in the left hemisphere's posterior superior temporal gyrus, this area is essential for processing auditory and visual language inputs. It integrates semantic and syntactic information, playing a key role in meaningful communication. The development of Wernicke's area during infancy and childhood is marked by rapid growth and refinement influenced by early language exposure and environmental stimuli. Neuroplasticity, the brain's ability to reorganize and adapt, is crucial for recovery from language impairments such as Wernicke's aphasia. This capacity for reorganization includes synaptic plasticity and axonal sprouting, which facilitate recovery through targeted rehabilitation and enriched environments. Recent research utilizing advanced neuroimaging and neuroanatomical tracing techniques has elucidated the connectivity of Wernicke's area with other language-related regions, such as Broca's area. Functional studies have revealed its specialized roles in processing different aspects of language, including phonological, semantic, and syntactic features. Moreover, investigations into language disorders and potential therapeutic interventions underscore the importance of harnessing neuroplasticity for effective treatment. Emerging technologies, such as non-invasive brain stimulation and multimodal imaging, offer promising avenues for further exploration of Wernicke's area and its role in language functions. These innovations hold the potential to enhance our understanding of language processing and improve therapeutic strategies for language impairments. In conclusion, Wernicke's area is central to language comprehension, and genetic and environmental factors influence its development. Understanding neuroplasticity and leveraging advanced research technologies can significantly advance our ability to address language-related disorders and enhance patient outcomes.

The role of variation in phonological and semantic working memory capacities in sentence comprehension: neural evidence from healthy and brain-damaged individuals.

Research on the role of working memory (WM) in language processing has typically focused on WM for phonological information. However, considerable behavioral evidence supports the existence of a separate semantic WM system that plays a greater role in language processing. We review the neural evidence that supports the distinction between phonological and semantic WM capacities and discuss how individual differences in these capacities relate to sentence processing. In terms of neural substrates, findings from multivariate functional MRI for healthy participants and voxel-based lesion-symptom mapping for brain-damaged participants imply that the left supramarginal gyrus supports phonological WM, whereas the left inferior frontal gyrus (LIFG) and angular gyrus support semantic WM. In sentence comprehension, individual variation in semantic but not phonological WM related to performance in resolving semantic information and the LIFG region implicated in semantic WM showed fMRI activation during the resolution of semantic interference. Moreover, variation for brain-damaged participants in the integrity of a fiber tract supporting semantic WM had a greater relation to the processing of complex sentences than did the integrity of fiber tracts supporting phonological WM. Overall, the neural findings provide converging evidence regarding the distinction of these two capacities and the greater contribution of individual differences in semantic than phonological WM capacity to sentence processing.

L1 production following brief L2 exposure: Evidence for cross-talk across comprehension and production.

Language control allows bilinguals to fluently shift between their languages. Here, we tested whether comprehension and production tasks initiate language control processes to the same extent, and whether these processes operate over specific concepts or globally. Seventy Hebrew-English bilinguals completed an L1 picture-naming production task in the first and third blocks, and either a reading aloud (word production) or an animacy judgment (word comprehension) task in their L2 in the second block. Further, concepts were either repeated across blocks or not. Results showed more filled pauses in the third block relative to the first block. Additionally, the size of this blocked-language order effect was similar following word production and word comprehension tasks in the L2, suggesting that production and comprehension tasks were similarly efficient in instigating control processes. Finally, both recurring and new concepts were affected, suggesting that mostly global language control is at play. These findings provide constraining evidence for fully understanding the scope of bilingual language control.

EPS mid-career prize: An integrated framework for the learning, recognition and interpretation of words.

In this article, I review the evidence on the involvement of sleep and consolidation in word learning and processing during language comprehension, focusing on implications for theory. The theoretical basis for the review is a complementary systems account of word learning involving flexible (hippocampal) and stable (cortical) pathways to lexical knowledge. I argue that the accumulated data are consistent with a role for both pathways in both learning and recognition of lexical items, with sleep and consolidation supporting the transfer of recent experience between the pathways. The level of involvement of each pathway is dependent on key factors, such as consistency with prior knowledge in the case of learning, and reliance on context and/or automaticity in the case of recognition. As a consequence, the notion of a mental lexicon cannot really be restricted to just the listener's stable knowledge about words: flexible knowledge and recent experiences are also important. Furthermore, I argue that the flexible pathway plays a critical role even in the absence of new lexical items. The available evidence suggests that this pathway encodes (and potentially consolidates) recent linguistic experiences, providing potential benefits to interpretation of subsequent language and the long-term retention of knowledge. In conclusion, I propose that a dual-pathway account incorporating both flexibility and stability is necessary to explain the learning, recognition, and interpretation of words.

Distinct functional connectivity patterns during naturalistic learning by adolescent first versus second language speakers.

Spoken lessons (lectures) are commonly used in schools as a medium for conveying educational content. In adolescence, experience-expectant maturation of language and cognitive systems supports learning; however, little is known about whether or how learners' language experiences interact with this integration process during learning. We examined functional connectivity using fMRI in 38 Spanish-English bilingual (L1-Spanish) and English monolingual (L1-English) adolescents during a naturalistic science video lesson in English. Seed analyses including the left inferior frontal gyrus (pars opercularis) and posterior middle temporal gyrus showed that L1-Spanish adolescents, when learning in their second language (L2), displayed widespread bilateral functional connectivity throughout the cortex while L1-English adolescents displayed mostly left-lateralized connectivity with core language regions over the course of the science lesson. Furthermore, we identified functional seed connectivity associated with better learning outcomes for adolescents with diverse language backgrounds. Importantly, functional connectivity patterns in L1-Spanish adolescents while learning in English also correlate with their Spanish cloze reading. Findings suggest that functional networks associated with higher-order language processing and cognitive control are differentially engaged for L1 vs. L2 speakers while learning new information through spoken language.

An ERP Study on the Processing of Subject-Verb and Object-Verb Gender Agreement in Punjabi.

This study was conducted with the aim of exploring the general parsing mechanisms involved in processing different kinds of dependency relations, namely verb agreement with subjects versus objects in Punjabi, an SOV Indo-Aryan language. Event related brain potentials (ERPs) were recorded as twenty-five native Punjabi speakers read transitive sentences. Critical stimuli were either fully acceptable as regards verb agreement, or alternatively violated gender agreement with the subject or object. A linear mixed-models analysis confirmed a P600 effect at the position of the verb for all violations, regardless of whether subject or object agreement was violated. These results thus suggest that an identical mechanism is involved in gender agreement computation in Punjabi regardless of whether the agreement is with the subject or the object argument.

Differences in perceptual representations in multilinguals' first, second, and third language.

Two experiments were conducted to investigate the differences in perceptual representations among multilingual individuals. In Experiment 1, the immediate sentence-picture verification paradigm was used to investigate perceptual representations in the working memory stage. The results suggest a match effect within the first language (Cantonese), but not within the second language (Mandarin) or the third language (English), showing perceptual representations only in first language comprehension. In Experiment 2, the delayed sentence-picture verification paradigm was used to investigate perceptual representations in long-term memory. Similarly, the results suggest a match effect within the first language (Mandarin), but not within the second language (English). The findings of both experiments suggest that the first language was perceptually represented, regardless of whether it was Cantonese or Mandarin, regardless of the processing in working memory or long-term memory. No evidence was found for perceptual representations in the later-learned languages, regardless of high or low proficiency. Our study has implications for theories of language comprehension and embodied cognition.

Early parafoveal semantic integration in natural reading.

Humans can read and comprehend text rapidly, implying that readers might process multiple words per fixation. However, the extent to which parafoveal words are previewed and integrated into the evolving sentence context remains disputed. We investigated parafoveal processing during natural reading by recording brain activity and eye movements using MEG and an eye tracker while participants silently read one-line sentences. The sentences contained an unpredictable target word that was either congruent or incongruent with the sentence context. To measure parafoveal processing, we flickered the target words at 60 Hz and measured the resulting brain responses (i.e. Rapid Invisible Frequency Tagging, RIFT) during fixations on the pre-target words. Our results revealed a significantly weaker tagging response for target words that were incongruent with the previous context compared to congruent ones, even within 100ms of fixating the word immediately preceding the target. This reduction in the RIFT response was also found to be predictive of individual reading speed. We conclude that semantic information is not only extracted from the parafovea but can also be integrated with the previous context before the word is fixated. This early and extensive parafoveal processing supports the rapid word processing required for natural reading. Our study suggests that theoretical frameworks of natural reading should incorporate the concept of deep parafoveal processing.

On the Mathematical Relationship Between Contextual Probability and N400 Amplitude.

Accounts of human language comprehension propose different mathematical relationships between the contextual probability of a word and how difficult it is to process, including linear, logarithmic, and super-logarithmic ones. However, the empirical evidence favoring any of these over the others is mixed, appearing to vary depending on the index of processing difficulty used and the approach taken to calculate contextual probability. To help disentangle these results, we focus on the mathematical relationship between corpus-derived contextual probability and the N400, a neural index of processing difficulty. Specifically, we use 37 contemporary transformer language models to calculate the contextual probability of stimuli from 6 experimental studies of the N400, and test whether N400 amplitude is best predicted by a linear, logarithmic, super-logarithmic, or sub-logarithmic transformation of the probabilities calculated using these language models, as well as combinations of these transformed metrics. We replicate the finding that on some datasets, a combination of linearly and logarithmically-transformed probability can predict N400 amplitude better than either metric alone. In addition, we find that overall, the best single predictor of N400 amplitude is sub-logarithmically-transformed probability, which for almost all language models and datasets explains all the variance in N400 amplitude otherwise explained by the linear and logarithmic transformations. This is a novel finding that is not predicted by any current theoretical accounts, and thus one that we argue is likely to play an important role in increasing our understanding of how the statistical regularities of language impact language comprehension.

Children's Interpretations of Numerically Quantified Expression Ambiguities: Evidence from Quantified Noun Phrases and Bare Cardinals.

Understanding how children comprehend text by forming links between sentences has been the focus of research for decades. Such research has consistently shown that children use anaphors and resolve ambiguities in a different manner than adults. The present study examined a less-studied anaphoric reference that arises when two numerically quantified expressions (e.g., "three cats… two cats…") are used in the text. Focusing on 249 six- to eight-year-old children and 50 adults for comparison, the study employed a picture selection task across six experiments to assess interpretative preferences in ambiguous and unambiguous discourses containing numerically quantified expressions. The findings indicate a pronounced difference in interpretative strategies: unlike adults, who predominantly adopted an anaphoric subset reading, children showed a consistent preference for the non-anaphoric reading, even in contexts explicitly disambiguated towards this interpretation. This preference persisted across various experimental manipulations, highlighting challenges in text integration and comprehension among children. Contributing to the developmental trajectory of language comprehension, this study underscores the complexity of cognitive development and linguistic interpretation, revealing significant developmental differences in processing numerically quantified expressions and anaphoric references within discourse.

The bigger your pupils, the better my comprehension: an ERP study of how pupil size and gaze of the speaker affect syntactic processing.

Gaze direction and pupil dilation play a critical role in communication and social interaction due to their ability to redirect and capture our attention and their relevance for emotional information. The present study aimed to explore whether the pupil size and gaze direction of the speaker affect language comprehension. Participants listened to sentences that could be correct or contain a syntactic anomaly, while the static face of a speaker was manipulated in terms of gaze direction (direct, averted) and pupil size (mydriasis, miosis). Left anterior negativity (LAN) and P600 linguistic event-related potential components were observed in response to syntactic anomalies across all conditions. The speaker's gaze did not impact syntactic comprehension. However, the amplitude of the LAN component for mydriasis (dilated pupil) was larger than for miosis (constricted pupil) condition. Larger pupils are generally associated with care, trust, interest, and attention, which might facilitate syntactic processing at early automatic stages. The result also supports the permeable and context-dependent nature of syntax. Previous studies also support an automatic nature of syntax (fast and efficient), which combined with the permeability to relevant sources of communicative information, such as pupil size and emotions, is highly adaptive for language comprehension and social interaction.

Dynamically predicting comprehension difficulties through physiological data and intelligent wearables.

Comprehending digital content written in natural language online is vital for many aspects of life, including learning, professional tasks, and decision-making. However, facing comprehension difficulties can have negative consequences for learning outcomes, critical thinking skills, decision-making, error rate, and productivity. This paper introduces an innovative approach to predict comprehension difficulties at the local content level (e.g., paragraphs). Using affordable wearable devices, we acquire physiological responses non-intrusively from the autonomous nervous system, specifically pulse rate variability, and electrodermal activity. Additionally, we integrate data from a cost-effective eye-tracker. Our machine learning algorithms identify 'hotspots' within the content and regions corresponding to a high cognitive load. These hotspots represent real-time predictors of comprehension difficulties. By integrating physiological data with contextual information (such as the levels of experience of individuals), our approach achieves an accuracy of 72.11% ± 2.21, a precision of 0.77, a recall of 0.70, and an f1 score of 0.73. This study opens possibilities for developing intelligent, cognitive-aware interfaces. Such interfaces can provide immediate contextual support, mitigating comprehension challenges within content. Whether through translation, content generation, or content summarization using available Large Language Models, this approach has the potential to enhance language comprehension.

Exploring the Interplay Between Language Comprehension and Cortical Tracking: The Bilingual Test Case.

Cortical tracking, the synchronization of brain activity to linguistic rhythms is a well-established phenomenon. However, its nature has been heavily contested: Is it purely epiphenomenal or does it play a fundamental role in speech comprehension? Previous research has used intelligibility manipulations to examine this topic. Here, we instead varied listeners' language comprehension skills while keeping the auditory stimulus constant. To do so, we tested 22 native English speakers and 22 Spanish/Catalan bilinguals learning English as a second language (SL) in an EEG cortical entrainment experiment and correlated the responses with the magnitude of the N400 component of a semantic comprehension task. As expected, native listeners effectively tracked sentential, phrasal, and syllabic linguistic structures. In contrast, SL listeners exhibited limitations in tracking sentential structures but successfully tracked phrasal and syllabic rhythms. Importantly, the amplitude of the neural entrainment correlated with the amplitude of the detection of semantic incongruities in SLs, showing a direct connection between tracking and the ability to understand speech. Together, these findings shed light on the interplay between language comprehension and cortical tracking, to identify neural entrainment as a fundamental principle for speech comprehension.

Speech Prosody Serves Temporal Prediction of Language via Contextual Entrainment.

Temporal prediction assists language comprehension. In a series of recent behavioral studies, we have shown that listeners specifically employ rhythmic modulations of prosody to estimate the duration of upcoming sentences, thereby speeding up comprehension. In the current human magnetoencephalography (MEG) study on participants of either sex, we show that the human brain achieves this function through a mechanism termed entrainment. Through entrainment, electrophysiological brain activity maintains and continues contextual rhythms beyond their offset. Our experiment combined exposure to repetitive prosodic contours with the subsequent presentation of visual sentences that either matched or mismatched the duration of the preceding contour. During exposure to prosodic contours, we observed MEG coherence with the contours, which was source-localized to right-hemispheric auditory areas. During the processing of the visual targets, activity at the frequency of the preceding contour was still detectable in the MEG; yet sources shifted to the (left) frontal cortex, in line with a functional inheritance of the rhythmic acoustic context for prediction. Strikingly, when the target sentence was shorter than expected from the preceding contour, an omission response appeared in the evoked potential record. We conclude that prosodic entrainment is a functional mechanism of temporal prediction in language comprehension. In general, acoustic rhythms appear to endow language for employing the brain's electrophysiological mechanisms of temporal prediction.

Meaning as mentalization.

The way we establish meaning has been a profound question not only in language research but in developmental science as well. The relation between linguistic form and content has been loosened up in recent pragmatic approaches to communication, showing that code-based models of language comprehension must be augmented by context-sensitive, pragmatic-inferential mechanisms to recover the speaker's intended meaning. Language acquisition has traditionally been thought to involve building a mental lexicon and extracting syntactic rules from noisy linguistic input, while communicative-pragmatic inferences have also been argued to be indispensable. Recent research findings exploring the electrophysiological indicator of semantic processing, the N400, have raised serious questions about the traditional separation between semantic decoding and pragmatic inferential processes. The N400 appears to be sensitive to mentalization-the ability to attribute beliefs to social partners-already from its developmental onset. This finding raises the possibility that mentalization may not simply contribute to pragmatic inferences that enrich linguistic decoding processes but that the semantic system may be functioning in a fundamentally mentalistic manner. The present review first summarizes the key contributions of pragmatic models of communication to language comprehension. Then, it provides an overview of how communicative intentions are interpreted in developmental theories of communication, with a special emphasis on mentalization. Next, it discusses the sensitivity of infants to the information-transmitting potential of language, their ability to pick up its code-like features, and their capacity to track language comprehension of social partners using mentalization. In conclusion, I argue that the recovery of meaning during linguistic communication is not adequately modeled as a process of code-based semantic retrieval complemented by pragmatic inferences. Instead, the semantic system may establish meaning, as intended, during language comprehension and acquisition through mentalistic attribution of content to communicative partners.

Mental simulation of colour properties during language comprehension: influence of context and comprehension stages.

Many studies have shown that mental simulation may occur during language comprehension. Supporting evidence is derived from the matching effects in the sentence-picture verification (SPV) task often used to assess mental simulations of object properties, such as size, orientation, and shape. However, mixed results have been obtained regarding object colour, with researchers reporting matching or mismatching effects. This study investigated the impact of colour information clarity within sentences on the process of mental simulation during language comprehension. Employing the SPV task and using novel objects, we examined whether there is a mental simulation of colour after excluding typical/atypical colour bias and how varying levels of colour information clarity in sentences influence the emergence of matching effects at different stages of comprehension. To address these issues, we conducted two experiments. In Experiment 1, the participants read normal sentences and subsequently engaged in picture verification with a novel object after a 500 ms delay. In Experiment 2, the participants encountered sentences containing both clear and unclear colour information and, after either a 0 ms or 1500 ms interval, completed picture verification tasks with a novel object. Null effects were found in the 500 ms condition for normal sentences and the 0 ms condition for unclear colour information sentences. A mismatching effect appeared in the 0 ms condition after clear colour information sentences, and a matching effect appeared in the 1500 ms condition for all sentences. The results indicated that after excluding colour bias, the participants still formed mental simulations of colour during language comprehension. Our results also indicated that ongoing colour simulation with time pressure impacted the participant responses. The participants ignored unclear colour information under time pressure, but without time pressure, they constructed simulations that were as detailed as possible, regardless of whether the implicit colour information in the sentence was clear.

Understanding words in context: A naturalistic EEG study of children's lexical processing.

When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.