Diverging neural dynamics for syntactic structure building in naturalistic speaking and listening.

The neural correlates of sentence production are typically studied using task paradigms that differ considerably from the experience of speaking outside of an experimental setting. In this fMRI study, we aimed to gain a better understanding of syntactic processing in spontaneous production versus naturalistic comprehension in three regions of interest (BA44, BA45, and left posterior middle temporal gyrus). A group of participants (n = 16) was asked to speak about the events of an episode of a TV series in the scanner. Another group of participants (n = 36) listened to the spoken recall of a participant from the first group. To model syntactic processing, we extracted word-by-word metrics of phrase-structure building with a top-down and a bottom-up parser that make different hypotheses about the timing of structure building. While the top-down parser anticipates syntactic structure, sometimes before it is obvious to the listener, the bottom-up parser builds syntactic structure in an integratory way after all of the evidence has been presented. In comprehension, neural activity was found to be better modeled by the bottom-up parser, while in production, it was better modeled by the top-down parser. We additionally modeled structure building in production with two strategies that were developed here to make different predictions about the incrementality of structure building during speaking. We found evidence for highly incremental and anticipatory structure building in production, which was confirmed by a converging analysis of the pausing patterns in speech. Overall, this study shows the feasibility of studying the neural dynamics of spontaneous language production.

Information-theoretic principles in incremental language production.

I apply a recently emerging perspective on the complexity of action selection, the rate-distortion theory of control, to provide a computational-level model of errors and difficulties in human language production, which is grounded in information theory and control theory. Language production is cast as the sequential selection of actions to achieve a communicative goal subject to a capacity constraint on cognitive control. In a series of calculations, simulations, corpus analyses, and comparisons to experimental data, I show that the model directly predicts some of the major known qualitative and quantitative phenomena in language production, including semantic interference and predictability effects in word choice; accessibility-based ("easy-first") production preferences in word order alternations; and the existence and distribution of disfluencies including filled pauses, corrections, and false starts. I connect the rate-distortion view to existing models of human language production, to probabilistic models of semantics and pragmatics, and to proposals for controlled language generation in the machine learning and reinforcement learning literature.

Predictive Brain Activity Shows Congruent Semantic Specificity in Language Comprehension and Production.

Sentence fragments strongly predicting a specific subsequent meaningful word elicit larger preword slow waves, prediction potentials (PPs), than unpredictive contexts. To test the current predictive processing models, 128-channel EEG data were collected from both sexes to examine whether (1) different semantic PPs are elicited in language comprehension and production and (2) whether these PPs originate from the same specific "prediction area(s)" or rather from widely distributed category-specific neuronal circuits reflecting the meaning of the predicted item. Slow waves larger after predictable than unpredictable contexts were present both before subjects heard the sentence-final word in the comprehension experiment and before they pronounced the sentence-final word in the production experiment. Crucially, cortical sources underlying the semantic PP were distributed across several cortical areas and differed between the semantic categories of the expected words. In both production and comprehension, the anticipation of animal words was reflected by sources in posterior visual areas, whereas predictable tool words were preceded by sources in the frontocentral sensorimotor cortex. For both modalities, PP size increased with higher cloze probability, thus further confirming that it reflects semantic prediction, and with shorter latencies with which participants completed sentence fragments. These results sit well with theories viewing distributed semantic category-specific circuits as the mechanistic basis of semantic prediction in the two modalities.

Four dimensions of naturalistic language production in aphasia after stroke.

There is a rich tradition of research on the neuroanatomical correlates of spoken language production in aphasia using constrained tasks (e.g., picture naming), which offer controlled insights into the distinct processes that govern speech and language (i.e., lexical-semantic access, morphosyntactic construction, phonological encoding, speech motor programming/execution). Yet these tasks do not necessarily reflect everyday language use. In contrast, naturalistic language production (also referred to as connected speech or discourse) more closely approximates typical processing demands, requiring the dynamic integration of all aspects of speech and language. The brain bases of naturalistic language production remain relatively unknown, however, in part because of the difficulty in deriving features that are salient, quantifiable, and interpretable relative to both speech-language processes and the extant literature. The present cross-sectional observational study seeks to address these challenges by leveraging a validated and comprehensive auditory-perceptual measurement system that yields four explanatory dimensions of performance-Paraphasia (misselection of words and sounds), Logopenia (paucity of words), Agrammatism (grammatical omissions), and Motor speech (impaired speech motor programming/execution). We used this system to characterize naturalistic language production in a large and representative sample of individuals with acute post-stroke aphasia (n = 118). Scores on each of the four dimensions were correlated with lesion metrics, and multivariate associations among the dimensions and brain regions were then explored. Our findings revealed distinct yet overlapping neuroanatomical correlates throughout the left-hemisphere language network. Paraphasia and Logopenia were associated primarily with posterior regions, spanning both dorsal and ventral streams, which are critical for lexical-semantic access and phonological encoding. In contrast, Agrammatism and Motor speech were associated primarily with anterior regions of the dorsal stream that are involved in morphosyntactic construction and speech motor planning/execution respectively. Collectively, we view these results as constituting a brain-behavior model of naturalistic language production in aphasia, aligning with both historical and contemporary accounts of the neurobiology of spoken language production.

The effects of endogenous semantic variables during productive lexical retrieval: A Behavioral-Neural Dual Swinging Model (DSM).

Conceptual preparation is the very initial step in language production. Endogenous semantic variables, reflecting the inherent semantic properties of concepts, could influence the productive lexical retrieval by modulating both conceptual activation and lexical selection. Yet, empirical understandings on this process and underlying mechanisms remain limited. Here, inspired by previous theoretical models and preliminary findings, we proposed a Behavioral-Neural Dual Swinging Model (DSM), revealing the swinging process between conceptual facilitation and lexical interference and extending to neural resource allocation during these processes. To further test the model, we examined the joint effect of semantic richness and semantic density on productive word retrieval both behaviorally and neurally, using a picture naming paradigm. Results nicely support the DSM by showing that the productive retrieval is driven by the swinging between semantic richness-induced conceptual facilitation primarily managed in semantic-related regions and semantic density-induced lexical interference managed in control-related regions. Moreover, the conceptual facilitation accumulated from semantic richness plays a decisive role, mitigating interference from competitors as well as the neural demands in control-related regions.

The role of input vs. output phonological working memory in narrative production: Evidence from case series and case study approaches.

Separable input and output phonological working memory (WM) capacities have been proposed, with the input capacity supporting speech recognition and the output capacity supporting production. We examined the role of input vs. output phonological WM in narrative production, examining speech rate and pronoun ratio - two measures with prior evidence of a relation to phonological WM. For speech rate, a case series approach with individuals with aphasia found no significant independent contribution of input or output phonological WM capacity after controlling for single-word production. For pronoun ratio, there was some suggestion of a role for input phonological WM. Thus, neither finding supported a specific role for an output phonological buffer in speech production. In contrast, two cases demonstrating dissociations between input and output phonological WM capacities provided suggestive evidence of predicted differences in narrative production, though follow-up research is needed. Implications for case series vs. case study approaches are discussed.

Impact of theta transcranial alternating current stimulation on language production in adult classic galactosemia patients.

Patients with classic galactosemia (CG), an inborn error of galactose metabolism, suffer from impairments in cognition, including language processing. Potential causes are atypical brain oscillations. Recent electroencephalogram (EEG) showed differences in the P300 event-related-potential (ERP) and alterations in the alpha/theta-range during speech planning. This study investigated whether transcranial alternating current stimulation (tACS) at theta-frequency compared to sham can cause a normalization of the ERP post stimulation and improves language performance. Eleven CG patients and fourteen healthy controls participated in two tACS-sessions (theta 6.5 Hz/sham). They were engaged in an active language task, describing animated scenes at three moments, that is, pre/during/post stimulation. Pre and post stimulation, behavior (naming accuracy, voice-onset-times; VOT) and mean-amplitudes of ERP were compared, by means of a P300 time-window analysis and cluster-based-permutation testing during speech planning. The results showed that theta stimulation, not sham, significantly reduced naming error-percentage in patients, not in controls. Theta did not systematically speed up naming beyond a general learning effect, which was larger for the patients. The EEG analysis revealed a significant pre-post stimulation effect (P300/late positivity), in patients and during theta stimulation only. In conclusion, theta-tACS improved accuracy in language performance in CG patients compared to controls and altered the P300 and late positive ERP-amplitude, suggesting a lasting effect on neural oscillation and behavior.

Sustained pacifier use is associated with smaller vocabulary sizes at 1 and 2 years of age: A cross-sectional study.

Pacifier use during childhood has been hypothesized to interfere with language processing, but, to date, there is limited evidence revealing detrimental effects of prolonged pacifier use on infant vocabulary learning. In the present study, parents of 12- and 24-month-old infants were recruited in Oslo (Norway). The sample included 1187 monolingual full-term born (without visual, auditory, or cognitive impairments) infants: 452 (230 girls; 222 boys) 12-month-olds and 735 (345 girls; 390 boys) 24-month-olds. Parents filled out an online Norwegian Communicative Development Inventory (CDI), which assesses the vocabulary in comprehension and production for 12-month-old infants and in production only for 24-month-old infants. CDI scores were transformed into age- and sex-adjusted percentiles using Norwegian norms. Additionally, parents retrospectively reported their child's daytime pacifier use, in hours, at 2-month intervals, from birth to the assessment date. Maternal education was used to control, in the analyses, for the socio-economic status. We found that greater pacifier use in an infant's lifespan was associated with lower vocabulary size. Pacifier use later in life was more negatively associated with vocabulary size than precocious use, and increased the odds of being a low language scorer. In sum, our study moves beyond the findings of momentary effects of experimentally induced "impairment" in articulators' movement on speech perception and suggests that, from 12 months of age, constraints on the infant's speech articulators (pacifier use) may be negatively associated with word comprehension and production. RESEARCH HIGHLIGHT: We examined the relationship between pacifier use and vocabulary sizes in production at 24 months of age and comprehension and production at 12 months of age. Lifespan Pacifier Use (LPU) was negatively correlated with vocabulary sizes in comprehension and production among 12-month-old infants and negatively correlated with production for 24-month-olds. Later pacifier use was found to be more negatively correlated with vocabulary size in infants, as compared to more precocious use. The amount of pacifier use in the 2 months prior to a child's second birthday was predictive of a higher prevalence of low vocabulary scores in 24-month-olds.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production.

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

Age-related reorganization of functional network architecture in semantic cognition.

Cognitive aging is associated with widespread neural reorganization processes in the human brain. However, the behavioral impact of such reorganization is not well understood. The current neuroimaging study investigated age differences in the functional network architecture during semantic word retrieval in young and older adults. Combining task-based functional connectivity, graph theory and cognitive measures of fluid and crystallized intelligence, our findings show age-accompanied large-scale network reorganization even when older adults have intact word retrieval abilities. In particular, functional networks of older adults were characterized by reduced decoupling between systems, reduced segregation and efficiency, and a larger number of hub regions relative to young adults. Exploring the predictive utility of these age-related changes in network topology revealed high, albeit less efficient, performance for older adults whose brain graphs showed stronger dedifferentiation and reduced distinctiveness. Our results extend theoretical accounts on neurocognitive aging by revealing the compensational potential of the commonly reported pattern of network dedifferentiation when older adults can rely on their prior knowledge for successful task processing. However, we also demonstrate the limitations of such compensatory reorganization and show that a youth-like network architecture in terms of balanced integration and segregation is associated with more economical processing.

Non-sentential responses to requests for information.

When faced with requests for information ("Where did you go last night?"), why do speakers make non-sentential replies ("The movies") rather than full sentence replies ("I went to the movies")? We examine the role that pragmatic factors (politeness and formality) and memory factors (the speaker's ability to retrieve the answer to the question) play in determining whether speakers generate a non-sentential reply. Participants answered a series of questions about their lives. Pragmatic factors affected the participants' responses. Speakers instructed to be polite or formal made fewer non-sentential replies than speakers who did not receive specific instructions. Memory retrieval (indexed both by the time required for the participant to begin their response and by the presence of disfluencies at the beginning of the response) did not have a straightforward relationship to the production of non-sentential replies. The effect of response latency and disfluencies depended on whether the participants were told to be polite or formal (or if they were given no instruction at all).

Competition accumulates in successive retrieval of proper names.

Proper names are especially prone to retrieval failures and tip-of-the-tongue states (TOTs)-a phenomenon wherein a person has a strong feeling of knowing a word but cannot retrieve it. Current research provides mixed evidence regarding whether related names facilitate or compete with target-name retrieval. We examined this question in two experiments using a novel paradigm where participants either read a prime name aloud (Experiment 1) or classified a written prime name as famous or non-famous (Experiment 2) prior to naming a celebrity picture. Successful retrievals decreased with increasing trial number (and was dependent on the number of previously presented similar famous people) in both experiments, revealing a form of accumulating interference between multiple famous names. However, trial number had no effect on TOTs, and within each trial famous prime names increased TOTs only in Experiment 2. These results can be explained within a framework that assumes competition for selection at the point of lexical retrieval, such that successful retrievals decrease after successive retrievals of proper names of depicted faces of semantically similar people. By contrast, the effects of written prime words only occur when prime names are sufficiently processed, and do not provide evidence for competition but may reflect improved retrieval relative to a "don't know" response.

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language.

Sign language serves as the primary mode of communication for the deaf community. With technological advancements, it is crucial to develop systems capable of enhancing communication between deaf and hearing individuals. This paper reviews recent state-of-the-art methods in sign language recognition, translation, and production. Additionally, we introduce a rule-based system, called ruLSE, for generating synthetic datasets in Spanish Sign Language. To check the usefulness of these datasets, we conduct experiments with two state-of-the-art models based on Transformers, MarianMT and Transformer-STMC. In general, we observe that the former achieves better results (+3.7 points in the BLEU-4 metric) although the latter is up to four times faster. Furthermore, the use of pre-trained word embeddings in Spanish enhances results. The rule-based system demonstrates superior performance and efficiency compared to Transformer models in Sign Language Production tasks. Lastly, we contribute to the state of the art by releasing the generated synthetic dataset in Spanish named synLSE.

Timed picture naming norms for 800 photographs of 200 objects in English.

The present study presents picture-naming norms for a large set of 800 high-quality photographs of 200 natural objects and artefacts spanning a range of categories, with four unique images per object. Participants were asked to provide a single, most appropriate name for each image seen. We report recognition latencies for each image, and several normed variables for the provided names: agreement, H-statistic (i.e. level of naming uncertainty), Zipf word frequency and word length. Rather than simply focusing on a single name per image (i.e. the modal or most common name), analysis of recognition latencies showed that it is important to consider the diversity of labels that participants may ascribe to each pictured object. The norms therefore provide a list of candidate labels per image with weighted measures of word length and frequency per image that incorporate all provided names, as well as modal measures based on the most common name only.

First Language Lexical Attrition in a First Language Setting: A Multi-Measure Approach Testing Teachers of English.

Research on first language (L1) attrition typically focuses on immigrant populations in their second language (L2) environment, yet we know comparably little about L1 attrition in the L1 setting. This study used two lexical tasks to test L1 attrition, a time-sensitive word decision task and a video retelling. Chinese teachers of English vs. Chinese teachers of other subjects (N = 25/group) were recruited at a secondary school in China. The aim was to provide an exploratory basis of the L2 influence on L1 lexical attrition in the L1 environment, both on the level of lexical comprehension and production. Mixed-effects models were used to analyse multiple measures including response accuracy and reaction times in comprehension, and lexical diversity, density, sophistication, and accuracy in production. The results showed Chinese teachers' L1 lexical attrition in the form of longer response times to high-frequency Chinese words compared to non-English Chinese teachers, and the use of significantly fewer sophisticated words in their retellings. Also, teachers of English were faster and more accurate in decisions about Chinese borrowings from English, suggesting L2-driven influence on their mental lexicon. Considering participants' background information, analyses showed that increased L2 exposure and frequency of use can predict L1 lexical attrition.

Using a translanguaging framework to examine language production in a trilingual person with aphasia.

When language abilities in aphasia are assessed in clinical and research settings, the standard practice is to examine each language of a multilingual person separately. But many multilingual individuals, with and without aphasia, mix their languages regularly when they communicate with other speakers who share their languages. We applied a novel approach to scoring language production of a multilingual person with aphasia. Our aim was to discover whether the assessment outcome would differ meaningfully when we count accurate responses in only the target language of the assessment session versus when we apply a translanguaging framework, that is, count all accurate responses, regardless of the language in which they were produced. The participant is a Farsi-German-English speaking woman with chronic moderate aphasia. We examined the participant's performance on two picture-naming tasks, an answering wh-question task, and an elicited narrative task. The results demonstrated that scores in English, the participant's third-learned and least-impaired language did not differ between the two scoring methods. Performance in German, the participant's moderately impaired second language benefited from translanguaging-based scoring across the board. In Farsi, her weakest language post-CVA, the participant's scores were higher under the translanguaging-based scoring approach in some but not all of the tasks. Our findings suggest that whether a translanguaging-based scoring makes a difference in the results obtained depends on relative language abilities and on pragmatic constraints, with additional influence of the linguistic distances between the languages in question.

The Time Course of Language Production as Revealed by Pattern Classification of MEG Sensor Data.

Language production involves a complex set of computations, from conceptualization to articulation, which are thought to engage cascading neural events in the language network. However, recent neuromagnetic evidence suggests simultaneous meaning-to-speech mapping in picture naming tasks, as indexed by early parallel activation of frontotemporal regions to lexical semantic, phonological, and articulatory information. Here we investigate the time course of word production, asking to what extent such "earliness" is a distinctive property of the associated spatiotemporal dynamics. Using MEG, we recorded the neural signals of 34 human subjects (26 males) overtly naming 134 images from four semantic object categories (animals, foods, tools, clothes). Within each category, we covaried word length, as quantified by the number of syllables contained in a word, and phonological neighborhood density to target lexical and post-lexical phonological/phonetic processes. Multivariate pattern analyses searchlights in sensor space distinguished the stimulus-locked spatiotemporal responses to object categories early on, from 150 to 250 ms after picture onset, whereas word length was decoded in left frontotemporal sensors at 250-350 ms, followed by the latency of phonological neighborhood density (350-450 ms). Our results suggest a progression of neural activity from posterior to anterior language regions for the semantic and phonological/phonetic computations preparing overt speech, thus supporting serial cascading models of word production.SIGNIFICANCE STATEMENT Current psycholinguistic models make divergent predictions on how a preverbal message is mapped onto articulatory output during the language planning. Serial models predict a cascading sequence of hierarchically organized neural computations from conceptualization to articulation. In contrast, parallel models posit early simultaneous activation of multiple conceptual, phonological, and articulatory information in the language system. Here we asked whether such earliness is a distinctive property of the neural dynamics of word production. The combination of the millisecond precision of MEG with multivariate pattern analyses revealed subsequent onset times for the neural events supporting semantic and phonological/phonetic operations, progressing from posterior occipitotemporal to frontal sensor areas. The findings bring new insights for refining current theories of language production.

Utterance planning under message uncertainty: evidence from a novel picture-naming paradigm.

Language researchers view utterance planning as implicit decision-making: producers must choose the words, sentence structures, and various other linguistic features to communicate their message. To date, much of the research on utterance planning has focused on situations in which the speaker knows the full message to convey. Less is known about circumstances in which speakers begin utterance planning before they are certain about their message. In three picture-naming experiments, we used a novel paradigm to examine how speakers plan utterances before a full message is known. In Experiments 1 and 2, participants viewed displays showing two pairs of objects, followed by a cue to name one pair. In an Overlap condition, one object appeared in both pairs, providing early information about one of the objects to name. In a Different condition, there was no object overlap. Across both spoken and typed responses, participants tended to name the overlapping target first in the Overlap condition, with shorter initiation latencies compared with other utterances. Experiment 3 used a semantically constraining question to provide early information about the upcoming targets, and participants tended to name the more likely target first in their response. These results suggest that in situations of uncertainty, producers choose word orders that allow them to begin early planning. Producers prioritize message components that are certain to be needed and continue planning the rest when more information becomes available. Given similarities to planning strategies for other goal-directed behaviors, we suggest continuity between decision-making processes in language and other cognitive domains.

Precision of phonological errors in aphasia supports resource models of phonological working memory in language production.

Working memory (WM) is critical for many cognitive functions including language production. A key feature of WM is its capacity limitation. Two models have been proposed to account for such capacity limitation: slot models and resource models. In recent years, resource models have found support in both visual and auditory perception, but do they also extend to production? We investigate this by analyzing sublexical errors from four individuals with aphasia. Using tools from computational linguistics, we first define the concept of "precision" of sublexical errors. We then demonstrate that such precision decreases with increased working memory load, i.e., word length, as predicted by resource models. Finally, we rule out alternative accounts of this effect, such as articulatory simplification. These data provide the first evidence for the applicability of the resource model to production and further point to the generalizability of this account as a model of resource division in WM.

Two sides of the same coin? Comparing structural priming between production and comprehension in choice data and in reaction times.

Although structural priming seems to rely on the same mechanisms in production and comprehension, effects are not always consistent between modalities. Methodological differences often result in different data types, namely choice data in production and reaction time data in comprehension. In a structural priming experiment with English ditransitives, we collected choice data and reaction time data in both modalities. The choice data showed priming of the DO and PO dative. The reaction times revealed priming of the PO dative. In production, PO targets were chosen faster after a PO prime than after a baseline prime. In comprehension, DO targets were read slower after a PO prime than after a baseline prime. This result can be explained from competition between alternatives during structure selection. Priming leads to facilitation of the primed structure or inhibition of the opposite structure depending on the relative frequency of structures, which may differ across modalities.