Biological constraints on neural network models of cognitive function.

Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling.

Predictive Brain Activity Shows Congruent Semantic Specificity in Language Comprehension and Production.

Sentence fragments strongly predicting a specific subsequent meaningful word elicit larger preword slow waves, prediction potentials (PPs), than unpredictive contexts. To test the current predictive processing models, 128-channel EEG data were collected from both sexes to examine whether (1) different semantic PPs are elicited in language comprehension and production and (2) whether these PPs originate from the same specific "prediction area(s)" or rather from widely distributed category-specific neuronal circuits reflecting the meaning of the predicted item. Slow waves larger after predictable than unpredictable contexts were present both before subjects heard the sentence-final word in the comprehension experiment and before they pronounced the sentence-final word in the production experiment. Crucially, cortical sources underlying the semantic PP were distributed across several cortical areas and differed between the semantic categories of the expected words. In both production and comprehension, the anticipation of animal words was reflected by sources in posterior visual areas, whereas predictable tool words were preceded by sources in the frontocentral sensorimotor cortex. For both modalities, PP size increased with higher cloze probability, thus further confirming that it reflects semantic prediction, and with shorter latencies with which participants completed sentence fragments. These results sit well with theories viewing distributed semantic category-specific circuits as the mechanistic basis of semantic prediction in the two modalities.

Causal Influence of Linguistic Learning on Perceptual and Conceptual Processing: A Brain-Constrained Deep Neural Network Study of Proper Names and Category Terms.

Language influences cognitive and conceptual processing, but the mechanisms through which such causal effects are realized in the human brain remain unknown. Here, we use a brain-constrained deep neural network model of category formation and symbol learning and analyze the emergent model's internal mechanisms at the neural circuit level. In one set of simulations, the network was presented with similar patterns of neural activity indexing instances of objects and actions belonging to the same categories. Biologically realistic Hebbian learning led to the formation of instance-specific neurons distributed across multiple areas of the network, and, in addition, to cell assembly circuits of "shared" neurons responding to all category instances-the network correlates of conceptual categories. In two separate sets of simulations, the network learned the same patterns together with symbols for individual instances ["proper names" (PN)] or symbols related to classes of instances sharing common features ["category terms" (CT)]. Learning CT remarkably increased the number of shared neurons in the network, thereby making category representations more robust while reducing the number of neurons of instance-specific ones. In contrast, proper name learning prevented a substantial reduction of instance-specific neurons and blocked the overgrowth of category general cells. Representational similarity analysis further confirmed that the neural activity patterns of category instances became more similar to each other after category-term learning, relative to both learning with PN and without any symbols. These network-based mechanisms for concepts, PN, and CT explain why and how symbol learning changes object perception and memory, as revealed by experimental studies.

Brain-constrained neural modeling explains fast mapping of words to meaning.

Although teaching animals a few meaningful signs is usually time-consuming, children acquire words easily after only a few exposures, a phenomenon termed "fast-mapping." Meanwhile, most neural network learning algorithms fail to achieve reliable information storage quickly, raising the question of whether a mechanistic explanation of fast-mapping is possible. Here, we applied brain-constrained neural models mimicking fronto-temporal-occipital regions to simulate key features of semantic associative learning. We compared networks (i) with prior encounters with phonological and conceptual knowledge, as claimed by fast-mapping theory, and (ii) without such prior knowledge. Fast-mapping simulations showed word-specific representations to emerge quickly after 1-10 learning events, whereas direct word learning showed word-meaning mappings only after 40-100 events. Furthermore, hub regions appeared to be essential for fast-mapping, and attention facilitated it, but was not strictly necessary. These findings provide a better understanding of the critical mechanisms underlying the human brain's unique ability to acquire new words rapidly.

Instantaneous neural processing of communicative functions conveyed by speech prosody.

During conversations, speech prosody provides important clues about the speaker's communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. Here, the neurophysiological basis of intonation and speech act understanding were investigated with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level. Already approximately 100 ms after the sentence-final word differing in prosody, questions, and statements expressed with the same sentences led to different neurophysiological activity recorded in the event-related potential. Interestingly, low-pass filtered sentences and acoustically matched nonvocal musical signals failed to show any neurophysiological dissociations, thus suggesting that the physical intonation alone cannot explain this modulation. Our results show rapid neurophysiological indexes of prosodic communicative information processing that emerge only when pragmatic and lexico-semantic information are fully expressed. The early enhancement of question-related activity compared with statements was due to sources in the articulatory-motor region, which may reflect the richer action knowledge immanent to questions, namely the expectation of the partner action of answering the question. The present findings demonstrate a neurophysiological correlate of prosodic communicative information processing, which enables humans to rapidly detect and understand speaker intentions in linguistic interactions.

Bo-NO-bouba-kiki: picture-word mapping but no spontaneous sound symbolic speech-shape mapping in a language trained bonobo.

Humans share the ability to intuitively map 'sharp' or 'round' pseudowords, such as 'bouba' versus 'kiki', to abstract edgy versus round shapes, respectively. This effect, known as sound symbolism, appears early in human development. The phylogenetic origin of this phenomenon, however, is unclear: are humans the only species capable of experiencing correspondences between speech sounds and shapes, or could similar effects be observed in other animals? Thus far, evidence from an implicit matching experiment failed to find evidence of this sound symbolic matching in great apes, suggesting its human uniqueness. However, explicit tests of sound symbolism have never been conducted with nonhuman great apes. In the present study, a language-competent bonobo completed a cross-modal matching-to-sample task in which he was asked to match spoken English words to pictures, as well as 'sharp' or 'round' pseudowords to shapes. Sound symbolic trials were interspersed among English words. The bonobo matched English words to pictures with high accuracy, but did not show any evidence of spontaneous sound symbolic matching. Our results suggest that speech exposure/comprehension alone cannot explain sound symbolism. This lends plausibility to the hypothesis that biological differences between human and nonhuman primates could account for the putative human specificity of this effect.

Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity.

With strong and valid predictions, grasping a message is easy, whereas more demanding processing is required in the absence of robust expectations. We here demonstrate that brain correlates of the interplay between prediction and perception mechanisms in the understanding of meaningful sentences. Sentence fragments that strongly predict subsequent words induced anticipatory brain activity preceding the expected words; this potential was absent if context did not strongly predict subsequent words. Subjective reports of certainty about upcoming words and objective corpus-based measures correlated with the size of the anticipatory signal, thus establishing its status as a semantic prediction potential (SPP). Crucially, there was an inverse correlation between the SPP and the N400 brain response. The main cortical generators of SPP and N400 were found in inferior prefrontal cortex and posterior temporal cortex, respectively. Interestingly, sentence meaning was reflected by both measures, with additional category-specific sources of SPPs and N400s falling into parieto-temporo-occipital (visual) and frontocentral (sensorimotor) areas for animal- and tool-related words, respectively. These results show that the well-known brain index of semantic comprehension, N400, has an antecedent with different brain localization but similar semantic discriminatory function. We discuss whether N400 dynamics may causally depend on mechanisms underlying SPP size and sources.

Modelling concrete and abstract concepts using brain-constrained deep neural networks.

A neurobiologically constrained deep neural network mimicking cortical area function relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically 'ground' concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their 'shared neurons', thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Intensive aphasia therapy improves low mood in fluent post-stroke aphasia: Evidence from a case-controlled study.

Introduction: Depressive symptoms are a major drawback of aphasia, negatively impacting on functional outcomes. In a previous study, Intensive Language-Action Therapy (ILAT) was effective in improving depression and low mood in persons with chronic non-fluent aphasia. We present a proof-of-concept case-control study that evaluates language and mood outcomes amongst persons with fluent post-stroke aphasia.Participants: Thirteen Spanish speaking persons with fluent aphasia due to chronic stroke lesions in the left hemisphere participated in the study.Intervention: Five participants (intervention group) received ILAT for 3 h/day during two consecutive weeks, for an overall of 30 h, and 8 participants (control group) entered a waiting-list no-treatment arm.Results: The main finding was that participants receiving active treatment showed significant improvements on depression and aphasia severity scores, whereas no significant changes were found in the control group.Conclusions: The implementation of ILAT was efficient in improving clinical language deficits in people with fluent aphasia and contributes to improvement in mood after therapy.Trial registration: EUDRACT (2008-008481-12).

Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals.

A class of semantic theories defines concepts in terms of statistical distributions of lexical items, basing meaning on vectors of word co-occurrence frequencies. A different approach emphasizes abstract hierarchical taxonomic relationships among concepts. However, the functional relevance of these different accounts and how they capture information-encoding of lexical meaning in the brain still remains elusive. We investigated to what extent distributional and taxonomic models explained word-elicited neural responses using cross-validated representational similarity analysis (RSA) of functional magnetic resonance imaging (fMRI) and model comparisons. Our findings show that the brain encodes both types of semantic information, but in distinct cortical regions. Posterior middle temporal regions reflected lexical-semantic similarity based on hierarchical taxonomies, in coherence with the action-relatedness of specific semantic word categories. In contrast, distributional semantics best predicted the representational patterns in left inferior frontal gyrus (LIFG, BA 47). Both representations coexisted in the angular gyrus supporting semantic binding and integration. These results reveal that neuronal networks with distinct cortical distributions across higher-order association cortex encode different representational properties of word meanings. Taxonomy may shape long-term lexical-semantic representations in memory consistently with the sensorimotor details of semantic categories, whilst distributional knowledge in the LIFG (BA 47) may enable semantic combinatorics in the context of language use. Our approach helps to elucidate the nature of semantic representations essential for understanding human language.

Multisensory cueing facilitates naming in aphasia.

BACKGROUND: Impaired naming is a ubiquitous symptom in all types of aphasia, which often adversely impacts independence, quality of life, and recovery of affected individuals. Previous research has demonstrated that naming can be facilitated by phonological and semantic cueing strategies that are largely incorporated into the treatment of anomic disturbances. Beneficial effects of cueing, whereby naming becomes faster and more accurate, are often attributed to the priming mechanisms occurring within the distributed language network. OBJECTIVE: We proposed and explored two novel cueing techniques: (1) Silent Visuomotor Cues (SVC), which provided articulatory information of target words presented in the form of silent videos, and (2) Semantic Auditory Cues (SAC), which consisted of acoustic information semantically relevant to target words (ringing for "telephone"). Grounded in neurophysiological evidence, we hypothesized that both SVC and SAC might aid communicative effectiveness possibly by triggering activity in perceptual and semantic language regions, respectively. METHODS: Ten participants with chronic non-fluent aphasia were recruited for a longitudinal clinical intervention. Participants were split into dyads (i.e., five pairs of two participants) and required to engage in a turn-based peer-to-peer language game using the Rehabilitation Gaming System for aphasia (RGSa). The objective of the RGSa sessions was to practice communicative acts, such as making a request. We administered SVCs and SACs in a pseudorandomized manner at the moment when the active player selected the object to be requested from the interlocutor. For the analysis, we compared the times from selection to the reception of the desired object between cued and non-cued trials. RESULTS: Naming accuracy, as measured by a standard clinical scale, significantly improved for all stimuli at each evaluation point, including the follow-up. Moreover, the results yielded beneficial effects of both SVC and SAC cues on word naming, especially at the early intervention sessions when the exposure to the target lexicon was infrequent. CONCLUSIONS: This study supports the efficacy of the proposed cueing strategies which could be integrated into the clinic or mobile technology to aid naming even at the chronic stages of aphasia. These findings are consistent with sensorimotor accounts of language processing, suggesting a coupling between language, motor, and semantic brain regions. TRIAL REGISTRATION: NCT02928822 . Registered 30 May 2016.

Augmented Dyadic Therapy Boosts Recovery of Language Function in Patients With Nonfluent Aphasia.

Background and Purpose- Evidence suggests that therapy can be effective in recovering from aphasia, provided that it consists of socially embedded, intensive training of behaviorally relevant tasks. However, the resources of healthcare systems are often too limited to provide such treatment at sufficient dosage. Hence, there is a need for evidence-based, cost-effective rehabilitation methods. Here, we asked whether virtual reality-based treatment grounded in the principles of use-dependent learning, behavioral relevance, and intensity positively impacts recovery from nonfluent aphasia. Methods- Seventeen patients with chronic nonfluent aphasia underwent intensive therapy in a randomized, controlled, parallel-group trial. Participants were assigned to the control group (N=8) receiving standard treatment or to the experimental group (N=9) receiving augmented embodied therapy with the Rehabilitation Gaming System for aphasia. All Rehabilitation Gaming System for aphasia sessions were supervised by an assistant who monitored the patients but did not offer any elements of standard therapy. Both interventions were matched for intensity and materials. Results- Our results revealed that at the end of the treatment both groups significantly improved on the primary outcome measure (Boston Diagnostic Aphasia Examination: control group, P=0.04; experimental group, P=0.01), and the secondary outcome measure (lexical access-vocabulary test: control group, P=0.01; experimental group, P=0.007). However, only the Rehabilitation Gaming System for aphasia group improved on the Communicative Aphasia Log ( P=0.01). The follow-up assessment (week 16) demonstrated that while both groups retained vocabulary-related changes (control group, P=0.01; experimental group, P=0.007), only the Rehabilitation Gaming System for aphasia group showed therapy-induced improvements in language ( P=0.01) and communication ( P=0.05). Conclusions- Our results demonstrate the effectiveness of Rehabilitation Gaming System for aphasia for improving language and communication in patients with chronic aphasia suggesting that current challenges faced by the healthcare system in the treatment of stroke might be effectively addressed by augmenting traditional therapy with computer-based methods. Clinical Trial Registration- URL: https://www.clinicaltrials.gov . Unique identifier: NCT02928822.

Prediction mechanisms in motor and auditory areas and their role in sound perception and language understanding.

Is the meaning of an expected stimulus manifest in brain activity even before it appears? Although theories of predictive coding see anticipatory activity as crucial for the understanding of brain function, few studies have explored neurophysiologically manifest semantic predictions. Here, we report predictive negative-going potentials before the onset of action (i.e. whistle and hand clap) and non-action (i.e. pure tone, water drop) sounds. These prediction potentials (PP) indexed the meaning of action-related sounds. Dependent on the body-part-relationship of sound stimuli, neural sources were relatively more prominent in dorsal or ventral motor areas. In contrast, meaningless sounds (pure tones) activated predictive sources in temporal areas close to the auditory cortex; complex environmental sounds induced an anticipatory positivity broadly distributed over the scalp. We also found a systematic relationship between predictive activity and a Mismatch Negativity (MMN) like response to unexpected meaningful words which were presented as rare deviant stimuli amongst frequently repeated sounds. This deviant-elicited potential indexed semantic priming between action sounds and action-related words and semantic mismatch (prediction error). These results suggest a systematic link between perceptual/semantic prediction and matching mechanisms in the processing of sounds and words.

Neural Correlates of Semantic Prediction and Resolution in Sentence Processing.

Most brain-imaging studies of language comprehension focus on activity following meaningful stimuli. Testing adult human participants with high-density EEG, we show that, already before the presentation of a critical word, context-induced semantic predictions are reflected by a neurophysiological index, which we therefore call the semantic readiness potential (SRP). The SRP precedes critical words if a previous sentence context constrains the upcoming semantic content (high-constraint contexts), but not in unpredictable (low-constraint) contexts. Specific semantic predictions were indexed by SRP sources within the motor system-in dorsolateral hand motor areas for expected hand-related words (e.g., "write"), but in ventral motor cortex for face-related words ("talk"). Compared with affirmative sentences, negated ones led to medial prefrontal and more widespread motor source activation, the latter being consistent with predictive semantic computation of alternatives to the negated expected concept. Predictive processing of semantic alternatives in negated sentences is further supported by a negative-going event-related potential at ∼400 ms (N400), which showed the typical enhancement to semantically incongruent sentence endings only in high-constraint affirmative contexts, but not to high-constraint negated ones. These brain dynamics reveal the interplay between semantic prediction and resolution (match vs error) processing in sentence understanding.SIGNIFICANCE STATEMENT Most neuroscientists agree on the eminent importance of predictive mechanisms for understanding basic as well as higher brain functions. This contrasts with a sparseness of brain measures that directly reflects specific aspects of prediction, as they are relevant in the processing of language and thought. Here we show that when critical words are strongly expected in their sentence context, a predictive brain response reflects meaning features of these anticipated symbols already before they appear. The granularity of the semantic predictions was so fine grained that the cortical sources in sensorimotor and medial prefrontal cortex even distinguished between predicted face- or hand-related action words (e.g., the words "lick" or "pick") and between affirmative and negated sentence meanings.

Neurocomputational Consequences of Evolutionary Connectivity Changes in Perisylvian Language Cortex.

The human brain sets itself apart from that of its primate relatives by specific neuroanatomical features, especially the strong linkage of left perisylvian language areas (frontal and temporal cortex) by way of the arcuate fasciculus (AF). AF connectivity has been shown to correlate with verbal working memory-a specifically human trait providing the foundation for language abilities-but a mechanistic explanation of any related causal link between anatomical structure and cognitive function is still missing. Here, we provide a possible explanation and link, by using neurocomputational simulations in neuroanatomically structured models of the perisylvian language cortex. We compare networks mimicking key features of cortical connectivity in monkeys and humans, specifically the presence of relatively stronger higher-order "jumping links" between nonadjacent perisylvian cortical areas in the latter, and demonstrate that the emergence of working memory for syllables and word forms is a functional consequence of this structural evolutionary change. We also show that a mere increase of learning time is not sufficient, but that this specific structural feature, which entails higher connectivity degree of relevant areas and shorter sensorimotor path length, is crucial. These results offer a better understanding of specifically human anatomical features underlying the language faculty and their evolutionary selection advantage.SIGNIFICANCE STATEMENT Why do humans have superior language abilities compared to primates? Recently, a uniquely human neuroanatomical feature has been demonstrated in the strength of the arcuate fasciculus (AF), a fiber pathway interlinking the left-hemispheric language areas. Although AF anatomy has been related to linguistic skills, an explanation of how this fiber bundle may support language abilities is still missing. We use neuroanatomically structured computational models to investigate the consequences of evolutionary changes in language area connectivity and demonstrate that the human-specific higher connectivity degree and comparatively shorter sensorimotor path length implicated by the AF entail emergence of verbal working memory, a prerequisite for language learning. These results offer a better understanding of specifically human anatomical features for language and their evolutionary selection advantage.

Efficacy of intensive aphasia therapy in patients with chronic stroke: a randomised controlled trial.

OBJECTIVE: Recent evidence has fuelled the debate on the role of massed practice in the rehabilitation of chronic post-stroke aphasia. Here, we further determined the optimal daily dosage and total duration of intensive speech-language therapy. METHODS: Individuals with chronic aphasia more than 1 year post-stroke received Intensive Language-Action Therapy in a randomised, parallel-group, blinded-assessment, controlled trial. Participants were randomly assigned to one of two outpatient groups who engaged in either highly-intensive practice (Group I: 4 hours daily) or moderately-intensive practice (Group II: 2 hours daily). Both groups went through an initial waiting period and two successive training intervals. Each phase lasted 2 weeks. Co-primary endpoints were defined after each training interval. RESULTS: Thirty patients-15 per group-completed the study. A primary outcome measure (Aachen Aphasia Test) revealed no gains in language performance after the waiting period, but indicated significant progress after each training interval (gradual 2-week t-score change [CI]: 1.7 [±0.4]; 0.6 [±0.5]), independent of the intensity level applied (4-week change in Group I: 2.4 [±1.2]; in Group II: 2.2 [±0.8]). A secondary outcome measure (Action Communication Test) confirmed these findings in the waiting period and in the first training interval. In the second training interval, however, only patients with moderately-intensive practice continued to make progress (Time-by-Group interaction: P=0.009, η2=0.13). CONCLUSIONS: Our results suggest no added value from more than 2 hours of daily speech-language therapy within 4 weeks. Instead, these results demonstrate that even a small 2-week increase in treatment duration contributes substantially to recovery from chronic post-stroke aphasia.

Representational Similarity Mapping of Distributional Semantics in Left Inferior Frontal, Middle Temporal, and Motor Cortex.

Language comprehension engages a distributed network of frontotemporal, parietal, and sensorimotor regions, but it is still unclear how meaning of words and their semantic relationships are represented and processed within these regions and to which degrees lexico-semantic representations differ between regions and semantic types. We used fMRI and representational similarity analysis to relate word-elicited multivoxel patterns to semantic similarity between action and object words. In left inferior frontal (BA 44-45-47), left posterior middle temporal and left precentral cortex, the similarity of brain response patterns reflected semantic similarity among action-related verbs, as well as across lexical classes-between action verbs and tool-related nouns and, to a degree, between action verbs and food nouns, but not between action verbs and animal nouns. Instead, posterior inferior temporal cortex exhibited a reverse response pattern, which reflected the semantic similarity among object-related nouns, but not action-related words. These results show that semantic similarity is encoded by a range of cortical areas, including multimodal association (e.g., anterior inferior frontal, posterior middle temporal) and modality-preferential (premotor) cortex and that the representational geometries in these regions are partly dependent on semantic type, with semantic similarity among action-related words crossing lexical-semantic category boundaries.

Electrophysiological Evidence for Early and Interactive Symbol Access and Rule Processing in Retrieving and Combining Language Constructions.

The human brain stores an immense repertoire of linguistic symbols (morphemes, words) and combines them into a virtually unlimited set of well-formed strings (phrases, sentences) that serve as efficient communicative tools. Communication is hampered, however, if strings include meaningless items (e.g., "pseudomorphemes"), or if the rules for combining string elements are violated. Prior research suggests that, when participants attentively process sentences in a linguistic task, syntactic processing can occur quite early, but lexicosemantic processing, or any interaction involving this factor, is manifest later in time (ca. 400 msec or later). In contrast, recent evidence from passive speech perception paradigms suggests early processing of both combinatorial (morphosyntactic) and storage-related (lexicosemantic) properties. A crucial question is whether these parallel processes might also interact early in processing. Using ERPs in an orthogonal design, we presented spoken word strings to participants while they were distracted from incoming speech to obtain information about automatic language processing mechanisms unaffected by task-related strategies. Stimuli were either (1) well-formed miniconstructions (short pronoun-verb sentences), (2) "unstored" strings containing a pseudomorpheme, (3) "ill-combined" strings violating subject-verb agreement rules, or (4) double violations including both types of errors. We found that by 70-210 msec after the onset of the phrase-final syllable that disambiguated the strings, interactions of lexicosemantic and morphosyntactic deviance were evident in the ERPs. These results argue against serial processing of lexical storage, morphosyntactic combination and their interaction, and in favor of early, simultaneous, and interactive processing of symbols and their combinatorial structures.

The cortical dynamics of speaking: Lexical and phonological knowledge simultaneously recruit the frontal and temporal cortex within 200 ms.

Language production models typically assume that retrieving a word for articulation is a sequential process with substantial functional delays between conceptual, lexical, phonological and motor processing, respectively. Nevertheless, explicit evidence contrasting the spatiotemporal dynamics between different word production components is scarce. Here, using anatomically constrained magnetoencephalography during overt meaningful speech production, we explore the speed with which lexico-semantic versus acoustic-articulatory information of a to-be-uttered word become first neurophysiologically manifest in the cerebral cortex. We demonstrate early modulations of brain activity by the lexical frequency of a word in the temporal cortex and the left inferior frontal gyrus, simultaneously with activity in the motor and the posterior superior temporal cortex reflecting articulatory-acoustic phonological features (+LABIAL vs. +CORONAL) of the word-initial speech sounds (e.g., Monkey vs. Donkey). The specific nature of the spatiotemporal pattern correlating with a word's frequency and initial phoneme demonstrates that, in the course of speech planning, lexico-semantic and phonological-articulatory processes emerge together rapidly, drawing in parallel on temporal and frontal cortex. This novel finding calls for revisions of current brain language theories of word production.

Somatotopic Semantic Priming and Prediction in the Motor System.

The recognition of action-related sounds and words activates motor regions, reflecting the semantic grounding of these symbols in action information; in addition, motor cortex exerts causal influences on sound perception and language comprehension. However, proponents of classic symbolic theories still dispute the role of modality-preferential systems such as the motor cortex in the semantic processing of meaningful stimuli. To clarify whether the motor system carries semantic processes, we investigated neurophysiological indexes of semantic relationships between action-related sounds and words. Event-related potentials revealed that action-related words produced significantly larger stimulus-evoked (Mismatch Negativity-like) and predictive brain responses (Readiness Potentials) when presented in body-part-incongruent sound contexts (e.g., "kiss" in footstep sound context; "kick" in whistle context) than in body-part-congruent contexts, a pattern reminiscent of neurophysiological correlates of semantic priming. Cortical generators of the semantic relatedness effect were localized in areas traditionally associated with semantic memory, including left inferior frontal cortex and temporal pole, and, crucially, in motor areas, where body-part congruency of action sound-word relationships was indexed by a somatotopic pattern of activation. As our results show neurophysiological manifestations of action-semantic priming in the motor cortex, they prove semantic processing in the motor system and thus in a modality-preferential system of the human brain.