Distinct neural mechanisms for action access and execution in the human brain: insights from an fMRI study.

Goal-directed actions are fundamental to human behavior, whereby inner goals are achieved through mapping action representations to motor outputs. The left premotor cortex (BA6) and the posterior portion of Broca's area (BA44) are two modulatory poles of the action system. However, how these regions support the representation-output mapping within the system is not yet understood. To address this, we conducted a finger-tapping functional magnetic resonance imaging experiment using action categories ranging from specific to general. Our study found distinct neural behaviors in BA44 and BA6 during action category processing and motor execution. During access of action categories, activity in a posterior portion of BA44 (pBA44) decreased linearly as action categories became less specific. Conversely, during motor execution, activity in BA6 increased linearly with less specific categories. These findings highlight the differential roles of pBA44 and BA6 in action processing. We suggest that pBA44 facilitates access to action categories by utilizing motor information from the behavioral context while the premotor cortex integrates motor information to execute the selected action. This finding enhances our understanding of the interplay between prefrontal cortical regions and premotor cortex in mapping action representation to motor execution and, more in general, of the cortical mechanisms underlying human behavior.

Cleaning up the Brickyard: How Theory and Methodology Shape Experiments in Cognitive Neuroscience of Language.

The capacity for language is a defining property of our species, yet despite decades of research, evidence on its neural basis is still mixed and a generalized consensus is difficult to achieve. We suggest that this is partly caused by researchers defining "language" in different ways, with focus on a wide range of phenomena, properties, and levels of investigation. Accordingly, there is very little agreement among cognitive neuroscientists of language on the operationalization of fundamental concepts to be investigated in neuroscientific experiments. Here, we review chains of derivation in the cognitive neuroscience of language, focusing on how the hypothesis under consideration is defined by a combination of theoretical and methodological assumptions. We first attempt to disentangle the complex relationship between linguistics, psychology, and neuroscience in the field. Next, we focus on how conclusions that can be drawn from any experiment are inherently constrained by auxiliary assumptions, both theoretical and methodological, on which the validity of conclusions drawn rests. These issues are discussed in the context of classical experimental manipulations as well as study designs that employ novel approaches such as naturalistic stimuli and computational modeling. We conclude by proposing that a highly interdisciplinary field such as the cognitive neuroscience of language requires researchers to form explicit statements concerning the theoretical definitions, methodological choices, and other constraining factors involved in their work.

Hierarchical syntactic processing is beyond mere associating: Functional magnetic resonance imaging evidence from a novel artificial grammar.

Grammar is central to any natural language. In the past decades, the artificial grammar of the An Bn type in which a pair of associated elements can be nested in the other pair was considered as a desirable model to mimic human language syntax without semantic interference. However, such a grammar relies on mere associating mechanisms, thus insufficient to reflect the hierarchical nature of human syntax. Here, we test how the brain imposes syntactic hierarchies according to the category relations on linearized sequences by designing a novel artificial "Hierarchical syntactic structure-building Grammar" (HG), and compare this to the An Bn grammar as a "Nested associating Grammar" (NG) based on multilevel associations. Thirty-six healthy German native speakers were randomly assigned to one of the two grammars. Both groups performed a grammaticality judgment task on auditorily presented word sequences generated by the corresponding grammar in the scanner after a successful explicit behavioral learning session. Compared to the NG group, we found that the HG group showed a (a) significantly higher involvement of Brodmann area (BA) 44 in Broca's area and the posterior superior temporal gyrus (pSTG); and (b) qualitatively distinct connectivity between the two regions. Thus, the present study demonstrates that the build-up process of syntactic hierarchies on the basis of category relations critically relies on a distinctive left-hemispheric syntactic network involving BA 44 and pSTG. This indicates that our novel artificial grammar can constitute a suitable experimental tool to investigate syntax-specific processes in the human brain.

Functional neuroanatomy of language without speech: An ALE meta-analysis of sign language.

Sign language (SL) conveys linguistic information using gestures instead of sounds. Here, we apply a meta-analytic estimation approach to neuroimaging studies (N = 23; subjects = 316) and ask whether SL comprehension in deaf signers relies on the same primarily left-hemispheric cortical network implicated in spoken and written language (SWL) comprehension in hearing speakers. We show that: (a) SL recruits bilateral fronto-temporo-occipital regions with strong left-lateralization in the posterior inferior frontal gyrus known as Broca's area, mirroring functional asymmetries observed for SWL. (b) Within this SL network, Broca's area constitutes a hub which attributes abstract linguistic information to gestures. (c) SL-specific voxels in Broca's area are also crucially involved in SWL, as confirmed by meta-analytic connectivity modeling using an independent large-scale neuroimaging database. This strongly suggests that the human brain evolved a lateralized language network with a supramodal hub in Broca's area which computes linguistic information independent of speech.

Language and action in Broca's area: Computational differentiation and cortical segregation.

Actions have been proposed to follow hierarchical principles similar to those hypothesized for language syntax. These structural similarities are claimed to be reflected in the common involvement of certain neural populations of Broca's area, in the Inferior Frontal Gyrus (IFG). In this position paper, we follow an influential hypothesis in linguistic theory to introduce the syntactic operation Merge and the corresponding motor/conceptual interfaces. We argue that actions hierarchies do not follow the same principles ruling language syntax. We propose that hierarchy in the action domain lies in predictive processing mechanisms mapping sensory inputs and statistical regularities of action-goal relationships. At the cortical level, distinct Broca's subregions appear to support different types of computations across the two domains. We argue that anterior BA44 is a major hub for the implementation of the syntactic operation Merge. On the other hand, posterior BA44 is recruited in selecting premotor mental representations based on the information provided by contextual signals. This functional distinction is corroborated by a recent meta-analysis (Papitto, Friederici, & Zaccarella, 2020). We conclude by suggesting that action and language can meet only where the interfaces transfer abstract computations either to the external world or to the internal mental world.

Psycholinguistic norms for more than 300 lexical signs in German Sign Language (DGS).

Sign language offers a unique perspective on the human faculty of language by illustrating that linguistic abilities are not bound to speech and writing. In studies of spoken and written language processing, lexical variables such as, for example, age of acquisition have been found to play an important role, but such information is not as yet available for German Sign Language (Deutsche Gebärdensprache, DGS). Here, we present a set of norms for frequency, age of acquisition, and iconicity for more than 300 lexical DGS signs, derived from subjective ratings by 32 deaf signers. We also provide additional norms for iconicity and transparency for the same set of signs derived from ratings by 30 hearing non-signers. In addition to empirical norming data, the dataset includes machine-readable information about a sign's correspondence in German and English, as well as annotations of lexico-semantic and phonological properties: one-handed vs. two-handed, place of articulation, most likely lexical class, animacy, verb type, (potential) homonymy, and potential dialectal variation. Finally, we include information about sign onset and offset for all stimulus clips from automated motion-tracking data. All norms, stimulus clips, data, as well as code used for analysis are made available through the Open Science Framework in the hope that they may prove to be useful to other researchers: https://doi.org/10.17605/OSF.IO/MZ8J4.

The topographical organization of motor processing: An ALE meta-analysis on six action domains and the relevance of Broca's region.

Action is a cover term used to refer to a large set of motor processes differing in domain specificities (e.g. execution or observation). Here we review neuroimaging evidence on action processing (N = 416; Subjects = 5912) using quantitative Activation Likelihood Estimation (ALE) and Meta-Analytic Connectivity Modeling (MACM) approaches to delineate the functional specificities of six domains: (1) Action Execution, (2) Action Imitation, (3) Motor Imagery, (4) Action Observation, (5) Motor Learning, (6) Motor Preparation. Our results show distinct functional patterns for the different domains with convergence in posterior BA44 (pBA44) for execution, imitation and imagery processing. The functional connectivity network seeding in the motor-based localized cluster of pBA44 differs from the connectivity network seeding in the (language-related) anterior BA44. The two networks implement distinct cognitive functions. We propose that the motor-related network encompassing pBA44 is recruited when processing movements requiring a mental representation of the action itself.

Universal neural basis of structure building evidenced by network modulations emerging from Broca's area: The case of Chinese.

The basic steps in building up language involve binding words of different categories into a hierarchical structure. To what extent these steps are universal or differ across languages is an open issue. Here we examine the neural dynamics of phrase structure building in Chinese-a language that in contrast to other languages heavily depends on contextual semantic information. We used functional magnetic resonance imaging and dynamic causal modeling to identify the relevant brain regions and their dynamic relations. Language stimuli consisted of syntax-driving determiners, semantics-embedded classifiers, and nonverbal symbols making up for two-component sequences manipulated by the factors structure (phrase/list) and number of words (2-word/1-word). Processing phrases compared with word lists elicited greater activation in the anterior part of Broca's area, Brodmann area (BA) 45, and the left posterior superior/middle temporal gyri (pSTG/pMTG), while processing two words against one word led to stronger involvement of the left BA 45, BA 44, and insula. Differential network modulations emerging from subparts of Broca's area revealed that phrasal construction in particular highly modulated the direct connection from BA 44 to left pMTG, suggesting BA 44's primary role in phrase structure building. Conversely, the involvement of BA 45 rather appears sensitive to the reliance on lexico-semantic information in Chinese. Against the background of previous findings from other languages, the present results indicate that phrase structure building has a universal neural basis within the left fronto-temporal network. Most importantly, they provide the first evidence demonstrating that the structure-building network may be modulated by language-specific characteristics.

Building by Syntax: The Neural Basis of Minimal Linguistic Structures.

Language comes in utterances in which words are bound together according to a simple rule-based syntactic computation (merge), which creates linguistic hierarchies of potentially infinite length-phrases and sentences. In the current functional magnetic resonance imaging study, we compared prepositional phrases and sentences-both involving merge-to word lists-not involving merge-to explore how this process is implemented in the brain. We found that merge activates the pars opercularis of the left inferior frontal gyrus (IFG; Brodmann Area [BA] 44) and a smaller region in the posterior superior temporal sulcus (pSTS). Within the IFG, sentences engaged a more anterior portion of the area (pars triangularis, BA 45)-compared with phrases-which showed activity peak in BA 44. As prepositional phrases, in contrast to sentences, do not contain verbs, activity in BA 44 may reflect structure-building syntactic processing, while the involvement of BA 45 may reflect the encoding of propositional meaning initiated by the verb. The pSTS appears to work together with the IFG during thematic role assignment not only at the sentential level, but also at the phrasal level. The present results suggest that merge, the process of binding words together into syntactic hierarchies, is primarily supported by BA 44 in the IFG.

Continuous Theta-Burst Stimulation on the Left Posterior Inferior Frontal Gyrus Perturbs Complex Syntactic Processing Stability in Mandarin Chinese.

The structure of human language is inherently hierarchical. The left posterior inferior frontal gyrus (LpIFG) is proposed to be a core region for constructing syntactic hierarchies. However, it remains unclear whether LpIFG plays a causal role in syntactic processing in Mandarin Chinese and whether its contribution depends on syntactic complexity, working memory, or both. We addressed these questions by applying inhibitory continuous theta-burst stimulation (cTBS) over LpIFG. Thirty-two participants processed sentences containing embedded relative clauses (i.e., complex syntactic processing), syntactically simpler coordinated sentences (i.e., simple syntactic processing), and non-hierarchical word lists (i.e., word list processing) after receiving real or sham cTBS. We found that cTBS significantly increased the coefficient of variation, a representative index of processing stability, in complex syntactic processing (esp., when subject relative clause was embedded) but not in the other two conditions. No significant changes in d' and reaction time were detected in these conditions. The findings suggest that (a) inhibitory effect of cTBS on the LpIFG might be prominent in perturbing the complex syntactic processing stability but subtle in altering the processing quality; and (b) the causal role of the LpIFG seems to be specific for syntactic processing rather than working memory capacity, further evidencing their separability in LpIFG. Collectively, these results support the notion of the LpIFG as a core region for complex syntactic processing across languages.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm.

Introduction: Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level. Methods: This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up. Results: Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated. Discussion: Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.

Electrophysiological correlates of basic semantic composition in people with aphasia.

The neuroanatomical correlates of basic semantic composition have been investigated in previous neuroimaging and lesion studies, but research on the electrophysiology of the involved processes is scarce. A large literature on sentence-level event-related potentials (ERPs) during semantic processing has identified at least two relevant components - the N400 and the P600. Other studies demonstrated that these components are reduced and/or delayed in people with aphasia (PWA). However, it remains to be shown if these findings generalize beyond the sentence level. Specifically, it is an open question if an alteration in ERP responses in PWA can also be observed during basic semantic composition, providing a potential future diagnostic tool. The present study aimed to elucidate the electrophysiological dynamics of basic semantic composition in a group of post-stroke PWA. We included 20 PWA and 20 age-matched controls (mean age 58 years) and measured ERP responses while they performed a plausibility judgment task on two-word phrases that were either meaningful ("anxious horse"), anomalous ("anxious wood") or had the noun replaced by a pseudoword ("anxious gufel"). The N400 effect for anomalous versus meaningful phrases was similar in both groups. In contrast, unlike the control group, PWA did not show an N400 effect between pseudoword and meaningful phrases. Moreover, both groups exhibited a parietal P600 effect towards pseudoword phrases, while PWA showed an additional P600 over frontal electrodes. Finally, PWA showed an inverse correlation between the magnitude of the N400 and P600 effects: PWA exhibiting no or even reversed N400 effects towards anomalous and pseudoword phrases showed a stronger P600 effect. These results may reflect a compensatory mechanism which allows PWA to arrive at the correct interpretation of the phrase. When compositional processing capacities are impaired in the early N400 time-window, PWA may make use of a more elaborate re-analysis process reflected in the P600.

Syntax through the looking glass: A review on two-word linguistic processing across behavioral, neuroimaging and neurostimulation studies.

In recent years a growing number of studies on syntactic processing has employed basic two-word constructions (e.g., "the tree") to characterize the fundamental aspects of linguistic composition. This large body of evidence allows, for the first time, to closely examine which cognitive processes and neural substrates support the combination of two syntactic units into a more complex one, mirroring the nature of combinatory operations described in theoretical linguistics. The present review comprehensively examines behavioral, neuroimaging and neurostimulation studies investigating basic syntactic composition, covering more than forty years of psycho- and neuro-linguistic research. Across several paradigms, four key features of syntactic composition have emerged: (1) the rule-based and (2) automatic nature of the combinatorial process, (3) a central role of Broca's area and the posterior temporal lobe in representing and combining syntactic features, and (4) the reliance on efficient bottom-up integration rather than top-down prediction.

Online neurostimulation of Broca's area does not interfere with syntactic predictions: A combined TMS-EEG approach to basic linguistic combination.

Categorical predictions have been proposed as the key mechanism supporting the fast pace of syntactic composition in language. Accordingly, grammar-based expectations are formed-e.g., the determiner "a" triggers the prediction for a noun-and facilitate the analysis of incoming syntactic information, which is then checked against a single or few other word categories. Previous functional neuroimaging studies point towards Broca's area in the left inferior frontal gyrus (IFG) as one fundamental cortical region involved in categorical prediction during incremental language processing. Causal evidence for this hypothesis is however still missing. In this study, we combined Electroencephalography (EEG) and Transcranial Magnetic Stimulation (TMS) to test whether Broca's area is functionally relevant in predictive mechanisms for language. We transiently perturbed Broca's area during the first word in a two-word construction, while simultaneously measuring the Event-Related Potential (ERP) correlates of syntactic composition. We reasoned that if Broca's area is involved in predictive mechanisms for syntax, disruptive TMS during the first word would mitigate the difference in the ERP responses for predicted and unpredicted categories in basic two-word constructions. Contrary to this hypothesis, perturbation of Broca's area at the predictive stage did not affect the ERP correlates of basic composition. The correlation strength between the electrical field induced by TMS and the ERP responses further confirmed this pattern. We discuss the present results considering an alternative account of the role of Broca's area in syntactic composition, namely the bottom-up integration of words into constituents, and of compensatory mechanisms within the language predictive network.

Neural classification maps for distinct word combinations in Broca's area.

Humans are equipped with the remarkable ability to comprehend an infinite number of utterances. Relations between grammatical categories restrict the way words combine into phrases and sentences. How the brain recognizes different word combinations remains largely unknown, although this is a necessary condition for combinatorial unboundedness in language. Here, we used functional magnetic resonance imaging and multivariate pattern analysis to explore whether distinct neural populations of a known language network hub-Broca's area-are specialized for recognizing distinct simple word combinations. The phrases consisted of a noun (flag) occurring either with a content word, an adjective (green flag), or with a function word, a determiner (that flag). The key result is that the distribution of neural populations classifying word combination in Broca's area seems sensitive to neuroanatomical subdivisions within this area, irrespective of task. The information patterns for adjective + noun were localized in its anterior part (BA45) whereas those for determiner + noun were localized in its posterior part (BA44). Our findings provide preliminary answers to the fundamental question of how lexical and grammatical category information interact during simple word combination, with the observation that Broca's area is sensitive to the recognition of categorical relationships during combinatory processing, based on different demands placed on syntactic and semantic information. This supports the hypothesis that the combinatorial power of language consists of some neural computation capturing phrasal differences when processing linguistic input.

Chimpanzees produce diverse vocal sequences with ordered and recombinatorial properties.

The origins of human language remains a major question in evolutionary science. Unique to human language is the capacity to flexibly recombine a limited sound set into words and hierarchical sequences, generating endlessly new sentences. In contrast, sequence production of other animals appears limited, stunting meaning generation potential. However, studies have rarely quantified flexibility and structure of vocal sequence production across the whole repertoire. Here, we used such an approach to examine the structure of vocal sequences in chimpanzees, known to combine calls used singly into longer sequences. Focusing on the structure of vocal sequences, we analysed 4826 recordings of 46 wild adult chimpanzees from Taï National Park. Chimpanzees produced 390 unique vocal sequences. Most vocal units emitted singly were also emitted in two-unit sequences (bigrams), which in turn were embedded into three-unit sequences (trigrams). Bigrams showed positional and transitional regularities within trigrams with certain bigrams predictably occurring in either head or tail positions in trigrams, and predictably co-occurring with specific other units. From a purely structural perspective, the capacity to organize single units into structured sequences offers a versatile system potentially suitable for expansive meaning generation. Further research must show to what extent these structural sequences signal predictable meanings.

Controlling Video Stimuli in Sign Language and Gesture Research: The OpenPoseR Package for Analyzing OpenPose Motion-Tracking Data in R.

Researchers in the fields of sign language and gesture studies frequently present their participants with video stimuli showing actors performing linguistic signs or co-speech gestures. Up to now, such video stimuli have been mostly controlled only for some of the technical aspects of the video material (e.g., duration of clips, encoding, framerate, etc.), leaving open the possibility that systematic differences in video stimulus materials may be concealed in the actual motion properties of the actor's movements. Computer vision methods such as OpenPose enable the fitting of body-pose models to the consecutive frames of a video clip and thereby make it possible to recover the movements performed by the actor in a particular video clip without the use of a point-based or markerless motion-tracking system during recording. The OpenPoseR package provides a straightforward and reproducible way of working with these body-pose model data extracted from video clips using OpenPose, allowing researchers in the fields of sign language and gesture studies to quantify the amount of motion (velocity and acceleration) pertaining only to the movements performed by the actor in a video clip. These quantitative measures can be used for controlling differences in the movements of an actor in stimulus video clips or, for example, between different conditions of an experiment. In addition, the package also provides a set of functions for generating plots for data visualization, as well as an easy-to-use way of automatically extracting metadata (e.g., duration, framerate, etc.) from large sets of video files.

Differential contributions of left-hemispheric language regions to basic semantic composition.

Semantic composition, the ability to combine single words to form complex meanings, is a core feature of human language. Despite growing interest in the basis of semantic composition, the neural correlates and the interaction of regions within this network remain a matter of debate. We designed a well-controlled two-word fMRI paradigm in which phrases only differed along the semantic dimension while keeping syntactic information alike. Healthy participants listened to meaningful ("fresh apple"), anomalous ("awake apple") and pseudoword phrases ("awake gufel") while performing an implicit and an explicit semantic task. We identified neural signatures for distinct processes during basic semantic composition. When lexical information is kept constant across conditions and the evaluation of phrasal plausibility is examined (meaningful vs. anomalous phrases), a small set of mostly left-hemispheric semantic regions, including the anterior part of the left angular gyrus, is found active. Conversely, when the load of lexical information-independently of phrasal plausibility-is varied (meaningful or anomalous vs. pseudoword phrases), conceptual combination involves a wide-spread left-hemispheric network comprising executive semantic control regions and general conceptual representation regions. Within this network, the functional coupling between the left anterior inferior frontal gyrus, the bilateral pre-supplementary motor area and the posterior angular gyrus specifically increases for meaningful phrases relative to pseudoword phrases. Stronger effects in the explicit task further suggest task-dependent neural recruitment. Overall, we provide a separation between distinct nodes of the semantic network, whose functional contributions depend on the type of compositional process under analysis.

Dissociable contributions of frontal and temporal brain regions to basic semantic composition.

Semantic composition is the ability to combine single words to form complex meanings and is an essential component for successful communication. Evidence from neuroimaging studies suggests that semantic composition engages a widely distributed left-hemispheric network, including the anterior temporal lobe, the inferior frontal gyrus and the angular gyrus. To date, the functional relevance of these regions remains unclear. Here, we investigate the impact of lesions to key regions in the semantic network on basic semantic composition. We conducted a multivariate lesion-behaviour mapping study in 36 native German speaking participants with chronic lesions to the language network after left-hemispheric stroke. During the experiment, participants performed a plausibility judgement task on auditorily presented adjective-noun phrases that were either meaningful ('anxious horse'), anomalous ('anxious salad') or had the noun replaced by a pseudoword ('anxious gufel'), as well as a single-word control condition ('horse'). We observed that reduced accuracy for anomalous phrases is associated with lesions in left anterior inferior frontal gyrus, whereas increased reaction times for anomalous phrases correlates with lesions in anterior-to-mid temporal lobe. These results indicate that anterior inferior frontal gyrus is relevant for accurate semantic decisions, while anterior-to-mid temporal lobe lesions lead to slowing of the decision for anomalous two-word phrases. These differential effects of lesion location support the notion that anterior inferior frontal gyrus affords executive control for decisions on semantic composition while anterior-to-mid temporal lobe lesions slow the semantic processing of the individual constituents of the phrase.