Embodied Processing at Six Linguistic Granularity Levels: A Consensus Paper.

Language processing is influenced by sensorimotor experiences. Here, we review behavioral evidence for embodied and grounded influences in language processing across six linguistic levels of granularity. We examine (a) sub-word features, discussing grounded influences on iconicity (systematic associations between word form and meaning); (b) words, discussing boundary conditions and generalizations for the simulation of color, sensory modality, and spatial position; (c) sentences, discussing boundary conditions and applications of action direction simulation; (d) texts, discussing how the teaching of simulation can improve comprehension in beginning readers; (e) conversations, discussing how multi-modal cues improve turn taking and alignment; and (f) text corpora, discussing how distributional semantic models can reveal how grounded and embodied knowledge is encoded in texts. These approaches are converging on a convincing account of the psychology of language, but at the same time, there are important criticisms of the embodied approach and of specific experimental paradigms. The surest way forward requires the adoption of a wide array of scientific methods. By providing complimentary evidence, a combination of multiple methods on various levels of granularity can help us gain a more complete understanding of the role of embodiment and grounding in language processing.

Embodiment and learning of abstract concepts (such as algebraic topology and regression to the mean).

This video is a proof of concept that ideas from embodied cognition can be used to understand how the brain and cognitive systems deal with very abstract concepts. The video teaches regression to the mean using three ideas. The first idea is directly related to embodied cognition: abstract concepts are grounded in perceptual, motor, and emotional systems by using successive levels of grounding within an extended procedure. The second idea is that this sort of grounding often requires formal instruction: a teacher needs to develop the sequence in which the concepts are grounded and the methods of grounding. That is, at least some abstract concepts are unlikely to be learned through an individual's unstructured interactions with the world. The third idea is that humans are hyper-social, thus making formal instruction possible. To the extent that the viewer learns the abstract concept of regression to the mean, then the video demonstrates how an embodied theory of abstract concepts could work.

Grounding (fairly) complex numerical knowledge: an educational example.

In this article, we contextualize and discuss an on-line contribution to this special issue in which a video-recorded lecture demonstrates the teaching of an abstract mathematical concept, namely regression to the mean. We first motivate the pertinence of this example from the perspective of embodied cognition. Then, we identify mechanisms of teaching that reflect embodied cognitive practices, such as the concreteness fading approach. Rather than a comprehensive review of multiple extensive literatures, this article provides the interested reader with several sources or entries into those literatures.

A pre-registered, multi-lab non-replication of the action-sentence compatibility effect (ACE).

The Action-sentence Compatibility Effect (ACE) is a well-known demonstration of the role of motor activity in the comprehension of language. Participants are asked to make sensibility judgments on sentences by producing movements toward the body or away from the body. The ACE is the finding that movements are faster when the direction of the movement (e.g., toward) matches the direction of the action in the to-be-judged sentence (e.g., Art gave you the pen describes action toward you). We report on a pre-registered, multi-lab replication of one version of the ACE. The results show that none of the 18 labs involved in the study observed a reliable ACE, and that the meta-analytic estimate of the size of the ACE was essentially zero.

The gleam-glum effect: /i:/ versus /λ/ phonemes generically carry emotional valence.

The gleam-glum effect is a novel sound symbolic finding that words with the /i:/-phoneme (like gleam) are perceived more positive emotionally than matched words with the /Λ/-phoneme (like glum). We provide data that not only confirm the effect but also are consistent with an explanation that /i:/ and /Λ/ articulation tend to co-occur with activation of positive versus negative emotional facial musculature respectively. Three studies eliminate selection bias by including all applicable English words from the English Lexicon Project (Balota et al., 2007) and the Warriner et al. (2013) database and every possible Mandarin Pinyin combination that differ only in the middle phoneme (/i:/ vs /Λ/). In Study 1, 61 U.S. undergraduates rated monosyllabic English /i:/ words as robustly more positive than matched /Λ/ words. Study 2 analyzed the Warriner et al. (2013) valence ratings, extending the gleam-glum effect to all applicable words in the database. In Study 3, 38 U.S. participants (using English) and 37 participants in China (using Mandarin Pinyin) rated word pairs under three conditions that moderate musculature activity: Read aloud (Enhance), read silently (Control), and read silently while chewing gum (Interfere). Indeed, the effect was both replicated and was significantly larger when facial musculature was enhanced than when interfered with, and the two language populations did not significantly differ. These findings confirm a robust gleam-glum effect, despite semantic noise, in English and Mandarin Pinyin. Furthermore, these data are consistent with the hypothesis that this type of sound symbolism arises from the overlap in muscles used both in articulation and emotion expression. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Culture, ecology, and grounded procedures.

We propose that grounded procedures may help explain psychological variations across cultures. Here we offer a set of novel predictions based on the interplay between the social and physical ecology, chronic sensorimotor experience, and cultural norms.

Moved by Reading in a Spanish-Speaking, Dual Language Learner Population.

Purpose: The aim of this study was to evaluate the effectiveness of an English-only version and a Spanish-support version of an embodied reading comprehension intervention (Moved by Reading) consisting of 3 stages (physical manipulation, imagined manipulation, and transfer) for Spanish-English dual language learners. Method: Sixty-one dual language learners in Arizona were randomly assigned to 4 groups (Spanish-support control, Spanish-support intervention, English-only control, and English-only intervention). Analyses of variance were used to compare control and intervention groups and to compare groups according to the language of the intervention. Results: Children in the Spanish-support intervention group significantly outperformed both control groups during the physical manipulation stage, whereas children in the English-only intervention group outperformed both control groups in the imagined manipulation stage, but there was little transfer to a new, unrelated text. Conclusions: The Moved by Reading intervention, in both its English-only and Spanish-support versions, improved performance on comprehension questions, but in different stages of the intervention. The Spanish-support version of the intervention was most effective in the physical manipulation stage, whereas the English-only version was more effective in the imagined manipulation stage. Neither version was effective in producing significant transfer effects.

Retrieving Against the Flow: Incoherence Between Optic Flow and Movement Direction Has Little Effect on Memory for Order.

Research from multiple areas in neuroscience suggests a link between self-locomotion and memory. In two free recall experiments with adults, we looked for a link between (a) memory, and (b) the coherence of movement and optic flow. In both experiments, participants heard lists of words while on a treadmill and wearing a virtual reality (VR) headset. In the first experiment, the VR scene and treadmill were stationary during encoding. During retrieval, all participants walked forward, but the VR scene was stationary, moved forward, or moved backwards. In the second experiment, during encoding all participants walked forward and viewed a forward-moving VR scene. During retrieval, all participants continued to walk forward but the VR scene was stationary, forward-moving, or backward-moving. In neither experiment was there a significant difference in the amount recalled, or output order strategies, attributable to differences in movement conditions. Thus, any effects of movement on memory are more limited than theories of hippocampal function and theories in cognitive psychology anticipate.

Contribution of Embodiment to Solving the Riddle of Infantile Amnesia.

At least since the late nineteenth century, researchers have sought an explanation for infantile amnesia (IA)-the lack of autobiographical memories dating from early childhood-and childhood amnesia (CA), faster forgetting of events up until the age of about seven. Evidence suggests that IA occurs across altricial species, and a number of studies using animal models have converged on the hypothesis that maturation of the hippocampus is an important factor. But why does the hippocampus mature at one time and not another, and how does that maturation relate to memory? Our hypothesis is rooted in theories of embodied cognition, and it provides an explanation both for hippocampal development and the end of IA. Specifically, the onset of locomotion prompts the alignment of hippocampal place cells and grid cells to the environment, which in turn facilitates the ontogeny of long-term episodic memory and the end of IA. That is, because the animal can now reliably discriminate locations, location becomes a stable cue for memories. Furthermore, as the mode of human locomotion shifts from crawling to walking, there is an additional shift in the alignment of the hippocampus that marks the beginning of adult-like episodic memory and the end of CA. Finally, given a reduction in self-locomotion and exploration with aging, the hypothesis suggests a partial explanation for cognitive decline with aging.

Consequences of joint action: Entanglement with your partner.

Primates are expert tool users because they can adapt their body schemas to form a hand-tool joint representation that affords effective wielding. Here we extend the scope of this mechanism by proposing that humans are experts in social tasks because they can adjust their body schemas to incorporate the kinematics of partners, thus forming an interpersonal joint body schema. Participants engaged with a confederate in a 2-handed sawing task requiring each to use 1 hand. After active movement coordination, an interpersonal body schema was demonstrated in 2 ways. First, there was interference between visual stimulation near 1 person's body and vibrotactile localization on the other person's body. Second, after active movement coordination, the motor output of 1 partner (attempting to draw straight lines) was affected by the viewed actions of the other partner (drawing ovals). This adaptation of the body schema may underlie the formation of cultural groups. In fact, participants with interdependent self-construals (typical of Asian cultures) form a stronger interpersonal joint body schema than do participants with independent self-construals (typical of North American and Western European cultures).

Few believe the world is flat: How embodiment is changing the scientific understanding of cognition.

Science has changed many of our dearly held and commonsensical (but incorrect) beliefs. For example, few still believe the world is flat, and few still believe the sun orbits the earth. Few still believe humans are unrelated to the rest of the animal kingdom, and soon few will believe human thinking is computer-like. Instead, as with all animals, our thoughts are based on bodily experiences, and our thoughts and behaviors are controlled by bodily and neural systems of perception, action, and emotion interacting with the physical and social environments. We are embodied; nothing more. Embodied cognition is about cognition formatted in sensorimotor experience, and sensorimotor systems make those thoughts dynamic. Even processes that seem abstract, such as language comprehension and goal understanding, are embodied. Thus, embodied cognition is not limited to 1 type of thought or another: It is cognition.

Response to Mahon: Unburdening cognition from abstract symbols.

B. Z. Mahon (see record 2015-22897-004) has provided a real service to those researchers investigating human cognition by clearly framing much of the debate between an embodied approach to cognition and an abstract alternative. There are several areas in which Mahon and A. M. Glenberg agree. For example, Glenberg (although not all embodiment theorists) agrees that a major question is the degree to which bodily and neural sensorimotor activity constitutes cognition. In addition, the evidence is clear that there is something akin to a spread of activation amongst systems, and the very fact of this spread makes it difficult to distinguish embodied from nonembodied positions. There are also several areas about which the authors disagree. For example, Mahon suggests that we need to consider predictions of alternative hypotheses. The alternative he offers is that there is a distinction between concepts and sensorimotor activity. Although it is reasonable to consider this alternative, there is no description of what these nonembodied concepts are in Mahon's essay, just that they are not sensorimotor. Glenberg offers a partial compromise to the argument.

Sensory motor mechanisms unify psychology: the embodiment of culture.

Sensorimotor mechanisms can unify explanations at cognitive, social, and cultural levels. As an example, we review how anticipated motor effort is used by individuals and groups to judge distance: the greater the anticipated effort the greater the perceived distance. Anticipated motor effort can also be used to understand cultural differences. People with interdependent self- construals interact almost exclusively with in-group members, and hence there is little opportunity to tune their sensorimotor systems for interaction with out-group members. The result is that interactions with out-group members are expected to be difficult and out-group members are perceived as literally more distant. In two experiments we show (a) interdependent Americans, compared to independent Americans, see American confederates (in-group) as closer; (b) interdependent Arabs, compared to independent Arabs, perceive Arab confederates (in- group) as closer, whereas interdependent Americans perceive Arab confederates (out-group) as farther. These results demonstrate how the same embodied mechanism can seamlessly contribute to explanations at the cognitive, social, and cultural levels.

The motor system contributes to comprehension of abstract language.

If language comprehension requires a sensorimotor simulation, how can abstract language be comprehended? We show that preparation to respond in an upward or downward direction affects comprehension of the abstract quantifiers "more and more" and "less and less" as indexed by an N400-like component. Conversely, the semantic content of the sentence affects the motor potential measured immediately before the upward or downward action is initiated. We propose that this bidirectional link between motor system and language arises because the motor system implements forward models that predict the sensory consequences of actions. Because the same movement (e.g., raising the arm) can have multiple forward models for different contexts, the models can make different predictions depending on whether the arm is raised, for example, to place an object or raised as a threat. Thus, different linguistic contexts invoke different forward models, and the predictions constitute different understandings of the language.

From the Revolution to Embodiment: 25 Years of Cognitive Psychology.

In 1988, the cognitive revolution had become institutionalized: Cognition was the manipulation of abstract symbols by rules. But, much like institutionalized political parties, some of the ideas were becoming stale. Where was action? Where was the self? How could cognition be smoothly integrated with emotions, with social psychology, with development, with clinical analyses? Around that time, thinkers in linguistics, philosophy, artificial intelligence, biology, and psychology were formulating the idea that just as overt behavior depends on the specifics of the body in action, so might cognition depend on the body. Here we characterize (some would say caricature) the strengths and weaknesses of cognitive psychology of that era, and then we describe what has come to be called embodied cognition: how cognition arises through the dynamic interplay of brain controlling bodily action controlling perception, which changes the brain. We focus on the importance of action and how action shapes perception, the self, and language. Having the body in action as a central consideration for theories of cognition promises, we believe, to help unify psychology.

Language comprehension warps the mirror neuron system.

Is the mirror neuron system (MNS) used in language understanding? According to embodied accounts of language comprehension, understanding sentences describing actions makes use of neural mechanisms of action control, including the MNS. Consequently, repeatedly comprehending sentences describing similar actions should induce adaptation of the MNS thereby warping its use in other cognitive processes such as action recognition and prediction. To test this prediction, participants read blocks of multiple sentences where each sentence in the block described transfer of objects in a direction away or toward the reader. Following each block, adaptation was measured by having participants predict the end-point of videotaped actions. The adapting sentences disrupted prediction of actions in the same direction, but (a) only for videos of biological motion, and (b) only when the effector implied by the language (e.g., the hand) matched the videos. These findings are signatures of the MNS.