Visual recognition of words learned with gestures induces motor resonance in the forearm muscles.

According to theories of Embodied Cognition, memory for words is related to sensorimotor experiences collected during learning. At a neural level, words encoded with self-performed gestures are represented in distributed sensorimotor networks that resonate during word recognition. Here, we ask whether muscles involved in gesture execution also resonate during word recognition. Native German speakers encoded words by reading them (baseline condition) or by reading them in tandem with picture observation, gesture observation, or gesture observation and execution. Surface electromyogram (EMG) activity from both arms was recorded during the word recognition task and responses were detected using eye-tracking. The recognition of words encoded with self-performed gestures coincided with an increase in arm muscle EMG activity compared to the recognition of words learned under other conditions. This finding suggests that sensorimotor networks resonate into the periphery and provides new evidence for a strongly embodied view of recognition memory.

Motor Cortex Causally Contributes to Vocabulary Translation following Sensorimotor-Enriched Training.

The role of the motor cortex in perceptual and cognitive functions is highly controversial. Here, we investigated the hypothesis that the motor cortex can be instrumental for translating foreign language vocabulary. Human participants of both sexes were trained on foreign language (L2) words and their native language translations over 4 consecutive days. L2 words were accompanied by complementary gestures (sensorimotor enrichment) or pictures (sensory enrichment). Following training, participants translated the auditorily presented L2 words that they had learned. During translation, repetitive transcranial magnetic stimulation was applied bilaterally to a site within the primary motor cortex (Brodmann area 4) located in the vicinity of the arm functional compartment. Responses within the stimulated motor region have previously been found to correlate with behavioral benefits of sensorimotor-enriched L2 vocabulary learning. Compared to sham stimulation, effective perturbation by repetitive transcranial magnetic stimulation slowed down the translation of sensorimotor-enriched L2 words, but not sensory-enriched L2 words. This finding suggests that sensorimotor-enriched training induced changes in L2 representations within the motor cortex, which in turn facilitated the translation of L2 words. The motor cortex may play a causal role in precipitating sensorimotor-based learning benefits, and may directly aid in remembering the native language translations of foreign language words following sensorimotor-enriched training. These findings support multisensory theories of learning while challenging reactivation-based theories.SIGNIFICANCE STATEMENT Despite the potential for sensorimotor enrichment to serve as a powerful tool for learning in many domains, its underlying brain mechanisms remain largely unexplored. Using transcranial magnetic stimulation and a foreign language (L2) learning paradigm, we found that sensorimotor-enriched training can induce changes in L2 representations within the motor cortex, which in turn causally facilitate the translation of L2 words. The translation of recently acquired L2 words may therefore rely not only on auditory information stored in memory or on modality-independent L2 representations, but also on the sensorimotor context in which the words have been experienced.

Visual Sensory Cortices Causally Contribute to Auditory Word Recognition Following Sensorimotor-Enriched Vocabulary Training.

Despite a rise in the use of "learning by doing" pedagogical methods in praxis, little is known as to how the brain benefits from these methods. Learning by doing strategies that utilize complementary information ("enrichment") such as gestures have been shown to optimize learning outcomes in several domains including foreign language (L2) training. Here we tested the hypothesis that behavioral benefits of gesture-based enrichment are critically supported by integrity of the biological motion visual cortices (bmSTS). Prior functional neuroimaging work has implicated the visual motion cortices in L2 translation following sensorimotor-enriched training; the current study is the first to investigate the causal relevance of these structures in learning by doing contexts. Using neuronavigated transcranial magnetic stimulation and a gesture-enriched L2 vocabulary learning paradigm, we found that the bmSTS causally contributed to behavioral benefits of gesture-enriched learning. Visual motion cortex integrity benefitted both short- and long-term learning outcomes, as well as the learning of concrete and abstract words. These results adjudicate between opposing predictions of two neuroscientific learning theories: While reactivation-based theories predict no functional role of specialized sensory cortices in vocabulary learning outcomes, the current study supports the predictive coding theory view that these cortices precipitate sensorimotor-based learning benefits.

Embodied Learning: Why at School the Mind Needs the Body.

Despite all methodological efforts made in the last three decades, Western instruction grounds on traditional principles. Most educational programs follow theories that are mentalistic, i.e., they separate the mind from the body. At school, learners sit, watch, listen, and write. The aim of this paper is to present embodied learning as an alternative to mentalistic education. Similarly, this paper wants to describe embodied learning from a neuroscientific perspective. After a brief historical overview, I will review studies highlighting the behavioral effectiveness of embodied instruction in second language learning, mathematics and spatial thinking. On this base, I will discuss some of the brain mechanisms driving embodied learning and describe its advantages, clearly pleading in favor of instructional practice that reunites body and mind.

Depth of Encoding Through Observed Gestures in Foreign Language Word Learning.

Word learning is basic to foreign language acquisition, however time consuming and not always successful. Empirical studies have shown that traditional (visual) word learning can be enhanced by gestures. The gesture benefit has been attributed to depth of encoding. Gestures can lead to depth of encoding because they trigger semantic processing and sensorimotor enrichment of the novel word. However, the neural underpinning of depth of encoding is still unclear. Here, we combined an fMRI and a behavioral study to investigate word encoding online. In the scanner, participants encoded 30 novel words of an artificial language created for experimental purposes and their translation into the subjects' native language. Participants encoded the words three times: visually, audiovisually, and by additionally observing semantically related gestures performed by an actress. Hemodynamic activity during word encoding revealed the recruitment of cortical areas involved in stimulus processing. In this study, depth of encoding can be spelt out in terms of sensorimotor brain networks that grow larger the more sensory modalities are linked to the novel word. Word retention outside the scanner documented a positive effect of gestures in a free recall test in the short term.

Guided Embodiment and Potential Applications of Tutor Systems in Language Instruction and Rehabilitation.

Intelligent tutor systems (ITSs) in mobile devices take us through learning tasks and make learning ubiquitous, autonomous, and at low cost (Nye, 2015). In this paper, we describe guided embodiment as an ITS essential feature for second language learning (L2) and aphasia rehabilitation (ARe) that enhances efficiency in the learning process. In embodiment, cognitive processes, here specifically language (re)learning are grounded in actions and gestures (Pecher and Zwaan, 2005; Fischer and Zwaan, 2008; Dijkstra and Post, 2015). In order to guide users through embodiment, ITSs must track action and gesture, and give corrective feed-back to achieve the users' goals. Therefore, sensor systems are essential to guided embodiment. In the next sections, we describe sensor systems that can be implemented in ITS for guided embodiment.

Recently learned foreign abstract and concrete nouns are represented in distinct cortical networks similar to the native language.

In the native language, abstract and concrete nouns are represented in distinct areas of the cerebral cortex. Currently, it is unknown whether this is also the case for abstract and concrete nouns of a foreign language. Here, we taught adult native speakers of German 45 abstract and 45 concrete nouns of a foreign language. After learning the nouns for 5 days, participants performed a vocabulary translation task during functional magnetic resonance imaging. Translating abstract nouns in contrast to concrete nouns elicited responses in regions that are also responsive to abstract nouns in the native language: the left inferior frontal gyrus and the left middle and superior temporal gyri. Concrete nouns elicited larger responses in the angular gyri bilaterally and the left parahippocampal gyrus than abstract nouns. The cluster in the left angular gyrus showed psychophysiological interaction (PPI) with the left lingual gyrus. The left parahippocampal gyrus showed PPI with the posterior cingulate cortex. Similar regions have been previously found for concrete nouns in the native language. The results reveal similarities in the cortical representation of foreign language nouns with the representation of native language nouns that already occur after 5 days of vocabulary learning. Furthermore, we showed that verbal and enriched learning methods were equally suitable to teach foreign abstract and concrete nouns. Hum Brain Mapp 38:4398-4412, 2017. © 2017 Wiley Periodicals, Inc.

Enrichment Effects of Gestures and Pictures on Abstract Words in a Second Language.

Laboratory research has demonstrated that multisensory enrichment promotes verbal learning in a foreign language (L2). Enrichment can be done in various ways, e.g., by adding a picture that illustrates the L2 word's meaning or by the learner performing a gesture to the word (enactment). Most studies have tested enrichment on concrete but not on abstract words. Unlike concrete words, the representation of abstract words is deprived of sensory-motor features. This has been addressed as one of the reasons why abstract words are difficult to remember. Here, we ask whether a brief enrichment training by means of pictures and by self-performed gestures also enhances the memorability of abstract words in L2. Further, we explore which of these two enrichment strategies is more effective. Twenty young adults learned 30 novel abstract words in L2 according to three encoding conditions: (1) reading, (2) reading and pairing the novel word to a picture, and (3) reading and enacting the word by means of a gesture. We measured memory performance in free and cued recall tests, as well as in a visual recognition task. Words encoded with gestures were better remembered in the free recall in the native language (L1). When recognizing the novel words, participants made less errors for words encoded with gestures compared to words encoded with pictures. The reaction times in the recognition task did not differ across conditions. The present findings support, even if only partially, the idea that enactment promotes learning of abstract words and that it is superior to enrichment by means of pictures even after short training.

Exploring the Neural Representation of Novel Words Learned through Enactment in a Word Recognition Task.

Vocabulary learning in a second language is enhanced if learners enrich the learning experience with self-performed iconic gestures. This learning strategy is called enactment. Here we explore how enacted words are functionally represented in the brain and which brain regions contribute to enhance retention. After an enactment training lasting 4 days, participants performed a word recognition task in the functional Magnetic Resonance Imaging (fMRI) scanner. Data analysis suggests the participation of different and partially intertwined networks that are engaged in higher cognitive processes, i.e., enhanced attention and word recognition. Also, an experience-related network seems to map word representation. Besides core language regions, this latter network includes sensory and motor cortices, the basal ganglia, and the cerebellum. On the basis of its complexity and the involvement of the motor system, this sensorimotor network might explain superior retention for enactment.

Visual and motor cortices differentially support the translation of foreign language words.

At present, it is largely unclear how the human brain optimally learns foreign languages. We investigated teaching strategies that utilize complementary information ("enrichment"), such as pictures or gestures, to optimize vocabulary learning outcome. We found that learning while performing gestures was more efficient than the common practice of learning with pictures and that both enrichment strategies were better than learning without enrichment ("verbal learning"). We tested the prediction of an influential cognitive neuroscience theory that provides explanations for the beneficial behavioral effects of enrichment: the "multisensory learning theory" attributes the benefits of enrichment to recruitment of brain areas specialized in processing the enrichment. To test this prediction, we asked participants to translate auditorily presented foreign words during fMRI. Multivariate pattern classification allowed us to decode from the brain activity under which enrichment condition the vocabulary had been learned. The visual-object-sensitive lateral occipital complex (LOC) represented auditory words that had been learned with pictures. The biological motion superior temporal sulcus (bmSTS) and motor areas represented auditory words that had been learned with gestures. Importantly, brain activity in these specialized visual and motor brain areas correlated with behavioral performance. The cortical activation pattern found in the present study strongly supports the multisensory learning theory in contrast to alternative explanations. In addition, the results highlight the importance of learning foreign language vocabulary with enrichment, particularly with self-performed gestures.

Bringing back the body into the mind: gestures enhance word learning in foreign language.

Foreign language education in the twenty-first century still teaches vocabulary mainly through reading and listening activities. This is due to the link between teaching practice and traditional philosophy of language, where language is considered to be an abstract phenomenon of the mind. However, a number of studies have shown that accompanying words or phrases of a foreign language with gestures leads to better memory results. In this paper, I review behavioral research on the positive effects of gestures on memory. Then I move to the factors that have been addressed as contributing to the effect, and I embed the reviewed evidence in the theoretical framework of embodiment. Finally, I argue that gestures accompanying foreign language vocabulary learning create embodied representations of those words. I conclude by advocating the use of gestures in future language education as a learning tool that enhances the mind.

Intelligent virtual agents as language trainers facilitate multilingualism.

IN THIS PAPER WE INTRODUCE A NEW GENERATION OF LANGUAGE TRAINERS: intelligent virtual agents (IVAs) with human appearance and the capability to teach foreign language vocabulary. We report results from studies that we have conducted with Billie, an IVA employed as a vocabulary trainer, as well as research findings on the acceptance of the agent as a trainer by adults and children. The results show that Billie can train humans as well as a human teacher can and that both adults and children accept the IVA as a trainer. The advantages of IVAs are multiple. First, their teaching methods can be based on neuropsychological research findings concerning memory and learning practice. Second, virtual teachers can provide individualized training. Third, they coach users during training, are always supportive, and motivate learners to train. Fourth, agents will reside in the user's mobile devices and thus be at the user's disposal everywhere and anytime. Agents in apps will make foreign language training accessible to anybody at low cost. This will enable people around the world, including physically, financially, and geographically disadvantaged persons, to learn a foreign language and help to facilitate multilingualism.

The impact of iconic gestures on foreign language word learning and its neural substrate.

Vocabulary acquisition represents a major challenge in foreign language learning. Research has demonstrated that gestures accompanying speech have an impact on memory for verbal information in the speakers' mother tongue and, as recently shown, also in foreign language learning. However, the neural basis of this effect remains unclear. In a within-subjects design, we compared learning of novel words coupled with iconic and meaningless gestures. Iconic gestures helped learners to significantly better retain the verbal material over time. After the training, participants' brain activity was registered by means of fMRI while performing a word recognition task. Brain activations to words learned with iconic and with meaningless gestures were contrasted. We found activity in the premotor cortices for words encoded with iconic gestures. In contrast, words encoded with meaningless gestures elicited a network associated with cognitive control. These findings suggest that memory performance for newly learned words is not driven by the motor component as such, but by the motor image that matches an underlying representation of the word's semantics.