Experiential grounding of abstract concepts: Processing of abstract mental state concepts engages brain regions involved in mentalizing, automatic speech, and lip movements.

concepts like mental state concepts lack a physical referent, which can be directly perceived. Classical theories therefore claim that abstract concepts require amodal representations detached from experiential brain systems. However, grounded cognition approaches suggest an involvement of modal experiential brain regions in the processing of abstract concepts. In the present functional magnetic resonance imaging study, we investigated the relation of the processing of abstract mental state concepts to modal experiential brain systems in a fine-grained fashion. Participants performed lexical decisions on abstract mental state as well as on verbal association concepts as control category. Experiential brain systems related to the processing of mental states, generating verbal associations, automatic speech as well as hand and lip movements were determined by corresponding localizer tasks. Processing of abstract mental state concepts neuroanatomically overlapped with activity patterns associated with processing of mental states, generating verbal associations, automatic speech and lip movements. Hence, mental state concepts activate the mentalizing brain network, complemented by perceptual-motor brain regions involved in simulation of visual or action features associated with social interactions, linguistic brain regions as well as face-motor brain regions recruited for articulation. The present results provide compelling evidence for the rich grounding of abstract mental state concepts in experiential brain systems related to mentalizing, verbal communication and mouth action.

Academic training increases grounding of scientific concepts in experiential brain systems.

Scientific concepts typically transcendent our sensory experiences. Traditional approaches to science education therefore assume a shift towards amodal or verbal knowledge representations during academic training. Grounded cognition approaches, in contrast, predict a maintenance of grounding of the concepts in experiential brain networks or even an increase. To test these competing approaches, the present study investigated the semantic content of scientific psychological concepts and identified the corresponding neural circuits using functional magnetic resonance imaging (fMRI) in undergraduate psychology students (beginners) and in graduated psychologists (advanced learners). During fMRI scanning, participants were presented with words denoting scientific psychological concepts within a lexical decision task (e.g. "conditioning", "habituation"). The individual semantic property content of each concept was related to brain activity during abstract concept processing. In both beginners and advanced learners, visual and motor properties activated brain regions also involved in perception and action, while mental state properties increased activity in brain regions also recruited by emotional-social scene observation. Only in advanced learners, social constellation properties elicited brain activity overlapping with emotional-social scene observation. In line with grounded cognition approaches, the present results highlight the importance of experiential information for constituting the meaning of abstract scientific concepts during the course of academic training.

How consumption and reward features affect desire for food, consumption intentions, and behaviour.

Understanding the mechanisms that underlie desire and intentions may assist in the search for strategies to promote the selection and consumption of more sustainable and healthier products. Therefore, we conducted two experiments to examine how cognitive representations influence desire and intentions for various savoury dishes. In Experiment 1, 1000 participants were allocated to one of five conditions, listing either the typical, sensory, context, hedonic, or health features of 20 popular dishes to assess cognitive representations, before rating their present moment desire to consume each dish. Although there was no direct effect of condition on desire, there was a significant mediating effect of condition on desire through the proportion of consumption and reward features listed (i.e., sensory, context, and hedonic words). In Experiment 2, 892 participants were allocated to one of four conditions, listing either the typical, sensory, context, or health features for the same 20 dishes, before rating their intention to consume each dish over the next four weeks. At a 4-week follow-up, participants rated how often they had consumed each dish. Again, there was no direct effect of condition on intentions, although there was a significant mediating effect of condition on intentions through consumption and reward features. This suggests that mentally simulating a previous consumption experience increases intentions to consume the dish in mind. The results also showed a positive indirect effect of consumption and reward features on behaviour through an increase in intentions. Describing healthy and sustainable products in terms of the rewarding consumption experience may increase desire and intentions to consume them, improving the health of both people and the planet.

Language with vision: A study on grounded word and sentence embeddings.

Grounding language in vision is an active field of research seeking to construct cognitively plausible word and sentence representations by incorporating perceptual knowledge from vision into text-based representations. Despite many attempts at language grounding, achieving an optimal equilibrium between textual representations of the language and our embodied experiences remains an open field. Some common concerns are the following. Is visual grounding advantageous for abstract words, or is its effectiveness restricted to concrete words? What is the optimal way of bridging the gap between text and vision? To what extent is perceptual knowledge from images advantageous for acquiring high-quality embeddings? Leveraging the current advances in machine learning and natural language processing, the present study addresses these questions by proposing a simple yet very effective computational grounding model for pre-trained word embeddings. Our model effectively balances the interplay between language and vision by aligning textual embeddings with visual information while simultaneously preserving the distributional statistics that characterize word usage in text corpora. By applying a learned alignment, we are able to indirectly ground unseen words including abstract words. A series of evaluations on a range of behavioral datasets shows that visual grounding is beneficial not only for concrete words but also for abstract words, lending support to the indirect theory of abstract concepts. Moreover, our approach offers advantages for contextualized embeddings, such as those generated by BERT (Devlin et al, 2018), but only when trained on corpora of modest, cognitively plausible sizes. Code and grounded embeddings for English are available at ( https://github.com/Hazel1994/Visually_Grounded_Word_Embeddings_2 ).

Grounding of abstract concepts related to power.

Grounded cognition assumes that language and concepts are understood using simulations in different modalities. Evidence for this assumption mainly stems from studies using concrete concepts. Less evidence for grounding exists for abstract concepts, which are assumed to be grounded via metaphors associated with them or via experiences with them in specific situations. In the present study, we developed a new paradigm and investigated grounding of abstract concepts related to power or the exercise of power. As stimulus material, we chose pairs of concepts, for example, democracy and dictatorship. Participants were presented each concept separately and were asked to create a visual image in their mind. Then they were asked to rate images on several aspects. Afterwards they were asked to draw a sketch of the image. Results showed that drawings of high-power concepts had a larger vertical extension than low-power concepts. Results of the questions depended on the specific concepts. For instance, wealth (high-power) was rated as more colorful than poverty (low-power), but democracy (low-power) was rated as more colorful than dictatorship (high-power). These results may partly be explained by the valence of the concepts. Drawings often contained persons, objects, and situations, but were rarely abstract. Sometimes drawings contained metaphorical content and sometimes the content of drawings related to specific experiences. In conclusion, abstract concepts related to power can be depicted visually via grounding in different ways, such as using metaphors, experiences, and actions.

Imagined eating - An investigation of priming and sensory-specific satiety.

While obesity remains a pressing issue, the wider population continues to be exposed to more digital food content than ever before. Much research has demonstrated the priming effect of visual food content, i.e., exposure to food cues increasing appetite and food intake. In contrast, some recent research points out that repeated imagined consumption can facilitate satiate and decrease food intake. Such findings have been suggested as potential remedies to excessive food cue exposure. However, the practically limitless variety of digital food content available today may undermine satiation attempts. The present work aims to replicate and extend prior findings by introducing a within-subjects baseline comparison, disentangling general and (sensory-) specific eating desires, as well as considering the moderating influence of visual and flavour stimulus variety. Three online studies (n = 1149 total) manipulated food colour and flavour variety and reproducibly revealed a non-linear dose-response pattern of imagined eating: 3 repetitions primed, while 30 repetitions satiated. Priming appeared to be specific to the taste of the exposed stimulus, and satiation, contrary to prior literature, appeared to be more general. Neither colour nor flavour variety reliably moderated any of the responses. Therefore, the results suggest that a more pronounced variety may be required to alter imagery-induced satiation.

Sensorimotor distance: A grounded measure of semantic similarity for 800 million concept pairs.

Experimental design and computational modelling across the cognitive sciences often rely on measures of semantic similarity between concepts. Traditional measures of semantic similarity are typically derived from distance in taxonomic databases (e.g. WordNet), databases of participant-produced semantic features, or corpus-derived linguistic distributional similarity (e.g. CBOW), all of which are theoretically problematic in their lack of grounding in sensorimotor experience. We present a new measure of sensorimotor distance between concepts, based on multidimensional comparisons of their experiential strength across 11 perceptual and action-effector dimensions in the Lancaster Sensorimotor Norms. We demonstrate that, in modelling human similarity judgements, sensorimotor distance has comparable explanatory power to other measures of semantic similarity, explains variance in human judgements which is missed by other measures, and does so with the advantages of remaining both grounded and computationally efficient. Moreover, sensorimotor distance is equally effective for both concrete and abstract concepts. We further introduce a web-based tool ( https://lancaster.ac.uk/psychology/smdistance ) for easily calculating and visualising sensorimotor distance between words, featuring coverage of nearly 800 million word pairs. Supplementary materials are available at https://osf.io/d42q6/ .

Quantifying social semantics: An inclusive definition of socialness and ratings for 8388 English words.

It has been proposed that social experience plays an important role in the grounding of concepts, and socialness has been proffered as a fundamental organisational principle underpinning semantic representation in the human brain. However, the empirical support for these hypotheses is limited by inconsistencies in the way socialness has been defined and measured. To further advance theory, the field must establish a clearer working definition, and research efforts could be facilitated by the availability of an extensive set of socialness ratings for individual concepts. Therefore, in the current work, we employed a novel and inclusive definition to test the extent to which socialness is reliably perceived as a broad construct, and we report socialness norms for over 8000 English words, including nouns, verbs, and adjectives. Our inclusive socialness measure shows good reliability and validity, and our analyses suggest that the socialness ratings capture aspects of word meaning which are distinct to those measured by other pertinent semantic constructs, including concreteness and emotional valence. Finally, in a series of regression analyses, we show for the first time that the socialness of a word's meaning explains unique variance in participant performance on lexical tasks. Our dataset of socialness norms has considerable item overlap with those used in both other lexical/semantic norms and in available behavioural mega-studies. They can help target testable predictions about brain and behaviour derived from multiple representation theories and neurobiological accounts of social semantics.

Differential temporo-spatial pattern of electrical brain activity during the processing of abstract concepts related to mental states and verbal associations.

Refined grounded cognition accounts propose that abstract concepts might be grounded in brain circuits involved in mentalizing. In the present event-related potential (ERP) study, we compared the time course of neural processing in response to semantically predefined abstract mental states and verbal association concepts during a lexical decision task. In addition to scalp ERPs, source estimates of underlying volume brain activity were determined to reveal spatio-temporal clusters of greater electrical brain activity to abstract mental state vs. verbal association concepts, and vice versa. Source estimates suggested early (onset 194 ms), but short-lived enhanced activity (offset 210 ms) to verbal association concepts in left occipital regions. Increased occipital activity might reflect retrieval of visual word form or access to visual conceptual features of associated words. Increased estimated source activity to mental state concepts was obtained in visuo-motor (superior parietal, pre- and postcentral areas) and mentalizing networks (lateral and medial prefrontal areas, insula, precuneus, temporo-parietal junction) with an onset of 212 ms, which extended to later time windows. The time course data indicated two processing phases: An initial conceptual access phase, in which linguistic and modal brain circuits rapidly process features depending on their relevance, and a later conceptual elaboration phase, in which elaborative processing within feature-specific networks further refines the concept. This study confirms the proposal that abstract concepts are based on representations in distinct neural circuits depending on their semantic feature content. The present research also highlights the importance of investigating sets of abstract concepts with a defined semantic content.

Task-Dependent Functional and Effective Connectivity during Conceptual Processing.

Conceptual knowledge is central to cognition. Previous neuroimaging research indicates that conceptual processing involves both modality-specific perceptual-motor areas and multimodal convergence zones. For example, our previous functional magnetic resonance imaging (fMRI) study revealed that both modality-specific and multimodal regions respond to sound and action features of concepts in a task-dependent fashion (Kuhnke P, Kiefer M, Hartwigsen G. 2020b. Task-dependent recruitment of modality-specific and multimodal regions during conceptual processing. Cereb Cortex. 30:3938-3959.). However, it remains unknown whether and how modality-specific and multimodal areas interact during conceptual tasks. Here, we asked 1) whether multimodal and modality-specific areas are functionally coupled during conceptual processing, 2) whether their coupling depends on the task, 3) whether information flows top-down, bottom-up or both, and 4) whether their coupling is behaviorally relevant. We combined psychophysiological interaction analyses with dynamic causal modeling on the fMRI data of our previous study. We found that functional coupling between multimodal and modality-specific areas strongly depended on the task, involved both top-down and bottom-up information flow, and predicted conceptually guided behavior. Notably, we also found coupling between different modality-specific areas and between different multimodal areas. These results suggest that functional coupling in the conceptual system is extensive, reciprocal, task-dependent, and behaviorally relevant. We propose a new model of the conceptual system that incorporates task-dependent functional interactions between modality-specific and multimodal areas.

Images of the unseen: extrapolating visual representations for abstract and concrete words in a data-driven computational model.

Theories of grounded cognition assume that conceptual representations are grounded in sensorimotor experience. However, abstract concepts such as jealousy or childhood have no directly associated referents with which such sensorimotor experience can be made; therefore, the grounding of abstract concepts has long been a topic of debate. Here, we propose (a) that systematic relations exist between semantic representations learned from language on the one hand and perceptual experience on the other hand, (b) that these relations can be learned in a bottom-up fashion, and (c) that it is possible to extrapolate from this learning experience to predict expected perceptual representations for words even where direct experience is missing. To test this, we implement a data-driven computational model that is trained to map language-based representations (obtained from text corpora, representing language experience) onto vision-based representations (obtained from an image database, representing perceptual experience), and apply its mapping function onto language-based representations for abstract and concrete words outside the training set. In three experiments, we present participants with these words, accompanied by two images: the image predicted by the model and a random control image. Results show that participants' judgements were in line with model predictions even for the most abstract words. This preference was stronger for more concrete items and decreased for the more abstract ones. Taken together, our findings have substantial implications in support of the grounding of abstract words, suggesting that we can tap into our previous experience to create possible visual representation we don't have.

Situating desire: Situational cues affect desire for food through eating simulations.

How do situations influence food desire? Although eating typically occurs in rich background situations, research on food desire often focuses on the properties of foods and consumers, rather than on the situations in which eating takes place. Here, we take a grounded cognition perspective and suggest that a situation that is congruent with consuming a food increases simulations of eating it, which, in turn, affect desire, and the expected and actual liking of the food. We tested this idea in four pre-registered experiments (N = 524). Participants processed an image of a food presented in a congruent situation, an incongruent situation, or no background situation. Compared to the incongruent situation, the congruent situation increased expected liking of the food and desire, and this was partially or fully mediated by eating simulations. The congruent situation also increased salivation, a physiological indicator of preparing to eat. However, there was only weak and indirect evidence for congruence effects on actual liking of the food when tasted. These findings show that situational cues can affect desire for food through eating simulations. Thus, background situations play an important but understudied role in human food desires. We address implications for research using food images, and for applications to promote healthy and sustainable eating behaviour.

Pleasure vs. identity: More eating simulation language in meat posts than plant-based posts on social media #foodtalk.

Current levels of meat consumption in Western societies are unsustainable and contribute to the climate emergency. However, most people are not reducing their intake. Here, we examine the language used on social media to describe meat and plant-based foods, since the ways people think and communicate about food could hinder the transition towards sustainable eating. In two pre-registered studies, we analysed the degree to which the language in food posts on Instagram reflects eating simulations, which have been found to be associated with desire for appetitive stimuli. Specifically, thinking about or presenting foods or drinks in terms of rewarding simulations (i.e., re-experiences of enjoying their consumption) has been found to increase their appeal. Here, we analysed the words used in Instagram hashtags (NStudy1 = 852; NStudy2 = 3104) and caption text (NStudy1 = 682) to examine how much they refer to eating simulations (e.g., taste, texture, enjoyment, eating context) or to other food-related features (e.g., ingredients, preparation, health, category information). As hypothesized, meat posts contained more eating simulation hashtags than plant-based and vegetarian posts, which instead contained more eating-independent hashtags, for example referring to health or to vegan identity. Findings for the text words were generally in the same direction but much weaker. Thus, meat food posts contained hashtag language that is likely more appealing to mainstream consumers, because it refers to the enjoyable experience of eating the food, rather than the food being healthy or identity affirming. This pattern reflects polarisation surrounding sustainable foods, which may hinder the shift towards plant-based diets needed to curb climate change.

Is color continuously activated in mental simulations across a broader discourse context?

Previous studies have provided contradictory information regarding the activation of perceptual information in a changing discourse context. The current study examines the continued activation of color in mental simulations across one (Experiment 1), two (Experiment 2), and five sentences (Experiment 3), using a sentence-picture verification paradigm. In Experiment 1, the sentence either contained a reference to a color (e.g., a red bicycle) or no reference to a color (e.g., bicycle). In Experiments 2 and 3, either the first or the final sentence contained a reference to a color. Participants responded to pictures either matching the color mentioned in the sentence, or shown in grayscale. The results illustrated that color was activated in mental simulations when the final sentence contained a reference to color. When the target object (e.g., bicycle) was mentioned in all sentences (i.e., in Experiment 2), color remained activated in the mental simulation, even when only the first sentence made a reference to a color. When the focus of the story was shifted elsewhere and the target object was not present across all sentences (i.e., in Experiment 3), color was no longer activated in the mental simulation. These findings suggest that color remains active in mental simulations so long as the target object is present in every sentence. As soon as the focus of the story shifts to another event, this perceptual information is deactivated in the mental simulation. As such, there is no continued activation of color across a broader discourse context.

Redundancy, isomorphism, and propagative mechanisms between emotional and amodal representations of words: A computational study.

Some proposals claim that language acts as a link to propagate emotional and other modal information. Thus, there is an eminently amodal path of emotional propagation in the mental lexicon. Following these proposals, we present a computational model that emulates a linking mechanism (mapping function) between emotional and amodal representations of words using vector space models, emotional feature-based models, and neural networks. We analyzed three central concepts within the embodiment debate (redundancy, isomorphism, and propagative mechanisms) comparing two alternative hypotheses: semantic neighborhood hypothesis versus specific dimensionality hypothesis. Univariate and multivariate neural networks were trained for dimensional (N = 11,357) and discrete emotions (N = 2,266), and later we analyzed its predictions in a test set (N = 4,167 and N = 875, respectively). We showed how this computational model could propagate emotional responses to words without a direct emotional experience via amodal propagation, but no direct relations were found between emotional rates and amodal distances. Thereby, we found that there were clear redundancy and propagative mechanisms, but no isomorphism should be assumed. Results suggested that it was necessary to establish complex links to go beyond amodal distances of vector spaces. In this way, although the emotional rates of semantic neighborhoods could predict the emotional rates of target words, the mapping function of specific amodal features seemed to simulate emotional responses better. Thus, both hypotheses would not be mutually exclusive. We also showed that discrete emotions could have simpler relations between modal and amodal representations than dimensional emotions. All these results and their theoretical implications are discussed.

Left posterior inferior parietal cortex causally supports the retrieval of action knowledge.

Conceptual knowledge is central to human cognition. The left posterior inferior parietal lobe (pIPL) is implicated by neuroimaging studies as a multimodal hub representing conceptual knowledge related to various perceptual-motor modalities. However, the causal role of left pIPL in conceptual processing remains unclear. Here, we transiently disrupted left pIPL function with transcranial magnetic stimulation (TMS) to probe its causal relevance for the retrieval of action and sound knowledge. We compared effective TMS over left pIPL with sham TMS, while healthy participants performed three different tasks-lexical decision, action judgment, and sound judgment-on words with a high or low association to actions and sounds. We found that pIPL-TMS selectively impaired action judgments on low sound-low action words. For the first time, we directly related computational simulations of the TMS-induced electrical field to behavioral performance, which revealed that stronger stimulation of left pIPL is associated with worse performance for action but not sound judgments. These results indicate that left pIPL causally supports conceptual processing when action knowledge is task-relevant and cannot be compensated by sound knowledge. Our findings suggest that left pIPL is specialized for the retrieval of action knowledge, challenging the view of left pIPL as a multimodal conceptual hub.

The role of the motor system in generating creative thoughts.

Neurocognitive research is pertinent to developing mechanistic models of how humans generate creative thoughts. Such models usually overlook the role of the motor cortex in creative thinking. The framework of embodied or grounded cognition suggests that creative thoughts (e.g. using a shoe as a hammer, improvising a piano solo) are partially served by simulations of motor activity associated with tools and their use. The major hypothesis stemming from the embodied or grounded account is that, while the motor system is used to execute actions, simulations within this system also support higher-order cognition, creativity included. That is, the cognitive process of generating creative output, not just executing it, is deeply embedded in motor processes. Here, we highlight a collection of neuroimaging research that implicates the motor system in generating creative thoughts, including some evidence for its functionally necessary role in generating creative output. Specifically, the grounded or embodied framework suggests that generating creative output may, in part, rely on motor simulations of possible actions, and that these simulations may by partially implemented in the motor regions themselves. In such cases, action simulations (i.e. reactivating or re-using the motor system), do not result in overt action but instead are used to support higher-order cognitive goals like generating creative uses or improvising.

The Psychology of Desire and Implications for Healthy Hydration.

This article discusses the cognitive mechanisms underlying the motivation to consume sugar-sweetened beverages (SSBs) and outlines implications for developing healthy hydration habits. While the detrimental health consequences of consuming SSBs are well understood, the psychological processes underlying the motivation to consume them are understudied. To address this gap, the current article applies a grounded cognition theory of desire and motivated behaviour, which can be used as a framework to understand and potentially change the motivation for SSBs and healthier alternatives, such as water. The grounded cognition theory of desire argues that people represent foods and drinks through potentially rewarding simulations, or re-experiences, of consuming them. These simulations, in turn, can increase desire and motivated behaviour. In line with this theory, research on eating behaviour shows that people think about attractive food in terms of what it feels like to eat it and in terms of relevant eating situations and that these simulations predict the desire to eat. Similarly, emerging research on SSBs shows that people represent these beverages in terms of the sensory and rewarding experiences of drinking them, more so than water, and especially if they consume them often. These simulations, in turn, predict the desire for sugary drinks and actual consumption. This has implications for attempts to increase healthy hydration: in order to facilitate healthy choices, the immediate pleasure to be gained from consuming a healthy beverage should be emphasized, rather than its long-term benefits. Repeatedly facilitating healthy drink choices in similar situations can ultimately contribute to the development of healthy hydration habits.

Using consumption and reward simulations to increase the appeal of plant-based foods.

The production of meat is a main contributor to current dangerous levels of greenhouse gas emissions. However, the shift to more plant-based diets is hampered by consumers finding meat-based foods more attractive than plant-based foods. How can plant-based foods best be described to increase their appeal to consumers? Based on the grounded cognition theory of desire, we suggest that descriptions that trigger simulations, or re-experiences, of eating and enjoying a food will increase the attractiveness of a food, compared to descriptions emphasizing ingredients. In Study 1, we first examined the descriptions of ready meals available in four large UK supermarkets (N = 240). We found that the labels of meat-based foods contained more references to eating simulations than vegetarian foods, and slightly more than plant-based foods, and that this varied between supermarkets. In Studies 2 and 3 (N = 170, N = 166, pre-registered), we manipulated the labels of plant-based and meat-based foods to either include eating simulation words or not. We assessed the degree to which participants reported that the description made them think about eating the food (i.e., induced eating simulations), and how attractive they found the food. In Study 2, where either sensory or eating context words were added, we found no differences with control labels. In Study 3, however, where simulation-based labels included sensory, context, and hedonic words, we found that simulation-based descriptions increased eating simulations and attractiveness. Moreover, frequent meat eaters found plant-based foods less attractive, but this was attenuated when plant-based foods were described with simulation-inducing words. We suggest that language that describes rewarding eating experiences can be used to facilitate the shift toward healthy and sustainable diets.

Prediction mechanisms in motor and auditory areas and their role in sound perception and language understanding.

Is the meaning of an expected stimulus manifest in brain activity even before it appears? Although theories of predictive coding see anticipatory activity as crucial for the understanding of brain function, few studies have explored neurophysiologically manifest semantic predictions. Here, we report predictive negative-going potentials before the onset of action (i.e. whistle and hand clap) and non-action (i.e. pure tone, water drop) sounds. These prediction potentials (PP) indexed the meaning of action-related sounds. Dependent on the body-part-relationship of sound stimuli, neural sources were relatively more prominent in dorsal or ventral motor areas. In contrast, meaningless sounds (pure tones) activated predictive sources in temporal areas close to the auditory cortex; complex environmental sounds induced an anticipatory positivity broadly distributed over the scalp. We also found a systematic relationship between predictive activity and a Mismatch Negativity (MMN) like response to unexpected meaningful words which were presented as rare deviant stimuli amongst frequently repeated sounds. This deviant-elicited potential indexed semantic priming between action sounds and action-related words and semantic mismatch (prediction error). These results suggest a systematic link between perceptual/semantic prediction and matching mechanisms in the processing of sounds and words.