Gesturing during disfluent speech: A pragmatic account.

People are more likely to gesture when their speech is disfluent. Why? According to an influential proposal, speakers gesture when they are disfluent because gesturing helps them to produce speech. Here, we test an alternative proposal: People may gesture when their speech is disfluent because gestures serve as a pragmatic signal, telling the listener that the speaker is having problems with speaking. To distinguish between these proposals, we tested the relationship between gestures and speech disfluencies when listeners could see speakers' gestures and when they were prevented from seeing their gestures. If gesturing helps speakers to produce words, then the relationship between gesture and disfluency should persist regardless of whether gestures can be seen. Alternatively, if gestures during disfluent speech are pragmatically motivated, then the tendency to gesture more when speech is disfluent should disappear when the speaker's gestures are invisible to the listener. Results showed that speakers were more likely to gesture when their speech was disfluent, but only when the listener could see their gestures and not when the listener was prevented from seeing them, supporting a pragmatic account of the relationship between gestures and disfluencies. People tend to gesture more when speaking is difficult, not because gesturing facilitates speech production, but rather because gestures comment on the speaker's difficulty presenting an utterance to the listener.

No support for a causal role of primary motor cortex in construing meaning from language: An rTMS study.

Embodied cognition theories predict a functional involvement of sensorimotor processes in language understanding. In a preregistered experiment, we tested this idea by investigating whether interfering with primary motor cortex (M1) activation can change how people construe meaning from action language. Participants were presented with sentences describing actions (e.g., "turning off the light") and asked to choose between two interpretations of their meaning, one more concrete (e.g., "flipping a switch") and another more abstract (e.g., "going to sleep"). Prior to this task, participants' M1 was disrupted using repetitive transcranial magnetic stimulation (rTMS). The results yielded strong evidence against the idea that M1-rTMS affects meaning construction (BF01 > 30). Additional analyses and control experiments suggest that the absence of effect cannot be accounted for by failure to inhibit M1, lack of construct validity of the task, or lack of power to detect a small effect. In sum, these results do not support a causal role for primary motor cortex in building meaning from action language.

Hidden Differences in Phenomenal Experience.

In addition to the many easily observable differences between people, there are also differences in people's subjective experiences that are harder to observe, and which, as a consequence, remain hidden. For example, people vary widely in how much visual imagery they experience. But those who cannot see in their mind's eye, tend to assume everyone is like them. Those who can, assume everyone else can as well. We argue that a study of such hidden phenomenal differences has much to teach cognitive science. Uncovering and describing this variation (a search for unknown unknowns) may help predict otherwise puzzling differences in human behavior. The very existence of certain differences can also act as a stress test for some cognitive theories. Finally, studying hidden phenomenal differences is the first step toward understanding what kinds of environments may mask or unmask links between phenomenal experience and observable behavior.

Spatial metaphors and the design of everyday things.

People use space (e.g., left-right, up-down) to think about a variety of non-spatial concepts like time, number, similarity, and emotional valence. These spatial metaphors can be used to inform the design of user interfaces, which visualize many of these concepts in space. Traditionally, researchers have relied on patterns in language to discover habits of metaphorical thinking. However, advances in cognitive science have revealed that many spatial metaphors remain unspoken, shaping people's preferences, memories, and actions independent of language - and even in contradiction to language. Here we argue that cognitive science can impact our everyday lives by informing the design of physical and digital objects via the spatial metaphors in people's minds. We propose a simple principle for predicting which spatial metaphors organize people's non-spatial concepts based on the structure of their linguistic, cultural, and bodily experiences. By leveraging the latent metaphorical structure of people's minds, we can design objects and interfaces that help people think.

The Order of Magnitude: Why SNARC-like Tasks (Still) Cannot Support a Generalized Magnitude System.

According to proponents of the generalized magnitude system proposal (GMS), SNARC-like effects index spatial mappings of magnitude and provide crucial evidence for the existence of a GMS. Casasanto and Pitt (2019) have argued that these effects, instead, reflect mappings of ordinality, which people compute on the basis of differences among stimuli that vary either qualitatively (e.g., musical pitches) or quantitatively (e.g., dots of different sizes). In response to our paper, Prpic et al. (2021) argued that both magnitude and ordinality play a role in SNARC-like effects. Here, we address each of their arguments and conclude that magnitude is relevant to these effects only insofar as it serves as a basis for ordinality. For this reason and others, SNARC or SNARC-like effects cannot provide evidence for the putative generalized magnitude system.

Do gestures really facilitate speech production?

Why do people gesture when they speak? According to one influential proposal, the Lexical Retrieval Hypothesis (LRH), gestures serve a cognitive function in speakers' minds by helping them find the right spatial words. Do gestures also help speakers find the right words when they talk about abstract concepts that are spatialized metaphorically? If so, then preventing people from gesturing should increase the rate of disfluencies during speech about both literal and metaphorical space. Here, we sought to conceptually replicate the finding that preventing speakers from gesturing increases disfluencies in speech with literal spatial content (e.g., the rocket went up), which has been interpreted as evidence for the LRH, and to extend this pattern to speech with metaphorical spatial content (e.g., my grades went up). Across three measures of speech disfluency (disfluency rate, speech rate, and rate of nonjuncture filled pauses), we found no difference in disfluency between speakers who were allowed to gesture freely and speakers who were not allowed to gesture, for any category of speech (literal spatial content, metaphorical spatial content, and no spatial content). This large dataset (7,969 phrases containing 2,075 disfluencies) provided no support for the idea that gestures help speakers find the right words, even for speech with literal spatial content. Upon reexamining studies cited as evidence for the LRH and related proposals over the past 5 decades, we conclude that there is, in fact, no reliable evidence that preventing gestures impairs speaking. Together, these findings challenge long-held beliefs about why people gesture when they speak. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Expertise Modulates Neural Stimulus-Tracking.

How does the brain anticipate information in language? When people perceive speech, low-frequency (<10 Hz) activity in the brain synchronizes with bursts of sound and visual motion. This phenomenon, called cortical stimulus-tracking, is thought to be one way that the brain predicts the timing of upcoming words, phrases, and syllables. In this study, we test whether stimulus-tracking depends on domain-general expertise or on language-specific prediction mechanisms. We go on to examine how the effects of expertise differ between frontal and sensory cortex. We recorded electroencephalography (EEG) from human participants who were experts in either sign language or ballet, and we compared stimulus-tracking between groups while participants watched videos of sign language or ballet. We measured stimulus-tracking by computing coherence between EEG recordings and visual motion in the videos. Results showed that stimulus-tracking depends on domain-general expertise, and not on language-specific prediction mechanisms. At frontal channels, fluent signers showed stronger coherence to sign language than to dance, whereas expert dancers showed stronger coherence to dance than to sign language. At occipital channels, however, the two groups of participants did not show different patterns of coherence. These results are difficult to explain by entrainment of endogenous oscillations, because neither sign language nor dance show any periodicity at the frequencies of significant expertise-dependent stimulus-tracking. These results suggest that the brain may rely on domain-general predictive mechanisms to optimize perception of temporally-predictable stimuli such as speech, sign language, and dance.

Spatial concepts of number, size, and time in an indigenous culture.

In industrialized groups, adults implicitly map numbers, time, and size onto space according to cultural practices like reading and counting (e.g., from left to right). Here, we tested the mental mappings of the Tsimane', an indigenous population with few such cultural practices. Tsimane' adults spatially arranged number, size, and time stimuli according to their relative magnitudes but showed no directional bias for any domain on any spatial axis; different mappings went in different directions, even in the same participant. These findings challenge claims that people have an innate left-to-right mapping of numbers and that these mappings arise from a domain-general magnitude system. Rather, the direction-specific mappings found in industrialized cultures may originate from direction-agnostic mappings that reflect the correlational structure of the natural world.

The Reverse Chameleon Effect: Negative Social Consequences of Anatomical Mimicry.

Bodily mimicry often makes the mimickee have more positive feelings about the mimicker. Yet, little is known about the causes of mimicry's social effects. When people mimic each other's bodily movements face to face, they can either adopt a mirrorwise perspective (moving in the same absolute direction) or an anatomical perspective (moving in the same direction relative to their own bodies). Mirrorwise mimicry maximizes visuo-spatial similarity between the mimicker and mimickee, whereas anatomical mimicry maximizes the similarity in the states of their motor systems. To compare the social consequences of visuo-spatial and motoric similarity, we asked participants to converse with an embodied virtual agent (VIRTUO), who mimicked their head movements either mirrorwise, anatomically, or not at all. Compared to participants who were not mimicked, those who were mimicked mirrorwise tended to rate VIRTUO more positively, but those who were mimicked anatomically rated him more negatively. During face-to-face conversation, mirrorwise and anatomical mimicry have opposite social consequences. Results suggest that visuo-spatial similarity between mimicker and mimickee, not similarity in motor system activity, gives rise to the positive social effects of bodily mimicry.

Temporal focus and time spatialization across cultures.

The temporal focus hypothesis (TFH) proposes that whether the past or the future is conceptualized as being located in front depends on temporal focus: the balance of attention paid to the past (tradition) and the future (progress). How general is the TFH, and to what extent can cultures and subcultures be placed on a single line relating time spatialization and temporal focus in spite of stark differences in language, religion, history, and economic development? Data from 10 Western (sub)cultural groups (N = 1198,) were used to derive a linear model relating aggregated temporal focus and proportion of future-in-front responses. This model then successfully fitted 10 independently collected (sub)cultural groups in China and Vietnam (N = 899). Further analysis of the whole data set (N = 2,097) showed that the group-level relation arose at the individual level and allowed precise quantification of its influence. Finally, in an effort to apply the model to all relevant published data sets, we included recent data from Britain and South Africa: The former, but not the latter, fitted the model well. Temporal focus is a central factor that shapes how people around the world think of time in spatial terms.

Unconscious Number Discrimination in the Human Visual System.

How do humans compute approximate number? According to one influential theory, approximate number representations arise in the intraparietal sulcus and are amodal, meaning that they arise independent of any sensory modality. Alternatively, approximate number may be computed initially within sensory systems. Here we tested for sensitivity to approximate number in the visual system using steady state visual evoked potentials. We recorded electroencephalography from humans while they viewed dotclouds presented at 30 Hz, which alternated in numerosity (ranging from 10 to 20 dots) at 15 Hz. At this rate, each dotcloud backward masked the previous dotcloud, disrupting top-down feedback to visual cortex and preventing conscious awareness of the dotclouds' numerosities. Spectral amplitude at 15 Hz measured over the occipital lobe (Oz) correlated positively with the numerical ratio of the stimuli, even when nonnumerical stimulus attributes were controlled, indicating that subjects' visual systems were differentiating dotclouds on the basis of their numerical ratios. Crucially, subjects were unable to discriminate the numerosities of the dotclouds consciously, indicating the backward masking of the stimuli disrupted reentrant feedback to visual cortex. Approximate number appears to be computed within the visual system, independently of higher-order areas, such as the intraparietal sulcus.

Hand-use norms for Dutch and English manual action verbs: Implicit measures from a pantomime task.

Many studies use manual action verbs to test whether people use neural systems for controlling manual actions to understand language about those actions. Yet, few of these studies empirically establish how people use their hands to perform the actions described by those verbs, relying instead on explicit self-report measures. Here, participants pantomimed the manual actions described by a large set of Dutch (N = 251) and English (N = 250) verbs, allowing us to approximate the extent to which people use each of their hands to perform these actions. After the pantomime task, participants also provided explicit ratings of each of these actions. The results from the pantomime task showed that most manual actions cannot be described accurately as either "unimanual" or "bimanual." With a few exceptions, unimanual action verbs do not describe actions that are performed with only one hand, and bimanual verbs do not describe actions that are performed by using both hands equally. Instead, individual actions vary continuously in the extent to which people use their non-dominant hand to perform them, and in the extent to which people consistently prefer one hand or the other to perform them. Finally, by comparing participants' implicit behavior to their explicit ratings, we found that participants' self-report showed only limited correspondence with their observed motor behavior. We provide all of our measures in both raw and summary format, offering researchers a precision tool for constructing stimulus sets for experiments on embodied cognition.

The correlations in experience principle: How culture shapes concepts of time and number.

People use space to conceptualize abstract domains like time and number. This tendency may be a cognitive universal, but the specifics of people's implicit space-time and space-number associations vary across cultures. In Western cultures, both time and numbers are arranged in people's minds along an imaginary horizontal line, from left to right, but in other cultures the directions of the mental timeline (MTL) and mental number line (MNL) are reversed. How does culture shape our abstract concepts? Using time and number as a testbed, we propose and test a general principle, which we call the CORrelations in Experience (CORE) principle, according to which different aspects of experience should selectively affect different abstract concepts. Across 3 training experiments, the MTL was shaped by experiences that provide a correlation between space and time, whereas the MNL was shaped by experiences that provide a correlation between space and number. These findings reveal that the MTL and MNL have distinct experiential bases, supporting the CORE principle and challenging the widespread claim that both mappings are determined by a common set of cultural experiences (e.g., reading, writing, visual scanning). The CORE principle provides an account of how domains like time and number, universal fixtures of the natural world, can be conceptualized in culture-specific ways: People spatialize abstract domains in their minds according to the ways those domains are spatialized in their experience. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Faulty Magnitude Detector: Why SNARC-Like Tasks Cannot Support a Generalized Magnitude System.

Do people represent space, time, number, and other conceptual domains using a generalized magnitude system (GMS)? To answer this question, numerous studies have used the spatial-numerical association of response codes (SNARC) task and its variants. Yet, for a combination of reasons, SNARC-like effects cannot provide evidence for a GMS, even in principle. Rather, these effects support a broader theory of how people use space metaphorically to scaffold their understanding of myriad non-spatial domains, whether or not these domains exhibit variation in magnitude.

Metaphors we learn by: Directed motor action improves word learning.

Can performing simple motor actions help people learn the meanings of words? Here we show that placing vocabulary flashcards in particular locations after studying them helps students learn the definitions of novel words with positive or negative emotional valence. After studying each card, participants placed it on one of two shelves (top or bottom), according to its valence. Participants who were instructed to place positive cards on the top shelf and negative cards on the bottom shelf, consistent with metaphors that link "good" with "up," remembered the words' definitions better than participants who followed the opposite spatial mapping, and better than control participants who placed all of the cards on the desktop. Saying "up" and "down" after studying the cards was ineffective, suggesting a privileged role for motor action in activating space-valence associations that partly constitute the meanings of emotionally charged words. These results provide a first demonstration that mental metaphors can be activated strategically to improve (or impair) word learning: We call this the strategic use of mental metaphor (SUMM) effect. Even when multiple factors known to enhance encoding of verbal materials into long-term memory were matched across conditions (e.g., study time, repetition, distinctiveness, depth of processing), metaphor-congruent motor actions led to better elaborated, more durable memories.

tDCS to premotor cortex changes action verb understanding: Complementary effects of inhibitory and excitatory stimulation.

Processing the meaning of action language correlates with somatotopic activity in premotor cortex (PMC). A previous neurostimulation study supported a causal contribution of PMC activity to action verb understanding, but the direction of the effect was unexpected: inhibiting PMC made participants respond faster to action verbs. Here we investigated the effects of PMC excitation and inhibition on action verb understanding using tDCS. Right-handed participants received tDCS stimulation with the anodal electrode (presumed to be excitatory) and cathodal electrode (presumed to be inhibitory) placed over left and right PMC, respectively, or with the reverse configuration. After completing the stimulation protocol, participants made lexical decisions on unimanual action verbs (e.g., throw) and abstract verbs (e.g., think). tDCS configuration selectively affected how accurately participants responded to unimanual action verbs. When the anode was positioned over left PMC we observed a relative impairment in performance for right-hand responses (i.e. the hand with which these participants typically perform unimanual actions). By contrast, when the cathode was positioned over left PMC we observed a relative improvement. tDCS configuration did not differentially affect responses to abstract verbs. These complementary effects of excitatory and inhibitory tDCS clarify the functional role of premotor hand areas in understanding action language.

Approach motivation in human cerebral cortex.

Different regions of the human cerebral cortex are specialized for different emotions, but the principles underlying this specialization have remained unknown. According to the sword and shield hypothesis, hemispheric specialization for affective motivation, a basic dimension of human emotion, varies across individuals according to the way they use their hands to perform approach- and avoidance-related actions. In a test of this hypothesis, here we measured approach motivation before and after five sessions of transcranial direct current stimulation to increase excitation in the left or right dorsolateral prefrontal cortex, in healthy adults whose handedness ranged from strongly left-handed to strongly right-handed. The strength and direction of participants' handedness predicted whether electrical stimulation to frontal cortex caused an increase or decrease in their experience of approach-related emotions. The organization of approach motivation in the human cerebral cortex varies across individuals as predicted by the organization of the individuals' motor systems. These results show that the large-scale cortical organization of abstract concepts corresponds with the way people use their hands to interact with the world. Affective motivation may re-use neural circuits that evolved for performing approach- and avoidance-related motor actions.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

Spatializing Emotion: No Evidence for a Domain-General Magnitude System.

People implicitly associate different emotions with different locations in left-right space. Which aspects of emotion do they spatialize, and why? Across many studies people spatialize emotional valence, mapping positive emotions onto their dominant side of space and negative emotions onto their non-dominant side, consistent with theories of metaphorical mental representation. Yet other results suggest a conflicting mapping of emotional intensity (a.k.a., emotional magnitude), according to which people associate more intense emotions with the right and less intense emotions with the left - regardless of their valence; this pattern has been interpreted as support for a domain-general system for representing magnitudes. To resolve the apparent contradiction between these mappings, we first tested whether people implicitly map either valence or intensity onto left-right space, depending on which dimension of emotion they attend to (Experiments 1a, b). When asked to judge emotional valence, participants showed the predicted valence mapping. However, when asked to judge emotional intensity, participants showed no systematic intensity mapping. We then tested an alternative explanation of findings previously interpreted as evidence for an intensity mapping (Experiments 2a, b). These results suggest that previous findings may reflect a left-right mapping of spatial magnitude (i.e., the size of a salient feature of the stimuli) rather than emotion. People implicitly spatialize emotional valence, but, at present, there is no clear evidence for an implicit lateral mapping of emotional intensity. These findings support metaphor theory and challenge the proposal that mental magnitudes are represented by a domain-general metric that extends to the domain of emotion.

Visual cortex entrains to sign language.

Despite immense variability across languages, people can learn to understand any human language, spoken or signed. What neural mechanisms allow people to comprehend language across sensory modalities? When people listen to speech, electrophysiological oscillations in auditory cortex entrain to slow ([Formula: see text]8 Hz) fluctuations in the acoustic envelope. Entrainment to the speech envelope may reflect mechanisms specialized for auditory perception. Alternatively, flexible entrainment may be a general-purpose cortical mechanism that optimizes sensitivity to rhythmic information regardless of modality. Here, we test these proposals by examining cortical coherence to visual information in sign language. First, we develop a metric to quantify visual change over time. We find quasiperiodic fluctuations in sign language, characterized by lower frequencies than fluctuations in speech. Next, we test for entrainment of neural oscillations to visual change in sign language, using electroencephalography (EEG) in fluent speakers of American Sign Language (ASL) as they watch videos in ASL. We find significant cortical entrainment to visual oscillations in sign language <5 Hz, peaking at [Formula: see text]1 Hz. Coherence to sign is strongest over occipital and parietal cortex, in contrast to speech, where coherence is strongest over the auditory cortex. Nonsigners also show coherence to sign language, but entrainment at frontal sites is reduced relative to fluent signers. These results demonstrate that flexible cortical entrainment to language does not depend on neural processes that are specific to auditory speech perception. Low-frequency oscillatory entrainment may reflect a general cortical mechanism that maximizes sensitivity to informational peaks in time-varying signals.