Does emotion influence visual perception? Depends on how you look at it.

One of the biggest challenges in the study of emotion-cognition interaction is addressing the question of whether and how emotions influence processes of perception as distinct from other higher-level cognitive processes. Most theories of emotion agree that an emotion episode begins with a sensory experience - such as a visual percept - that elicits a cascade of affective, cognitive, physiological, and/or behavioural responses (the ordering and inclusion of those latter components being forever debated). However, for decades, a subset of philosophers and scientists have suggested that the presumed perception → emotion relationship is in fact bidirectional, with emotion also altering the perceptual process. In the present review we reflect on the history and empirical support (or, some might argue, lack thereof) for the notion that emotion influences visual perception. We examine ways in which researchers have attempted to test the question, and the ways in which this pursuit is so difficult. As is the case with the ongoing debate about the cognitive penetrability of perception, we conclude that nothing is conclusive in the debate about the emotional penetrability of perception. We nonetheless don rose-coloured glasses as we look forward to the future of this research topic.

A sad thumbs up: incongruent gestures and disrupted sensorimotor activity both slow processing of facial expressions.

Recognising a facial expression is more difficult when the expresser's body conveys incongruent affect. Existing research has documented such interference for universally recognisable bodily expressions. However, it remains unknown whether learned, conventional gestures can interfere with facial expression processing. Study 1 participants (N = 62) viewed videos of people simultaneously producing facial expressions and hand gestures and reported the valence of either the face or hand. Responses were slower and less accurate when the face-hand pairing was incongruent compared to congruent. We hypothesised that hand gestures might exert an even stronger influence on facial expression processing when other routes to understanding the meaning of a facial expression, such as with sensorimotor simulation, are disrupted. Participants in Study 2 (N = 127) completed the same task, but the facial mobility of some participants was restricted, which disrupted face processing in prior work. The hand-face congruency effect from Study 1 was replicated. The facial mobility manipulation affected males only, and it did not moderate the congruency effect. The present work suggests the affective meaning of conventional gestures is processed automatically and can interfere with face perception, but does not suggest that perceivers rely more on gestures when sensorimotor face processing is disrupted.

Functional Smiles: Tools for Love, Sympathy, and War.

A smile is the most frequent facial expression, but not all smiles are equal. A social-functional account holds that smiles of reward, affiliation, and dominance serve basic social functions, including rewarding behavior, bonding socially, and negotiating hierarchy. Here, we characterize the facial-expression patterns associated with these three types of smiles. Specifically, we modeled the facial expressions using a data-driven approach and showed that reward smiles are symmetrical and accompanied by eyebrow raising, affiliative smiles involve lip pressing, and dominance smiles are asymmetrical and contain nose wrinkling and upper-lip raising. A Bayesian-classifier analysis and a detection task revealed that the three smile types are highly distinct. Finally, social judgments made by a separate participant group showed that the different smile types convey different social messages. Our results provide the first detailed description of the physical form and social messages conveyed by these three types of functional smiles and document the versatility of these facial expressions.

Perceiving emotions: Cueing social categorization processes and attentional control through facial expressions.

Individuals spontaneously categorise other people on the basis of their gender, ethnicity and age. But what about the emotions they express? In two studies we tested the hypothesis that facial expressions are similar to other social categories in that they can function as contextual cues to control attention. In Experiment 1 we associated expressions of anger and happiness with specific proportions of congruent/incongruent flanker trials. We also created consistent and inconsistent category members within each of these two general contexts. The results demonstrated that participants exhibited a larger congruency effect when presented with faces in the emotional group associated with a high proportion of congruent trials. Notably, this effect transferred to inconsistent members of the group. In Experiment 2 we replicated the effects with faces depicting true and false smiles. Together these findings provide consistent evidence that individuals spontaneously utilise emotions to categorise others and that such categories determine the allocation of attentional control.

Social functionality of human emotion.

Answers to the question "What are human emotions for?" have stimulated highly productive programs of research on emotional phenomena in psychology and neuroscience in the past decade. Although a variety of functions have been proposed and examined at different levels of abstraction, what is undeniable is that when emotional processing is compromised, most things social go awry. In this review we survey the research findings documenting the functions of emotion and link these to new discoveries about how emotion is accurately processed and transmitted. We focus specifically on emotion processing in dyads and groups, which reflects the current scientific trend. Within dyads, emotional expressions and learning and understanding through vicarious emotion are the phenomena of interest. Behavioral and brain mechanisms supporting their successful occurrence are evaluated. At the group level, group emotions and group-based emotions, two very different phenomena, are discussed, and mechanistic accounts are reviewed.

Ancestral Diversity: A Socioecological Account of Emotion Culture.

Cultural differences in emotion expression, experience, and regulation can cause misunderstandings with lasting effects on interpersonal, intergroup, and international relations. A full account of the factors responsible for the emergence of different cultures of emotion is therefore urgent. Here we propose that the ancestral diversity of regions of the world, determined by colonization and sometimes forced migration of humans over centuries, explains significant variation in cultures of emotion. We review findings that relate the ancestral diversity of the world's countries to present-day differences in display rules for emotional expression, the clarity of expressions, and the use of specific facial expressions such as the smile. Results replicate at the level of the states of the United States, which also vary in ancestral diversity. Further, we suggest that historically diverse contexts provide opportunities for individuals to exercise physiological processes that support emotion regulation, resulting in average regional differences in cardiac vagal tone. We conclude that conditions created by the long-term commingling of the world's people have predictable effects on the evolution of emotion cultures and provide a roadmap for future research to analyze causation and isolate mechanisms linking ancestral diversity to emotion.

Red and blue states: dichotomized maps mislead and reduce perceived voting influence.

In the United States the color red has come to represent the Republican party, and blue the Democratic party, in maps of voting patterns. Here we test the hypothesis that voting maps dichotomized into red and blue states leads people to overestimate political polarization compared to maps in which states are represented with continuous gradations of color. We also tested whether any polarizing effect is due to partisan semantic associations with red and blue, or if alternative hues produce similar effects. In Study 1, participants estimated the hypothetical voting patterns of eight swing states on maps with dichotomous or continuous red/blue or orange/green color schemes. A continuous gradient mitigated the polarizing effects of red/blue maps on voting predictions. We also found that a novel hue pair, green/orange, decreased perceived polarization. Whether this effect was due to the novelty of the hues or the fact that the hues were not explicitly labeled "Democrat" and "Republican" was unclear. In Study 2, we explicitly assigned green/orange hues to the two parties. Participants viewed electoral maps depicting results from the 2020 presidential election and estimated the voting margins for a subset of states. We replicated the finding that continuous red/blue gradient reduced perceived polarization, but the novel hues did not reduce perceived polarization. Participants also expected their hypothetical vote to matter more when viewing maps with continuous color gradations. We conclude that the dichotomization of electoral maps (not the particular hues) increases perceived voting polarization and reduces a voter's expected influence on election outcomes.

Face coverings differentially alter valence judgments of emotional expressions.

Face masks that prevent disease transmission obscure facial expressions, impairing nonverbal communication. We assessed the impact of lower (masks) and upper (sunglasses) face coverings on emotional valence judgments of clearly valenced (fearful, happy) and ambiguously valenced (surprised) expressions, the latter of which have both positive and negative meaning. Masks, but not sunglasses, impaired judgments of clearly valenced expressions compared to faces without coverings. Drift diffusion models revealed that lower, but not upper, face coverings slowed evidence accumulation and affected differences in non-judgment processes (i.e., stimulus encoding, response execution time) for all expressions. Our results confirm mask-interference effects in nonverbal communication. The findings have implications for nonverbal and intergroup communication, and we propose guidance for implementing strategies to overcome mask-related interference.

Living in ancestrally diverse states of the United States is associated with greater vagal tone.

Historically, exposure to dissimilar others ("strangers") was a physiologically arousing event-resulting in avoidance, distrust, and even conflict. Despite this, contemporary migration patterns are increasing intergroup contact. What gives rise to an individual's ability to regulate their arousal such that social engagement with outgroup members is possible? We propose that cultural practices that evolve in ancestrally diverse, compared to ancestrally homogeneous, societies provide more opportunities for society members to engage in emotion regulation. This regulatory exercise, in turn, promotes higher vagal tone-a physiological indicator of one's ability to effectively manage arousal in social interaction. In a secondary analysis of data from the MIDUS 2 Biomarker Project, we find that the ancestral diversity of the states of the United States significantly predicts the average vagal tone of the state's citizens. The findings suggest that social context is associated with predictable and significant adaptations of human physiology over individual lifetimes.

Wearing N95, Surgical, and Cloth Face Masks Compromises the Perception of Emotion.

According to the familiar axiom, the eyes are the window to the soul. However, wearing masks to prevent the spread of viruses such as COVID-19 involves obscuring a large portion of the face. Do the eyes carry sufficient information to allow for the accurate perception of emotions in dynamic expressions obscured by masks? What about the perception of the meanings of specific smiles? We addressed these questions in two studies. In the first, participants saw dynamic expressions of happiness, disgust, anger, and surprise that were covered by N95, surgical, or cloth masks or were uncovered and rated the extent to which the expressions conveyed each of the same four emotions. Across conditions, participants perceived significantly lower levels of the expressed (target) emotion in masked faces, and this was particularly true for expressions composed of more facial action in the lower part of the face. Higher levels of other (non-target) emotions were also perceived in masked expressions. In the second study, participants rated the extent to which three categories of smiles (reward, affiliation, and dominance) conveyed positive feelings, reassurance, and superiority, respectively. Masked smiles communicated less of the target signal than unmasked smiles, but not more of other possible signals. The present work extends recent studies of the effects of masked faces on the perception of emotion in its novel use of dynamic facial expressions (as opposed to still images) and the investigation of different types of smiles. Supplementary Information: The online version contains supplementary material available at 10.1007/s42761-021-00097-z.

Exploring the Relationship Between Fiction Reading and Emotion Recognition.

Fiction reading experience affects emotion recognition abilities, yet the causal link remains underspecified. Current theory suggests fiction reading promotes the simulation of fictional minds, which supports emotion recognition skills. We examine the extent to which contextualized statistical experience with emotion category labels in language is associated with emotion recognition. Using corpus analyses, we demonstrate fiction texts reliably use emotion category labels in an emotive sense (e.g., cry of relief), whereas other genres often use alternative senses (e.g., hurricane relief fund). Furthermore, fiction texts were shown to be a particularly reliable source of information about complex emotions. The extent to which these patterns affect human emotion concepts was analyzed in two behavioral experiments. In experiment 1 (n = 134), experience with fiction text predicted recognition of emotions employed in an emotive sense in fiction texts. In experiment 2 (n = 387), fiction reading experience predicted emotion recognition abilities, overall. These results suggest that long-term language experience, and fiction reading, in particular, supports emotion concepts through exposure to these emotions in context.

Correction to: Evidence for Distinct Facial Signals of Reward, Affiliation, and Dominance from Both Perception and Production Tasks.

[This corrects the article DOI: 10.1007/s42761-020-00024-8.].

Evidence for Distinct Facial Signals of Reward, Affiliation, and Dominance from Both Perception and Production Tasks.

Smiles are nonverbal signals that convey social information and influence the social behavior of recipients, but the precise form and social function of a smile can be variable. In previous work, we have proposed that there are at least three physically distinct types of smiles associated with specific social functions: reward smiles signal positive affect and reinforce desired behavior; affiliation smiles signal non-threat and promote peaceful social interactions; dominance smiles signal feelings of superiority and are used to negotiate status hierarchies. The present work advances the science of the smile by addressing a number of questions that directly arise from this smile typology. What do perceivers think when they see each type of smile (Study 1)? How do perceivers behave in response to each type of smile (Study 2)? Do people produce three physically distinct smiles in response to contexts related to each of the three social functions of smiles (Study 3)? We then use an online machine learning platform to uncover the labels that lay people use to conceptualize the smile of affiliation, which is a smile that serves its social function but lacks a corresponding lay concept. Taken together, the present findings support the conclusion that reward, affiliation, and dominance smiles are distinct signals with specific social functions. These findings challenge the traditional assumption that smiles merely convey whether and to what extent a smiler is happy and demonstrate the utility of a social-functional approach to the study of facial expression.

The Simulation of Smiles (SIMS) model: Embodied simulation and the meaning of facial expression.

Recent application of theories of embodied or grounded cognition to the recognition and interpretation of facial expression of emotion has led to an explosion of research in psychology and the neurosciences. However, despite the accelerating number of reported findings, it remains unclear how the many component processes of emotion and their neural mechanisms actually support embodied simulation. Equally unclear is what triggers the use of embodied simulation versus perceptual or conceptual strategies in determining meaning. The present article integrates behavioral research from social psychology with recent research in neurosciences in order to provide coherence to the extant and future research on this topic. The roles of several of the brain's reward systems, and the amygdala, somatosensory cortices, and motor centers are examined. These are then linked to behavioral and brain research on facial mimicry and eye gaze. Articulation of the mediators and moderators of facial mimicry and gaze are particularly useful in guiding interpretation of relevant findings from neurosciences. Finally, a model of the processing of the smile, the most complex of the facial expressions, is presented as a means to illustrate how to advance the application of theories of embodied cognition in the study of facial expression of emotion.

Disrupting facial action increases risk taking.

Sometimes risk involves taking actions that in and of themselves elicit emotion, often fearful emotions. Across two studies we test the hypothesis that preventing facial actions associated with fear and anxiety responses during a risky decision task leads to greater risk taking. We first demonstrate that while performing the balloon analogue risk task (Lejuez et al., 2002), individuals make grimaces associated with anxious anticipation. In Study 1 (n = 120), experimental condition participants had inflexible medical tape attached to their foreheads to disrupt movement of the brow, and they wore a mouth guard that interfered with actions involving the mouth. Tape was also applied to control participants' faces, but it did not disrupt facial action, and they did not wear a mouth guard. All participants performed the balloon analogue risk task, in which a greater number of balloon pumps signals more risk taking. Study 2 (n = 202) served as a replication and minor extension that added a second risk task also predicted to elicit anxious anticipation (i.e., a jack-in-the-box toy). As hypothesized, disrupting the activation of facial muscles led to more balloon pumps and lever turns. Our findings suggest that facial expressions modulate risk taking. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Social Structure Predicts Eye Contact Tolerance in Nonhuman Primates: Evidence from a Crowd-Sourcing Approach.

In most primates, eye contact is an implicit signal of threat, and often connotes social status and imminent physical aggression. However, in humans and some of the gregarious nonhuman primates, eye contact is tolerated more and may be used to communicate other emotional and mental states. What accounts for the variation in this critical social cue across primate species? We crowd-sourced primatologists and found a strong linear relationship between eye contact tolerance and primate social structure such that eye contact tolerance increased as social structures become more egalitarian. In addition to constituting the first generalizable demonstration of this relationship, our findings serve to inform the related question of why eye contact is deferentially avoided in some human cultures, while eye contact is both frequent and even encouraged in others.

Social anxiety and the interpretation of morphed facial expressions following exclusion and inclusion.

BACKGROUND AND OBJECTIVES: Negative interpretation biases are postulated to play etiological and maintaining roles in social anxiety (SA). However, empirical support for interpretation biases of facial expression in SA is inconsistent. Given the importance of signals of (dis)approval in SA, our objective was to examine whether SA is associated with enhanced sensitivity to such signals especially following exclusion. METHODS: In Study 1, participants (N = 139) underwent an exclusion/inclusion manipulation and were then presented with video clips of smiles gradually changing into disgust expressions (smile-to-disgust). In Study 2 (N = 203), participants saw smile-to-disgust as well as disgust-to-smile clips following an exclusion/inclusion manipulation. Participants' task in both studies was to detect the offset of the initial expression. RESULTS: Results of Study 1 show that detection latency of smiles' disappearance is negatively associated with SA severity. The results of Study 2 suggest that this association is stronger following exclusion, and specific to the smile-to-disgust as opposed to the disgust-to-smile, transitions. LIMITATIONS: Our studies did not examine whether the observed interpretation bias was specific to SA. CONCLUSIONS: Our findings support and refine cognitive theories of SA, suggesting that interpretation biases for facial information in SA may be especially pronounced following exclusion.

Emotional conception: how embodied emotion concepts guide perception and facial action.

This study assessed embodied simulation via electromyography (EMG) as participants first encoded emotionally ambiguous faces with emotion concepts (i.e., "angry,""happy") and later passively viewed the faces without the concepts. Memory for the faces was also measured. At initial encoding, participants displayed more smiling-related EMG activity in response to faces paired with "happy" than in response to faces paired with "angry." Later, in the absence of concepts, participants remembered happiness-encoded faces as happier than anger-encoded faces. Further, during passive reexposure to the ambiguous faces, participants' EMG indicated spontaneous emotion-specific mimicry, which in turn predicted memory bias. No specific EMG activity was observed when participants encoded or viewed faces with non-emotion-related valenced concepts, or when participants encoded or viewed Chinese ideographs. From an embodiment perspective, emotion simulation is a measure of what is currently perceived. Thus, these findings provide evidence of genuine concept-driven changes in emotion perception. More generally, the findings highlight embodiment's role in the representation and processing of emotional information.

Embodiment of emotion concepts.

Theories of embodied cognition hold that higher cognitive processes operate on perceptual symbols and that concept use involves partial reactivations of the sensory-motor states that occur during experience with the world. On this view, the processing of emotion knowledge involves a (partial) reexperience of an emotion, but only when access to the sensory basis of emotion knowledge is required by the task. In 2 experiments, participants judged emotional and neutral concepts corresponding to concrete objects (Experiment 1) and abstract states (Experiment 2) while facial electromyographic activity was recorded from the cheek, brow, eye, and nose regions. Results of both studies show embodiment of specific emotions in an emotion-focused but not a perceptual-focused processing task on the same words. A follow up in Experiment 3, which blocked selective facial expressions, suggests a causal, rather than simply a correlational, role for embodiment in emotion word processing. Experiment 4, using a property generation task, provided support for the conclusion that emotions embodied in conceptual tasks are context-dependent situated simulations rather than associated emotional reactions. Implications for theories of embodied simulation and for emotion theories are discussed.

The attribution of primary and secondary emotions to the in-group and to the out-group: the case of equal status countries.

We examined the attribution of primary and secondary emotions in the context of equal status groups with a non-conflictual relationship, that is, Germans and French. In Study 1 (N = 169), we found that in such an intergroup context, there was no differential attribution of secondary emotions but an over-attribution of primary emotions to the out-group. Only high identifiers tended to attribute more secondary emotions to the in-group than to the out-group. In Study 2 (N = 423), the role of the identification with the in-group and a superordinate group (Europe) in the process of infrahumanization was examined. Participants' national versus European identification was primed. The results did not differ between these two conditions. As in Study 1, an over-attribution of primary emotions to the out-group was observed. Concerning the secondary emotions, the classical infrahumanization effect occurred, that is, an over-attribution of secondary emotions to the in-group.