Other ethnicity effects in ensemble coding of facial expressions.

Cultural difference in ensemble emotion perception is an important research question, providing insights into the complexity of human cognition and social interaction. Here, we conducted two experiments to investigate how emotion perception would be affected by other ethnicity effects and ensemble coding. In Experiment 1, two groups of Asian and Caucasian participants were tasked with assessing the average emotion of faces from their ethnic group, other ethnic group, and mixed ethnicity groups. Results revealed that participants exhibited relatively accurate yet amplified emotion perception of their group faces, with a tendency to overestimate the weight of the faces from the other ethnic group. In Experiment 2, Asian participants were instructed to discern the emotion of a target face surrounded by faces from Caucasian and Asian faces. Results corroborated earlier findings, indicating that while participants accurately perceived emotions in faces of their ethnicity, their perception of Caucasian faces was noticeably influenced by the presence of surrounding Asian faces. These findings collectively support the notion that the other ethnicity effect stems from differential emotional amplification inherent in ensemble coding of emotion perception.

Repulsion bias is insensitive to spatial attention, yet expands during active working memory maintenance.

Our brain sometimes represents visual information in a biased manner. Multiple visual features presented simultaneously or sequentially may interact with each other when we perceive them or maintain them in visual working memory (WM), giving rise to report bias. How goal-directed attention influences target representation is not fully understood, especially concerning whether attention towards distractors modulates report bias for the target. Our study investigated the WM biases of the target when it is concurrent with (1) one attended distractor only, (2) one unattended distractor only, and (3) both kinds of distractors during perception. It was found that the target WM is reported as being repelled away from concurrent distractors, attended or unattended, suggesting attention is not necessary for the occurrence of repulsion bias during perception. Furthermore, goal-directed attention towards the distractors modulates the strength of interitem interaction, and the repulsion bias was found to be stronger when attention was directed toward the distractor than when it was not. However, the exaggerated repulsion associated with the attended distractor is likely due to increased relevance to the memory task and (or) WM load instead of spatial attention. In contrast, spatial attention towards the distractor increases the chances of misreporting the distractor for the target.

The background assimilation effect: Facial emotional perception is affected by surrounding stimuli.

The perception of facial emotion is not only determined by the physical features of the face itself but also be influenced by the emotional information of the background or surrounding information. However, the details of such effect are not fully understood. Here, the authors tested the perceived emotion of a target face surrounded by stimuli with different levels of emotional valence. In Experiment 1, four types of objects were divided into three groups (negative, unpleasant flowers and unpleasant animals; mildly negative (neutral), houses; positive, pleasant flowers). In Experiment 2, three groups of surrounding faces with different social-emotional valence (negative, neutral, and positive) were formed with the memory of affective personal knowledge. The data from two experiments showed that the perception of facial emotion can be influenced and modulated by the emotional valence of the surrounding stimuli, which can be explained by assimilation: the positive stimuli increased the valence of a target face, while the negative stimuli comparatively decreased it. Furthermore, the neutral stimuli also increased the valence of the target, which could be explained by the social positive effect. Therefore, the process of assimilation is likely to be a high-level emotional cognition rather than a low-level visual perception. The results of this study may help us better understand face perception in realistic scenarios.

Modeling facial perception in group context from a serial perception perspective.

By utilizing statistical properties and summary statistics, the visual system can efficiently integrate perception of spatially and temporally adjacent stimuli into perception of a given target. For instance, perception of a target face can either be biased positively toward previous faces (e.g. the serial dependence effect) or be biased negatively by surrounding faces in the same trial/space (e.g. spatial ensemble averaging). However, both aspects were investigated separately. As spatial and temporal processing share the same purpose to reduce redundancy in visual processing, if one statistical processing occurs, would the statistical processing in the other domain still exist or be discarded? We investigated this question by exploring whether serial dependence of face perception (of attractiveness and averageness) survives when the changed face perception in the group context occurs. The results of Markov Chain modeling and conventional methods suggested that serial dependence (the temporal aspect) co-occurs with changed face perception in the group context (the spatial aspect). We also utilized the Hidden Markov modeling, as a new mathematical method, to model statistical processing from both domains. The results confirmed the co-occurrence of temporal effect and changed face perception in the group context for both attractiveness and averageness, suggesting potentially different spatial and temporal compression mechanisms in high-level vision. Further modeling and cluster analysis further revealed that the detailed computation of spatially and temporally adjacent faces in the attractiveness and averageness processing were similar yet different among different individuals. This work builds a bridge to understanding mathematical principles underlying changed face perception in the group context from the serial perspective.

Categorical encoding of moving colors during location tracking.

Categorical perception (CP) describes our tendency to perceive the visual world in a categorical manner, suggesting that high-level cognition may affect perception. While most studies are conducted in static visual scenes, Sun and colleagues found CP effects of color in multiple object tracking (MOT). This study used functional magnetic resonance imaging to investigate the neural mechanism behind the categorical effects of color in MOT. Categorical effects were associated with activities in a broad range of brain regions, including both the ventral (V4, middle temporal gyrus) and dorsal pathways (MT + /V5, inferior parietal lobule) of feature processing, as well as frontal regions (middle frontal gyrus, medial superior frontal gyrus). We proposed that these regions are hierarchically organized and responsible for distinct functions. The color-selective V4 encodes color categories, making cross-category colors more discriminable than within-category colors. Meanwhile, the language and/or semantic regions encode the verbal information of the colors. Both visual and nonvisual codes of color categories then modulate the activities of motion-sensitive MT + areas and frontal areas responsible for attentional processes.

Disrupted visual input unveils the computational details of artificial neural networks for face perception.

Background: Convolutional Neural Network (DCNN), with its great performance, has attracted attention of researchers from many disciplines. The studies of the DCNN and that of biological neural systems have inspired each other reciprocally. The brain-inspired neural networks not only achieve great performance but also serve as a computational model of biological neural systems. Methods: Here in this study, we trained and tested several typical DCNNs (AlexNet, VGG11, VGG13, VGG16, DenseNet, MobileNet, and EfficientNet) with a face ethnicity categorization task for experiment 1, and an emotion categorization task for experiment 2. We measured the performance of DCNNs by testing them with original and lossy visual inputs (various kinds of image occlusion) and compared their performance with human participants. Moreover, the class activation map (CAM) method allowed us to visualize the foci of the "attention" of these DCNNs. Results: The results suggested that the VGG13 performed the best: Its performance closely resembled human participants in terms of psychophysics measurements, it utilized similar areas of visual inputs as humans, and it had the most consistent performance with inputs having various kinds of impairments. Discussion: In general, we examined the processing mechanism of DCNNs using a new paradigm and found that VGG13 might be the most human-like DCNN in this task. This study also highlighted a possible paradigm to study and develop DCNNs using human perception as a benchmark.

Attention Modulates the Ensemble Coding of Facial Expressions.

Ensemble coding and attention are two mechanisms utilized by our visual system to overcome the limitation of visual processing when confronted with the overwhelming visual information. Recent evidence in ensemble coding of size suggests that the attended items contributed more to the averaging. On the other hand, some new evidence also indicates that reduced attention jeopardies the perceptual averaging of stimuli. What is the relationship between attention and ensemble coding? To answer this question, in the current study, we tested whether an exogenous attentional cue would influence the reported mean emotion of a crowd. We showed participants a group of four faces with different emotions. Participants' attention was guided to the happiest or saddest face (attention conditions), or not to any specific face (baseline condition). The results supported the notion that the attention alters the ensemble perception of the facial expression by elevating the weight of that face in the ensemble representation. This opens the question for the neural mechanisms of ensemble coding and its connection to visual attention.

A general serial dependence among various facial traits: Evidence from Markov Chain and derivative of Gaussian.

The human vision system can extract a stable representation of the always-changing visual world. However, the mechanism underlying such perceptual continuity remains unclear. A possible candidate is the serial dependence: visual perception of an object is positively biased toward the visual input from the recent past. Does the visual system use one pattern of serial dependence for general purposes? Or different patterns of serial dependence for different visual tasks? Because different social facial traits (e.g., trustworthiness and dominance) are dissociable, it is reasonable to assume that the perception of different facial characteristics would require different patterns of serial dependences. In this study, we examine the existence and the similarities of the serial dependence(s) in the evaluation of seven facial characteristics (i.e., attractiveness, trustworthiness, confidence, dominance, intelligence, age, and aggressiveness). The convergent evidence from conventional Derivative of Gaussian fitting and Markov Chain modeling demonstrated that (1) serial dependence exists in judgments of all seven social facial characteristic, (2) the serial dependences of them are highly similar, and (3) the serial dependence follows the efficient coding. Thus it is highly possible that there exists a general serial dependence mechanism for (at least high-level) vision processing. Moreover, we used the Markov Chain modeling to better describe the transitional pattern of serial dependence, which is a kind of Markov process. These findings may shed light on future works regarding serial dependence, as well as face perception.

Friend effects framework: Contrastive and hierarchical processing in cheerleader effects.

Cheerleader effects, group attractiveness effects, and divisive normalization are all characterized by faces appearing more attractive when seen within a group. However, it is possible that your friends could have a detrimental effect upon your attractiveness too: if these group effects arose partly as a contrastive process between your face and your friends, then highly attractive friends may diminish your attractiveness. We confirm this hypothesis across two experiments by showing that the presence of highly attractive friends can indeed make you appear less attractive (i.e., a reverse cheerleader effect), suggesting friend effects are driven in part by a contrastive process against the group. However, these effects are also influenced by your own attractiveness in a fashion that appears consistent with hierarchical encoding, where less attractive targets benefit more from being viewed in an increasingly unattractive group than attractive targets. Our final experiment demonstrates that the company of others not only alters our attractiveness, but also induces shifts in how average or distinctive a target face appears too, with these averageness effects associated with the friend effects observed in our first experiment. We present a Friend Effects Framework within which 'friend effects' is an umbrella term for the positive (e.g., cheerleader effects, group attractiveness effects) and negative (i.e., the reverse cheerleader effect) ways in which hierarchical encoding, group contrastive effects, and other influences of friends can have on your attractiveness.

Perception of attractive and unattractive face groups is driven by distinct spatial frequencies.

Our visual system is able to extract information on facial attractiveness from groups of faces that contain both coarse and detailed information. This raises the question: What information is extracted from a face group? Is the attractiveness perception of multiple faces driven by high or low spatial frequency that can highlight the local or global information of the faces, respectively? In the first experiment, we adapted participants to four unattractive faces with full bandwidth (FB), high spatial frequency (HSF), and low spatial frequency (LSF). We observed significant aftereffects in the HSF faces adaptation condition, which suggests that the perception of multiple unattractive faces is largely driven by HSF information. In the second experiment, we found a similar but different pattern in the direct-rating tasks, suggesting distinct perception mechanisms in unattractive versus attractive faces. In the third experiment, both the adaptation and direct-rating paradigms suggested that perception of multiple attractive faces is largely driven by LSF information. Overall, results from the three experiments together found that perception of multiple attractive and unattractive faces depends on visual information from different spatial frequencies, suggesting distinctive mechanisms in processing attractive and unattractive groups of faces.

Ensemble statistics shape face adaptation and the cheerleader effect.

When confronted with a scene of emotional faces, our brains automatically average the individual facial expressions together to create the gist of the collective emotion. Here, we tested whether this ensemble averaging could also occur for facial attractiveness, and in turn shape 2 related face perception phenomena: adaptation and the cheerleader effect. In our first 2 experiments, we showed that adaptation aftereffects could indeed be shaped by ensemble statistics; viewing an increasingly unattractive group of faces conversely increased attractiveness judgments for a subsequently presented face. Not only did group adaptation aftereffects occur, but their effects were equivalent to those observed from the morphed average face of the group, suggesting that the visual system had averaged the group together. In our last 2 experiments, we showed that viewing a target face in an increasingly unattractive group led to the target being perceived as increasingly more attractive: a "cheerleader" effect. Moreover, our results suggest that this cheerleader effect likely comprises of both a social positive effect and a contrastive process, requiring variance between the surrounding and target faces; that is, the visual system appeared incapable of boosting a target's attractiveness when all of the faces in the scene were identical. Furthermore, the mean group attractiveness ratings strongly predicted both the cheerleader effect and adaptation aftereffects, with the latter 2 also interrelated. This suggests that ensemble statistics is the common underlying process linking each of these phenomena. To be perceived as beautiful, being surrounded by unattractive friends may help. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Adaptation reveals that facial expression averaging occurs during rapid serial presentation.

How do we interpret the rapidly changing visual stimuli we encounter? How does our past visual experience shape our perception? Recent work has suggested that our visual system is able to interpret multiple faces presented temporally via integration or ensemble coding. Visual adaptation is widely used to probe such short term plasticity. Here we use an adaptation paradigm to investigate whether integration or averaging of emotional faces occurs during a rapid serial visual presentation (RSVP). In four experiments, we tested whether the RSVP of distinct emotional faces could induce adaptation aftereffects and whether these aftereffects were of similar magnitudes as their statistically averaged face. Experiment 1 showed that the RSVP faces could generate significant facial expression aftereffects (FEAs) across happy and sad emotions. Experiment 2 revealed that the magnitudes of the FEAs from RSVP faces and their paired average faces were comparable and significantly correlated. Experiment 3 showed that the FEAs depended on the mean emotion of the face stream, regardless of variations in emotion or the temporal frequency of the stream. Experiment 4 further indicated that the emotion of the average face of the stream, but not the emotion of individual faces matched for identity to the test faces, determined the FEAs. Together, our results suggest that the visual system interprets rapidly presented faces by ensemble coding, and thus implies the formation of a facial expression norm in face space.