The Crowd-Emotion-Amplification Effect.

How do people go about reading a room or taking the temperature of a crowd? When people catch a brief glimpse of an array of faces, they can focus their attention on only some of the faces. We propose that perceivers preferentially attend to faces exhibiting strong emotions and that this generates a crowd-emotion-amplification effect-estimating a crowd's average emotional response as more extreme than it actually is. Study 1 (N = 50) documented the crowd-emotion-amplification effect. Study 2 (N = 50) replicated the effect even when we increased exposure time. Study 3 (N = 50) used eye tracking to show that attentional bias to emotional faces drives amplification. These findings have important implications for many domains in which individuals must make snap judgments regarding a crowd's emotionality, from public speaking to controlling crowds.

Perceiving gaze from head and eye rotations: An integrative challenge for children and adults.

Gaze is an emergent visual feature. A person's gaze direction is perceived not just based on the rotation of their eyes, but also their head. At least among adults, this integrative process appears to be flexible such that one feature can be weighted more heavily than the other depending on the circumstances. Yet it is unclear how this weighting might vary across individuals or across development. When children engage emergent gaze, do they prioritize cues from the head and eyes similarly to adults? Is the perception of gaze among individuals with autism spectrum disorder (ASD) emergent, or is it reliant on a single feature? Sixty adults (M = 29.86 years-of-age), thirty-seven typically developing children and adolescents (M = 9.3 years-of-age; range = 7-15), and eighteen children with ASD (M = 9.72 years-of-age; range = 7-15) viewed faces with leftward, rightward, or direct head rotations in conjunction with leftward or rightward pupil rotations, and then indicated whether the face was looking leftward or rightward. All individuals, across development and ASD status, used head rotation to infer gaze direction, albeit with some individual differences. However, the use of pupil rotation was heavily dependent on age. Finally, children with ASD used pupil rotation significantly less than typically developing (TD) children when inferring gaze direction, even after accounting for age. Our approach provides a novel framework for understanding individual and group differences in gaze as it is actually perceived-as an emergent feature. Furthermore, this study begins to address an important gap in ASD literature, taking the first look at emergent gaze perception in this population.

Perceptual averaging of facial expressions requires visual awareness and attention.

Humans, as highly social animals, are regularly exposed to the faces of conspecifics-often more than one at a time. This feature of social living is important for understanding face perception, not just because it means that information from faces is available in bulk, but also because it changes the way individuals are perceived. For instance, when two faces are seen nearby one another, they tend to look like each other. This phenomenon of perceptual averaging is robust when both faces are seen and attended. But in everyday life, some faces may not receive the full benefit of attention, or they may not be visible at all. We evaluated whether perceptual averaging of relatively complex and simple information on faces, including facial expression and head orientation, can still occur even in these circumstances. In particular, we used object-substitution masking (OSM) and a dual-task designed to disrupt visual awareness and attention, respectively, during evaluations of briefly presented face pairs. Disruptions of awareness or attention eliminated averaging of facial expression, whereas orientation averaging persisted in spite of these challenges. These results demonstrate boundary conditions for the process of perceptual averaging. More generally, they provide insight into how the visual system processes multitudes of objects, both simple and complex, both with and without attention and awareness.

Ensemble Perception of Dynamic Emotional Groups.

Crowds of emotional faces are ubiquitous, so much so that the visual system utilizes a specialized mechanism known as ensemble coding to see them. In addition to being proximally close, members of emotional crowds, such as a laughing audience or an angry mob, often behave together. The manner in which crowd members behave-in sync or out of sync-may be critical for understanding their collective affect. Are ensemble mechanisms sensitive to these dynamic properties of groups? Here, observers estimated the average emotion of a crowd of dynamic faces. The members of some crowds changed their expressions synchronously, whereas individuals in other crowds acted asynchronously. Observers perceived the emotion of a synchronous group more precisely than the emotion of an asynchronous crowd or even a single dynamic face. These results demonstrate that ensemble representation is particularly sensitive to coordinated behavior, and they suggest that shared behavior is critical for understanding emotion in groups.

Ensemble perception of size in 4-5-year-old children.

Groups of objects are nearly everywhere we look. Adults can perceive and understand the 'gist' of multiple objects at once, engaging ensemble-coding mechanisms that summarize a group's overall appearance. Are these group-perception mechanisms in place early in childhood? Here, we provide the first evidence that 4-5-year-old children use ensemble coding to perceive the average size of a group of objects. Children viewed a pair of trees, with each containing a group of differently sized oranges. We found that, in order to determine which tree had the larger oranges overall, children integrated the sizes of multiple oranges into ensemble representations. This pooling occurred rapidly, and it occurred despite conflicting information from numerosity, continuous extent, density, and contrast. An ideal observer analysis showed that although children's integration mechanisms are sensitive, they are not yet as efficient as adults'. Overall, our results provide a new insight into the way children see and understand the environment, and they illustrate the fundamental nature of ensemble coding in visual perception.

Perceiving crowd attention: ensemble perception of a crowd's gaze.

In nearly every interpersonal encounter, people readily gather socio-visual cues to guide their behavior. Intriguingly, social information is most effective in directing behavior when it is perceived in crowds. For example, the shared gaze of a crowd is more likely to direct attention than is a single person's gaze. Are people equipped with mechanisms to perceive a crowd's gaze as an ensemble? Here, we provide the first evidence that the visual system extracts a summary representation of a crowd's attention; observers rapidly pooled information from multiple crowd members to perceive the direction of a group's collective gaze. This pooling occurred in high-level stages of visual processing, with gaze perceived as a global-level combination of information from head and pupil rotation. These findings reveal an important and efficient mechanism for assessing crowd gaze, which could underlie the ability to perceive group intentions, orchestrate joint attention, and guide behavior.

Similarity in motion binds and bends judgments of aspect ratio.

It is well known that objects become grouped in perceptual organization when they share some visual feature, like a common direction of motion. Less well known is that grouping can change how people perceive a set of objects. For example, when a pair of shapes consistently share a common region of space, their aspect ratios tend to be perceived as more similar (are attracted toward each other). Conversely, when shapes are assigned to different regions in space their aspect ratios repel from each other. Here we examine whether the visual system produce both attractive and repulsive distortions when the state of grouping between a pair of shapes changes on a moment-to-moment basis. Observers viewed a pair of ellipses that differed in terms of how flat or tall they were and reported the aspect ratio of one ellipse from the pair. Each ellipse was defined by a cloud of coherently-moving dots, and the dots within the two ellipses had either the same or different directions of motion, varying from trial-to-trial. We found that the cued ellipse's aspect ratio was reported to be repelled from the aspect ratio of the uncued ellipse when the shapes had different directions of motion compared to when they had the same direction of motion. These results suggest that the visual system can adaptively alter visual experience based on grouping, in particular, repelling the appearance of objects when they do not appear to go together, and it can do so quickly and flexibly.

Anxious youth and adults share threat-biased interpretations of linguistic and visual ambiguity: A proof of concept study.

Interpretation bias, or the threatening appraisal of ambiguous information, has been linked to anxiety disorder. Interpretation bias has been demonstrated for linguistic (e.g., evaluation of ambiguous sentences) and visual judgments (e.g., categorizing emotionally ambiguous facial expressions). It is unclear how these separate components of bias might be associated. We examined linguistic and visual interpretation biases in youth and emerging adults with (n = 44) and without (n = 40) anxiety disorder, and in youth-parent dyads (n = 40). Linguistic and visual biases were correlated with each other, and with anxiety. Compared to non-anxious participants, those with anxiety demonstrated stronger biases, and linguistic bias was especially predictive of anxiety symptoms and diagnosis. Age did not moderate these relationships. Parent linguistic bias was correlated with youth anxiety but not linguistic bias; parent and youth visual biases were correlated. Linguistic and visual interpretation biases are linked in clinically-anxious youth and emerging adults.

Humans use summary statistics to perceive auditory sequences.

In vision, humans use summary statistics (e.g., the average facial expression of a crowd) to efficiently perceive the gist of groups of features. Here, we present direct evidence that ensemble coding is also important for auditory processing. We found that listeners could accurately estimate the mean frequency of a set of logarithmically spaced pure tones presented in a temporal sequence (Experiment 1). Their performance was severely reduced when only a subset of tones from a given sequence was presented (Experiment 2), which demonstrates that ensemble coding is based on a substantial number of the tones in a sequence. This precise ensemble coding occurred despite very limited representation of individual tones from the sequence: Listeners were poor at identifying specific individual member tones (Experiment 3) and at determining their positions in the sequence (Experiment 4). Together, these results indicate that summary statistical coding is not limited to visual processing and is an important auditory mechanism for extracting ensemble frequency information from sequences of sounds.

Gaze perception from head and pupil rotations in 2D and 3D: Typical development and the impact of autism spectrum disorder.

The study of gaze perception has largely focused on a single cue (the eyes) in two-dimensional settings. While this literature suggests that 2D gaze perception is shaped by atypical development, as in Autism Spectrum Disorder (ASD), gaze perception is in reality contextually-sensitive, perceived as an emergent feature conveyed by the rotation of the pupils and head. We examined gaze perception in this integrative context, across development, among children and adolescents developing typically or with ASD with both 2D and 3D stimuli. We found that both groups utilized head and pupil rotations to judge gaze on a 2D face. But when evaluating the gaze of a physically-present, 3D robot, the same ASD observers used eye cues less than their typically-developing peers. This demonstrates that emergent gaze perception is a slowly developing process that is surprisingly intact, albeit weakened in ASD, and illustrates how new technology can bridge visual and clinical science.

Amplification in the evaluation of multiple emotional expressions over time.

Social interactions are dynamic and unfold over time. To make sense of social interactions, people must aggregate sequential information into summary, global evaluations. But how do people do this? Here, to address this question, we conducted nine studies (N = 1,583) using a diverse set of stimuli. Our focus was a central aspect of social interaction-namely, the evaluation of others' emotional responses. The results suggest that when aggregating sequences of images and videos expressing varying degrees of emotion, perceivers overestimate the sequence's average emotional intensity. This tendency for overestimation is driven by stronger memory of more emotional expressions. A computational model supports this account and shows that amplification cannot be explained only by nonlinear perception of individual exemplars. Our results demonstrate an amplification effect in the perception of sequential emotional information, which may have implications for the many types of social interactions that involve repeated emotion estimation.

Internal curvature signal and noise in low- and high-level vision.

How does internal processing contribute to visual pattern perception? By modeling visual search performance, we estimated internal signal and noise relevant to perception of curvature, a basic feature important for encoding of three-dimensional surfaces and objects. We used isolated, sparse, crowded, and face contexts to determine how internal curvature signal and noise depended on image crowding, lateral feature interactions, and level of pattern processing. Observers reported the curvature of a briefly flashed segment, which was presented alone (without lateral interaction) or among multiple straight segments (with lateral interaction). Each segment was presented with no context (engaging low-to-intermediate-level curvature processing), embedded within a face context as the mouth (engaging high-level face processing), or embedded within an inverted-scrambled-face context as a control for crowding. Using a simple, biologically plausible model of curvature perception, we estimated internal curvature signal and noise as the mean and standard deviation, respectively, of the Gaussian-distributed population activity of local curvature-tuned channels that best simulated behavioral curvature responses. Internal noise was increased by crowding but not by face context (irrespective of lateral interactions), suggesting prevention of noise accumulation in high-level pattern processing. In contrast, internal curvature signal was unaffected by crowding but modulated by lateral interactions. Lateral interactions (with straight segments) increased curvature signal when no contextual elements were added, but equivalent interactions reduced curvature signal when each segment was presented within a face. These opposing effects of lateral interactions are consistent with the phenomena of local-feature contrast in low-level processing and global-feature averaging in high-level processing.

Awareness becomes necessary between adaptive pattern coding of open and closed curvatures.

Visual pattern processing becomes increasingly complex along the ventral pathway, from the low-level coding of local orientation in the primary visual cortex to the high-level coding of face identity in temporal visual areas. Previous research using pattern aftereffects as a psychophysical tool to measure activation of adaptive feature coding has suggested that awareness is relatively unimportant for the coding of orientation, but awareness is crucial for the coding of face identity. We investigated where along the ventral visual pathway awareness becomes crucial for pattern coding. Monoptic masking, which interferes with neural spiking activity in low-level processing while preserving awareness of the adaptor, eliminated open-curvature aftereffects but preserved closed-curvature aftereffects. In contrast, dichoptic masking, which spares spiking activity in low-level processing while wiping out awareness, preserved open-curvature aftereffects but eliminated closed-curvature aftereffects. This double dissociation suggests that adaptive coding of open and closed curvatures straddles the divide between weakly and strongly awareness-dependent pattern coding.

Simultaneous shape repulsion and global assimilation in the perception of aspect ratio.

Although local interactions involving orientation and spatial frequency are well understood, less is known about spatial interactions involving higher level pattern features. We examined interactive coding of aspect ratio, a prevalent two-dimensional feature. We measured perception of two simultaneously flashed ellipses by randomly post-cueing one of them and having observers indicate its aspect ratio. Aspect ratios interacted in two ways. One manifested as an aspect-ratio-repulsion effect. For example, when a slightly tall ellipse and a taller ellipse were simultaneously flashed, the less tall ellipse appeared flatter and the taller ellipse appeared even taller. This repulsive interaction was long range, occurring even when the ellipses were presented in different visual hemifields. The other interaction manifested as a global assimilation effect. An ellipse appeared taller when it was a part of a global vertical organization than when it was a part of a global horizontal organization. The repulsion and assimilation effects temporally dissociated as the former slightly strengthened, and the latter disappeared when the ellipse-to-mask stimulus onset asynchrony was increased from 40 to 140 ms. These results are consistent with the idea that shape perception emerges from rapid lateral and hierarchical neural interactions.

Ensemble perception includes information from multiple spatial scales.

Most visual scenes contain information at different spatial scales, including the local and global, or the detail and gist. Global processes have become increasingly implicated in research examining summary statistical perception, initially as the output of ensemble coding, and more recently as a gating mechanism for selecting which information is included in the averaging process itself. Yet local and global processing are known to be rapidly integrated by the visual system, and it is plausible that global-level information, like spatial organization, may be included as an input during ensemble coding. We tested this hypothesis using an ensemble shape-perception task in which observers evaluated the mean aspect ratios of sets of ellipses. In addition to varying the aspect ratios of the individual shapes, we independently varied the spatial arrangements of the sets so that they had either flat or tall organizations at the global level. We found that observers made precise summary judgments about the average aspect ratios of the sets by integrating information from multiple shapes. More importantly, global flat and tall organizations were incorporated into ensemble judgments about the sets; summary judgments were biased in the directions of the global spatial arrangements on each trial. This global-to-local integration even occurred when the global organizations were masked. Our results demonstrate that the process of summary representation can include information from both the local and global scales. The gist is not just an output of ensemble representation - it can be included as an input to the mechanism itself.

Anger bias in the evaluation of crowds.

People are good at categorizing the emotions of individuals and crowds of faces. People also make mistakes when classifying emotion. When they do so with judgments of individuals, these errors tend to be negatively biased, potentially serving a protective function. For example, a face with a subtle expression is more likely to be categorized as angry than happy. Yet surprisingly little is known about the errors people make when evaluating multiple faces. We found that perceivers were biased to classify faces as angry, especially when evaluating crowds. This amplified bias depended on uncertainty, occurring when categorization was difficult, and it reached peak intensity for crowds with four members. Drift diffusion modeling revealed the mechanisms behind this bias, including an early response component and more efficient processing of anger from crowds with subtle expressions. Our findings introduce bias as an important new dimension for understanding how perceivers make judgments about crowds. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Integration and segmentation conflict during ensemble coding of shape.

During ensemble coding, the visual system extracts summary information from input that has been integrated, facilitating gist-level judgments about objects and features that belong together. In contrast, input can be segmented, allowing for quick categorical distinctions between objects. Integration and segmentation usually work in parallel but may sometimes conflict in the context of ensemble coding. To investigate this possibility, we examined summary perception of aspect ratio (i.e., "tallness/flatness"). Aspect ratio has a category boundary (e.g., a circle), and individual aspect ratios may be perceptually exaggerated-segmented-away from this boundary. We predicted that summary perception of multiple aspect ratios would be disrupted when, as a set, they spanned the category boundary, since integration and segmentation would then be at odds. We found that when observers reported the average aspect ratio of a set of ellipses, they were less sensitive to the mean of sets that included both tall and flat ellipses, compared to sets comprised of tall or flat ellipses. Follow-up experiments suggest this occurred because segmentation distorted the appearance of ellipses away from the category boundary, exaggerating set heterogeneity. These experiments advance understanding of how the visual system summarizes information by showing that integration and segmentation can conflict. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Is this my group or not? The role of ensemble coding of emotional expressions in group categorization.

When exposed to others' emotional responses, people often make rapid decisions as to whether these others are members of their group or not. These group categorization decisions have been shown to be extremely important to understanding group behavior. Yet, despite their prevalence and importance, we know very little about the attributes that shape these categorization decisions. To address this issue, we took inspiration from ensemble coding research and developed a task designed to reveal the influence of the mean and variance of group members' emotions on participants' group categorization. In Study 1, we verified that group categorization decreases when the group's mean emotion is different from the participant's own emotional response. In Study 2, we established that people identify a group's mean emotion more accurately when its variance is low rather than high. In Studies 3 and 4, we showed that participants were more likely to self-categorize as members of groups with low emotional variance, even if their own emotions fell outside of the range of group emotions they saw, and that this preference is seen for judgments of both positive and negative group emotions. In Study 5, we showed that this unique preference for low group emotional variance is special to group categorization and does not appear in a more basic face categorization task. Our studies reveal unexplored and important tendencies in group categorization based on group emotions. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Long-lasting effects of subliminal affective priming from facial expressions.

Unconscious processing of stimuli with emotional content can bias affective judgments. Is this subliminal affective priming merely a transient phenomenon manifested in fleeting perceptual changes, or are long-lasting effects also induced? To address this question, we investigated memory for surprise faces 24 h after they had been shown with 30-ms fearful, happy, or neutral faces. Surprise faces subliminally primed by happy faces were initially rated as more positive, and were later remembered better, than those primed by fearful or neutral faces. Participants likely to have processed primes supraliminally did not respond differentially as a function of expression. These results converge with findings showing memory advantages with happy expressions, though here the expressions were displayed on the face of a different person, perceived subliminally, and not present at test. We conclude that behavioral biases induced by masked emotional expressions are not ephemeral, but rather can last at least 24 h.

Within-hemifield perceptual averaging of facial expressions predicted by neural averaging.

High-level visual neurons in the ventral stream typically have large receptive fields, supporting position-invariant object recognition but entailing poor spatial resolution. Consequently, when multiple objects fall within their large receptive fields, unless selective attention is deployed, their responses are averages of responses to the individual objects. We investigated a behavioral consequence of this neural averaging in the perception of facial expressions. Two faces (7 degrees -apart) were briefly presented (100-ms, backward-masked) either within the same visual hemifield (within-hemifield condition) or in different hemifields (between-hemifield condition). Face pairs included happy, angry, and valence-neutral faces, and observers rated the emotional valence of a post-cued face. Perceptual averaging of facial expressions was predicted only for the within-hemifield condition because the receptive fields of 'face-tuned' neurons are primarily confined within the contralateral field; the between-hemifield condition served to control for post-perceptual effects. Consistent with averaging, valence-neutral faces appeared more positive when paired with a happy face than when paired with an angry face, and affective intensities of happy and angry faces were reduced by accompanying valence-neutral or opposite-valence faces, in the within-hemifield relative to the between-hemifield condition. We thus demonstrated within-hemifield perceptual averaging of a complex feature as predicted by neural averaging in the ventral visual stream.