The functional role of spatial anisotropies in ensemble perception.

BACKGROUND: The human brain can rapidly represent sets of similar stimuli by their ensemble summary statistics, like the average orientation or size. Classic models assume that ensemble statistics are computed by integrating all elements with equal weight. Challenging this view, here, we show that ensemble statistics are estimated by combining parafoveal and foveal statistics in proportion to their reliability. In a series of experiments, observers reproduced the average orientation of an ensemble of stimuli under varying levels of visual uncertainty. RESULTS: Ensemble statistics were affected by multiple spatial biases, in particular, a strong and persistent bias towards the center of the visual field. This bias, evident in the majority of subjects and in all experiments, scaled with uncertainty: the higher the uncertainty in the ensemble statistics, the larger the bias towards the element shown at the fovea. CONCLUSION: Our findings indicate that ensemble perception cannot be explained by simple uniform pooling. The visual system weights information anisotropically from both the parafovea and the fovea, taking the intrinsic spatial anisotropies of vision into account to compensate for visual uncertainty.

Consistent social information perceived in animated backgrounds improves ensemble perception of facial expressions.

Observers can rapidly extract the mean emotion from a set of faces with remarkable precision, known as ensemble coding. Previous studies have demonstrated that matched physical backgrounds improve the precision of ongoing ensemble tasks. However, it remains unknown whether this facilitation effect still occurs when matched social information is perceived from the backgrounds. In two experiments, participants decided whether the test face in the retrieving phase appeared more disgusted or neutral than the mean emotion of the face set in the encoding phase. Both phases were paired with task-irrelevant animated backgrounds, which included either the forward movement trajectory carrying the "cooperatively chasing" information, or the backward movement trajectory conveying no such chasing information. The backgrounds in the encoding and retrieving phases were either mismatched (i.e., forward and backward replays of the same trajectory), or matched (i.e., two identical forward movement trajectories in Experiment 1, or two different forward movement trajectories in Experiment 2). Participants in both experiments showed higher ensemble precisions and better discrimination sensitivities when backgrounds matched. The findings suggest that consistent social information perceived from memory-related context exerts a context-matching facilitation effect on ensemble coding and, more importantly, this effect is independent of consistent physical information.

Attention Modulates the Ensemble Coding of Facial Expressions.

Ensemble coding and attention are two mechanisms utilized by our visual system to overcome the limitation of visual processing when confronted with the overwhelming visual information. Recent evidence in ensemble coding of size suggests that the attended items contributed more to the averaging. On the other hand, some new evidence also indicates that reduced attention jeopardies the perceptual averaging of stimuli. What is the relationship between attention and ensemble coding? To answer this question, in the current study, we tested whether an exogenous attentional cue would influence the reported mean emotion of a crowd. We showed participants a group of four faces with different emotions. Participants' attention was guided to the happiest or saddest face (attention conditions), or not to any specific face (baseline condition). The results supported the notion that the attention alters the ensemble perception of the facial expression by elevating the weight of that face in the ensemble representation. This opens the question for the neural mechanisms of ensemble coding and its connection to visual attention.

Neural representations of ensemble coding in the occipital and parietal cortices.

The human visual system is able to extract summary statistics from sets of similar items, but the underlying neural mechanism remains poorly understood. Using functional magnetic resonance imaging (fMRI) and an encoding model, we examined how the neural representation of ensemble coding is constructed by manipulating the task-relevance of ensemble features. We found a gradual increase in orientation-selective responses to the mean orientation of multiple stimuli along the visual hierarchy only when these orientations were task-relevant. Such responses to the ensemble orientation were present in the extrastriate area, V3, even when the mean orientation was not task-relevant, indicating that the ensemble representation can co-exist with the task-relevant individual feature representation. Ensemble orientations were also represented in frontal regions, but those representations were robust only when each mean orientation was linked to a motor response dimension. Together, our findings suggest that the neural representation of the ensemble percept is formed by pooling signals at multiple levels of the visual processing stream.

Differential neurodynamics and connectivity in the dorsal and ventral visual pathways during perception of emotional crowds and individuals: a MEG study.

Reading the prevailing emotion of groups of people ("crowd emotion") is critical to understanding their overall intention and disposition. It alerts us to potential dangers, such as angry mobs or panicked crowds, giving us time to escape. A critical aspect of processing crowd emotion is that it must occur rapidly, because delays often are costly. Although knowing the timing of neural events is crucial for understanding how the brain guides behaviors using coherent signals from a glimpse of multiple faces, this information is currently lacking in the literature on face ensemble coding. Therefore, we used magnetoencephalography to examine the neurodynamics in the dorsal and ventral visual streams and the periamygdaloid cortex to compare perception of groups of faces versus individual faces. Forty-six participants compared two groups of four faces or two individual faces with varying emotional expressions and chose which group or individual they would avoid. We found that the dorsal stream was activated as early as 68 msec after the onset of stimuli containing groups of faces. In contrast, the ventral stream was activated later and predominantly for individual face stimuli. The latencies of the dorsal stream activation peaks correlated with participants' response times for facial crowds. We also found enhanced connectivity earlier between the periamygdaloid cortex and the dorsal stream regions for crowd emotion perception. Our findings suggest that ensemble coding of facial crowds proceeds rapidly and in parallel by engaging the dorsal stream to mediate adaptive social behaviors, via a distinct route from single face perception.

Exaggerated groups: amplification in ensemble coding of temporal and spatial features.

The human visual system represents summary statistical information (e.g. average) along many visual dimensions efficiently. While studies have indicated that approximately the square root of the number of items in a set are effectively integrated through this ensemble coding, how those samples are determined is still unknown. Here, we report that salient items are preferentially weighted over the other less salient items, by demonstrating that the perceived means of spatial (i.e. size) and temporal (i.e. flickering temporal frequency (TF)) features of the group of items are positively biased as the number of items in the group increases. This illusory 'amplification effect' was not the product of decision bias but of perceptual bias. Moreover, our visual search experiments with similar stimuli suggested that this amplification effect was due to attraction of visual attention to the salient items (i.e. large or high TF items). These results support the idea that summary statistical information is extracted from sets with an implicit preferential weighting towards salient items. Our study suggests that this saliency-based weighting may reflect a more optimal and efficient integration strategy for the extraction of spatio-temporal statistical information from the environment, and may thus be a basic principle of ensemble coding.

Emotional cues and social anxiety resolve ambiguous perception of biological motion.

Perceptions of ambiguous biological motion are modulated by different individual cognitive abilities (such as inhibition and empathy) and emotional states (such as anxiety). This study explored facing-the-viewer bias (FTV) in perceiving ambiguous directions of biological motion, and investigated whether task-irrelevant simultaneous face emotional cues in the background and the individual social anxiety traits could affect FTV. We found that facial motion cues as background affect sociobiologically relevant scenarios, including biological motion, but not non-biological situations (conveyed through random dot motion). Individuals with high anxiety traits demonstrated a more dominant FTV bias than individuals with low anxiety traits. Ensemble coding-like processing of task-irrelevant multiple emotional cues could magnify the facing-the-viewer bias than did in the single emotional cue. Overall, those findings suggest a correlation between high-level emotional processing and high-level motion perception (subjective to attentional control) contributes to facing-the-viewer bias.

Getting to know you: The development of mechanisms underlying face learning.

Nearly every study investigating the development of face recognition has focused on the ability to tell people apart using one or two tightly controlled images to represent each identity. Such research ignores the challenge of recognizing the same person despite variability in appearance. Whereas natural variation in appearance makes unfamiliar faces difficult to recognize, by 6 years of age people easily recognize multiple images of familiar faces. Two mechanisms are proposed to underlie the process by which adults become familiar with newly encountered faces. We provide the first examination of the development of these mechanisms during childhood (6-11 years). In Experiment 1, we examined children's (6- to 10-year-olds') and adults' ability to engage in ensemble coding-the ability to rapidly extract an average representation of an identity from several instances. In Experiment 2, we examined children's ability to use within-person variability in appearance to recognize novel instances of a newly encountered identity. We created a child-friendly perceptual matching task, and the number of images to which participants were exposed varied across targets. Although children were less accurate than adults overall in Experiment 2, we found no age-related improvement in either ensemble coding or the ability to benefit from exposure to within-person variability in appearance when learning a new face, suggesting that both abilities are developed by 6 years of age. We discuss the implications of these findings for understanding the nature of mechanisms underlying face learning and other developmental processes such as language and music.

Acute systemic MK-801 induced functional uncoupling between hippocampal areas CA3 and CA1 with distant effect in the retrosplenial cortex.

The hippocampus and retrosplenial cortex are integrated within a higher-order cognitive circuit supporting relational (spatial, contextual, episodic) forms of learning and memory. Hippocampal place cells can coordinate multiple parallel representations in the same physical environment. Novel environment exploration triggers expression of immediate-early genes (IEGs) Arc and Homer1a in spatial context-specific ensembles of CA1 and CA3 neurons. Less is know about ensemble coding in the retrosplenial cortex (RSC), a region directly connected and functionally coupled to CA1. Hippocampal and retrosplenial damage is found in patients with schizophrenia alongside cognitive deficits affecting relational memory. Systemic administration of non-competitive NMDAR antagonists such as MK-801 is used to model psychosis in animals and humans. Acute systemic MK-801 (0.15 mg/kg) impaired cognitive control in rats and ensemble code for spatial context in CA1. Here, we use expression of immediate-early genes Arc and Homer 1a to examine ensemble coding in rat CA3 and RSC to test if the effect of MK-801 extends upstream and downstream of CA1, respectively. Different rats explored the same context twice (A/A), explored two distinct contexts (A/B) or remained in their home cage (CC). In contrast to CA1, MK-801 did not affect ensemble coding in CA3. Unlike CA3 and CA1, similarity of RSC ensembles active during exploration did not reflect change in spatial context, but MK-801 (0.15 mg/kg) increased similarity in RSC ensembles active during spontaneous behavior in the home cage. The data provide support for MK-801-induced functional uncoupling between CA3 and CA1 and suggest that ensemble coding deficit may extend downstream of CA1. This deficit may reflect hyperassociative state in the cognitive circuit underlying cognitive disorganization in psychosis. © 2016 Wiley Periodicals, Inc.

Perception of the average size of multiple objects in chimpanzees (Pan troglodytes).

Humans can extract statistical information, such as the average size of a group of objects or the general emotion of faces in a crowd without paying attention to any individual object or face. To determine whether summary perception is unique to humans, we investigated the evolutional origins of this ability by assessing whether chimpanzees, which are closely related to humans, can also determine the average size of multiple visual objects. Five chimpanzees and 18 humans were able to choose the array in which the average size was larger, when presented with a pair of arrays, each containing 12 circles of different or the same sizes. Furthermore, both species were more accurate in judging the average size of arrays consisting of 12 circles of different or the same sizes than they were in judging the average size of arrays consisting of a single circle. Our findings could not be explained by the use of a strategy in which the chimpanzee detected the largest or smallest circle among those in the array. Our study provides the first evidence that chimpanzees can perceive the average size of multiple visual objects. This indicates that the ability to compute the statistical properties of a complex visual scene is not unique to humans, but is shared between both species.

Ensemble Perception of Dynamic Emotional Groups.

Crowds of emotional faces are ubiquitous, so much so that the visual system utilizes a specialized mechanism known as ensemble coding to see them. In addition to being proximally close, members of emotional crowds, such as a laughing audience or an angry mob, often behave together. The manner in which crowd members behave-in sync or out of sync-may be critical for understanding their collective affect. Are ensemble mechanisms sensitive to these dynamic properties of groups? Here, observers estimated the average emotion of a crowd of dynamic faces. The members of some crowds changed their expressions synchronously, whereas individuals in other crowds acted asynchronously. Observers perceived the emotion of a synchronous group more precisely than the emotion of an asynchronous crowd or even a single dynamic face. These results demonstrate that ensemble representation is particularly sensitive to coordinated behavior, and they suggest that shared behavior is critical for understanding emotion in groups.

Effect of Vertical Stretching on the Extraction of Mean Identity From Faces.

Observers can extract the mean identity from a set of faces and falsely recognise it as a genuine set member. The current experiment demonstrated that this 'perceptual averaging' also occurs with vertically stretched faces. On each trial, participants decided whether a target face was present in a preceding set of four faces. In the control condition, the faces were all normally proportioned; in the stretched set condition, the face sets were stretched but the targets were normal; and in the stretched target condition, the face sets were normal but the targets were stretched. In all three conditions, participants falsely identified the set mean as a face that had been presented within the set, implying that this identity-averaging effect is based on high-level identity information rather than the low-level physical characteristics of the face stimuli.

Perceiving crowd attention: ensemble perception of a crowd's gaze.

In nearly every interpersonal encounter, people readily gather socio-visual cues to guide their behavior. Intriguingly, social information is most effective in directing behavior when it is perceived in crowds. For example, the shared gaze of a crowd is more likely to direct attention than is a single person's gaze. Are people equipped with mechanisms to perceive a crowd's gaze as an ensemble? Here, we provide the first evidence that the visual system extracts a summary representation of a crowd's attention; observers rapidly pooled information from multiple crowd members to perceive the direction of a group's collective gaze. This pooling occurred in high-level stages of visual processing, with gaze perceived as a global-level combination of information from head and pupil rotation. These findings reveal an important and efficient mechanism for assessing crowd gaze, which could underlie the ability to perceive group intentions, orchestrate joint attention, and guide behavior.

Ensemble crowd perception: a viewpoint-invariant mechanism to represent average crowd identity.

Individuals can rapidly and precisely judge the average of a set of similar items, including both low-level (Ariely, 2001) and high-level objects (Haberman & Whitney, 2007). However, to date, it is unclear whether ensemble perception is based on viewpoint-invariant object representations. Here, we tested this question by presenting participants with crowds of sequentially presented faces. The number of faces in each crowd and the viewpoint of each face varied from trial to trial. This design required participants to integrate information from multiple viewpoints into one ensemble percept. Participants reported the mean identity of crowds (e.g., family resemblance) using an adjustable, forward-oriented test face. Our results showed that participants accurately perceived the mean crowd identity even when required to incorporate information across multiple face orientations. Control experiments showed that the precision of ensemble coding was not solely dependent on the length of time participants viewed the crowd. Moreover, control analyses demonstrated that observers did not simply sample a subset of faces in the crowd but rather integrated many faces into their estimates of average crowd identity. These results demonstrate that ensemble perception can operate at the highest levels of object recognition after 3-D viewpoint-invariant faces are represented.

Ensemble coding of facial identity is robust, but may not contribute to face learning.

Ensemble coding - the rapid extraction of a perceptual average - has been proposed as a potential mechanism underlying face learning. We tested this proposal across five pre-registered experiments in which four ambient images of an identity were presented in the study phase. In Experiments 1 and 2a-c, participants were asked whether a test image was in the study array; these experiments examined the robustness of ensemble coding. Experiment 1 replicated ensemble coding in an online sample; participants recognize images from the study array and the average of those images. Experiments 2a-c provide evidence that ensemble coding meets several criteria of a possible learning mechanism: It is robust to changes in head orientation (± 60o), survives a short (30s) delay, and persists when images of two identities are interleaved during the study phase. Experiment 3 examined whether ensemble coding is sufficient for face learning (i.e., facilitates recognition of novel images of a target identity). Each study array comprised four ambient images (variability + average), a single image, or an average of four images (average only). Participants were asked whether a novel test image showed the identity from a study array. Performance was best in the four-image condition, with no difference between the single-image and average-only conditions. We conclude that ensemble coding of facial identity is robust but that the perceptual average per se is not sufficient for face learning.

The role of visual working memory in capacity-limited cross-modal ensemble coding.

Ensemble coding refers to the brain's ability to rapidly extract summary statistics, such as average size and average cost, from a large set of visual stimuli. Although ensemble coding is thought to circumvent a capacity limit of visual working memory, we recently observed a VWM-like capacity limit in an ensemble task where observers extracted the average sweetness of groups of food pictures (i.e., they could only integrate information from four out of six available items), thus suggesting the involvement of VWM in this novel form of cross-modal ensemble coding. Therefore, across two experiments we investigated if this cross-modal ensemble capacity limit could be explained by individual differences in VWM processing. To test this, observers performed both an ensemble task and a VWM task, and we determined 1) how much information they integrated into their ensemble percepts, and 2) how much information they remembered from those displays. Interestingly, we found that individual differences in VWM capacity did not explain differences in performance on the ensemble coding task (i.e., high-capacity individuals did not have significantly higher "ensemble abilities" than low-capacity individuals). While our data cannot definitively state whether or not VWM is necessary to perform the ensemble task, we conclude that it is certainly not sufficient to support this cognitive process. We speculate that the capacity limit may be explained by 1) a bottleneck at the perceptual stage (i.e., a failure to process multiple visual features across multiple items, as there are no singular features that convey taste), or 2) the interaction of multiple cognitive systems (e.g., VWM, gustatory working memory, long term memory). Our results highlight the importance of examining ensemble perception across multiple sensory and cognitive domains to provide a clearer picture of the mechanisms underlying everyday behavior.

Sex ratios in vocal ensembles affect perceptions of threat and belonging.

People often interact with groups (i.e., ensembles) during social interactions. Given that group-level information is important in navigating social environments, we expect perceptual sensitivity to aspects of groups that are relevant for personal threat as well as social belonging. Most ensemble perception research has focused on visual ensembles, with little research looking at auditory or vocal ensembles. Across four studies, we present evidence that (i) perceivers accurately extract the sex composition of a group from voices alone, (ii) judgments of threat increase concomitantly with the number of men, and (iii) listeners' sense of belonging depends on the number of same-sex others in the group. This work advances our understanding of social cognition, interpersonal communication, and ensemble coding to include auditory information, and reveals people's ability to extract relevant social information from brief exposures to vocalizing groups.

Other ethnicity effects in ensemble coding of facial expressions.

Cultural difference in ensemble emotion perception is an important research question, providing insights into the complexity of human cognition and social interaction. Here, we conducted two experiments to investigate how emotion perception would be affected by other ethnicity effects and ensemble coding. In Experiment 1, two groups of Asian and Caucasian participants were tasked with assessing the average emotion of faces from their ethnic group, other ethnic group, and mixed ethnicity groups. Results revealed that participants exhibited relatively accurate yet amplified emotion perception of their group faces, with a tendency to overestimate the weight of the faces from the other ethnic group. In Experiment 2, Asian participants were instructed to discern the emotion of a target face surrounded by faces from Caucasian and Asian faces. Results corroborated earlier findings, indicating that while participants accurately perceived emotions in faces of their ethnicity, their perception of Caucasian faces was noticeably influenced by the presence of surrounding Asian faces. These findings collectively support the notion that the other ethnicity effect stems from differential emotional amplification inherent in ensemble coding of emotion perception.

Ensemble Coding of Crowd with Cross-Category Facial Expressions.

Ensemble coding allows observers to form an average to represent a set of elements. However, it is unclear whether observers can extract an average from a cross-category set. Previous investigations on this issue using low-level stimuli yielded contradictory results. The current study addressed this issue by presenting high-level stimuli (i.e., a crowd of facial expressions) simultaneously (Experiment 1) or sequentially (Experiment 2), and asked participants to complete a member judgment task. The results showed that participants could extract average information from a group of cross-category facial expressions with a short perceptual distance. These findings demonstrate cross-category ensemble coding of high-level stimuli, contributing to the understanding of ensemble coding and providing inspiration for future research.

Automatic processing of unattended mean emotion: Evidence from visual mismatch responses.

The mean emotion from multiple facial expressions can be extracted rapidly and precisely. However, it remains debated whether mean emotion processing is automatic which can occur under no attention. To address this question, we used a passive oddball paradigm and recorded event-related brain potentials when participants discriminated the changes in the central fixation while a set of four faces was presented in the periphery. The face set consisted of one happy and three angry expressions (mean negative) or one angry and three happy expressions (mean positive), and the mean negative and mean positive face sets were shown with a probability of 20% (deviant) and 80% (standard) respectively in the sequence, or the vice versa. The cluster-based permutation analyses showed that the visual mismatch negativity started early at around 92 ms and was also observed in later time windows when the mean emotion was negative, while a mismatch positivity was observed at around 168-266 ms when the mean emotion was positive. The results suggest that there might be different mechanisms underlying the processing of mean negative and mean positive emotions. More importantly, the brain can detect the changes in the mean emotion automatically, and ensemble coding for multiple facial expressions can occur in an automatic fashion without attention.