How neural representations of newly learnt faces change over time: Event-related brain potential evidence for overnight consolidation.

Previous experiments have shown that a brief encounter with a previously unfamiliar person leads to the establishment of new facial representations, which can be activated by completely novel pictures of the newly learnt face. The present study examined how stable such novel neural representations are over time, and, specifically, how they become consolidated within the first 24 h after learning. Using event-related brain potentials (ERPs) in a between-participants design, we demonstrate that clear face familiarity effects in the occipito-temporal N250 are evident immediately after learning. These effects then undergo change, with a nearly complete absence of familiarity-related ERP differences 4 h after the initial encounter. Critically, 24 h after learning, the original familiarity effect re-emerges. These findings suggest that the neural correlates of novel face representations are not stable over time but change during the first day after learning. The resulting pattern of change is consistent with a process of consolidation.

Personal familiarity of faces, animals, objects, and scenes: Distinct perceptual and overlapping conceptual representations.

While face, object, and scene recognition are often studied at a basic categorization level (e.g. "a face", "a car", "a kitchen"), we frequently recognise individual items of these categories as unique entities (e.g. "my mother", "my car", "my kitchen"). This recognition of individual identity is essential to appropriate behaviour in our world. However, relatively little is known about how we recognise individually familiar visual stimuli. Using event-related brain potentials, the present study examined whether and to what extent the underlying neural representations of personally familiar items are similar or different across different categories. In three experiments, we examined the recognition of personally highly familiar faces, animals, indoor scenes, and objects. We observed relatively distinct familiarity effects in an early time window (200-400 ms), with a clearly right-lateralized occipito-temporal scalp distribution for human faces and more bilateral and posterior distributions for other stimulus categories, presumably reflecting access to at least partly discrete visual long-term representations. In contrast, we found clearly overlapping familiarity effects in a later time window (starting 400 to 500 ms after stimulus onset), again with a mainly right occipito-temporal scalp distribution, for all stimulus categories. These later effects appear to reflect the sustained activation of conceptual properties relevant to any potential interaction. We conclude that familiarity for items from the various visual stimulus categories tested here is represented differently at the perceptual level, while relatively overlapping conceptual mechanisms allow for the preparation of impending potential interaction with the environment.

A first impression of the future.

We offer a response to six commentaries on our target article 'Understanding trait impressions from faces'. A broad consensus emerged with authors emphasizing the importance of increasing the diversity of faces and participants, integrating research on impressions beyond the face, and continuing to develop methods needed for data-driven approaches. We propose future directions for the field based on these themes.

Understanding trait impressions from faces.

Impressions from faces are made remarkably quickly and they can underpin behaviour in a wide variety of social contexts. Over the last decade many studies have sought to trace the links between facial cues and social perception and behaviour. One such body of work has shown clear overlap between the fields of face perception and social stereotyping by demonstrating a role for conceptual stereotypes in impression formation from faces. We integrate these results involving conceptual influences on impressions with another substantial body of research in visual cognition which demonstrates that much of the variance in impressions can be predicted from perceptual, data-driven models using physical cues in face images. We relate this discussion to the phylogenetic, cultural, individual and developmental origins of facial impressions and define priority research questions for the field including investigating non-WEIRD cultures, tracking the developmental trajectory of impressions and determining the malleability of impression formation.

The roles of shape and texture in the recognition of familiar faces.

The surface texture of the face is proposed to be the dominant cue in face recognition. In this study, we investigated the role of shape information in face recognition. We compared the roles of shape and surface texture in the recognition of face identity using familiar and unfamiliar hybrid faces in which the average shape from one facial identity was combined with the average texture of a different identity. In the first experiment (n = 53), participants had to match the name of a familiar person to one of eight hybrid face images. In texture trials, all images had the correct shape, but only one image had the correct texture. In shape trials, all images had the correct texture, but only one image had the correct shape. Importantly, neither task could be performed by perceptual matching. Although performance was lower for the shape trials (81%) compared to texture trials (99%), both were significantly above chance (12.5%). In the second experiment (n = 110), participants had to name hybrid faces. There were two potentially correct answers for each face image: one based on the texture and one based on the shape. Participants reported the correct name based on the texture on 61% of trials and the correct name based on the shape on 12% of trials. In the third experiment (n = 19), fMR-adaptation was used to measure the neural sensitivity to changes in the shape or texture. The core face-selective regions showed a similar sensitivity to shape and texture. These findings confirm that texture is the dominant cue for face recognition, but also show that shape plays an important role in the recognition and neural response to familiar faces.

Familiarity is familiarity is familiarity: Event-related brain potentials reveal qualitatively similar representations of personally familiar and famous faces.

Humans excel in familiar face recognition, but often find it hard to make identity judgements of unfamiliar faces. Understanding of the factors underlying the substantial benefits of familiarity is at present limited, but the effect is sometimes qualified by the way in which a face is known-for example, personal acquaintance sometimes gives rise to stronger familiarity effects than exposure through the media. Given the different quality of personal versus media knowledge, for example in one's emotional response or level of interaction, some have suggested qualitative differences between representations of people known personally or from media exposure. Alternatively, observed differences could reflect quantitative differences in the level of familiarity. We present 4 experiments investigating potential contributory influences to face familiarity effects in which observers view pictures showing their friends, favorite celebrities, celebrities they dislike, celebrities about whom they have expressed no opinion, and their own face. Using event-related potential indices with high temporal resolution and multiple highly varied everyday ambient images as a strong test of face recognition, we focus on the N250 and the later Sustained Familiarity Effect (SFE). All known faces show qualitatively similar responses relative to unfamiliar faces. Regardless of personal- or media-based familiarity, N250 reflects robust visual representations, successively refined over increasing exposure, while SFE appears to reflect the amount of identity-specific semantic information known about a person. These modulations of visual and semantic representations are consistent with face recognition models which emphasize the degree of familiarity but do not distinguish between different types of familiarity. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Predicting attractiveness from face parts reveals multiple covarying cues.

In most studies of facial attractiveness perception, judgments are based on the whole face images. Here we investigated how attractiveness judgments from parts of faces compare to perceived attractiveness of the whole face, and to each other. We manipulated the extent and regions of occlusion, where either the left/right or the top/bottom half of the face was occluded. We also further segmented the face into relatively small horizontal regions involving the forehead, eyes, nose, or mouth. The results demonstrated the correlated nature of face regions, such that an attractiveness judgment for one face part can be highly predictive of the attractiveness of the whole face or the other parts. The left/right half of the face created more accurate predictions than the top/bottom half. Judgments involving a larger area of the face (i.e., left/right or top/bottom halves) produced more accurate predictions than those derived from smaller regions, such as the eyes or the mouth alone, but even the smallest and most featureless region investigated (the forehead) provided useful information. The correlated nature of the attractiveness of face parts shows that perceived attractiveness is determined by multiple covarying cues that the visual system can exploit to determine attractiveness from a single glance.

Detecting a viewer's familiarity with a face: Evidence from event-related brain potentials and classifier analyses.

Human observers recognize the faces of people they know efficiently and without apparent effort. Consequently, recognizing a familiar face is often assumed to be an automatic process beyond voluntary control. However, there are circumstances in which a person might seek to hide their recognition of a particular face. The present study therefore used event-related potentials (ERPs) and a classifier based on logistic regression to determine if it is possible to detect whether a viewer is familiar with a particular face, regardless of whether the participant is willing to acknowledge it or not. In three experiments, participants were presented with highly variable "ambient" images of personally familiar and unfamiliar faces, while performing an incidental butterfly detection task (Experiment 1), an explicit familiarity judgment task (Experiment 2), and a concealed familiarity task in which they were asked to deny familiarity with one truly known facial identity while acknowledging familiarity with a second known identity (Experiment 3). In all three experiments, we observed substantially more negative ERP amplitudes at occipito-temporal electrodes for familiar relative to unfamiliar faces starting approximately 200 ms after stimulus onset. Both the earlier N250 familiarity effect, reflecting visual recognition of a known face, and the later sustained familiarity effect, reflecting the integration of visual with additional identity-specific information, were similar across experiments and thus independent of task demands. These results were further supported by the classifier analysis. We conclude that ERP correlates of familiar face recognition are largely independent of voluntary control and discuss potential applications in forensic settings.

The interplay between gaze cueing and facial trait impressions.

The gaze cueing effect involves the rapid orientation of attention to follow the gaze direction of another person. Previous studies reported reciprocal influences between social variables and the gaze cueing effect, with modulation of gaze cueing by social features of face stimuli and modulation of the observer's social judgements from the validity of the gaze cues themselves. However, it remains unclear which social dimensions can affect-and be affected by-gaze cues. We used computer-averaged prototype face-like images with high and low levels of perceived trustworthiness and dominance to investigate the impact of these two fundamental social impression dimensions on the gaze cueing effect. Moreover, by varying the proportions of valid and invalid gaze cues across three experiments, we assessed whether gaze cueing influences observers' impressions of dominance and trustworthiness through incidental learning. Bayesian statistical analyses provided clear evidence that the gaze cueing effect was not modulated by facial social trait impressions (Experiments 1-3). However, there was uncertain evidence of incidental learning of social evaluations following the gaze cueing task. A decrease in perceived trustworthiness for non-cooperative low dominance faces (Experiment 2) and an increase in dominance ratings for faces whose gaze behaviour contradicted expectations (Experiment 3) appeared, but further research is needed to clarify these effects. Thus, this study confirms that attentional shifts triggered by gaze direction involve a robust and relatively automatic process, which could nonetheless influence social impressions depending on perceived traits and the gaze behaviour of faces providing the cues.

Trait evaluations of faces and voices: Comparing within- and between-person variability.

Human faces and voices are rich sources of information that can vary in many different ways. Most of the literature on face/voice perception has focused on understanding how people look and sound different to each other (between-person variability). However, recent studies highlight the ways in which the same person can look and sound different on different occasions (within-person variability). Across three experiments, we examined how within- and between-person variability relate to one another for social trait impressions by collecting trait ratings attributed to multiple face images and voice recordings of the same people. We find that within-person variability in social trait evaluations is at least as great as between-person variability. Using different stimulus sets across experiments, trait impressions of voices are consistently more variable within people than between people-a pattern that is only evident occasionally when judging faces. Our findings highlight the importance of understanding within-person variability, showing how judgments of the same person can vary widely on different encounters and quantify how this pattern differs for voice and face perception. The work consequently has implications for theoretical models proposing that voices can be considered "auditory faces" and imposes limitations to the "kernel of truth" hypothesis of trait evaluations. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Face perception across the adult lifespan: evidence for age-related changes independent of general intelligence.

It is well-documented that face perception - including facial expression and identity recognition ability - declines with age. To date, however, it is not yet well understood whether this age-related decline reflects face-specific effects, or instead can be accounted for by well-known declines in general intelligence. We examined this issue using a relatively large, healthy, age-diverse (18-88 years) sample (N = 595) who were assessed on well-established measures of face perception and general intelligence. Replicating previous work, we observed that facial expression recognition, facial identity recognition, and general intelligence all showed declines with age. Of importance, the age-related decline of expression and identity recognition was present even when the effects of general intelligence were statistically controlled. Moreover, facial expression and identity ability each showed significant unique associations with age. These results indicate that face perception ability becomes poorer as we age, and that this decline is to some extent relatively focal in nature. Results are in line with a hierarchical structure of face perception ability, and suggest that age appears to have independent effects on the general and specific face processing levels within this structure.

Insights from computational models of face recognition: A reply to Blauch, Behrmann and Plaut.

We agree with Blauch, Behrmann, and Plaut (2020) on a number of points, and are reassured that their data bear out our previous findings. We discuss differences in modelling style, and the usefulness of different types of model for supporting psychological understanding. We emphasise the role that within-person variability plays in recognising familiar faces and clarify the range over which it is idiosyncratic. The combination of image analysis with top-down support to cohere different images of the same person seems to be an important characteristic of successful models.

Consistent evidence of a link between Alexithymia and general intelligence.

Alexithymia is a personality construct characterised most notably by a difficulty in identifying and expressing feelings. Although the emotional difficulties in alexithymia are well established, to date little work has examined its relationship to broader cognitive abilities, such as general intelligence. Across three independent, healthy adult samples (Ns = 389, 318, & 273), we examined whether alexithymia was associated with general intelligence. In all three samples, we observed a significant negative association between alexithymia and general intelligence. In two of the samples, general intelligence was a significant predictor of alexithymia even when accounting for performance on tests of facial emotion recognition ability and supramodal emotion recognition ability (measured with faces, bodies, and voices). From a theoretical perspective, these results suggest that models of alexithymia need to incorporate a role for more generalised cognitive functioning. From a practical perspective, studies examining links between alexithymia and clinical disorders, many of which have known links to general intelligence, should consider including a measure of general intelligence in order to adjust for this potentially confounding factor.

Face and Voice Perception: Understanding Commonalities and Differences.

Faces and voices are of high importance in interpersonal communication, and there are notable parallels between face and voice perception. However, these parallels do not sit entirely comfortably with the full range of available evidence. In this review, we evaluate parallels between the functional and neural organisation of face and voice perception, while locating these in the context of ways in which faces and voices also differ. We take the discussion to the next level by asking why these commonalities and differences exist. A novel synthesis is offered, grounded in the interaction between intrinsic characteristics of faces and voices and the demands of everyday life, showing how the pattern of findings reflects a system that can respond optimally to different everyday demands.

Perceptual integration and the composite face effect.

The composite face paradigm is widely used to investigate holistic perception of faces. In the paradigm, parts from different faces (usually the top and bottom halves) are recombined. The principal criterion for holistic perception is that responses involving the component parts of composites in which the parts are aligned into a face-like configuration are disrupted compared with the same parts in a misaligned (not face-like) format. This is often taken as evidence that seeing a whole face in the aligned condition interferes with perceiving its separate parts, but the extent to which the effect is perceptually driven remains unclear. We used salient perceptual categories of gender (male or female) and race (Asian or Caucasian appearance) to create composite stimuli from parts of faces that varied orthogonally on these characteristics. In Experiment 1, participants categorised the gender of the parts of aligned composite and misaligned images created from parts with the same (congruent) or different (incongruent) gender and the same (congruent) or different (incongruent) race. In Experiment 2, the same stimuli were used but the task changed to categorising race. In both experiments, there was a strong influence of the task-relevant manipulation on the composite effect, with slower responses to aligned stimuli with incongruent gender in Experiment 1 and incongruent race in Experiment 2. In contrast, the task-irrelevant variable (race in Experiment 1, gender in Experiment 2) did not exert much influence on the composite effect in either experiment. These findings show that although holistic integration of salient visual properties makes a strong contribution to the composite face effect, it clearly also involves targeted processing of an attended visual characteristic.

Emotion recognition ability: Evidence for a supramodal factor and its links to social cognition.

Accurate recognition of others' emotions is an important skill for successful social interaction. Unsurprisingly, it has been an enduring topic of interest, and notable individual differences have been observed. Despite this focus, the underlying functional architecture of this ability has not been thoroughly investigated, particularly concerning emotion recognition across different sensory domains and stimulus modalities. Using a structural equation modelling approach, Study 1 (N = 284) established the structure of emotion recognition ability across three expressive domains - face, body and voice - and observed strong evidence for a superordinate 'supramodal' emotion recognition factor, over and above domain-specific factors. Additionally, we observed a significant moderate negative association between this superordinate factor and alexithymia. In Study 2 (N = 218), findings indicated that supramodal emotion recognition ability and face identity recognition are two related but independent constructs. In Study 3 (N = 249), we examined links from both supramodal emotion recognition and face identity recognition to broader cognitive ability, and observed that general intelligence was a significant predictor of supramodal emotion recognition ability. In contrast, there was no association between intelligence and face identity recognition ability. Across three independent samples, then, our findings offer strong support for an emotion recognition ability factor existing across visual and auditory domains encompassing social signals conveyed by face, body and voice, and outline its associations to broader cognitive and affective traits.

Do facial first impressions reflect a shared social reality?

Influential facial impression models have repeatedly shown that trustworthiness, youthful-attractiveness, and dominance dimensions subserve a wide variety of first impressions formed from strangers' faces, suggestive of a shared social reality. However, these models are built from impressions aggregated across observers. Critically, recent work has now shown substantial inter-observer differences in facial impressions, raising the important question of whether these dimensional models based on aggregated group data are meaningful at the individual observer level. We addressed this question with a novel case series approach, using factor analyses of ratings of twelve different traits to build individual models of facial impressions for different observers. Strikingly, three dimensions of trustworthiness, youthful/attractiveness, and competence/dominance appeared across the majority of these individual observer models, demonstrating that the dimensional approach is indeed meaningful at the individual level. Nonetheless, we also found differences in the stability of the competence/dominance dimension across observers. Taken together, results suggest that individual differences in impressions arise in the context of a largely common structure that supports a shared social reality.

Dose-dependent modulation of the visually evoked N1/N170 by perceptual surprise: a clear demonstration of prediction-error signalling.

Prediction-error checking processes play a key role in predictive coding models of perception. However, neural indices of such processes have yet to be unambiguously demonstrated. To date, experimental paradigms aiming to study such phenomena have relied upon the relative frequency of stimulus repeats and/or 'unexpected' events that are physically different from 'expected' events. These features of experimental design leave open alternative explanations for the observed effects. A definitive demonstration requires that presumed prediction error-related responses should show contextual dependency (rather than simply effects of frequency or repetition) and should not be attributable to low-level stimulus differences. Most importantly, prediction-error signals should show dose dependency with respect to the degree to which expectations are violated. Here, we exploit a novel experimental paradigm specifically designed to address these issues, using it to interrogate early latency event-related potentials (ERPs) to contextually expected and unexpected visual stimuli. In two electroencephalography (EEG) experiments, we demonstrate that an N1/N170 evoked potential is robustly modulated by unexpected perceptual events ('perceptual surprise') and shows dose-dependent sensitivity with respect to both the influence of prior information and the extent to which expectations are violated. This advances our understanding of perceptual predictions in the visual domain by clearly identifying these evoked potentials as an index of visual surprise.

Facial identity across the lifespan.

We can recognise people that we know across their lifespan. We see family members age, and we can recognise celebrities across long careers. How is this possible, despite the very large facial changes that occur as people get older? Here we analyse the statistical properties of faces as they age, sampling photos of the same people from their 20s to their 70s. Across a number of simulations, we observe that individuals' faces retain some idiosyncratic physical properties across the adult lifespan that can be used to support moderate levels of age-independent recognition. However, we found that models based exclusively on image-similarity only achieved limited success in recognising faces across age. In contrast, more robust recognition was achieved with the introduction of a minimal top-down familiarisation procedure. Such models can incorporate the within-person variability associated with a particular individual to show a surprisingly high level of generalisation, even across the lifespan. The analysis of this variability reveals a powerful statistical tool for understanding recognition, and demonstrates how visual representations may support operations typically thought to require conceptual properties.

Prediction-error signals to violated expectations about person identity and head orientation are doubly-dissociated across dorsal and ventral visual stream regions.

Predictive coding theories of perception highlight the importance of constantly updated internal models of the world to predict future sensory inputs. Importantly, such theories suggest that prediction-error signalling should be specific to the violation of predictions concerning distinct attributes of the same stimulus. To interrogate this as yet untested prediction, we focused on two different aspects of face perception (identity and orientation) and investigated whether cortical regions which process particular stimulus attributes also signal prediction violations with respect to those same stimulus attributes. We employed a paradigm using sequential trajectories of images to create perceptual expectations about face orientation and identity, and then parametrically violated each attribute. Using MEG data, we identified double dissociations of expectancy violations in the dorsal and ventral visual streams, such that the right fusiform gyrus showed greater prediction-error signals to identity violations than to orientation violations, whereas the left angular gyrus showed the converse pattern of results. Our results suggest that perceptual prediction-error signalling is directly linked to regions associated with the processing of different stimulus properties.