Modulation of face processing by top-down attention: Insights from early ERP waveforms.

The face fusiform area (FFA) plays a pivotal role in face recognition, yet the precise timeline of its activity remains debated. Using EEG, we conducted three experiments to investigate how expectancy-consistent versus expectancy-inconsistent visual stimuli influence processing dynamics. Participants viewed images of faces, houses, and tools (Experiment 1), celebrity faces (Experiment 2), or animal faces (Experiment 3), preceded by a priming question. Notably, both conditions presented identical visual stimulation, ensuring that observed differences stemmed from cognitive processing rather than sensory input. Our results from Experiments 2 and 3 reveal that while the initial 150 ms period, crucial for unconscious face detection, remained unaffected, subsequent processing exhibited a delay of several milliseconds for expectancy-inconsistent stimuli, indicating additional processing time required for unexpected recognition. Importantly, no significant differences were observed in Experiment 1, where less demanding tasks or generic mental imagery were used, suggesting that the priming effect was not as pronounced in this context. These findings underscore the critical role of the period immediately following the first 150 ms in face identification and individuation, highlighting the influence of top-down attention on face recognition dynamics. This study provides novel insights into the temporal dynamics of face processing and the neural mechanisms underlying top-down attentional modulation.

The Influence of the Other-Race Effect on Susceptibility to Face Morphing Attacks.

Facial morphs created between two identities resemble both of the faces used to create the morph. Consequently, humans and machines are prone to mistake morphs made from two identities for either of the faces used to create the morph. This vulnerability has been exploited in "morph attacks" in security scenarios. Here, we asked whether the "other-race effect" (ORE)-the human advantage for identifying own- vs. other-race faces-exacerbates morph attack susceptibility for humans. We also asked whether face-identification performance in a deep convolutional neural network (DCNN) is affected by the race of morphed faces. Caucasian (CA) and East-Asian (EA) participants performed a face-identity matching task on pairs of CA and EA face images in two conditions. In the morph condition, different-identity pairs consisted of an image of identity "A" and a 50/50 morph between images of identity "A" and "B". In the baseline condition, morphs of different identities never appeared. As expected, morphs were identified mistakenly more often than original face images. Of primary interest, morph identification was substantially worse for cross-race faces than for own-race faces. Similar to humans, the DCNN performed more accurately for original face images than for morphed image pairs. Notably, the deep network proved substantially more accurate than humans in both cases. The results point to the possibility that DCNNs might be useful for improving face identification accuracy when morphed faces are presented. They also indicate the significance of the race of a face in morph attack susceptibility in applied settings.

How emotions shape memory? The moderating effect of healthy emotionality on eyewitness testimony.

The influence of emotions on memory is a significant topic in the psychology of eyewitness testimony. However, conflicting results have arisen, possibly due to varying approaches and methodologies across studies. These discrepancies might also arise from inadequate consideration of individual differences in emotionality. Therefore, this study aims to analyze the moderating effect of healthy emotionality on the relationship between emotion and memory of criminal events. The results of our laboratory experiment (N = 150) conducted with VR technology indicate that eyewitnesses of crimes, unlike observers of neutral events, recall details concerning the perpetrators' actions immediately preceding the crime act better. Notably, individuals with lower scores on a scale measuring healthy emotionality (ESQ) demonstrate enhanced recollection for these details. At the same time, emotionality plays no significant role in recollection in repeated measurement, as well as in remembering the neutral event. The emotions experienced during crime observation appear to hinder the recollection of perpetrator appearance and behavior unrelated to the crime. These findings are discussed in light of the adaptive role of negative emotions in detecting danger and preparing for unpleasant stimuli.

Face identification proficiency test designed using item response theory.

Measures of face-identification proficiency are essential to ensure accurate and consistent performance by professional forensic face examiners and others who perform face-identification tasks in applied scenarios. Current proficiency tests rely on static sets of stimulus items and so cannot be administered validly to the same individual multiple times. To create a proficiency test, a large number of items of "known" difficulty must be assembled. Multiple tests of equal difficulty can be constructed then using subsets of items. We introduce the Triad Identity Matching (TIM) test and evaluate it using item response theory (IRT). Participants view face-image "triads" (N = 225) (two images of one identity, one image of a different identity) and select the different identity. In Experiment 3, university students (N = 197) showed wide-ranging accuracy on the TIM test, and IRT modeling demonstrated that the TIM items span various difficulty levels. In Experiment 3, we used IRT-based item metrics to partition the test into subsets of specific difficulties. Simulations showed that subsets of the TIM items yielded reliable estimates of subject ability. In Experiments 3a and b, we found that the student-derived IRT model reliably evaluated the ability of non-student participants and that ability generalized across different test sessions. In Experiment 3c, we show that TIM test performance correlates with other common face-recognition tests. In summary, the TIM test provides a starting point for developing a framework that is flexible and calibrated to measure proficiency across various ability levels (e.g., professionals or populations with face-processing deficits).

Sex differences in eyewitness memory: A scoping review.

Researchers in cognitive and forensic psychology have long been interested in the impact of individual differences on eyewitness memory. The sex of the eyewitness is one such factor, with a body of research spanning over 50 years that has sought to determine if and how eyewitness memory differs between males and females. This research has significant implications across the criminal justice system, particularly in the context of gendered issues such as sexual assault. However, the findings have been inconsistent, and there is still a lack of consensus across the literature. A scoping review and analysis of the literature was performed to examine the available evidence regarding whether sex differences in eyewitness memory exist, what explanations have been proposed for any differences found, and how this research has been conducted. Through a strategic search of seven databases, 22 relevant articles were found and reviewed. Results demonstrated that despite the mixed nature of the methodologies and findings, the research suggests that neither males nor females have superior performance in the total amount of accurate information reported, but rather that females may have better memory for person-related details while males may perform better for details related to the surrounding environment. There was also consistent evidence for the own-gender bias. There was some consensus that differences in selective attention between males and females may underlie these sex differences in eyewitness memory. However, none of the studies directly tested this suggested attentional factor, and thus future research is needed to investigate this using a more systematic and empirical approach.

Modeling naturalistic face processing in humans with deep convolutional neural networks.

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.

Radial bias in face identification.

Human vision in the periphery is most accurate for stimuli that point towards the fovea. This so-called radial bias has been linked with the organization and spatial selectivity of neurons at the lowest levels of the visual system, from retinal ganglion cells onwards. Despite evidence that the human visual system is radially biased, it is not yet known whether this bias persists at higher levels of processing, or whether high-level representations are invariant to this low-level orientation bias. We used the case of face identity recognition to address this question. The specialized high-level mechanisms that support efficient face recognition are highly dependent on horizontally oriented information, which convey the most useful identity cues in the fovea. We show that face selective mechanisms are more sensitive on the horizontal meridian (to the left and right of fixation) compared to the vertical meridian (above and below fixation), suggesting that the horizontal cues in the face are better extracted on the horizontal meridian, where they align with the radial bias. The results demonstrate that the radial bias is maintained at high-level recognition stages and emphasize the importance of accounting for the radial bias in future investigation of visual recognition processes in peripheral vision.

Effect of mask coverage on face identification in Taiwanese men and women.

Mask wearing is the easiest and most effective way to avoid COVID-19 infection; however, it affects interpersonal activities, especially face identification. This study examined the effects of three mask coverage levels (full coverage, FC; coverage up to the middle [MB] or bottom of the nose bridge [BB]) on face identification accuracy and time. A total of 115 university students (60 men and 55 women) were recruited to conduct a computer-based simulation test consisting of 30 questions (10 questions [five face images each of men and women] for the three mask coverage levels). One unmasked target face and four face images with a specified mask coverage level were designed for each question, and the participants were requested to select the same face from the four covered face images on the basis of the target face. The ANOVA results indicated that identification accuracy was significantly affected by sex (p < 0.01) and the mask coverage level (p < 0.001), whereas identification time was only influenced by sex (p < 0.05). The multiple comparison results indicated that the identification accuracy rate for faces wearing a mask with FC (90.3%) was significantly lower than for those wearing masks with coverage up to the MB (93.7%) and BB (94.9%) positions; however, no difference in identification accuracy rate was observed between the MB and BB levels. Women exhibited a higher identification accuracy rate than men (94.1% vs. 91.9%) in identifying unfamiliar faces, even though they may spend less time identifying the images. A smaller mask coverage level (i.e., the BB level) does not facilitate face identification. The findings can be served as a reference for people to trade-off between wearing a mask and interpersonal interaction in their daily activities.

FRMDB: Face Recognition Using Multiple Points of View.

Although face recognition technology is currently integrated into industrial applications, it has open challenges, such as verification and identification from arbitrary poses. Specifically, there is a lack of research about face recognition in surveillance videos using, as reference images, mugshots taken from multiple Points of View (POVs) in addition to the frontal picture and the right profile traditionally collected by national police forces. To start filling this gap and tackling the scarcity of databases devoted to the study of this problem, we present the Face Recognition from Mugshots Database (FRMDB). It includes 28 mugshots and 5 surveillance videos taken from different angles for 39 distinct subjects. The FRMDB is intended to analyze the impact of using mugshots taken from multiple points of view on face recognition on the frames of the surveillance videos. To validate the FRMDB and provide a first benchmark on it, we ran accuracy tests using two CNNs, namely VGG16 and ResNet50, pre-trained on the VGGFace and VGGFace2 datasets for the extraction of face image features. We compared the results to those obtained from a dataset from the related literature, the Surveillance Cameras Face Database (SCFace). In addition to showing the features of the proposed database, the results highlight that the subset of mugshots composed of the frontal picture and the right profile scores the lowest accuracy result among those tested. Therefore, additional research is suggested to understand the ideal number of mugshots for face recognition on frames from surveillance videos.

Confidence judgments are associated with face identification accuracy: Findings from a confidence forced-choice task.

Confidence is assumed to be an indicator of identification accuracy in legal practices (e.g., forensic face examination). However, it is not clear whether people can evaluate the correctness of their face-identification decisions reliably using confidence reports. In the current experiment, confidence in the correctness of the perceptual decision was measured with a confidence forced-choice methodology: Upon completion of two perceptual face-identity matching trials, the participants were asked to compare the two decisions and to select the trial on which they felt more confident. On each face-identity matching trial, participants viewed three face images (two same-identity images, one different-identity image) and were instructed to select the image of the different identity. In order to measure the extent to which difficulty level informs confidence decisions, we selected face-image triads using item-difficulty estimates extracted from psychometric modeling applied in a prior study. The difference in difficulty between the paired face-image triads predicted the proportion of high-confidence judgments allocated to the easier trial of the pair. Consistent with the impact of difficulty monitoring on confidence judgments, performance was significantly more accurate on trials associated with higher confidence. Overall, the results suggested that people reliably evaluate the correctness of their perceptual face-identity matching decisions and use trial difficulty to evaluate confidence.

Stable individual differences in unfamiliar face identification: Evidence from simultaneous and sequential matching tasks.

Matching identity in images of unfamiliar faces is difficult: Images of the same person can look different and images of different people can look similar. Recent studies have capitalized on individual differences in the ability to distinguish match (same ID) vs. mismatch (different IDs) face pairs to inform models of face recognition. We addressed two significant gaps in the literature by examining the stability of individual differences in both sensitivity to identity and response bias. In Study 1, 210 participants completed a battery of four tasks in each of two sessions separated by one week. Tasks varied in protocol (same/different, lineup, sorting) and stimulus characteristics (low vs. high within-person variability in appearance). In Study 2, 148 participants completed a battery of three tasks in a single session. Stimuli were presented simultaneously on some trials and sequentially on others, introducing short-term memory demands. Principal components analysis revealed two components that were stable across time and tasks: sensitivity to identity and bias. Analyses of response times suggest that individual differences in bias reflect decision-making processes. We discuss the implications of our findings in applied settings and for models of face recognition.

Detecting a familiar person behind the surgical mask: recognition without identification among masked versus sunglasses-covered faces.

Previous research has shown that even when famous people's identities cannot be discerned from faces that have been filtered with monochromatic noise, these unidentifiable famous faces still tend to receive higher familiarity ratings than similarly filtered non-famous faces. Experiment 1 investigated whether a similar face recognition without identification effect would occur among faces whose identification was hindered through the wearing of a surgical mask. Among a mixture of famous and non-famous faces wearing surgical masks and hoods, participants rated how familiar each person seemed then attempted to identify the person. Though surgical masks significantly impaired identification of the famous faces, unidentified masked famous faces received higher familiarity ratings on average than the non-famous masked faces, suggesting that a sense of familiarity could still occur even when identification was impaired by the mask. Experiment 2 compared faces covered by surgical masks with faces covered by sunglasses. Though sunglasses impaired face identification more than surgical masks, the magnitude of the face recognition without identification effect was the same in both cases. This pattern suggests that holistic face processing is not a requirement for the sense of familiarity with a face, and that different facial feature types can contribute.

Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models.

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component-based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a "face space" based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

Incongruence in Lighting Impairs Face Identification.

The effect of uniform lighting on face identity processing is little understood, despite its potential influence on our ability to recognize faces. Here, we investigated how changes in uniform lighting level affected face identification performance during face memory tests. Observers were tasked with learning a series of faces, followed by a memory test where observers judged whether the faces presented were studied before or novel. Face stimuli were presented under uniform bright or dim illuminations, and lighting across the face learning and the memory test sessions could be the same ("congruent") or different ("incongruent"). This led to four experimental conditions: (1) Bright/Dim (learning bright faces, testing on dim faces); (2) Bright/Bright; (3) Dim/Bright; and (4) Dim/Dim. Our results revealed that incongruent lighting levels across sessions (Bright/Dim and Dim/Bright) significantly reduced sensitivity (d') to faces and introduced conservative biases compared to congruent lighting levels (Bright/Bright and Dim/Dim). No significant differences in performance were detected between the congruent lighting conditions (Bright/Bright vs. Dim/Dim) and between the incongruent lighting conditions (Bright/Dim vs. Dim/Bright). Thus, incongruent lighting deteriorated performance in face identification. These findings implied that the level of uniform lighting should be considered in an illumination-specific face representation and potential applications such as eyewitness testimony.

Cultural Differences in Face Recognition and Potential Underlying Mechanisms.

The ability to recognize a face is crucial for the success of social interactions. Understanding the visual processes underlying this ability has been the focus of a long tradition of research. Recent advances in the field have revealed that individuals having different cultural backgrounds differ in the type of visual information they use for face processing. However, the mechanisms that underpin these differences remain unknown. Here, we revisit recent findings highlighting group differences in face processing. Then, we integrate these results in a model of visual categorization developed in the field of psychophysics: the RAP framework. On the basis of this framework, we discuss potential mechanisms, whether face-specific or not, that may underlie cross-cultural differences in face perception.

Performance of typical and superior face recognizers on a novel interactive face matching procedure.

Unfamiliar simultaneous face matching is error prone. Reducing incorrect identification decisions will positively benefit forensic and security contexts. The absence of view-independent information in static images likely contributes to the difficulty of unfamiliar face matching. We tested whether a novel interactive viewing procedure that provides the user with 3D structural information as they rotate a facial image to different orientations would improve face matching accuracy. We tested the performance of 'typical' (Experiment 1) and 'superior' (Experiment 2) face recognizers, comparing their performance using high-quality (Experiment 3) and pixelated (Experiment 4) Facebook profile images. In each trial, participants responded whether two images featured the same person with one of these images being either a static face, a video providing orientation information, or an interactive image. Taken together, the results show that fluid orientation information and interactivity prompt shifts in criterion and support matching performance. Because typical and superior face recognizers both benefited from the structural information provided by the novel viewing procedures, our results point to qualitatively similar reliance on pictorial encoding in these groups. This also suggests that interactive viewing tools can be valuable in assisting face matching in high-performing practitioner groups.

Does stereopsis improve face identification? A study using a virtual reality display with integrated eye-tracking and pupillometry.

Stereopsis is a powerful depth cue for humans, which may also contribute to object recognition. In particular, we surmise that face identification would benefit from the availability of stereoscopic depth cues, since facial perception may be based on three-dimensional (3D) representations. In this study, a virtual reality (VR) headset with integrated eye-tracking was used to present stereoscopic images of faces. As a monoscopic contrast condition, identical images of faces were displayed to the two eyes. We monitored the participants' gaze behavior and pupil diameters while they performed a sample-to-match face identification task. We found that accuracy was superior in the stereoscopic condition compared to the monoscopic condition for frontal and intermediate views, but not profiles. Moreover, pupillary diameters were smaller when identifying stereoscopically seen faces than when viewing them without stereometric cues, which we interpret as lower processing load for the former than the latter conditions. The analysis of gaze showed that participants tended to focus on regions of the face rich in volumetric information, more so in the stereoscopic condition than the monoscopic condition. Together, these findings suggest that a 3D representation of faces may be the natural format used by the visual system when assessing face identity. Stereoscopic information, by providing depth information, assists the construction of robust facial representations in memory.

Face Recognition at a Distance for a Stand-Alone Access Control System.

Although access control based on human face recognition has become popular in consumer applications, it still has several implementation issues before it can realize a stand-alone access control system. Owing to a lack of computational resources, lightweight and computationally efficient face recognition algorithms are required. The conventional access control systems require significant active cooperation from the users despite its non-aggressive nature. The lighting/illumination change is one of the most difficult and challenging problems for human-face-recognition-based access control applications. This paper presents the design and implementation of a user-friendly, stand-alone access control system based on human face recognition at a distance. The local binary pattern (LBP)-AdaBoost framework was employed for face and eyes detection, which is fast and invariant to illumination changes. It can detect faces and eyes of varied sizes at a distance. For fast face recognition with a high accuracy, the Gabor-LBP histogram framework was modified by substituting the Gabor wavelet with Gaussian derivative filters, which reduced the facial feature size by 40% of the Gabor-LBP-based facial features, and was robust to significant illumination changes and complicated backgrounds. The experiments on benchmark datasets produced face recognition accuracies of 97.27% on an E-face dataset and 99.06% on an XM2VTS dataset, respectively. The system achieved a 91.5% true acceptance rate with a 0.28% false acceptance rate and averaged a 5.26 frames/sec processing speed on a newly collected face image and video dataset in an indoor office environment.

Face averages and multiple images in a live matching task.

We know from previous research that unfamiliar face matching (determining whether two simultaneously presented images show the same person or not) is very error-prone. A small number of studies in laboratory settings have shown that the use of multiple images or a face average, rather than a single image, can improve face matching performance. Here, we tested 1,999 participants using four-image arrays and face averages in two separate live matching tasks. Matching a single image to a live person resulted in numerous errors (79.9% accuracy across both experiments), and neither multiple images (82.4% accuracy) nor face averages (76.9% accuracy) improved performance. These results are important when considering possible alterations which could be made to photo-ID. Although multiple images and face averages have produced measurable improvements in performance in recent laboratory studies, they do not produce benefits in a real-world live face matching context.

Perceptual identification task points to continuity between implicit memory and recall.

Adopting a continuous identification task (CID-R) with embedded questions about prior occurrence, recent research has proposed that implicit and explicit memory are underpinned by a single memory system, since there is a systematic relationship between implicit memory (measured by identification) and explicit memory (measured by subjective report of recognition; for an example, see Berry, Shanks, & Henson, 2008). We were interested in whether this pattern would extend to recall of information from a study phase (Experiment 1) or recall from semantic memory (Experiment 2). We developed a degraded face identification version of the CID-R task using Gaussian blur. We reproduced previous results regarding the relationship between explicit responses on the recognition task (old/new) and stimuli identification, pointing to a continuity between explicit and implicit memory. Critically, we also found that the strength of the implicit effect (i.e., stimuli identification) was predicted by the accuracy in recall (retrieval of context in Experiment 1 and correct responses to general knowledge questions about the face in Experiment 2). Our results support the idea that memory is unidimensional and related to memory trace strength; both for recall and recognition, and interestingly, for semantic and episodic recall.