Spatiotemporal dynamics of similarity-based neural representations of facial identity.

Humans' remarkable ability to quickly and accurately discriminate among thousands of highly similar complex objects demands rapid and precise neural computations. To elucidate the process by which this is achieved, we used magnetoencephalography to measure spatiotemporal patterns of neural activity with high temporal resolution during visual discrimination among a large and carefully controlled set of faces. We also compared these neural data to lower level "image-based" and higher level "identity-based" model-based representations of our stimuli and to behavioral similarity judgments of our stimuli. Between ∼50 and 400 ms after stimulus onset, face-selective sources in right lateral occipital cortex and right fusiform gyrus and sources in a control region (left V1) yielded successful classification of facial identity. In all regions, early responses were more similar to the image-based representation than to the identity-based representation. In the face-selective regions only, responses were more similar to the identity-based representation at several time points after 200 ms. Behavioral responses were more similar to the identity-based representation than to the image-based representation, and their structure was predicted by responses in the face-selective regions. These results provide a temporally precise description of the transformation from low- to high-level representations of facial identity in human face-selective cortex and demonstrate that face-selective cortical regions represent multiple distinct types of information about face identity at different times over the first 500 ms after stimulus onset. These results have important implications for understanding the rapid emergence of fine-grained, high-level representations of object identity, a computation essential to human visual expertise.

I see what you're saying: voice signals influence children's judgments of direct and averted gaze.

Adults use gaze and voice signals as cues to the mental and emotional states of others. We examined the influence of voice cues on children's judgments of gaze. In Experiment 1, 6-year-olds, 8-year-olds, and adults viewed photographs of faces fixating the center of the camera lens and a series of positions to the left and right and judged whether gaze was direct or averted. On each trial, participants heard the participant-directed voice cue (e.g., "I see you"), an object-directed voice cue (e.g., "I see that"), or no voice. In 6-year-olds, the range of directions of gaze leading to the perception of eye contact (the cone of gaze) was narrower for trials with object-directed voice cues than for trials with participant-directed voice cues or no voice. This effect was absent in 8-year-olds and adults, both of whom had a narrower cone of gaze than 6-year-olds. In Experiment 2, we investigated whether voice cues would influence adults' judgments of gaze when the task was made more difficult by limiting the duration of exposure to the face. Adults' cone of gaze was wider than in Experiment 1, and the effect of voice cues was similar to that observed in 6-year-olds in Experiment 1. Together, the results indicate that object-directed voice cues can decrease the width of the cone of gaze, allowing more adult-like judgments of gaze in young children, and that voice cues may be especially effective when the cone of gaze is wider because of immaturity (Experiment 1) or limited exposure (Experiment 2).

Gradual improvement in fine-grained sensitivity to triadic gaze after 6 years of age.

The current research compared the ability of adults and children to determine where another person is looking in shared visual space (triadic gaze). In Experiment 1, children (6-, 8-, 10-, and 14-year-olds) and adults viewed photographs of a model fixating a series of positions separated by 1.6° along the horizontal plane. The task was to indicate whether the model was looking to the left or right of one of three target positions (midline, 6.4° left, or 6.4° right). By 6 years of age, thresholds were quite small (M=1.94°) but were roughly twice as large as those of adults (M=1.05°). Thresholds decreased to adult-like levels around 10 years of age. All age groups showed the same pattern of higher sensitivity for central targets than peripheral targets and of misjudging gaze toward peripheral targets as farther from midline than it really was. In subsequent experiments, we evaluated possible reasons for the higher thresholds in 6- and 8-year-olds. In Experiment 2, the thresholds of 6-year-olds did not improve when the range of deviations from the target position that the model fixated covered a much wider range. In Experiment 3, 8-year-olds were less sensitive than adults to small shifts in eye position even though the task required only matching faces with the same eye position and not determining where the person was looking. These findings suggest that by 6 years of age, children are quite sensitive to triadic gaze, which may support inferences about others' interests and intentions. Subsequent improvements in sensitivity involve, at least in part, an increase in sensitivity to eye position.

The development of fine-grained sensitivity to eye contact after 6 years of age.

Adults use eye contact as a cue to the mental and emotional states of others. Here, we examined developmental changes in the ability to discriminate between eye contact and averted gaze. Children (6-, 8-, 10-, and 14-year-olds) and adults (n=18/age) viewed photographs of a model fixating the center of a camera lens and a series of positions to the left/right or upward/downward and judged whether the model's gaze was direct or averted to the left/right or upward/downward. The horizontal range of fixation positions leading to the perception of direct gaze (the cone of gaze) was more than 50% larger in 6-year-olds than in adults, but it was adult-like and smaller than the vertical cone of gaze by 8 years of age. The vertical cone of gaze was large and statistically adult-like by age 6, with only a small linear reduction thereafter. In all age groups, the horizontal cone of gaze was centered on the bridge of the participant's nose and the vertical cone was centered slightly below the participant's eye height. These findings indicate that until after age 6, relatively poor sensitivity to direct versus averted gaze limits children's ability to use gaze cues to make social judgments.

Children's representations of facial expression and identity: identity-contingent expression aftereffects.

This investigation used adaptation aftereffects to examine developmental changes in the perception of facial expressions. Previous studies have shown that adults' perceptions of ambiguous facial expressions are biased following adaptation to intense expressions. These expression aftereffects are strong when the adapting and probe expressions share the same facial identity but are mitigated when they are posed by different identities. We extended these findings by comparing expression aftereffects and categorical boundaries in adults versus 5- to 9-year-olds (n=20/group). Children displayed adult-like aftereffects and categorical boundaries for happy/sad by 7 years of age and for fear/anger by 9 years of age. These findings suggest that both children and adults perceive expressions according to malleable dimensions in which representations of facial expression are partially integrated with facial identity.

Successful Reorganization of Category-Selective Visual Cortex following Occipito-temporal Lobectomy in Childhood.

Investigations of functional (re)organization in children who have undergone large cortical resections offer a unique opportunity to elucidate the nature and extent of cortical plasticity. We report findings from a 3-year investigation of a child, U.D., who underwent surgical removal of the right occipital and posterior temporal lobes at age 6 years 9 months. Relative to controls, post-surgically, U.D. showed age-appropriate intellectual performance and visuoperceptual face and object recognition skills. Using fMRI at five different time points, we observed a persistent hemianopia and no visual field remapping. In category-selective visual cortices, however, object- and scene-selective regions in the intact left hemisphere were stable early on, but regions subserving face and word recognition emerged later and evinced competition for cortical representation. These findings reveal alterations in the selectivity and topography of category-selective regions when confined to a single hemisphere and provide insights into dynamic functional changes in extrastriate cortical architecture.

Subcortical Facilitation of Behavioral Responses to Threat.

Behavioral responses to threat are critical to survival. Several cortical and subcortical brain regions respond selectively to threat. However, the relation of these neural responses and their underlying representations to behavior is unclear. We examined the contribution of lower-order subcortical representations to behavioral responses to threat in adult humans. In Experiments 1 and 2, participants viewed pairs of images presented to the same eye or to different eyes. We observed a monocular advantage, which indicates subcortical facilitation, for ancestral threats (snakes, spiders), but not for modern threats, positive images, or neutral images. In Experiment 3, we presented pairs of snakes or neutral images into the temporal or nasal hemifield. For snakes only, we observed a temporal hemifield advantage, which indicates facilitation by the retino-tectal subcortical pathway. These results advance the current understanding of processing of threat by adult humans by revealing the characteristics of behaviors driven by a lower-order neural mechanism that is specialized for the processing of ancestral threats. The results also contribute to ongoing debates concerning the biological generality of neural mechanisms for processing of complex, emotionally-relevant stimuli by providing evidence for conservation of lower-order neural mechanisms for processing of ancestral threats across both ontogeny and phylogeny.

A comparison of spatial frequency tuning for judgments of eye gaze and facial identity.

Humans use the direction of eye gaze and facial identity to make important social judgments. We carried out the first measurements of spatial frequency (SF) tuning for judgments of eye gaze, and compared SF tuning for judgments of facial identity and eye gaze. In Experiment 1, participants discriminated between leftward and rightward shifts of gaze, or between two male faces or two female faces. Faces were masked with visual noise that blocked one of 10 SF bands. For each task and masking SF, we measured contrast thresholds for human observers, and used an ideal observer to measure the amount of visual information available to perform the task. As in previous research, low to mid SFs were most important for judgments of facial identity. Mid to high SFs were most important for judgments of eye gaze, and the highest SF important for these judgments was higher than that for identity. In Experiment 2, participants discriminated horizontal and vertical shifts of gaze. The highest SF important for judgments of gaze did not differ between the horizontal and vertical axes. However, SFs above and below this value were more important for judgments of vertical shifts of gaze than for horizontal shifts of gaze. These results suggest that the visual system relies on higher SFs for judgments of eye gaze than for judgments of facial identity, and that SF tuning is broader for judgments of vertical shifts of gaze than for horizontal shifts of gaze.

Adults with autism spectrum disorder show evidence of figural aftereffects with male and female faces.

The norm-based coding model of face perception posits that face perception involves an implicit comparison of observed faces to a representation of an average face (prototype) that is shaped by experience. Using some methods, observers with autism spectrum disorder (ASD) have shown atypical face perception, but other methods suggest preserved face perception. Here, we used a figural aftereffects paradigm to test whether adults with ASD showed evidence of norm-based coding of faces, and whether they encode separate prototypes for male and female faces, as typical observers do. Following prolonged exposure to distorted faces that differ from their stored prototype, neurotypical adults show aftereffects: their prototype shifts in the direction of the adapting face. We measured aftereffects following adaptation to one distorted gender. There were no significant group differences in the size or direction of the aftereffects; both groups showed sex-selective aftereffects after adapting to expanded female faces but showed aftereffects for both sexes after adapting to contracted face of either sex, demonstrating that adults with and without ASD show evidence of partially dissociable male and female face prototypes. This is the first study to examine sex-selective prototypes using figural aftereffects in adults with ASD and replicates the findings of previous studies examining aftereffects in adults with ASD. The results contrast with studies reporting diminished adaptation in children with ASD.

Norm-based coding of facial identity in adults with autism spectrum disorder.

It is unclear whether reported deficits in face processing in individuals with autism spectrum disorders (ASD) can be explained by deficits in perceptual face coding mechanisms. In the current study, we examined whether adults with ASD showed evidence of norm-based opponent coding of facial identity, a perceptual process underlying the recognition of facial identity in typical adults. We began with an original face and an averaged face and then created an anti-face that differed from the averaged face in the opposite direction from the original face by a small amount (near adaptor) or a large amount (far adaptor). To test for norm-based coding, we adapted participants on different trials to the near versus far adaptor, then asked them to judge the identity of the averaged face. We varied the size of the test and adapting faces in order to reduce any contribution of low-level adaptation. Consistent with the predictions of norm-based coding, high functioning adults with ASD (n = 27) and matched typical participants (n = 28) showed identity aftereffects that were larger for the far than near adaptor. Unlike results with children with ASD, the strength of the aftereffects were similar in the two groups. This is the first study to demonstrate norm-based coding of facial identity in adults with ASD.

Bandwidths for the perception of head orientation decrease during childhood.

Adults use the orientation of people's heads as a cue to the focus of their attention. We examined developmental changes in mechanisms underlying sensitivity to head orientation during childhood. Eight-, 10-, 12-year-olds, and adults were adapted to a frontal face view or a 20° left or right side view before judging the orientation of a face at or near frontal. After frontal adaptation, there were no age differences in judgments of head orientation. However, after adaptation to a 20° left or right side view, aftereffects were larger and sensitivity to head orientation was lower in 8- and 10-year-olds than in adults, with no difference between 12-year-olds and adults. A computational model indicates that these results can be modeled as a consequence of decreasing neural tuning bandwidths and decreasing additive internal noise during childhood, and/or as a consequence of increasing inhibition during childhood. These results provide the first evidence that neural mechanisms underlying sensitivity to head orientation undergo considerable refinement during childhood.

Strategies for perceiving facial expressions in adults with autism spectrum disorder.

Rutherford and McIntosh (J Autism Dev Disord 37:187­196, 2007) demonstrated that individuals with autism spectrum disorder (ASD) are more tolerant than controls of exaggerated schematic facial expressions, suggesting that they may use an alternative strategy when processing emotional expressions. The current study was designed to test this finding using photographs of real people. In addition, two control tasks were added to eliminate alternative explanations. We replicated the findings of Rutherford and McIntosh (J Autism Dev Disord 37:187­196, 2007) and also demonstrated that adults with ASD do not show this tolerance when evaluating how realistic the expressions are. These results suggest adults with ASD employ a rule-based strategy to a greater extent than typical adults when processing facial expressions but not when processing other aspects of faces.

The influences of face inversion and facial expression on sensitivity to eye contact in high-functioning adults with autism spectrum disorders.

We examined the influences of face inversion and facial expression on sensitivity to eye contact in high-functioning adults with and without an autism spectrum disorder (ASD). Participants judged the direction of gaze of angry, fearful, and neutral faces. In the typical group only, the range of directions of gaze leading to the perception of eye contact (the cone of gaze) was narrower for upright than inverted faces. In both groups, the cone of gaze was wider for angry faces than for fearful or neutral faces. These results suggest that in high-functioning adults with ASD, the perception of eye contact is not tuned to be finer for upright than inverted faces, but that information is nevertheless integrated across expression and gaze direction.

Telling one face from another: electrocortical correlates of facial characteristics among individual female faces.

Research investigating the neural correlates of face processing has emphasized differences in neural activity when participants view faces versus other stimulus categories (e.g., houses). Much less is known about the neural mechanisms underlying the discrimination among individual faces. Using a large number of female faces, here we show that the amplitude of the face-sensitive N170 electrocortical component is related to a range of facial characteristics. The right N170 amplitude was related to eye color and face width. The left N170 amplitude was related to eye shape and face proportions, suggesting a functional dissociation between hemispheres. In contrast, the amplitude of the P100 and N250 components was largely unaffected by these facial characteristics. Consistent with recent findings in non-human primates, we identify for the first time evidence of human electrocortical brain potentials that are sensitive to variations in specific facial characteristics, a prerequisite for recognizing the identity of individual faces.