Behavioral and Neural Adaptation in Approach Behavior.

People often make approachability decisions based on perceived facial trustworthiness. However, it remains unclear how people learn trustworthiness from a population of faces and whether this learning influences their approachability decisions. Here we investigated the neural underpinning of approach behavior and tested two important hypotheses: whether the amygdala adapts to different trustworthiness ranges and whether the amygdala is modulated by task instructions and evaluative goals. We showed that participants adapted to the stimulus range of perceived trustworthiness when making approach decisions and that these decisions were further modulated by the social context. The right amygdala showed both linear response and quadratic response to trustworthiness level, as observed in prior studies. Notably, the amygdala's response to trustworthiness was not modulated by stimulus range or social context, a possible neural dynamic adaptation. Together, our data have revealed a robust behavioral adaptation to different trustworthiness ranges as well as a neural substrate underlying approach behavior based on perceived facial trustworthiness.

Top-down attention switches coupling between low-level and high-level areas of human visual cortex.

Top-down attention is an essential cognitive ability, allowing our finite brains to process complex natural environments by prioritizing information relevant to our goals. Previous evidence suggests that top-down attention operates by modulating stimulus-evoked neural activity within visual areas specialized for processing goal-relevant information. We show that top-down attention also has a separate influence on the background coupling between visual areas: adopting different attentional goals resulted in specific patterns of noise correlations in the visual system, whereby intrinsic activity in the same set of low-level areas was shared with only those high-level areas relevant to the current goal. These changes occurred independently of evoked activity, persisted without visual stimulation, and predicted behavioral success in deploying attention better than the modulation of evoked activity. This attentional switching of background connectivity suggests that attention may help synchronize different levels of the visual processing hierarchy, forming state-dependent functional pathways in human visual cortex to prioritize goal-relevant information.

A model of binocular rivalry and cross-orientation suppression.

Binocular rivalry and cross-orientation suppression are well-studied forms of competition in visual cortex, but models of these two types of competition are in tension with one another. Binocular rivalry occurs during the presentation of dichoptic grating stimuli, where two orthogonal gratings presented separately to the two eyes evoke strong alternations in perceptual dominance. Cross-orientation suppression occurs during the presentation of plaid stimuli, where the responses to a component grating presented to both eyes is weakened by the presence of a superimposed orthogonal grating. Conventional models of rivalry that rely on strong competition between orientation-selective neurons incorrectly predict rivalry between the components of plaids. Lowering the inhibitory weights in such models reduces rivalry for plaids, but also reduces it for dichoptic gratings. Using an exhaustive grid search, we show that this problem cannot be solved simply by adjusting the parameters of the model. Instead, we propose a robust class of models that rely on ocular opponency neurons, previously proposed as a mechanism for efficient stereo coding, to yield rivalry only for dichoptic gratings, not for plaids. This class of models reconciles models of binocular rivalry with the divisive normalization framework that has been used to explain cross-orientation. Our model makes novel predictions that we confirmed with psychophysical tests.

Task-invariant brain responses to the social value of faces.

In two fMRI experiments (n = 44) using tasks with different demands-approach-avoidance versus one-back recognition decisions-we measured the responses to the social value of faces. The face stimuli were produced by a parametric model of face evaluation that reduces multiple social evaluations to two orthogonal dimensions of valence and power [Oosterhof, N. N., & Todorov, A. The functional basis of face evaluation. Proceedings of the National Academy of Sciences, U.S.A., 105, 11087-11092, 2008]. Independent of the task, the response within regions of the occipital, fusiform, and lateral prefrontal cortices was sensitive to the valence dimension, with larger responses to low-valence faces. Additionally, there were extensive quadratic responses in the fusiform gyri and dorsal amygdala, with larger responses to faces at the extremes of the face valence continuum than faces in the middle. In all these regions, participants' avoidance decisions correlated with brain responses, with faces more likely to be avoided evoking stronger responses. The findings suggest that both explicit and implicit face evaluation engage multiple brain regions involved in attention, affect, and decision making.

A statistical model of facial attractiveness.

Previous research has identified facial averageness and sexual dimorphism as important factors in facial attractiveness. The averageness and sexual dimorphism accounts provide important first steps in understanding what makes faces attractive, and should be valued for their parsimony. However, we show that they explain relatively little of the variance in facial attractiveness, particularly for male faces. As an alternative to these accounts, we built a regression model that defines attractiveness as a function of a face's position in a multidimensional face space. The model provides much more predictive power than the averageness and sexual dimorphism accounts and reveals previously unreported components of attractiveness. The model shows that averageness is attractive in some dimensions but not in others and resolves previous contradictory reports about the effects of sexual dimorphism on the attractiveness of male faces.

Distributed representations of dynamic facial expressions in the superior temporal sulcus.

Previous research on the superior temporal sulcus (STS) has shown that it responds more to facial expressions than to neutral faces. Here, we extend our understanding of the STS in two ways. First, using targeted high-resolution fMRI measurements of the lateral cortex and multivoxel pattern analysis, we show that the response to seven categories of dynamic facial expressions can be decoded in both the posterior STS (pSTS) and anterior STS (aSTS). We were also able to decode patterns corresponding to these expressions in the frontal operculum (FO), a structure that has also been shown to respond to facial expressions. Second, we measured the similarity structure of these representations and found that the similarity structure in the pSTS significantly correlated with the perceptual similarity structure of the expressions. This was the case regardless of whether we used pattern classification or more traditional correlation techniques to extract the neural similarity structure. These results suggest that distributed representations in the pSTS could underlie the perception of facial expressions.

Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces.

People make trait inferences based on facial appearance despite little evidence that these inferences accurately reflect personality. The authors tested the hypothesis that these inferences are driven in part by structural resemblance to emotional expressions. The authors first had participants judge emotionally neutral faces on a set of trait dimensions. The authors then submitted the face images to a Bayesian network classifier trained to detect emotional expressions. By using a classifier, the authors can show that neutral faces perceived to possess various personality traits contain objective resemblance to emotional expression. In general, neutral faces that are perceived to have positive valence resemble happiness, faces that are perceived to have negative valence resemble disgust and fear, and faces that are perceived to be threatening resemble anger. These results support the idea that trait inferences are in part the result of an overgeneralization of emotion recognition systems. Under this hypothesis, emotion recognition systems, which typically extract accurate information about a person's emotional state, are engaged during the perception of neutral faces that bear subtle resemblance to emotional expressions. These emotions could then be misattributed as traits.

The social evaluation of faces: a meta-analysis of functional neuroimaging studies.

Neuroscience research on the social evaluation of faces has accumulated over the last decade, yielding divergent results. We used a meta-analytic technique, multi-level kernel density analysis (MKDA), to analyze 29 neuroimaging studies on face evaluation. Across negative face evaluations, we observed the most consistent activations in bilateral amygdala. Across positive face evaluations, we observed the most consistent activations in medial prefrontal cortex, pregenual anterior cingulate cortex (pgACC), medial orbitofrontal cortex (mOFC), left caudate and nucleus accumbens (NAcc). Based on additional analyses comparing linear and non-linear responses, we propose a ventral/dorsal dissociation within the amygdala, wherein separate populations of neurons code for face valence and intensity, respectively. Finally, we argue that some of the differences between studies are attributable to differences in the typicality of face stimuli. Specifically, extremely attractive faces are more likely to elicit responses in NAcc/caudate and mOFC.

Normal binocular rivalry in autism: implications for the excitation/inhibition imbalance hypothesis.

Autism is characterized by disruption in multiple dimensions of perception, emotion, language and social cognition. Many hypotheses for the underlying neurophysiological basis have been proposed. Among these is the excitation/inhibition (E/I) imbalance hypothesis, which states that levels of cortical excitation and inhibition are disrupted in autism. We tested this theory in the visual system, because vision is one of the better understood systems in neuroscience, and because the E/I imbalance theory has been proposed to explain hypersensitivity to sensory stimuli in autism. We conducted two experiments on binocular rivalry, a well-studied psychophysical phenomenon that depends critically on excitation and inhibition levels in cortex. Using a computational model, we made specific predictions about how imbalances in excitation and inhibition levels would affect perception during two aspects of binocular rivalry: mixed perception (Experiment 1) and traveling waves (Experiment 2). We found no significant differences in either of these phenomena between high-functioning adults with autism and controls, and no evidence for a relationship between these measurements and the severity of autism. These results do not conclusively rule out an excitation/inhibition imbalance in the visual system of those with autism, but they suggest that such an imbalance, if it exists, is likely to be small in magnitude.

Brain systems for assessing the affective value of faces.

Cognitive neuroscience research on facial expression recognition and face evaluation has proliferated over the past 15 years. Nevertheless, large questions remain unanswered. In this overview, we discuss the current understanding in the field, and describe what is known and what remains unknown. In §2, we describe three types of behavioural evidence that the perception of traits in neutral faces is related to the perception of facial expressions, and may rely on the same mechanisms. In §3, we discuss cortical systems for the perception of facial expressions, and argue for a partial segregation of function in the superior temporal sulcus and the fusiform gyrus. In §4, we describe the current understanding of how the brain responds to emotionally neutral faces. To resolve some of the inconsistencies in the literature, we perform a large group analysis across three different studies, and argue that one parsimonious explanation of prior findings is that faces are coded in terms of their typicality. In §5, we discuss how these two lines of research--perception of emotional expressions and face evaluation--could be integrated into a common, cognitive neuroscience framework.

Reprint of: The amygdala and FFA track both social and non-social face dimensions.

The amygdala is thought to perform a number of social functions, and has received much attention for its role in processing social properties of faces. In particular, it has been shown to respond more to facial expressions than to neutral faces, and more to positively valenced and negatively valenced faces than faces in the middle of the continuum. However, when these findings are viewed in the context of a multidimensional face space, an important question emerges. Face space is a vector space where every face can be represented as a point in the space. The origin of the space represents the average face. In this context, positively valenced and negatively valenced faces are further away from the average face than faces in the middle of the continuum. It is therefore unclear if the amygdala response to positively valenced and negatively valenced faces is due to their social properties or to their general distance from the average face. Here, we compared the amygdala response to a set of faces that varied along two dimensions centered around the average face but differing in social content. In both the amygdala and much of the posterior face network, we observed a similar response to both dimensions, with stronger responses to the extremes of the dimensions than to faces near the average face. These findings suggest that the responses in these regions to socially relevant faces may be partially due to general distance from the average face.

The amygdala and FFA track both social and non-social face dimensions.

The amygdala is thought to perform a number of social functions, and has received much attention for its role in processing social properties of faces. In particular, it has been shown to respond more to facial expressions than to neutral faces, and more to positively valenced and negatively valenced faces than faces in the middle of the continuum. However, when these findings are viewed in the context of a multidimensional face space, an important question emerges. Face space is a vector space where every face can be represented as a point in the space. The origin of the space represents the average face. In this context, positively valenced and negatively valenced faces are further away from the average face than faces in the middle of the continuum. It is therefore unclear if the amygdala response to positively valenced and negatively valenced faces is due to their social properties or to their general distance from the average face. Here, we compared the amygdala response to a set of faces that varied along two dimensions centered around the average face but differing in social content. In both the amygdala and much of the posterior face network, we observed a similar response to both dimensions, with stronger responses to the extremes of the dimensions than to faces near the average face. These findings suggest that the responses in these regions to socially relevant faces may be partially due to general distance from the average face.

Graded representations of emotional expressions in the left superior temporal sulcus.

Perceptual categorization is a fundamental cognitive process that gives meaning to an often graded sensory environment. Previous research has subdivided the visual pathway into posterior regions that processes the physical properties of a stimulus, and frontal regions that process more abstract properties such as category information. The superior temporal sulcus (STS) is known to be involved in face and emotion perception, but the nature of its processing remains unknown. Here, we used targeted fMRI measurements of the STS to investigate whether its representations of facial expressions are categorical or noncategorical. Multivoxel pattern analysis showed that even though subjects were performing a categorization task, the left STS contained graded, noncategorical representations. In the right STS, representations showed evidence for both stimulus-related gradations and a categorical boundary.

Nonlinear amygdala response to face trustworthiness: contributions of high and low spatial frequency information.

Previous neuroimaging research has shown amygdala sensitivity to the perceived trustworthiness of neutral faces, with greater responses to untrustworthy compared with trustworthy faces. This observation is consistent with the common view that the amygdala encodes fear and is preferentially responsive to negative stimuli. However, some studies have shown greater amygdala activation to positive compared with neutral stimuli. The first goal of this study was to more fully characterize the amygdala response to face trustworthiness by modeling its activation with both linear and nonlinear predictors. Using fMRI, we report a nonmonotonic response profile, such that the amygdala responds strongest to highly trustworthy and highly untrustworthy faces. This finding complicates future attempts to make inferences about mental states based on activation in the amygdala. The second goal of the study was to test for modulatory effects of image spatial frequency filtering on the amygdala response. We predicted greater amygdala sensitivity to face trustworthiness for low spatial frequency images compared with high spatial frequency images. Instead, we found that both frequency ranges provided sufficient information for the amygdala to differentiate faces on trustworthiness. This finding is consistent with behavioral results and suggests that trustworthiness information may reach the amygdala through pathways carrying both coarse and fine resolution visual signals.