'Reading the Mind in the Eyes': an fMRI study of adolescents with autism and their siblings.

BACKGROUND: Mentalizing deficits are a hallmark of the autism spectrum condition (ASC) and a potential endophenotype for atypical social cognition in ASC. Differences in performance and neural activation on the 'Reading the Mind in the Eyes' task (the Eyes task) have been identified in individuals with ASC in previous studies. METHOD: Performance on the Eyes task along with the associated neural activation was examined in adolescents with ASC (n = 50), their unaffected siblings (n = 40) and typically developing controls (n = 40). Based on prior literature that males and females with ASC display different cognitive and associated neural characteristics, analyses were stratified by sex. Three strategies were applied to test for endophenotypes at the level of neural activation: (1) identifying and locating conjunctions of ASC-control and sibling-control differences; (2) examining whether the sibling group is comparable to the ASC or intermediate between the ASC and control groups; and (3) examining spatial overlaps between ASC-control and sibling-control differences across multiple thresholds. RESULTS: Impaired behavioural performance on the Eyes task was observed in males with ASC compared to controls, but only at trend level in females; and no difference in performance was identified between sibling and same-sex control groups in both sexes. Neural activation showed a substantial endophenotype effect in the female groups but this was only modest in the male groups. CONCLUSIONS: Behavioural impairment on complex emotion recognition associated with mental state attribution is a phenotypic, rather than an endophenotypic, marker of ASC. However, the neural response during the Eyes task is a potential endophenotypic marker for ASC, particularly in females.

It is all in the face: carotenoid skin coloration loses attractiveness outside the face.

Recently, the importance of skin colour for facial attractiveness has been recognized. In particular, dietary carotenoid-induced skin colour has been proposed as a signal of health and therefore attractiveness. While perceptual results are highly consistent, it is currently not clear whether carotenoid skin colour is preferred because it poses a cue to current health condition in humans or whether it is simply seen as a more aesthetically pleasing colour, independently of skin-specific signalling properties. Here, we tested this question by comparing attractiveness ratings of faces to corresponding ratings of meaningless scrambled face images matching the colours and contrasts found in the face. We produced sets of face and non-face stimuli with either healthy (high-carotenoid coloration) or unhealthy (low-carotenoid coloration) colour and asked participants for attractiveness ratings. Results showed that, while for faces increased carotenoid coloration significantly improved attractiveness, there was no equivalent effect on perception of scrambled images. These findings are consistent with a specific signalling system of current condition through skin coloration in humans and indicate that preferences are not caused by sensory biases in observers.

Deficits in facial, body movement and vocal emotional processing in autism spectrum disorders.

BACKGROUND: Previous behavioural and neuroimaging studies of emotion processing in autistic spectrum disorder (ASD) have focused on the use of facial stimuli. To date, however, no studies have examined emotion processing in autism across a broad range of social signals. METHOD: This study addressed this issue by investigating emotion processing in a group of 23 adults with ASD and 23 age- and gender-matched controls. Recognition of basic emotions ('happiness', 'sadness', 'anger', disgust' and 'fear') was assessed from facial, body movement and vocal stimuli. The ability to make social judgements (such as approachability) from facial stimuli was also investigated. RESULTS: Significant deficits in emotion recognition were found in the ASD group relative to the control group across all stimulus domains (faces, body movements and voices). These deficits were seen across a range of emotions. The ASD group were also impaired in making social judgements compared to the control group and this correlated with impairments in basic emotion recognition. CONCLUSIONS: This study demonstrates that there are significant and broad-ranging deficits in emotion processing in ASD present across a range of stimulus domains and in the auditory and visual modality; they cannot therefore be accounted for simply in terms of impairments in face processing or in the visual modality alone. These results identify a core deficit affecting the processing of a wide range of emotional information in ASD, which contributes to the impairments in social function seen in people with this condition.

Emotion recognition in progressive supranuclear palsy.

Progressive supranuclear palsy (PSP) is an atypical parkinsonian syndrome characterised by akinesis, rigidity, falls, supranuclear gaze palsy and cognitive, particularly executive, dysfunction. This study examined the extent to which emotion recognition is affected by PSP. Although deficits in the recognition of emotion have been reported in several diseases which share clinicopathological characteristics with PSP, it has never been studied systematically in PSP. Twenty-four patients with probable or definite PSP and matched healthy controls were studied using tests of facial identity and facial emotion recognition. Patients were not impaired in recognising famous faces, but they showed significant deficits in the recognition of emotions, particularly negative emotions. Moreover, emotion recognition was strongly correlated with the severity of other cognitive deficits in PSP, but not disease duration. Deficits in emotion recognition form an integral part of the cognitive spectrum of the disease. The findings point to the pathological involvement of key regions necessary for the processing of emotions and to a subtype of PSP with cognitive and emotion recognition impairments. The acknowledgement of deficits in emotion recognition is important for management of both patients and their carers.

Impaired recognition and experience of disgust following brain injury.

Huntington's disease can particularly affect people's recognition of disgust from facial expressions, and functional neuroimaging research has demonstrated that facial expressions of disgust consistently engage different brain areas (insula and putamen) than other facial expressions. However, it is not known whether these particular brain areas process only facial signals of disgust or disgust signals from multiple modalities. Here we describe evidence, from a patient with insula and putamen damage, for a neural system for recognizing social signals of disgust from multiple modalities.

In the face of threat: neural and endocrine correlates of impaired facial emotion recognition in cocaine dependence.

The ability to recognize facial expressions of emotion in others is a cornerstone of human interaction. Selective impairments in the recognition of facial expressions of fear have frequently been reported in chronic cocaine users, but the nature of these impairments remains poorly understood. We used the multivariate method of partial least squares and structural magnetic resonance imaging to identify gray matter brain networks that underlie facial affect processing in both cocaine-dependent (n = 29) and healthy male volunteers (n = 29). We hypothesized that disruptions in neuroendocrine function in cocaine-dependent individuals would explain their impairments in fear recognition by modulating the relationship with the underlying gray matter networks. We found that cocaine-dependent individuals not only exhibited significant impairments in the recognition of fear, but also for facial expressions of anger. Although recognition accuracy of threatening expressions co-varied in all participants with distinctive gray matter networks implicated in fear and anger processing, in cocaine users it was less well predicted by these networks than in controls. The weaker brain-behavior relationships for threat processing were also mediated by distinctly different factors. Fear recognition impairments were influenced by variations in intelligence levels, whereas anger recognition impairments were associated with comorbid opiate dependence and related reduction in testosterone levels. We also observed an inverse relationship between testosterone levels and the duration of crack and opiate use. Our data provide novel insight into the neurobiological basis of abnormal threat processing in cocaine dependence, which may shed light on new opportunities facilitating the psychosocial integration of these patients.

Recognition of facial emotion in nine individuals with bilateral amygdala damage.

Findings from several case studies have shown that bilateral amygdala damage impairs recognition of emotions in facial expressions, especially fear. However, one study did not find such an impairment, and, in general, comparison across studies has been made difficult because of the different stimuli and tasks employed. In a collaborative study to facilitate such comparisons, we report here the recognition of emotional facial expressions in nine subjects with bilateral amygdala damage, using a sensitive and quantitative assessment. Compared to controls, the subjects as a group were significantly impaired in recognizing fear, although individual performances ranged from severely impaired to essentially normal. Most subjects were impaired on several negative emotions in addition to fear, but no subject was impaired in recognizing happy expressions. An analysis of response consistency showed that impaired recognition of fear could not be attributed simply to mistaking fear for another emotion. While it remains unclear why some subjects with amygdala damage included here are not impaired on our task, the results overall are consistent with the idea that the amygdala plays an important role in triggering knowledge related to threat and danger signaled by facial expressions.

Face processing impairments after encephalitis: amygdala damage and recognition of fear.

Face processing and facial emotion recognition were investigated in five post-encephalitic people of average or above-average intelligence. Four of these people (JC, YW, RB and SE) had extensive damage in the region of the amygdala. A fifth post-encephalitic person with predominantly hippocampal damage and relative sparing of the amygdala (RS) participated, allowing us to contrast the effects of temporal lobe damage including and excluding the amygdala region. The findings showed impaired recognition of fear following bilateral temporal lobe damage when this included the amygdala. For JC, this was part of a constellation of deficits on face processing tasks, with impaired recognition of several emotions. SE, YW and RB, however, showed relatively circumscribed deficits. Although they all had some problems in recognizing or naming famous faces, and had poor memory for faces on the Warrington Recognition Memory Test, none showed a significant impairment on the Benton Test of Facial Recognition, indicating relatively good perception of the face's physical structure. In a test of recognition of basic emotions (happiness, surprise, fear, sadness, disgust and anger), SE, YW and RB achieved normal levels of performance in comparison to our control group for all emotions except fear. Their results contrast with those of RS, with relative sparing of the amygdala region and unimpaired recognition of emotion, pointing clearly toward the importance of the amygdala in the recognition of fear.

Computer-enhanced emotion in facial expressions.

Benson & Perrett's (1991 b) computer-based caricature procedure was used to alter the positions of anatomical landmarks in photographs of emotional facial expressions with respect to their locations in a reference norm face (e.g. a neutral expression). Exaggerating the differences between an expression and its norm produces caricatured images, whereas reducing the differences produces 'anti-caricatures'. Experiment 1 showed that caricatured (+50% different from neutral) expressions were recognized significantly faster than the veridical (0%, undistorted) expressions. This held for all six basic emotions from the Ekman & Friesen (1976) series, and the effect generalized across different posers. For experiment 2, caricatured (+50%) and anti-caricatured (-50%) images were prepared using two types of reference norm; a neutral-expression norm, which would be optimal if facial expression recognition involves monitoring changes in the positioning of underlying facial muscles, and a perceptually-based norm involving an average of the expressions of six basic emotions (excluding neutral) in the Ekman & Friesen (1976) series. The results showed that the caricatured images were identified significantly faster, and the anti-caricatured images significantly slower, than the veridical expressions. Furthermore, the neutral-expression and average-expression norm caricatures produced the same pattern of results.

Disgust implicated in obsessive-compulsive disorder.

Psychiatric classificatory systems consider obsessions and compulsions as forms of anxiety disorder. However, the neurology of diseases associated with obsessive-compulsive symptoms suggests the involvement of fronto-striatal regions likely to be involved in the mediation of the emotion of disgust, suggesting that dysfunctions of disgust should be considered alongside anxiety in the pathogenesis of obsessive-compulsive behaviours. We therefore tested recognition of facial expressions of basic emotions (including disgust) by groups of participants with obsessive-compulsive disorder (OCD) and with Gilles de la Tourette's syndrome (GTS) with an without co-present obsessive-compulsive behaviours (GTS with OCB; GTS without OCB). A group of people suffering from panic disorder and generalized anxiety were also included in the study. Both groups with obsessive-compulsive symptoms (OCD; GTS with OCB) showed impaired recognition of facial expressions of disgust. Such problems were not evident in participants with panic disorder and generalized anxiety, or for participants with GTS without obsessions or compulsions, indicating that the deficit is closely related to the presence of obsessive-compulsive symptoms. Participants with OCD were able to assign words to emotion categories without difficulty, showing that their problem with disgust is linked to a failure to recognize this emotion in others and not a comprehension or response criterion effect. Impaired recognition of disgust is consistent with the neurology of OCD and with the idea that abnormal experience of disgust may be involved in the genesis of obsessions and compulsions.

Neural responses to facial and vocal expressions of fear and disgust.

Neuropsychological studies report more impaired responses to facial expressions of fear than disgust in people with amygdala lesions, and vice versa in people with Huntington's disease. Experiments using functional magnetic resonance imaging (fMRI) have confirmed the role of the amygdala in the response to fearful faces and have implicated the anterior insula in the response to facial expressions of disgust. We used fMRI to extend these studies to the perception of fear and disgust from both facial and vocal expressions. Consistent with neuropsychological findings, both types of fearful stimuli activated the amygdala. Facial expressions of disgust activated the anterior insula and the caudate-putamen; vocal expressions of disgust did not significantly activate either of these regions. All four types of stimuli activated the superior temporal gyrus. Our findings therefore (i) support the differential localization of the neural substrates of fear and disgust; (ii) confirm the involvement of the amygdala in the emotion of fear, whether evoked by facial or vocal expressions; (iii) confirm the involvement of the anterior insula and the striatum in reactions to facial expressions of disgust; and (iv) suggest a possible general role for the perception of emotional expressions for the superior temporal gyrus.

Caricaturing facial expressions.

The physical differences between facial expressions (e.g. fear) and a reference norm (e.g. a neutral expression) were altered to produce photographic-quality caricatures. In Experiment 1, participants rated caricatures of fear, happiness and sadness for their intensity of these three emotions; a second group of participants rated how 'face-like' the caricatures appeared. With increasing levels of exaggeration the caricatures were rated as more emotionally intense, but less 'face-like'. Experiment 2 demonstrated a similar relationship between emotional intensity and level of caricature for six different facial expressions. Experiments 3 and 4 compared intensity ratings of facial expression caricatures prepared relative to a selection of reference norms - a neutral expression, an average expression, or a different facial expression (e.g. anger caricatured relative to fear). Each norm produced a linear relationship between caricature and rated intensity of emotion; this finding is inconsistent with two-dimensional models of the perceptual representation of facial expression. An exemplar-based multidimensional model is proposed as an alternative account.

Facial expression megamix: tests of dimensional and category accounts of emotion recognition.

We report four experiments investigating the perception of photographic quality continua of interpolated ('morphed') facial expressions derived from prototypes of the 6 emotions in the Ekman and Friesen (1976) series (happiness, surprise, fear, sadness, disgust and anger). In Experiment 1, morphed images made from all possible pairwise combinations of expressions were presented in random order; subjects identified these as belonging to distinct expression categories corresponding to the prototypes at each end of the relevant continuum. This result was replicated in Experiment 2, which also included morphs made from a prototype with a neutral expression, and allowed 'neutral' as a response category. These findings are inconsistent with the view that facial expressions are recognised by locating them along two underlying dimensions, since such a view predicts that at least some transitions between categories should involve neutral regions or identification as a different emotion. Instead, they suggest that facial expressions of basic emotions are recognised by their fit to discrete categories. Experiment 3 used continua involving 6 emotions to demonstrate best discrimination of pairs of stimuli falling across category boundaries; this provides further evidence of categorical perception of facial expressions of emotion. However, in both Experiment 1 and Experiment 2, reaction time data showed that increasing distance from the prototype had a definite cost on ability to identify emotion in the resulting morphed face. Moreover, Experiment 4 showed that subjects had some insight into which emotions were blended to create specific morphed images. Hence, categorical perception effects were found even though subjects were sensitive to physical properties of these morphed facial expressions. We suggest that rapid classification of prototypes and better across boundary discriminability reflect the underlying organisation of human categorisation abilities.

Configural information in facial expression perception.

Composite facial expressions were prepared by aligning the top half of one expression (e.g., anger) with the bottom half of another (e.g., happiness). Experiment 1 shows that participants are slower to identify the expression in either half of these composite images relative to a "noncomposite" control condition in which the 2 halves are misaligned. This parallels the composite effect for facial identity (A. W. Young, D. Hellawell, & D. C. Hay, 1987), and like its identity counterpart, the effect is disrupted by inverting the stimuli (Experiment 2). Experiment 3 shows that no composite effect is found when the top and bottom sections contain different models' faces posing the same expression; this serves to exclude many nonconfigural interpretations of the composite effect (e.g., that composites are more "attention-grabbing" than noncomposites). Finally, Experiment 4 demonstrates that the composite effects for identity and expression operate independently of one another.

A principal component analysis of facial expressions.

Pictures of facial expressions from the Ekman and Friesen set (Ekman, P., Friesen, W. V., (1976). Pictures of facial affect. Palo Alto, California: Consulting Psychologists Press) were submitted to a principal component analysis (PCA) of their pixel intensities. The output of the PCA was submitted to a series of linear discriminant analyses which revealed three principal findings: (1) a PCA-based system can support facial expression recognition, (2) continuous two-dimensional models of emotion (e.g. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161-1178) are reflected in the statistical structure of the Ekman and Friesen facial expressions, and (3) components for coding facial expression information are largely different to components for facial identity information. The implications for models of face processing are discussed.

A novel functional brain imaging endophenotype of autism: the neural response to facial expression of emotion.

Siblings of individuals with autism have over 20 times the population risk of autism. Evidence of comparable, but less marked, cognitive and social communication deficits in siblings suggests a role for these traits in the search for biomarkers of familial risk. However, no neuroimaging biomarkers of familial risk have been identified to date. Here we show, for the first time, that the neural response to facial expression of emotion differs between unaffected siblings and healthy controls with no family history of autism. Strikingly, the functional magnetic resonance imaging (fMRI) response to happy versus neutral faces was significantly reduced in unaffected siblings compared with controls within a number of brain areas implicated in empathy and face processing. The response in unaffected siblings did not differ significantly from the response in autism. Furthermore, investigation of the response to faces versus fixation crosses suggested that, within the context of this study, an atypical response specifically to happy faces, rather than to faces in general, accounts for the observed sibling versus controls difference and is a clear biomarker of familial risk. Our findings suggest that an atypical implicit response to facial expression of emotion may form the basis of impaired emotional reactivity in autism and in the broader autism phenotype in relatives. These results demonstrate that the fMRI response to facial expression of emotion is a candidate neuroimaging endophenotype for autism, and may offer far-reaching insights into the etiology of autism.

Paying attention to emotional images with impact.

Emotional stimuli receive high processing priority in attention and memory. This processing "advantage" is generally thought to be predominantly mediated by arousal. However, recent data suggest that ratings of an image's affective "impact" may be a better predictor of recollection than arousal or valence. One interpretation of these findings is that high-impact images may draw an individual's attention, thus facilitating subsequent processing. We investigated the allocation of visual attention to negative emotional images that differed in impact but were matched for valence, arousal, and other characteristics. Participants viewed a central image flanked by 2 neutral indoor or outdoor scenes and made speeded judgments about whether the neutral scenes matched. In Experiment 1, responses were slower on high-impact relative to low-impact or neutral trials. In Experiment 2, responses on high-arousal relative to low-arousal trials did not differ significantly. These data provide evidence for differential allocation of attention to distinct sets of negative, equally arousing images, and argue against the prevailing view that heightened attention to and processing of emotional stimuli relate simply to arousal or valence.

Disgust and happiness recognition correlate with anteroventral insula and amygdala volume respectively in preclinical Huntington's disease.

Patients with Huntington's disease (HD) can show disproportionate impairments in recognizing facial signals of disgust, but the neural basis of this deficit remains unclear. Functional imaging studies have implicated the anterior insula in the ability to recognize disgust, but have identified other structures as well, including the basal ganglia. In view of variable insula and basal ganglia volume changes in HD, we used voxel-based morphometry to map regional variations in gray matter (GM) volume in participants carrying the mutation for HD, and correlated this with their performance on a test of facial emotion recognition for six basic emotions (disgust, fear, anger, happiness, sadness, surprise). The volume of the anteroventral insula was strongly correlated with performance on the disgust recognition task. The amygdala volume (bilaterally) correlated with the ability to recognize happy facial expressions. There was marked specificity of the regional correlations for the emotion involved. Recognition of other emotion expressions, or more general cognitive or motor performance as measured by a standardized rating scale, did not correlate with regional brain volume in this group. Control participants showed no effect for any measure. The strong linear correlations for disgust and happiness recognition imply direct involvement of the anterior insula in disgust appreciation, and a similar role for the amygdala in recognizing happy facial expressions. The absence of a significant correlation with the basal ganglia suggests a less critical role for these structures in disgust recognition than has previously been suggested. The findings also highlight the role of neurodegenerative diseases combined with statistical imaging techniques in elucidating the brain basis of behavior and cognition.

Neuropsychology of fear and loathing.

For over 60 years, ideas about emotion in neuroscience and psychology have been dominated by a debate on whether emotion can be encompassed within a single, unifying model. In neuroscience, this approach is epitomized by the limbic system theory and, in psychology, by dimensional models of emotion. Comparative research has gradually eroded the limbic model, and some scientists have proposed that certain individual emotions are represented separately in the brain. Evidence from humans consistent with this approach has recently been obtained by studies indicating that signals of fear and disgust are processed by distinct neural substrates. We review this research and its implications for theories of emotion.