The contribution of dynamics to macaque body and face patch responses.

Previous functional imaging studies demonstrated body-selective patches in the primate visual temporal cortex, comparing activations to static bodies and static images of other categories. However, the use of static instead of dynamic displays of moving bodies may have underestimated the extent of the body patch network. Indeed, body dynamics provide information about action and emotion and may be processed in patches not activated by static images. Thus, to map with fMRI the full extent of the macaque body patch system in the visual temporal cortex, we employed dynamic displays of natural-acting monkey bodies, dynamic monkey faces, objects, and scrambled versions of these videos, all presented during fixation. We found nine body patches in the visual temporal cortex, starting posteriorly in the superior temporal sulcus (STS) and ending anteriorly in the temporal pole. Unlike for static images, body patches were present consistently in both the lower and upper banks of the STS. Overall, body patches showed a higher activation by dynamic displays than by matched static images, which, for identical stimulus displays, was less the case for the neighboring face patches. These data provide the groundwork for future single-unit recording studies to reveal the spatiotemporal features the neurons of these body patches encode. These fMRI findings suggest that dynamics have a stronger contribution to population responses in body than face patches.

Mental Imagery Follows Similar Cortical Reorganization as Perception: Intra-Modal and Cross-Modal Plasticity in Congenitally Blind.

Cortical plasticity in congenitally blind individuals leads to cross-modal activation of the visual cortex and may lead to superior perceptual processing in the intact sensory domains. Although mental imagery is often defined as a quasi-perceptual experience, it is unknown whether it follows similar cortical reorganization as perception in blind individuals. In this study, we show that auditory versus tactile perception evokes similar intra-modal discriminative patterns in congenitally blind compared with sighted participants. These results indicate that cortical plasticity following visual deprivation does not influence broad intra-modal organization of auditory and tactile perception as measured by our task. Furthermore, not only the blind, but also the sighted participants showed cross-modal discriminative patterns for perception modality in the visual cortex. During mental imagery, both groups showed similar decoding accuracies for imagery modality in the intra-modal primary sensory cortices. However, no cross-modal discriminative information for imagery modality was found in early visual cortex of blind participants, in contrast to the sighted participants. We did find evidence of cross-modal activation of higher visual areas in blind participants, including the representation of specific-imagined auditory features in visual area V4.

When a smile becomes a fist: the perception of facial and bodily expressions of emotion in violent offenders.

Previous reports have suggested an enhancement of facial expression recognition in women as compared to men. It has also been suggested that men versus women have a greater attentional bias towards angry cues. Research has shown that facial expression recognition impairments and attentional biases towards anger are enhanced in violent criminal male offenders. Bodily expressions of anger form a more direct physical threat as compared to facial expressions. In four experiments, we tested how 29 imprisoned aggressive male offenders perceive body expressions by other males. The performance of all participants in a matching-to-sample task dropped significantly when the distracting image showed an angry posture. Violent offenders misjudged fearful body movements as expressing anger significantly more often than the control group. When violent offenders were asked to categorize facial expressions and ignore the simultaneously presented congruent or incongruent posture, they performed worse than the control group, specifically, when a smile was combined with an aggressive posture. Finally, violent offenders showed a greater congruency effect than controls when viewing postures as part of an emotionally congruent social scene and did not perform above chance when categorizing a happy posture presented in a fight scene. The results suggest that violent offenders have difficulties in processing emotional incongruence when aggressive stimuli are involved and a possible bias towards aggressive body language.

Similarities and differences in perceiving threat from dynamic faces and bodies. An fMRI study.

Neuroscientific research on the perception of emotional signals has mainly focused on how the brain processes threat signals from photographs of facial expressions. Much less is known about body postures or about the processing of dynamic images. We undertook a systematic comparison of the neurofunctional network dedicated to processing facial and bodily expressions. Two functional magnetic resonance imaging (fMRI) experiments investigated whether areas involved in processing social signals are activated differently by threatening signals (fear and anger) from facial or bodily expressions. The amygdala (AMG) was more active for facial than for bodily expressions. Body stimuli triggered higher activation than face stimuli in a number of areas. These were the cuneus, fusiform gyrus (FG), extrastriate body area (EBA), temporoparietal junction (TPJ), superior parietal lobule (SPL), primary somatosensory cortex (SI), as well as the thalamus. Emotion-specific effects were found in TPJ and FG for bodies and faces alike. EBA and superior temporal sulcus (STS) were more activated by threatening bodies.

Tease or threat? Judging social interactions from bodily expressions.

We casually observe many interactions that do not really concern us. Yet sometimes we need to be able to rapidly appraise whether an interaction between two people represents a real threat for one of them rather than an innocent tease. Using functional magnetic resonance imaging, we investigated whether small differences in the body language of two interacting people are picked up by the brain even if observers are performing an unrelated task. Fourteen participants were scanned while watching 3-s movies (192 trials and 96 scrambles) showing a male person either threatening or teasing a female one. In one task condition, observers categorized the interaction as threatening or teasing, and in the other, they monitored randomly appearing dots and categorized the color. Our results clearly show that right amygdala responds more to threatening than to teasing situations irrespective of the observers' task. When observers' attention is not explicitly directed to the situation, this heightened amygdala activation goes together with increased activity in body sensitive regions in fusiform gyrus, extrastriate body area-human motion complex and superior temporal sulcus and is associated with a better behavioral performance of the participants during threatening situations. In addition, regions involved in action observation (inferior frontal gyrus, temporoparietal junction, and inferior parietal lobe) and preparation (premotor, putamen) show increased activation for threat videos. Also regions involved in processing moral violations (temporoparietal junction, hypothalamus) reacted selectively to the threatening interactions. Taken together, our results show which brain regions react selectively to witnessing a threatening interaction even if the situation is not attended because the observers perform an unrelated task.

Audiovisual emotion recognition in schizophrenia: reduced integration of facial and vocal affect.

Since Kraepelin called dementia praecox what we nowadays call schizophrenia, cognitive dysfunction has been regarded as central to its psychopathological profile. Disturbed experience and integration of emotions are, both intuitively and experimentally, likely to be intermediates between basic, non-social cognitive disturbances and functional outcome in schizophrenia. While a number of studies have consistently proven that, as part of social cognition, recognition of emotional faces and voices is disturbed in schizophrenics, studies on multisensory integration of facial and vocal affect are rare. We investigated audiovisual integration of emotional faces and voices in three groups: schizophrenic patients, non-schizophrenic psychosis patients and mentally healthy controls, all diagnosed by means of the Schedules of Clinical Assessment in Neuropsychiatry (SCAN 2.1). We found diminished crossmodal influence of emotional faces on emotional voice categorization in schizophrenics, but not in non-schizophrenia psychosis patients. Results are discussed in the perspective of recent theories on multisensory integration.

Neuroscience. More to seeing than meets the eye.

A single event may elicit several different sensory stimuli such as vision, sound, and touch. But how does the brain know which of the many different stimuli arriving in the sensory cortex of the brain are connected? In her Perspective, de Gelder discusses new findings showing that when a touch is applied on the same side of the body as a visual cue, vision is enhanced (Macaluso et al.). She explains that this effect is due to neurons projecting from the somatosensory (touch) area of the sensory cortex back to the visual cortex. These neurons keep the visual cortex informed about tactile stimuli elicited at the same time as the visual stimuli.

Decreased differential activity in the amygdala in response to fearful expressions in Type D personality.

Recent advances in functional brain imaging offer unique opportunities to explore the neurofunctional basis of tools used to assess personality differences which have proven their clinical usefulness. In this functional magnetic resonance imaging (fMRI) study, the focus was on the amygdala activation and we investigated whether individual differences in activity of the amygdala following presentation of emotional expressions in the face and the whole body may be systematically related to the presence of Type D (distressed) personality or to its constituting factors, Negative Affectivity (NA) and Social Inhibition (SI). Our results show that the observed difference in amygdala activity between fearful and neutral expressions was present in participants that did not meet the criteria for Type D personality, while this effect was absent in participants that could be classified as Type D personality. Our correlation analyses further showed that the activation in the left amygdala elicited by fearful versus neutral bodily expressions correlated negatively with the Negative Affectivity score. The same pattern was observed for the right amygdala for fearful facial and bodily expressions when contrasted with neutral facial and bodily expressions.

Offenders become the victim in virtual reality: impact of changing perspective in domestic violence.

The role of empathy and perspective-taking in preventing aggressive behaviors has been highlighted in several theoretical models. In this study, we used immersive virtual reality to induce a full body ownership illusion that allows offenders to be in the body of a victim of domestic abuse. A group of male domestic violence offenders and a control group without a history of violence experienced a virtual scene of abuse in first-person perspective. During the virtual encounter, the participants' real bodies were replaced with a life-sized virtual female body that moved synchronously with their own real movements. Participants' emotion recognition skills were assessed before and after the virtual experience. Our results revealed that offenders have a significantly lower ability to recognize fear in female faces compared to controls, with a bias towards classifying fearful faces as happy. After being embodied in a female victim, offenders improved their ability to recognize fearful female faces and reduced their bias towards recognizing fearful faces as happy. For the first time, we demonstrate that changing the perspective of an aggressive population through immersive virtual reality can modify socio-perceptual processes such as emotion recognition, thought to underlie this specific form of aggressive behaviors.

Rapid detection of fear in body expressions, an ERP study.

Recent findings indicate that the perceptual processing of fearful expressions in the face can already be initiated around 100-120 ms after stimulus presentation, demonstrating that emotional information of a face can be encoded before the identity of the face is fully recognized. At present it is not clear whether fear signals from body expressions may be encoded equally as rapid. To answer this question we investigated the early temporal dynamics of perceiving fearful body expression by measuring EEG. Participants viewed images of whole body actions presented either in a neutral or a fearful version. We observed an early emotion effect on the P1 peak latency around 112 ms post stimulus onset hitherto only found for facial expressions. Also consistent with the majority of facial expression studies, the N170 component elicited by perceiving bodies proved not to be sensitive for the expressed fear. In line with previous work, its vertex positive counterpart, the VPP, did show a condition-specific influence for fearful body expression. Our results indicate that the information provided by fearful body expression is already encoded in the early stages of visual processing, and suggest that similar early processing mechanisms are involved in the perception of fear in faces and bodies.

Mismatch negativity predicts recovery from the vegetative state.

OBJECTIVE: Mismatch negativity (MMN) is an automatic event related brain response, well investigated in the acute phase after severe brain injury: the presence of a MMN is often found to predict the emergence from coma, and the exclusion of shifting into a vegetative state (VS). In the present study MMN was examined during recovery from VS. METHODS: Ten vegetative patients were repeatedly examined every 2 weeks for an average period of 3.5 months. Amplitudes and latencies were related to the patients' recovery from VS to consciousness, and to a healthy norm group. In addition, MMN was examined on its prognostic value in VS patients, in predicting recovery to consciousness and long-term functional outcome. RESULTS: With recovery to consciousness MMN-amplitudes increased. A sudden increase was seen in MMN amplitude when patients started to show inconsistent behavioural responses to simple commands. At this level MMN resembled the MMN response as was seen in the norm group. In addition, the MMN-amplitude and latency during the first measurement predicted the patients' outcome on recovery to consciousness. CONCLUSIONS: With recovery from VS to consciousness the ability to process auditory stimulus deviance increases. A sudden enhancement in MMN-amplitude preceded overt communication with the environment. This might be indicative of the consolidation of neural networks underlying overt communication. Moreover, MMN can be helpful in identifying the ability to recover from VS. SIGNIFICANCE: MMN can be used to track recovery from the vegetative state in the post-acute phase after severe brain injury. In addition, MMN can be used to predict the ability to recover from the vegetative state.

Clear signals or mixed messages: inter-individual emotion congruency modulates brain activity underlying affective body perception.

The neural basis of emotion perception has mostly been investigated with single face or body stimuli. However, in daily life one may also encounter affective expressions by groups, e.g. an angry mob or an exhilarated concert crowd. In what way is brain activity modulated when several individuals express similar rather than different emotions? We investigated this question using an experimental design in which we presented two stimuli simultaneously, with same or different emotional expressions. We hypothesized that, in the case of two same-emotion stimuli, brain activity would be enhanced, while in the case of two different emotions, one emotion would interfere with the effect of the other. The results showed that the simultaneous perception of different affective body expressions leads to a deactivation of the amygdala and a reduction of cortical activity. It was revealed that the processing of fearful bodies, compared with different-emotion bodies, relied more strongly on saliency and action triggering regions in inferior parietal lobe and insula, while happy bodies drove the occipito-temporal cortex more strongly. We showed that this design could be used to uncover important differences between brain networks underlying fearful and happy emotions. The enhancement of brain activity for unambiguous affective signals expressed by several people simultaneously supports adaptive behaviour in critical situations.

The perception of emotion in body expressions.

During communication, we perceive and express emotional information through many different channels, including facial expressions, prosody, body motion, and posture. Although historically the human body has been perceived primarily as a tool for actions, there is now increased understanding that the body is also an important medium for emotional expression. Indeed, research on emotional body language is rapidly emerging as a new field in cognitive and affective neuroscience. This article reviews how whole-body signals are processed and understood, at the behavioral and neural levels, with specific reference to their role in emotional communication. The first part of this review outlines brain regions and spectrotemporal dynamics underlying perception of isolated neutral and affective bodies, the second part details the contextual effects on body emotion recognition, and final part discusses body processing on a subconscious level. More specifically, research has shown that body expressions as compared with neutral bodies draw upon a larger network of regions responsible for action observation and preparation, emotion processing, body processing, and integrative processes. Results from neurotypical populations and masking paradigms suggest that subconscious processing of affective bodies relies on a specific subset of these regions. Moreover, recent evidence has shown that emotional information from the face, voice, and body all interact, with body motion and posture often highlighting and intensifying the emotion expressed in the face and voice.

Paradoxical configuration effects for faces and objects in prosopagnosia.

Selective impairment in recognition of faces (prosopagnosia) has been advanced as an argument for a brain module dedicated to face processing and focusing on the specific configural properties of faces. Loss of the inversion effect supposedly strengthened the argument ([10]: de Gelder B, Bachoud-Levi AC, Degos JD. Inversion superiority in visual agnosia may be common to a variety of orientation polarised objects besides faces. Vision Research, 1998;38:2855-61; [20]: Farah MJ, Wilson K, Drain H, Tanaka J. The inverted face inversion effect in prosopagnosia: Evidence for mandatory, face-specific perceptual mechanisms. Vision Research 1995b;35:2089-93). The present study of prosopagnosic patient LH reports that he has lost the normal pattern of superior performance with upright faces and objects and shows instead paradoxical inversion effect for faces but also for objects. Experiment 2 investigated whether LH's use of features based route for processing upright objects would be hindered by the whole-based encoding when processing upright objects. The data show the same context effect for objects as was found for faces. Therefore the inversion effect does not present decisive evidence for the existence of a face module. Moreover, the importance of configuration-based recognition known to be crucial for face processing, must also be taken seriously for object recognition.

The lateralization of lip-reading: a second look.

Photographs of unfamiliar speaking faces were matched by normal right-handed subjects on the basis of perceived mouth-shape (i.e. visible speech sound) across different face-views. A clear left-hemisphere (RVF) processing advantage emerged, which was absent when the task was that of identity matching. In contrast to earlier proposals, the extraction of lip-shape from face photographs may be better managed by left-hemisphere- than right-hemisphere mechanisms even at its initial stages. This may contribute to the observed patterns of dissociations in speech-reading and in audiovisual speech-processing in neurological patients.

Multiple-domain dissociation between impaired visual perception and preserved mental imagery in a patient with bilateral extrastriate lesions.

A brain-damaged patient is described whose pattern of performance provides insight into both the functional mechanisms and the neural structures involved in visual mental imagery. The patient became severely agnosic, alexic, achromatopsic and prosopagnosic following bilateral brain lesions in the temporo-occipital cortex. However, her mental imagery for the same visual entities that she could not perceive was perfectly preserved. This clear-cut dissociation held across all the major domains of high-level vision: object recognition, reading, colour and face processing. Our findings, together with other reports on domain-specific dissociations and functional brain imaging studies, provide evidence to support the view that visual perception and visual mental imagery are subserved by independent functional mechanisms, which do not share the same cortical implementation. In particular, our results suggest that mental imagery abilities need not be mediated by early visual cortices.

Configural face processes in acquired and developmental prosopagnosia: evidence for two separate face systems?

Configural face processes were tested using face recognition and face detection tasks in a comparison of acquired and developmental prosopagnosia. In the recognition task the two patients showed a very different pattern. The developmental patient does not show an inversion effect while the acquired prosopagnosia patient is better at matching inverted than normal stimuli. Moreover, there is no effect of face context on matching features in the developmental case while the acquired prosopagnosia patient shows a strong negative effect of context. However, in a speeded face detection task both patients are similarly unimpaired. The results are consistent with the existence of two separate face systems, one involved in face detection and the other in face recognition.

Event-related TMS over the right posterior parietal cortex induces ipsilateral visuo-spatial interference.

The right posterior parietal cortex (PPC) is implicated in visuo-spatial processing, as illustrated by patients with visuo-spatial neglect, but the precise time-course of its contribution is still an open question. In the present study we assessed whether single-pulse transcranial magnetic stimulation (TMS) can interfere with the performance of normal subjects in a standard visuo-spatial task. Participants had to perform a landmark task while TMS was applied over the right PPC, the homologue region in the left hemisphere or the right primary motor cortex. Stimulation was time-locked to the stimulus presentation with a stimulus onset asynchrony (SOA) varying between 50 and 200 ms. Our results indicate that TMS interfered mainly with the visuo-spatial task when applied over the right PPC at an early stage (50 ms post-stimulus). The interference effect of single-pulse TMS in the present visuo-spatial processing is revealed by a processing cost for ipsilateral targets. These results are in agreement with neuropsychological and brain imaging studies showing a right hemispheric dominance in visuo-spatial processing but add crucial information about the time-course of visuo-spatial processing within the right PPC.

The time-course of intermodal binding between seeing and hearing affective information.

Intermodal binding between affective information that is seen as well as heard triggers a mandatory process of audiovisual integration. In order to track the time course of this audiovisual binding, event related brain potentials were recorded while subjects saw facial expression and concurrently heard auditory fragment. The results suggest that the combination of the two inputs is early in time (110 ms post-stimulus) and translates as a specific enhancement in amplitude of the auditory NI component. These findings are compatible with previous functional neuroimaging results of audiovisual speech showing strong audiovisual interactions in auditory cortex in the form of magnetic response amplifications, as well as with electrophysiological studies demonstrating early audiovisual interactions (before 200 ms post-stimulus). Moreover, our results show that the informational content present in the two modalities plays a crucial role in triggering the intermodal binding process.

Non-conscious recognition of affect in the absence of striate cortex.

Functional neuroimaging experiments have shown that recognition of emotional expressions does not depend on awareness of visual stimuli and that unseen fear stimuli can activate the amygdala via a colliculopulvinar pathway. Perception of emotional expressions in the absence of awareness in normal subjects has some similarities with the unconscious recognition of visual stimuli which is well documented in patients with striate cortex lesions (blindsight). Presumably in these patients residual vision engages alternative extra-striate routes such as the superior colliculus and pulvinar. Against this background, we conjectured that a blindsight subject (GY) might recognize facial expressions presented in his blind field. The present study now provides direct evidence for this claim.