Effects of cue modality and emotional category on recognition of nonverbal emotional signals in schizophrenia.

BACKGROUND: Impaired interpretation of nonverbal emotional cues in patients with schizophrenia has been reported in several studies and a clinical relevance of these deficits for social functioning has been assumed. However, it is unclear to what extent the impairments depend on specific emotions or specific channels of nonverbal communication. METHODS: Here, the effect of cue modality and emotional categories on accuracy of emotion recognition was evaluated in 21 patients with schizophrenia and compared to a healthy control group (n = 21). To this end, dynamic stimuli comprising speakers of both genders in three different sensory modalities (auditory, visual and audiovisual) and five emotional categories (happy, alluring, neutral, angry and disgusted) were used. RESULTS: Patients with schizophrenia were found to be impaired in emotion recognition in comparison to the control group across all stimuli. Considering specific emotions more severe deficits were revealed in the recognition of alluring stimuli and less severe deficits in the recognition of disgusted stimuli as compared to all other emotions. Regarding cue modality the extent of the impairment in emotional recognition did not significantly differ between auditory and visual cues across all emotional categories. However, patients with schizophrenia showed significantly more severe disturbances for vocal as compared to facial cues when sexual interest is expressed (alluring stimuli), whereas more severe disturbances for facial as compared to vocal cues were observed when happiness or anger is expressed. CONCLUSION: Our results confirmed that perceptual impairments can be observed for vocal as well as facial cues conveying various social and emotional connotations. The observed differences in severity of impairments with most severe deficits for alluring expressions might be related to specific difficulties in recognizing the complex social emotional information of interpersonal intentions as compared to "basic" emotional states. Therefore, future studies evaluating perception of nonverbal cues should consider a broader range of social and emotional signals beyond basic emotions including attitudes and interpersonal intentions. Identifying specific domains of social perception particularly prone for misunderstandings in patients with schizophrenia might allow for a refinement of interventions aiming at improving social functioning.

I can't keep your face and voice out of my head: neural correlates of an attentional bias toward nonverbal emotional cues.

Emotional information can be conveyed by verbal and nonverbal cues with the latter often suggested to exert a greater influence in shaping our perceptions of others. The present functional magnetic resonance imaging study sought to explore attentional biases toward nonverbal signals by investigating the interaction of verbal and nonverbal cues. Results obtained in this study underline the previous suggestions of a "nonverbal dominance" in emotion communication by evidencing implicit effects of nonverbal cues on emotion judgements even when attention is directed away from nonverbal signals and focused on verbal cues. Attentional biases toward nonverbal signals appeared to be reflected in increasing activation of the dorsolateral prefrontal cortex (DLPFC) assumed to reflect increasing difficulties to suppress nonverbal cues during task conditions that asked to shift attention away from nonverbal signals. Aside the DLPFC, results suggest the right amygdala to play a role in attention control mechanisms related to the processing of emotional cues. Analyses conducted to determine the cerebral correlates of the individual ability to shift attention between verbal and nonverbal sources of information indicated that higher task-switching abilities seem to be associated with the up-regulation of right amygdala activation during explicit judgments of nonverbal cues, whereas difficulties in task-switching seem to be related to a down-regulation.

Nonverbal signals speak up: association between perceptual nonverbal dominance and emotional intelligence.

Emotional communication uses verbal and nonverbal means. In case of conflicting signals, nonverbal information is assumed to have a stronger impact. It is unclear, however, whether perceptual nonverbal dominance varies between individuals and whether it is linked to emotional intelligence. Using audiovisual stimulus material comprising verbal and nonverbal emotional cues that were varied independently, perceptual nonverbal dominance profiles and their relations to emotional intelligence were examined. Nonverbal dominance was found in every participant, ranging from 55 to 100%. Moreover, emotional intelligence, particularly the ability to understand emotions, correlated positively with nonverbal dominance. Furthermore, higher overall emotional intelligence as well as a higher ability to understand emotions were linked to smaller reaction time differences between emotionally incongruent and congruent stimuli. The association between perceptual nonverbal dominance and emotional intelligence, and more specifically the ability to understand emotions, might reflect an adaptive process driven by the experience of higher authenticity in nonverbal cues.

Impairments in recognition of emotional facial expressions, affective prosody, and multisensory facilitation of response time in high-functioning autism.

Introduction: Deficits in emotional perception are common in autistic people, but it remains unclear to which extent these perceptual impairments are linked to specific sensory modalities, specific emotions or multisensory facilitation. Methods: This study aimed to investigate uni- and bimodal perception of emotional cues as well as multisensory facilitation in autistic (n = 18, mean age: 36.72 years, SD: 11.36) compared to non-autistic (n = 18, mean age: 36.41 years, SD: 12.18) people using auditory, visual and audiovisual stimuli. Results: Lower identification accuracy and longer response time were revealed in high-functioning autistic people. These differences were independent of modality and emotion and showed large effect sizes (Cohen's d 0.8-1.2). Furthermore, multisensory facilitation of response time was observed in non-autistic people that was absent in autistic people, whereas no differences were found in multisensory facilitation of accuracy between the two groups. Discussion: These findings suggest that processing of auditory and visual components of audiovisual stimuli is carried out more separately in autistic individuals (with equivalent temporal demands required for processing of the respective unimodal cues), but still with similar relative improvement in accuracy, whereas earlier integrative multimodal merging of stimulus properties seems to occur in non-autistic individuals.

Cerebral integration of verbal and nonverbal emotional cues: impact of individual nonverbal dominance.

Emotional communication is essential for successful social interactions. Emotional information can be expressed at verbal and nonverbal levels. If the verbal message contradicts the nonverbal expression, usually the nonverbal information is perceived as being more authentic, revealing the "true feelings" of the speaker. The present fMRI study investigated the cerebral integration of verbal (sentences expressing the emotional state of the speaker) and nonverbal (facial expressions and tone of voice) emotional signals using ecologically valid audiovisual stimulus material. More specifically, cerebral activation associated with the relative impact of nonverbal information on judging the affective state of a speaker (individual nonverbal dominance index, INDI) was investigated. Perception of nonverbally expressed emotions was associated with bilateral activation within the amygdala, fusiform face area (FFA), temporal voice area (TVA), and the posterior temporal cortex as well as in the midbrain and left inferior orbitofrontal cortex (OFC)/left insula. Verbally conveyed emotions were linked to increased responses bilaterally in the TVA. Furthermore, the INDI correlated with responses in the left amygdala elicited by nonverbal and verbal emotional stimuli. Correlation of the INDI with the activation within the medial OFC was observed during the processing of communicative signals. These results suggest that individuals with a higher degree of nonverbal dominance have an increased sensitivity not only to nonverbal but to emotional stimuli in general.

Correlates of individual voice and face preferential responses during resting state.

Human nonverbal social signals are transmitted to a large extent by vocal and facial cues. The prominent importance of these cues is reflected in specialized cerebral regions which preferentially respond to these stimuli, e.g. the temporal voice area (TVA) for human voices and the fusiform face area (FFA) for human faces. But it remained up to date unknown whether there are respective specializations during resting state, i.e. in the absence of any cues, and if so, whether these representations share neural substrates across sensory modalities. In the present study, resting state functional connectivity (RSFC) as well as voice- and face-preferential activations were analysed from functional magnetic resonance imaging (fMRI) data sets of 60 healthy individuals. Data analysis comprised seed-based analyses using the TVA and FFA as regions of interest (ROIs) as well as multi voxel pattern analyses (MVPA). Using the face- and voice-preferential responses of the FFA and TVA as regressors, we identified several correlating clusters during resting state spread across frontal, temporal, parietal and occipital regions. Using these regions as seeds, characteristic and distinct network patterns were apparent with a predominantly convergent pattern for the bilateral TVAs whereas a largely divergent pattern was observed for the bilateral FFAs. One region in the anterior medial frontal cortex displayed a maximum of supramodal convergence of informative connectivity patterns reflecting voice- and face-preferential responses of both TVAs and the right FFA, pointing to shared neural resources in supramodal voice and face processing. The association of individual voice- and face-preferential neural activity with resting state connectivity patterns may support the perspective of a network function of the brain beyond an activation of specialized regions.

Reduced impact of nonverbal cues during integration of verbal and nonverbal emotional information in adults with high-functioning autism.

Background: When receiving mismatching nonverbal and verbal signals, most people tend to base their judgment regarding the current emotional state of others primarily on nonverbal information. However, individuals with high-functioning autism (HFA) have been described as having difficulties interpreting nonverbal signals. Recognizing emotional states correctly is highly important for successful social interaction. Alterations in perception of nonverbal emotional cues presumably contribute to misunderstanding and impairments in social interactions. Methods: To evaluate autism-specific differences in the relative impact of nonverbal and verbal cues, 18 adults with HFA (14 male and four female subjects, mean age 36.7 years (SD 11.4) and 18 age, gender and IQ-matched typically developed controls [14 m/4 f, mean age 36.4 years (SD 12.2)] rated the emotional state of speakers in video sequences with partly mismatching emotional signals. Standardized linear regression coefficients were calculated as a measure of the reliance on the nonverbal and verbal components of the videos for each participant. Regression coefficients were then compared between groups to test the hypothesis that autistic adults base their social evaluations less strongly on nonverbal information. Further exploratory analyses were performed for differences in valence ratings and response times. Results: Compared to the typically developed control group, nonverbal cue reliance was reduced in adults with high-functioning autism [t(23.14) = -2.44, p = 0.01 (one-sided)]. Furthermore, the exploratory analyses showed a tendency to avoid extreme answers in the HFA group, observable via less positive as well as less negative valence ratings in response to emotional expressions of increasingly strong valence. In addition, response time was generally longer in HFA compared to the control group [F (1, 33) = 10.65, p = 0.004]. Conclusion: These findings suggest reduced impact of nonverbal cues and longer processing times in the analysis of multimodal emotional information, which may be associated with a subjectively lower relevance of this information and/or more processing difficulties for people with HFA. The less extreme answering tendency may indicate a lower sensitivity for nonverbal valence expression in HFA or result from a tendency to avoid incorrect answers when confronted with greater uncertainty in interpreting emotional states.

Attenuated impression of irony created by the mismatch of verbal and nonverbal cues in patients with autism spectrum disorder.

Perception of irony has been observed to be impaired in adults with autism spectrum disorder. In typically developed adults, the mismatch of verbal and nonverbal emotional cues can be perceived as an expression of irony even in the absence of any further contextual information. In this study, we evaluate to what extent high functioning autists perceive this incongruence as expressing irony. Our results show that incongruent verbal and nonverbal signals create an impression of irony significantly less often in participants with high-functioning autism than in typically developed control subjects. The extent of overall autistic symptomatology as measured with the autism-spectrum questionnaire (AQ), however, does not correlate with the reduced tendency to attribute incongruent stimuli as expressing irony. Therefore, the attenuation in irony attribution might rather be related to specific subdomains of autistic traits, such as a reduced tendency to interpret communicative signals in terms of complex intentional mental states. The observed differences in irony attribution support the assumption that a less pronounced tendency to engage in higher order mentalization processes might underlie the impairment of pragmatic language understanding in high functioning autism.

Neurobiological correlates of emotional intelligence in voice and face perception networks.

Facial expressions and voice modulations are among the most important communicational signals to convey emotional information. The ability to correctly interpret this information is highly relevant for successful social interaction and represents an integral component of emotional competencies that have been conceptualized under the term emotional intelligence. Here, we investigated the relationship of emotional intelligence as measured with the Salovey-Caruso-Emotional-Intelligence-Test (MSCEIT) with cerebral voice and face processing using functional and structural magnetic resonance imaging. MSCEIT scores were positively correlated with increased voice-sensitivity and gray matter volume of the insula accompanied by voice-sensitivity enhanced connectivity between the insula and the temporal voice area, indicating generally increased salience of voices. Conversely, in the face processing system, higher MSCEIT scores were associated with decreased face-sensitivity and gray matter volume of the fusiform face area. Taken together, these findings point to an alteration in the balance of cerebral voice and face processing systems in the form of an attenuated face-vs-voice bias as one potential factor underpinning emotional intelligence.

Perception of Verbal and Nonverbal Emotional Signals in Women With Borderline Personality Disorder: Evidence of a Negative Bias and an Increased Reliance on Nonverbal Cues.

Studies conducted in patients diagnosed with borderline personality disorder (BPD) have documented a variety of anomalies concerning patients' abilities to interpret emotional signals. Attempting to clarify the bases of these anomalies, the current literature draws attention to a possible role of dysfunctional expectations, such as the expectation of social rejection. Dysfunctional expectations, however, may not only bias social interpretations, but may also focus attention on social cues most important in conveying emotional messages, such as nonverbal signals. To explore these assumptions, 30 female BPD patients were tasked to judge the valence of emotional states conveyed by combinations of verbal and nonverbal emotional cues. Compared to controls, BPD patients exhibited a negative bias in their interpretations and relied more on available nonverbal cues. Shifts in the relative importance of nonverbal cues appeared to be rooted mainly in a reduced reliance on positive verbal cues presumably deemed less credible by BPD patients.

Integration of verbal and nonverbal emotional signals in patients with schizophrenia: Decreased nonverbal dominance.

In day-to-day social interaction, emotions are usually expressed by verbal (e.g. spoken words) and nonverbal signals (e.g. facial expressions, prosody). In case of conflicting signals nonverbal signals are perceived as being the more reliable source of information. Deficits in interpreting nonverbal signals - as described for patients with schizophrenic disorders - might interfere with the ability to integrate verbal and nonverbal social cues into a meaningful whole. The aim of this study was to examine how schizophrenic disorders influence the integration of verbal and nonverbal signals. For this purpose short video sequences were presented to 21 patients with schizophrenia and 21 healthy controls. Each sequence showed an actor speaking a short sentence with independently varying emotional connotations at the verbal and the nonverbal level. The participants rated the valence of the speaker's emotional state on a four-point scale (from very negative to very positive). The relative impact of nonverbal cues as compared to verbal cues on these ratings was evaluated. Both groups base their decisions primarily on nonverbal information. However, this effect is significantly less prominent in the patient group. Patients tend to base their decisions less on nonverbal signals and more on verbal information than healthy controls.

Effects of Emotional Intelligence on the Impression of Irony Created by the Mismatch between Verbal and Nonverbal Cues.

Emotional information is conveyed through verbal and nonverbal signals, with nonverbal cues often being considered the decisive factor in the judgment of others' emotional states. The aim of the present study was to examine how verbal and nonverbal cues are integrated by perceivers. More specifically, we tested whether the mismatch between verbal and nonverbal information was perceived as an expression of irony. Moreover, we investigated the effects of emotional intelligence on the impression of irony. The findings revealed that the mismatch between verbal and nonverbal information created the impression of irony. Furthermore, participants higher in emotional intelligence were faster at rating such stimuli as ironic expressions.

Laughter perception in social anxiety.

BACKGROUND: Laughter is a powerful signal of social acceptance or rejection while the fear of being embarrassed and humiliated is central in social anxiety (SA). This type of anxiety is associated with cognitive biases indicating increased sensitivity to social threat as well as with deficits in emotion regulation. Both are thought to be implicated in the maintenance of social anxiety. METHOD: Using laughter as a novel stimulus, we investigated cognitive biases and their modulation through emotion regulation and cue ambiguity in individuals with varying degrees of SA (N = 60). RESULTS: A combination of a negative laughter interpretation bias and an attention bias away from joyful/social inclusive laughter in SA was observed. Both biases were not attributable to effects of general anxiety and were closely correlated with the concept of gelotophobia, the fear of being laughed at. DISCUSSION: Thus, our study demonstrates altered laughter perception in SA. Furthermore, it highlights the usefulness of laughter as a highly prevalent social signal for future research on the interrelations of interpretation and attention biases in SA and their modulation through emotion regulation.

Cerebral processing of prosodic emotional signals: evaluation of a network model using rTMS.

A great number of functional imaging studies contributed to developing a cerebral network model illustrating the processing of prosody in the brain. According to this model, the processing of prosodic emotional signals is divided into three main steps, each related to different brain areas. The present study sought to evaluate parts of the aforementioned model by using low-frequency repetitive transcranial magnetic stimulation (rTMS) over two important brain regions identified by the model: the superior temporal cortex (Experiment 1) and the inferior frontal cortex (Experiment 2). The aim of both experiments was to reduce cortical activity in the respective brain areas and evaluate whether these reductions lead to measurable behavioral effects during prosody processing. However, results obtained in this study revealed no rTMS effects on the acquired behavioral data. Possible explanations for these findings are discussed in the paper.

They are laughing at me: cerebral mediation of cognitive biases in social anxiety.

The fear of embarrassment and humiliation is the central element of social anxiety. This frequent condition is associated with cognitive biases indicating increased sensitivity to signals of social threat, which are assumed to play a causal role in the maintenance of social anxiety. Here, we employed laughter, a potent medium for the expression of acceptance and rejection, as an experimental stimulus in participants selected for varying degrees of social anxiety to identify cerebral mediators of cognitive biases in social anxiety using functional magnetic resonance imaging in combination with mediation analysis. We directly demonstrated that cerebral activation patterns within the dorsal attention network including the left dorsolateral and dorsomedial prefrontal cortex mediate the influence of social anxiety on laughter perception. This mediation proved to be specific for social anxiety after correction for measures of general state and trait anxiety and occurred most prominently under bimodal audiovisual laughter presentation when compared with monomodal auditory or visual laughter cues. Considering the possibility to modulate cognitive biases and cerebral activity by neuropsychological trainings, non-invasive electrophysiological stimulation and psychotherapy, this study represents a starting point for a whole line of translational research projects and identifies promising targets for electrophysiological interventions aiming to alleviate cognitive biases and symptom severity in social anxiety.

Non-verbal emotion communication training induces specific changes in brain function and structure.

The perception of emotional cues from voice and face is essential for social interaction. However, this process is altered in various psychiatric conditions along with impaired social functioning. Emotion communication trainings have been demonstrated to improve social interaction in healthy individuals and to reduce emotional communication deficits in psychiatric patients. Here, we investigated the impact of a non-verbal emotion communication training (NECT) on cerebral activation and brain structure in a controlled and combined functional magnetic resonance imaging (fMRI) and voxel-based morphometry study. NECT-specific reductions in brain activity occurred in a distributed set of brain regions including face and voice processing regions as well as emotion processing- and motor-related regions presumably reflecting training-induced familiarization with the evaluation of face/voice stimuli. Training-induced changes in non-verbal emotion sensitivity at the behavioral level and the respective cerebral activation patterns were correlated in the face-selective cortical areas in the posterior superior temporal sulcus and fusiform gyrus for valence ratings and in the temporal pole, lateral prefrontal cortex and midbrain/thalamus for the response times. A NECT-induced increase in gray matter (GM) volume was observed in the fusiform face area. Thus, NECT induces both functional and structural plasticity in the face processing system as well as functional plasticity in the emotion perception and evaluation system. We propose that functional alterations are presumably related to changes in sensory tuning in the decoding of emotional expressions. Taken together, these findings highlight that the present experimental design may serve as a valuable tool to investigate the altered behavioral and neuronal processing of emotional cues in psychiatric disorders as well as the impact of therapeutic interventions on brain function and structure.