Recruitment of Language-, Emotion- and Speech-Timing Associated Brain Regions for Expressing Emotional Prosody: Investigation of Functional Neuroanatomy with fMRI.

We aimed to progress understanding of prosodic emotion expression by establishing brain regions active when expressing specific emotions, those activated irrespective of the target emotion, and those whose activation intensity varied depending on individual performance. BOLD contrast data were acquired whilst participants spoke non-sense words in happy, angry or neutral tones, or performed jaw-movements. Emotion-specific analyses demonstrated that when expressing angry prosody, activated brain regions included the inferior frontal and superior temporal gyri, the insula, and the basal ganglia. When expressing happy prosody, the activated brain regions also included the superior temporal gyrus, insula, and basal ganglia, with additional activation in the anterior cingulate. Conjunction analysis confirmed that the superior temporal gyrus and basal ganglia were activated regardless of the specific emotion concerned. Nevertheless, disjunctive comparisons between the expression of angry and happy prosody established that anterior cingulate activity was significantly higher for angry prosody than for happy prosody production. Degree of inferior frontal gyrus activity correlated with the ability to express the target emotion through prosody. We conclude that expressing prosodic emotions (vs. neutral intonation) requires generic brain regions involved in comprehending numerous aspects of language, emotion-related processes such as experiencing emotions, and in the time-critical integration of speech information.

The role of left and right dorsolateral prefrontal cortex in semantic processing: A transcranial direct current stimulation study.

For complex linguistic strings such as idioms, making a decision as to the correct meaning may require complex top-down cognitive control such as the suppression of incorrect alternative meanings. In the study presented here, we used transcranial direct current stimulation to test the hypothesis that a domain general dorsolateral prefrontal cognitive control network is involved in constraining the complex processing involved. Specifically, we sought to test prominent theoretical stances on the division of labour across dorsolateral prefrontal cortex in the left- and right-hemispheres of the brain, including the role of salience and fine vs. coarse semantic coding. 32 healthy young adult participants were randomly allocated to one of two stimulation montage groups (LH anodal/RH cathodal or RH anodal/LH cathodal). Participants were tested twice, completing a semantic decision task after either receiving active or sham stimulation. The semantic decision task required participants to judge the relatedness of an idiom and a target word. The target word was figuratively related, literally related, or unrelated to the idiom. Control non-literal non-idiomatic sentences were also included that only had a literal meaning. The results showed that left-hemisphere dorsolateral prefrontal cortex is highly involved in processing figurative language, whereas both left- and right- dorsolateral prefrontal cortex contributed to literal language processing. In comparison, semantic processing for the non-idiomatic control sentences did not require domain general cognitive control as it relates to suppression of the rejected alternative meaning. The results are discussed in terms of the interplay between need for domain general cognitive control in understanding the meaning of complex sentences, hemispheric differences in semantic processing, and salience detection.

What is the Value of Embedding Artificial Emotional Prosody in Human-Computer Interactions? Implications for Theory and Design in Psychological Science.

In computerized technology, artificial speech is becoming increasingly important, and is already used in ATMs, online gaming and healthcare contexts. However, today's artificial speech typically sounds monotonous, a main reason for this being the lack of meaningful prosody. One particularly important function of prosody is to convey different emotions. This is because successful encoding and decoding of emotions is vital for effective social cognition, which is increasingly recognized in human-computer interaction contexts. Current attempts to artificially synthesize emotional prosody are much improved relative to early attempts, but there remains much work to be done due to methodological problems, lack of agreed acoustic correlates, and lack of theoretical grounding. If the addition of synthetic emotional prosody is not of sufficient quality, it may risk alienating users instead of enhancing their experience. So the value of embedding emotion cues in artificial speech may ultimately depend on the quality of the synthetic emotional prosody. However, early evidence on reactions to synthesized non-verbal cues in the facial modality bodes well. Attempts to implement the recognition of emotional prosody into artificial applications and interfaces have perhaps been met with greater success, but the ultimate test of synthetic emotional prosody will be to critically compare how people react to synthetic emotional prosody vs. natural emotional prosody, at the behavioral, socio-cognitive and neural levels.

A possible contributory mechanism for impaired idiom perception in schizophrenia.

In this review, we focus on the ability of people with schizophrenia to correctly perceive the meaning of idioms; figurative language expressions in which intended meaning is not derived from the meaning of constituent words. We collate evidence on how idiom perception is impaired, ascertain the clinical relevance of this impairment, and consider possible psychological and neural mechanisms behind the impairment. In reviewing extant literature, we searched the PubMed database, from 1975-2014, focussing on articles that directly concerned schizophrenia and idioms, with follow up searches to explore the viability of possible underlying mechanisms. We learn that there is clear evidence of impairment, with a tendency to err towards literal interpretations unless the figurative meaning is salient, and despite contextual cues to figurative interpretations. Given the importance of idioms in everyday language, the potential impact is significant. Clinically, impaired idiom perception primarily relates to positive symptoms of schizophrenia, but also to negative symptoms. The origins of the impairment remain speculation, with impaired executive function, impaired semantic functions, and impaired context processing all proposed to explain the phenomenon. We conclude that a possible contributory mechanism at the neural level is an impaired dorsolateral prefrontal cortex system for cognitive control over semantic processing.

The overlapping relationship between emotion perception and theory of mind.

Socio-cognitive skills are crucial for successful interpersonal interactions. Two particularly important socio-cognitive processes are emotion perception (EP) and theory of mind (ToM), but agreement is lacking on terminology and conceptual links between these constructs. Here we seek to clarify the relationship between the two at multiple levels, from concept to neuroanatomy. EP is often regarded as a low-level perceptual process necessary to decode affective cues, while ToM is usually seen as a higher-level cognitive process involving mental state deduction. In information processing models, EP tends to precede ToM. At the neuroanatomical level, lesion study data suggest that EP and ToM are both right-hemisphere based, but there is also evidence that ToM requires temporal-cingulate networks, whereas EP requires partially separable regions linked to distinct emotions. Common regions identified in fMRI studies of EP and ToM have included medial prefrontal cortex and temporal lobe areas, but differences emerge depending on the perceptual, cognitive and emotional demands of the EP and ToM tasks. For the future, clarity of definition of EP and ToM will be paramount to produce distinct task manipulations and inform models of socio-cognitive processing.

Theory of Mind in Bipolar Disorder, with Comparison to the Impairments Observed in Schizophrenia.

Our ability to make sense of information on the potential intentions and dispositions of others is of paramount importance for understanding their communicative intent, and for judging what an appropriate reaction might be. Thus, anything that impinges on this ability has the potential to cause significant social impairment, and compromise an individual's level of functioning. Both bipolar disorder and schizophrenia are known to feature theory of mind impairment. We conducted a theoretical review to determine the extent and types of theory of mind impairment in bipolar disorder, and evaluate their relationship to medication and symptoms. We also considered possible mediatory mechanisms, and set out to discover what else could be learnt about the impairment in bipolar disorder by comparison to the profile of impairment in schizophrenia. The literature established that in bipolar disorder (i) some form of theory of mind impairment has been observed in all mood states, including euthymia, (ii) the form of theory of mind assessed and task used to make the assessment influence the impairment observed, and (iii) there might be some relationship to cognitive impairment, although a relationship to standard clinical variables was harder to establish. What also became clear in the literature on bipolar disorder itself was the possible relationship of theory of mind impairment to history of psychotic symptoms. Direct comparative studies, including patients with schizophrenia, were thus examined, and provided several important directions for future research on the bases of impairment in bipolar disorder. Particularly prominent was the issue of whether theory of mind impairment could be considered a candidate endophenotype for the psychoses, although current evidence suggests that this may be premature. The differences in impairment across schizophrenia and bipolar disorder may, however, have genuine differential effects on social functioning and the likely success of remediation.

Perception of emotion-related conflict in human communications: what are the effects of schizophrenia?

Our ability to make sense of emotional cues is of paramount importance for understanding state of mind and communicative intent. However, emotional cues often conflict with each other; this presents a significant challenge for people with schizophrenia. We conducted a theoretical review to determine the extent and types of impaired processing of emotion-related conflict in schizophrenia; we evaluated the relationship with medication and symptoms, and considered possible mediatory mechanisms. The literature established that people with schizophrenia demonstrated impaired function: (i) when passively exposed to emotion cues whilst performing an unrelated task, (ii) when selectively attending to one source of emotion cues whilst trying to ignore interference from another source, and (iii) when trying to resolve conflicting emotion cues and judge meta-communicative intent. These deficits showed associations with both negative and positive symptoms. There was limited evidence for antipsychotic medications attenuating impaired emotion perception when there are conflicting cues, with further direct research needed. Impaired attentional control and context processing may underlie some of the observed impairments. Neuroanatomical correlates are likely to involve interhemispheric transfer via the corpus callosum, limbic regions such as the amygdala, and possibly dorsolateral prefrontal and anterior cingulate cortex through their role in conflict processing.

Type 2 diabetes and memory: using neuroimaging to understand the mechanisms.

The most robust and frequently reported cognitive deficits in type 2 diabetes (DM2) are those that relate to memory. Behavioural research has identified a number of potential contributory physiological factors, including abnormalities in glucose metabolism, such as hyperglycaemia and hypoglycaemia. The impact of these mechanisms on memory has been further investigated through the use of both structural and functional neuroimaging. Structural brain imaging has indicated that memory impairments in DM2 are associated with global atrophy of the brain. Further data suggest that localised atrophy in the hippocampal area, a brain region critical to memory formation and consolidation, may be primarily responsible for the memory deficits seen in this population. Functional imaging data has corroborates these findings, with functional magnetic resonance imaging (fMRI) suggesting reduced connectivity between the hippocampus and surrounding brain regions, particularly the frontal and temporal gyri. Despite this, little functional neuroimaging research has directly investigated differences in regional brain activity between healthy and DM2 participants whilst memory tasks are being performed. By using neuroimaging techniques to their full potential, we can acquire a fuller, more comprehensive picture of the impact that DM2 has on memory.

Age-related decline in emotional prosody discrimination: acoustic correlates.

It is now accepted that older adults have difficulty recognizing prosodic emotion cues, but it is not clear at what processing stage this ability breaks down. We manipulated the acoustic characteristics of tones in pitch, amplitude, and duration discrimination tasks to assess whether impaired basic auditory perception coexisted with our previously demonstrated age-related prosodic emotion perception impairment. It was found that pitch perception was particularly impaired in older adults, and that it displayed the strongest correlation with prosodic emotion discrimination. We conclude that an important cause of age-related impairment in prosodic emotion comprehension exists at the fundamental sensory level of processing.

Attitudinal prosody: what we know and directions for future study.

Prosodic aspects of speech such as pitch, duration and amplitude constitute nonverbal cues that supplement or modify the meaning of the spoken word, to provide valuable clues as to a speakers' state of mind. It can thus indicate what emotion a person is feeling (emotional prosody), or their attitude towards an event, person or object (attitudinal prosody). Whilst the study of emotional prosody has gathered pace, attitudinal prosody now deserves equal attention. In social cognition, understanding attitudinal prosody is important in its own right, since it can convey powerful constructs such as confidence, persuasion, sarcasm and superiority. In this review, it is examined what prosody is, how it conveys attitudes, and which attitudes prosody can convey. The review finishes by considering the neuroanatomy associated with attitudinal prosody, and put forward the hypothesis that this cognition is mediated by the right cerebral hemisphere, particularly posterior superior lateral temporal cortex, with an additional role for the basal ganglia, and limbic regions such as the medial prefrontal cortex and amygdala. It is suggested that further exploration of its functional neuroanatomy is greatly needed, since it could provide valuable clues about the value of current prosody nomenclature and its separability from other types of prosody at the behavioural level.

Further characterisation of the functional neuroanatomy associated with prosodic emotion decoding.

INTRODUCTION: Current models of prosodic emotion comprehension propose a three stage cognition mediated by temporal lobe auditory regions through to inferior and orbitofrontal regions. Cumulative evidence suggests that its mediation may be more flexible though, with a facility to respond in a graded manner based on the need for executive control. The location of this fine-tuning system is unclear, as is its similarity to the cognitive control system. METHODS: In the current study, need for executive control was manipulated in a block-design functional MRI study by systematically altering the proportion of incongruent trials across time, i.e., trials for which participants identified prosodic emotions in the face of conflicting lexico-semantic emotion cues. Resultant Blood Oxygenation Level Dependent contrast data were analysed according to standard procedures using Statistical Parametric Mapping v8 (Ashburner et al., 2009). RESULTS: In the parametric analyses, superior (medial) frontal gyrus activity increased linearly with increased need for executive control. In the separate analyses of each level of incongruity, results suggested that the baseline prosodic emotion comprehension system was sufficient to deal with low proportions of incongruent trials, whereas a more widespread frontal lobe network was required for higher proportions. CONCLUSIONS: These results suggest an executive control system for prosodic emotion comprehension exists which has the capability to recruit superior (medial) frontal gyrus in a graded manner and other frontal regions once demand exceeds a certain threshold. The need to revise current models of prosodic emotion comprehension and add a fourth processing stage are discussed.

Contributions of emotional prosody comprehension deficits to the formation of auditory verbal hallucinations in schizophrenia.

Deficits in emotional processing have been widely described in schizophrenia. Associations of positive symptoms with poor emotional prosody comprehension (EPC) have been reported at the phenomenological, behavioral, and neural levels. This review focuses on the relation between emotional processing deficits and auditory verbal hallucinations (AVH). We explore the possibility that the relation between AVH and EPC in schizophrenia might be mediated by the disruption of a common mechanism intrinsic to auditory processing, and that, moreover, prosodic feature processing deficits play a pivotal role in the formation of AVH. The review concludes with proposing a mechanism by which AVH are constituted and showing how different aspects of our neuropsychological model can explain the constellation of subjective experiences which occur in relation to AVH.

The specificity of age-related decline in interpretation of emotion cues from prosody.

Older adults are not as good as younger adults at decoding prosodic emotions. We sought to determine the specificity of this finding. Performance of older and younger adults was compared on a prosodic emotion task, a "pure" prosodic emotion task, a linguistic prosody task, and a "pure" linguistic prosody task. Older adults were less accurate at interpreting prosodic emotion cues and nonemotional contours, concurrent semantic processing worsened interpretation, and performance was further degraded when identifying negative emotions and questions. Older adults display a pervasive problem interpreting prosodic cues, but further study is required to clarify the stage at which performance declines.

Linear increases in BOLD response associated with increasing proportion of incongruent trials across time in a colour Stroop task.

Selective attention is popularly assessed with colour Stroop tasks in which participants name the ink colour of colour words, whilst resisting interference from the natural tendency to read the words. Prior studies hinted that the key brain regions (dorsolateral prefrontal (dlPFC) and anterior cingulate cortex (ACC)) may vary their degree of involvement, dependent on attentional demand. This study aimed to determine whether a parametrically varied increase in attentional demand resulted in linearly increased activity in these regions, and/or whether additional regions would be recruited during high attentional demand. Twenty-eight healthy young adults underwent fMRI whilst naming the font colour of colour words. Linear increases in BOLD response were assessed with increasing percentage incongruent trials per block (0, 20, 40, 60, 80, and 100%). Whilst ACC activation increased linearly according to incongruity level, dlPFC activity appeared constant. Together with behavioural evidence of reduced Stroop interference, these data support a load-dependent conflict-related response in ACC, but not dlPFC.

Decoding emotional prosody in Parkinson's disease and its potential neuropsychological basis.

Parkinson's disease patients may have difficulty decoding prosodic emotion cues. These data suggest that the basal ganglia are involved, but may reflect dorsolateral prefrontal cortex dysfunction. An auditory emotional n-back task and cognitive n-back task were administered to 33 patients and 33 older adult controls, as were an auditory emotional Stroop task and cognitive Stroop task. No deficit was observed on the emotion decoding tasks; this did not alter with increased frontal lobe load. However, on the cognitive tasks, patients performed worse than older adult controls, suggesting that cognitive deficits may be more prominent. The impact of frontal lobe dysfunction on prosodic emotion cue decoding may only become apparent once frontal lobe pathology rises above a threshold.

fMRI evidence for the effect of verbal complexity on lateralisation of the neural response associated with decoding prosodic emotion.

The pattern of intonation accompanying an utterance provides a powerful cue as to a speaker's emotional state of mind. Most prior lesion studies have demonstrated that the nodal point for decoding these prosodic emotion cues is mediated by unimodal auditory cortex in the right posterior lateral temporal lobe. However, functional neuroimaging has brought with it increasing attention to the equivalent left hemisphere region in this role. This study used fMRI to quantitatively assess the hypothesis that involvement of the left posterior lateral temporal lobe depended on the linguistic load or verbal complexity of the prosodic emotion stimuli. BOLD contrast data was acquired on a 3T scanner whilst 16 healthy young adults identified the prosodic emotion in three conditions: 'sentences' comprised of words, a repeated monosyllable, and a single prolonged syllable (asyllabic). Whole-brain analyses were performed using SPM5 and supplemented by posterior lateral temporal lobe region of interest (ROI) analyses. The whole-brain analyses appeared to show bilateral temporal lobe activation across the conditions, however, the ROI analyses indicated a highly significant decrease in activity in the left ROI as verbal complexity decreased. Changes in right ROI activity were not statistically significant. Our results indicate that the likelihood of observing a notable left temporal lobe response in functional neuroimaging studies of emotional prosody comprehension depends on the verbal complexity of the prosodic emotion stimuli. Despite the right hemisphere dominance underlying this task, the left hemisphere region may be co-activated in its attempt to extract phonetic-segmental information from the acoustic stimuli whether or not the stimuli contain meaningful phonetic-segmental information.

fMRI delineation of working memory for emotional prosody in the brain: commonalities with the lexico-semantic emotion network.

Decoding emotional prosody is crucial for successful social interactions, and continuous monitoring of emotional intent via prosody requires working memory. It has been proposed by Ross and others that emotional prosody cognitions in the right hemisphere are organized in an analogous fashion to propositional language functions in the left hemisphere. This study aimed to test the applicability of this model in the context of prefrontal cortex working memory functions. BOLD response data were therefore collected during performance of two emotional working memory tasks by participants undergoing fMRI. In the prosody task, participants identified the emotion conveyed in pre-recorded sentences, and working memory load was manipulated in the style of an N-back task. In the matched lexico-semantic task, participants identified the emotion conveyed by sentence content. Block-design neuroimaging data were analyzed parametrically with SPM5. At first, working memory for emotional prosody appeared to be right-lateralized in the PFC, however, further analyses revealed that it shared much bilateral prefrontal functional neuroanatomy with working memory for lexico-semantic emotion. Supplementary separate analyses of males and females suggested that these language functions were less bilateral in females, but their inclusion did not alter the direction of laterality. It is concluded that Ross et al.'s model is not applicable to prefrontal cortex working memory functions, that evidence that working memory cannot be subdivided in prefrontal cortex according to material type is increased, and that incidental working memory demands may explain the frontal lobe involvement in emotional prosody comprehension as revealed by neuroimaging studies.

The psychological, neurochemical and functional neuroanatomical mediators of the effects of positive and negative mood on executive functions.

In this review we evaluate the cognitive and neural effects of positive and negative mood on executive function. Mild manipulations of negative mood appear to have little effect on cognitive control processes, whereas positive mood impairs aspects of updating, planning and switching. These cognitive effects may be linked to neurochemistry: with positive mood effects mediated by dopamine while negative mood effects may be mediated by serotonin levels. Current evidence on the effects of mood on regional brain activity during executive functions, indicates that the prefrontal cortex is a recurrent site of integration between mood and cognition. We conclude that there is a disparity between the importance of this topic and awareness of how mood affects, executive functions in the brain. Most behavioural and neuroimaging studies of executive function in normal samples do not explore the potential role of variations in mood, yet the evidence we outline indicates that even mild fluctuations in mood can have a significant influence on neural activation and cognition.

Anterior cingulate activity and level of cognitive conflict: explicit comparisons.

The role of anterior cingulate cortex (ACC) in attention is a matter of debate. One hypothesis suggests that its role is to monitor response-level conflict, but explicit evidence is somewhat lacking. In this study, the activation of ACC was compared in (a) color and number standard Stroop tasks in which response preparation and interference shared modality (response-level conflict) and (b) color and number matching Stroop tasks in which response preparation and interference did not share modality (non-response-level conflict). In the congruent conditions, there was no effect of task type. In the interference conditions, anterior cingulate activity in the matching tasks was less than that in the standard tasks. These results support the hypothesis that ACC specifically mediates generalized modality-independent selection processes invoked by response competition.