Affective Prosody and Its Impact on the Neurology of Language, Depression, Memory and Emotions.

Based on the seminal publications of Paul Broca and Carl Wernicke who established that aphasic syndromes (disorders of the verbal-linguistic aspects of communication) were predominantly the result of focal left-hemisphere lesions, "language" is traditionally viewed as a lateralized function of the left hemisphere. This, in turn, has diminished and delayed the acceptance that the right hemisphere also has a vital role in language, specifically in modulating affective prosody, which is essential for communication competency and psychosocial well-being. Focal lesions of the right hemisphere may result in disorders of affective prosody (aprosodic syndromes) that are functionally and anatomically analogous to the aphasic syndromes that occur following focal left-hemisphere lesions. This paper will review the deductive research published over the last four decades that has elucidated the neurology of affective prosody which, in turn, has led to a more complete and nuanced understanding of the neurology of language, depression, emotions and memory. In addition, the paper will also present the serendipitous clinical observations (inductive research) and fortuitous inter-disciplinary collaborations that were crucial in guiding and developing the deductive research processes that culminated in the concept that primary emotions and related display behaviors are a lateralized function of the right hemisphere and social emotions, and related display behaviors are a lateralized function of the left hemisphere.

Differential Hemispheric Lateralization of Emotions and Related Display Behaviors: Emotion-Type Hypothesis.

There are two well-known hypotheses regarding hemispheric lateralization of emotions. The Right Hemisphere Hypothesis (RHH) postulates that emotions and associated display behaviors are a dominant and lateralized function of the right hemisphere. The Valence Hypothesis (VH) posits that negative emotions and related display behaviors are modulated by the right hemisphere and positive emotions and related display behaviors are modulated by the left hemisphere. Although both the RHH and VH are supported by extensive research data, they are mutually exclusive, suggesting that there may be a missing factor in play that may provide a more accurate description of how emotions are lateralization in the brain. Evidence will be presented that provides a much broader perspective of emotions by embracing the concept that emotions can be classified into primary and social types and that hemispheric lateralization is better explained by the Emotion-type Hypothesis (ETH). The ETH posits that primary emotions and related display behaviors are modulated by the right hemisphere and social emotions and related display behaviors are modulated by the left hemisphere.

Disorders of vocal emotional expression and comprehension: The aprosodias.

Language is traditionally considered to be a dominant function of the left hemisphere because of its role in modulating the propositional or literal aspects of language (what is said). This chapter, however, will address the vital role of the right hemisphere in modulating the nonverbal, affective-prosodic aspects of speech (how it is said) that are essential for appropriate interpersonal interactions, communication competency, and psychosocial well-being. Focal right hemisphere lesions cause various disorders of affective prosody (aprosodic syndromes) that are analogous to the various aphasic syndromes that occur following focal left hemisphere lesions. Disorders of affective prosody may present clinically as loss of psychosocial well-being with disruption of interpersonal relationships. This chapter will review the research published over the last four decades that has helped to elucidate the neurobiology of affective prosody and the pathophysiology underlying the aprosodic syndromes.

Interpersonal prosodic correlation in frontotemporal dementia.

Communication accommodation describes how individuals adjust their communicative style to that of their conversational partner. We predicted that interpersonal prosodic correlation related to pitch and timing would be decreased in behavioral variant frontotemporal dementia (bvFTD). We predicted that the interpersonal correlation in a timing measure and a pitch measure would be increased in right temporal FTD (rtFTD) due to sparing of the neural substrate for speech timing and pitch modulation but loss of social semantics. We found no significant effects in bvFTD, but conversations including rtFTD demonstrated higher interpersonal correlations in speech rate than healthy controls.

Aprosodia and prosoplegia with right frontal neurodegeneration.

Affective prosody and facial expression are essential components of human communication. Aprosodic syndromes are associated with focal right cerebral lesions that impair the affective-prosodic aspects of language, but are rarely identified because affective prosody is not routinely assessed by clinicians. Inability to produce emotional faces (affective prosoplegia) is a related and important aspect of affective communication has overlapping neuroanatomic substrates with affective prosody. We describe a patient with progressive aprosodia and prosoplegia who had right greater than left perisylvian and temporal atrophy with an anterior predominance. We discuss the importance of assessing affective prosody and facial expression to arrive at an accurate clinical diagnosis.

Neurophysiology of spontaneous facial expressions: II. Motor control of the right and left face is partially independent in adults.

Facial expressions are described traditionally as monolithic or unitary entities. However, humans have the capacity to produce facial blends of emotion in which the upper and lower face simultaneously display different expressions. Recent neuroanatomical studies in monkeys have demonstrated that there are separate cortical motor areas for controlling the upper and lower face in each hemisphere that, presumably, also occur in humans. Using high-speed videography, we began measuring the movement dynamics of spontaneous facial expressions, including facial blends, to develop a more complete understanding of the neurophysiology underlying facial expressions. In our part 1 publication in Cortex (2016), we found that hemispheric motor control of the upper and lower face is overwhelmingly independent; 242 (99%) of the expressions were classified as demonstrating independent hemispheric motor control whereas only 3 (1%) were classified as demonstrating dependent hemispheric motor control. In this companion paper (part 2), 251 unitary facial expressions that occurred on either the upper or lower face were analyzed. 164 (65%) expressions demonstrated dependent hemispheric motor control whereas 87 (35%) expressions demonstrated independent or dual hemispheric motor control, indicating that some expressions represent facial blends of emotion that occur across the vertical facial axis. These findings also support the concepts that 1) spontaneous facial expressions are organized predominantly across the horizontal facial axis and secondarily across the vertical facial axis and 2) facial expressions are complex, multi-component, motoric events. Based on the Emotion-type hypothesis of cerebral lateralization, we propose that facial expressions modulated by a primary-emotional response to an environmental event are initiated by the right hemisphere on the left side of the face whereas facial expressions modulated by a social-emotional response to an environmental event are initiated by the left hemisphere on the right side of the face.

Altered saccadic targets when processing facial expressions under different attentional and stimulus conditions.

Depending on a subject's attentional bias, robust changes in emotional perception occur when facial blends (different emotions expressed on upper/lower face) are presented tachistoscopically. If no instructions are given, subjects overwhelmingly identify the lower facial expression when blends are presented to either visual field. If asked to attend to the upper face, subjects overwhelmingly identify the upper facial expression in the left visual field but remain slightly biased to the lower facial expression in the right visual field. The current investigation sought to determine whether differences in initial saccadic targets could help explain the perceptual biases described above. Ten subjects were presented with full and blend facial expressions under different attentional conditions. No saccadic differences were found for left versus right visual field presentations or for full facial versus blend stimuli. When asked to identify the presented emotion, saccades were directed to the lower face. When asked to attend to the upper face, saccades were directed to the upper face. When asked to attend to the upper face and try to identify the emotion, saccades were directed to the upper face but to a lesser degree. Thus, saccadic behavior supports the concept that there are cognitive-attentional pre-attunements when subjects visually process facial expressions. However, these pre-attunements do not fully explain the perceptual superiority of the left visual field for identifying the upper facial expression when facial blends are presented tachistoscopically. Hence other perceptual factors must be in play, such as the phenomenon of virtual scanning.

Neurophysiology of spontaneous facial expressions: I. Motor control of the upper and lower face is behaviorally independent in adults.

Facial expressions are described traditionally as monolithic entities. However, humans have the capacity to produce facial blends, in which the upper and lower face simultaneously display different emotional expressions. This, in turn, has led to the Component Theory of facial expressions. Recent neuroanatomical studies in monkeys have demonstrated that there are separate cortical motor areas for controlling the upper and lower face that, presumably, also occur in humans. The lower face is represented on the posterior ventrolateral surface of the frontal lobes in the primary motor and premotor cortices and the upper face is represented on the medial surface of the posterior frontal lobes in the supplementary motor and anterior cingulate cortices. Our laboratory has been engaged in a series of studies exploring the perception and production of facial blends. Using high-speed videography, we began measuring the temporal aspects of facial expressions to develop a more complete understanding of the neurophysiology underlying facial expressions and facial blends. The goal of the research presented here was to determine if spontaneous facial expressions in adults are predominantly monolithic or exhibit independent motor control of the upper and lower face. We found that spontaneous facial expressions are very complex and that the motor control of the upper and lower face is overwhelmingly independent, thus robustly supporting the Component Theory of facial expressions. Seemingly monolithic expressions, be they full facial or facial blends, are most likely the result of a timing coincident rather than a synchronous coordination between the ventrolateral and medial cortical motor areas responsible for controlling the lower and upper face, respectively. In addition, we found evidence that the right and left face may also exhibit independent motor control, thus supporting the concept that spontaneous facial expressions are organized predominantly across the horizontal facial axis and secondarily across the vertical axis.

Postural instability in Parkinson Disease: to step or not to step.

Postural instability is a key feature of Parkinson Disease that is associated with falls and morbidity. We designed a pull apparatus to quantitatively measure the force needed to pull subjects off-balance. Thirteen Controls and eight individuals with Parkinson Disease (PD) were evaluated. All individuals with PD reported subjective symptoms of postural instability and were symptomatic for approximately 9.4years when tested. No significant differences were found between Controls and PD subjects in the magnitude of force required to pull them off-balance. None of the Controls fell and all took a step into the direction of pull to maintain their balance. 59% of the time PD subjects fell because they did not take a step in the direction of pull to maintain their center of mass (COM) over their feet, thus indicating a deficiency in postural reflexes. If they fell on the first pull, PD subjects did not show a learning effect when pulled multiple times in the same direction. The utility of the Pull Test to detect postural instability is related to the subject's behavioral response, not the force needed to pull them off balance. Our findings may also help explain certain features of the PD gait as an attempt by subjects to avoid postural instability by not placing their COM in gravitationally unstable positions.

Decoding facial blends of emotion: visual field, attentional and hemispheric biases.

Most clinical research assumes that modulation of facial expressions is lateralized predominantly across the right-left hemiface. However, social psychological research suggests that facial expressions are organized predominantly across the upper-lower face. Because humans learn to cognitively control facial expression for social purposes, the lower face may display a false emotion, typically a smile, to enable approach behavior. In contrast, the upper face may leak a person's true feeling state by producing a brief facial blend of emotion, i.e. a different emotion on the upper versus lower face. Previous studies from our laboratory have shown that upper facial emotions are processed preferentially by the right hemisphere under conditions of directed attention if facial blends of emotion are presented tachistoscopically to the mid left and right visual fields. This paper explores how facial blends are processed within the four visual quadrants. The results, combined with our previous research, demonstrate that lower more so than upper facial emotions are perceived best when presented to the viewer's left and right visual fields just above the horizontal axis. Upper facial emotions are perceived best when presented to the viewer's left visual field just above the horizontal axis under conditions of directed attention. Thus, by gazing at a person's left ear, which also avoids the social stigma of eye-to-eye contact, one's ability to decode facial expressions should be enhanced.

Attitudinal prosody: what we know and directions for future study.

Prosodic aspects of speech such as pitch, duration and amplitude constitute nonverbal cues that supplement or modify the meaning of the spoken word, to provide valuable clues as to a speakers' state of mind. It can thus indicate what emotion a person is feeling (emotional prosody), or their attitude towards an event, person or object (attitudinal prosody). Whilst the study of emotional prosody has gathered pace, attitudinal prosody now deserves equal attention. In social cognition, understanding attitudinal prosody is important in its own right, since it can convey powerful constructs such as confidence, persuasion, sarcasm and superiority. In this review, it is examined what prosody is, how it conveys attitudes, and which attitudes prosody can convey. The review finishes by considering the neuroanatomy associated with attitudinal prosody, and put forward the hypothesis that this cognition is mediated by the right cerebral hemisphere, particularly posterior superior lateral temporal cortex, with an additional role for the basal ganglia, and limbic regions such as the medial prefrontal cortex and amygdala. It is suggested that further exploration of its functional neuroanatomy is greatly needed, since it could provide valuable clues about the value of current prosody nomenclature and its separability from other types of prosody at the behavioural level.

Posed versus spontaneous facial expressions are modulated by opposite cerebral hemispheres.

Clinical research has indicated that the left face is more expressive than the right face, suggesting that modulation of facial expressions is lateralized to the right hemisphere. The findings, however, are controversial because the results explain, on average, approximately 4% of the data variance. Using high-speed videography, we sought to determine if movement-onset asymmetry was a more powerful research paradigm than terminal movement asymmetry. The results were very robust, explaining up to 70% of the data variance. Posed expressions began overwhelmingly on the right face whereas spontaneous expressions began overwhelmingly on the left face. This dichotomy was most robust for upper facial expressions. In addition, movement-onset asymmetries did not predict terminal movement asymmetries, which were not significantly lateralized. The results support recent neuroanatomic observations that upper versus lower facial movements have different forebrain motor representations and recent behavioral constructs that posed versus spontaneous facial expressions are modulated preferentially by opposite cerebral hemispheres and that spontaneous facial expressions are graded rather than non-graded movements.

Affective prosody: what do comprehension errors tell us about hemispheric lateralization of emotions, sex and aging effects, and the role of cognitive appraisal.

The Aprosodia Battery was developed to distinguish different patterns of affective-prosodic deficits in patients with left versus right brain damage by using affective utterances with incrementally reduced verbal-articulatory demands. It has also been used to assess affective-prosodic performance in various clinical groups, including patients with schizophrenia, PTSD, multiple sclerosis, alcohol abuse and Alzheimer disease and in healthy adults, as means to explore maturational-aging effects. To date, all studies using the Aprosodia Battery have yielded statistically robust results. This paper describes an extensive, quantitative error analysis using previous results from the Aprosodia Battery in patients with left and right brain damage, age-equivalent controls (old adults), and a group of young adults. This inductive analysis was performed to address three major issues in the literature: (1) sex and (2) maturational-aging effects in comprehending affective prosody and (3) differential hemispheric lateralization of emotions. We found no overall sex effects for comprehension of affective prosody. There were, however, scattered sex effects related to a particular affect, suggesting that these differences were related to cognitive appraisal rather than primary perception. Results in the brain damaged groups did not support the Valence Hypothesis of emotional lateralization but did support the Right Hemisphere Hypothesis of emotional lateralization. When comparing young versus old adults, a robust maturational-aging effect was observed in overall error rates and in the distribution of errors across affects. This effect appears to be mediated, in part, by cognitive appraisal, causing an alteration in the salience of different affective-prosodic stimuli with increasing age. In addition, the maturational-aging effects lend support for the Emotion-Type hypothesis of emotional lateralization and the "classic aging effect" that is due primarily to decline of right hemisphere cognitive functions in senescence. The results of our inductive analysis may help direct future deductive research efforts, exploring the neuropsychology of emotional communication, by taking into account the potentially confounding influence of (1) methodological differences involving construction of test stimuli and assessment procedures, (2) developmental, maturational and aging effects related to cognitive appraisal and (3) whether a stimulus has a primary or social-emotional bias.

Cerebral localization of functions and the neurology of language: fact versus fiction or is it something else?

Over the last 15 years there has been a burgeoning number of publications using functional brain imaging (>40,000 articles based on an ISI/Web of Science search) to localize behavioral and cognitive processes to specific areas in the human brain that are often not confirmed by traditional, lesion-based studies. Thus, there is a need to reassess what cerebral localization of functions is and is not. Otherwise, there is no rational way to interpret the escalating claims of localization in the functional imaging literature that is taking on the appearance of neurophysiologic "phrenology". This article will present arguments to suggest that functional localization in the brain is a robust but very dynamic, four-dimensional process. It is a learned phenomenon driven over time by large-scale, spatially distributed, neural networks seeking to efficiently maximize the processing, storage, and manipulation of information for cognitive and behavioral operations. Because of historical considerations and space limitations, the main focus will be on localization of language-related functions whose theoretical neurological basis can be generalized for any complex cognitive-behavioral function.

Deficits in affective prosody comprehension: family history of alcoholism versus alcohol exposure.

BACKGROUND: Abstinent alcoholics have deficits in comprehending the affective intonation in speech. Prior work suggests that these deficits are due to alcohol exposure rather than preexisting risk factors for alcoholism. The present paper examines whether family history of alcoholism is a contributor to affective prosody deficits in alcoholics. METHODS: Fifty-eight healthy, nonabusing young adults with and without a family history of alcoholism or other substance abuse (29 FH+ and 29 FH-) were compared on affective prosody comprehension using the Aprosodia Battery. A secondary analysis was done comparing affective prosody comprehension in FH+ and FH- detoxified alcoholics from an earlier study (17 FH+ and 14 FH-). RESULTS: Performance on the Aprosodia Battery was not related to FH status in either the healthy, nonabusing sample or in the detoxified alcoholic group. CONCLUSIONS: The present study lends support to previous research suggesting that deficits in affective prosody comprehension observed in detoxified alcoholics are associated with a history of heavy drinking rather than with a family history of alcoholism.

Differences in coated-platelet production between frontotemporal dementia and Alzheimer disease.

OBJECTIVE: Coated-platelets are a subset of platelets produced by dual-agonist activation with collagen and thrombin. These platelets retain full-length amyloid precursor protein on their surface and correlate inversely with disease severity in Alzheimer disease (AD). We have now investigated coated-platelet production and its relationship with disease severity in frontotemporal dementia (FTD) patients to determine whether our earlier observations were unique to AD. METHODS: Coated-platelet levels were assayed in 40 FTD, 40 AD patients, and 40 controls. Both patient groups were equally divided between mild-stage (Clinical Dementia Rating < or =1) and advanced stage dementia (Clinical Dementia Rating >1). RESULTS: Coated-platelet levels were not significantly different between patients with early-stage and advanced stage FTD (P=0.9), whereas early-stage AD patients had significantly higher levels than advanced stage AD (P<0.001). In addition, coated-platelet production was significantly elevated in early-stage AD versus early-stage FTD patients (P=0.01). CONCLUSIONS: In contrast to AD, there is no significant relationship between disease severity and coated-platelet levels in FTD. Differences in coated-platelet levels between early-stage AD and early-stage FTD patients warrant further investigation for potential clinical applications in helping to differentiate between these 2 disorders.

Comprehension of affective prosody in veterans with chronic posttraumatic stress disorder.

Posttraumatic stress disorder (PTSD) is one of the few psychiatric conditions in which a subjective decrease in emotional range serves as a diagnostic criterion. In order to investigate whether veterans with chronic PTSD also experienced objective limitations in emotional perception, the authors administered the Aprosodia Battery to a group of 11 veterans with chronic PTSD, nine subjects with right hemisphere damage, seven subjects with left hemisphere damage, and 12 comparison subjects. The patients with PTSD displayed significant deficiencies in the comprehension and discriminative components of affective speech, similar in severity and performance profile on the Aprosodia Battery to the individuals with focal right hemisphere damage due to ischemic infarction.

Rate of progression in Alzheimer's disease correlates with coated-platelet levels--a longitudinal study.

Coated-platelets represent a subset of platelets produced by activation with both collagen and thrombin that retain full-length amyloid precursor protein on their surface. In our initial cross-sectional study, coated-platelet levels correlated inversely with disease severity in Alzheimer's disease (AD). Higher levels were observed in the early stage compared with the advanced stage of the disease. In this longitudinal study, we investigated for the first time the relationship between initial coated-platelet levels and disease progression in individuals with AD. Coated-platelet levels were assayed in 25 patients with AD who where then clinically monitored for 2 years. A significant linear correlation (r = 0.47, P = 0.017) was detected between the initial coated-platelet levels and disease progression measured in the Mini-Mental State Examination score. The most severe decline was noted in individuals with the highest initial coated-platelet production. These findings support our previous observations from cross-sectional studies and suggest the need for additional study of coated-platelets as a link to the sequence of events leading to the development of AD.

fMRI evidence for the effect of verbal complexity on lateralisation of the neural response associated with decoding prosodic emotion.

The pattern of intonation accompanying an utterance provides a powerful cue as to a speaker's emotional state of mind. Most prior lesion studies have demonstrated that the nodal point for decoding these prosodic emotion cues is mediated by unimodal auditory cortex in the right posterior lateral temporal lobe. However, functional neuroimaging has brought with it increasing attention to the equivalent left hemisphere region in this role. This study used fMRI to quantitatively assess the hypothesis that involvement of the left posterior lateral temporal lobe depended on the linguistic load or verbal complexity of the prosodic emotion stimuli. BOLD contrast data was acquired on a 3T scanner whilst 16 healthy young adults identified the prosodic emotion in three conditions: 'sentences' comprised of words, a repeated monosyllable, and a single prolonged syllable (asyllabic). Whole-brain analyses were performed using SPM5 and supplemented by posterior lateral temporal lobe region of interest (ROI) analyses. The whole-brain analyses appeared to show bilateral temporal lobe activation across the conditions, however, the ROI analyses indicated a highly significant decrease in activity in the left ROI as verbal complexity decreased. Changes in right ROI activity were not statistically significant. Our results indicate that the likelihood of observing a notable left temporal lobe response in functional neuroimaging studies of emotional prosody comprehension depends on the verbal complexity of the prosodic emotion stimuli. Despite the right hemisphere dominance underlying this task, the left hemisphere region may be co-activated in its attempt to extract phonetic-segmental information from the acoustic stimuli whether or not the stimuli contain meaningful phonetic-segmental information.