Covert digital manipulation of vocal emotion alter speakers' emotional states in a congruent direction.

Research has shown that people often exert control over their emotions. By modulating expressions, reappraising feelings, and redirecting attention, they can regulate their emotional experience. These findings have contributed to a blurring of the traditional boundaries between cognitive and emotional processes, and it has been suggested that emotional signals are produced in a goal-directed way and monitored for errors like other intentional actions. However, this interesting possibility has never been experimentally tested. To this end, we created a digital audio platform to covertly modify the emotional tone of participants' voices while they talked in the direction of happiness, sadness, or fear. The result showed that the audio transformations were being perceived as natural examples of the intended emotions, but the great majority of the participants, nevertheless, remained unaware that their own voices were being manipulated. This finding indicates that people are not continuously monitoring their own voice to make sure that it meets a predetermined emotional target. Instead, as a consequence of listening to their altered voices, the emotional state of the participants changed in congruence with the emotion portrayed, which was measured by both self-report and skin conductance level. This change is the first evidence, to our knowledge, of peripheral feedback effects on emotional experience in the auditory domain. As such, our result reinforces the wider framework of self-perception theory: that we often use the same inferential strategies to understand ourselves as those that we use to understand others.

Voice emotion recognition by Mandarin-speaking pediatric cochlear implant users in Taiwan.

OBJECTIVES: To explore the effects of obligatory lexical tone learning on speech emotion recognition and the cross-culture differences between United States and Taiwan for speech emotion understanding in children with cochlear implant. METHODS: This cohort study enrolled 60 cochlear-implanted (cCI) Mandarin-speaking, school-aged children who underwent cochlear implantation before 5 years of age and 53 normal-hearing children (cNH) in Taiwan. The emotion recognition and the sensitivity of fundamental frequency (F0) changes for those school-aged cNH and cCI (6-17 years old) were examined in a tertiary referred center. RESULTS: The mean emotion recognition score of the cNH group was significantly better than the cCI. Female speakers' vocal emotions are more easily to be recognized than male speakers' emotion. There was a significant effect of age at test on voice recognition performance. The average score of cCI with full-spectrum speech was close to the average score of cNH with eight-channel narrowband vocoder speech. The average performance of voice emotion recognition across speakers for cCI could be predicted by their sensitivity to changes in F0. CONCLUSIONS: Better pitch discrimination ability comes with better voice emotion recognition for Mandarin-speaking cCI. Besides the F0 cues, cCI are likely to adapt their voice emotion recognition by relying more on secondary cues such as intensity and duration. Although cross-culture differences exist for the acoustic features of voice emotion, Mandarin-speaking cCI and their English-speaking cCI peer expressed a positive effect for age at test on emotion recognition, suggesting the learning effect and brain plasticity. Therefore, further device/processor development to improve presentation of pitch information and more rehabilitative efforts are needed to improve the transmission and perception of voice emotion in Mandarin. LEVEL OF EVIDENCE: 3.

Voice emotion perception and production in cochlear implant users.

Voice emotion is a fundamental component of human social interaction and social development. Unfortunately, cochlear implant users are often forced to interface with highly degraded prosodic cues as a result of device constraints in extraction, processing, and transmission. As such, individuals with cochlear implants frequently demonstrate significant difficulty in recognizing voice emotions in comparison to their normal hearing counterparts. Cochlear implant-mediated perception and production of voice emotion is an important but relatively understudied area of research. However, a rich understanding of the voice emotion auditory processing offers opportunities to improve upon CI biomedical design and to develop training programs benefiting CI performance. In this review, we will address the issues, current literature, and future directions for improved voice emotion processing in cochlear implant users.

Mental status assessment of disaster relief personnel by vocal affect display based on voice emotion recognition.

BACKGROUND: Disaster relief personnel tend to be exposed to excessive stress, which can be a cause of mental disorders. To prevent from mental disorders, frequent assessment of mental status is important. This pilot study aimed to examine feasibility of stress assessment using vocal affect display (VAD) indices as calculated by our proposed algorithms in a situation of comparison between different durations of stay in stricken area as disaster relief operation, which is an environment highly likely to induce stress. METHODS: We used Sensibility Technology (ST) software to analyze VAD from voices of participants exposed to extreme stress for either long or short durations, and we proposed algorithms for indices of low VAD (VAD-L), high VAD (VAD-H), and VAD ratio (VAD-R), calculated from the intensity of emotions as measured by voice emotion analysis. As a preliminary validation, 12 members of Japan Self-Defense Forces dispatched overseas for long (3 months or more) or short (about a week) durations were asked to record their voices saying 11 phrases repeatedly across 6 days during their dispatch. RESULTS: In the validation, the two groups showed an inverse relationship in VAD-L and VAD-H, in that long durations in disaster zones resulted in higher values of both VAD-L and VAD-R, and lower values of VAD-H, compared with short durations. Interestingly, phrases produced varied results in terms of group differences and VAD indices, demonstrating the sensitivity of the ST. CONCLUSIONS: A comparison of the values obtained for the different groups of subjects clarified that there were tendencies of the VAD-L, VAD-H, and VAD-R indices observed for each group of participants. The results suggest the possibility of using ST software in the measurement of affective aspects related to mental health from vocal behavior.