Improving emotion perception in cochlear implant users: insights from machine learning analysis of EEG signals.

BACKGROUND: Although cochlear implants can restore auditory inputs to deafferented auditory cortices, the quality of the sound signal transmitted to the brain is severely degraded, limiting functional outcomes in terms of speech perception and emotion perception. The latter deficit negatively impacts cochlear implant users' social integration and quality of life; however, emotion perception is not currently part of rehabilitation. Developing rehabilitation programs incorporating emotional cognition requires a deeper understanding of cochlear implant users' residual emotion perception abilities. METHODS: To identify the neural underpinnings of these residual abilities, we investigated whether machine learning techniques could be used to identify emotion-specific patterns of neural activity in cochlear implant users. Using existing electroencephalography data from 22 cochlear implant users, we employed a random forest classifier to establish if we could model and subsequently predict from participants' brain responses the auditory emotions (vocal and musical) presented to them. RESULTS: Our findings suggest that consistent emotion-specific biomarkers exist in cochlear implant users, which could be used to develop effective rehabilitation programs incorporating emotion perception training. CONCLUSIONS: This study highlights the potential of machine learning techniques to improve outcomes for cochlear implant users, particularly in terms of emotion perception.

An influence of menopausal symptoms on mental health, emotion perception, and quality of life: a multi-faceted approach.

PURPOSE: The menopausal transition brings with it many physical, cognitive, and affective changes in a woman's life, impacting quality of life. Whereas prior work has examined impact on general mental health and cognitive function, research on basic affective processing during menopause remains scarce. METHODS: Using a median-split procedure, this pre-registered study examined the impact of stronger (N = 46 women) vs. milder (N = 47 women) menopausal symptoms using a behavioural task of subjective emotion perception (embody) and a passive eye tracking viewing task of emotional faces in addition to self-report questionnaires. After 3 months, participants completed the questionnaires again to examine whether objective measures of emotion perception (eye tracking) might predict mental health at follow-up. RESULTS: As anticipated, women with stronger vs. milder menopausal symptoms reported increased symptoms of anxiety, depression, stress, emotion regulation difficulties, and lower quality of life during both time points. While no evidence was found in the behavioural task, eye tracking data indicated blunted emotion perception in women with high menopausal symptoms, while women with low symptoms spent more time looking at happy faces relative to fearful or surprised faces. Although eye tracking or hormonal data did not predict mental health at follow-up, a higher estradiol/FSH ratio indicated a higher quality of life. CONCLUSIONS: This study documented an impact of the menopausal transition and strength of menopausal symptoms in particular on objective emotion perception as well as mental health and quality of life in women suffering from stronger vs. milder menopausal symptoms. Clinical implications are discussed.

Dexamphetamine increased speech and visual unimodal illusions in healthy participants without affecting temporal binding window.

OBJECTIVE: Stimuli received beyond a very short timeframe, known as temporal binding windows (TBWs), are perceived as separate events. In previous audio-visual multisensory integration (McGurk effect) studies, widening of TBWs has been observed in people with schizophrenia. The present study aimed to determine if dexamphetamine could increase TBWs in unimodal auditory and unimodal visual illusions that may have some validity as experimental models for auditory and visual hallucinations in psychotic disorders. METHODS: A double-blind, placebo-controlled, counter-balanced crossover design with permuted block randomisation for drug order was followed. Dexamphetamine (0.45 mg/kg, PO, q.d.) was administered to healthy participants. Phantom word illusion (speech illusion) and visual-induced flash illusion/VIFI (visual illusion) tests were measured to determine if TBWs were altered as a function of delay between stimuli presentations. Word emotional content for phantom word illusions was also analysed. RESULTS: Dexamphetamine significantly increased the total number of phantom words/speech illusions (p < 0.01) for pooled 220-1100 ms ISIs in kernel density estimation and the number of positive valence words heard (beta = 2.20, 95% CI [1.86, 2.55], t = 12.46, p < 0.001) with a large effect size (std. beta = 1.05, 95% CI [0.89, 1.22]) relative to placebo without affecting the TBWs. For the VIFI test, kernel density estimation for pooled 0-801 ms ISIs showed a significant difference (p < 0.01) in the data distributions of number of target flash (es) perceived by participants after receiving dexamphetamine as compared with placebo. CONCLUSIONS: Overall, healthy participants who were administered dexamphetamine (0.45 mg/kg, PO, q.d.) experienced increases in auditory and visual illusions in both phantom word illusion and VIFI tests without affecting their TBWs.

Perceiving the outlier in the crowd: The influence of facial identity.

The accurate perception of groups with outliers can help us identify potential risks. However, it is unclear how outliers affect the perception of group emotion. To address this question, we conducted a study on group emotion perception in the context of facial identity. We presented 74 participants with pictures of crowds, and asked them to evaluate the valence ratios and intensity of the crowd by means of the Emotional Aperture Measure. The results revealed that outlier emotions were often overestimated within crowds. Moreover, we found that the emotional expression of a close friend modulated the perception of outliers. Specifically, when a close friend expressed the group emotion, participants overestimated the outlier less than when a close friend expressed the outlier emotion. These results suggest that people can detect outliers within groups, and that their perception of group emotion is influenced by close friends. Thus, we provide evidence that facial identity affects group emotion perception.

Emotion in motion: perceiving fear in the behaviour of individuals from minimal motion capture displays.

The ability to quickly and accurately recognise emotional states is adaptive for numerous social functions. Although body movements are a potentially crucial cue for inferring emotions, few studies have studied the perception of body movements made in naturalistic emotional states. The current research focuses on the use of body movement information in the perception of fear expressed by targets in a virtual heights paradigm. Across three studies, participants made judgments about the emotional states of others based on motion-capture body movement recordings of those individuals actively engaged in walking a virtual plank at ground-level or 80 stories above a city street. Results indicated that participants were reliably able to differentiate between height and non-height conditions (Studies 1 & 2), were more likely to spontaneously describe target behaviour in the height condition as fearful (Study 2) and their fear estimates were highly calibrated with the fear ratings from the targets (Studies 1-3). Findings show that VR height scenarios can induce fearful behaviour and that people can perceive fear in minimal representations of body movement.

Semantic effects on the perception of emotional prosody in native and non-native Chinese speakers.

While previous research has found an in-group advantage (IGA) favouring native speakers in emotional prosody perception over non-native speakers, the effects of semantics on emotional prosody perception remain unclear. This study investigated the effects of semantics on emotional prosody perception in Chinese words and sentences for native and non-native Chinese speakers. The critical manipulation was the congruence of prosodic (positive, negative) and semantic (positive, negative, and neutral) valence. Participants listened to a series of audio clips and judged whether the emotional prosody was positive or negative for each utterance. The results revealed an IGA effect: native speakers perceived emotional prosody more accurately and quickly than non-native speakers in Chinese words and sentences. Furthermore, a semantic congruence effect was observed in Chinese words, where both native and non-native speakers recognised emotional prosody more accurately in the semantic-prosody congruent condition than in the incongruent condition. However, in Chinese sentences, this congruence effect was only present for non-native speakers. Additionally, the IGA effect and semantic congruence effect on emotional prosody perception were influenced by prosody valence. These findings illuminate the role of semantics in emotional prosody perception, highlighting perceptual differences between native and non-native Chinese speakers.

Interoceptive sensitivity and perception of others' emotions: an investigation based on a two-stage model.

Recent research shows that sensitivity to interoceptive sensitivity is associated with a more granular experience of emotions. These studies suggest that individuals sensitive to their interoceptive signals can better perceive somatic physiological changes as compared to their counterparts. Therefore, they discriminate among a wide and subtle range of emotions. Further, the perception of others' emotions could be based on our own emotional experiences. However, whether interoceptive sensitivity is related to the perception of others' emotions remains unclear. Therefore, this study examined the relationship between interoceptive sensitivity and emotional perception. Considering the model that emotion perception comprises two processes, categorization of facial expressions and approach-avoidance responses, this study examined both categorizations of facial expressions and approach-avoidance responses. The results showed no relationship between interoceptive sensitivity and the perception of emotion, which suggests that interoceptive sensitivity is related to the experience of emotion but does not affect the granularity of emotional perception. Future studies should diversely and empirically examine the role of the body in emotional perception from the perspective of interoceptive sensitivity.

Recognition and perception of emotions in juvenile myoclonic epilepsy.

OBJECTIVE: Perception and recognition of emotions are fundamental prerequisites of human life. Patients with juvenile myoclonic epilepsy (JME) may have emotional and behavioral impairments that might influence socially desirable interactions. We aimed to investigate perception and recognition of emotions in patients with JME by means of neuropsychological tests and functional magnetic resonance imaging (fMRI). METHODS: Sixty-five patients with JME (median age = 27 years, interquartile range [IQR] = 23-34) were prospectively recruited at the Department of Neurology, Christian Doppler University Hospital, Paracelsus Medical University, Salzburg, Austria. Patients were compared to 68 healthy controls (median age = 24 years, IQR = 21-31), matched for sex, age, and education. All study participants underwent the Networks of Emotion Processing test battery (NEmo), an fMRI paradigm of "dynamic fearful faces," a structured interview for psychiatric and personality disorders, and comprehensive neuropsychological testing. RESULTS: JME patients versus healthy controls demonstrated significant deficits in emotion recognition in facial and verbal tasks of all emotions, especially fear. fMRI revealed decreased amygdala activation in JME patients as compared to healthy controls. Patients were at a higher risk of experiencing psychiatric disorders as compared to healthy controls. Cognitive evaluation revealed impaired attentional and executive functioning, namely psychomotor speed, tonic alertness, divided attention, mental flexibility, and inhibition of automated reactions. Duration of epilepsy correlated negatively with parallel prosodic and facial emotion recognition in NEmo. Deficits in emotion recognition were not associated with psychiatric comorbidities, impaired attention and executive functions, types of seizures, and treatment. SIGNIFICANCE: This prospective study demonstrated that as compared to healthy subjects, patients with JME had significant deficits in recognition and perception of emotions as shown by neuropsychological tests and fMRI. The results of this study may have importance for psychological/psychotherapeutic interventions in the management of patients with JME.

Associations between self-reported sleep, overnight memory consolidation, and emotion perception: A large-scale online study in the general population.

Experimental studies suggest that short or disrupted sleep impairs memory consolidation, mood, and perception of emotional stimuli. However, studies have chiefly relied on laboratory-based study designs and small sample sizes. The aim of this fully online and pre-registered study was to investigate the association between sleep and overnight memory consolidation, emotion perception, and affect in a large, self-selected UK sample. A total of 1646 participants (473 completed) took part in an online study, where they completed a declarative (word-pairs) memory task, emotion perception task (valence ratings of images), and rated their affect within 2 h of bed-time. The following morning, participants reported on their state affect, sleep for the previous night, completed a cued recall task for the previously presented word-pairs, rated the valence of previously viewed images, and completed a surprise recognition task. Demographic data and habitual sleep quality and duration (sleep traits) were also recorded. Habitual sleep traits were associated with immediate recall for the word-pairs task, while self-reported sleep parameters for the specific night were not associated with overnight memory consolidation. Neither habitual sleep traits, nor nightly sleep parameters were associated with unpleasantness ratings to negative stimuli or overnight habituation. Habitual poor sleep was associated with less positive and more negative affect, and morning affect was predicted by the specific night's sleep. This study suggests that overnight emotional processing and declarative memory may not be associated with self-reported sleep across individuals. More work is needed to understand how findings from laboratory-based studies extrapolate to real-world samples and contexts.

Vocal emotion perception in schizophrenia and its diagnostic significance.

BACKGROUND: Cognitive and emotional impairment are among the core features of schizophrenia; assessment of vocal emotion recognition may facilitate the detection of schizophrenia. We explored the differences between cognitive and social aspects of emotion using vocal emotion recognition and detailed clinical characterization. METHODS: Clinical symptoms and social and cognitive functioning were assessed by trained clinical psychiatrists. A vocal emotion perception test, including an assessment of emotion recognition and emotional intensity, was conducted. One-hundred-six patients with schizophrenia (SCZ) and 230 healthy controls (HCs) were recruited. RESULTS: Considering emotion recognition, scores for all emotion categories were significantly lower in SCZ compared to HC. Considering emotional intensity, scores for anger, calmness, sadness, and surprise were significantly lower in the SCZs. Vocal recognition patterns showed a trend of unification and simplification in SCZs. A direct correlation was confirmed between vocal recognition impairment and cognition. In diagnostic tests, only the total score of vocal emotion recognition was a reliable index for the presence of schizophrenia. CONCLUSIONS: This study shows that patients with schizophrenia are characterized by impaired vocal emotion perception. Furthermore, explicit and implicit vocal emotion perception processing in individuals with schizophrenia are viewed as distinct entities. This study provides a voice recognition tool to facilitate and improve the diagnosis of schizophrenia.

Exploring the perception of stress in childhood and early adolescence.

Stressful life experiences may jeopardize the healthy development of children. To improve interventions, more knowledge is needed on the perception of stress by children. In adults, stress is regarded as a state of low valence and high arousal. It is unclear whether children perceive stress similarly. In the current study, 35 children of the general population completed three tasks aiming to provide insight into their knowledge of the concept stress. In the first task, participants were asked about their verbal knowledge of the concept stress. In the second task, they rated the valence and arousal of eight emotion-evoking vignettes. In the final task, participants completed an experience sampling survey for at least 1 day, consisting of a stress thermometer and pictorial scales of valence and arousal. Participants' perception of stress was found to be mainly valence focused. Age and sex were found to play a role in the degree of arousal focus. Older participants differentiated more in arousal levels than younger participants, as did girls in comparison with boys. Because the perception of stress depends on developmental and other individual factors, using stress as a single measurement dimension in a survey is not recommended.

Beyond enjoyment: Young children consider the normative goodness of activity engagement when attributing happiness.

Individuals are typically happy when engaging in enjoyable activities. But many enjoyable activities could be harmful when engaged in to excess. Do children consider the normative goodness of activity engagement levels when attributing happiness? To examine this question, we presented children with enjoyable activities that are often harmless in moderation but harmful in excess. When told that engaging in their favorite activities at their preferred amount was either normatively good (i.e., harmless and permitted) or normatively bad (i.e., harmful and forbidden), 10- and 11-year-old and 7- and 8-year-old children (Study 1) and even 5-year-old children (Studies 2 and 3 with simplified methods) attributed less happiness when the engagement level was normatively bad than when it was normatively good both to themselves and to another child. Young children also perceived normatively bad engagement as less interesting and pleasurable (Study 3). The findings suggest that children consider the normative goodness of activity engagement (rather than enjoyment alone) when attributing happiness, illuminating how children understand happiness.

The Study on the Identification of Musical Passages for an Emotion Perception Scale for People With Developmental Disabilities.

BACKGROUND: Emotion recognition is essential to the social adjustment and social interaction of people with intellectual and developmental disabilities (IDDs). Given that music is a medium for expressing and conveying human emotion, we conducted this preliminary study to identify musical passages representing the basic human emotions of happiness, sadness, and anger, with the goal of developing a music-based emotion perception scale for IDDs. METHODS: To identify musical passages for emotion perception, 20 certified music therapists evaluated 100 selected musical passages and established 60 pieces that yielded the highest agreement for each emotion category. During the second phase of this study, 300 neurotypical participants rated 60 passages in terms of the perceived type and intensity of emotions expressed. RESULTS: The 60 passages showed high reliability and were statistically classified into three factors: happiness, sadness, and anger. The k-means cluster analysis yielded a cut-off score of 41 for the low emotion perception group (F = 1120.63, P < 0.001). The hierarchical logistic regression analysis revealed that only model 3 (musical passages) was significantly associated with low emotion perception (step χ² = 227.8, P < 0.001). CONCLUSION: The selected musical passages demonstrated high reliability and established three factors for identifying perceptions of happiness, sadness, and anger. Neither psychological status nor individual demographic characteristics affected the emotion perception results.

Words matter when inferring emotions: a conceptual replication and extension.

It is long known that facial configurations play a critical role when inferring mental and emotional states from others. Nevertheless, there is still a scientific debate on how we infer emotions from facial configurations. The theory of constructed emotion (TCE) suggests that we may infer different emotions from the same facial configuration, depending on the context (e.g. provided by visual and lexical cues) in which they are perceived. For instance, a recent study found that participants were more accurate in inferring mental and emotional states across three different datasets (i.e. RMET, static and dynamic emojis) when words were provided (i.e. forced-choice task), compared to when they were not (i.e. free-labelling task), suggesting that words serve as contexts that modulate the inference from facial configurations. The goal of the current within-subject study was to replicate and extend these findings by adding a fourth dataset (KDEF-dyn), consisting of morphed human faces (to increase the ecological validity). Replicating previous findings, we observed that words increased accuracy across the three (previously used) datasets, an effect that was also observed for the facial morphed stimuli. Our findings are in line with the TCE, providing support for the importance of contextual verbal cues in emotion perception.

"But not the music": psychopathic traits and difficulties recognising and resonating with the emotion in music.

Recognising and responding appropriately to emotions is critical to adaptive psychological functioning. Psychopathic traits (e.g. callous, manipulative, impulsive, antisocial) are related to differences in recognition and response when emotion is conveyed through facial expressions and language. Use of emotional music stimuli represents a promising approach to improve our understanding of the specific emotion processing difficulties underlying psychopathic traits because it decouples recognition of emotion from cues directly conveyed by other people (e.g. facial signals). In Experiment 1, participants listened to clips of emotional music and identified the emotional content (Sample 1, N = 196) or reported on their feelings elicited by the music (Sample 2, N = 197). Participants accurately recognised (t(195) = 32.78, p < .001, d = 4.69) and reported feelings consistent with (t(196) = 7.84, p < .001, d = 1.12) the emotion conveyed in the music. However, psychopathic traits were associated with reduced emotion recognition accuracy (F(1, 191) = 19.39, p < .001) and reduced likelihood of feeling the emotion (F(1, 193) = 35.45, p < .001), particularly for fearful music. In Experiment 2, we replicated findings for broad difficulties with emotion recognition (Sample 3, N = 179) and emotional resonance (Sample 4, N = 199) associated with psychopathic traits. Results offer new insight into emotion recognition and response difficulties that are associated with psychopathic traits.

Development and validation of the first adaptive test of emotion perception in music.

ABSTRACTThe Musical Emotion Discrimination Task (MEDT) is a short, non-adaptive test of the ability to discriminate emotions in music. Test-takers hear two performances of the same melody, both played by the same performer but each trying to communicate a different basic emotion, and are asked to determine which one is "happier", for example. The goal of the current study was to construct a new version of the MEDT using a larger set of shorter, more diverse music clips and an adaptive framework to expand the ability range for which the test can deliver measurements. The first study analysed responses from a large sample of participants (N = 624) to determine how musical features contributed to item difficulty, which resulted in a quantitative model of musical emotion discrimination ability rooted in Item Response Theory (IRT). This model informed the construction of the adaptive MEDT. A second study contributed preliminary evidence for the validity and reliability of the adaptive MEDT, and demonstrated that the new version of the test is suitable for a wider range of abilities. This paper therefore presents the first adaptive musical emotion discrimination test, a new resource for investigating emotion processing which is freely available for research use.

He must be mad; she might be sad: perceptual and decisional aspects of emotion recognition in ambiguous faces.

While the recognition of ambiguous emotions is crucial for successful social interactions, previous work has shown that they are perceived differently depending on whether they are viewed on male or female faces. The present paper aims to shed light on this phenomenon by exploring two hypotheses: the confounded signal hypothesis, which posits the existence of perceptual overlaps between emotions and gendered morphotypes, and the social role hypothesis, according to which the observer's responses are biased by stereotypes. Participants were asked to categorise blended faces (i.e. artificial faces made ambiguous by mixing two emotions) in a forced-choice task. Six emotions were used to create each blend (neutral, surprise, sadness, fear, happiness, anger), for a total of 15 expressions. We then applied signal detection theory - considering both the morphotype of the stimuli and the participants' gender - to distinguish participants' perceptual processes from their response biases. The results showed a perceptual advantage for anger on male faces and for sadness on female faces. However, different strategies were deployed when labelling emotions on gendered morphotypes. In particular, a response bias towards angry male faces establishes their special status, as they resulted in both excellent detection and a tendency to be over-reported, especially by women.

White matter association tracts underlying language and theory of mind: An investigation of 809 brains from the Human Connectome Project.

Language and theory of mind (ToM) are the cognitive capacities that allow for the successful interpretation and expression of meaning. While functional MRI investigations are able to consistently localize language and ToM to specific cortical regions, diffusion MRI investigations point to an inconsistent and sometimes overlapping set of white matter tracts associated with these two cognitive domains. To further examine the white matter tracts that may underlie these domains, we use a two-tensor tractography method to investigate the white matter microstructure of 809 participants from the Human Connectome Project. 20 association white matter tracts (10 in each hemisphere) are uniquely identified by leveraging a neuroanatomist-curated automated white matter tract atlas. The fractional anisotropy (FA), mean diffusivity (MD), and number of streamlines (NoS) are measured for each white matter tract. Performance on neuropsychological assessments of semantic memory (NIH Toolbox Picture Vocabulary Test, TPVT) and emotion perception (Penn Emotion Recognition Test, PERT) are used to measure critical subcomponents of the language and ToM networks, respectively. Regression models are constructed to examine how structural measurements of left and right white matter tracts influence performance across these two assessments. We find that semantic memory performance is influenced by the number of streamlines of the left superior longitudinal fasciculus III (SLF-III), and emotion perception performance is influenced by the number of streamlines of the right SLF-III. Additionally, we find that performance on both semantic memory & emotion perception is influenced by the FA of the left arcuate fasciculus (AF). The results point to multiple, overlapping white matter tracts that underlie the cognitive domains of language and ToM. Results are discussed in terms of hemispheric dominance and concordance with prior investigations.

Positive hysteresis in emotion recognition: Face processing visual regions are involved in perceptual persistence, which mediates interactions between anterior insula and medial prefrontal cortex.

Facial emotion perception can be studied from the point of view of dynamic systems whose output may depend not only on current input but also on prior history - a phenomenon known as hysteresis. In cognitive neuroscience, hysteresis has been described as positive (perceptual persistence) or negative (fatigue of current percept) depending on whether perceptual switching occurs later or earlier than actual physical stimulus changes. However, its neural correlates remain elusive. We used dynamic transitions between emotional expressions and combined behavioral assessment with functional magnetic resonance imaging (fMRI) to investigate the underlying circuitry of perceptual hysteresis in facial emotion recognition. Our findings revealed the involvement of face-selective visual areas - fusiform face area (FFA) and superior temporal sulcus (STS) - in perceptual persistence as well as the right anterior insula. Moreover, functional connectivity analyses revealed an interplay between the right anterior insula and medial prefrontal cortex, which showed to be dependent on the presence of positive hysteresis. Our results support the hypothesis that high-order regions are involved in perceptual stabilization and decision during perceptual persistence (positive hysteresis) and add evidence to the role of the anterior insula as a hub of sensory information in perceptual decision-making.

Eye gaze direction modulates nonconscious affective contextual effect.

Facial emotion recognition is inherently contextualized and may automatically incorporate affective information from the context. Here we investigate whether this affective contextual effect is modulated by a prominent social cue, namely, the gaze direction of the contextualized emotional face. We demonstrate that the perceived emotional expression of a visible target face is biased toward the emotion of an invisible contextual face, with this nonconscious affective contextual modulation dependent on the gaze direction of the target face. In particular, a target face gazing toward a contextual face induced a larger affective contextual effect than a face gazing away. Furthermore, this gaze modulation effect specifically occurred for invisible fearful contexts and hinged on individual trait anxiety levels. These findings show that social information delivered by gaze cues can modulate the fear-specific affective contextual effect without awareness, shedding new light on how compound socio-affective signals are automatically integrated into our perception of others' emotions.