Associated valence impacts early visual processing of letter strings: Evidence from ERPs in a cross-modal learning paradigm.

Emotion effects in event-related potentials (ERPs) during reading have been observed at very short latencies of around 100 to 200 ms after word onset. The nature of these effects remains a matter of debate: First, it is possible that they reflect semantic access, which might thus occur much faster than proposed by most reading models. Second, it is possible that associative learning of a word's shape might contribute to the emergence of emotion effects during visual processing. The present study addressed this question by employing an associative learning paradigm on pronounceable letter strings (pseudowords). In a learning session, letter strings were associated with positive, neutral, or negative valence by means of monetary gain, loss, or zero outcome. Crucially, half of the stimuli were learned in the visual modality, while the other half was presented acoustically, allowing for experimental separation of associated valence and physical percept. In a test session one or two days later, acquired letter strings were presented in an old/new decision task while we recorded ERPs. Behavioural data showed an advantage for gain-associated stimuli both during learning and in the delayed old/new task. Early emotion effects in ERPs were limited to visually acquired letter strings, but absent for acoustically acquired letter strings. These results imply that associative learning of a word's visual features might play an important role in the emergence of emotion effects at the stage of perceptual processing.

Implicit reward associations impact face processing: Time-resolved evidence from event-related brain potentials and pupil dilations.

The present study aimed at investigating whether associated motivational salience causes preferential processing of inherently neutral faces similar to emotional expressions by means of event-related brain potentials (ERPs) and changes of the pupil size. To this aim, neutral faces were implicitly associated with monetary outcome, while participants (N = 44) performed a face-matching task with masked primes that ensured performance around chance level and thus an equal proportion of gain, loss, and zero outcomes. During learning, motivational context strongly impacted the processing of the fixation, prime and mask stimuli prior to the target face, indicated by enhanced amplitudes of subsequent ERP components and increased pupil size. In a separate test session, previously associated faces as well as novel faces with emotional expressions were presented within the same task but without motivational context and performance feedback. Most importantly, previously gain-associated faces amplified the LPC, although the individually contingent face-outcome assignments were not made explicit during the learning session. Emotional expressions impacted the N170 and EPN components. Modulations of the pupil size were absent in both motivationally-associated and emotional conditions. Our findings demonstrate that neural representations of neutral stimuli can acquire increased salience via implicit learning, with an advantage for gain over loss associations.

Spatiotemporal pattern of appraising social and emotional relevance: Evidence from event-related brain potentials.

Social information is particularly relevant for the human species because of its direct link to guiding physiological responses and behavior. Accordingly, extant functional magnetic resonance imaging (fMRI) data suggest that social content may form a unique stimulus dimension. It remains largely unknown, however, how neural activity underlying social (versus nonsocial) information processing temporally unfolds, and how such social information appraisal may interact with the processing of other stimulus characteristics, particularly emotional meaning. Here, we presented complex visual scenes differing in both social (vs. nonsocial) and emotional relevance (positive, negative, neutral) intermixed with scrambled versions of these pictures to N = 24 healthy young adults. Event-related brain potentials (ERPs) to intact pictures were examined for gaining insight to the dynamics of appraisal of both dimensions, implemented within the brain. Our main finding is an early interaction between social and emotional relevance due to enhanced amplitudes of early ERP components to emotionally positive and neutral pictures of social compared to nonsocial content, presumably reflecting rapid allocation of attention and counteracting an overall negativity bias. Importantly, our ERP data show high similarity with previously observed fMRI data using the same stimuli, and source estimations located the ERP effects in overlapping occipitotemporal brain areas. Our novel data suggest that relevance detection may occur already as early as around 100 ms after stimulus onset and may combine relevance checks not only examining intrinsic pleasantness/emotional valence but also social content as a unique, highly relevant stimulus dimension.

What makes the hedonic experience of a meal in a top restaurant special and retrievable in the long term? Meal-related, social and personality factors.

Knowing what makes a top gastronomy experience unique and retrievable in the long term is of interest for scientific and economic reasons. Recent attempts to isolate predictors of the hedonic evaluation of food have afforded several factors, such as individual and social attributes, or liking/disliking profiles. However, in these studies relevant variables have been examined in isolation without an integrative perspective. Here we investigated 80 guests enjoying a 23-course meal in a top gastronomy restaurant, in groups of four. Our main question concerned the factors driving the overall evaluation of the meal at its conclusion and after three months. To this aim we administered the Big Five Personality Inventory before the meal, dish-by-dish hedonic ratings, and a multi-dimensional Meal Experience Questionnaire (MEQ) at the end of the meal. Hedonic evaluations of the meal were collected immediately after the meal and three months later. Better immediate overall evaluations were predicted by both the number of peaks in dish-by-dish ratings and by positive ratings of the final dish. Both factors and the number of troughs were also critical for the long-term evaluation after three months. The MEQ dimensions overall interest, valence and distraction predicted immediate evaluations, while the long-term evaluations were determined by interest and high scores on the personality traits agreeableness and conscientiousness. High consistency of the hedonic ratings within quartets indicated the relevance of commensality for the meal experience. The present findings highlight the simultaneous relevance of food- and personality-related factors and commensality for a top gastronomy meal experience in the short and long-run. The uncovered relationships are of theoretical interest and for those involved in designing meals for consumers in various settings.

Motivational Salience Modulates Early Visual Cortex Responses across Task Sets.

Motivationally relevant stimuli benefit from strengthened sensory processing. It is unclear, however, if motivational value of positive and negative valence has similar or dissociable effects on early visual processing. Moreover, whether these perceptual effects are task-specific, stimulus-specific, or more generally feature-based is unknown. In this study, we compared the effects of positive and negative motivational value on early sensory processing using ERPs. We tested the extent to which these effects could generalize to new task contexts and to stimuli sharing common features with the motivationally significant ones. At the behavioral level, stimuli paired with positive incentives were learned faster than stimuli paired with neutral or negative outcomes. The ERP results showed that monetary loss elicited higher neural activity in V1 (at the C1 level) compared with reward, whereas the latter influenced postperceptual processing stages (P300). Importantly, the early loss-related effect generalized to new contexts and to new stimuli with common features, whereas the later reward effects did not spill over to the new context. These results suggest that acquired negative motivational salience can influence early sensory processing by means of plastic changes in feature-based processing in V1.

Associated motivational salience impacts early sensory processing of human faces.

Facial expressions of emotion have an undeniable processing advantage over neutral faces, discernible both at behavioral level and in emotion-related modulations of several event-related potentials (ERPs). Recently it was proposed that also inherently neutral stimuli might gain salience through associative learning mechanisms. The present study investigated whether acquired motivational salience leads to processing advantages similar to biologically determined origins of inherent emotional salience by applying an associative learning paradigm to human face processing. Participants (N=24) were trained to categorize neutral faces to salience categories by receiving different monetary outcomes. ERPs were recorded in a subsequent test phase consisting of gender decisions on previously associated faces, as well as on familiarized and novel faces expressing happy, angry or no emotion. Previously reward-associated faces boosted the P1 component, indicating that acquired reward-associations modulate early sensory processing in extrastriate visual cortex. However, ERP modulations to emotional - primarily angry - expressions expanded to subsequent processing stages, as reflected in well-established emotion-related ERPs. The present study offers new evidence that motivational salience associated to inherently neutral stimuli can sharpen sensory encoding but does not obligatorily lead to preferential processing at later stages.

Dissociating the Influence of Affective Word Content and Cognitive Processing Demands on the Late Positive Potential.

The late positive potential (LPP) elicited by affective stimuli in the event-related brain potential (ERP) is often assumed to be a member of the P3 family. The present study addresses the relationship of the LPP to the classic P3b in a published data set, using a non-parametric permutation test for topographical comparisons, and residue iteration decomposition to assess the temporal features of the LPP and the P3b by decomposing the ERP into several component clusters according to their latency variability. The experiment orthogonally manipulated arousal and valence of words, which were either read or judged for lexicality. High-arousing and positive valenced words induced a larger LPP than low-arousing and negative valenced words, respectively, and the LDT elicited a larger P3b than reading. The experimental manipulation of arousal, valence, and task yielded main effects without any interactions on ERP amplitude in the LPP/P3b time range. The arousal and valence effects partially differed from the task effect in scalp topography; in addition, whereas the late positive component elicited by affective stimuli, defined as LPP, was stimulus-locked, the late positive component elicited by task demand, defined as P3b, was mainly latency-variable. Therefore LPP and P3b manifest different subcomponents.

Emotion effects on the n170: a question of reference?

We investigated whether face-specific processes as indicated by the N170 in event-related brain potentials (ERPs) are modulated by emotional significance in facial expressions. Results yielded that emotional modulations over temporo-occipital electrodes typically used to measure the N170 were less pronounced when ERPs were referred to mastoids than when average reference was applied. This offers a potential explanation as to why the literature has so far yielded conflicting evidence regarding effects of emotional facial expressions on the N170. However, spatial distributions of the N170 and emotion effects across the scalp were distinguishable for the same time point, suggesting different neural sources for the N170 and emotion processing. We conclude that the N170 component itself is unaffected by emotional facial expressions, with overlapping activity from the emotion-sensitive early posterior negativity accounting for amplitude modulations over typical N170 electrodes. Our findings are consistent with traditional models of face processing assuming face and emotion encoding to be parallel and independent processes.

Classification of dynamic facial expressions of emotion presented briefly.

A number of studies have shown an impact of speed of a developing facial expression of emotion on its recognition and perceived naturalness. Still, the impact of speed at constant, short presentation times, as normally used in many experiments is unclear. In the present study participants classified faces displaying facial expressions of six basic emotions in static and dynamic presentation modes and three different types of neutral movements. Stimuli were created with computer software that allows fine-grained control over action units and dynamic features. Rise times in dynamic expressions varied between 200 and 900 ms. Results replicated classical findings showing better performance for expressions of happiness, and frequent confusions among morphologically similar expressions, and a general dynamic facilitation for most expressions. Importantly, dynamic presentation as such facilitated a more accurate classification, but variations in speed at the fast range studied here had no noticeable effect for expressions of anger, fear, happiness, and surprise. The main exception was sadness, which was best recognised at slow speed and in static pictures, and disgust, which was most unambiguously categorised at fast to moderate speed.

Measuring the speed of recognising facially expressed emotions.

Faces provide identity- and emotion-related information-basic cues for mastering social interactions. Traditional models of face recognition suggest that following a very first initial stage the processing streams for facial identity and expression depart. In the present study we extended our previous multivariate investigations of face identity processing abilities to the speed of recognising facially expressed emotions. Analyses are based on a sample of N=151 young adults. First, we established a measurement model with a higher order factor for the speed of recognising facially expressed emotions (SRE). This model has acceptable fit without specifying emotion-specific relations between indicators. Next, we assessed whether SRE can be reliably distinguished from the speed of recognising facial identity (SRI) and found latent factors for SRE and SRI to be perfectly correlated. In contrast, SRE and SRI were both only moderately related to a latent factor for the speed of recognising non-face stimuli (SRNF). We conclude that the processing of facial stimuli-and not the processing of facially expressed basic emotions-is the critical component of SRE. These findings are at variance with suggestions of separate routes for processing facial identity and emotional facial expressions and suggest much more communality between these streams as far as the aspect of processing speed is concerned.

Visual competition attenuates emotion effects during overt attention shifts.

Numerous different objects are simultaneously visible in a person's visual field, competing for attention. This competition has been shown to affect eye-movements and early neural responses toward stimuli, while the role of a stimulus' emotional meaning for mechanisms of overt attention shifts under competition is unclear. The current study combined EEG and eye-tracking to investigate effects of competition and emotional content on overt shifts of attention to human face stimuli. Competition prolonged the latency of the P1 component and of saccades, while faces showing emotional expressions elicited an early posterior negativity (EPN). Remarkably, the emotion-related modulation of the EPN was attenuated when two stimuli were competing for attention compared to non-competition. In contrast, no interaction effects of emotional expression and competition were observed on other event-related potentials. This finding indicates that competition can decelerate attention shifts in general and also diminish the emotion-driven attention capture, measured through the smaller effects of emotional expression on EPN amplitude. Reduction of the brain's responsiveness to emotional content in the presence of distractors contradicts models that postulate fully automatic processing of emotions.

How German and Italian Laypeople Reason about Distributive Shortages during COVID-19.

(1) Background: The COVID-19 pandemic provided a unique opportunity to investigate how moral reasoning is influenced by individuals' exposure to a crisis and by personal, societal and temporal proximity. We examined how Italians and Germans judged different behaviors that arose because of the pandemic, which affected health and societal matters. (2) Methods: Over the course of four months and three assessment periods, we used an observational online survey to assess participants' judgments regarding seven scenarios that addressed distributive shortages during the pandemic. (3) Results: Overall, there was no clear answering pattern across all scenarios. For a variation of triage and pandemic restrictions, most participants selected a mean value, which can be interpreted as deferring the choice. For the other scenarios, most participants used the extremes of the scale, thereby reflecting a clear opinion of the public regarding the moral issue. In addition, moral reasoning varied across the two countries, assessment periods, fear, and age. (4) Conclusions: By using scenarios that were taken from real-life experiences, the current study addresses criticism that moral research mostly relies on unrealistic scenarios that lack in external validity, plausibility, and proximity to everyday situations. In addition, it shows how lay people regard measures of public health and societal decision-making.

On the automaticity of emotion processing in words and faces: event-related brain potentials evidence from a superficial task.

The degree to which emotional aspects of stimuli are processed automatically is controversial. Here, we assessed the automatic elicitation of emotion-related brain potentials (ERPs) to positive, negative, and neutral words and facial expressions in an easy and superficial face-word discrimination task, for which the emotional valence was irrelevant. Both emotional words and facial expressions impacted ERPs already between 50 and 100 ms after stimulus onset, possibly reflecting rapid relevance detection. Following this initial processing stage only emotionality in faces but not in words was associated with an early posterior negativity (EPN). Therefore, when emotion is irrelevant in a task which requires superficial stimulus analysis, automatically enhanced sensory encoding of emotional content appears to occur only for evolutionary prepared emotional stimuli, as reflected in larger EPN amplitudes to faces, but not to symbolic word stimuli.

Overt and covert attention shifts to emotional faces: Combining EEG, eye tracking, and a go/no-go paradigm.

In everyday life, faces with emotional expressions quickly attract attention and eye movements. To study the neural mechanisms of such emotion-driven attention by means of event-related brain potentials (ERPs), tasks that employ covert shifts of attention are commonly used, in which participants need to inhibit natural eye movements towards stimuli. It remains, however, unclear how shifts of attention to emotional faces with and without eye movements differ from each other. The current preregistered study aimed to investigate neural differences between covert and overt emotion-driven attention. We combined eye tracking with measurements of ERPs to compare shifts of attention to faces with happy, angry, or neutral expressions when eye movements were either executed (go conditions) or withheld (no-go conditions). Happy and angry faces led to larger EPN amplitudes, shorter latencies of the P1 component, and faster saccades, suggesting that emotional expressions significantly affected shifts of attention. Several ERPs (N170, EPN, LPC) were augmented in amplitude when attention was shifted with an eye movement, indicating an enhanced neural processing of faces if eye movements had to be executed together with a reallocation of attention. However, the modulation of ERPs by facial expressions did not differ between the go and no-go conditions, suggesting that emotional content enhances both covert and overt shifts of attention. In summary, our results indicate that overt and covert attention shifts differ but are comparably affected by emotional content.

"Finding an Emotional Face" Revisited: Differences in Own-Age Bias and the Happiness Superiority Effect in Children and Young Adults.

People seem to differ in their visual search performance involving emotionally expressive faces when these expressions are seen on faces of others close to their age (peers) compared to faces of non-peers, known as the own-age bias (OAB). This study sought to compare search advantages in angry and happy faces detected on faces of adults and children on a pool of children (N = 77, mean age = 5.57) and adults (N = 68, mean age = 21.48). The goals of this study were to (1) examine the developmental trajectory of expression recognition and (2) examine the development of an OAB. Participants were asked to find a target face displaying an emotional expression among eight neutral faces. Results showed that children and adults found happy faces significantly faster than angry and fearful faces regardless of it being present on the faces of peers or non-peers. Adults responded faster to the faces of peers regardless of the expression. Furthermore, while children detected angry faces significantly faster compared to fearful ones, we found no such difference in adults. In contrast, adults detected all expressions significantly faster when they appeared on the faces of other adults compared to the faces of children. In sum, we found evidence for development in detecting facial expressions and also an age-dependent increase in OAB. We suggest that the happy face could have an advantage in visual processing due to its importance in social situations and its overall higher frequency compared to other emotional expressions. Although we only found some evidence on the OAB, using peer or non-peer faces should be a theoretical consideration of future research because the same emotion displayed on non-peers' compared to peers' faces may have different implications and meanings to the perceiver.

Emotions in cognitive conflicts are not aversive but are task specific.

It has been suggested that cognitive conflicts require effortful processing and, therefore, are aversive (Botvinick, 2007). In the present study, we compared conflicts emerging from the inhibition of a predominant response tendency in a go/no-go task with those between incompatible response activations in a Simon task in a within-subjects design, using the same type of stimuli. Whereas no-go trials elicited reduced skin conductance and pupillometric responses, but prolonged corrugator muscle activity, as compared with go trials, incompatible and compatible Simon trials were indistinguishable with respect to these parameters. Furthermore, the conflict-sensitive N2 components of the event-related brain potential were similar in amplitude, but showed significantly different scalp distributions, indicating dissociable neural generator systems. The present findings suggest the involvement of different emotional and cognitive processes in both types of cognitive conflicts-none being aversive, however. In addition, the N2 findings call into question claims of common monitoring systems for all kinds of cognitive conflicts.

A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion.

Human faces express emotions, informing others about their affective states. In order to measure expressions of emotion, facial Electromyography (EMG) has widely been used, requiring electrodes and technical equipment. More recently, emotion recognition software has been developed that detects emotions from video recordings of human faces. However, its validity and comparability to EMG measures is unclear. The aim of the current study was to compare the Affectiva Affdex emotion recognition software by iMotions with EMG measurements of the zygomaticus mayor and corrugator supercilii muscle, concerning its ability to identify happy, angry and neutral faces. Twenty participants imitated these facial expressions while videos and EMG were recorded. Happy and angry expressions were detected by both the software and by EMG above chance, while neutral expressions were more often falsely identified as negative by EMG compared to the software. Overall, EMG and software values correlated highly. In conclusion, Affectiva Affdex software can identify facial expressions and its results are comparable to EMG findings.

Hormonal and modality specific effects on males' emotion recognition ability.

Successful emotion recognition is a key component of human socio-emotional communication skills. However, little is known about the factors impacting males' accuracy in emotion recognition tasks. This pre-registered study examined potential candidates, focusing on the modality of stimulus presentation, emotion category and individual baseline hormone levels. In an additional exploratory analysis, we examined the association of testosterone x cortisol interaction with recognition accuracy and reaction times. We obtained accuracy and reaction time scores from 282 males who categorized voice, face and voice-face stimuli for nonverbal emotional content. Results showed that recognition accuracy was significantly higher in the audio-visual than in the auditory or visual modality. While Spearman's rank correlations showed no significant association of testosterone (T) with recognition accuracy or with response times for specific emotions, the logistic and linear regression models uncovered some evidence for a positive association between T and recognition accuracy as well as between cortisol (C) and reaction time. In addition, the overall effect size of T by C interaction with recognition accuracy and reaction time was significant, but small. Our results establish that audio-visual congruent stimuli enhance recognition accuracy and provide novel empirical support by showing that the interaction of testosterone and cortisol relates to males' accuracy and response times in emotion recognition tasks.

Vascular response to social cognitive performance measured by infrared thermography: A translational study from mouse to man.

To assess complex social recognition in mice, we previously developed the SocioBox paradigm. Unexpectedly, 4 weeks after performing in the SocioBox, mice displayed robust social avoidance during Y-maze sociability testing. This unique "sociophobia" acquisition could be documented in independent cohorts. We therefore employed infrared thermography as a non-invasive method of stress-monitoring during SocioBox testing (presentation of five other mice) versus empty box. A higher Centralization Index (body/tail temperature) in the SocioBox correlated negatively with social recognition memory and, after 4 weeks, with social preference in the Y-maze. Assuming that social stimuli might be associated with characteristic thermo-responses, we exposed healthy men (N = 103) with a comparably high intelligence level to a standardized test session including two cognitive tests with or without social component (face versus pattern recognition). In some analogy to the Centralization Index (within-subject measure) used in mice, the Reference Index (ratio nose/malar cheek temperature) was introduced to determine the autonomic facial response/flushing during social recognition testing. Whereas cognitive performance and salivary cortisol were comparable across human subjects and tests, the Face Recognition Test was associated with a characteristic Reference Index profile. Infrared thermography may have potential for discriminating disturbed social behaviors.

Emotions in word and face processing: early and late cortical responses.

Recent research suggests that emotion effects in word processing resemble those in other stimulus domains such as pictures or faces. The present study aims to provide more direct evidence for this notion by comparing emotion effects in word and face processing in a within-subject design. Event-related brain potentials (ERPs) were recorded as participants made decisions on the lexicality of emotionally positive, negative, and neutral German verbs or pseudowords, and on the integrity of intact happy, angry, and neutral faces or slightly distorted faces. Relative to neutral and negative stimuli both positive verbs and happy faces elicited posterior ERP negativities that were indistinguishable in scalp distribution and resembled the early posterior negativities reported by others. Importantly, these ERP modulations appeared at very different latencies. Therefore, it appears that similar brain systems reflect the decoding of both biological and symbolic emotional signals of positive valence, differing mainly in the speed of meaning access, which is more direct and faster for facial expressions than for words.