Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces.

Decoding someone's facial expressions provides insights into his or her emotional experience. Recently, Automatic Facial Coding (AFC) software has been developed to provide measurements of emotional facial expressions. Previous studies provided first evidence for the sensitivity of such systems to detect facial responses in study participants. In the present experiment, we set out to generalise these results to affective responses as they can occur in variable social interactions. Thus, we presented facial expressions (happy, neutral, angry) and instructed participants (N = 64) to either actively mimic, to look at them passively (n = 21), or to inhibit their own facial reaction (n = 22). A video stream for AFC and an electromyogram (EMG) of the zygomaticus and corrugator muscles were registered continuously. In the mimicking condition, both AFC and EMG differentiated well between facial expressions in response to the different emotional pictures. In the passive viewing and in the inhibition condition AFC did not detect changes in facial expressions whereas EMG was still highly sensitive. Although only EMG is sensitive when participants intend to conceal their facial reactions, these data extend previous findings that Automatic Facial Coding is a promising tool for the detection of intense facial reaction.

Perception of children's faces with unilateral coronal synostosis--an eye-tracking investigation.

PURPOSE: Premature unilateral coronal craniosynostosis results in distinctive cranial and facial abnormalities of varying severity, including orbital dystopia and an abnormal head shape. As the face is affected, these children may encounter stigmatization. To avoid this scenario, many parents elect for their child to undergo surgical correction. Laypeople's perception of children with either untreated or treated unilateral coronal craniosynostosis (UCS) has not yet been objectively evaluated. METHODS: This study introduces eye tracking as an objective instrument in order to evaluate the perception of 14 children with coronal synostosis, both pre- and postoperatively. Age-matched healthy children served as a control group. Using standardized photos, the involuntary eye movements and the fixations of 30 unaffected laypeople were evaluated. RESULTS: In the untreated children, whose faces were characterized by striking orbital dystopia, the eyes drew more attention than those of the healthy children. The results of our study demonstrate that the operative correction of unilateral coronal synostosis results in the normalization of the asymmetry of the fronto-orbital region, whereas the C-shaped deformity of the midface, which is not addressed via surgery, subsequently attracts more attention. CONCLUSION: Eye tracking objectively evaluates both the perception of craniofacial abnormalities and the extent of the approximation of normality after surgical correction. We introduce eye tracking as an objective measurement tool for craniofacial abnormalities for the first time.

Not so harmless anymore: how context impacts the perception and electrocortical processing of neutral faces.

Our first impression of others is highly influenced by their facial appearance. However, the perception and evaluation of faces is not only guided by internal features such as facial expressions, but also highly dependent on contextual information such as secondhand information (verbal descriptions) about the target person. To investigate the time course of contextual influences on cortical face processing, event-related brain potentials were investigated in response to neutral faces, which were preceded by brief verbal descriptions containing cues of affective valence (negative, neutral, positive) and self-reference (self-related vs. other-related). ERP analysis demonstrated that early and late stages of face processing are enhanced by negative and positive as well as self-relevant descriptions, although faces per se did not differ perceptually. Affective ratings of the faces confirmed these findings. Altogether, these results demonstrate for the first time both on an electrocortical and behavioral level how contextual information modifies early visual perception in a top-down manner.

The effects of an unexpected spider stimulus on skin conductance responses and eye movements: an inattentional blindness study.

Are spiders evolutionarily relevant threat cues that capture attention automatically and preattentively-also in non-fearful persons? Previous studies concerning this question did not examine responses to unexpected spider stimuli, although this is an ecologically valid situation. Therefore, we conducted an inattentional blindness (IB) experiment. While unselected participants (N = 120) were focusing their attention on a visual discrimination task, suddenly a spider or a flower image appeared. The rates of reported detection were not enhanced for spider images, and fear of spiders had no moderating influence on IB rates. However, spider pictures in general elicited greater skin conductance responses than flower pictures, and this difference was also observed for pictures that were not reported afterward. Moreover, eye movement data revealed that the spider stimulus was initially fixated more often than the flower stimulus. In sum, our findings imply that enhanced reactions to unexpected spider images are independent of fear level and include early engagement processes. Awareness of a spider picture does not seem to be a prerequisite for elevated arousal.

Training machine learning algorithms for automatic facial coding: The role of emotional facial expressions' prototypicality.

Automatic facial coding (AFC) is a promising new research tool to efficiently analyze emotional facial expressions. AFC is based on machine learning procedures to infer emotion categorization from facial movements (i.e., Action Units). State-of-the-art AFC accurately classifies intense and prototypical facial expressions, whereas it is less accurate for non-prototypical and less intense facial expressions. A potential reason might be that AFC is typically trained with standardized and prototypical facial expression inventories. Because AFC would be useful to analyze less prototypical research material as well, we set out to determine the role of prototypicality in the training material. We trained established machine learning algorithms either with standardized expressions from widely used research inventories or with unstandardized emotional facial expressions obtained in a typical laboratory setting and tested them on identical or cross-over material. All machine learning models' accuracies were comparable when trained and tested with held-out dataset from the same dataset (acc. = [83.4% to 92.5%]). Strikingly, we found a substantial drop in accuracies for models trained with the highly prototypical standardized dataset when tested in the unstandardized dataset (acc. = [52.8%; 69.8%]). However, when they were trained with unstandardized expressions and tested with standardized datasets, accuracies held up (acc. = [82.7%; 92.5%]). These findings demonstrate a strong impact of the training material's prototypicality on AFC's ability to classify emotional faces. Because AFC would be useful for analyzing emotional facial expressions in research or even naturalistic scenarios, future developments should include more naturalistic facial expressions for training. This approach will improve the generalizability of AFC to encode more naturalistic facial expressions and increase robustness for future applications of this promising technology.

Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery.

Even more than in cognitive research applications, moving fMRI to the clinic and the drug development process requires the generation of stable and reliable signal changes. The performance characteristics of the fMRI paradigm constrain experimental power and may require different study designs (e.g., crossover vs. parallel groups), yet fMRI reliability characteristics can be strongly dependent on the nature of the fMRI task. The present study investigated both within-subject and group-level reliability of a combined three-task fMRI battery targeting three systems of wide applicability in clinical and cognitive neuroscience: an emotional (face matching), a motivational (monetary reward anticipation) and a cognitive (n-back working memory) task. A group of 25 young, healthy volunteers were scanned twice on a 3T MRI scanner with a mean test-retest interval of 14.6 days. FMRI reliability was quantified using the intraclass correlation coefficient (ICC) applied at three different levels ranging from a global to a localized and fine spatial scale: (1) reliability of group-level activation maps over the whole brain and within targeted regions of interest (ROIs); (2) within-subject reliability of ROI-mean amplitudes and (3) within-subject reliability of individual voxels in the target ROIs. Results showed robust evoked activation of all three tasks in their respective target regions (emotional task=amygdala; motivational task=ventral striatum; cognitive task=right dorsolateral prefrontal cortex and parietal cortices) with high effect sizes (ES) of ROI-mean summary values (ES=1.11-1.44 for the faces task, 0.96-1.43 for the reward task, 0.83-2.58 for the n-back task). Reliability of group level activation was excellent for all three tasks with ICCs of 0.89-0.98 at the whole brain level and 0.66-0.97 within target ROIs. Within-subject reliability of ROI-mean amplitudes across sessions was fair to good for the reward task (ICCs=0.56-0.62) and, dependent on the particular ROI, also fair-to-good for the n-back task (ICCs=0.44-0.57) but lower for the faces task (ICC=-0.02-0.16). In conclusion, all three tasks are well suited to between-subject designs, including imaging genetics. When specific recommendations are followed, the n-back and reward task are also suited for within-subject designs, including pharmaco-fMRI. The present study provides task-specific fMRI reliability performance measures that will inform the optimal use, powering and design of fMRI studies using comparable tasks.

You see what you avoid: Fear of spiders and avoidance are associated with predominance of spiders in binocular rivalry.

What we see is the result of an efficient selection of cues in the visual stream. In addition to physical characteristics this process is also influenced by emotional salience of the cues. Previously, we showed in spider phobic patients that fear-related pictures gain preferential access to consciousness in binocular rivalry. We set out to replicate this in an independent unselected sample and examine the relationship of this perceptual bias with a range of symptom clusters. To this end, we recruited 79 participants with variable degrees of fear of spiders. To induce binocular rivalry, a picture of either a spider or a flower was projected to one eye, and a neutral geometric pattern to the other eye. Participants continuously reported what they saw. We correlated indices of perceptual dominance (first percept, dominance duration) with individual fear of spiders and with scores on specific symptom clusters of fear of spiders (i.e., vigilance, fixation, and avoidance coping). Overall, higher fear of spiders correlates with more predominace of spider pictures. In addition, this perceptual bias is uniquely associated with avoidance coping. Interestingly, this demonstrates that a perceptual bias, which is not intentionally controlled, is linked with an instrumental coping behavior, that has been implicated in the maintenance of pathological fear.

Time is a great healer: Peak-end memory bias in anxiety - Induced by threat of shock.

Recently, we demonstrated that the peak-end memory bias, which is well established in the context of pain, can also be observed in anxiety: Retrospective evaluations of a frightening experience are worse when peak anxiety is experienced at the end of an episode. Here, we set out to conceptually replicate and extend this finding with rigorous experimental control in a threat of shock paradigm. We induced two intensity levels of anxiety by presenting visual cues that indicated different strengths of electric stimuli. Each of the 59 participants went through one of two conditions that only differed in the order of moderate and high threat phases. As a manipulation check, orbicularis-EMG to auditory startle probes, electrodermal activity, and state anxiety confirmed the effects of the specific threat exposure. Critically, after some time had passed, participants for whom exposure had ended with high threat reported more anxiety for the entire episode than those for whom it ended with moderate threat. Moreover, they ranked their experience as more aversive when compared to other unpleasant everyday experiences. This study overcomes several previous limitations and speaks to the generalizability of the peak-end bias. Most notably, the findings bear implications for exposure therapy in clinical anxiety.

What's in a face: Automatic facial coding of untrained study participants compared to standardized inventories.

Automatic facial coding (AFC) is a novel research tool to automatically analyze emotional facial expressions. AFC can classify emotional expressions with high accuracy in standardized picture inventories of intensively posed and prototypical expressions. However, classification of facial expressions of untrained study participants is more error prone. This discrepancy requires a direct comparison between these two sources of facial expressions. To this end, 70 untrained participants were asked to express joy, anger, surprise, sadness, disgust, and fear in a typical laboratory setting. Recorded videos were scored with a well-established AFC software (FaceReader, Noldus Information Technology). These were compared with AFC measures of standardized pictures from 70 trained actors (i.e., standardized inventories). We report the probability estimates of specific emotion categories and, in addition, Action Unit (AU) profiles for each emotion. Based on this, we used a novel machine learning approach to determine the relevant AUs for each emotion, separately for both datasets. First, misclassification was more frequent for some emotions of untrained participants. Second, AU intensities were generally lower in pictures of untrained participants compared to standardized pictures for all emotions. Third, although profiles of relevant AU overlapped substantially across the two data sets, there were also substantial differences in their AU profiles. This research provides evidence that the application of AFC is not limited to standardized facial expression inventories but can also be used to code facial expressions of untrained participants in a typical laboratory setting.

Effect of Cleft Lip on Adolescent Evaluation of Faces: An Eye-Tracking Study.

PURPOSE: Adolescence is a delicate phase during life in which self-stigmatization increases and acceptance by peers becomes more important. However, little is known about how adolescents with cleft lip and/or palate experience this stage of life. Therefore, the purpose of this study was to investigate how cleft lip and/or palate (CLP) adolescents are looked at by their peers and how they look at others with/without CLP. METHODS: In this prospective, cross-sectional study 54 observers (CLP versus control) performed an eye-tracking task and gave attractiveness/ valence ratings. For this purpose, they were shown pictures of patients with and without CLP with neutral or smiling facial expressions. RESULTS: Adolescents with CLP were looked at differently compared to their unaffected peers, with shorter fixations of the eyes and longer fixations of the nose and mouth. Smiling altered the scan path toward the mouth for all faces. Contrary to the control group, adolescents with CLP tended to spend less time fixating the eyes. In the attractiveness/valence ratings, CLP adolescents were rated more negatively. CONCLUSIONS: Adolescents with cleft lip and/or palate look differently at peers and are also viewed with an alternate scan path.

Visual face perception of adults with unilateral cleft lip and palate in comparison to controls--an eye-tracking study.

OBJECTIVE: To assess how faces with a cleft lip and palate are perceived and to study how faces with and without a unilateral cleft lip and palate are viewed by individuals with a unilateral cleft lip and/or palate in comparison to nonaffected controls. DESIGN: Prospective clinical study. SETTING: Department of Orthodontics and Department of Psychology, University of Wuerzburg. PARTICIPANTS: Thirty-three participants (20 men and 13 women; mean age, 25.4 ± 6.6 years) with a unilateral cleft lip and/or palate and a control group of 30 participants (15 men and 15 women; mean age, 26.8 ± 3.4 years) were enrolled in this study. MAIN OUTCOME MEASURES: Eye movements were analyzed via an eye-tracking camera while all participants looked at pictures of faces with and without a unilateral cleft lip and palate. RESULTS: The nose and the mouth area of pictures of faces with a unilateral cleft lip and palate were looked at significantly longer by both groups. Additionally, the participants with a unilateral cleft lip and/or palate looked at faces with and without a unilateral cleft lip and palate differently, taking more time to view the nose and less time to view the eyes compared with the participants without a cleft lip and palate. CONCLUSION: When perceiving a face with a unilateral cleft lip and palate, the observer's gaze is distracted to the nose and mouth area. Moreover, participants with a unilateral cleft lip and/or palate themselves focused greater attention on those features that are anomalous on their own faces in comparison to participants without a cleft lip and palate. Specifically, this different scanpath is reflected in the cumulative duration of the eye movements as well as in the initial facial scan pattern.

Emotional sounds guide visual attention to emotional pictures: An eye-tracking study with audio-visual stimuli.

For the realm of visual cues, it has been well documented that attention is preferentially oriented toward emotionally relevant cues. Preliminary evidence suggests that emotional cues from other sensory modalities may also steer visual attention toward emotional pictures. However, more research is needed to elucidate the mechanisms that are involved. Therefore, a novel design was used to investigate whether emotional sounds promote attentional orientation toward emotional pictures. To this end, 48 participants viewed pairs of pictures with either neutral or unpleasant content in a free-viewing paradigm. In addition, neutral or unpleasant sounds were presented either on the left-hand or on the right-hand side of the monitor. Eye movements were recorded as an index of visual spatial attention toward the pictures. Most interestingly, position and valence of the sounds independently modulated visual orienting towards unpleasant pictures. For initial capture and sustained attention, orienting towards unpleasant pictures was significantly enhanced when any sound was heard on the same side as the unpleasant picture. In addition, unpleasant sounds (irrespective of the side) boosted leftward bias of initial attention toward emotionally congruent pictures. Taken together, this study clearly shows that emotional auditory cues guide visual spatial allocation of attention specifically to emotionally congruent pictures. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Girls' Stuff? Maternal Gender Stereotypes and Their Daughters' Fear.

One of the most robust findings in psychopathology is the fact that specific phobias are more prevalent in women than in men. Although there are several theoretical accounts for biological and social contributions to this gender difference, empirical data are surprisingly limited. Interestingly, there is evidence that individuals with stereotypical feminine characteristics are more fearful than those with stereotypical masculine characteristics; this is beyond biological sex. Because gender role stereotypes are reinforced by parental behavior, we aimed to examine the relationship of maternal gender stereotypes and children's fear. Dyads of 38 mothers and their daughters (between ages 6 and 10) were included. We assessed maternal implicit and explicit gender stereotypes as well as their daughters' self-reported general fearfulness, specific fear of snakes, and approach behavior toward a living snake. First, mothers' fear of snakes significantly correlated with their daughters' fear of snakes. Second, mothers' gender stereotypes significantly correlated with their daughters' self-reported fear. Specifically, maternal implicit gender stereotypes were associated with daughters' fear of snakes and fear ratings in response to the snake. Moreover, in children, self-reported fear correlated with avoidance of the fear-relevant animal. Together, these results provide first evidence for a potential role of parental gender stereotypes in the development and maintenance of fear in their offspring.

Impact of facial asymmetry in visual perception: a 3-dimensional data analysis.

INTRODUCTION: The aim of this controlled study was to analyze the degree and localization of 3-dimensional (3D) facial asymmetry in adult patients with cleft lip and palate (CLP) compared with a control group and its impact on the visual perception of faces. METHODS: The degree of 3D asymmetry was analyzed with a novel method without landmarks in 18 adults with complete unilateral CLP and 18 adults without congenital anomalies. Furthermore, the CLP and control faces were rated for appearance, symmetry, and facial expression by 30 participants. RESULTS: The results showed that adults with CLP had significantly greater asymmetry in their facial soft tissues compared with the control group. Moreover, the lower face, and particularly the midface, had greater asymmetry in the CLP patients. The perceptual ratings showed that adults with CLP were judged much more negatively than those in the control group. CONCLUSIONS: With sophisticated 3D analysis, the real morphology of a face can be calculated and asymmetric regions precisely identified. The greatest asymmetry in CLP patients is in the midface. These results underline the importance of symmetry in the perception of faces. In general, the greater the facial asymmetry near the midline of the face, the more negative the evaluation of the face in direct face-to-face interactions.

Read My Face: Automatic Facial Coding Versus Psychophysiological Indicators of Emotional Valence and Arousal.

Facial expressions provide insight into a person's emotional experience. To automatically decode these expressions has been made possible by tremendous progress in the field of computer vision. Researchers are now able to decode emotional facial expressions with impressive accuracy in standardized images of prototypical basic emotions. We tested the sensitivity of a well-established automatic facial coding software program to detect spontaneous emotional reactions in individuals responding to emotional pictures. We compared automatically generated scores for valence and arousal of the Facereader (FR; Noldus Information Technology) with the current psychophysiological gold standard of measuring emotional valence (Facial Electromyography, EMG) and arousal (Skin Conductance, SC). We recorded physiological and behavioral measurements of 43 healthy participants while they looked at pleasant, unpleasant, or neutral scenes. When viewing pleasant pictures, FR Valence and EMG were both comparably sensitive. However, for unpleasant pictures, FR Valence showed an expected negative shift, but the signal differentiated not well between responses to neutral and unpleasant stimuli, that were distinguishable with EMG. Furthermore, FR Arousal values had a stronger correlation with self-reported valence than with arousal while SC was sensitive and specifically associated with self-reported arousal. This is the first study to systematically compare FR measurement of spontaneous emotional reactions to standardized emotional images with established psychophysiological measurement tools. This novel technology has yet to make strides to surpass the sensitivity of established psychophysiological measures. However, it provides a promising new measurement technique for non-contact assessment of emotional responses.

Toward and away from spiders: eye-movements in spider-fearful participants.

Highly fearful individuals show attentional biases toward threat. However, it is still unclear whether initial engagement of attention toward threat or difficulties to disengage from threat is the underlying mechanism. We used eye-tracking to investigate how quickly fear-relevant pictures are identified and whether they distract from the allocation of attention toward neutral targets. Pairs of fear-relevant and neutral pictures were presented to 18 high and 16 low spider-fearful participants. They were instructed to either fixate on a target or to fixate on the opposite picture, while eye movements were monitored continuously. Overall, fear-relevant targets were fixated more quickly than neutral targets. Spider-fearful participants had longer latencies when they had to identify the fear-relevant but fixate the neutral picture. Thus, attentional allocation toward threat was not specifically enhanced in fearful participants. Instead, they had difficulties to disengage attention from fear-relevant information. This disengagement deficit could be a cause, a correlate, or the result of phobic fear.

Attention and amygdala activity: an fMRI study with spider pictures in spider phobia.

Facilitated detection of threatening visual cues is thought to be adaptive. In theory, detection of threat cues should activate the amygdala independently from allocation of attention. However, previous studies using emotional facial expressions as well as phobic cues yielded contradictory results. We used fMRI to examine whether the allocation of attention to components of superimposed spider and bird displays modulates amygdala activation. Nineteen spider-phobic women were instructed to identify either a moving or a stationary animal in briefly presented double-exposure displays. Amygdala activation followed a dose-response relationship: Compared to congruent neutral displays (two birds), amygdala activation was most pronounced in response to congruent phobic displays (two spiders) and less but still significant in response to mixed displays (spider and bird) when attention was focused on the phobic component. When attention was focused on the neutral component, mixed displays did not result in significant amygdala activation. This was confirmed in a significant parametric graduation of the amygdala activation in the order of congruent phobic displays, mixed displays with attention focus on the spider, mixed displays with focus on the bird and congruent neutral displays. These results challenge the notion that amygdala activation in response to briefly presented phobic cues is independent from attention.

When spiders appear suddenly: spider-phobic patients are distracted by task-irrelevant spiders.

Fear is thought to facilitate the detection of threatening stimuli. Few studies have examined the effects of task-irrelevant phobic cues in search tasks that do not involve semantic categorization. In a combined reaction time and eye-tracking experiment we investigated whether peripheral visual cues capture initial attention and distract from the execution of goal-directed eye movements. Twenty-one spider-phobic patients and 21 control participants were instructed to search for a color singleton while ignoring task-irrelevant abrupt-onset distractors which contained either a small picture of a spider (phobic), a flower (non-phobic, but similar to spiders in shape), a mushroom (non-phobic, and not similar to spiders in shape), or no picture. As expected, patients' reaction times were longer on trials with spider distractors. However, eye movements revealed that this was not due to attentional capture by spider distractors; patients more often fixated on all distractors with pictures, but their reaction times were delayed by longer fixation durations on spider distractors. These data do not support automatic capture of attention by phobic cues but suggest that phobic patients fail to disengage attention from spiders.

Threat vs. Threat: Attention to Fear-Related Animals and Threatening Faces.

It is generally thought to be adaptive that fear relevant stimuli in the environment can capture and hold our attention; and in psychopathology attentional allocation is thought to be cue-specific. Such hypervigilance toward threatening cues or difficulty to disengage attention from threat has been demonstrated for a variety of stimuli, for example, toward evolutionary prepared animals or toward socially relevant facial expressions. Usually, specific stimuli have been examined in individuals with particular fears (e.g., animals in animal fearful and faces in socially fearful participants). However, different kinds of stimuli are rarely examined in one study. Thus, it is unknown how different categories of threatening stimuli compete for attention and how specific kinds of fears modulate these attentional processes. In this study, we used a free viewing paradigm: pairs of pictures with threat-related content (spiders or angry faces) or neutral content (butterflies or neutral faces) were presented side by side (i.e., spiders and angry faces, angry and neutral faces, spiders and butterflies, butterflies and neutral faces). Eye-movements were recorded while spider fearful, socially anxious, or non-anxious participants viewed the picture pairs. Results generally replicate the finding that unpleasant pictures more effectively capture attention in the beginning of a trial compared to neutral pictures. This effect was more pronounced in spider fearful participants: the higher the fear the quicker they were in looking at spiders. This was not the case for high socially anxious participants and pictures of angry faces. Interestingly, when presented next to each other, there was no preference in initial orientation for either spiders or angry faces. However, neutral faces were looked at more quickly than butterflies. Regarding sustained attention, we found no general preference for unpleasant pictures compared to neutral pictures.

Here is looking at you: emotional faces predominate in binocular rivalry.

Two incompatible pictures compete for perceptual dominance when they are presented to one eye each. This so-called binocular rivalry results in an alternation of dominant and suppressed percepts. In accordance with current theories of emotion processing, the authors' previous research has suggested that emotionally arousing pictures predominate in this perceptual process. Three experiments were run with pictures of emotional facial expressions that are known to induce emotions while being well controlled in terms of physical characteristics. In Experiment 1, photographs of emotional and neutral facial expressions were presented of the same actor to minimize physical differences. In Experiment 2, schematic emotional expressions were presented to further eliminate low-level differences. In Experiment 3, a probe-detection task was conducted to control for possible response-biases. Together, these data clearly demonstrate that emotional facial expressions predominate over neutral expressions; they are more often the first percept and they are perceived for longer durations. This is not caused by physical stimulus properties or by response-biases. This novel approach supports that emotionally significant visual stimuli are preferentially perceived.