Psychophysiological responses to eye contact with a humanoid robot: Impact of perceived intentionality.

Eye contact with a social robot has been shown to elicit similar psychophysiological responses to eye contact with another human. However, it is becoming increasingly clear that the attention- and affect-related psychophysiological responses differentiate between direct (toward the observer) and averted gaze mainly when viewing embodied faces that are capable of social interaction, whereas pictorial or pre-recorded stimuli have no such capability. It has been suggested that genuine eye contact, as indicated by the differential psychophysiological responses to direct and averted gaze, requires a feeling of being watched by another mind. Therefore, we measured event-related potentials (N170 and frontal P300) with EEG, facial electromyography, skin conductance, and heart rate deceleration responses to seeing a humanoid robot's direct versus averted gaze, while manipulating the impression of the robot's intentionality. The results showed that the N170 and the facial zygomatic responses were greater to direct than to averted gaze of the robot, and independent of the robot's intentionality, whereas the frontal P300 responses were more positive to direct than to averted gaze only when the robot appeared intentional. The study provides further evidence that the gaze behavior of a social robot elicits attentional and affective responses and adds that the robot's seemingly autonomous social behavior plays an important role in eliciting higher-level socio-cognitive processing.

Neural specialization to human faces at the age of 7 months.

Sensitivity to human faces has been suggested to be an early emerging capacity that promotes social interaction. However, the developmental processes that lead to cortical specialization to faces has remained unclear. The current study investigated both cortical sensitivity and categorical specificity through event-related potentials (ERPs) previously implicated in face processing in 7-month-old infants (N290) and adults (N170). Using a category-specific repetition/adaptation paradigm, cortical specificity to human faces, or control stimuli (cat faces), was operationalized as changes in ERP amplitude between conditions where a face probe was alternated with categorically similar or dissimilar adaptors. In adults, increased N170 for human vs. cat faces and category-specific release from adaptation for face probes alternated with cat adaptors was found. In infants, a larger N290 was found for cat vs. human probes. Category-specific repetition effects were also found in infant N290 and the P1-N290 peak-to-peak response where latter indicated category-specific release from adaptation for human face probes resembling that found in adults. The results suggest cortical specificity to human faces during the first year of life. Encoding of unfamiliar cat stimuli might explain N290 amplification found in infants.

Individual differences in pupil dilation to others' emotional and neutral eyes with varying pupil sizes.

Sensitivity to others' emotional signals is an important factor for social interaction. While many studies of emotional reactivity focus on facial emotional expressions, signals such as pupil dilation which can indicate arousal, may also affect observers. For example, observers' pupils dilate when viewing someone with dilated pupils, so-called pupillary contagion. Yet it is unclear how pupil size and emotional expression interact as signals. Further, examining individual differences in emotional reactivity to others can shed light on its mechanisms and potential outcomes. In the current study, adults' (N = 453) pupil size was assessed while they viewed images of the eye region of individuals varying in emotional expression (neutral, happy, sad, fearful, angry) and pupil size (large, medium, small). Participants showed pupillary contagion regardless of the emotional expression. Individual differences in demographics (gender, age, socioeconomic status) and psychosocial factors (anxiety, depression, sleep problems) were also examined, yet the only factor related to pupillary contagion was socioeconomic status, with higher socioeconomic status predicting less pupillary contagion for emotionally-neutral stimuli. The results suggest that while pupillary contagion is a robust phenomenon, it can vary meaningfully across individuals.

Investigating pupillometry to detect emotional regulation difficulties in post-traumatic stress disorder.

OBJECTIVE: Individuals with posttraumatic stress disorder (PTSD) have been found to exhibit emotional regulation difficulties. However, the specific neural mechanisms that underlie these difficulties remain understudied. This study aimed to use pupillometry as an index function of parasympathetic nervous system activation, to investigate the mechanisms underlying emotional regulation difficulties in individuals with PTSD. METHOD: A total of 87 trauma-exposed mothers (34 with PTSD and 53 non-PTSD controls) completed an eye tracking assessment in which pupillary dilation in response to emotionally valenced stimuli was measured. The participants also completed two self-report measures of emotional regulation, namely the Difficulties in Emotional Regulation Scale and the Emotional Regulations Questionnaire. Linear mixed-effect modelling was used to assess potential group differences. RESULTS: The PTSD group exhibited increased pupillary dilation to positively valenced stimuli compared to the non-PTSD group. However, no significant associations between the self-report measures and pupillary response to emotionally valenced stimuli were found. CONCLUSION: Increased pupillary dilation in PTSD may reflect impaired parasympathetic nervous system processes. The lack of association of these measures with self-reported emotion regulation may suggest reporting biases. Larger studies with more generalised populations are required to consolidate these preliminary findings.

Pupillary and Attentional Responses to Infant Facial Expressions in Mothers Across Socioeconomic Variations.

Maternal responses to infant facial expressions were examined in two socioeconomically diverse samples of South African mothers (Study I, N = 111; and Study II, N = 214; age: 17-44 years) using pupil and gaze tracking. Study I showed increased pupil response to infant distress expressions in groups recruited from private as compared to public maternity clinics, possibly reflecting underlying differences in socioeconomic status (SES) across the groups. Study II, sampling uniformly low-SES neighborhoods, found increased pupil dilation and faster orientation to expressions of infant distress, but only in the highest income group. These results are consistent with maternal physiological and attentional sensitivity to infant distress cues but challenge the universality of this sensitivity across socioeconomic diversity.

Attachment security and cortical responses to fearful faces in infants.

The present study measured event-related potentials (ERPs) to investigate whether cortical responses to facial expressions of fear are associated with the development of secure and insecure patterns of infant-mother attachment during the first year. Based on previous findings showing reduced attentional biases to fearful faces in infants with insecure and disorganized attachment, we hypothesized that insecure and disorganized attachment would be associated with reduced ERP differentiation of fearful from non-fearful faces. ERPs to facial expressions were measured at 7 months of age and attachment was assessed at 14 months of age with the Strange Situation Procedure (n = 61). Occipitotemporal face-sensitive ERP responses particularly in the time range of the N290 component were related to attachment security at 14 months. Only securely attached infants showed age-typical cortical discrimination of fearful from non-fearful faces at 7 months, whereas a similar pattern of ERP responses was not observed in infants with insecure and disorganized attachment. These results add to previous findings by suggesting that patterns of secure and insecure infant attachment are related to early-emerging differences in the perceptual processing of facial emotions, which could have implications for the development of social competence.

Atypical Pattern of Frontal EEG Asymmetry for Direct Gaze in Young Children with Autism Spectrum Disorder.

This study examined approach-motivation related brain activity (frontal electroencephalogram [EEG] asymmetry) in response to direct and averted gaze in 3- to 6-year-old typically developing (TD) children, children with autism spectrum disorder (ASD), and those with intellectual disability (ID). We found that, in TD children, direct gaze elicited greater approach-related frontal EEG activity than did downcast gaze. This pattern of activity was in contrast to that observed in children with ASD, who showed greater approach-related activity in response to downcast gaze than to direct gaze. ID children did not differ in their responses to different gaze conditions. These findings indicate that another person's direct gaze does not elicit approach-motivation related brain activity in young children with ASD.

Infants' attention bias to faces as an early marker of social development.

Infants have a strong tendency to look at faces. We examined individual variations in this attentional bias in 7-month-old infants by using a face-distractor competition paradigm and tested in a longitudinal sample whether these variations were associated with outcomes reflecting social behavior at 24 and 48 months of age (i.e., spontaneous helping, emotion understanding, mentalizing, and callous-unemotional traits; N = 100-138). The results showed a robust and distinct attention bias to faces at 7 months, particularly when faces were displaying a fearful expression. This bias declined between 7 and 24 months and there were no significant correlations in attention dwell times between 7 and 24 months of age. Variations in attention to faces at 7 months were not associated with emotion understanding or mentalizing abilities at 48 months of age, but increased attention to faces at 7 months (regardless of facial expression) was related to more frequent helping responses at 24 months and reduced callous-unemotional traits at 48 months of age. Thus, while the results fail to associate infants' face bias with later-emerging emotion understanding and mentalizing capacities, they are consistent with a model whereby increased attention to faces in infancy is linked with the development of affective empathy and responsivity to others' needs.

Mothers' pupillary responses to infant facial expressions.

BACKGROUND: Human parental care relies heavily on the ability to monitor and respond to a child's affective states. The current study examined pupil diameter as a potential physiological index of mothers' affective response to infant facial expressions. METHODS: Pupillary time-series were measured from 86 mothers of young infants in response to an array of photographic infant faces falling into four emotive categories based on valence (positive vs. negative) and arousal (mild vs. strong). RESULTS: Pupil dilation was highly sensitive to the valence of facial expressions, being larger for negative vs. positive facial expressions. A separate control experiment with luminance-matched non-face stimuli indicated that the valence effect was specific to facial expressions and cannot be explained by luminance confounds. Pupil response was not sensitive to the arousal level of facial expressions. CONCLUSIONS: The results show the feasibility of using pupil diameter as a marker of mothers' affective responses to ecologically valid infant stimuli and point to a particularly prompt maternal response to infant distress cues.

Perceptual interference processing in preschool children, with and without prenatal exposure to selective serotonin reuptake inhibitors.

BACKGROUND: Prenatal exposure to maternal depression, with or without maternal medical antidepressant treatment, may pose a risk to the child's cognitive and behavioral development. Targeting one of the core functions of behavioral regulation, we investigated both behavioral and neural indices of interference suppression in both exposed and control participants at preschool age. METHODS: Children (N = 80, M = 68.60 months, SD = 5.57) with prenatal exposure to maternal depression with (SSRI, N = 21) and without (DEP, N = 33) antidepressant treatment were tested together with unexposed children (CON, N = 26) on a behavioral flanker task while recording event-related potentials (ERPs). ERPs were extracted from trials with congruent/incongruent flankers and speeded/slow response times (RT). FINDINGS: Effects of flanker congruence were found in both behavioral indices and the late slow wave ERP (LSW, 500-800 ms), across all groups in the expected directions. Further, increased amplitude of the N2 (350-450 ms) and the LSW potential was found in trials with speeded vs slow RT. Interestingly, the parietal N2 in speeded trials showed decreased latency among children in the CON group but not among the other children. No other effects of group on ERP or behavioral measures were found. CONCLUSION: While interference effects were evident in behavioral and ERP measures, prenatal exposure to SSRIs and DEP was not directly associated with abilities of interference suppression. However, RT in the flanker task was associated with N2 and LSW potentials. Importantly, the interaction between RT and participant group upon parietal N2 latency may suggest effects of prenatal exposure on neural efficiency.

Transcutaneous vagus nerve stimulation modulates tinnitus-related beta- and gamma-band activity.

OBJECTIVES: The ability of a treatment method to interfere with tinnitus-related neural activity patterns, such as cortical gamma rhythms, has been suggested to indicate its potential in relieving tinnitus. Therapeutic modulation of gamma-band oscillations with vagus nerve stimulation has been recently reported in epileptic patients. The aim of this study was to investigate the effects of transcutaneous vagus nerve stimulation (tVNS) on neural oscillatory patterns. DESIGN: We calculated the power spectral density and synchrony of magnetoencephalography recordings during auditory stimulation in seven tinnitus patients and eight normal-hearing control subjects. Comparisons between subject groups were performed to reveal electrophysiological markers of tinnitus. tVNS-specific effects within each group were studied by comparing recording blocks with and without tVNS. We also investigated the correlation of each measure with individual ratings of tinnitus distress, as measured by the tinnitus handicap inventory questionnaire. RESULTS: Tinnitus patients differed from controls in the baseline condition (no tVNS applied), measured by both cortical oscillatory power and synchronization, particularly at beta and gamma frequencies. Importantly, we found tVNS-induced changes in synchrony, correlating strongly with tinnitus handicap inventory scores, at whole-head beta-band (r = -0.857, p = 0.007), whole-head gamma-band (r = -0.952, p = 0.0003), and frontal gamma-band (r = -0.952, p = 0.0003). CONCLUSIONS: We conclude that tVNS was successful in modulating tinnitus-related beta- and gamma-band activity and thus could have potential as a treatment method for tinnitus.

Widely applicable MATLAB routines for automated analysis of saccadic reaction times.

Saccadic reaction time (SRT) is a widely used dependent variable in eye-tracking studies of human cognition and its disorders. SRTs are also frequently measured in studies with special populations, such as infants and young children, who are limited in their ability to follow verbal instructions and remain in a stable position over time. In this article, we describe a library of MATLAB routines (Mathworks, Natick, MA) that are designed to (1) enable completely automated implementation of SRT analysis for multiple data sets and (2) cope with the unique challenges of analyzing SRTs from eye-tracking data collected from poorly cooperating participants. The library includes preprocessing and SRT analysis routines. The preprocessing routines (i.e., moving median filter and interpolation) are designed to remove technical artifacts and missing samples from raw eye-tracking data. The SRTs are detected by a simple algorithm that identifies the last point of gaze in the area of interest, but, critically, the extracted SRTs are further subjected to a number of postanalysis verification checks to exclude values contaminated by artifacts. Example analyses of data from 5- to 11-month-old infants demonstrated that SRTs extracted with the proposed routines were in high agreement with SRTs obtained manually from video records, robust against potential sources of artifact, and exhibited moderate to high test-retest stability. We propose that the present library has wide utility in standardizing and automating SRT-based cognitive testing in various populations. The MATLAB routines are open source and can be downloaded from http://www.uta.fi/med/icl/methods.html .

Developmental precursors of social brain networks: the emergence of attentional and cortical sensitivity to facial expressions in 5 to 7 months old infants.

Biases in attention towards facial cues during infancy may have an important role in the development of social brain networks. The current study used a longitudinal design to examine the stability of infants' attentional biases towards facial expressions and to elucidate how these biases relate to emerging cortical sensitivity to facial expressions. Event-related potential (ERP) and attention disengagement data were acquired in response to the presentation of fearful, happy, neutral, and phase-scrambled face stimuli from the same infants at 5 and 7 months of age. The tendency to disengage from faces was highly consistent across both ages. However, the modulation of this behavior by fearful facial expressions was uncorrelated between 5 and 7 months. In the ERP data, fear-sensitive activity was observed over posterior scalp regions, starting at the latency of the N290 wave. The scalp distribution of this sensitivity to fear in ERPs was dissociable from the topography of face-sensitive modulation within the same latency range. While attentional bias scores were independent of co-registered ERPs, attention bias towards fearful faces at 5 months of age predicted the fear-sensitivity in ERPs at 7 months of age. The current results suggest that the attention bias towards fear could be involved in the developmental tuning of cortical networks for social signals of emotion.

Dynamic eye tracking based metrics for infant gaze patterns in the face-distractor competition paradigm.

OBJECTIVE: To develop new standardized eye tracking based measures and metrics for infants' gaze dynamics in the face-distractor competition paradigm. METHOD: Eye tracking data were collected from two samples of healthy 7-month-old (total n = 45), as well as one sample of 5-month-old infants (n = 22) in a paradigm with a picture of a face or a non-face pattern as a central stimulus, and a geometric shape as a lateral stimulus. The data were analyzed by using conventional measures of infants' initial disengagement from the central to the lateral stimulus (i.e., saccadic reaction time and probability) and, additionally, novel measures reflecting infants gaze dynamics after the initial disengagement (i.e., cumulative allocation of attention to the central vs. peripheral stimulus). RESULTS: The results showed that the initial saccade away from the centrally presented stimulus is followed by a rapid re-engagement of attention with the central stimulus, leading to cumulative preference for the central stimulus over the lateral stimulus over time. This pattern tended to be stronger for salient facial expressions as compared to non-face patterns, was replicable across two independent samples of 7-month-old infants, and differentiated between 7 and 5 month-old infants. CONCLUSION: The results suggest that eye tracking based assessments of infants' cumulative preference for faces over time can be readily parameterized and standardized, and may provide valuable techniques for future studies examining normative developmental changes in preference for social signals. SIGNIFICANCE: Standardized measures of early developing face preferences may have potential to become surrogate biomarkers of neurocognitive and social development.

Motherhood and oxytocin receptor genetic variation are associated with selective changes in electrocortical responses to infant facial expressions.

Recent studies suggest that parental caregiving is associated with adaptive changes in neurocognitive responses to emotional cues and oxytocin function, possibly reflecting the increased need of parents to monitor infants' emotional states. In the current study, we investigated whether the changes associated with motherhood and oxytocin receptor genetic variation rs53576 are specific to the processing of infant cues as opposed to a more general increase in responsiveness to emotional cues. We measured event-related brain potentials (ERPs) and behavioral recognition responses from mothers of young infants (n = 48) and nulliparous females (n = 46) to infant and adult faces displaying strong and mild intensity emotional expressions. Mothers and GG allele carriers of the OXTR gene showed an early latency (∼100 ms) differential frontal ERP response to strong intensity facial expressions, and mothers also showed modulation of the posterior EPN waveform by negative valence. The early frontal ERP modulation was associated with faster emotion recognition performance across participants. Most importantly, these effects were highly specific to infant facial expressions. The results point to a dissociable neurocognitive system that is involved in monitoring infants' emotional cues and may be important in supporting parental caregiving in humans.

A graphical user interface for infant ERP analysis.

Recording of event-related potentials (ERPs) is one of the best-suited technologies for examining brain function in human infants. Yet the existing software packages are not optimized for the unique requirements of analyzing artifact-prone ERP data from infants. We developed a new graphical user interface that enables an efficient implementation of a two-stage approach to the analysis of infant ERPs. In the first stage, video records of infant behavior are synchronized with ERPs at the level of individual trials to reject epochs with noncompliant behavior and other artifacts. In the second stage, the interface calls MATLAB and EEGLAB (Delorme & Makeig, Journal of Neuroscience Methods 134(1):9-21, 2004) functions for further preprocessing of the ERP signal itself (i.e., filtering, artifact removal, interpolation, and rereferencing). Finally, methods are included for data visualization and analysis by using bootstrapped group averages. Analyses of simulated and real EEG data demonstrated that the proposed approach can be effectively used to establish task compliance, remove various types of artifacts, and perform representative visualizations and statistical comparisons of ERPs. The interface is available for download from http://www.uta.fi/med/icl/methods/eeg.html in a format that is widely applicable to ERP studies with special populations and open for further editing by users.

Regulatory variant of the TPH2 gene and early life stress are associated with heightened attention to social signals of fear in infants.

BACKGROUND: Cross-species evidence suggests that genetic and experiential factors act early in development to establish individual emotional traits, but little is known about the mechanisms that emerge during this period to mediate long-term outcomes. Here, we tested the hypothesis that known genetic and environmental risk conditions may heighten infants' natural tendency to attend to threat-alerting stimuli, resulting in a cognitive bias that may contribute to emotional vulnerability. METHODS: Data from two samples of 5-7-month-old infants (N = 139) were used to examine whether established candidate variations in the serotonin-system genes, i.e., TPH2 SNP rs4570625 (-703 G/T) and HTR1A SNP rs6295 (-1019 G/C), and early rearing condition (maternal stress and depressive symptoms) are associated with alterations in infants' attention to facial expressions. Infants were tested with a paradigm that assesses the ability to disengage attention from a centrally presented stimulus (a nonface control stimulus or a neutral, happy, or fearful facial expression) toward the location of a new stimulus in the visual periphery (a geometric shape). RESULTS: TPH2 -703 T-carrier genotype (i.e., TT homozygotes and heterozygotes), presence of maternal stress and depressive symptoms, and a combination of the T-carrier genotype and maternal depressive symptoms were associated with a relatively greater difficulty disengaging attention from fearful facial expressions. No associations were found with infants' temperamental traits. CONCLUSIONS: Alterations in infants' natural attentional bias toward fearful facial expressions may emerge prior to the manifestation of emotional and social behaviors and provide a sensitive marker of early emotional development.

Detection of shouted speech in noise: human and machine.

High vocal effort has characteristic acoustic effects on speech. This study focuses on the utilization of this information by human listeners and a machine-based detection system in the task of detecting shouted speech in the presence of noise. Both female and male speakers read Finnish sentences using normal and shouted voice in controlled conditions, with the sound pressure level recorded. The speech material was artificially corrupted by noise and supplemented with pure noise. The human performance level was statistically evaluated by a listening test, where the subjects labeled noisy samples according to whether shouting was heard or not. A Bayesian detection system was constructed and statistically evaluated. Its performance was compared against that of human listeners, substituting different spectrum analysis methods in the feature extraction stage. Using features capable of taking into account the spectral fine structure (i.e., the fundamental frequency and its harmonics), the machine reached the detection level of humans even in the noisiest conditions. In the listening test, male listeners detected shouted speech significantly better than female listeners, especially with speakers making a smaller vocal effort increase for shouting.

Signal-to-noise ratio adaptive post-filtering method for intelligibility enhancement of telephone speech.

Post-filtering can be utilized to improve the quality and intelligibility of telephone speech. Previous studies have shown that energy reallocation with a high-pass type filter works effectively in improving the intelligibility of speech in difficult noise conditions. The present study introduces a signal-to-noise ratio adaptive post-filtering method that utilizes energy reallocation to transfer energy from the first formant to higher frequencies. The proposed method adapts to the level of the background noise so that, in favorable noise conditions, the post-filter has a flat frequency response and the effect of the post-filtering is increased as the level of the ambient noise increases. The performance of the proposed method is compared with a similar post-filtering algorithm and unprocessed speech in subjective listening tests which evaluate both intelligibility and listener preference. The results indicate that both of the post-filtering methods maintain the quality of speech in negligible noise conditions and are able to provide intelligibility improvement over unprocessed speech in adverse noise conditions. Furthermore, the proposed post-filtering algorithm performs better than the other post-filtering method under evaluation in moderate to difficult noise conditions, where intelligibility improvement is mostly required.

Conversational quality evaluation of artificial bandwidth extension of telephone speech.

Artificial bandwidth extension methods have been developed to improve the quality and intelligibility of narrowband telephone speech and to reduce the difference with wideband speech. Such methods have commonly been evaluated with objective measures or subjective listening-only tests, but conversational evaluations have been rare. This article presents a conversational evaluation of two methods for the artificial bandwidth extension of telephone speech. Bandwidth-extended narrowband speech is compared with narrowband and wideband speech in a test setting including a simulated telephone connection, realistic conversation tasks, and various background noise conditions. The responses of the subjects indicate that speech processed with one of the methods is preferred to narrowband speech in noise, but wideband speech is superior to both narrowband and bandwidth-extended speech. Bandwidth extension was found to be beneficial for telephone conversation in noisy listening conditions.