Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach.

Surface facial electromyography (EMG) is commonly used to detect emotions from subtle facial expressions. Although there are established procedures for collecting EMG data and some aspects of their processing, there is little agreement among researchers about the optimal way to process the EMG signal, so that the study-unrelated variability (noise) is removed, and the emotion-related variability is best detected. The aim of the current paper was to establish an optimal processing pipeline for EMG data for identifying emotional expressions in facial muscles. We identified the most common processing steps from existing literature and created 72 processing pipelines that represented all the different processing choices. We applied these pipelines to a previously published dataset from a facial mimicry experiment, where 100 adult participants observed happy and sad facial expressions, whilst the activity of their facial muscles, zygomaticus major and corrugator supercilii, was recorded with EMG. We used a resampling approach and subsets of the original data to investigate the effect and robustness of different processing choices on the performance of a logistic regression model that predicted the mimicked emotion (happy/sad) from the EMG signal. In addition, we used a random forest model to identify the most important processing steps for the sensitivity of the logistic regression model. Three processing steps were found to be most impactful: baseline correction, standardisation within muscles, and standardisation within subjects. The chosen feature of interest and the signal averaging had little influence on the sensitivity to the effect. We recommend an optimal processing pipeline, share our code and data, and provide a step-by-step walkthrough for researchers.

Self-projection in early childhood: No evidence for a common underpinning of episodic memory, episodic future thinking, theory of mind, and spatial navigation.

Buckner and Carroll [Trends in Cognitive Sciences (2007), Vol. 11, pp. 49-57] argued that episodic memory (EM), episodic future thinking (EFT), theory of mind (ToM), and spatial navigation all build on the same mental mechanism-self-projection, that is, the ability to disengage from the immediate present and shift perspective to alternative temporal, mental, or spatial situations. Developmental studies indeed show that all four abilities undergo profound developmental changes around 4 years of age, and there are first indications of developmental interrelations between some of the abilities. However, strong evidence for the self-projection account, namely that all four abilities are interrelated in their emergence during early childhood, is still lacking. To thoroughly investigate the self-projection hypothesis, we tested 151 4-year-old children on 12 different tasks assessing their EM, EFT, ToM, and spatial navigation skills (3 tasks per ability). Structural equation modeling under maximum likelihood estimation was conducted on a final sample of 144 children to evaluate a model with the 12 tasks as indicators and self-projection as the latent factor. The model showed a very good fit to the data. However, the factor loadings, indicating the strength of association between the latent factor and the indicators, were very low-which speaks against the validity of the measurement model. In summary, the results do not support the assumption of a common latent factor underlying the various abilities EM, EFT, ToM, and spatial navigation. Implications of our results for the self-projection account and possible related theoretical and methodological challenges are discussed.

Eating in Synch: An investigation of parent-infant behaviour coordination during feeding interactions.

During feeding, parents have been described to move their mouth as if they were eating themselves. Such matching of behaviours between parents and their infants during face-to-face interactions represents an example of behavioural synchrony. To date, however, the function of these synchronous eating-like mouth movements by the caregiver remains unexplored. To address this question, two competing hypotheses were tested: 1) the instructional hypothesis proposing that parents make eating-like mouth movements, such as opening and closing their mouth, to demonstrate to their infants what they need to do; 2) the mimicry hypothesis suggesting that parents imitate their infant's mouth actions to enhance affiliation. To test these hypotheses, we examined the temporal dependencies between parents' and infants' mouth movements. We reasoned that parents' mouth movements would occur before their infants' if they serve an instructional purpose, but that they would happen after, if parents mimic their infants. Additionally, we expected that parents' matching mouth movements would be more likely when their infants looked at them in both cases. To examine these hypotheses, fifteen caregivers were observed as they were feeding their six-month-old infants. Time-window sequential analysis was conducted to quantify how likely parents were to display mouth opening and closing before or after their infants did. The results revealed that parents' mouth movements were more likely to follow infants' movements and are thus in line with the mimicry hypothesis. Interestingly, these mouth movements of parents were independent of infant's gaze.

Neural processing of self-produced and externally generated events in 3-month-old infants.

Did I make that sound? Differentiating whether sensory events are caused by us or the environment is pivotal for our sense of agency. Adults can predict the sensory effects of their actions, which results in attenuated processing of self-produced events compared with externally generated events. Yet, little is known about whether young infants predict and discriminate self-produced events from externally produced events. Using electroencephalography (EEG), 3-month-olds' neural response to the same audiovisual stimulus was compared between a Self-produced condition and externally generated conditions with predictable timing (External-Regular) and irregular timing (External-Irregular). We hypothesized that if 3-month-olds predict self-produced events, their event-related potentials should be smallest for the Self-produced condition, strongest for the External-Irregular condition, and in between for the External-Regular condition. Cluster-based permutation tests indicated a more positive deflection (300-470 ms) for irregular stimuli compared with regular stimuli over the vertex. Contrasting the Self-produced and External-Irregular conditions showed a statistical trend within the same time window. Although not fully conclusive, this might suggest the emerging differentiation between self-produced and less predictable external events. However, there was no statistical evidence that infants differentiated self-produced events from temporally predictable external events. Our findings shed light on the emerging sense of agency and suggest that 3-month-olds are transitioning toward predicting and discriminating the consequences of their actions.

Infants tailor their attention to maximize learning.

Infants' remarkable learning abilities allow them to rapidly acquire many complex skills. It has been suggested that infants achieve this learning by optimally allocating their attention to relevant stimuli in the environment, but the underlying mechanisms remain poorly understood. Here, we modeled infants' looking behavior during a learning task through an ideal learner that quantified the informational structure of environmental stimuli. We show that saccadic latencies, looking time, and time spent engaged with a stimulus sequence are explained by the properties of the learning environments, including the level of surprise of the stimulus, overall predictability of the environment, and progress in learning the environmental structure. These findings reveal the factors that shape infants' advanced learning, emphasizing their predisposition to seek out stimuli that maximize learning.

Nine-month-old infants update their predictive models of a changing environment.

Humans generate internal models of their environment to predict events in the world. As the environments change, our brains adjust to these changes by updating their internal models. Here, we investigated whether and how 9-month-old infants differentially update their models to represent a dynamic environment. Infants observed a predictable sequence of stimuli, which were interrupted by two types of cues. Following the update cue, the pattern was altered, thus, infants were expected to update their predictions for the upcoming stimuli. Because the pattern remained the same after the no-update cue, no subsequent updating was required. Infants showed an amplified negative central (Nc) response when the predictable sequence was interrupted. Late components such as the PSW were also evoked in response to unexpected stimuli; however, we found no evidence for a differential response to the informational value of surprising cues at later stages of processing. Infants rather learned that surprising cues always signal a change in the environment that requires updating. Interestingly, infants responded with an amplified neural response to the absence of an expected change, suggesting a top-down modulation of early sensory processing in infants. Our findings corroborate emerging evidence showing that infants build predictive models early in life.

Eurosibs: Towards robust measurement of infant neurocognitive predictors of autism across Europe.

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that affects social communication skills and flexible behaviour. Developing new treatment approaches for ASD requires early identification of the factors that influence later behavioural outcomes. One fruitful research paradigm has been the prospective study of infants with a first degree relative with ASD, who have around a 20% likelihood of developing ASD themselves. Early findings have identified a range of candidate neurocognitive markers for later ASD such as delayed attention shifting or neural responses to faces, but given the early stage of the field most sample sizes are small and replication attempts remain rare. The Eurosibs consortium is a European multisite neurocognitive study of infants with an older sibling with ASD conducted across nine sites in five European countries. In this manuscript, we describe the selection and standardization of our common neurocognitive testing protocol. We report data quality assessments across sites, showing that neurocognitive measures hold great promise for cross-site consistency in diverse populations. We discuss our approach to ensuring robust data analysis pipelines and boosting future reproducibility. Finally, we summarise challenges and opportunities for future multi-site research efforts.

Infants differentially update their internal models of a dynamic environment.

Unexpected events provide us with opportunities for learning about what to expect from the world around us. Using a saccadic-planning paradigm, we investigated whether and how infants and adults represent the statistics of a changing environment (i.e. build an internal model of the environment). Participants observed differently colored bees that appeared at an unexpected location every few trials. The color cues indicated whether the subsequent bees would appear at this new location (i.e. update trials) or at the same location as previously (i.e. no-update trials). Infants learned the predictive value of the color cues and updated their internal models when necessary. Unlike infants, adults had a tendency to update their models each time they observed a change in the structure. We argue that infants are open to learning from current evidence due to being less influenced by their prior knowledge. This is an advantageous learning strategy to form accurate representations in dynamic environments, which is fundamental for successful adaptation.

How preschoolers and adults represent their joint action partner's behavior.

We investigated the cognitive mechanisms underlying turn-taking joint action in 42-month-old children (Experiment 1) and adults (Experiment 2) using a behavioral task of dressing a virtual bear together. We aimed to investigate how participants represent a partners' behavior, i.e., in terms of specific action kinematics or of action effects. The bear was dressed by pressing a smaller and a bigger button. In the Action-response task, instructions asked participants to respond to the partner by pressing the same or opposite button; in the Action-effect task they had to respond to the partner's action effect by dressing the bear with the lacking part of the clothing, which in some cases implied pressing the same button and in other cases implied pressing the opposite button. In 50% of the trials, the partner's association between each button and the ensuing effect (dressing the bear with t-shirt or pants) was reversed, while it never changed for participants. Both children and adults showed no effect of physical congruency of actions, but showed impaired performance in the Action-effect task if their partner achieved her effect through a different action-effect association than their own. These results suggest that, when encoding their partner's actions, agents are influenced by action-effect associations that they learnt through their own experience. While interference led to overt errors in children, it caused longer reaction times in adults, suggesting that a flexible cognitive control (that is still in development in young children) is required to take on the partner's perspective.

Unravelling the contributions of motor experience and conceptual knowledge in action perception: A training study.

Prior knowledge affects how we perceive the world and the sensorimotor system actively guides our perception. An ongoing dispute regards the extent to which prior motor knowledge versus conceptual knowledge modulates the observation of others' actions. Research indicates that motor experience increases motor activation during action perception. Other research, however, has shown that conceptual familiarity with actions also modulates motor activation, i.e., increased motor activation during observation of unfamiliar, compared to conceptually familiar, actions. To begin to disentangle motor from conceptual contributions to action perception, we uniquely combined motoric and conceptual interventions into one design. We experimentally manipulated participants' experience with both motoric skills and conceptual knowledge, via motor training of kinematically challenging actions and contextual information about the action, respectively, in a week-long training session. Measurements of the effects on motor activity measured via electroencephalography (EEG) during pre- and post-training action observation were compared. We found distinct, non-interacting effects of both manipulations: Motor training increased motor activation, whereas additional conceptual knowledge decreased motor activation. The findings indicate that both factors influence action perception in a distinct and parallel manner. This research speaks to previously irreconcilable findings and provides novel insights about the distinct roles of motor and conceptual contributions to action perception.

Neural mirroring and social interaction: Motor system involvement during action observation relates to early peer cooperation.

Whether we hand over objects to someone, play a team sport, or make music together, social interaction often involves interpersonal action coordination, both during instances of cooperation and entrainment. Neural mirroring is thought to play a crucial role in processing other's actions and is therefore considered important for social interaction. Still, to date, it is unknown whether interindividual differences in neural mirroring play a role in interpersonal coordination during different instances of social interaction. A relation between neural mirroring and interpersonal coordination has particularly relevant implications for early childhood, since successful early interaction with peers is predictive of a more favorable social development. We examined the relation between neural mirroring and children's interpersonal coordination during peer interaction using EEG and longitudinal behavioral data. Results showed that 4-year-old children with higher levels of motor system involvement during action observation (as indicated by lower beta-power) were more successful in early peer cooperation. This is the first evidence for a relation between motor system involvement during action observation and interpersonal coordination during other instances of social interaction. The findings suggest that interindividual differences in neural mirroring are related to interpersonal coordination and thus successful social interaction.

The role of action prediction and inhibitory control for joint action coordination in toddlers.

From early in life, young children eagerly engage in social interactions. Yet, they still have difficulties in performing well-coordinated joint actions with others. Adult literature suggests that two processes are important for smooth joint action coordination: action prediction and inhibitory control. The aim of the current study was to disentangle the potential role of these processes in the early development of joint action coordination. Using a simple turn-taking game, we assessed 2½-year-old toddlers' joint action coordination, focusing on timing variability and turn-taking accuracy. In two additional tasks, we examined their action prediction capabilities with an eye-tracking paradigm and examined their inhibitory control capabilities with a classic executive functioning task (gift delay task). We found that individual differences in action prediction and inhibitory action control were distinctly related to the two aspects of joint action coordination. Toddlers who showed more precision in their action predictions were less variable in their action timing during the joint play. Furthermore, toddlers who showed more inhibitory control in an individual context were more accurate in their turn-taking performance during the joint action. On the other hand, no relation between timing variability and inhibitory control or between turn-taking accuracy and action prediction was found. The current results highlight the distinct role of action prediction and inhibitory action control for the quality of joint action coordination in toddlers. Underlying neurocognitive mechanisms and implications for processes involved in joint action coordination in general are discussed.

You'll never crawl alone: neurophysiological evidence for experience-dependent motor resonance in infancy.

Lately, neuroscience is showing a great interest in examining the functional and neural mechanisms which support action observation and understanding. Recent studies have suggested that our motor skills crucially affect the way in which we perceive the actions generated by others, by showing stronger motor resonance for observation of actions that are established in one's motor repertoire. In the present study we extend previous findings that were based on expert motor skills in adults to the natural development of actions in infants. To investigate the effect of natural motor experience on motor resonance during action observation, 14- to 16-month-old infants' EEG was recorded during observation of action videos. Stronger mu- and beta-desynchronizations were found for observation of crawling compared to walking videos and the size of the effect was strongly related to the infant's own crawling experience. This suggests that already early in life one's own action experience is closely related to how actions of others are perceived.