A field test of computer-vision-based gaze estimation in psychology.

Computer-vision-based gaze estimation refers to techniques that estimate gaze direction directly from video recordings of the eyes or face without the need for an eye tracker. Although many such methods exist, their validation is often found in the technical literature (e.g., computer science conference papers). We aimed to (1) identify which computer-vision-based gaze estimation methods are usable by the average researcher in fields such as psychology or education, and (2) evaluate these methods. We searched for methods that do not require calibration and have clear documentation. Two toolkits, OpenFace and OpenGaze, were found to fulfill these criteria. First, we present an experiment where adult participants fixated on nine stimulus points on a computer screen. We filmed their face with a camera and processed the recorded videos with OpenFace and OpenGaze. We conclude that OpenGaze is accurate and precise enough to be used in screen-based experiments with stimuli separated by at least 11 degrees of gaze angle. OpenFace was not sufficiently accurate for such situations but can potentially be used in sparser environments. We then examined whether OpenFace could be used with horizontally separated stimuli in a sparse environment with infant participants. We compared dwell measures based on OpenFace estimates to the same measures based on manual coding. We conclude that OpenFace gaze estimates may potentially be used with measures such as relative total dwell time to sparse, horizontally separated areas of interest, but should not be used to draw conclusions about measures such as dwell duration.

Investigating the development of the autonomic nervous system in infancy through pupillometry.

We aim to investigate early developmental trajectories of the autonomic nervous system (ANS) as indexed by the pupillary light reflex (PLR) in infants with (i.e. preterm birth, feeding difficulties, or siblings of children with autism spectrum disorder) and without (controls) increased likelihood for atypical ANS development. We used eye-tracking to capture the PLR in 216 infants in a longitudinal follow-up study spanning 5 to 24 months of age, and linear mixed models to investigate effects of age and group on three PLR parameters: baseline pupil diameter, latency to constriction and relative constriction amplitude. An increase with age was found in baseline pupil diameter (F(3,273.21) = 13.15, p < 0.001, [Formula: see text] = 0.13), latency to constriction (F(3,326.41) = 3.84, p = 0.010, [Formula: see text] = 0.03) and relative constriction amplitude(F(3,282.53) = 3.70, p = 0.012, [Formula: see text] = 0.04). Group differences were found for baseline pupil diameter (F(3,235.91) = 9.40, p < 0.001, [Formula: see text] = 0.11), with larger diameter in preterms and siblings than in controls, and for latency to constriction (F(3,237.10) = 3.48, p = 0.017, [Formula: see text] = 0.04), with preterms having a longer latency than controls. The results align with previous evidence, with development over time that could be explained by ANS maturation. To better understand the cause of the group differences, further research in a larger sample is necessary, combining pupillometry with other measures to further validate its value.

Larger pupil dilation to nonsocial sounds in infants with subsequent autism diagnosis.

BACKGROUND: Studies of infants with an elevated likelihood of autism spectrum disorder can identify basic developmental processes that are associated with subsequently emerging clinical symptoms. Atypical responsiveness to sounds in infancy is such a potential early marker of autism. Here, we used pupillometry to quantify reactivity to social and nonsocial sounds in infants with a subsequent diagnosis. Previous research suggest that pupil dilation reflects attentional alerting, and link it to the locus coeruleus norepinephrine system. METHODS: We measured pupil dilation responses to child-directed speech and the sound of running water; sounds infants often hear in their everyday life. The final sample consisted of 99 ten-month-old infants (52 girls), of whom 68 had an elevated likelihood of autism and 31 were typically developing low-likelihood infants. At follow-up (36 months of age), 18 children in the elevated-likelihood group were diagnosed with autism. RESULTS: Compared to infants without diagnosis, the infants who were subsequently diagnosed with autism had larger pupil dilation when listening to nonsocial sounds, while reactivity to speech was strikingly similar between groups. In the total sample, more pupil dilation to the nonsocial sound was associated with higher levels of autistic symptoms. We also found that on a trial-by-trial basis, across all conditions and groups, more pupil dilation was associated with making fewer gaze shifts. CONCLUSIONS: This study did not find evidence of atypical pupillary reactivity to child-directed speech early in life in autism. Instead, the results suggest that certain nonsocial sounds elicit atypically strong alerting responses in infants with a subsequent autism diagnosis. These findings may have important theoretical and clinical implications.

Pupil size and pupillary light reflex in early infancy: heritability and link to genetic liability to schizophrenia.

BACKGROUND: Measures based on pupillometry, such as the pupillary light reflex (PLR) and baseline pupil size, reflect physiological responses linked to specific neural circuits that have been implicated as atypical in some psychiatric and neurodevelopmental conditions. METHODS: We investigated the contribution of genetic and environmental factors to the baseline pupil size and the PLR in 510 infant twins assessed at 5 months of age (281 monozygotic and 229 dizygotic pairs), and its associations with common genetic variants associated with neurodevelopmental (autism spectrum disorder and attention deficit hyperactivity disorder) and mental health (bipolar disorder, major depressive disorder and schizophrenia) conditions using genome-wide polygenic scores (GPSs). RESULTS: Univariate twin modelling showed high heritability at 5 months for both pupil size (h2 = .64) and constriction in response to light (h2 = .62), and bivariate twin modeling indicated substantial independence between the genetic factors influencing each (rG = .38). A statistically significant positive association between infant tonic pupil size and the GPS for schizophrenia was found (ß = .15, p = .024), while there was no significant association with the GPS for autism or any other GPSs. CONCLUSIONS: This study shows that some pupil measures are highly heritable in early infancy, although substantially independent in their genetic etiologies, and associated with common genetic variants linked to schizophrenia. It illustrates how genetically informed studies of infants may help us understand early physiological responses associated with psychiatric disorders which emerge much later in life.

T1-Weighted/T2-Weighted Ratio Mapping at 5 Months Captures Individual Differences in Behavioral Development and Differentiates Infants at Familial Risk for Autism from Controls.

Identifying structural measures that capture early brain development and are sensitive to individual differences in behavior is a priority in developmental neuroscience, with potential implications for our understanding of both typical and atypical populations. T1-weighted/T2-weighted (T1w/T2w) ratio mapping, which previously has been linked to myelination, represents an interesting candidate measure in this respect, as an accessible measure from standard magnetic resonance imaging (MRI) sequences. Yet, its value as an early infancy measure remains largely unexplored. Here, we compared T1w/T2w ratio in 5-month-old infants at familial risk (n = 27) for autism spectrum disorder (ASD) to those without elevated autism risk (n = 16). We found lower T1w/T2w ratio in infants at high risk for ASD within widely distributed regions, spanning both white and gray matter. In regions differing between groups, higher T1w/T2w ratio was robustly associated with higher age at scan (range: ~ 4-6.5 months), implying sensitivity to maturation at short developmental timescales. Further, higher T1w/T2w ratio within these regions was associated with higher scores on measures of concurrent developmental level. These findings suggest that T1w/T2w ratio is a developmentally sensitive measure that should be explored further in future studies of both typical and atypical infant populations.

A Pre-registered sticky mittens study: active training does not increase reaching and grasping in a swedish context.

Several studies have previously investigated the effects of sticky mittens training on reaching and grasping development. However, recent critique casted doubts on the robustness of the motor effect of this training. The current study presents a pre-registered report that aimed to generalize these effects to Swedish infants. Three-month-old infants N = 96, 51 females, mostly White middle class in Uppsala, received daily, parent-led sticky mittens or observational training for 2 weeks or no training in 2019. Reaching and grasping abilities were assessed before and after training, using motion tracking and a 4-step reaching task. Sticky mittens training did not facilitate successful reaching. These results indicate that beneficial motor effects of sticky mittens training did not generalize to this sample.

Development of the pupillary light reflex from 9 to 24 months: association with common autism spectrum disorder (ASD) genetic liability and 3-year ASD diagnosis.

BACKGROUND: Although autism spectrum disorder (ASD) is heritable, the mechanisms through which genes contribute to symptom emergence remain unclear. Investigating candidate intermediate phenotypes such as the pupillary light reflex (PLR) prospectively from early in development could bridge genotype and behavioural phenotype. METHODS: Using eye tracking, we longitudinally measured the PLR at 9, 14 and 24 months in a sample of infants (N = 264) enriched for a family history of ASD; 27 infants received an ASD diagnosis at 3 years. We examined the 9- to 24-month developmental trajectories of PLR constriction latency (onset; ms) and amplitude (%) and explored their relation to categorical 3-year ASD outcome, polygenic liability for ASD and dimensional 3-year social affect (SA) and repetitive/restrictive behaviour (RRB) traits. Polygenic scores for ASD (PGSASD ) were calculated for 190 infants. RESULTS: While infants showed a decrease in latency between 9 and 14 months, higher PGSASD was associated with a smaller decrease in latency in the first year (ß = -.16, 95% CI = -0.31, -0.002); infants with later ASD showed a significantly steeper decrease in latency (a putative 'catch-up') between 14 and 24 months relative to those with other outcomes (typical: ß = .54, 95% CI = 0.08, 0.99; other: ß = .53, 95% CI = 0.02, 1.04). Latency development did not associate with later dimensional variation in ASD-related traits. In contrast, change in amplitude was not related to categorical ASD or genetics, but decreasing 9- to 14-month amplitude was associated with higher SA (ß = .08, 95% CI = 0.01, 0.14) and RRB (ß = .05, 95% CI = 0.004, 0.11) traits. CONCLUSIONS: These findings corroborate PLR development as possible intermediate phenotypes being linked to both genetic liability and phenotypic outcomes. Future work should incorporate alternative measures (e.g. functionally informed structural and genetic measures) to test whether distinct neural mechanisms underpin PLR alterations.

Eye tracking in human interaction: Possibilities and limitations.

There is a long history of interest in looking behavior during human interaction. With the advance of (wearable) video-based eye trackers, it has become possible to measure gaze during many different interactions. We outline the different types of eye-tracking setups that currently exist to investigate gaze during interaction. The setups differ mainly with regard to the nature of the eye-tracking signal (head- or world-centered) and the freedom of movement allowed for the participants. These features place constraints on the research questions that can be answered about human interaction. We end with a decision tree to help researchers judge the appropriateness of specific setups.

Reduced orienting to audiovisual synchrony in infancy predicts autism diagnosis at 3 years of age.

BACKGROUND: Effective multisensory processing develops in infancy and is thought to be important for the perception of unified and multimodal objects and events. Previous research suggests impaired multisensory processing in autism, but its role in the early development of the disorder is yet uncertain. Here, using a prospective longitudinal design, we tested whether reduced visual attention to audiovisual synchrony is an infant marker of later-emerging autism diagnosis. METHODS: We studied 10-month-old siblings of children with autism using an eye tracking task previously used in studies of preschoolers. The task assessed the effect of manipulations of audiovisual synchrony on viewing patterns while the infants were observing point light displays of biological motion. We analyzed the gaze data recorded in infancy according to diagnostic status at 3 years of age (DSM-5). RESULTS: Ten-month-old infants who later received an autism diagnosis did not orient to audiovisual synchrony expressed within biological motion. In contrast, both infants at low-risk and high-risk siblings without autism at follow-up had a strong preference for this type of information. No group differences were observed in terms of orienting to upright biological motion. CONCLUSIONS: This study suggests that reduced orienting to audiovisual synchrony within biological motion is an early sign of autism. The findings support the view that poor multisensory processing could be an important antecedent marker of this neurodevelopmental condition.

Infants' preference for individual agents within chasing interactions.

Infants, like adults, are able to discriminate between chasing and non-chasing interactions when watching animations with simple geometric shapes. But where infants derive the necessary information for discrimination and how chasing detection influences later visual attention has been previously unexplored. Here, using eye tracking, we investigated how 5- and 12-month-old infants (N=94) distribute their visual attention among individual members within different interactions depending on a type of interaction. Infant gaze was examined when observing animations depicting chasing and following interactions compared with animations displaying randomly moving shapes. Results demonstrate that when observing chasing and following interactions, all infants strongly preferred to attend to the agent that initiates an interaction and trails behind another. Low-level features, such as changes in agent-specific velocity profiles, could not account for this preference (Study 2). Rather, the strong preference for the agent going behind seems to be dependent on the initial goal-directed or "heat-seeking" motion of one agent toward another (Study 3). The current set of experiments suggests that, similar to adults, 5-months-olds' visual attention depends on the motion features of an individual agent within the interaction and is fine-tuned to agents that display goal-directed motion toward other agents.

The TimeStudio Project: An open source scientific workflow system for the behavioral and brain sciences.

This article describes a new open source scientific workflow system, the TimeStudio Project, dedicated to the behavioral and brain sciences. The program is written in MATLAB and features a graphical user interface for the dynamic pipelining of computer algorithms developed as TimeStudio plugins. TimeStudio includes both a set of general plugins (for reading data files, modifying data structures, visualizing data structures, etc.) and a set of plugins specifically developed for the analysis of event-related eyetracking data as a proof of concept. It is possible to create custom plugins to integrate new or existing MATLAB code anywhere in a workflow, making TimeStudio a flexible workbench for organizing and performing a wide range of analyses. The system also features an integrated sharing and archiving tool for TimeStudio workflows, which can be used to share workflows both during the data analysis phase and after scientific publication. TimeStudio thus facilitates the reproduction and replication of scientific studies, increases the transparency of analyses, and reduces individual researchers' analysis workload. The project website ( http://timestudioproject.com ) contains the latest releases of TimeStudio, together with documentation and user forums.

People overlook subtractive changes differently depending on age, culture, and task.

Previous work has explored transformative strategies that adds or removes components to change an original structure or state, and showed that adults tend to search for additive solutions far more often than subtractive ones. In the current study, we replicated a Lego building task and a grid-based symmetry task from a previous study, and also introduced a novel digital puzzle task. We investigated limitations in the previous study as well as extended the investigation of the subtraction neglect in a sample of children and across two cultures. Results partially confirm previous results, and extends the literature by showing that 9-10 year old children were more likely to ignore subtractive transformations than adults. However, we found both task-based and cultural variations in strategy use in adults from Sweden and the USA. We conclude that a subtraction neglect involves complex cognitive processes that are dependent on the task, culture, and age.

Directed causal effect with PCMCI in hyperscanning EEG time series.

Social activities are likely to cause effects or reactivity in the brains of the people involved in collaborative social situations. This study assesses a new method, Tigramite, for time domain analysis of directed causality between the prefrontal cortex (PFC) of persons in such situations. An experimental situation using hyperscanning EEG was applied while individuals led and followed each other in finger-tapping rhythms. This structured task has a long duration and a high likelihood of inter-brain causal reactions in the prefrontal cortices. Tigramite is a graph-based causal discovery method to identify directed causal relationships in observational time series. Tigramite was used to analyze directed causal connections within and between the PFC. Significantly directed causality within and between brains could be detected during the social interactions. This is the first empirical evidence the Tigramite can reveal inter- and intra-brain-directed causal effects in hyperscanning EEG time series. The findings are promising for further studies of causality in neural networks during social activities using Tigramite on EEG in the time domain.

Context dependent cognitive development in Bhutanese children.

We assessed risk/protective factors for cognitive development of Bhutanese children (504 3-5 year-olds, 49% girls, major ethnicities Ngalop 26%, Tshangla 30%, Lhotsampa 34%) using a non-verbal test of cognitive capacity (SON-R) and primary caregiver interviews. Cognitive capacity was related to the family's SES and whether the family belonged to the primary Buddhist majority ethnic groups (Ngalop or Tshangla) or primarily Hindu minorities (Lhotsampa). In majority families more engagement in Buddhist practices was associated with higher cognitive capacity in children. Minority children were more impacted by parents autonomous-relatedness values. Results demonstrate that cognitive development is dependent on the financial and educational context of the family, societal events, and culture specific risk/protective factors that differ across sub-groups (majority/minority, culture/religion).

Infant responses to direct gaze and associations to autism: A live eye-tracking study.

LAY ABSTRACT: When other people look directly towards us, we often respond by looking back at them, and such direct-gaze responses are important for establishing eye contact. Atypical eye contact is common in autism, but how and when this aspect of autism develops is not well understood. Here, we studied whether how much and how quickly infants respond to others' direct gaze is associated with autism in toddlerhood. We did this by measuring direct-gaze responses in a playful social interaction using live eye tracking. The study included 169 infants, of whom 129 had an elevated likelihood of developing autism due to having a first-degree family member with the condition, and 40 with typical likelihood of autism. In the elevated likelihood group, 35 were diagnosed with autism spectrum disorder at 3 years of age, and 94 were not. The results showed that infants in all three groups tended to increase their looking towards the adult's face after the adult looked directly at them. However, neither how much nor how quickly the infants responded to direct gaze by looking back at the adult reliably differentiated the infants with or without subsequent autism. While infants in the elevated likelihood of autism and subsequent diagnosis group tended to look away quicker from faces with direct gaze than infants in the typical likelihood group, this measure did not differentiate between the two elevated likelihood groups. We interpret the results as supporting the view that atypical direct-gaze responses are not early markers of autism.

Global motion processing in infants' visual cortex and the emergence of autism.

Autism is a heritable and common neurodevelopmental condition, with behavioural symptoms typically emerging around age 2 to 3 years. Differences in basic perceptual processes have been documented in autistic children and adults. Specifically, data from many experiments suggest links between autism and alterations in global visual motion processing (i.e., when individual motion information is integrated to perceive an overall coherent pattern). Yet, no study has investigated whether a distinctive organization of global motion processing precede the emergence of autistic symptoms in early childhood. Here, using a validated infant electroencephalography (EEG) experimental paradigm, we first establish the normative activation profiles for global form, global motion, local form, and local motion in the visual cortex based on data from two samples of 5-month-old infants (total n = 473). Further, in a sample of 5-month-olds at elevated likelihood of autism (n = 52), we show that a different topographical organization of global motion processing is associated with autistic symptoms in toddlerhood. These findings advance the understanding of neural organization of infants' basic visual processing, and its role in the development of autism.

Remote, tablet-based assessment of gaze following: a nationwide infant twin study.

Introduction: Much of our understanding of infant psychological development relies on an in-person, laboratory-based assessment. This limits research generalizability, scalability, and equity in access. One solution is the development of new, remotely deployed assessment tools that do not require real-time experimenter supervision. Methods: The current nationwide (Sweden) infant twin study assessed participants remotely via their caregiver's tablets (N = 104, ages 3 to 17 months). To anchor our findings in previous research, we used a gaze-following task where experimental and age effects are well established. Results: Closely mimicking results from conventional eye tracking, we found that a full head movement elicited more gaze following than isolated eye movements. Furthermore, predictably, we found that older infants followed gaze more frequently than younger infants. Finally, while we found no indication of genetic contributions to gaze-following accuracy, the latency to disengage from the gaze cue and orient toward a target was significantly more similar in monozygotic twins than in dizygotic twins, an indicative of heritability. Discussion: Together, these results highlight the potential of remote assessment of infants' psychological development, which can improve generalizability, inclusion, and scalability in developmental research.

What are you looking at? Gaze following with and without target objects in ASD and typical development.

LAY ABSTRACT: During the first year of life, infants start to align their attention with that of other people. This ability is called joint attention and facilitates social learning and language development. Although children with autism spectrum disorder (ASD) are known to engage less in joint attention compared to other children, several experimental studies have shown that they follow other's gaze (a requirement for visual joint attention) to the same extent as other children. In this study, infants' eye movements were measured at age 10, 14, and 18 months while watching another person look in a certain direction. A target object was either present or absent in the direction of the other person's gaze. Some of the infants were at elevated likelihood of ASD, due to having an older autistic sibling. At age 3 years, infants were assessed for a diagnosis of ASD. Results showed that infants who met diagnostic criteria at 3 years followed gaze to the same extent as other infants. However, they then looked back at the model faster than typically developing infants when no target object was present. When a target object was present, there was no difference between groups. These results may be in line with the view that directly after gaze following, infants with later ASD are less influenced by other people's gaze when processing the common attentional focus. The study adds to our understanding of both the similarities and differences in looking behaviors between infants who later receive an ASD diagnosis and other infants.

How Does Temperament in Toddlers at Elevated Likelihood for Autism Relate to Symptoms of Autism and ADHD at Three Years of Age?

The current study investigated longitudinal associations between parent-rated temperament, observed exuberance and accelerometer activity level at 18-months and symptoms of ASD and ADHD at 36-months in a sample of 54 children at elevated likelihood for ASD. For the specific parent-rated temperament scales, most observed significant associations appeared to be specific for either ASD or ADHD symptoms. Indeed, by controlling for overlapping symptoms a different pattern of associations emerged. These results illustrate how temperamental measures may signal risk for later ASD versus ADHD symptomatology in infants at elevated likelihood for ASD. In addition, they indicate the potential of adopting a broader view on neurodevelopmental disorders by investigating not only ASD traits, but also co-occurring disorders such as ADHD in samples of elevated likelihood for ASD.

Using mu rhythm desynchronization to measure mirror neuron activity in infants.

The Mirror Neuron System hypothesis stating that observed actions are projected onto the observer's own action system assigns an important role to development, because only actions mastered by the observer can be mirrored. The purpose of the present study was to investigate whether there is evidence of a functioning mirror neuron system (MNS) in 8-month-old infants. High-density EEG was used to assess the mu rhythm desynchronization in an action observation task where the infants observed a live model. To reduce noise, ICA decompositions were used. The results show a higher desynchronization of the mu rhythm when infants observed a goal-directed action than when they observed a spatially similar non-goal-directed movement. The localizations of the sources are in agreement with those proposed by the MNS hypothesis. This indicates that the MNS is functioning at this age.