A field test of computer-vision-based gaze estimation in psychology.

Computer-vision-based gaze estimation refers to techniques that estimate gaze direction directly from video recordings of the eyes or face without the need for an eye tracker. Although many such methods exist, their validation is often found in the technical literature (e.g., computer science conference papers). We aimed to (1) identify which computer-vision-based gaze estimation methods are usable by the average researcher in fields such as psychology or education, and (2) evaluate these methods. We searched for methods that do not require calibration and have clear documentation. Two toolkits, OpenFace and OpenGaze, were found to fulfill these criteria. First, we present an experiment where adult participants fixated on nine stimulus points on a computer screen. We filmed their face with a camera and processed the recorded videos with OpenFace and OpenGaze. We conclude that OpenGaze is accurate and precise enough to be used in screen-based experiments with stimuli separated by at least 11 degrees of gaze angle. OpenFace was not sufficiently accurate for such situations but can potentially be used in sparser environments. We then examined whether OpenFace could be used with horizontally separated stimuli in a sparse environment with infant participants. We compared dwell measures based on OpenFace estimates to the same measures based on manual coding. We conclude that OpenFace gaze estimates may potentially be used with measures such as relative total dwell time to sparse, horizontally separated areas of interest, but should not be used to draw conclusions about measures such as dwell duration.

Replacing eye trackers in ongoing studies: A comparison of eye-tracking data quality between the Tobii Pro TX300 and the Tobii Pro Spectrum.

The Tobii Pro TX300 is a popular eye tracker in developmental eye-tracking research, yet it is no longer manufactured. If a TX300 breaks down, it may have to be replaced. The data quality of the replacement eye tracker may differ from that of the TX300, which may affect the experimental outcome measures. This is problematic for longitudinal and multi-site studies, and for researchers replacing eye trackers between studies. We, therefore, ask how the TX300 and its successor, the Tobii Pro Spectrum, compare in terms of eye-tracking data quality. Data quality-operationalized through precision, accuracy, and data loss-was compared between eye trackers for three age groups (around 5-months, 10-months, and 3-years). Precision was better for all gaze position signals obtained with the Spectrum in comparison to the TX300. Accuracy of the Spectrum was higher for the 5-month-old and 10-month-old children. For the three-year-old children, accuracy was similar across both eye trackers. Gaze position signals from the Spectrum exhibited lower proportions of data loss, and the duration of the data loss periods tended to be shorter. In conclusion, the Spectrum produces gaze position signals with higher data quality, especially for the younger infants. Implications for data analysis are discussed.

Two-year-olds at elevated risk for ASD can learn novel words from their parents.

Children diagnosed with autism spectrum disorder (ASD) often have smaller vocabularies in infancy compared to typically-developing children. To understand whether their smaller vocabularies stem from problems in learning, our study compared a prospective risk sample of 18 elevated risk and 11 lower risk 24-month-olds on current vocabulary size and word learning ability using a paradigm in which parents teach their child words. Results revealed that both groups learned novel words, even though parents indicated that infants at elevated risk of ASD knew fewer words. This suggests that these early compromised vocabularies cannot be solely linked to difficulties in word formations.

The emergence of a theta social brain network during infancy.

Infants' socio-cognitive ability develops dramatically during the first year of life. From the perspective of ontogeny, the early development of social behavior allows for parent-child attachment, which in turn enhances survival. Thus, it is theorized that the development of social behavior, driven by social brain networks, forms the core of developmental acquisitions during this period. Further, understanding the maturation within the neural networks during social development is crucial to obtain a better grasp of the development of social developmental disorders. Therefore, we performed a longitudinal study in 854 infants measured at around 5 and 10 months to map the development of functional networks in the brain when infants were processing social and non-social videos. Using EEG, we focused on the frequency bands most commonly connected to social behavior: theta and alpha. We found that alpha networks remained relatively stable over the first year of life and showed no selectivity for social versus non-social stimuli, theta networks, showed strong global reconfigurations. The development of the theta networks progressed from a parietal occipital network in early infancy to a frontoparietal network towards the end of the first year of life. This reconfiguration coincided with selectivity for social versus non-social stimuli, with infants approaching the end of their first year of life showing increased synchronicity of theta communication when watching social videos versus non-social videos. Our findings provide strong evidence for the involvement of a frontoparietal theta network in the development of the social brain.

Exploring emotional face processing in 5-month-olds: The relation with quality of parent-child interaction and spatial frequencies.

It is unclear whether infants differentially process emotional faces in the brain at 5 months of age. Contradictory findings of previous research indicate that additional factors play a role in this process. The current study investigated whether five-month-old infants show differential brain activity between emotional faces. Furthermore, we explored the relation between emotional face processing and (I) stimulus characteristics, specifically the spatial frequency content, and (II) parent, child, and dyadic qualities of interaction characteristics. Face-sensitive components (i.e., N290, P400, Nc) in response to neutral and fearful faces that contained only lower or higher spatial frequencies were assessed. Quality of parent-child interaction was assessed with the Manchester Assessment of Caregiver Infant Interaction (MACI). The results show that, as a full group, none of the components differed between emotional expressions. However, when splitting the group based on median MACI scores, infants who showed high quality of interaction (i.e., more attentiveness to caregiver, positive and negative affect, and liveliness) processed emotions differently, whereas infants who showed low quality did not. These results indicate that a sub-group of infants show differential emotional face processing at 5 months of age, which seem to relate to quality of their behavior during the parent-child interaction.

Individual differences in visual attention and self-regulation: A multimethod longitudinal study from infancy to toddlerhood.

Given the importance of self-regulation for a broad range of developmental outcomes, identifying reliable precursors of self-regulation early in development is important for early prevention of developmental problems. The aim of this study was to examine whether three visual attention measures (fixation duration, variation in fixation duration, and disengagement) in infancy (9.10-11.43 months of age) predicted effortful control and compliance in toddlerhood (26.71-31.80 months). The sample consisted of 74 children (50% boys). In infancy, two eye-tracking tasks were conducted: a visual search task to assess fixation duration and variation in fixation duration (n = 58) and the gap-overlap task to assess disengagement (n = 49). In toddlerhood, children's effortful control (n = 65) and compliance (n = 65) were assessed by parent reports and observed during a delay of gratification task and a cleanup session together with the parents, respectively. Using full information maximum likelihood to account for missing data, multiple regression analyses revealed that, when all three measures of visual attention were taken into account, longer fixations and less variation in fixation duration in infancy predicted better effortful control. Disengagement did not predict effortful control. Compliance in toddlerhood was not predicted by any of the visual attention measures. These findings may indicate that visual attentional measures in infancy predict relatively independent forms of self-regulation in toddlerhood. Future studies are necessary to elucidate the mechanisms that underlie the association between (variation in) fixation duration in infancy and effortful control in toddlerhood.

Noise-robust fixation detection in eye movement data: Identification by two-means clustering (I2MC).

Eye-tracking research in infants and older children has gained a lot of momentum over the last decades. Although eye-tracking research in these participant groups has become easier with the advance of the remote eye-tracker, this often comes at the cost of poorer data quality than in research with well-trained adults (Hessels, Andersson, Hooge, Nyström, & Kemner Infancy, 20, 601-633, 2015; Wass, Forssman, & Leppänen Infancy, 19, 427-460, 2014). Current fixation detection algorithms are not built for data from infants and young children. As a result, some researchers have even turned to hand correction of fixation detections (Saez de Urabain, Johnson, & Smith Behavior Research Methods, 47, 53-72, 2015). Here we introduce a fixation detection algorithm-identification by two-means clustering (I2MC)-built specifically for data across a wide range of noise levels and when periods of data loss may occur. We evaluated the I2MC algorithm against seven state-of-the-art event detection algorithms, and report that the I2MC algorithm's output is the most robust to high noise and data loss levels. The algorithm is automatic, works offline, and is suitable for eye-tracking data recorded with remote or tower-mounted eye-trackers using static stimuli. In addition to application of the I2MC algorithm in eye-tracking research with infants, school children, and certain patient groups, the I2MC algorithm also may be useful when the noise and data loss levels are markedly different between trials, participants, or time points (e.g., longitudinal research).

Octave effect in auditory attention.

After hearing a tone, the human auditory system becomes more sensitive to similar tones than to other tones. Current auditory models explain this phenomenon by a simple bandpass attention filter. Here, we demonstrate that auditory attention involves multiple pass-bands around octave-related frequencies above and below the cued tone. Intriguingly, this "octave effect" not only occurs for physically presented tones, but even persists for the missing fundamental in complex tones, and for imagined tones. Our results suggest neural interactions combining octave-related frequencies, likely located in nonprimary cortical regions. We speculate that this connectivity scheme evolved from exposure to natural vibrations containing octave-related spectral peaks, e.g., as produced by vocal cords.

An in-depth look at saccadic search in infancy.

Two questions were posed in the present study: (1) Do infants search for discrepant items in the absence of instructions? We outline where previous research has been inconclusive in answering this question. (2) In what manner do infants search, and what are the fixation and saccade characteristics in saccadic search? A thorough characterization of saccadic search in infancy is of great importance as a reference for future eye-movement studies in infancy. We presented 10-month-old infants with 24 visual search displays in two separate sessions within two weeks. We report that infant saccadic search performance at 10 months is above what may be expected by our model of chance, and is dependent on the specific target. Infant fixation and saccade characteristics show similarities to adult fixation and saccade characteristics in saccadic search. All findings were highly consistent across two separate sessions on the group level. An examination of the reliability of saccadic search revealed that test-retest reliability for oculomotor characteristics was high, particularly for fixation duration. We suggest that future research into saccadic search in infancy adopt the presented model of chance as a baseline against which to compare search performance. Researchers investigating both the typical and atypical development of visual search may benefit from the presented results.

The area-of-interest problem in eyetracking research: A noise-robust solution for face and sparse stimuli.

A problem in eyetracking research is choosing areas of interest (AOIs): Researchers in the same field often use widely varying AOIs for similar stimuli, making cross-study comparisons difficult or even impossible. Subjective choices while choosing AOIs cause differences in AOI shape, size, and location. On the other hand, not many guidelines for constructing AOIs, or comparisons between AOI-production methods, are available. In the present study, we addressed this gap by comparing AOI-production methods in face stimuli, using data collected with infants and adults (with autism spectrum disorder [ASD] and matched controls). Specifically, we report that the attention-attracting and attention-maintaining capacities of AOIs differ between AOI-production methods, and that this matters for statistical comparisons in one of three groups investigated (the ASD group). In addition, we investigated the relation between AOI size and an AOI's attention-attracting and attention-maintaining capacities, as well as the consequences for statistical analyses, and report that adopting large AOIs solves the problem of statistical differences between the AOI methods. Finally, we tested AOI-production methods for their robustness to noise, and report that large AOIs-using the Voronoi tessellation method or the limited-radius Voronoi tessellation method with large radii-are most robust to noise. We conclude that large AOIs are a noise-robust solution in face stimuli and, when implemented using the Voronoi method, are the most objective of the researcher-defined AOIs. Adopting Voronoi AOIs in face-scanning research should allow better between-group and cross-study comparisons.

Implicit perceptual memory modulates early visual processing of ambiguous images.

The way we perceive the present visual environment is influenced by past visual experiences. Here we investigated the neural basis of such experience dependency. We repeatedly presented human observers with an ambiguous visual stimulus (structure-from-motion) that can give rise to two distinct perceptual interpretations. Past visual experience is known to influence the perception of such stimuli. We recorded fast dynamics of neural activity shortly after stimulus onset using event-related electroencephalography. The number of previous occurrences of a certain percept modulated early posterior brain activity starting as early as 50 ms after stimulus onset. This modulation developed across hundreds of percept repetitions, reflecting several minutes of accumulating perceptual experience. Importantly, there was no such modulation when the mere number of previous stimulus presentations was considered regardless of how they were perceived. This indicates that the effect depended on previous perception rather than previous visual input. The short latency and posterior scalp location of the effect suggest that perceptual history modified bottom-up stimulus processing in early visual cortex. We propose that bottom-up neural responses to a given visual presentation are shaped, in part, by feedback modulation that occurred during previous presentations, thus allowing these responses to be biased in light of previous perceptual decisions.

Qualitative tests of remote eyetracker recovery and performance during head rotation.

What are the decision criteria for choosing an eyetracker? Often the choice is based on specifications by the manufacturer of the validity (accuracy) and reliability (precision) of measurements that can be achieved using a particular eyetracker. These specifications are mostly achieved under optimal conditions-for example, by using an artificial eye or trained participants fixed in a chinrest. Research, however, does not always take place in optimal conditions: For instance, when investigating eye movements in infants, school children, and patient groups with disorders such as attention-deficit hyperactivity disorder, it is practically impossible to restrict movement. We modeled movements often seen in infant research in two behaviors: (1) looking away from and back to the screen, to investigate eyetracker recovery, and (2) head orientations, to investigate eyetracker performance with nonoptimal orientations of the eyes. We investigated how eight eyetracking setups by three manufacturers (SMI, Tobii, and LC Technologies) coped with these modeled behaviors in adults. We report that the tested SMI eyetrackers dropped in sampling frequency when the eyes were not visible to the eyetracker, whereas the other systems did not, and discuss the potential consequences thereof. Furthermore, we report that the tested eyetrackers varied in their rates of data loss and systematic offsets during shifted head orientations. We conclude that (prospective) eye-movement researchers who cannot restrict movement or nonoptimal head orientations in their participants might benefit from testing their eyetracker in nonoptimal conditions. Additionally, researchers should be aware of the data loss and inaccuracies that might result from nonoptimal head orientations.

Parallel development of ERP and behavioural measurements of visual segmentation.

Visual segmentation, a process in which elements are integrated into a form and segregated from the background, is known to differ from adults at infancy. The further developmental trajectory of this process, and of the underlying brain mechanisms, during childhood and adolescence is unknown. The aim of the study was to investigate the developmental trajectory of ERP reflections of visual segmentation, and to relate this to behavioural performance. One hundred and eleven typically developing children from 7 to 18 years of age were divided into six age groups. Each child performed two visual tasks. In a texture segmentation task, the difference in event-related potential (ERP) response to homogeneous (no visual segmentation) and checkered stimuli (visual segmentation) was investigated. In addition, behavioural performance on integration of elements into contours was measured. Both behavioural and ERP measurements of visual segmentation differed from adults in 7-12 year-old children. Behaviourally, young children were less able to integrate elements into a contour than older children. In addition, a developmental change was present in the ERP pattern evoked by homogeneous versus checkered stimuli. The largest differences in behaviour and ERPs were found between 7-8- and 9-10-, and between 11-12- and 13-14-year-old children, indicating the strongest development between those age groups. Behavioural as well as ERP measurements at 13-14 years of age showed similar results to those of adults. These results reveal that visual segmentation continues to develop until early puberty. Only by 13-14 years of age, children do integrate and segregate visual information as adults do. These results can be interpreted in terms of functional connectivity within the visual cortex.

The effect of attenuating noradrenergic neurotransmission by clonidine on brain activity measures of visuospatial attention.

OBJECTIVE: In the current study, we investigated the role of noradrenaline in directing (bias) and disengagement of visuospatial attention. METHODS: We assessed the effect of clonidine on event-related brain potential (ERP) reflections of bias and disengagement in a double-blind placebo-controlled crossover design. An initial dose of 200-µg clonidine was replaced by 100 µg because of marked side effects. Twenty-one healthy male participants performed the visual-spatial cueing task while an electroencephalogram (EEG) was recorded. The behavioral output is the validity effect (benefit of cueing in terms of reaction time to targets). ERP indices for bias were the cue-related early directing attention negativity and late directing attention positivity, and the target-elicited P1 and N1 modulations by validity ('validity-effect'). The ERP index for disengagement was the target-elicited 'late positive deflection' modulation by validity. Behavioral analyses were performed on 16 participants, electrophysiological analyses on a subset (n=9). RESULTS: Clonidine attenuated the N1 effect, albeit in a subsample. Neither cue-elicited ERPs nor the behavioral validity effect were affected. Clonidine-induced blood pressure reduction was correlated with the reduction of the late positive deflection effect under clonidine. CONCLUSION: Clonidine attenuated the result of bias in a subsample and may have a modulating effect on disengagement.

How Do Psychology Professors View the Relation Between Scientific Knowledge and Its Applicability and Societal Relevance?

How do researchers in psychology view the relation between scientific knowledge, its applicability, and its societal relevance? Most research on psychological science and its benefits to society is discussed from a bird's eye view (a meta-scientific perspective), by identifying general trends such as psychology's dominant focus on lab-based experiments and general descriptive theories. In recent years, several critics have argued that this focus has come at the cost of reduced practical and societal relevance. In this study, we interviewed Dutch psychology professors to gauge their views about the relation between psychological research and its relevance to society. We found that psychology professors engaged in a variety of activities to engage science with society, from work in clinical and applied settings, to consultancy, education, and science communication. However, we found that the role of theory when applying scientific knowledge to practical problems is far from straightforward. While most participants regarded theories as relevant to understanding general contexts of application, psychological theories were seldom directly related to specific applications. We compare and discuss our findings in the light of recent discussions about the lack of applicability and societal relevance of psychological science.

Neural processing of high and low spatial frequency information in faces changes across development: qualitative changes in face processing during adolescence.

Face perception in adults depends on skilled processing of interattribute distances ('configural' processing), which is disrupted for faces presented in inverted orientation (face inversion effect or FIE). Children are not proficient in configural processing, and this might relate to an underlying immaturity to use facial information in low spatial frequency (SF) ranges, which capture the coarse information needed for configural processing. We hypothesized that during adolescence a shift from use of high to low SF information takes place. Therefore, we studied the influence of SF content on neural face processing in groups of children (9-10 years), adolescents (14-15 years) and young adults (21-29 years) by measuring event-related potentials (ERPs) to upright and inverted faces which varied in SF content. Results revealed that children show a neural FIE in early processing stages (i.e. P1; generated in early visual areas), suggesting a superficial, global facial analysis. In contrast, ERPs of adults revealed an FIE at later processing stages (i.e. N170; generated in face-selective, higher visual areas). Interestingly, adolescents showed FIEs in both processing stages, suggesting a hybrid developmental stage. Furthermore, adolescents and adults showed FIEs for stimuli containing low SF information, whereas such effects were driven by both low and high SF information in children. These results indicate that face processing has a protracted maturational course into adolescence, and is dependent on changes in SF processing. During adolescence, sensitivity to configural cues is developed, which aids the fast and holistic processing that is so special for faces.

The direct and indirect effects of parenting behaviors and functional brain network efficiency on self-regulation from infancy to early childhood: A longitudinal mediation model.

There is growing interest in the hypothesis that early parenting behaviors impact children's self-regulation by affecting children's developing brain networks. Yet, most prior research on the development of self-regulation has focused on either environmental or neurobiological factors. The aim of the current study was to expand the literature by examining direct and indirect effects of variations in parenting behaviors (support and stimulation) and efficiency of functional brain networks (small-worldness) on individual differences in child self-regulation, using a three-wave longitudinal model in a sample of 109 infants and their mothers. Results revealed that parental support predicted child self-regulation at 5 months, 10 months, and 3 years of age. This effect was not mediated by infants' small-worldness within the alpha and theta rhythm. Parental stimulation predicted higher levels of infants' alpha small-worldness, whereas parental support predicted lower levels of infants' theta small-worldness. Thus, parents may need to stimulate their infants to explore the environment autonomously in order to come to more efficient functional brain networks. The findings of the current study highlight potential influences of both extrinsic environmental factors and intrinsic neurobiological factors in relation to child self-regulation, emphasizing the role of parental support as a form of external regulation during infancy, when the brain is not yet sufficiently developed to perform self-regulation itself.

Social Responsiveness Scale-aided analysis of the clinical impact of copy number variations in autism.

Recent array-based studies have detected a wealth of copy number variations (CNVs) in patients with autism spectrum disorders (ASD). Since CNVs also occur in healthy individuals, their contributions to the patient's phenotype remain largely unclear. In a cohort of children with symptoms of ASD, diagnosis of the index patient using ADOS-G and ADI-R was performed, and the Social Responsiveness Scale (SRS) was administered to the index patients, both parents, and all available siblings. CNVs were identified using SNP arrays and confirmed by FISH or array CGH. To evaluate the clinical significance of CNVs, we analyzed three families with multiple affected children (multiplex) and six families with a single affected child (simplex) in which at least one child carried a CNV with a brain-transcribed gene. CNVs containing genes that participate in pathways previously implicated in ASD, such as the phosphoinositol signaling pathway (PIK3CA, GIRDIN), contactin-based networks of cell communication (CNTN6), and microcephalin (MCPH1) were found not to co-segregate with ASD phenotypes. In one family, a loss of CNTN5 co-segregated with disease. This indicates that most CNVs may by themselves not be sufficient to cause ASD, but still may contribute to the phenotype by additive or epistatic interactions with inherited (transmitted) mutations or non-genetic factors. Our study extends the scope of genome-wide CNV profiling beyond de novo CNVs in sporadic patients and may aid in uncovering missing heritability in genome-wide screening studies of complex psychiatric disorders.

Reduced prepulse inhibition in adolescents at risk for psychosis: a 2-year follow-up study.

BACKGROUND: Reduced prepulse inhibition (PPI) of the auditory startle reflex is a hallmark feature of attention-processing deficits in patients with schizophrenia and other psychotic disorders. Recent evidence suggests that these deficits may also be present before the onset of psychosis in individuals at ultra-high risk (UHR) and become progressively worse as psychosis develops. We conducted a longitudinal follow-up study to observe the development of PPI over time in UHR adolescents and healthy controls. METHODS: Two-year follow-up data of PPI measures were compared between UHR adolescents and a matched control group of typically developing individuals. RESULTS: We included 42 UHR adolescents and 32 matched controls in our study. Compared with controls, UHR individuals showed reduced PPI at both assessments. Clinical improvement in UHR individuals was associated with an increase in PPI parameters. LIMITATIONS: A developmental increase in startle magnitude partially confined the interpretation of the association between clinical status and PPI. Furthermore, post hoc analyses for UHR individuals who became psychotic between assessments had limited power owing to a low transition rate (14%). CONCLUSION: Deficits in PPI are present before the onset of psychosis and represent a stable vulnerability marker over time in UHR individuals. The magnitude of this marker may partially depend on the severity of clinical symptoms.

The Reality of "Real-Life" Neuroscience: A Commentary on Shamay-Tsoory and Mendelsohn (2019).

The main thrust of Shamay-Tsoory and Mendelsohn's ecological approach is that "the use of real-life complex, dynamic, naturalistic stimuli provides a solid basis for understanding brain and behavior" (p. 851). Although we support the overall goal and objectives of Shamay-Tsoory and Mendelsohn's approach to "real-life" neuroscience, their review refers to the terms "ecological validity" and "representative design" in a manner different from that originally introduced by Egon Brunswik. Our aim is to clarify Brunswik's original definitions and briefly explain how these concepts pertain to the larger problem of generalizability, not just for history's sake, but because we believe that a proper understanding of these concepts is important for researchers who want to understand human behavior and the brain in the context of everyday experience, and because Brunswik's original ideas may contribute to Shamay-Tsoory and Mendelsohn's ecological approach. Finally, we argue that the popular and often misused concept of "ecological validity" is ill-formed, lacks specificity, and may even undermine the development of theoretically sound and tractable research.