Strategies for enhancing automatic fixation detection in head-mounted eye tracking.

Moving through a dynamic world, humans need to intermittently stabilize gaze targets on their retina to process visual information. Overt attention being thus split into discrete intervals, the automatic detection of such fixation events is paramount to downstream analysis in many eye-tracking studies. Standard algorithms tackle this challenge in the limiting case of little to no head motion. In this static scenario, which is approximately realized for most remote eye-tracking systems, it amounts to detecting periods of relative eye stillness. In contrast, head-mounted eye trackers allow for experiments with subjects moving naturally in everyday environments. Detecting fixations in these dynamic scenarios is more challenging, since gaze-stabilizing eye movements need to be reliably distinguished from non-fixational gaze shifts. Here, we propose several strategies for enhancing existing algorithms developed for fixation detection in the static case to allow for robust fixation detection in dynamic real-world scenarios recorded with head-mounted eye trackers. Specifically, we consider (i) an optic-flow-based compensation stage explicitly accounting for stabilizing eye movements during head motion, (ii) an adaptive adjustment of algorithm sensitivity according to head-motion intensity, and (iii) a coherent tuning of all algorithm parameters. Introducing a new hand-labeled dataset, recorded with the Pupil Invisible glasses by Pupil Labs, we investigate their individual contributions. The dataset comprises both static and dynamic scenarios and is made publicly available. We show that a combination of all proposed strategies improves standard thresholding algorithms and outperforms previous approaches to fixation detection in head-mounted eye tracking.

Noise estimation for head-mounted 3D binocular eye tracking using Pupil Core eye-tracking goggles.

Head-mounted, video-based eye tracking is becoming increasingly common and has promise in a range of applications. Here, we provide a practical and systematic assessment of the sources of measurement uncertainty for one such device - the Pupil Core - in three eye-tracking domains: (1) the 2D scene camera image; (2) the physical rotation of the eye relative to the scene camera 3D space; and (3) the external projection of the estimated gaze point location onto the target plane or in relation to world coordinates. We also assess eye camera motion during active tasks relative to the eye and the scene camera, an important consideration as the rigid arrangement of eye and scene camera is essential for proper alignment of the detected gaze. We find that eye camera motion, improper gaze point depth estimation, and erroneous eye models can all lead to added noise that must be considered in the experimental design. Further, while calibration accuracy and precision estimates can help assess data quality in the scene camera image, they may not be reflective of errors and variability in gaze point estimation. These findings support the importance of eye model constancy for comparisons across experimental conditions and suggest additional assessments of data reliability may be warranted for experiments that require the gaze point or measure eye movements relative to the external world.

A Pupil Segmentation Algorithm Based on Fuzzy Clustering of Distributed Information.

Pupil segmentation is critical for line-of-sight estimation based on the pupil center method. Due to noise and individual differences in human eyes, the quality of eye images often varies, making pupil segmentation difficult. In this paper, we propose a pupil segmentation method based on fuzzy clustering of distributed information, which first preprocesses the original eye image to remove features such as eyebrows and shadows and highlight the pupil area; then the Gaussian model is introduced into global distribution information to enhance the classification fuzzy affiliation for the local neighborhood, and an adaptive local window filter that fuses local spatial and intensity information is proposed to suppress the noise in the image and preserve the edge information of the pupil details. Finally, the intensity histogram of the filtered image is used for fast clustering to obtain the clustering center of the pupil, and this binarization process is used to segment the pupil for the next pupil localization. Experimental results show that the method has high segmentation accuracy, sensitivity, and specificity. It can accurately segment the pupil when there are interference factors such as light spots, light reflection, and contrast difference at the edge of the pupil, which is an important contribution to improving the stability and accuracy of the line-of-sight tracking.

The impact of slippage on the data quality of head-worn eye trackers.

Mobile head-worn eye trackers allow researchers to record eye-movement data as participants freely move around and interact with their surroundings. However, participant behavior may cause the eye tracker to slip on the participant's head, potentially strongly affecting data quality. To investigate how this eye-tracker slippage affects data quality, we designed experiments in which participants mimic behaviors that can cause a mobile eye tracker to move. Specifically, we investigated data quality when participants speak, make facial expressions, and move the eye tracker. Four head-worn eye-tracking setups were used: (i) Tobii Pro Glasses 2 in 50 Hz mode, (ii) SMI Eye Tracking Glasses 2.0 60 Hz, (iii) Pupil-Labs' Pupil in 3D mode, and (iv) Pupil-Labs' Pupil with the Grip gaze estimation algorithm as implemented in the EyeRecToo software. Our results show that whereas gaze estimates of the Tobii and Grip remained stable when the eye tracker moved, the other systems exhibited significant errors (0.8-3.1∘ increase in gaze deviation over baseline) even for the small amounts of glasses movement that occurred during the speech and facial expressions tasks. We conclude that some of the tested eye-tracking setups may not be suitable for investigating gaze behavior when high accuracy is required, such as during face-to-face interaction scenarios. We recommend that users of mobile head-worn eye trackers perform similar tests with their setups to become aware of its characteristics. This will enable researchers to design experiments that are robust to the limitations of their particular eye-tracking setup.

GlassesViewer: Open-source software for viewing and analyzing data from the Tobii Pro Glasses 2 eye tracker.

We present GlassesViewer, open-source software for viewing and analyzing eye-tracking data of the Tobii Pro Glasses 2 head-mounted eye tracker as well as the scene and eye videos and other data streams (pupil size, gyroscope, accelerometer, and TTL input) that this headset can record. The software provides the following functionality written in MATLAB: (1) a graphical interface for navigating the study- and recording structure produced by the Tobii Glasses 2; (2) functionality to unpack, parse, and synchronize the various data and video streams comprising a Glasses 2 recording; and (3) a graphical interface for viewing the Glasses 2's gaze direction, pupil size, gyroscope and accelerometer time-series data, along with the recorded scene and eye camera videos. In this latter interface, segments of data can furthermore be labeled through user-provided event classification algorithms or by means of manual annotation. Lastly, the toolbox provides integration with the GazeCode tool by Benjamins et al. (2018), enabling a completely open-source workflow for analyzing Tobii Pro Glasses 2 recordings.

Measuring dwell time percentage from head-mounted eye-tracking data--comparison of a frame-by-frame and a fixation-by-fixation analysis.

Although analysing software for eye-tracking data has significantly improved in the past decades, the analysis of gaze behaviour recorded with head-mounted devices is still challenging and time-consuming. Therefore, new methods have to be tested to reduce the analysis workload while maintaining accuracy and reliability. In this article, dwell time percentages to six areas of interest (AOIs), of six participants cycling on four different roads, were analysed both frame-by-frame and in a 'fixation-by-fixation' manner. The fixation-based method is similar to the classic frame-by-frame method but instead of assigning frames, fixations are assigned to one of the AOIs. Although some considerable differences were found between the two methods, a Pearson correlation of 0.930 points out a good validity of the fixation-by-fixation method. For the analysis of gaze behaviour over an extended period of time, the fixation-based approach is a valuable and time-saving alternative for the classic frame-by-frame analysis.

Early development of saliency-driven attention through object manipulation.

In the first years of life, infants progressively develop attention selection skills to gather information from visually clustered environments. As young as newborns, infants are sensitive to the distinguished differences in color, orientation, and luminance, which are the components of visual saliency. However, we know little about how saliency-driven attention emerges and develops socially through everyday free-viewing experiences. The present work assessed the saliency change in infants' egocentric scenes and investigated the impacts of manual engagements on infant object looking in the interactive context of object play. Thirty parent-infant dyads, including infants in two age groups (younger: 3- to 6-month-old; older: 9- to 12-month-old), completed a brief session of object play. Infants' looking behaviors were recorded by the head-mounted eye-tracking gear, and both parents' and infants' manual actions on objects were annotated separately for analyses. The present findings revealed distinct attention mechanisms that underlie the hand-eye coordination between parents and infants and within infants during object play: younger infants are predominantly biased toward the characteristics of the visual saliency accompanying the parent's handled actions on the objects; on the other hand, older infants gradually employed more attention to the object, regardless of the saliency in view, as they gained more self-generated manual actions. Taken together, the present work highlights the tight coordination between visual experiences and sensorimotor competence and proposes a novel dyadic pathway to sustained attention that social sensitivity to parents' hands emerges through saliency-driven attention, preparing infants to focus, follow, and steadily track moving targets in free-flow viewing activities.

Beyond screen time: Using head-mounted eye tracking to study natural behavior.

Head-mounted eye tracking is a new method that allows researchers to catch a glimpse of what infants and children see during naturalistic activities. In this chapter, we review how mobile, wearable eye trackers improve the construct validity of important developmental constructs, such as visual object experiences and social attention, in ways that would be impossible using screen-based eye tracking. Head-mounted eye tracking improves ecological validity by allowing researchers to present more realistic and complex visual scenes, create more interactive experimental situations, and examine how the body influences what infants and children see. As with any new method, there are difficulties to overcome. Accordingly, we identify what aspects of head-mounted eye-tracking study design affect the measurement quality, interpretability of the results, and efficiency of gathering data. Moreover, we provide a summary of best practices aimed at allowing researchers to make well-informed decisions about whether and how to apply head-mounted eye tracking to their own research questions.

Real-time processes in the development of action planning.

Across species and ages, planning multi-step actions is a hallmark of intelligence and critical for survival. Traditionally, researchers adopt a "top-down" approach to action planning by focusing on the ability to create an internal representation of the world that guides the next step in a multi-step action. However, a top-down approach does not inform on underlying mechanisms, so researchers can only speculate about how and why improvements in planning occur. The current study takes a "bottom-up" approach by testing developmental changes in the real-time, moment-to-moment interplay among perceptual, neural, and motor components of action planning using simultaneous video, motion-tracking, head-mounted eye tracking, and electroencephalography (EEG). Preschoolers (n = 32) and adults (n = 22) grasped a hammer with their dominant hand to pound a peg when the hammer handle pointed in different directions. When the handle pointed toward their non-dominant hand, younger children ("nonadaptive planners") used a habitual overhand grip that interfered with wielding the hammer, whereas adults and older children ("adaptive planners") used an adaptive underhand grip. Adaptive and nonadaptive children differed in when and where they directed their gaze to obtain visual information, neural activation of the motor system before reaching, and straightness of their reach trajectories. Nonadaptive children immediately used a habitual overhand grip before gathering visual information, leaving insufficient time to form a plan before acting. Our novel bottom-up approach transcends mere speculation by providing converging evidence that the development of action planning depends on a real-time "tug of war" between habits and information gathering and processing.

Attentional Flexibility Predicts A-Not-B Task Performance in 14-Month-Old-Infants: A Head-Mounted Eye Tracking Study.

Early individual differences in executive functions (EFs) are predictive of a range of developmental outcomes. However, despite the importance of EFs, little is known about the processes underlying these early individual differences. Therefore, we investigated the association between 14-month-old infants' attention on a reaching version of the A-not-B task and task success. We hypothesized that both strategic focused attention (measured as percentage looking time towards the correct location during delay) and attentional flexibility (measured as number of looks per second to available stimuli during delay) would relate positively to task performance. Infants performed the A-not-B task wearing a head-mounted eye tracker (N = 24). Results were trial-dependent and partially supported the hypotheses: (1) infants who were better able to flexibly shift attention between available stimuli on the first pre-switch trial showed better task performance overall; and (2) strategic focused attention to the hiding location during the first switch trial was positively related to performance on that particular trial only (trend-level effect). Thus, the study shows preliminary evidence that particularly attentional flexibility is a key factor underlying EF performance in young children. Advantages and challenges of working with head-mounted eye tracking in infants are discussed.

Transition From Crawling to Walking Changes Gaze Communication Space in Everyday Infant-Parent Interaction.

Acquisition of walking changes not only infants' locomotion itself but also infants' exploratory behavior and social interaction, such as gaze communication. To understand the ecological context in which gaze communication occurs and how it changes with walking development from the point of view of the spatial arrangement of infants, parents, and objects, we analyzed longitudinal data of daily eye contact scenes recorded from head-mounted eye trackers worn by parents as infants grew from 10 to 15.5 months, focusing on infant-parent distance and the number of objects between the dyad. A Bayesian state-space model revealed that the interpersonal distance at which infants initiated eye contact with their parents increased with the time ratio of walking to crawling. This result could not be explained by the developmental change in the amount of time that the infants were far from the parents, which is not limited to the gaze communication context. Moreover, the interpersonal distance at which the parents initiated eye contact with the infants did not increase with the time ratio of walking to crawling. The number of objects on the floor between infants and parents at the time of eye contact increased with interpersonal distance. Taken together, these results indicate that the transition from crawling to walking changes the ecological context in which infants initiate gaze communication to a visual environment characterized by a larger interpersonal distance and, therefore, more objects cluttered between the dyad. The present study has wider implications for the developmental change of shared attention in conjunction with walking development.

The Costs and Benefits of Development: The Transition From Crawling to Walking.

The transition from crawling to walking requires infants to relinquish their status as experienced, highly skilled crawlers in favor of being inexperienced, lowskilled walkers. Yet infants willingly undergo this developmental transition, despite incurring costs of shaky steps, frequent falls, and inability to gauge affordances for action in their new upright posture. Why do infants persist with walking when crawling serves the purpose of independent mobility? In this article, we present an integrative analysis of the costs and benefits associated with crawling and walking that challenges prior assumptions, and reveals deficits of crawling and benefits of upright locomotion that were previously overlooked. Inquiry into multiple domains of development reveals that the benefits of persisting with walking outweigh the costs: Compared to crawlers, walking infants cover more space more quickly, experience richer visual input, access and play more with distant objects, and interact in qualitatively new ways with caregivers.

Play, attention, and learning: how do play and timing shape the development of attention and influence classroom learning?

The behavioral and neurobiological connections between play and the development of critical cognitive functions, such as attention, remain largely unknown. We do not yet know how these connections relate to the formation of specific abilities, such as spatial ability, and to learning in formal environments, such as in the classroom. Insights into these issues would be beneficial not only for understanding play, attention, and learning individually, but also for the development of more efficacious systems for learning and for the treatment of neurodevelopmental disorders. Different operational definitions of play can incorporate or exclude varying types of behavior, emphasize varying developmental time points, and motivate different research questions. Relevant questions to be explored in this area include, How do particular kinds of play relate to the development of particular kinds of abilities later in life? How does play vary across societies and species in the context of evolution? Does play facilitate a shift from reactive to predictive timing, and is its connection to timing unique or particularly significant? This report will outline important research steps that need to be taken in order to address these and other questions about play, human activity, and cognitive functions.