Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors.

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.

MYFix: Automated Fixation Annotation of Eye-Tracking Videos.

In mobile eye-tracking research, the automatic annotation of fixation points is an important yet difficult task, especially in varied and dynamic environments such as outdoor urban landscapes. This complexity is increased by the constant movement and dynamic nature of both the observer and their environment in urban spaces. This paper presents a novel approach that integrates the capabilities of two foundation models, YOLOv8 and Mask2Former, as a pipeline to automatically annotate fixation points without requiring additional training or fine-tuning. Our pipeline leverages YOLO's extensive training on the MS COCO dataset for object detection and Mask2Former's training on the Cityscapes dataset for semantic segmentation. This integration not only streamlines the annotation process but also improves accuracy and consistency, ensuring reliable annotations, even in complex scenes with multiple objects side by side or at different depths. Validation through two experiments showcases its efficiency, achieving 89.05% accuracy in a controlled data collection and 81.50% accuracy in a real-world outdoor wayfinding scenario. With an average runtime per frame of 1.61 ± 0.35 s, our approach stands as a robust solution for automatic fixation annotation.

Improving Eye-Tracking Data Quality: A Framework for Reproducible Evaluation of Detection Algorithms.

High-quality eye-tracking data are crucial in behavioral sciences and medicine. Even with a solid understanding of the literature, selecting the most suitable algorithm for a specific research project poses a challenge. Empowering applied researchers to choose the best-fitting detector for their research needs is the primary contribution of this paper. We developed a framework to systematically assess and compare the effectiveness of 13 state-of-the-art algorithms through a unified application interface. Hence, we more than double the number of algorithms that are currently usable within a single software package and allow researchers to identify the best-suited algorithm for a given scientific setup. Our framework validation on retrospective data underscores its suitability for algorithm selection. Through a detailed and reproducible step-by-step workflow, we hope to contribute towards significantly improved data quality in scientific experiments.

Visual Tracking in Amblyopia: A Continuous Psychophysical Approach.

Purpose: This study aimed to explore the underlying mechanisms of the observed visuomotor deficit in amblyopia. Methods: Twenty-four amblyopic (25.8 ± 3.8 years; 15 males) and 22 normal participants (25.8 ± 2.1 years; 8 males) took part in the study. The participants were instructed to continuously track a randomly moving Gaussian target on a computer screen using a mouse. In experiment 1, the participants performed the tracking task at six different target sizes. In experiments 2 and 3, they were asked to track a target with the contrast adjusted to individual's threshold. The tracking performance was represented by the kernel function calculated as the cross-correlation between the target and mouse displacements. The peak, latency, and width of the kernel were extracted and compared between the two groups. Results: In experiment 1, target size had a significant effect on the kernel peak (F(1.649, 46.170) = 200.958, P = 4.420 × 10-22). At the smallest target size, the peak in the amblyopic group was significantly lower than that in the normal group (0.089 ± 0.023 vs. 0.107 ± 0.020, t(28) = -2.390, P = 0.024) and correlated with the contrast sensitivity function (r = 0.739, P = 0.002) in the amblyopic eyes. In experiments 2 and 3, with equally visible stimuli, there were still differences in the kernel between the two groups (all Ps < 0.05). Conclusions: When stimulus visibility was compensated, amblyopic participants still showed significantly poorer tracking performance.

Oculomotor routines for perceptual judgments.

In everyday life we frequently make simple visual judgments about object properties, for example, how big or wide is a certain object? Our goal is to test whether there are also task-specific oculomotor routines that support perceptual judgments, similar to the well-established exploratory routines for haptic perception. In a first study, observers saw different scenes with two objects presented in a photorealistic virtual reality environment. Observers were asked to judge which of two objects was taller or wider while gaze was tracked. All tasks were performed with the same set of virtual objects in the same scenes, so that we can compare spatial characteristics of exploratory gaze behavior to quantify oculomotor routines for each task. Width judgments showed fixations around the center of the objects with larger horizontal spread. In contrast, for height judgments, gaze was shifted toward the top of the objects with larger vertical spread. These results suggest specific strategies in gaze behavior that presumably are used for perceptual judgments. To test the causal link between oculomotor behavior and perception, in a second study, observers could freely gaze at the object or we introduced a gaze-contingent setup forcing observers to fixate specific positions on the object. Discrimination performance was similar between free-gaze and the gaze-contingent conditions for width and height judgments. These results suggest that although gaze is adapted for different tasks, performance seems to be based on a perceptual strategy, independent of potential cues that can be provided by the oculomotor system.

Study on the relationship between fixation characteristics and hit rate in psychological procedure training of free throw.

BACKGROUND: Free throw is an important means of scoring in basketball games. With the improvement of basketball competition level and the enhancement of confrontation degree, the number of free throws in the game gradually increases, so the score of free throw will have an important impact on the result of the game. The purpose of this study is to explore the relationship between visual attention characteristics and hit rate of basketball players in free throw psychological procedure training, so as to provide scientific basis for basketball teaching and training. METHODS: Forty players with similar free throw abilities were randomly assigned to the experimental group (10 males, 10 females) and control group (10 males, 10 females). The experimental group was free throw psychological procedure training, while the control group was trained with routine training, Eye movement indices (number of fixations, fixation duration, and pupil dilation) and the free throw hit rate and analyzed before and after the experiment. Group differences were examined using t-tests, while paired sample t-tests were conducted to compare pre- and post-test results within each group. The training time and training times of the two groups were the same. RESULTS: There were significant differences in fixation duration, number of fixations, pupil diameter and free throw hit rate between pre-test and post-test in the experimental group (P < 0.05). Post-test, there were significant differences in number of fixations, fixation duration, pupil diameter and free throw hit rate between the two groups (P < 0.05). There was a significant positive correlation between number of fixations and free throw hit rate in top (P < 0.01), and there was a significant positive correlation between fixation duration and hit rate in front (P < 0.01). CONCLUSIONS: The psychological procedure training can improve the visual information search strategy and information processing ability of free throw, and significantly improve the free throw hit rate. There was a positive correlation between the front fixation time and the free throw hit rate, and there was a positive correlation between the top number of fixations and the free throw hit rate.

Temporal dynamics of ocular torsion and vertical vergence during visual, vestibular, and visuovestibular rotations.

Ocular torsion and vertical divergence reflect the brain's sensorimotor integration of motion through the vestibulo-ocular reflex (VOR) and the optokinetic reflex (OKR) to roll rotations. Torsion and vergence however express different response patterns depending on several motion variables, but research on their temporal dynamics remains limited. This study investigated the onset times of ocular torsion (OT) and vertical vergence (VV) during visual, vestibular, and visuovestibular motion, as well as their relative decay rates following prolonged optokinetic stimulations. Temporal characteristics were retrieved from three separate investigations where the level of visual clutter and acceleration were controlled. Video eye-tracking was used to retrieve the eye-movement parameters from a total of 41 healthy participants across all trials. Ocular torsion consistently initiated earlier than vertical vergence, particularly evident under intensified visual information density, and higher clutter levels were associated with more balanced decay rates. Additionally, stimulation modality and accelerations affected the onsets of both eye movements, with visuovestibular motion triggering earlier responses compared to vestibular motion, and increased accelerations leading to earlier onsets for both movements. The present study showed that joint visuovestibular responses produced more rapid onsets, indicating a synergetic sensorimotor process. It also showed that visual content acted as a fusional force during the decay period, and imposed greater influence over the torsional onset compared to vergence. Acceleration, by contrast, did not affect the temporal relationship between the two eye movements. Altogether, these findings provide insights into the sensorimotor integration of the vestibulo-ocular and optokinetic reflex arcs.

Prefrontal cortex activity and functional organisation in dual-task ocular pursuit is affected by concurrent upper limb movement.

Tracking a moving object with the eyes seems like a simple task but involves areas of prefrontal cortex (PFC) associated with attention, working memory and prediction. Increasing the demand on these processes with secondary tasks can affect eye movements and/or perceptual judgments. This is particularly evident in chronic or acute neurological conditions such as Alzheimer's disease or mild traumatic brain injury. Here, we combined near infrared spectroscopy and video-oculography to examine the effects of concurrent upper limb movement, which provides additional afference and efference that facilitates tracking of a moving object, in a novel dual-task pursuit protocol. We confirmed the expected effects on judgement accuracy in the primary and secondary tasks, as well as a reduction in eye velocity when the moving object was occluded. Although there was limited evidence of oculo-manual facilitation on behavioural measures, performing concurrent upper limb movement did result in lower activity in left medial PFC, as well as a change in PFC network organisation, which was shown by Graph analysis to be locally and globally more efficient. These findings extend upon previous work by showing how PFC is functionally organised to support eye-hand coordination when task demands more closely replicate daily activities.

Eye movements track prioritized auditory features in selective attention to natural speech.

Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention. Strong anatomical overlap with brain regions critical for oculomotor processes suggests a joint network for attention and eye movements. However, the role of this shared network in complex, naturalistic environments remains understudied. Here, we investigated eye movements in relation to (un)attended sentences of natural speech. Combining simultaneously recorded eye tracking and magnetoencephalographic data with temporal response functions, we show that gaze tracks attended speech, a phenomenon we termed ocular speech tracking. Ocular speech tracking even differentiates a target from a distractor in a multi-speaker context and is further related to intelligibility. Moreover, we provide evidence for its contribution to neural differences in speech processing, emphasizing the necessity to consider oculomotor activity in future research and in the interpretation of neural differences in auditory cognition.

A behavioral advantage for the face pareidolia illusion in peripheral vision.

Investigation of visual illusions helps us understand how we process visual information. For example, face pareidolia, the misperception of illusory faces in objects, could be used to understand how we process real faces. However, it remains unclear whether this illusion emerges from errors in face detection or from slower, cognitive processes. Here, our logic is straightforward; if examples of face pareidolia activate the mechanisms that rapidly detect faces in visual environments, then participants will look at objects more quickly when the objects also contain illusory faces. To test this hypothesis, we sampled continuous eye movements during a fast saccadic choice task-participants were required to select either faces or food items. During this task, pairs of stimuli were positioned close to the initial fixation point or further away, in the periphery. As expected, the participants were faster to look at face targets than food targets. Importantly, we also discovered an advantage for food items with illusory faces but, this advantage was limited to the peripheral condition. These findings are among the first to demonstrate that the face pareidolia illusion persists in the periphery and, thus, it is likely to be a consequence of erroneous face detection.

Sustained attention operates via dissociable neural mechanisms across different eccentric locations.

In primates, foveal and peripheral vision have distinct neural architectures and functions. However, it has been debated if selective attention operates via the same or different neural mechanisms across eccentricities. We tested these alternative accounts by examining the effects of selective attention on the steady-state visually evoked potential (SSVEP) and the fronto-parietal signal measured via EEG from human subjects performing a sustained visuospatial attention task. With a negligible level of eye movements, both SSVEP and SND exhibited the heterogeneous patterns of attentional modulations across eccentricities. Specifically, the attentional modulations of these signals peaked at the parafoveal locations and such modulations wore off as visual stimuli appeared closer to the fovea or further away towards the periphery. However, with a relatively higher level of eye movements, the heterogeneous patterns of attentional modulations of these neural signals were less robust. These data demonstrate that the top-down influence of covert visuospatial attention on early sensory processing in human cortex depends on eccentricity and the level of saccadic responses. Taken together, the results suggest that sustained visuospatial attention operates differently across different eccentric locations, providing new understanding of how attention augments sensory representations regardless of where the attended stimulus appears.

Eye tracking evidence for the reinstatement of emotionally negative and neutral memories.

Recent eye tracking studies have linked gaze reinstatement-when eye movements from encoding are reinstated during retrieval-with memory performance. In this study, we investigated whether gaze reinstatement is influenced by the affective salience of information stored in memory, using an adaptation of the emotion-induced memory trade-off paradigm. Participants learned word-scene pairs, where scenes were composed of negative or neutral objects located on the left or right side of neutral backgrounds. This allowed us to measure gaze reinstatement during scene memory tests based on whether people looked at the side of the screen where the object had been located. Across two experiments, we behaviorally replicated the emotion-induced memory trade-off effect, in that negative object memory was better than neutral object memory at the expense of background memory. Furthermore, we found evidence that gaze reinstatement was related to recognition memory for the object and background scene components. This effect was generally comparable for negative and neutral memories, although the effects of valence varied somewhat between the two experiments. Together, these findings suggest that gaze reinstatement occurs independently of the processes contributing to the emotion-induced memory trade-off effect.

Spiking activity in the visual thalamus is coupled to pupil dynamics across temporal scales.

The processing of sensory information, even at early stages, is influenced by the internal state of the animal. Internal states, such as arousal, are often characterized by relating neural activity to a single "level" of arousal, defined by a behavioral indicator such as pupil size. In this study, we expand the understanding of arousal-related modulations in sensory systems by uncovering multiple timescales of pupil dynamics and their relationship to neural activity. Specifically, we observed a robust coupling between spiking activity in the mouse dorsolateral geniculate nucleus (dLGN) of the thalamus and pupil dynamics across timescales spanning a few seconds to several minutes. Throughout all these timescales, 2 distinct spiking modes-individual tonic spikes and tightly clustered bursts of spikes-preferred opposite phases of pupil dynamics. This multi-scale coupling reveals modulations distinct from those captured by pupil size per se, locomotion, and eye movements. Furthermore, coupling persisted even during viewing of a naturalistic movie, where it contributed to differences in the encoding of visual information. We conclude that dLGN spiking activity is under the simultaneous influence of multiple arousal-related processes associated with pupil dynamics occurring over a broad range of timescales.

Eye movement intervention facilitates concurrent perception and memory processing.

A widely used psychotherapeutic treatment for post-traumatic stress disorder (PTSD) involves performing bilateral eye movement (EM) during trauma memory retrieval. However, how this treatment-described as eye movement desensitization and reprocessing (EMDR)-alleviates trauma-related symptoms is unclear. While conventional theories suggest that bilateral EM interferes with concurrently retrieved trauma memories by taxing the limited working memory resources, here, we propose that bilateral EM actually facilitates information processing. In two EEG experiments, we replicated the bilateral EM procedure of EMDR, having participants engaging in continuous bilateral EM or receiving bilateral sensory stimulation (BS) as a control while retrieving short- or long-term memory. During EM or BS, we presented bystander images or memory cues to probe neural representations of perceptual and memory information. Multivariate pattern analysis of the EEG signals revealed that bilateral EM enhanced neural representations of simultaneously processed perceptual and memory information. This enhancement was accompanied by heightened visual responses and increased neural excitability in the occipital region. Furthermore, bilateral EM increased information transmission from the occipital to the frontoparietal region, indicating facilitated information transition from low-level perceptual representation to high-level memory representation. These findings argue for theories that emphasize information facilitation rather than disruption in the EMDR treatment.

Insights into the relationship between eye movements and personality traits in restricted visual fields.

Previous studies have suggested behavioral patterns, such as visual attention and eye movements, relate to individual personality traits. However, these studies mainly focused on free visual tasks, and the impact of visual field restriction remains inadequately understood. The primary objective of this study is to elucidate the patterns of conscious eye movements induced by visual field restriction and to examine how these patterns relate to individual personality traits. Building on previous research, we aim to gain new insights through two behavioral experiments, unraveling the intricate relationship between visual behaviors and individual personality traits. As a result, both Experiment 1 and Experiment 2 revealed differences in eye movements during free observation and visual field restriction. Particularly, simulation results based on the analyzed data showed clear distinctions in eye movements between free observation and visual field restriction conditions. This suggests that eye movements during free observation involve a mixture of conscious and unconscious eye movements. Furthermore, we observed significant correlations between conscious eye movements and personality traits, with more pronounced effects in the visual field restriction condition used in Experiment 2 compared to Experiment 1. These analytical findings provide a novel perspective on human cognitive processes through visual perception.

Scene construction in healthy aging - Exploring the interplay between task complexity and oculomotor behaviour.

Mounting evidence indicates a close correspondence between episodic memory, mental imagery, and oculomotor behaviour. It remains unclear, however, how oculomotor variables support endogenously driven forms of mental imagery and how this relationship changes across the adult lifespan. In this study we investigated age-related changes in oculomotor signatures during scene construction and explored how task complexity impacts these processes. Younger and cognitively healthy older participants completed a guided scene construction paradigm where scene complexity was manipulated according to the number of elements to be sequentially integrated. We recorded participants' eye movements and collected subjective ratings regarding their phenomenological experience. Overall, older adults rated their constructions as more vivid and more spatially integrated, while also generating more fixations and saccades relative to the younger group, specifically on control trials. Analyses of participants' total scan paths revealed that, in the early stages of scene construction, oculomotor behaviour changed as a function of task complexity within each group. Following the introduction of a second stimulus, older but not younger adults showed a significant decrease in the production of eye movements. Whether this shift in oculomotor behaviour serves a compensatory function to bolster task performance represents an important question for future research.

Behind mouse eyes: The function and control of eye movements in mice.

The mouse visual system has become the most popular model to study the cellular and circuit mechanisms of sensory processing. However, the importance of eye movements only started to be appreciated recently. Eye movements provide a basis for predictive sensing and deliver insights into various brain functions and dysfunctions. A plethora of knowledge on the central control of eye movements and their role in perception and behaviour arose from work on primates. However, an overview of various eye movements in mice and a comparison to primates is missing. Here, we review the eye movement types described to date in mice and compare them to those observed in primates. We discuss the central neuronal mechanisms for their generation and control. Furthermore, we review the mounting literature on eye movements in mice during head-fixed and freely moving behaviours. Finally, we highlight gaps in our understanding and suggest future directions for research.

Eye Tracking During Visual Paired-Comparison Tasks: A Systematic Review and Meta-Analysis of the Diagnostic Test Accuracy for Detecting Cognitive Decline.

Background: Alzheimer's disease and mild cognitive impairment (MCI) progress silently, making early diagnosis challenging, especially in less educated populations. The visual paired comparison (VPC) task, utilizing eye-tracking movement (ETM) technology, offers a promising alternative for early detection of memory decline. Objective: This systematic review and meta-analysis evaluated the efficacy of the VPC task, utilizing ETM as a tool for assessing age-related cognitive changes. Methods: A comprehensive search across five databases and grey literature focused on healthy and impaired memory participants assessed through the ETM-based VPC task. The primary outcomes were novelty preference scores and eye movement metrics. The risk of bias of the included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Random-effects meta-analyses calculated Hedges' g effect size. Sensitivity and specificity of the VPC were meta-analytically pooled. Results: The systematic review included 12 articles, involving 1,022 participants (aged 18 to 90 years, with education ranging from 6.5 to 20.0 years), with a low risk of bias and minimal applicability concerns across all items. Five studies contributed to the meta-analysis, revealing a significant effect favoring the VPC task for recognition memory detection (k = 9, g = -1.03). Pooled sensitivity and specificity analyses demonstrated VPC effectiveness as a recognition memory assessment tool (0.84 and 0.75, respectively). Conclusions: The VPC task, utilizing ETM, may serve as a biomarker for early memory decline detection. Its use as a digital eye-tracking tool presents a possible alternative to traditional tests, warranting further research for application in neurodegenerative disease diagnosis.

Quantifying task-related gaze.

Competing theories attempt to explain what guides eye movements when exploring natural scenes: bottom-up image salience and top-down semantic salience. In one study, we apply language-based analyses to quantify the well-known observation that task influences gaze in natural scenes. Subjects viewed ten scenes as if they were performing one of two tasks. We found that the semantic similarity between the task and the labels of objects in the scenes captured the task-dependence of gaze (t(39) = 13.083; p < 0.001). In another study, we examined whether image salience or semantic salience better predicts gaze during a search task, and if viewing strategies are affected by searching for targets of high or low semantic relevance to the scene. Subjects searched 100 scenes for a high- or low-relevance object. We found that image salience becomes a worse predictor of gaze across successive fixations, while semantic salience remains a consistent predictor (X2(1, N=40) = 75.148, p < .001). Furthermore, we found that semantic salience decreased as object relevance decreased (t(39) = 2.304; p = .027). These results suggest that semantic salience is a useful predictor of gaze during task-related scene viewing, and that even in target-absent trials, gaze is modulated by the relevance of a search target to the scene in which it might be located.

What eye and hand movements tell us about expectations towards argument order: An eye- and mouse-tracking study in German.

Previous research on real-time sentence processing in German has shown that listeners use the morphological marking of accusative case on a sentence-initial noun phrase to not only interpret the current argument as the object and patient, but also to predict a plausible agent. So far, less is known about the use of case marking to predict the semantic role of upcoming arguments after the subject/agent has been encountered. In the present study, we examined the use of case marking for argument interpretation in transitive as well as ditransitive structures. We aimed to control for multiple factors that could have influenced processing in previous studies, including the animacy of arguments, world knowledge, and the perceptibility of the case cue. Our results from eye- and mouse-tracking indicate that the exploitation of the first case cue that enables the interpretation of the unfolding sentence is influenced by (i) the strength of argument order expectation and (ii) the perceptual salience of the case cue. PsycINFO code: 2720 Linguistics & Language & Speech.