Understanding the Role of Eye Movement Pattern and Consistency in Isolated English Word Reading Through Hidden Markov Modeling.

In isolated English word reading, readers have the optimal performance when their initial eye fixation is directed to the area between the beginning and word center, that is, the optimal viewing position (OVP). Thus, how well readers voluntarily direct eye gaze to this OVP during isolated word reading may be associated with reading performance. Using Eye Movement analysis with Hidden Markov Models, we discovered two representative eye movement patterns during lexical decisions through clustering, which focused at the OVP and the word center, respectively. Higher eye movement similarity to the OVP-focusing pattern predicted faster lexical decision time in addition to cognitive abilities and lexical knowledge. However, the OVP-focusing pattern was associated with longer isolated single letter naming time, suggesting conflicting visual abilities required for identifying isolated letters and multi-letter words. In contrast, in both word and pseudoword naming, although clustering did not reveal an OVP-focused pattern, higher consistency of the first fixation as measured in entropy predicted faster naming time in addition to cognitive abilities and lexical knowledge. Thus, developing a consistent eye movement pattern focusing on the OVP is essential for word orthographic processing and reading fluency. This finding has important implications for interventions for reading difficulties.

Different temporal dynamics of foveal and peripheral visual processing during fixation.

Humans explore visual scenes by alternating short fixations with saccades directing the fovea to points of interest. During fixation, the visual system not only examines the foveal stimulus at high resolution, but it also processes the extrafoveal input to plan the next saccade. Although foveal analysis and peripheral selection occur in parallel, little is known about the temporal dynamics of foveal and peripheral processing upon saccade landing, during fixation. Here we investigate whether the ability to localize changes across the visual field differs depending on when the change occurs during fixation, and on whether the change localization involves foveal, extrafoveal processing, or both. Our findings reveal that the ability to localize changes in peripheral areas of the visual field improves as a function of time after fixation onset, whereas localization accuracy for foveal stimuli remains approximately constant. Importantly, this pattern holds regardless of whether individuals monitor only foveal or peripheral stimuli, or both simultaneously. Altogether, these results show that the visual system is more attuned to the foveal input early on during fixation, whereas change localization for peripheral stimuli progressively improves throughout fixation, possibly as a consequence of an increased readiness to plan the next saccade.

Sense of agency at a temporally-delayed gaze-contingent display.

The subjective feeling of being the author of one's actions and the subsequent consequences is referred to as a sense of agency. Such a feeling is crucial for usability in human-computer interactions, where eye movement has been adopted, yet this area has been scarcely investigated. We examined how the temporal action-feedback discrepancy affects the sense of agency concerning eye movement. Participants conducted a visual search for an array of nine Chinese characters within a temporally-delayed gaze-contingent display, blurring the peripheral view. The relative delay between each eye movement and the subsequent window movement varied from 0 to 4,000 ms. In the control condition, the window played a recorded gaze behavior. The mean authorship rating and the proportion of "self" responses in the categorical authorship report ("self," "delayed self," and "other") gradually decreased as the temporal discrepancy increased, with "other" being rarely reported, except in the control condition. These results generally mirror those of prior studies on hand actions, suggesting that sense of agency extends beyond the effector body parts to other modalities, and two different types of sense of agency that have different temporal characteristics are simultaneously operating. The mode of fixation duration shifted as the delay increased under 200-ms delays and was divided into two modes at 200-500 ms delays. The frequency of 0-1.5° saccades exhibited an increasing trend as the delay increased. These results demonstrate the influence of perceived action-effect discrepancy on action refinement and task strategy.

Empowering High-Level Spinal Cord Injury Patients in Daily Tasks With a Hybrid Gaze and FEMG-Controlled Assistive Robotic System.

Individuals with high-level spinal cord injuries often face significant challenges in performing essential daily tasks due to their motor impairments. Consequently, the development of reliable, hands-free human-computer interfaces (HCI) for assistive devices is vital for enhancing their quality of life. However, existing methods, including eye-tracking and facial electromyogram (FEMG) control, have demonstrated limitations in stability and efficiency. To address these shortcomings, this paper presents an innovative hybrid control system that seamlessly integrates gaze and FEMG signals. When deployed as a hybrid HCI, this system has been successfully used to assist individuals with high-level spinal cord injuries in performing activities of daily living (ADLs), including tasks like eating, pouring water, and pick-and-place. Importantly, our experimental results confirm that our hybrid control method expedites the performance in pick-place tasks, achieving an average completion time of 34.3 s, which denotes a 28.8% and 21.8% improvement over pure gaze-based control and pure FEMG-based control, respectively. With practice, participants experienced up to a 44% efficiency improvement using the hybrid control method. This state-of-the-art system offers a highly precise and reliable intention interface, suitable for daily use by individuals with high-level spinal cord injuries, ultimately enhancing their quality of life and independence.

Tests of abnormal gaze behavior increase the accuracy of mild cognitive impairment assessments.

Tests to detect mild cognitive impairment (MCI) should have high sensitivity and accuracy. Previously, we reported on a cognitive composition test (CCT) that we developed to detect MCI. In this study, we compared gaze behavior parameters, namely, gaze fixation duration and gaze-shift frequency, during the CCT in participants with MCI and healthy controls (HC) to determine whether these parameters would accurately detect MCI in older adults. Participants performed CCT-A, -B, and -C tasks with varying difficulty levels while wearing eye-tracking devices. Performance time, gaze fixation duration, and gaze-shift frequency were analyzed. Receiver operating characteristic curve analysis was performed to assess the identification accuracy. The MCI group was significantly slower in completing the CCT-C task and had a higher gaze-shift frequency into both the sample object space and workspace than the HC group. Gaze fixation duration in the sample object space increased in the MCI group as the CCT became difficult. Our findings indicated that combining the CCT with performance time and gaze pattern improved the accuracy of distinguishing between individuals with and without MCI and that patients with MCI have abnormal gaze behavior during cognitive tasks. Therefore, evaluation of gaze parameters may improve the accuracy of identifying patients with MCI.

Flicker adaptation improves acuity for briefly presented stimuli by reducing crowding.

Adaptation to flickering/dynamic noise improves visual acuity for briefly presented stimuli (Arnold et al., 2016). Here, we investigate whether such adaptation operates directly on our ability to see detail or by changing fixational eye movements and pupil size or by reducing visual crowding. Following earlier work, visual acuity was measured in observers who were either unadapted or who had adapted to a 60-Hz flickering noise pattern. Participants reported the orientation of a white tumbling-T target (four-alternative forced choice [4AFC], ⊤⊣⊥⊢). The target was presented for 110 ms either in isolation or flanked by randomly oriented T's (e.g., ⊣⊤⊢) followed by an isolated (+) or flanked (+++) mask, respectively. We measured fixation stability (using an infrared eye tracker) while observers performed the task (with and without adaptation). Visual acuity improved modestly (around 8.4%) for flanked optotypes following adaptation to flicker (mean, -0.038 ± 0.063 logMAR; p = 0.015; BF10 = 3.66) but did not when measured with isolated letters (mean, -0.008 ± 0.055 logMAR; p = 0.5; BF10 = 0.29). The magnitude of acuity improvement was associated with individuals' (unadapted) susceptibility to crowding (the ratio of crowded to uncrowded acuity; r = -0.58, p = 0.008, BF10 = 7.70) but to neither fixation stability nor pupil size. Confirming previous reports, flicker improved acuity for briefly presented stimuli, but we show that this was only the case for crowded letters. These improvements likely arise from attenuation of sensitivity to a transient low spatial frequency (SF) image structure (Arnold et al., 2016; Tagoh et al., 2022), which may, for example, reduce masking of high SFs by low SFs. We also suggest that this attenuation could reduce backward masking and so reduce foveal crowding.

How do referees visually explore? An in-situ examination of the referential head and eye movements of football referees.

The majority of a football referee's time is spent assessing open-play situations, yet little is known about how referees search for information during this uninterrupted play. The aim of the current study was to examine the exploratory gaze behaviour of elite and sub-elite football referees in open-play game situations. Four elite (i.e. national) and eight sub-elite (i.e. regional) referees officiated an in-situ football match while wearing a mobile eye-tracker to assess their gaze behaviour. Both referential head and eye movements (i.e. moving gaze away from and then back to the ball) were measured. Results showed gaze behaviour was characterised overall by more referential head than eye movements (~75 vs 25%), which were of longer duration (~950 vs 460 ms). Moreover, elite referees employed faster referential movements (~640 vs 730 ms), spending less time with their gaze away from the ball (carrier) than the sub-elite referees. Crucially, both the referential head and eye movements were coordinated relative to key events in the match, in this case passes, showing that referees anticipate the passes to ensure that the referential movements did not occur during passes, rather before or after. The results further our understanding of the coordinative gaze behaviours that underpin expertise in officiating.

Individual gaze shapes diverging neural representations.

Complex visual stimuli evoke diverse patterns of gaze, but previous research suggests that their neural representations are shared across brains. Here, we used hyperalignment to compare visual responses between observers viewing identical stimuli. We find that individual eye movements enhance cortical visual responses but also lead to representational divergence. Pairwise differences in the spatial distribution of gaze and in semantic salience predict pairwise representational divergence in V1 and inferior temporal cortex, respectively. This suggests that individual gaze sculpts individual visual worlds.

3D-ARM-Gaze: a public dataset of 3D Arm Reaching Movements with Gaze information in virtual reality.

3D-ARM-Gaze is a public dataset designed to provide natural arm movements together with visual and gaze information when reaching objects in a wide reachable space from a precisely controlled, comfortably seated posture. Participants were involved in picking and placing objects in various positions and orientations in a virtual environment, whereby a specific procedure maximized the workspace explored while ensuring a consistent seated posture by guiding participants to a predetermined neutral posture via visual feedback from the trunk and shoulders. These experimental settings enabled to capture natural arm movements with high median success rates (>98% objects reached) and minimal compensatory movements. The dataset regroups more than 2.5 million samples recorded from 20 healthy participants performing 14 000 single pick-and-place movements (700 per participant). While initially designed to explore novel prosthesis control strategies based on natural eye-hand and arm coordination, this dataset will also be useful to researchers interested in core sensorimotor control, humanoid robotics, human-robot interactions, as well as for the development and testing of associated solutions in gaze-guided computer vision.

Eye tracking study in children to assess mental calculation and eye movements.

Eye tracking technology is a high-potential tool for different mathematic cognition research areas. Moreover, there is a dire need for more studies that provide detailed information on the quality of registered eye data. This study aimed to illustrate the applicability of eye tracking in the examination of mathematical cognition, focusing specifically on primary school students completing a computerized mental arithmetic task. Results suggested that the eye tracking device effectively captured high-quality eye movement data when primary school children engaged in this specific task. Furthermore, significant negative correlations have been found between task performance and number of eye fixations. Finally, eye movements distinctions between "Areas of Interest" have been found, indicating different visual tracking associated with different components of arithmetic calculations. This study underscores the extensive possibilities for future research employing eye tracking devices during computerized calculation tasks as assessment tools to explore the complex visual and cognitive processes.

Pupil contagion variation with gaze, arousal, and autistic traits.

Pupillary contagion occurs when one's pupil size unconsciously adapts to the pupil size of an observed individual and is presumed to reflect the transfer of arousal. Importantly, when estimating pupil contagion, low level stimuli properties need to be controlled for, to ensure that observations of pupillary changes are due to internal change in arousal rather than the external differences between stimuli. Here, naturalistic images of children's faces depicting either small or large pupils were presented to a group of children and adolescents with a wide range of autistic traits, a third of whom had been diagnosed with autism. We examined the extent to which pupillary contagion reflects autonomic nervous system reaction through pupil size change, heart rate and skin conductance response. Our second aim was to determine the association between arousal reaction to stimuli and degree of autistic traits. Results show that pupil contagion and concomitant heart rate change, but not skin conductance change, was evident when gaze was restricted to the eye region of face stimuli. A positive association was also observed between pupillary contagion and autistic traits when participants' gaze was constrained to the eye region. Findings add to a broader understanding of the mechanisms underlying pupillary contagion and its association with autism.

Using eye tracking to assess learning of a multifunction prosthetic hand: an exploratory study from a rehabilitation perspective.

BACKGROUND: Eye tracking technology not only reveals the acquisition of visual information at fixation but also has the potential to unveil underlying cognitive processes involved in learning to use a multifunction prosthetic hand. It also reveals gaze behaviours observed during standardized tasks and self-chosen tasks. The aim of the study was to explore the use of eye tracking to track learning progress of multifunction hands at two different time points in prosthetic rehabilitation. METHODS: Three amputees received control training of a multifunction hand with new control strategy. Detailed description of control training was collected first. They wore Tobii Pro2 eye-tracking glasses and performed a set of standardized tasks (required to switch to different grips for each task) after one day of training and at one-year-follow-up (missing data for Subject 3 at the follow up due to socket problem). They also performed a self-chosen task (free to use any grip for any object) and were instructed to perform the task in a way how they would normally do at home. The gaze-overlaid videos were analysed using the Tobii Pro Lab and the following metrics were extracted: fixation duration, saccade amplitude, eye-hand latency, fixation count and time to first fixation. RESULTS: During control training, the subjects learned 3 to 4 grips. Some grips were easier, and others were more difficult because they forgot or were confused with the switching strategies. At the one-year-follow-up, a decrease in performance time, fixation duration, eye-hand latency, and fixation count was observed in Subject 1 and 2, indicating an improvement in the ability to control the multifunction hand and a reduction of cognitive load. An increase in saccade amplitude was observed in both subjects, suggesting a decrease in difficulty to control the prosthetic hand. During the standardized tasks, the first fixation of all three subjects were on the multifunction hand in all objects. During the self-chosen tasks, the first fixations were mostly on the objects first. CONCLUSION: The qualitative data from control training and the quantitative eye tracking data from clinical standardized tasks provided a rich exploration of cognitive processing in learning to control a multifunction hand. Many prosthesis users prefer multifunction hands and with this study we have demonstrated that a targeted prosthetic training protocol with reliable assessment methods will help to lay the foundation for measuring functional benefits of multifunction hands.

Design of a Gaze-Controlled Interactive Art System for the Elderly to Enjoy Life.

The impact of global population aging on older adults' health and emotional well-being is examined in this study, emphasizing innovative technological solutions to address their diverse needs. Changes in physical and mental functions due to aging, along with emotional challenges that necessitate attention, are highlighted. Gaze estimation and interactive art are utilized to develop an interactive system tailored for elderly users, where interaction is simplified through eye movements to reduce technological barriers and provide a soothing art experience. By employing multi-sensory stimulation, the system aims to evoke positive emotions and facilitate meaningful activities, promoting active aging. Named "Natural Rhythm through Eyes", it allows for users to interact with nature-themed environments via eye movements. User feedback via questionnaires and expert interviews was collected during public demonstrations in elderly settings to validate the system's effectiveness in providing usability, pleasure, and interactive experience for the elderly. Key findings include the following: (1) Enhanced usability of the gaze estimation interface for elderly users. (2) Increased enjoyment and engagement through nature-themed interactive art. (3) Positive influence on active aging through the integration of gaze estimation and interactive art. These findings underscore technology's potential to enhance well-being and quality of life for older adults navigating aging challenges.

Social face processing in chronic severe traumatic brain injury: Altered decoding of emotions and mental states but preserved gaze cueing of attention.

The processing of social information transmitted by facial stimuli is altered in individuals with traumatic brain injury (TBI). This study investigated whether these alterations also affect the mechanisms underlying the orienting of visual attention in response to eye-gaze signals. TBI patients and a control group of healthy individuals matched on relevant criteria completed a spatial cueing task. In this task, a lateral visual target was presented along with a task-irrelevant face, with the gaze averted to the left or right. Arrows pointing towards the left or right were also used as non-social control stimuli. Social cognition abilities were further investigated through tests based on decoding emotional expressions and mental states conveyed by facial stimuli. The decoding of emotions and mental states was worse in the TBI group than in the control group. However, both groups demonstrated reliable and comparable orienting of attention to both eye-gaze and arrow stimuli. Despite impairments in certain aspects of social face processing among TBI patients, gaze cueing of attention appears to be preserved in this neuropsychological population.

Prioritization of social information processing: Eye gaze elicits earlier vMMN than arrows.

Numerous research studies have demonstrated that eye gaze and arrows act as cues that automatically guide spatial attention. However, it remains uncertain whether the attention shifts triggered by these two types of stimuli vary in terms of automatic processing mechanisms. In our current investigation, we employed an equal probability paradigm to explore the likenesses and distinctions in the neural mechanisms of automatic processing for eye gaze and arrows in non-attentive conditions, using visual mismatch negative (vMMN) as an indicator of automatic processing. The sample size comprised 17 participants. The results indicated a significant interaction between time duration, stimulus material, and stimulus type. The findings demonstrated that both eye gaze and arrows were processed automatically, triggering an early vMMN, although with temporal variations. The vMMN for eye gaze occurred between 180 and 220 ms, whereas for arrows it ranged from 235 to 275 ms. Moreover, arrow stimuli produced a more pronounced vMMN amplitude. The earlier vMMN response to eye gaze compared with arrows implies the specificity and precedence of social information processing associated with eye gaze over the processing of nonsocial information with arrows. However, arrow could potentially elicit a stronger vMMN because of their heightened salience compared to the background, and the expansion of attention focusing might amplify the vMMN impact. This study offers insights into the similarities and differences in attention processing of social and non-social information under unattended conditions from the perspective of automatic processing.

Clinical Assessment of the Nystagmus Fixation Suppression Test: An Experimental Study.

BACKGROUND:  Assessment of nystagmus fixation suppression can be used as an additional diagnostic tool for patients with an acute vestibular syndrome to distinguish between a central or peripheral cause. We investigated the ability of physicians to detect fixation suppression using a nystagmus simulation model. METHODS:  We used a nystagmus simulator to measure the accuracy of the nystagmus fixation suppression test. Fixation suppression was assessed randomly in 6170 trials by 20 otorhinolaryngologists and neurologists, segregated into 2 groups based on their neurootological experience, a beginner and an experienced group. The simulator presented random nystagmus slow velocity (SPV) reductions and presented 3 conditions with either changed nystagmus frequency, amplitude, or both. RESULTS:  The cutoff for the discernment of fixation suppression ranged from 1.2 to 14°/s nystagmus velocity difference. The more intense the baseline nystagmus was, the more difficult was the detection of fixation suppression. There was not significant difference (P >.05) in the cutoff values in the experts group compared to the novices for all 3 different conditions. Both, novices and experts, detected frequency changes easier than differences of the nystagmus amplitude. Test sensitivity was very low (19%-65%) for discernment of small nystagmus velocity differences of <2°/s by experts. CONCLUSION:  In our study, there was no difference between experts and novices in detection of nystagmus suppression by visual fixation. The examiners could only detect large suppression effects at low-intensity baseline nystagmus. Overall, the sensitivity and accuracy of a clinical fixation suppression test is low and the assistance with a video-oculography device is highly recommended.

Human Eyes-Inspired Recurrent Neural Networks Are More Robust Against Adversarial Noises.

Humans actively observe the visual surroundings by focusing on salient objects and ignoring trivial details. However, computer vision models based on convolutional neural networks (CNN) often analyze visual input all at once through a single feedforward pass. In this study, we designed a dual-stream vision model inspired by the human brain. This model features retina-like input layers and includes two streams: one determining the next point of focus (the fixation), while the other interprets the visuals surrounding the fixation. Trained on image recognition, this model examines an image through a sequence of fixations, each time focusing on different parts, thereby progressively building a representation of the image. We evaluated this model against various benchmarks in terms of object recognition, gaze behavior, and adversarial robustness. Our findings suggest that the model can attend and gaze in ways similar to humans without being explicitly trained to mimic human attention and that the model can enhance robustness against adversarial attacks due to its retinal sampling and recurrent processing. In particular, the model can correct its perceptual errors by taking more glances, setting itself apart from all feedforward-only models. In conclusion, the interactions of retinal sampling, eye movement, and recurrent dynamics are important to human-like visual exploration and inference.

Foveolar Drusen Decrease Fixation Stability in Pre-Symptomatic AMD.

Purpose: This study aims at linking subtle changes of fixational eye movements (FEM) in controls and in patients with foveal drusen using adaptive optics retinal imaging in order to find anatomo-functional markers for pre-symptomatic age-related macular degeneration (AMD). Methods: We recruited 7 young controls, 4 older controls, and 16 patients with presymptomatic AMD with foveal drusen from the Silversight Cohort. A high-speed research-grade adaptive optics flood illumination ophthalmoscope (AO-FIO) was used for monocular retinal tracking of fixational eye movements. The system allows for sub-arcminute resolution, and high-speed and distortion-free imaging of the foveal area. Foveal drusen position and size were documented using gaze-dependent imaging on a clinical-grade AO-FIO. Results: FEM were measured with high precision (RMS-S2S = 0.0015 degrees on human eyes) and small foveal drusen (median diameter = 60 µm) were detected with high contrast imaging. Microsaccade amplitude, drift diffusion coefficient, and ISOline area (ISOA) were significantly larger for patients with foveal drusen compared with controls. Among the drusen participants, microsaccade amplitude was correlated to drusen eccentricity from the center of the fovea. Conclusions: A novel high-speed high-precision retinal tracking technique allowed for the characterization of FEM at the microscopic level. Foveal drusen altered fixation stability, resulting in compensatory FEM changes. Particularly, drusen at the foveolar level seemed to have a stronger impact on microsaccade amplitudes and ISOA. The unexpected anatomo-functional link between small foveal drusen and fixation stability opens up a new perspective of detecting oculomotor signatures of eye diseases at the presymptomatic stage.

Correcting for ERP latency jitter improves gaze-independent BCI decoding.

Objective.Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze.Approach.ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment.Main results.WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session.Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.