Automated Analysis Pipeline for Extracting Saccade, Pupil, and Blink Parameters Using Video-Based Eye Tracking.

The tremendous increase in the use of video-based eye tracking has made it possible to collect eye tracking data from thousands of participants. The traditional procedures for the manual detection and classification of saccades and for trial categorization (e.g., correct vs. incorrect) are not viable for the large datasets being collected. Additionally, video-based eye trackers allow for the analysis of pupil responses and blink behaviors. Here, we present a detailed description of our pipeline for collecting, storing, and cleaning data, as well as for organizing participant codes, which are fairly lab-specific but nonetheless, are important precursory steps in establishing standardized pipelines. More importantly, we also include descriptions of the automated detection and classification of saccades, blinks, "blincades" (blinks occurring during saccades), and boomerang saccades (two nearly simultaneous saccades in opposite directions where speed-based algorithms fail to split them), This is almost entirely task-agnostic and can be used on a wide variety of data. We additionally describe novel findings regarding post-saccadic oscillations and provide a method to achieve more accurate estimates for saccade end points. Lastly, we describe the automated behavior classification for the interleaved pro/anti-saccade task (IPAST), a task that probes voluntary and inhibitory control. This pipeline was evaluated using data collected from 592 human participants between 5 and 93 years of age, making it robust enough to handle large clinical patient datasets. In summary, this pipeline has been optimized to consistently handle large datasets obtained from diverse study cohorts (i.e., developmental, aging, clinical) and collected across multiple laboratory sites.

Attentional modulation of eye blinking is altered by sex, age, and task structure.

Spontaneous eye blinking is gaining popularity as a proxy for higher cognitive functions, as it is readily modulated by both environmental demands and internal processes. Prior studies were impoverished in sample size, sex representation and age distribution, making it difficult to establish a complete picture of the behavior. Here we present eye-tracking data from a large cohort of normative participants (n=604, 393 F, aged 5-93 years) performing two tasks: one with structured, discrete trials (interleaved pro/anti-saccade task; IPAST) and one with a less structured, continuous organization in which participants watch movies (free-viewing; FV). Sex- and age-based analyses revealed that females had higher blink rates between the ages of 22 and 58 years in the IPAST, and 22 and 34 years in FV. We derived a continuous measure of blink probability to reveal behavioral changes driven by stimulus appearance in both paradigms. In the IPAST, blinks were suppressed near stimulus appearance, particularly on correct anti-saccade trials, which we attribute to the stronger inhibitory control required for anti-saccades compared to pro-saccades. In FV, blink suppression occurred immediately after scene changes, and the effect was sustained on scenes where gaze clustered among participants (indicating engagement of attention). Females were more likely than males to blink during appearance of novel stimuli in both tasks, but only within the age bin of 18-44 years. The consistency of blink patterns in each paradigm endorses blinking as a sensitive index for changes in visual processing and attention, while sex and age differences drive interindividual variability.Significance Statement Eye-tracking is becoming useful as a non-invasive tool for detecting preclinical markers of neurological and psychiatric disease. Blinks are understudied despite being an important supplement to saccade and pupil eye-tracking metrics. The present study is a crucial step in developing a healthy baseline for blink behavior to compare to clinical groups. While many prior blink studies suffered from small sample sizes with relatively low age- and sex-diversity (review by Jongkees & Colzato, 2016), our large cohort of healthy participants has permitted a more detailed analysis of sex and age effects in blink behavior. Furthermore, our analysis techniques are robust to temporal changes in blink probability, greatly clarifying the relationship between blinking, visual processing, and inhibitory control mechanisms on visual tasks.

Cognitive correlates of antisaccade behaviour across multiple neurodegenerative diseases.

Oculomotor tasks generate a potential wealth of behavioural biomarkers for neurodegenerative diseases. Overlap between oculomotor and disease-impaired circuitry reveals the location and severity of disease processes via saccade parameters measured from eye movement tasks such as prosaccade and antisaccade. Existing studies typically examine few saccade parameters in single diseases, using multiple separate neuropsychological test scores to relate oculomotor behaviour to cognition; however, this approach produces inconsistent, ungeneralizable results and fails to consider the cognitive heterogeneity of these diseases. Comprehensive cognitive assessment and direct inter-disease comparison are crucial to accurately reveal potential saccade biomarkers. We remediate these issues by characterizing 12 behavioural parameters, selected to robustly describe saccade behaviour, derived from an interleaved prosaccade and antisaccade task in a large cross-sectional data set comprising five disease cohorts (Alzheimer's disease/mild cognitive impairment, amyotrophic lateral sclerosis, frontotemporal dementia, Parkinson's disease, and cerebrovascular disease; n = 391, age 40-87) and healthy controls (n = 149, age 42-87). These participants additionally completed an extensive neuropsychological test battery. We further subdivided each cohort by diagnostic subgroup (for Alzheimer's disease/mild cognitive impairment and frontotemporal dementia) or degree of cognitive impairment based on neuropsychological testing (all other cohorts). We sought to understand links between oculomotor parameters, their relationships to robust cognitive measures, and their alterations in disease. We performed a factor analysis evaluating interrelationships among the 12 oculomotor parameters and examined correlations of the four resultant factors to five neuropsychology-based cognitive domain scores. We then compared behaviour between the abovementioned disease subgroups and controls at the individual parameter level. We theorized that each underlying factor measured the integrity of a distinct task-relevant brain process. Notably, Factor 3 (voluntary saccade generation) and Factor 1 (task disengagements) significantly correlated with attention/working memory and executive function scores. Factor 3 also correlated with memory and visuospatial function scores. Factor 2 (pre-emptive global inhibition) correlated only with attention/working memory scores, and Factor 4 (saccade metrics) correlated with no cognitive domain scores. Impairment on several mostly antisaccade-related individual parameters scaled with cognitive impairment across disease cohorts, while few subgroups differed from controls on prosaccade parameters. The interleaved prosaccade and antisaccade task detects cognitive impairment, and subsets of parameters likely index disparate underlying processes related to different cognitive domains. This suggests that the task represents a sensitive paradigm that can simultaneously evaluate a variety of clinically relevant cognitive constructs in neurodegenerative and cerebrovascular diseases and could be developed into a screening tool applicable to multiple diagnoses.

Classification and staging of Parkinson's disease using video-based eye tracking.

INTRODUCTION: 83% of those diagnosed with Parkinson's Disease (PD) eventually progress to PD with mild cognitive impairment (PD-MCI) followed by dementia (PDD) - suggesting a complex spectrum of pathology concomitant with aging. Biomarkers sensitive and specific to this spectrum are required if useful diagnostics are to be developed that may supplement current clinical testing procedures. We used video-based eye tracking and machine learning to develop a simple, non-invasive test sensitive to PD and the stages of cognitive dysfunction. METHODS: From 121 PD (45 Cognitively Normal/45 MCI/20 Dementia/11 Other) and 106 healthy controls, we collected video-based eye tracking data on an interleaved pro/anti-saccade task. Features of saccade, pupil, and blink behavior were used to train a classifier to predict confidence scores for PD/PD-MCI/PDD diagnosis. RESULTS: The Receiver Operator Characteristic Area Under the Curve (ROC-AUC) of the classifier was 0.88, with the cognitive-dysfunction subgroups showing progressively increased AUC, and the AUC of PDD being 0.95. The classifier reached a sensitivity of 83% and a specificity of 78%. The confidence scores predicted PD motor and cognitive performance scores. CONCLUSION: Biomarkers of saccade, pupil, and blink were extracted from video-based eye tracking to create a classifier with high sensitivity to the landscape of PD cognitive and motor dysfunction. A complex landscape of PD is revealed through a quick, non-invasive eye tracking task and our model provides a framework for such a task to be used as a supplementary screening tool in the clinic.

Interleaved Pro/Anti-saccade Behavior Across the Lifespan.

The capacity for inhibitory control is an important cognitive process that undergoes dynamic changes over the course of the lifespan. Robust characterization of this trajectory, considering age continuously and using flexible modeling techniques, is critical to advance our understanding of the neural mechanisms that differ in healthy aging and neurological disease. The interleaved pro/anti-saccade task (IPAST), in which pro- and anti-saccade trials are randomly interleaved within a block, provides a simple and sensitive means of assessing the neural circuitry underlying inhibitory control. We utilized IPAST data collected from a large cross-sectional cohort of normative participants (n = 604, 5-93 years of age), standardized pre-processing protocols, generalized additive modeling, and change point analysis to investigate the effect of age on saccade behavior and identify significant periods of change throughout the lifespan. Maturation of IPAST measures occurred throughout adolescence, while subsequent decline began as early as the mid-20s and continued into old age. Considering pro-saccade correct responses and anti-saccade direction errors made at express (short) and regular (long) latencies was crucial in differentiating developmental and aging processes. We additionally characterized the effect of age on voluntary override time, a novel measure describing the time at which voluntary processes begin to overcome automated processes on anti-saccade trials. Drawing on converging animal neurophysiology, human neuroimaging, and computational modeling literature, we propose potential frontal-parietal and frontal-striatal mechanisms that may mediate the behavioral changes revealed in our analysis. We liken the models presented here to "cognitive growth curves" which have important implications for improved detection of neurological disease states that emerge during vulnerable windows of developing and aging.

Eye tracking identifies biomarkers in α-synucleinopathies versus progressive supranuclear palsy.

OBJECTIVES: This study (1) describes and compares saccade and pupil abnormalities in patients with manifest alpha-synucleinopathies (αSYN: Parkinson's disease (PD), Multiple System Atrophy (MSA)) and a tauopathy (progressive supranuclear palsy (PSP)); (2) determines whether patients with rapid-eye-movement sleep behaviour disorder (RBD), a prodromal stage of αSYN, already have abnormal responses that may indicate a risk for developing PD or MSA. METHODS: Ninety (46 RBD, 27 PD, 17 MSA) patients with an αSYN, 10 PSP patients, and 132 healthy age-matched controls (CTRL) were examined with a 10-min video-based eye-tracking task (Free Viewing). Participants were free to look anywhere on the screen while saccade and pupil behaviours were measured. RESULTS: PD, MSA, and PSP spent more time fixating the centre of the screen than CTRL. All patient groups made fewer macro-saccades (> 2◦ amplitude) with smaller amplitude than CTRL. Saccade frequency was greater in RBD than in other patients. Following clip change, saccades were temporarily suppressed, then rebounded at a slower pace than CTRL in all patient groups. RBD had distinct, although discrete saccade abnormalities that were more marked in PD, MSA, and even more in PSP. The vertical saccade rate was reduced in all patients and decreased most in PSP. Clip changes produced large increases or decreases in screen luminance requiring pupil constriction or dilation, respectively. PSP elicited smaller pupil constriction/dilation responses than CTRL, while MSA elicited the opposite. CONCLUSION: RBD patients already have discrete but less pronounced saccade abnormalities than PD and MSA patients. Vertical gaze palsy and altered pupil control differentiate PSP from αSYN.

Investigating Arousal, Saccade Preparation, and Global Luminance Effects on Microsaccade Behavior.

Microsaccades, small saccadic eye movements occurring during fixation, have been suggested to be modulated by various sensory, cognitive, and affective processes relating to arousal. Although the modulation of fatigue-related arousal on microsaccade behavior has previously been characterized, the influence of other aspects of arousal, such as emotional arousal, is less understood. Moreover, microsaccades are modulated by cognitive processes (e.g., voluntary saccade preparation) that could also be linked to arousal. To investigate the influence of emotional arousal, saccade preparation, and global luminance levels on microsaccade behavior, emotional auditory stimuli were presented prior to the onset of a fixation cue whose color indicated to look either at the peripheral stimulus (pro-saccade) or in the opposite direction of the stimulus (anti-saccade). Microsaccade behavior was found to be significantly modulated by saccade preparation and global luminance level, but not emotional arousal. In the pro- and anti-saccade task, microsaccade rate was lower during anti-saccade preparation as compared to pro-saccade preparation, though microsaccade dynamics were comparable during both trial types. Our results reveal a differential role of arousal linked to emotion, fatigue, saccade preparation, and global luminance level on microsaccade behavior.

Using an emotional saccade task to characterize executive functioning and emotion processing in attention-deficit hyperactivity disorder and bipolar disorder.

Despite distinct diagnostic criteria, attention-deficit hyperactivity disorder (ADHD) and bipolar disorder (BD) share cognitive and emotion processing deficits that complicate diagnoses. The goal of this study was to use an emotional saccade task to characterize executive functioning and emotion processing in adult ADHD and BD. Participants (21 control, 20 ADHD, 20 BD) performed an interleaved pro/antisaccade task (look toward vs. look away from a visual target, respectively) in which the sex of emotional face stimuli acted as the cue to perform either the pro- or antisaccade. Both patient groups made more direction (erroneous prosaccades on antisaccade trials) and anticipatory (saccades made before cue processing) errors than controls. Controls exhibited lower microsaccade rates preceding correct anti- vs. prosaccade initiation, but this task-related modulation was absent in both patient groups. Regarding emotion processing, the ADHD group performed worse than controls on neutral face trials, while the BD group performed worse than controls on trials presenting faces of all valence. These findings support the role of fronto-striatal circuitry in mediating response inhibition deficits in both ADHD and BD, and suggest that such deficits are exacerbated in BD during emotion processing, presumably via dysregulated limbic system circuitry involving the anterior cingulate and orbitofrontal cortex.

Comparing Pupil Light Response Modulation between Saccade Planning and Working Memory.

The signature of spatial attention effects has been demonstrated through saccade planning and working memory. Although saccade planning and working memory have been commonly linked to attention, the comparison of effects resulting from saccade planning and working memory is less explored. It has recently been shown that spatial attention interacts with local luminance at the attended location. When bright and dark patch stimuli are presented simultaneously in the periphery, thereby producing no change in global luminance, pupil size is nonetheless smaller when the locus of attention overlaps with the bright, compared to the dark patch stimulus (referred to as the local luminance modulation). Here, we used the local luminance modulation to directly compare the effects of saccade planning and spatial working memory. Participants were required to make a saccade towards a visual target location (visual-delay) or a remembered target location (memory-delay) after a variable delay, and the bright and dark patch stimuli were presented during the delay period between target onset and go signal. Greater pupil constriction was observed when the bright patch, compared to the dark patch, was spatially aligned with the target location in both tasks. However, the effects were diminished when there was no contingency implemented between the patch and target locations, particularly in the memory-delay task. Together, our results suggest the involvement of similar, but not identical, attentional mechanisms through saccade planning and working memory, and highlight a promising potential of local pupil luminance responses for probing visuospatial processing.