Microsaccade rate activity during the preparation of pro- and antisaccades.

Microsaccades belong to the category of fixational micromovements and may be crucial for image stability on the retina. Eye movement paradigms typically require fixational control, but this does not eliminate all oculomotor activity. The antisaccade task requires a planned eye movement in the direction opposite of an onset, allowing separation of planning and execution. We build on previous studies of microsaccades in the antisaccade task using a combination of fixed and mixed pro- and antisaccade blocks. We hypothesized that microsaccade rates may be reduced prior to the execution of antisaccades as compared with regular saccades (prosaccades). In two experiments, we measured microsaccades in four conditions across three trial blocks: one block each of fixed prosaccade and antisaccade trials, and a mixed block where both saccade types were randomized. We anticipated that microsaccade rates would be higher prior to antisaccades than prosaccades due to the need to preemptively suppress reflexive saccades during antisaccade generation. In Experiment 1, with monocular eye tracking, there was an interaction between the effects of saccade and block type on microsaccade rates, suggesting lower rates on antisaccade trials, but only within mixed blocks. In Experiment 2, eye tracking was binocular, revealing suppressed microsaccade rates on antisaccade trials. A cluster permutation analysis of the microsaccade rate over the course of a trial did not reveal any particular critical time for this difference in microsaccade rates. Our findings suggest that microsaccade rates reflect the degree of suppression of the oculomotor system during the antisaccade task.

Bayesian approximations to the theory of visual attention (TVA) in a foraging task.

Foraging as a natural visual search for multiple targets has increasingly been studied in humans in recent years. Here, we aimed to model the differences in foraging strategies between feature and conjunction foraging tasks found by Á. Kristjánsson et al. Bundesen proposed the theory of visual attention (TVA) as a computational model of attentional function that divides the selection process into filtering and pigeonholing. The theory describes a mechanism by which the strength of sensory evidence serves to categorise elements. We combined these ideas to train augmented Naïve Bayesian classifiers using data from Á. Kristjánsson et al. as input. Specifically, we attempted to answer whether it is possible to predict how frequently observers switch between different target types during consecutive selections (switches) during feature and conjunction foraging using Bayesian classifiers. We formulated 11 new parameters that represent key sensory and bias information that could be used for each selection during the foraging task and tested them with multiple Bayesian models. Separate Bayesian networks were trained on feature and conjunction foraging data, and parameters that had no impact on the model's predictability were pruned away. We report high accuracy for switch prediction in both tasks from the classifiers, although the model for conjunction foraging was more accurate. We also report our Bayesian parameters in terms of their theoretical associations with TVA parameters, πj (denoting the pertinence value), and ßi (denoting the decision-making bias).

Eye movement behavior in a real-world virtual reality task reveals ADHD in children.

Eye movements and other rich data obtained in virtual reality (VR) environments resembling situations where symptoms are manifested could help in the objective detection of various symptoms in clinical conditions. In the present study, 37 children with attention deficit hyperactivity disorder and 36 typically developing controls (9-13 y.o) played a lifelike prospective memory game using head-mounted display with inbuilt 90 Hz eye tracker. Eye movement patterns had prominent group differences, but they were dispersed across the full performance time rather than associated with specific events or stimulus features. A support vector machine classifier trained on eye movement data showed excellent discrimination ability with 0.92 area under curve, which was significantly higher than for task performance measures or for eye movements obtained in a visual search task. We demonstrated that a naturalistic VR task combined with eye tracking allows accurate prediction of attention deficits, paving the way for precision diagnostics.

Between the Scenes.

We constantly move our eyes to new information while inspecting a scene, but these patterns of eye movements change based on the task and goals of the observer. Inhibition of return (IOR) may facilitate visual search by reducing the likelihood of revisiting previously attended locations. However, IOR may present in any visual task, or it may be search-specific. We investigated the presence of IOR in foraging, memorization, change detection, and two versions of visual search. One version of search used a static search array that remained stable throughout the trial, but the second used a scene flickering paradigm similar to the change detection task. IOR was observed in both versions of visual search, memorization, and foraging, but not in change detection. Visual search and change detection both had temporal nonscene components, and we observed that IOR could be maintained despite the scene removal but only for search. Although IOR is maintained in scene coordinates, short disruptions to this scene are insufficient to completely remove the inhibitory tags. Finally, we compare return saccades in trials without a probe and observe fewer return saccades in tasks for which IOR was observed, providing further evidence that IOR might serve as a novelty drive.

Comparing saccadic and manual responses in the attention network test.

Attention is proposed to be a system of multiple functional networks, including alertness, orienting and executive control. A popular experimental paradigm for testing these networks and their interactions within a single design is the Attentional Networks Test (ANT) (Fan et al., 2002). The role of the oculomotor system in these various networks, however, has not been tested despite the strong link between attention and eye movements. We modified the executive control component of the manual response ANT version (ANTm) that allows testing the networks' involvement with oculomotor responses. Specifically, we used a central target to signal pro or anti-saccades that allows us to match the saccadic response compatibility of the original ANTm. We conducted three experiments to compare interactions of the networks between the traditional ANTm that used a flanker task response, our new ANTs with saccadic responses signalled with a fixation arrow, and a manual response version with the response arrow at fixation (ANTf). Results for all three experiments showed typical main effects of all three attention networks, but we observed differences in their interactions. The ANTm showed only an interaction of alerting enhancing the orienting; ANTs showed a congruency by orienting interaction with the orienting effect only observed for pro-saccades. The ANTf showed both alerting by orienting, and orienting by congruency. Although the saccadic response did differ from the original ANTm, key differences were also highlighted by the switch from peripheral to central target. Overall the proposed ANTf is a valid tool to test main effects of attentional networks. Further investigation of interaction differences between manual and oculomotor systems is required.

Inhibition of return: An information processing theory of its natures and significance.

Inhibition of return (IOR) is an inhibitory aftereffect of visuospatial orienting, typically resulting in slower responses to targets presented in an area that has been recently attended. Since its discovery, myriad research has sought to explain the causes and effects underlying this phenomenon. Here, we briefly summarize the history of the phenomenon, and describe the early work supporting the functional significance of IOR as a foraging facilitator. We then shine a light on the discordance in the literature with respect to mechanism-in particular the lack of theoretical constructs that can consistently explain innumerable dissociations. We then describe three diagnostics (central arrow targets, locus of slack logic and the psychological refractory period, and performance in speed-accuracy space) used to support our theory that there are two forms of inhibition of return-the form which is manifest being contingent upon the activation state of the reflexive oculomotor system. The input form, which operates to decrease the salience of inputs, is generated when the reflexive oculomotor system is suppressed; the output form, which operates to bias responding, is generated when the reflexive oculomotor system is not suppressed. Then, we subject a published data set, wherein inhibitory effects had been generated while the reflexive oculomotor system was either active or suppressed, to diffusion modelling. As we hypothesized, based on the aforementioned theory, the effects of the two forms of IOR were best accounted for by different drift diffusion parameters. The paper ends with a variety of suggestions for further research.

No evidence for an independent retinotopic reference frame for inhibition of return.

Inhibition of return (IOR) represents a delay in responding to a previously inspected location and is viewed as a crucial mechanism that sways attention toward novelty in visual search. Although most visual processing occurs in retinotopic, eye-centered, coordinates, IOR must be coded in spatiotopic, environmental, coordinates to successfully serve its role as a foraging facilitator. Early studies supported this suggestion but recent results have shown that both spatiotopic and retinotopic reference frames of IOR may co-exist. The present study tested possible sources for IOR at the retinotopic location including being part of the spatiotopic IOR gradient, part of hemifield inhibition and being an independent source of IOR. We conducted four experiments that alternated the cue-target spatial distance (discrete and contiguous) and the response modality (manual and saccadic). In all experiments, we tested spatiotopic, retinotopic and neutral (neither spatiotopic nor retinotopic) locations. We did find IOR at both the retinotopic and spatiotopic locations but no evidence for an independent source of retinotopic IOR for either of the response modalities. In fact, we observed the spread of IOR across entire validly cued hemifield including at neutral locations. We conclude that these results indicate a strategy to inhibit the whole cued hemifield or suggest a large horizontal gradient around the spatiotopically cued location. PUBLIC SIGNIFICANCE STATEMENT: We perceive the visual world around us as stable despite constant shifts of the retinal image due to saccadic eye movements. In this study, we explore whether Inhibition of return (IOR), a mechanism preventing us from returning to previously attended locations, operates in spatiotopic, world-centered or in retinal, eye-centered coordinates. We tested both saccadic and manual IOR at spatiotopic, retinotopic, and control locations. We did not find an independent retinotopic source of IOR for either of the response modalities. The results suggest that IOR spreads over the whole previously attended visual hemifield or there is a large horizontal spatiotopic gradient. The current results are in line with the idea of IOR being a foraging facilitator in visual search and contribute to our understanding of spatiotopically organized aspects of visual and attentional systems.

No Advantage for Separating Overt and Covert Attention in Visual Search.

We move our eyes roughly three times every second while searching complex scenes, but covert attention helps to guide where we allocate those overt fixations. Covert attention may be allocated reflexively or voluntarily, and speeds the rate of information processing at the attended location. Reducing access to covert attention hinders performance, but it is not known to what degree the locus of covert attention is tied to the current gaze position. We compared visual search performance in a traditional gaze-contingent display, with a second task where a similarly sized contingent window is controlled with a mouse, allowing a covert aperture to be controlled independently by overt gaze. Larger apertures improved performance for both the mouse- and gaze-contingent trials, suggesting that covert attention was beneficial regardless of control type. We also found evidence that participants used the mouse-controlled aperture somewhat independently of gaze position, suggesting that participants attempted to untether their covert and overt attention when possible. This untethering manipulation, however, resulted in an overall cost to search performance, a result at odds with previous results in a change blindness paradigm. Untethering covert and overt attention may therefore have costs or benefits depending on the task demands in each case.

Russian blues reveal the limits of language influencing colour discrimination.

Chromatic stimuli across a boundary of basic colour categories (BCCs; e.g. blue and green) are discriminated faster than colorimetrically equidistant colours within a given category. Russian has two BCCs for blue, sinij 'dark blue' and goluboj 'light blue'. These language-specific BCCs were reported to enable native Russian speakers to discriminate cross-boundary dark and light blues faster than English speakers (Winawer et al., 2007, PNAS, 4, 7780-7785). We re-evaluated this finding in two experiments that employed identical tasks as in the cited study. In Experiment 1, Russian and English speakers categorised colours as sinij/goluboj or dark blue/light blue respectively; this was followed by a colour discrimination task. In Experiment 2, Russian speakers initially performed the discrimination task on sinij/goluboj and goluboj/zelënyj 'green' sets. They then categorised these colours in three frequency contexts with each stimulus presented: (i) an equal number of times (unbiased); more frequent (ii) either sinij or goluboj; (iii) either goluboj or zelënyj. We observed a boundary response speed advantage for goluboj/zelënyj but not for sinij/goluboj. The frequency bias affected only the sinij/goluboj boundary such that in a lighter context, the boundary shifted towards lighter shades, and vice versa. Contrary to previous research, our results show that in Russian, stimulus discrimination at the lightness-defined blue BCC boundary is not reflected in processing speed. The sinij/goluboj boundary did have a sharper categorical transition than the dark blue/light blue boundary, but it was also affected by frequency and order biases, demonstrating that "Russian blues" are less well-structured than previously thought.

Salience Models: A Computational Cognitive Neuroscience Review.

The seminal model by Laurent Itti and Cristoph Koch demonstrated that we can compute the entire flow of visual processing from input to resulting fixations. Despite many replications and follow-ups, few have matched the impact of the original model-so what made this model so groundbreaking? We have selected five key contributions that distinguish the original salience model by Itti and Koch; namely, its contribution to our theoretical, neural, and computational understanding of visual processing, as well as the spatial and temporal predictions for fixation distributions. During the last 20 years, advances in the field have brought up various techniques and approaches to salience modelling, many of which tried to improve or add to the initial Itti and Koch model. One of the most recent trends has been to adopt the computational power of deep learning neural networks; however, this has also shifted their primary focus to spatial classification. We present a review of recent approaches to modelling salience, starting from direct variations of the Itti and Koch salience model to sophisticated deep-learning architectures, and discuss the models from the point of view of their contribution to computational cognitive neuroscience.

Temporal Limitations of the Standard Leaky Integrate and Fire Model.

Itti and Koch's Saliency Model has been used extensively to simulate fixation selection in a variety of tasks from visual search to simple reaction times. Although the Saliency Model has been tested for its spatial prediction of fixations in visual salience, it has not been well tested for their temporal accuracy. Visual tasks, like search, invariably result in a positively skewed distribution of saccadic reaction times over large numbers of samples, yet we show that the leaky integrate and fire (LIF) neuronal model included in the classic implementation of the model tends to produce a distribution shifted to shorter fixations (in comparison with human data). Further, while parameter optimization using a genetic algorithm and Nelder-Mead method does improve the fit of the resulting distribution, it is still unable to match temporal distributions of human responses in a visual task. Analysis of times for individual images reveal that the LIF algorithm produces initial fixation durations that are fixed instead of a sample from a distribution (as in the human case). Only by aggregating responses over many input images do they result in a distribution, although the form of this distribution still depends on the input images used to create it and not on internal model variability.

No supplementary evidence of attention to a spatial cue when saccadic facilitation is absent.

Attending a location in space facilitates responses to targets at that location when the time between cue and target is short. Certain types of exogenous cues - such as sudden peripheral onsets - have been described as reflexive and automatic. Recent studies however, have been showing many cases where exogenous cues are less automatic than previously believed and do not always result in facilitation. A lack of the behavioral facilitation, however, does not automatically necessitate a lack of underlying attention to that location. We test exogenous cueing in two experiments where facilitation is and is not likely to be observed with saccadic responses. We also test alternate measures linked to the allocation of attention such as saccadic curvature, microsaccades and pupil size. As expected, we find early facilitation as measured by saccadic reaction time when CTOAs are predictable but not when they are randomized within a block. We find no impact of the cue on microsaccade direction for either experiment, and only a slight dip in the frequency of microsaccades after the cue. We do find that change in pupil size to the cue predicts the magnitude of the validity effect, but only in the experiment where facilitation was observed. In both experiments, we observed a tendency for saccadic curvature to deviate away from the cued location and this was stronger for early CTOAs and toward vertical targets. Overall, we find that only change in pupil size is consistent with observed facilitation. Saccadic curvature is influenced by the onset of the cue, buts its direction is indicative of oculomotor inhibition whether we see RT facilitation or not. Microsaccades were not diagnostic in either experiment. Finally, we see little to no evidence of attention at the cued location in any additional measures when facilitation of saccadic responses is absent.

Implicit processing during change blindness revealed with mouse-contingent and gaze-contingent displays.

People often miss salient events that occur right in front of them. This phenomenon, known as change blindness, reveals the limits of visual awareness. Here, we investigate the role of implicit processing in change blindness using an approach that allows partial dissociation of covert and overt attention. Traditional gaze-contingent paradigms adapt the display in real time according to current gaze position. We compare such a paradigm with a newly designed mouse-contingent paradigm where the visual display changes according to the real-time location of a user-controlled mouse cursor, effectively allowing comparison of change detection with mainly overt attention (gaze-contingent display; Experiment 2) and untethered overt and covert attention (mouse-contingent display; Experiment 1). We investigate implicit indices of target detection during change blindness in eye movement and behavioral data, and test whether affective devaluation of unnoticed targets may contribute to change blindness. The results show that unnoticed targets are processed implicitly, but that the processing is shallower than if the target is consciously detected. Additionally, the partial untethering of covert attention with the mouse-contingent display changes the pattern of search and leads to faster detection of the changing target. Finally, although it remains possible that the deployment of covert attention is linked to implicit processing, the results fall short of establishing a direct connection.

Temporal ambiguity of onsets in a cueing task prevents facilitation but not inhibition of return.

Cueing effects, i.e., early facilitation of reaction time and inhibition of return (IOR), are well-established and robust phenomena characterizing exogenous orienting and are widely observed in experiments with a traditional Posner cueing paradigm. Krüger, MacInnes, and Hunt (2014) proposed that facilitatory effects of peripheral cues are the result of a cue-target perceptual merging due to re-entrant visual processing. To test the role and timing of these feedback mechanisms in peripheral cueing effects, we modified the traditional cueing task in Experiments 1-3 by interleaving pre- and post-cue trials at the valid and invalid location and random cue-target onset asynchrony (CTOA) ranging from -300 to +1,000 ms. Analysis of the manual reaction time distribution over CTOA showed well-pronounced IOR in the valid pre-cue condition and a small cost of perceptual merging in the post-cue condition, but no early facilitation of reaction time was observed in the pre-cue condition. In Experiment 4, we tested directly whether temporal ambiguity eliminated facilitation by restricting CTOAs to only the pre-cue time range and including a between-subject manipulation of a) random, b) mixed discrete, and c) blocked discrete CTOAs. Results obtained in the continuous and binned conditions showed no facilitation but robust IOR. We found both early facilitation and IOR in the blocked condition. Overall, the present findings show a small perceptual merging result without accompanying facilitation, suggesting different underlying mechanisms. Second, they demonstrate that early facilitation is likely to be affected by the presence or absence of temporal expectations and that the early onset of IOR might be masked by stronger facilitation in traditional cueing experiments.

A Generative Model of Cognitive State from Task and Eye Movements.

The early eye tracking studies of Yarbus provided descriptive evidence that an observer's task influences patterns of eye movements, leading to the tantalizing prospect that an observer's intentions could be inferred from their saccade behavior. We investigate the predictive value of task and eye movement properties by creating a computational cognitive model of saccade selection based on instructed task and internal cognitive state using a Dynamic Bayesian Network (DBN). Understanding how humans generate saccades under different conditions and cognitive sets links recent work on salience models of low-level vision with higher level cognitive goals. This model provides a Bayesian, cognitive approach to top-down transitions in attentional set in pre-frontal areas along with vector-based saccade generation from the superior colliculus. Our approach is to begin with eye movement data that has previously been shown to differ across task. We first present an analysis of the extent to which individual saccadic features are diagnostic of an observer's task. Second, we use those features to infer an underlying cognitive state that potentially differs from the instructed task. Finally, we demonstrate how changes of cognitive state over time can be incorporated into a generative model of eye movement vectors without resorting to an external decision homunculus. Internal cognitive state frees the model from the assumption that instructed task is the only factor influencing observers' saccadic behavior. While the inclusion of hidden temporal state does not improve the classification accuracy of the model, it does allow accurate prediction of saccadic sequence results observed in search paradigms. Given the generative nature of this model, it is capable of saccadic simulation in real time. We demonstrated that the properties from its generated saccadic vectors closely match those of human observers given a particular task and cognitive state. Many current models of vision focus entirely on bottom-up salience to produce estimates of spatial "areas of interest" within a visual scene. While a few recent models do add top-down knowledge and task information, we believe our contribution is important in three key ways. First, we incorporate task as learned attentional sets that are capable of self-transition given only information available to the visual system. This matches influential theories of bias signals by (Miller and Cohen Annu Rev Neurosci 24:167-202, 2001) and implements selection of state without simply shifting the decision to an external homunculus. Second, our model is generative and capable of predicting sequence artifacts in saccade generation like those found in visual search. Third, our model generates relative saccadic vector information as opposed to absolute spatial coordinates. This matches more closely the internal saccadic representations as they are generated in the superior colliculus.

Multiple Diffusion Models to Compare Saccadic and Manual Responses for Inhibition of Return.

Cuing a location in space produces a short-lived advantage in reaction time to targets at that location. This early advantage, however, switches to a reaction time cost and has been termed inhibition of return (IOR). IOR behaves differently for different response modalities, suggesting that it may not be a unified effect. This letter presents new data from two experiments testing the gradient of IOR with random, continuous cue-target Euclidean distance and cue-target onset asynchrony. These data were then used to train multiple diffusion models of saccadic and manual reaction time for these cuing experiments. Diffusion models can generate accurate distributions of reaction time data by modeling a response as a buildup of evidence toward a response threshold. If saccadic and attentional IOR are based on similar processes, then differences in distribution will be best explained by adjusting parameter values such as signal and noise within the same model structure. Although experimental data show differences in the timing of IOR across modality, best-fit models are shown to have similar model parameters for the gradient of IOR, suggesting similar underlying mechanisms for saccadic and manual IOR.

Just passing through? Inhibition of return in saccadic sequences.

Responses tend to be slower to previously fixated spatial locations, an effect known as "inhibition of return" (IOR). Saccades cannot be assumed to be independent, however, and saccade sequences programmed in parallel differ from independent eye movements. We measured the speed of both saccadic and manual responses to probes appearing in previously fixated locations when those locations were fixated as part of either parallel or independent saccade sequences. Saccadic IOR was observed in independent but not parallel saccade sequences, while manual IOR was present in both parallel and independent sequence types. Saccadic IOR was also short-lived, and dissipated with delays of more than ∼1500 ms between the intermediate fixation and the probe onset. The results confirm that the characteristics of IOR depend critically on the response modality used for measuring it, with saccadic and manual responses giving rise to motor and attentional forms of IOR, respectively. Saccadic IOR is relatively short-lived and is not observed at intermediate locations of parallel saccade sequences, while attentional IOR is long-lasting and consistent for all sequence types.

Attentional load interferes with target localization across saccades.

The retinal positions of objects in the world change with each eye movement, but we seem to have little trouble keeping track of spatial information from one fixation to the next. We examined the role of attention in trans-saccadic localization by asking participants to localize targets while performing an attentionally demanding secondary task. In the first experiment, attentional load decreased localization precision for a remembered target, but only when a saccade intervened between target presentation and report. We then repeated the experiment and included a salient landmark that shifted on half the trials. The shifting landmark had a larger effect on localization under high load, indicating that observers rely more on landmarks to make localization judgments under high than under low attentional load. The results suggest that attention facilitates trans-saccadic localization judgments based on spatial updating of gaze-centered coordinates when visual landmarks are not available. The availability of reliable landmarks (present in most natural circumstances) can compensate for the effects of scarce attentional resources on trans-saccadic localization.

Perceptual merging contributes to cueing effects.

An uninformative exogenous cue speeds target detection if cue and target appear in the same location separated by a brief temporal interval. This finding is usually ascribed to the orienting of spatial attention to the cued location. Here we examine the role of perceptual merging of the two trial events in speeded target detection. That is, the cue and target may be perceived as a single event when they appear in the same location. If so, cueing effects could reflect, in part, the binding of the perceived target onset to the earlier cue onset. We observed the traditional facilitation of cued over uncued targets and asked the same observers to judge target onset time by noting the time on a clock when the target appeared. Observers consistently judged the onset time of the target as being earlier than it appeared with cued targets judged as earlier than uncued targets. When the event order is reversed so that the target precedes the cue, perceived onset is accurate in both cued and uncued locations. This pattern of results suggests that perceptual merging does occur in exogenous cueing. A modified attention account is discussed that proposes reentrant processing, evident through perceptual merging, as the underlying mechanism of reflexive orienting of attention.

Driving forces in free visual search: An ethology.

Visual search typically involves sequences of eye movements under the constraints of a specific scene and specific goals. Visual search has been used as an experimental paradigm to study the interplay of scene salience and top-down goals, as well as various aspects of vision, attention, and memory, usually by introducing a secondary task or by controlling and manipulating the search environment. An ethology is a study of an animal in its natural environment, and here we examine the fixation patterns of the human animal searching a series of challenging illustrated scenes that are well-known in popular culture. The search was free of secondary tasks, probes, and other distractions. Our goal was to describe saccadic behavior, including patterns of fixation duration, saccade amplitude, and angular direction. In particular, we employed both new and established techniques for identifying top-down strategies, any influences of bottom-up image salience, and the midlevel attentional effects of saccadic momentum and inhibition of return. The visual search dynamics that we observed and quantified demonstrate that saccades are not independently generated and incorporate distinct influences from strategy, salience, and attention. Sequential dependencies consistent with inhibition of return also emerged from our analyses.