Expectations about motion direction affect perception and anticipatory smooth pursuit differently.

Smooth pursuit eye movements and visual motion perception rely on the integration of current sensory signals with past experience. Experience shapes our expectation of current visual events and can drive eye movement responses made in anticipation of a target, such as anticipatory pursuit. Previous research revealed consistent effects of expectation on anticipatory pursuit-eye movements follow the expected target direction or speed-and contrasting effects on motion perception, but most studies considered either eye movement or perceptual responses. The current study directly compared effects of direction expectation on perception and anticipatory pursuit within the same direction discrimination task to investigate whether both types of responses are affected similarly or differently. Observers (n = 10) viewed high-coherence random-dot kinematograms (RDKs) moving rightward and leftward with a probability of 50%, 70%, or 90% in a given block of trials to build up an expectation of motion direction. They were asked to judge motion direction of interleaved low-coherence RDKs (0%-15%). Perceptual judgements were compared with changes in anticipatory pursuit eye movements as a function of probability. Results show that anticipatory pursuit velocity scaled with probability and followed direction expectation (attraction bias), whereas perceptual judgments were biased opposite to direction expectation (repulsion bias). Control experiments suggest that the repulsion bias in perception was not caused by retinal slip induced by anticipatory pursuit, or by motion adaptation. We conclude that direction expectation can be processed differently for perception and anticipatory pursuit.NEW & NOTEWORTHY We show that expectations about motion direction that are based on long-term trial history affect perception and anticipatory pursuit differently. Whereas anticipatory pursuit direction was coherent with the expected motion direction (attraction bias), perception was biased opposite to the expected direction (repulsion bias). These opposite biases potentially reveal different ways in which perception and action utilize prior information and support the idea of different information processing for perception and pursuit.

Humans adapt their anticipatory eye movements to the volatility of visual motion properties.

Animal behavior constantly adapts to changes, for example when the statistical properties of the environment change unexpectedly. For an agent that interacts with this volatile setting, it is important to react accurately and as quickly as possible. It has already been shown that when a random sequence of motion ramps of a visual target is biased to one direction (e.g. right or left), human observers adapt their eye movements to accurately anticipate the target's expected direction. Here, we prove that this ability extends to a volatile environment where the probability bias could change at random switching times. In addition, we also recorded the explicit prediction of the next outcome as reported by observers using a rating scale. Both results were compared to the estimates of a probabilistic agent that is optimal in relation to the assumed generative model. Compared to the classical leaky integrator model, we found a better match between our probabilistic agent and the behavioral responses, both for the anticipatory eye movements and the explicit task. Furthermore, by controlling the level of preference between exploitation and exploration in the model, we were able to fit for each individual's experimental dataset the most likely level of volatility and analyze inter-individual variability across participants. These results prove that in such an unstable environment, human observers can still represent an internal belief about the environmental contingencies, and use this representation both for sensory-motor control and for explicit judgments. This work offers an innovative approach to more generically test the diversity of human cognitive abilities in uncertain and dynamic environments.

Reinforcement effects in anticipatory smooth eye movements.

When predictive information about target motion is available, anticipatory smooth pursuit eye movements (aSPEM) are consistently generated before target appearance, thereby reducing the typical sensorimotor delay between target motion onset and foveation. By manipulating the probability for target motion direction, we were able to bias the direction and mean velocity of aSPEM. This suggests that motion-direction expectancy has a strong effect on the initiation of anticipatory movements. To further understand the nature of anticipatory smooth eye movements, we investigated different effects of reinforcement on aSPEM. In a first experiment, the reinforcement was contingent to a particular anticipatory behavior. A monetary reward was associated to a criterion-matching anticipatory velocity as estimated online during the gap before target motion onset. Our results showed a small but significant effect of behavior-contingent monetary reward on aSPEM. In a second experiment, the proportion of rewarded trials was manipulated across motion directions (right vs. left) independently from participants' behavior. Our results indicate that a bias in expected reward does not systematically affect anticipatory eye movements. Overall, these findings strengthen the notion that anticipatory eye movements can be considered as an operant behavior (similar to visually guided ones), whereas the expectancy for a noncontingent reward cannot efficiently bias them.

Using Covert Response Activation to Test Latent Assumptions of Formal Decision-Making Models in Humans.

Most decisions that we make build upon multiple streams of sensory evidence and control mechanisms are needed to filter out irrelevant information. Sequential sampling models of perceptual decision making have recently been enriched by attentional mechanisms that weight sensory evidence in a dynamic and goal-directed way. However, the framework retains the longstanding hypothesis that motor activity is engaged only once a decision threshold is reached. To probe latent assumptions of these models, neurophysiological indices are needed. Therefore, we collected behavioral and EMG data in the flanker task, a standard paradigm to investigate decisions about relevance. Although the models captured response time distributions and accuracy data, EMG analyses of response agonist muscles challenged the assumption of independence between decision and motor processes. Those analyses revealed covert incorrect EMG activity ("partial error") in a fraction of trials in which the correct response was finally given, providing intermediate states of evidence accumulation and response activation at the single-trial level. We extended the models by allowing motor activity to occur before a commitment to a choice and demonstrated that the proposed framework captured the rate, latency, and EMG surface of partial errors, along with the speed of the correction process. In return, EMG data provided strong constraints to discriminate between competing models that made similar behavioral predictions. Our study opens new theoretical and methodological avenues for understanding the links among decision making, cognitive control, and motor execution in humans. SIGNIFICANCE STATEMENT: Sequential sampling models of perceptual decision making assume that sensory information is accumulated until a criterion quantity of evidence is obtained, from where the decision terminates in a choice and motor activity is engaged. The very existence of covert incorrect EMG activity ("partial error") during the evidence accumulation process challenges this longstanding assumption. In the present work, we use partial errors to better constrain sequential sampling models at the single-trial level.

Linking Theoretical Decision-making Mechanisms in the Simon Task with Electrophysiological Data: A Model-based Neuroscience Study in Humans.

A current challenge for decision-making research is in extending models of simple decisions to more complex and ecological choice situations. Conflict tasks (e.g., Simon, Stroop, Eriksen flanker) have been the focus of much interest, because they provide a decision-making context representative of everyday life experiences. Modeling efforts have led to an elaborated drift diffusion model for conflict tasks (DMC), which implements a superimposition of automatic and controlled decision activations. The DMC has proven to capture the diversity of behavioral conflict effects across various task contexts. This study combined DMC predictions with EEG and EMG measurements to test a set of linking propositions that specify the relationship between theoretical decision-making mechanisms involved in the Simon task and brain activity. Our results are consistent with a representation of the superimposed decision variable in the primary motor cortices. The decision variable was also observed in the EMG activity of response agonist muscles. These findings provide new insight into the neurophysiology of human decision-making. In return, they provide support for the DMC model framework.

Looking for symmetry: fixational eye movements are biased by image mirror symmetry.

Humans are highly sensitive to symmetry. During scene exploration, the area of the retina with dense light receptor coverage acquires most information from relevant locations determined by gaze fixation. We characterized patterns of fixational eye movements made by observers staring at synthetic scenes either freely (i.e., free exploration) or during a symmetry orientation discrimination task (i.e., active exploration). Stimuli could be mirror-symmetric or not. Both free and active exploration generated more saccades parallel to the axis of symmetry than along other orientations. Most saccades were small (<2°), leaving the fovea within a 4° radius of fixation. Analysis of saccade dynamics showed that the observed parallel orientation selectivity emerged within 500 ms of stimulus onset and persisted throughout the trials under both viewing conditions. Symmetry strongly distorted existing anisotropies in gaze direction in a seemingly automatic process. We argue that this bias serves a functional role in which adjusted scene sampling enhances and maintains sustained sensitivity to local spatial correlations arising from symmetry.

Eye tracking a self-moved target with complex hand-target dynamics.

Previous work has shown that the ability to track with the eye a moving target is substantially improved when the target is self-moved by the subject's hand compared with when being externally moved. Here, we explored a situation in which the mapping between hand movement and target motion was perturbed by simulating an elastic relationship between the hand and target. Our objective was to determine whether the predictive mechanisms driving eye-hand coordination could be updated to accommodate this complex hand-target dynamics. To fully appreciate the behavioral effects of this perturbation, we compared eye tracking performance when self-moving a target with a rigid mapping (simple) and a spring mapping as well as when the subject tracked target trajectories that he/she had previously generated when using the rigid or spring mapping. Concerning the rigid mapping, our results confirmed that smooth pursuit was more accurate when the target was self-moved than externally moved. In contrast, with the spring mapping, eye tracking had initially similar low spatial accuracy (though shorter temporal lag) in the self versus externally moved conditions. However, within ∼5 min of practice, smooth pursuit improved in the self-moved spring condition, up to a level similar to the self-moved rigid condition. Subsequently, when the mapping unexpectedly switched from spring to rigid, the eye initially followed the expected target trajectory and not the real one, thereby suggesting that subjects used an internal representation of the new hand-target dynamics. Overall, these results emphasize the stunning adaptability of smooth pursuit when self-maneuvering objects with complex dynamics.

Conflict tasks and the diffusion framework: Insight in model constraints based on psychological laws.

Formal models of decision-making have traditionally focused on simple, two-choice perceptual decisions. To date, one of the most influential account of this process is Ratcliff's drift diffusion model (DDM). However, the extension of the model to more complex decisions is not straightforward. In particular, conflicting situations, such as the Eriksen, Stroop, or Simon tasks, require control mechanisms that shield the cognitive system against distracting information. We adopted a novel strategy to constrain response time (RT) models by concurrently investigating two well-known empirical laws in conflict tasks, both at experimental and modeling levels. The two laws, predicted by the DDM, describe the relationship between mean RT and (i) target intensity (Piéron's law), (ii) standard deviation of RT (Wagenmakers-Brown's law). Pioneering work has shown that Piéron's law holds in the Stroop task, and has highlighted an additive relationship between target intensity and compatibility. We found similar results in both Eriksen and Simon tasks. Compatibility also violated Wagenmakers-Brown's law in a very similar and particular fashion in the two tasks, suggesting a common model framework. To investigate the nature of this commonality, predictions of two recent extensions of the DDM that incorporate selective attention mechanisms were simulated and compared to the experimental results. Both models predict Piéron's law and the violation of Wagenmakers-Brown's law by compatibility. Fits of the models to the RT distributions and accuracy data allowed us to further reveal their relative strengths and deficiencies. Combining experimental and computational results, this study sets the groundwork for a unified model of decision-making in conflicting environments.

Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Dynamic interaction between retinal and extraretinal signals in motion integration for smooth pursuit.

Due to the aperture problem, the initial direction of tracking responses to a translating bar is biased towards the direction orthogonal to the bar. This observation offers a powerful way to explore the interactions between retinal and extraretinal signals in controlling our actions. We conducted two experiments to probe these interactions by briefly (200 and 400 ms) blanking the moving target (45° or 135° tilted bar) during steady state (Experiment 1) and at different moments during the early phase of pursuit (Experiment 2). In Experiment 1, we found a marginal but statistically significant directional bias on target reappearance for all subjects in at least one blank condition (200 or 400 ms). In Experiment 2, no systematic significant directional bias was observed at target reappearance after a blank. These results suggest that the weighting of retinal and extraretinal signals is dynamically modulated during the different phases of pursuit. Based on our previous theoretical work on motion integration, we propose a new closed-loop two-stage recurrent Bayesian model where retinal and extraretinal signals are dynamically weighted based on their respective reliabilities and combined to compute the visuomotor drive. With a single free parameter, the model reproduces many aspects of smooth pursuit observed across subjects during and immediately after target blanking. It provides a new theoretical framework to understand how different signals are dynamically combined based on their relative reliability to adaptively control our actions. Overall, the model and behavioral results suggest that human subjects rely more strongly on prediction during the early phase than in the steady state phase of pursuit.

Saccadic trajectories deviate toward or away from optimally informative visual features.

The saccades' path is influenced by visual distractors, making their trajectory curve away or toward them. Previous research suggested that the more salient the distractor, the more pronounced is the curvature. We investigate the saliency of spatial visual features, predicted by a constrained maximum entropy model to be optimal or non-optimal information carriers in fast vision, by using them as distractors in a saccadic task. Their effect was compared to that of luminance-based control distractors. Optimal features evoke a larger saccadic curvature compared to non-optimal features, and the magnitude and direction of deviation change as a function of the delay between distractor and saccade onset. Effects were similar to those found with high-luminance versus low-luminance distractors. Therefore, model-predicted information optimal features interfere with target-oriented saccades, following a dynamic attraction-repulsion pattern. This suggests that the visuo-oculomotor system rapidly and automatically processes optimally informative features while programming visually guided eye movements.

Mammals achieve common neural coverage of visual scenes using distinct sampling behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across several fixations to construct a more complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact, and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of different species. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Information-optimal local features automatically attract covert and overt attention.

In fast vision, local spatial properties of the visual scene can automatically capture the observer's attention. We used specific local features, predicted by a constrained maximum-entropy model to be optimal information-carriers, as candidate "salient features''. Previous studies showed that participants choose these optimal features as "more salient" if explicitly asked. Here, we investigated the implicit saliency of these optimal features in two attentional tasks. In a covert-attention experiment, we measured the luminance-contrast threshold for discriminating the orientation of a peripheral gabor. In a gaze-orienting experiment, we analyzed latency and direction of saccades towards a peripheral target. In both tasks, two brief peripheral cues, differing in saliency according to the model, preceded the target, presented on the same (valid trials) or the opposite side (invalid trials) of the optimal cue. Results showed reduced contrast thresholds, saccadic latencies, and direction errors in valid trials, and the opposite in invalid trials, compared to baseline values obtained with equally salient cues. Also, optimal features triggered more anticipatory saccades. Similar effects emerged in a luminance-control condition. Overall, in fast vision, optimal features automatically attract covert and overt attention, suggesting that saliency is determined by information maximization criteria coupled with computational limitations.

Early Visual Saliency Based on Isolated Optimal Features.

Under fast viewing conditions, the visual system extracts salient and simplified representations of complex visual scenes. Saccadic eye movements optimize such visual analysis through the dynamic sampling of the most informative and salient regions in the scene. However, a general definition of saliency, as well as its role for natural active vision, is still a matter for discussion. Following the general idea that visual saliency may be based on the amount of local information, a recent constrained maximum-entropy model of early vision, applied to natural images, extracts a set of local optimal information-carriers, as candidate salient features. These optimal features proved to be more informative than others in fast vision, when embedded in simplified sketches of natural images. In the present study, for the first time, these features were presented in isolation, to investigate whether they can be visually more salient than other non-optimal features, even in the absence of any meaningful global arrangement (contour, line, etc.). In four psychophysics experiments, fast discriminability of a compound of optimal features (target) in comparison with a similar compound of non-optimal features (distractor) was measured as a function of their number and contrast. Results showed that the saliency predictions from the constrained maximum-entropy model are well verified in the data, even when the optimal features are presented in smaller numbers or at lower contrast. In the eye movements experiment, the target and the distractor compounds were presented in the periphery at different angles. Participants were asked to perform a simple choice-saccade task. Results showed that saccades can select informative optimal features spatially interleaved with non-optimal features even at the shortest latencies. Saccades' choice accuracy and landing position precision improved with SNR. In conclusion, the optimal features predicted by the reference model, turn out to be more salient than others, despite the lack of any clues coming from a global meaningful structure, suggesting that they get preferential treatment during fast image analysis. Also, peripheral fast visual processing of these informative local features is able to guide gaze orientation. We speculate that active vision is efficiently adapted to maximize information in natural visual scenes.

Competition between color and luminance for target selection in smooth pursuit and saccadic eye movements.

Visual processing of color and luminance for smooth pursuit and saccadic eye movements was investigated using a target selection paradigm. In two experiments, stimuli were varied along the dimensions color and luminance, and selection of the more salient target was compared in pursuit and saccades. Initial pursuit was biased in the direction of the luminance component whereas saccades showed a relative preference for color. An early pursuit response toward luminance was often reversed to color by a later saccade. Observers' perceptual judgments of stimulus salience, obtained in two control experiments, were clearly biased toward luminance. This choice bias in perceptual data implies that the initial short-latency pursuit response agrees with perceptual judgments. In contrast, saccades, which have a longer latency than pursuit, do not seem to follow the perceptual judgment of salience but instead show a stronger relative preference for color. These substantial differences in target selection imply that target selection processes for pursuit and saccadic eye movements use distinctly different weights for color and luminance stimuli.

Bayesian modeling of dynamic motion integration.

The quality of the representation of an object's motion is limited by the noise in the sensory input as well as by an intrinsic ambiguity due to the spatial limitation of the visual motion analyzers (aperture problem). Perceptual and oculomotor data demonstrate that motion processing of extended objects is initially dominated by the local 1D motion cues, related to the object's edges and orthogonal to them, whereas 2D information, related to terminators (or edge-endings), takes progressively over and leads to the final correct representation of global motion. A Bayesian framework accounting for the sensory noise and general expectancies for object velocities has proven successful in explaining several experimental findings concerning early motion processing [Weiss, Y., Adelson, E., 1998. Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision. MIT Technical report, A.I. Memo 1624]. In particular, these models provide a qualitative account for the initial bias induced by the 1D motion cue. However, a complete functional model, encompassing the dynamical evolution of object motion perception, including the integration of different motion cues, is still lacking. Here we outline several experimental observations concerning human smooth pursuit of moving objects and more particularly the time course of its initiation phase, which reflects the ongoing motion integration process. In addition, we propose a recursive extension of the Bayesian model, motivated and constrained by our oculomotor data, to describe the dynamical integration of 1D and 2D motion information. We compare the model predictions for object motion tracking with human oculomotor recordings.

Spatiotemporal dynamics of visual attention during saccade preparation: Independence and coupling between attention and movement planning.

During the preparation of a saccadic eye movement, a visual stimulus is more efficiently processed when it is spatially coincident with the saccadic target as compared to when the visual and the saccadic targets are displayed at different locations. We studied the coupling between visual selective attention and saccadic preparation by measuring orientation acuity of human subjects at different locations relative to the saccadic target and at different delays relative to the saccade cue onset. First, we generalized previous results (E. Castet, S. Jeanjean, A. Montagnini, D. Laugier, & G. S. Masson, 2006) revealing that a dramatic perceptual advantage at the saccadic target emerges dynamically within the first 150-200 ms from saccade cue onset. Second, by varying the validity of the spatial cue for the discrimination task, we encouraged subjects to modulate the spatial distribution of attentional resources independently from the automatic deployment to saccadic target. We found that an independent component of attention can be voluntarily deployed away from the saccadic target. The relative weight of the automatic versus the independent component of attention increases across time during saccadic preparation.

Dynamics of attentional deployment during saccadic programming.

The dynamics of attentional deployment before saccade execution was studied with a dual-task paradigm. Observers made a horizontal saccade whose direction was indicated by a symbolic precue and had to discriminate the orientation of a Gabor patch displayed at different delays after the precue (but before saccade onset). The patch location relative to the saccadic target was indicated to observers before each block. Therefore, on each trial, observers were informed simultaneously about the respective absolute locations of the saccadic and perceptual targets. The main result is that orientational acuity improved over a period of 150-200 ms after the precue onset at the saccadic target location, where overall performance is best, and at distant locations. This effect is due to attentional factors rather than to an alerting effect. It is also dependent on the efficiency of the temporal masks displayed before and after the Gabor patches.

Fixational saccades during grating detection and discrimination.

We investigated the patterns of fixational saccades in human observers performing two classical perceptual tasks: grating detection and discrimination. First, participants were asked to detect a vertical or tilted grating with one of three spatial frequencies and one of four luminance contrast levels. In the second experiment, participants had to discriminate the spatial frequency of two supra-threshold gratings. The gratings were always embedded in additive, high- or low-contrast pink noise. We observed that the patterns of fixational saccades were highly idiosyncratic among participants. Moreover, during the grating detection task, the amplitude and the number of saccades were inversely correlated with stimulus visibility. We did not find a systematic relationship between saccade parameters and grating frequency, apart from a slight decrease of saccade amplitude during grating discrimination with higher spatial frequencies. No consistent changes in the number and amplitude of fixational saccades with performance accuracy were reported. Surprisingly, during grating detection, saccade number and amplitude were similar in grating-with-noise and noise-only displays. Grating orientation did not affect substantially saccade direction in either task. The results challenge the idea that, when analyzing low-level spatial properties of visual stimuli, fixational saccades can be adapted in order to extract task-relevant information optimally. Rather, saccadic patterns seem to be overall modulated by task context, stimulus visibility and individual variability.

The urgency to look: prompt saccades to the benefit of perception.

Researchers have shown that the promptness to initiate a saccade is modulated by countless factors pertaining to the visual context and the task. However, experiments on saccadic eye movements are usually designed in such a way that oculomotor performance is dissociated from the natural role of saccades, namely that of making crucial perceptual information rapidly available for high-resolution, foveal analysis. Here, we demonstrate that the requirement to perform a difficult perceptual judgment at the saccade landing location can reduce saccadic latency (by >15%) and increase saccadic peak velocity. Importantly, the effect cannot be explained in terms of arousal, as latency changes are specific to the location where the perceptual judgement is required. These results indicate that mechanisms for voluntary saccade initiation are under the powerful indirect control of perceptual goals.