Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Saccadic trajectories deviate toward or away from optimally informative visual features.

The saccades' path is influenced by visual distractors, making their trajectory curve away or toward them. Previous research suggested that the more salient the distractor, the more pronounced is the curvature. We investigate the saliency of spatial visual features, predicted by a constrained maximum entropy model to be optimal or non-optimal information carriers in fast vision, by using them as distractors in a saccadic task. Their effect was compared to that of luminance-based control distractors. Optimal features evoke a larger saccadic curvature compared to non-optimal features, and the magnitude and direction of deviation change as a function of the delay between distractor and saccade onset. Effects were similar to those found with high-luminance versus low-luminance distractors. Therefore, model-predicted information optimal features interfere with target-oriented saccades, following a dynamic attraction-repulsion pattern. This suggests that the visuo-oculomotor system rapidly and automatically processes optimally informative features while programming visually guided eye movements.

Mammals achieve common neural coverage of visual scenes using distinct sampling behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across several fixations to construct a more complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact, and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of different species. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Sounds are remapped across saccades.

To achieve visual space constancy, our brain remaps eye-centered projections of visual objects across saccades. Here, we measured saccade trajectory curvature following the presentation of visual, auditory, and audiovisual distractors in a double-step saccade task to investigate if this stability mechanism also accounts for localized sounds. We found that saccade trajectories systematically curved away from the position at which either a light or a sound was presented, suggesting that both modalities are represented in eye-centered oculomotor centers. Importantly, the same effect was observed when the distractor preceded the execution of the first saccade. These results suggest that oculomotor centers keep track of visual, auditory and audiovisual objects by remapping their eye-centered representations across saccades. Furthermore, they argue for the existence of a supra-modal map which keeps track of multi-sensory object locations across our movements to create an impression of space constancy.

Stimulus blanking reveals contrast-dependent transsaccadic feature transfer.

Across saccadic eye movements, the visual system receives two successive static images corresponding to the pre- and the postsaccadic projections of the visual field on the retina. The existence of a mechanism integrating the content of these images is today still a matter of debate. Here, we studied the transfer of a visual feature across saccades using a blanking paradigm. Participants moved their eyes to a peripheral grating and discriminated a change in its orientation occurring during the eye movement. The grating was either constantly on the screen or briefly blanked during and after the saccade. Moreover, it either was of the same luminance as the background (i.e., isoluminant) or anisoluminant with respect to it. We found that for anisoluminant gratings, the orientation discrimination across saccades was improved when a blank followed the onset of the eye movement. Such effect was however abolished with isoluminant gratings. Additionally, performance was also improved when an anisoluminant grating presented before the saccade was followed by an isoluminant one. These results demonstrate that a detailed representation of the presaccadic image was transferred across saccades allowing participants to perform better on the transsaccadic orientation task. While such a transfer of visual orientation across saccade is masked in real-life anisoluminant conditions, the use of a blank and of an isoluminant postsaccadic grating allowed to reveal its existence.

Visual Organization of the Default Network.

The default network (DN) is a brain network with correlated activities spanning frontal, parietal, and temporal cortical lobes. The DN activates for high-level cognition tasks and deactivates when subjects are actively engaged in perceptual tasks. Despite numerous observations, the role of DN deactivation remains unclear. Using computational neuroimaging applied to a large dataset of the Human Connectome Project (HCP) and to two individual subjects scanned over many repeated runs, we demonstrate that the DN selectively deactivates as a function of the position of a visual stimulus. That is, we show that spatial vision is encoded within the DN by means of deactivation relative to baseline. Our results suggest that the DN functions as a set of high-level visual regions, opening up the possibility of using vision-science tools to understand its putative function in cognition and perception.

Sensitivity measures of visuospatial attention.

Measuring visual sensitivity has become popular to determine the spatial deployment of visual attention. Critically, the accuracy of the measurement depends on the quality of the stimulus used. We evaluated the strengths and weaknesses of six commonly used stimuli for assessing visual attention. While preparing an eye movement to a cued item, participants discriminated a stimulus-specific visual feature, either at the cued location or at other equidistant uncued locations. Stimuli differed in their visual features (digital letters, Gabors, crosses, pink noise, random dot kinematograms, and Gabor streams) and their presentation mode (static or dynamic stimuli). We evaluated these stimuli regarding their temporal and spatial specificity and their impact on saccade preparation. We assessed presaccadic visual sensitivity as a correlate of visual spatial attention and discuss the stimulus-specific time course, spatial specificity, and magnitude of the measured attention modulation. Irrespective of the stimulus type, we observed a clear increase of visual sensitivity at the cued location. Time course, spatial specificity, and magnitude of this improvement, however, were specific to each stimulus. Based on our findings, we present guidelines to select the stimulus best suited to measure visuospatial attention depending on the respective research question.

The spread of presaccadic attention depends on the spatial configuration of the visual scene.

When preparing a saccade, attentional resources are focused at the saccade target and its immediate vicinity. Here we show that this does not hold true when saccades are prepared toward a recently extinguished target. We obtained detailed maps of orientation sensitivity when participants prepared a saccade toward a target that either remained on the screen or disappeared before the eyes moved. We found that attention was mainly focused on the immediate surround of the visible target and spread to more peripheral locations as a function of the distance from the cue and the delay between the target's disappearance and the saccade. Interestingly, this spread was not accompanied with a spread of the saccade endpoint. These results suggest that presaccadic attention and saccade programming are two distinct processes that can be dissociated as a function of their interaction with the spatial configuration of the visual scene.

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity.

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we recorded the eye position of human participants engaged in a saccade task via eye tracking and assessed visual orientation discrimination performance at various locations during saccade preparation. Importantly, instead of using a single saccade target paradigm for which the saccade endpoint typically coincides roughly with the target, this protocol comprised the presentation of two nearby saccade targets, leading to a distinct spatial dissociation between target locations and saccade endpoint on a substantial number of trials. The paradigm allowed us to compare presaccadic visual discrimination performance at the endpoint of accurate saccades (landing at one of the saccade targets) and of averaging saccades (landing at an intermediate location in between the two targets). We observed a selective enhancement of visual sensitivity at the endpoint of accurate saccades but not at the endpoint of averaging saccades. Rather, before the execution of averaging saccades, visual sensitivity was equally enhanced at both targets, suggesting that saccade averaging follows from unresolved attentional selection among the saccade targets. These results argue against a mandatory coupling between visual attention and saccade programming based on a direct measure of presaccadic visual sensitivity rather than saccadic reaction times, which have been used in other protocols to draw similar conclusions. While our protocol provides a useful framework to investigate the relationship between visual attention and saccadic eye movements at the behavioral level, it can also be combined with electrophysiological measures to extend insights at the neuronal level.

Visual attention is not limited to the oculomotor range.

Both patients with eye movement disorders and healthy participants whose oculomotor range had been experimentally reduced have been reported to show attentional deficits at locations unreachable by their eyes. Whereas previous studies were mainly based on the evaluation of reaction times, we measured visual sensitivity before saccadic eye movements and during fixation at locations either within or beyond participants' oculomotor range. Participants rotated their heads to prevent them from performing large rightward saccades. In this posture, an attentional cue was presented inside or outside their oculomotor range. Participants either made a saccade to the cue or maintained fixation while they discriminated the orientation of a visual noise patch. In contrast to previous reports, we found that the cue attracted visual attention regardless of whether it was presented within or beyond participants' oculomotor range during both fixation and saccade preparation. Moreover, when participants aimed to look to a cue that they could not reach with their eyes, we observed no benefit at their actual saccade endpoint. This demonstrates that spatial attention is not coupled to the executed oculomotor program but instead can be deployed unrestrictedly also toward locations to which no saccade can be executed. Our results are compatible with the view that covert and overt attentional orienting are guided by feedback projections of visual and visuomotor neurons of the gaze control system, irrespective of oculomotor limitations.

Visual attention is not deployed at the endpoint of averaging saccades.

The premotor theory of attention postulates that spatial attention arises from the activation of saccade areas and that the deployment of attention is the consequence of motor programming. Yet attentional and oculomotor processes have been shown to be dissociable at the neuronal level in covert attention tasks. To investigate a potential dissociation at the behavioral level, we instructed human participants to move their eyes (saccade) towards 1 of 2 nearby, competing saccade targets. The spatial distribution of visual attention was determined using oriented visual stimuli presented either at the target locations, between them, or at several other equidistant locations. Results demonstrate that accurate saccades towards one of the targets were associated with presaccadic enhancement of visual sensitivity at the respective saccade endpoint compared to the nonsaccaded target location. In contrast, averaging saccades, landing between the 2 targets, were not associated with attentional facilitation at the saccade endpoint. Rather, attention before averaging saccades was equally deployed at the 2 target locations. Taken together, our results reveal that visual attention is not obligatorily coupled to the endpoint of a subsequent saccade. Rather, our results suggest that the oculomotor program depends on the state of attentional selection before saccade onset and that saccade averaging arises from unresolved attentional selection.

Pre-saccadic remapping relies on dynamics of spatial attention.

Each saccade shifts the projections of the visual scene on the retina. It has been proposed that the receptive fields of neurons in oculomotor areas are predictively remapped to account for these shifts. While remapping of the whole visual scene seems prohibitively complex, selection by attention may limit these processes to a subset of attended locations. Because attentional selection consumes time, remapping of attended locations should evolve in time, too. In our study, we cued a spatial location by presenting an attention-capturing cue at different times before a saccade and constructed maps of attentional allocation across the visual field. We observed no remapping of attention when the cue appeared shortly before saccade. In contrast, when the cue appeared sufficiently early before saccade, attentional resources were reallocated precisely to the remapped location. Our results show that pre-saccadic remapping takes time to develop suggesting that it relies on the spatial and temporal dynamics of spatial attention.

Variations in crowding, saccadic precision, and spatial localization reveal the shared topology of spatial vision.

Visual sensitivity varies across the visual field in several characteristic ways. For example, sensitivity declines sharply in peripheral (vs. foveal) vision and is typically worse in the upper (vs. lower) visual field. These variations can affect processes ranging from acuity and crowding (the deleterious effect of clutter on object recognition) to the precision of saccadic eye movements. Here we examine whether these variations can be attributed to a common source within the visual system. We first compared the size of crowding zones with the precision of saccades using an oriented clock target and two adjacent flanker elements. We report that both saccade precision and crowded-target reports vary idiosyncratically across the visual field with a strong correlation across tasks for all participants. Nevertheless, both group-level and trial-by-trial analyses reveal dissociations that exclude a common representation for the two processes. We therefore compared crowding with two measures of spatial localization: Landolt-C gap resolution and three-dot bisection. Here we observe similar idiosyncratic variations with strong interparticipant correlations across tasks despite considerably finer precision. Hierarchical regression analyses further show that variations in spatial precision account for much of the variation in crowding, including the correlation between crowding and saccades. Altogether, we demonstrate that crowding, spatial localization, and saccadic precision show clear dissociations, indicative of independent spatial representations, whilst nonetheless sharing idiosyncratic variations in spatial topology. We propose that these topological idiosyncrasies are established early in the visual system and inherited throughout later stages to affect a range of higher-level representations.

Presaccadic motion integration between current and future retinotopic locations of attended objects.

Object tracking across eye movements is thought to rely on presaccadic updating of attention between the object's current and its "remapped" location (i.e., the postsaccadic retinotopic location). We report evidence for a bifocal, presaccadic sampling between these two positions. While preparing a saccade, participants viewed four spatially separated random dot kinematograms, one of which was cued by a colored flash. They reported the direction of a coherent motion signal at the cued location while a second signal occurred simultaneously either at the cue's remapped location or at one of several control locations. Motion integration between the signals occurred only when the two motion signals were congruent and were shown at the cue and at its remapped location. This shows that the visual system integrates features between both the current and the future retinotopic locations of an attended object and that such presaccadic sampling is feature specific.

Remapping Attention Pointers: Linking Physiology and Behavior.

Our eyes rapidly scan visual scenes, displacing the projection on the retina with every move. Yet these frequent retinal image shifts do not appear to hamper vision. Two recent physiological studies shed new light on the role of attention in visual processing across saccadic eye movements.

Oculomotor selection underlies feature retention in visual working memory.

Oculomotor selection, spatial task relevance, and visual working memory (WM) are described as three processes highly intertwined and sustained by similar cortical structures. However, because task-relevant locations always constitute potential saccade targets, no study so far has been able to distinguish between oculomotor selection and spatial task relevance. We designed an experiment that allowed us to dissociate in humans the contribution of task relevance, oculomotor selection, and oculomotor execution to the retention of feature representations in WM. We report that task relevance and oculomotor selection lead to dissociable effects on feature WM maintenance. In a first task, in which an object's location was encoded as a saccade target, its feature representations were successfully maintained in WM, whereas they declined at nonsaccade target locations. Likewise, we observed a similar WM benefit at the target of saccades that were prepared but never executed. In a second task, when an object's location was marked as task relevant but constituted a nonsaccade target (a location to avoid), feature representations maintained at that location did not benefit. Combined, our results demonstrate that oculomotor selection is consistently associated with WM, whereas task relevance is not. This provides evidence for an overlapping circuitry serving saccade target selection and feature-based WM that can be dissociated from processes encoding task-relevant locations.

Attentional trade-offs maintain the tracking of moving objects across saccades.

In many situations like playing sports or driving a car, we keep track of moving objects, despite the frequent eye movements that drastically interrupt their retinal motion trajectory. Here we report evidence that transsaccadic tracking relies on trade-offs of attentional resources from a tracked object's motion path to its remapped location. While participants covertly tracked a moving object, we presented pulses of coherent motion at different locations to probe the allocation of spatial attention along the object's entire motion path. Changes in the sensitivity for these pulses showed that during fixation attention shifted smoothly in anticipation of the tracked object's displacement. However, just before a saccade, attentional resources were withdrawn from the object's current motion path and reflexively drawn to the retinal location the object would have after saccade. This finding demonstrates the predictive choice the visual system makes to maintain the tracking of moving objects across saccades.

Allocation of attention across saccades.

Whenever the eyes move, spatial attention must keep track of the locations of targets as they shift on the retina. This study investigated transsaccadic updating of visual attention to cued targets. While observers prepared a saccade, we flashed an irrelevant, but salient, color cue in their visual periphery and measured the allocation of spatial attention before and after the saccade using a tilt discrimination task. We found that just before the saccade, attention was allocated to the cue's future retinal location, its predictively "remapped" location. Attention was sustained at the cue's location in the world across the saccade, despite the change of retinal position whereas it decayed quickly at the retinal location of the cue, after the eye landed. By extinguishing the color cue across the saccade, we further demonstrate that the visual system relies only on predictive allocation of spatial attention, as the presence of the cue after the saccade did not substantially affect attentional allocation. These behavioral results support and extend physiological evidence showing predictive activation of visual neurons when an attended stimulus will fall in their receptive field after a saccade. Our results show that tracking of spatial locations across saccades is a plausible consequence of physiological remapping.

Apparent motion from outside the visual field, retinotopic cortices may register extra-retinal positions.

Observers made a saccade between two fixation markers while a probe was flashed sequentially at two locations on a side screen. The first probe was presented in the far periphery just within the observer's visual field. This target was extinguished and the observers made a large saccade away from the probe, which would have left it far outside the visual field if it had still been present. The second probe was then presented, displaced from the first in the same direction as the eye movement and by about the same distance as the saccade step. Because both eyes and probes shifted by similar amounts, there was little or no shift between the first and second probe positions on the retina. Nevertheless, subjects reported seeing motion corresponding to the spatial displacement not the retinal displacement. When the second probe was presented, the effective location of the first probe lay outside the visual field demonstrating that apparent motion can be seen from a location outside the visual field to a second location inside the visual field. Recent physiological results suggest that target locations are "remapped" on retinotopic representations to correct for the effects of eye movements. Our results suggest that the representations on which this remapping occurs include locations that fall beyond the limits of the retina.

Temporal dynamics of remapping captured by peri-saccadic continuous motion.

Different attention and saccade control areas contribute to space constancy by remapping target activity onto their expected post-saccadic locations. To visualize this dynamic remapping, we used a technique developed by Honda (2006) where a probe moved vertically while participants made a saccade across the motion path. Observers do not report any large excursions of the trace at the time of the saccade that would correspond to the classical peri-saccadic mislocalization effect. Instead, they reported that the motion trace appeared to be broken into two separate segments with a shift of approximately one-fifth of the saccade amplitude representing an overcompensation of the expected retinal displacement caused by the saccade. To measure the timing of this break in the trace, we introduced a second, physical shift that was the same size but opposite in direction to the saccade-induced shift. The trace appeared continuous most frequently when the physical shift was introduced at the midpoint of the saccade, suggesting that the compensation is in place when the saccade lands. Moreover, this simple linear shift made the combined traces appear continuous and linear, with no curvature. In contrast, Honda (2006) had reported that the pre- and post-saccadic portion of the trace appeared aligned and that there was often a small, visible excursion of the trace at the time of the saccade. To compare our results more directly, we increased the contrast of our moving probe in a third experiment. Now some observers reported seeing a deviation in the motion path but the misalignment remained present. We conclude that the large deviations at the time of saccade are generally masked for a continuously moving target but that there is nevertheless a residual misalignment between pre- and post-saccadic coordinates of approximately 20% of the saccade amplitude that normally goes unnoticed.