Attention to visual motion suppresses neuronal and behavioral sensitivity in nearby feature space.

BACKGROUND: Feature-based attention prioritizes the processing of the attended feature while strongly suppressing the processing of nearby ones. This creates a non-linearity or "attentional suppressive surround" predicted by the Selective Tuning model of visual attention. However, previously reported effects of feature-based attention on neuronal responses are linear, e.g., feature-similarity gain. Here, we investigated this apparent contradiction by neurophysiological and psychophysical approaches. RESULTS: Responses of motion direction-selective neurons in area MT/MST of monkeys were recorded during a motion task. When attention was allocated to a stimulus moving in the neurons' preferred direction, response tuning curves showed its minimum for directions 60-90° away from the preferred direction, an attentional suppressive surround. This effect was modeled via the interaction of two Gaussian fields representing excitatory narrowly tuned and inhibitory widely tuned inputs into a neuron, with feature-based attention predominantly increasing the gain of inhibitory inputs. We further showed using a motion repulsion paradigm in humans that feature-based attention produces a similar non-linearity on motion discrimination performance. CONCLUSIONS: Our results link the gain modulation of neuronal inputs and tuning curves examined through the feature-similarity gain lens to the attentional impact on neural population responses predicted by the Selective Tuning model, providing a unified framework for the documented effects of feature-based attention on neuronal responses and behavior.

Oculomotor system can differentially process red and green colors during saccade programming in the presence of a competing distractor.

Selective attention filters irrelevant information entering our brain to allow for fine-tuning of the relevant information processing. In the visual domain, shifts of attention are most often followed by a saccadic eye movement to objects and places of high relevance. Recent studies have shown that the stimulus color can affect saccade target selection and saccade trajectories. While those saccade modulations are based on perceptual color space, the level in the visual processing hierarchy at which color selection biases saccade programming remains unclear. As color has also been shown to influence manual response inhibition which is a key function of the prefrontal cortex, we hypothesized that the effects of color on executive functions would also inherently affect saccade programming. To test this hypothesis, we measured behavioral performance and saccade metrics during a modified saccadic Stroop task which reflects competition between color words ("RED" and "GREEN") and their color at the level of the prefrontal cortex. Our results revealed that the oculomotor system can differentially process red and green colors when planning a saccade in the presence of a competing distractor.

Oculomotor target selection is mediated by complex objects.

Oculomotor target selection often requires discriminating visual features, but it remains unclear how oculomotor substrates encoding saccade vectors functionally contribute to this process. One possibility is that oculomotor vector representations (observed directly as physiological activation or inferred from behavioral interference) of potential targets are continuously reweighted by task relevance computed elsewhere in specialized visual modules, whereas an alternative possibility is that oculomotor modules use local featural analyses to actively discriminate potential targets. Strengthening the former account, oculomotor vector representations have longer onset latencies for ventral- (i.e., color) than dorsal-stream features (i.e., luminance), suggesting that oculomotor vector representations originate from featurally relevant specialized visual modules. Here, we extended this reasoning by behaviorally examining whether the onset latency of saccadic interference elicited by visually complex stimuli is greater than is commonly observed for simple stimuli. We measured human saccade metrics (saccade curvature, endpoint deviations, saccade frequency, and error proportion) as a function of time after abrupt distractor onset. Distractors were novel, visually complex, and had to be discriminated from targets to guide saccades. The earliest saccadic interference latency was ∼110 ms, considerably longer than previous experiments, suggesting that sensory representations projected into the oculomotor system are gated to allow for sufficient featural processing to satisfy task demands. Surprisingly, initial oculomotor vector representations encoded features, as we manipulated the visual similarity between targets and distractors and observed increased vector modulation response magnitude and duration when the distractor was highly similar to the target. Oculomotor vector modulation was gradually extinguished over the time course of the experiment.NEW & NOTEWORTHY We challenge the role of the oculomotor system in discriminating features during saccadic target selection. Our data suggest that the onset latency of oculomotor vector representations is scaled by task difficulty and featural complexity, suggesting that featural computations are performed outside of the oculomotor system, which receives the output of these computations only after sufficient visual and cognitive processing. We also challenge the convention that initial oculomotor vector representations are feature invariant, as they encoded task relevance.

Gaze patterns during presentation of fixed and random phase radial frequency patterns.

Radial frequency (RF) patterns, circles which have had their radius modulated as a function of their polar angle, have been used in the examination of the integration of contour information around closed contour patterns. Typically, these patterns have been presented in a random orientation from trial-to-trial in order to maintain spatial uncertainty as to the location of the deformation on the pattern, as it may affect observer strategy and performance. However, the effect of fixed and random orientation (phase) on observer gaze strategies used to discriminate RF patterns has not been directly tested. This study compared fixation patterns across four conditions: fixed phase single cycle; random phase single cycle; fixed phase three cycle; and random phase three cycle RF3 patterns. The results showed that observers fixated on the known location of deformation for the fixed phase single cycle condition but used a more central fixation for the other three conditions. This strategy had a significant effect on observer thresholds for the fixed phase single cycle condition, with greater adherence to the strategy resulting in lower thresholds. It was also found that for the single cycle patterns observers tended to fixate on different locations on the pattern: on the maximum orientation difference from circular for the fixed phase pattern; and on the point of maximum curvature for the random phase pattern. These differences in gaze patterns are likely driven by the underlying local or global processing of the fixed or random phase single cycle patterns, respectively.

Long-term memory and hippocampal function support predictive gaze control during goal-directed search.

Eye movements during visual search change with prior experience for search stimuli. Previous studies measured these gaze effects shortly after initial viewing, typically during free viewing; it remains open whether the effects are preserved across long delays and for goal-directed search, and which memory system guides gaze. In Experiment 1, we analyzed eye movements of healthy adults viewing novel and repeated scenes while searching for a scene-embedded target. The task was performed across different time points to examine the repetition effects in long-term memory, and memory types were grouped based on explicit recall of targets. In Experiment 2, an amnesic person with bilateral extended hippocampal damage and the age-matched control group performed the same task with shorter intervals to determine whether or not the repetition effects depend on hippocampal function. When healthy adults explicitly remembered repeated target-scene pairs, search time and fixation duration decreased, and gaze was directed closer to the target region, than when they forgot targets. These effects were seen even after a one-month delay from their initial viewing, suggesting the effects are associated with long-term, explicit memory. Saccadic amplitude was not strongly modulated by scene repetition or explicit recall of targets. The amnesic person did not show explicit recall or implicit repetition effects, whereas his control group showed similar patterns to those seen in Experiment 1. The results reveal several aspects of gaze control that are influenced by long-term memory. The dependence of gaze effects on medial temporal lobe integrity support a role for this region in predictive gaze control.

Higher order, multifeatural object encoding by the oculomotor system.

Previous behavioral and physiological research has demonstrated that as the behavioral relevance of potential saccade goals increases, they elicit more competition during target selection processing as evidenced by increased saccade curvature and neural activity. However, these effects have only been demonstrated for lower order feature singletons, and it remains unclear whether more complicated featural differences between higher order objects also elicit vector modulation. Therefore, we measured human saccades curvature elicited by distractors bilaterally flanking a target during a visual search saccade task and systematically varied subsets of features shared between the two distractors and the target, referred to as objective similarity (OS). Our results demonstrate that saccades deviated away from the distractor highest in OS to the target and that there was a linear relationship between the magnitude of saccade deviation and the number of feature differences between the most similar distractor and the target. Furthermore, an analysis of curvature over the time course of the saccade demonstrated that curvature only occurred in the first 20-30 ms of the movement. Given the multifeatural complexity of the novel stimuli, these results suggest that saccadic target selection processing involves dynamically reweighting vector representations for movement planning to several possible targets based on their behavioral relevance. NEW & NOTEWORTHY We demonstrate that small featural differences between unfamiliar, higher order object representations modulate vector weights during saccadic target selection processing. Such effects have previously only been demonstrated for familiar, simple feature singletons (e.g., color) in which features characterize entire objects. The complexity and novelty of our stimuli suggest that the oculomotor system dynamically receives visual/cognitive information processed in the higher order representational networks of the cortical visual processing hierarchy and integrates this information for saccadic movement planning.

Rapid accumulation of inhibition accounts for saccades curved away from distractors.

Saccades curved toward a distractor are accompanied by a burst of neuronal activation at the distractor locus in the intermediate layers of the superior colliculus (SCi) ~30 ms before the initiation of a saccade. Although saccades curve away from inactivated SCi loci, whether inhibition is restricted to a similar critical epoch for saccades curved away from a distractor remains unclear. We examined this possibility by modeling human saccade curvature as a function of the time between onset of a task irrelevant luminance- or color-modulated distractor and initiation of an impending saccade, referred to as saccade distractor onset asynchrony (SDOA). Our results demonstrated that 70 ms of luminance-modulated distractor processing or 90 ms of color-modulated distractor processing was required to modulate saccade trajectories. As these behavioral, feature-based differences were temporally consistent with the cortically mediated neurophysiological differences in visual onset latencies between luminance and color stimuli observed in the oculomotor and visual system, this method provides a noninvasive means to estimate the timing of peak activation in the oculomotor system. As such, we modeled SDOA functions separately for saccades curved toward and away from distractors and observed that a similar temporal process determined the magnitude of saccade curvatures in both contexts, suggesting that saccades deviate away from a distractor due to a rapid accumulation of inhibition in the critical epoch before saccade initiation.NEW & NOTEWORTHY In this research article, we propose a novel, noninvasive approach to behaviorally model the time course of competitive oculomotor processing. Our results highly resembled those from previously published neurophysiological experiments utilizing similar oculomotor processing contexts, thus validating our approach. Furthermore, this methodology provided new insights into the underlying neural mechanism subserving oculomotor processing given that we applied it to a context with which the neural mechanism is more contentious, and the results clearly favored one view.

Hand placement near the visual stimulus improves orientation selectivity in V2 neurons.

Often, the brain receives more sensory input than it can process simultaneously. Spatial attention helps overcome this limitation by preferentially processing input from a behaviorally-relevant location. Recent neuropsychological and psychophysical studies suggest that attention is deployed to near-hand space much like how the oculomotor system can deploy attention to an upcoming gaze position. Here we provide the first neuronal evidence that the presence of a nearby hand enhances orientation selectivity in early visual processing area V2. When the hand was placed outside the receptive field, responses to the preferred orientation were significantly enhanced without a corresponding significant increase at the orthogonal orientation. Consequently, there was also a significant sharpening of orientation tuning. In addition, the presence of the hand reduced neuronal response variability. These results indicate that attention is automatically deployed to the space around a hand, improving orientation selectivity. Importantly, this appears to be optimal for motor control of the hand, as opposed to oculomotor mechanisms which enhance responses without sharpening orientation selectivity. Effector-based mechanisms for visual enhancement thus support not only the spatiotemporal dissociation of gaze and reach, but also the optimization of vision for their separate requirements for guiding movements.

Feature integration within and across visual streams occurs at different visual processing stages.

Direction repulsion is a perceptual illusion in which the directions of two superimposed surfaces are repulsed away from the real directions of motion. The repulsion is reduced when the surfaces differ in dorsal stream features such as speed. We have previously shown that segmenting the surfaces by color, a ventral stream feature, did not affect repulsion but instead reduced the time needed to process both surfaces. The current study investigated whether segmenting two superimposed surfaces by a feature coprocessed with direction in the dorsal stream (i.e., speed) would also reduce processing time. We found that increasing the speed of one or both surfaces reduced direction repulsion. Since color segmentation does not affect direction repulsion, these results suggest that motion processing integrates speed and direction prior to forming an object representation that includes ventral stream features such as color. Like our previous results for differences in color, differences in speed also decreased processing time. Therefore, the reduction in processing time derives from a later processing stage where both ventral and dorsal features bound into the object representations can reduce the time needed for decision making when those features differentiate the superimposed surfaces from each other.

Identifying a distractor produces object-based inhibition in an allocentric reference frame for saccade planning.

We investigated whether distractor inhibition occurs relative to the target or fixation in a perceptual decision-making task using a purely saccadic response. Previous research has shown that during the process of discriminating a target from distractor, saccades made to a target deviate towards the distractor. Once discriminated, the distractor is inhibited, and trajectories deviate away from the distractor. Saccade deviation magnitudes provide a sensitive measure of target-distractor competition dependent on the distance between them. While saccades are planned in an egocentric reference frame (locations represented relative to fixation), object-based inhibition has been shown to occur in an allocentric reference frame (objects represented relative to each other independent of fixation). By varying the egocentric and allocentric distances of the target and distractor, we found that only egocentric distances contributed to saccade trajectories shifts towards the distractor during active decision-making. When the perceptual decision-making process was complete, and the distractor was inhibited, both ego- and allocentric distances independently contributed to saccade trajectory shifts away from the distractor. This is consistent with independent spatial and object-based inhibitory mechanisms. Therefore, we suggest that distractor inhibition is maintained in cortical visual areas with allocentric maps which then feeds into oculomotor areas for saccade planning.

Oculomotor feature discrimination is cortically mediated.

Eye movements are often directed toward stimuli with specific features. Decades of neurophysiological research has determined that this behavior is subserved by a feature-reweighting of the neural activation encoding potential eye movements. Despite the considerable body of research examining feature-based target selection, no comprehensive theoretical account of the feature-reweighting mechanism has yet been proposed. Given that such a theory is fundamental to our understanding of the nature of oculomotor processing, we propose an oculomotor feature-reweighting mechanism here. We first summarize the considerable anatomical and functional evidence suggesting that oculomotor substrates that encode potential eye movements rely on the visual cortices for feature information. Next, we highlight the results from our recent behavioral experiments demonstrating that feature information manifests in the oculomotor system in order of featural complexity, regardless of whether the feature information is task-relevant. Based on the available evidence, we propose an oculomotor feature-reweighting mechanism whereby (1) visual information is projected into the oculomotor system only after a visual representation manifests in the highest stage of the cortical visual processing hierarchy necessary to represent the relevant features and (2) these dynamically recruited cortical module(s) then perform feature discrimination via shifting neural feature representations, while also maintaining parity between the feature representations in cortical and oculomotor substrates by dynamically reweighting oculomotor vectors. Finally, we discuss how our behavioral experiments may extend to other areas in vision science and its possible clinical applications.

Target-Distractor Competition Modulates Saccade Trajectories in Space and Object Space.

Saccade planning and execution can be affected by a multitude of factors present in a target selection task. Recent studies have shown that the similarity between a target and nearby distractors affects the curvature of saccade trajectories, because of target-distractor competition. To further understand the nature of this competition, we varied the distance between and the similarity of complex target and distractor objects in a delayed match-to-sample task to examine their effects on human saccade trajectories and better understand the underlying neural circuitry. For trials with short saccadic reaction times (SRTs) when target-distractor competition is still active, the distractor is attractive and saccade trajectories are deviated toward the distractor. We found a robust effect of distance consistent with saccade vector averaging, whereas the effect of similarity suggested the existence of an object-based suppressive surround. At longer SRTs, there was sufficient time for competition between the objects to complete and the distractor to be repulsive, which resulted in saccade trajectory deviations away from the distractor exhibiting the effects of a spatial suppressive surround. In terms of similarity, as the target-distractor similarity decreased, the initial saccade angle shifted toward the target, reflecting stronger distractor inhibition. There were no interactions between distance and similarity at any point in the time course of target-distractor competition. Together, saccade trajectories reflect target-distractor competition that is affected independently by both spatial and object space suppressive surrounds. The differences in saccade trajectories at short and long SRTs distinguish between active and completed decision-making processes.

Motion distractors perturb saccade programming later in time than static distractors.

The mechanism that reweights oculomotor vectors based on visual features is unclear. However, the latency of oculomotor visual activations gives insight into their antecedent featural processing. We compared the oculomotor processing time course of grayscale, task-irrelevant static and motion distractors during target selection by continuously measuring a battery of human saccadic behavioral metrics as a function of time after distractor onset. The motion direction was towards or away from the target and the motion speed was fast or slow. We compared static and motion distractors and observed that both distractors elicited curved saccades and shifted endpoints at short latencies (∼25 ms). After 50 ms, saccade trajectory biasing elicited by motion distractors lagged static distractor trajectory biasing by 10 ms. There were no such latency differences between distractor motion directions or motion speeds. This pattern suggests that additional processing of motion stimuli occurred prior to the propagation of visual information into the oculomotor system. We examined the interaction of distractor processing time (DPT) with two additional factors: saccadic reaction time (SRT) and saccadic amplitude. Shorter SRTs were associated with shorter DPT latencies of biased saccade trajectories. Both SRT and saccadic amplitude were associated with the magnitude of saccade trajectory biases.

The role of temporal cortex in the control of attention.

Attention is an indispensable component of active vision. Contrary to the widely accepted notion that temporal cortex processing primarily focusses on passive object recognition, a series of very recent studies emphasize the role of temporal cortex structures, specifically the superior temporal sulcus (STS) and inferotemporal (IT) cortex, in guiding attention and implementing cognitive programs relevant for behavioral tasks. The goal of this theoretical paper is to advance the hypothesis that the temporal cortex attention network (TAN) entails necessary components to actively participate in attentional control in a flexible task-dependent manner. First, we will briefly discuss the general architecture of the temporal cortex with a focus on the STS and IT cortex of monkeys and their modulation with attention. Then we will review evidence from behavioral and neurophysiological studies that support their guidance of attention in the presence of cognitive control signals. Next, we propose a mechanistic framework for executive control of attention in the temporal cortex. Finally, we summarize the role of temporal cortex in implementing cognitive programs and discuss how they contribute to the dynamic nature of visual attention to ensure flexible behavior.

Color Modulates Feature Integration.

Bayesian models of object recognition propose the resolution of ambiguity through probabilistic integration of prior experience with available sensory information. Color, even when task-irrelevant, has been shown to modulate high-level cognitive control tasks. However, it remains unclear how color modulations affect lower-level perceptual processing. We investigated whether color affects feature integration using the flash-jump illusion. This illusion occurs when an apparent motion stimulus, a rectangular bar appearing at different locations along a motion trajectory, changes color at a single position. Observers misperceive this color change as occurring farther along the trajectory of motion. This mislocalization error is proposed to be produced by a Bayesian perceptual framework dependent on responses in area V4. Our results demonstrated that the color of the flash modulated the magnitude of the flash-jump illusion such that participants reported less of a shift, i.e., a more veridical flash location, for both red and blue flashes, as compared to green and yellow. Our findings extend color-dependent modulation effects found in higher-order executive functions into lower-level Bayesian perceptual processes. Our results also support the theory that feature integration is a Bayesian process. In this framework, color modulations play an inherent and automatic role as different colors have different weights in Bayesian perceptual processing.

A motion-dependent distortion of retinotopy in area V4.

When one element in an apparent motion sequence differs in color from the others, it is perceived as shifted along the motion trajectory. We examined whether V4 neurons encode the physical or perceived location of this "flashed" element by recording neuronal responses while monkeys viewed these stimuli. The retinotopic locus of V4 activity evoked by the flashed element shifted along the motion trajectory. The magnitude of the shift is consistent with the perceptual shift in humans viewing identical stimuli. This retinotopic distortion depended on the presence of a flashed element but was observed for both color-selective and non-color-selective neurons. The distortion was undiminished when the flashed element terminated the sequence, a condition that reduced the perceptual shift in humans. These findings are consistent with a Bayesian model of localization in which perceived location is derived from position signals optimally integrated across visual areas.

Adaptation of gender derived from biological motion.

Human observers adapted to complex biological motions that distinguish males from females: viewing the gait of one gender biased judgments of subsequent gaits toward the opposite gender. This adaptation was not simply due to local features of the stimuli but instead relied upon the global motion of the figures. These results suggest the existence of neurons selective for gender and demonstrate that gender-from-motion judgments are not fixed but depend upon recent viewing history.

Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing.

It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing.

Visuomotor origins of covert spatial attention.

Covert spatial attention produces biases in perceptual performance and neural processing of behaviorally relevant stimuli in the absence of overt orienting movements. The neural mechanism that gives rise to these effects is poorly understood. This paper surveys past evidence of a relationship between oculomotor control and visual spatial attention and more recent evidence of a causal link between the control of saccadic eye movements by frontal cortex and covert visual selection. Both suggest that the mechanism of covert spatial attention emerges as a consequence of the reciprocal interactions between neural circuits primarily involved in specifying the visual properties of potential targets and those involved in specifying the movements needed to fixate them.