Saccades to partially occluded objects: Perceptual completion mediates oculomotor control.

Oculomotor behavior typically consists of directing gaze to objects in complex scenes for the purpose of extracting detailed perceptual information. Here, we probed the nature of the visual representations over which saccades to objects are computed. We contrasted an image-based oculomotor control hypothesis, holding that saccades are computed solely over information explicit in the retinal image, and an object-based oculomotor control hypothesis, holding that saccades are computed over object representations reflecting the three-dimensional structure of the scene. We recorded saccade landing positions to partially occluded objects in a naturalistic search task. In Experiment 1, saccade landing positions were biased toward the center of the perceptually completed object. Experiment 2 demonstrated that the bias held even when it would have been strategically advantageous to avoid it. Experiment 3 demonstrated that the bias was not due to image-level differences generated by the presence of occluders. The results indicate that saccade motor programs are computed, at least in part, over object-level representations reflecting the completion of occluded surfaces.

Object-mediated overwriting across saccades.

How are visual sensory representations that are acquired peripherally from a saccade target related to sensory representations generated foveally after the saccade? We tested the hypothesis that, when the two representations are perceived to belong to the same object, the post-saccadic value tends to overwrite the pre-saccadic value. Participants executed a saccade to a colored target object, which sometimes changed during the saccade by ±15°, 30°, or 45° in color space. They were post-cued to report either the pre-saccadic or post-saccadic color in a continuous report procedure. Substantial overwriting of the pre-saccadic color by the post-saccadic color was observed. Moreover, the introduction of a brief post-saccadic blank interval (which disrupted the perception of object correspondence) led to a substantial reduction in overwriting. The results provide the first direct evidence for an object-mediated overwriting mechanism across saccades, in which post-saccadic values automatically replace pre-saccadic values.

Dynamic interactions between visual working memory and saccade target selection.

Recent psychophysical experiments have shown that working memory for visual surface features interacts with saccadic motor planning, even in tasks where the saccade target is unambiguously specified by spatial cues. Specifically, a match between a memorized color and the color of either the designated target or a distractor stimulus influences saccade target selection, saccade amplitudes, and latencies in a systematic fashion. To elucidate these effects, we present a dynamic neural field model in combination with new experimental data. The model captures the neural processes underlying visual perception, working memory, and saccade planning relevant to the psychophysical experiment. It consists of a low-level visual sensory representation that interacts with two separate pathways: a spatial pathway implementing spatial attention and saccade generation, and a surface feature pathway implementing color working memory and feature attention. Due to bidirectional coupling between visual working memory and feature attention in the model, the working memory content can indirectly exert an effect on perceptual processing in the low-level sensory representation. This in turn biases saccadic movement planning in the spatial pathway, allowing the model to quantitatively reproduce the observed interaction effects. The continuous coupling between representations in the model also implies that modulation should be bidirectional, and model simulations provide specific predictions for complementary effects of saccade target selection on visual working memory. These predictions were empirically confirmed in a new experiment: Memory for a sample color was biased toward the color of a task-irrelevant saccade target object, demonstrating the bidirectional coupling between visual working memory and perceptual processing.

Are selection history effects limited to implicit forms of memory? Evidence from intertrial repetition.

Selection history effects in visual attention are typically considered implicit memory effects. In three experiments, we investigated if a key selection history effect, intertrial priming, could be based on the incidental application of explicit memory. In the basic search task (Experiment 1), participants searched for real-world objects from different categories. We examined nonpredictive, intertrial repetition at two levels: (1) the repetition of target location from trial N-1 to trial N and (2) the repetition of target location and color within a category. Reliable repetition advantages were observed at both levels. In Experiments 2-4, we examined whether participants had explicit access to the target values driving the selection history effects here. In Experiments 2 and 3, participants could reliably report the properties of the immediately preceding search target. In Experiment 4, participants could reliably report the properties of the last target exemplar they had found in each of the 36 categories. These data indicate that guidance by selection history was based on the nonstrategic application of memory representations that could be explicitly retrieved and reported. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Visual working memory modulates rapid eye movements to simple onset targets.

Representations in visual working memory (VWM) influence attention and gaze control in complex tasks, such as visual search, that require top-down selection to resolve stimulus competition. VWM and visual attention clearly interact, but the mechanism of that interaction is not well understood. In the research reported here, we demonstrated that in the absence of stimulus competition or goal-level biases, VWM representations of object features influence the spatiotemporal dynamics of extremely simple eye movements. The influence of VWM therefore extends into the most basic operations of the oculomotor system.

Visual working memory modulates low-level saccade target selection: evidence from rapidly generated saccades in the global effect paradigm.

In three experiments, we examined the influence of visual working memory (VWM) on the metrics of saccade landing position in a global effect paradigm. Participants executed a saccade to the more eccentric object in an object pair appearing on the horizontal midline, to the left or right of central fixation. While completing the saccade task, participants maintained a color in VWM for an unrelated memory task. Either the color of the saccade target matched the memory color (target match), the color of the distractor matched the memory color (distractor match), or the colors of neither object matched the memory color (no match). In the no-match condition, saccades tended to land at the midpoint between the two objects: the global, or averaging, effect. However, when one of the two objects matched VWM, the distribution of landing position shifted toward the matching object, both for target match and for distractor match. VWM modulation of landing position was observed even for the fastest quartile of saccades, with a mean latency as low as 112 ms. Effects of VWM on such rapidly generated saccades, with latencies in the express-saccade range, indicate that VWM interacts with the initial sweep of visual sensory processing, modulating perceptual input to oculomotor systems and thereby biasing oculomotor selection. As a result, differences in memory match produce effects on landing position similar to the effects generated by differences in physical salience.

Category-specific learning of color, orientation, and position regularities guide visual search.

In six experiments, we examined how object categories structure the learning of environmental regularities to guide visual search. Participants searched for pictures of exemplars from a set of real-world categories in a repeated search task modeled on the contextual cuing literature. Each trial began with a category label cue, followed by a search array of natural object photographs, with one target object matching the category label. Participants completed a series of search blocks, each containing one search trial per category. Individual categories were assigned either to the Repeated condition or to the Novel condition. For Repeated categories, a perceptual feature value of target objects remained constant across each search for that category: color (Experiments 1 and 3), orientation (Experiment 2), and position (Experiment 4). For Novel categories, the relevant feature value varied randomly for each search for that category. We observed a categorical cuing effect, with faster improvement in reaction time across blocks for Repeated compared with Novel categories. This effect reflected both the episodic retrieval of the immediately preceding search episode in that category and cumulative learning across multiple searches within a category. The cuing effect was observed from the very first repetition, a point in the experiment where the learning effect was not plausibly strategic. Finally, participants could reliably retrieve and report the repeated values in memory tests administered either at the end of the experiment or when the effect first emerged (Experiments 5 and 6), demonstrating that nonstrategic guidance of attention can be driven by explicitly available memory. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Simultaneous control of attention by multiple working memory representations.

Working memory representations play a key role in controlling attention by making it possible to shift attention to task-relevant objects. Visual working memory has a capacity of three to four objects, but recent studies suggest that only one representation can guide attention at a given moment. We directly tested this proposal by monitoring eye movements while observers performed a visual search task in which they attempted to limit attention to objects drawn in two colors. When the observers were motivated to attend to one color at a time, they searched many consecutive items of one color (long run lengths) and exhibited a delay prior to switching gaze from one color to the other (switch cost). In contrast, when they were motivated to attend to both colors simultaneously, observers' gaze switched back and forth between the two colors frequently (short run lengths), with no switch cost. Thus, multiple working memory representations can concurrently guide attention.

An object-mediated updating account of insensitivity to transsaccadic change.

Recent evidence has suggested that relatively precise information about the location and visual form of a saccade target object is retained across a saccade. However, this information appears to be available for report only when the target is removed briefly, so that the display is blank when the eyes land. We hypothesized that the availability of precise target information is dependent on whether a post-saccade object is mapped to the same object representation established for the presaccade target. If so, then the post-saccade features of the target overwrite the presaccade features, a process of object mediated updating in which visual masking is governed by object continuity. In two experiments, participants' sensitivity to the spatial displacement of a saccade target was improved when that object changed surface feature properties across the saccade, consistent with the prediction of the object-mediating updating account. Transsaccadic perception appears to depend on a mechanism of object-based masking that is observed across multiple domains of vision. In addition, the results demonstrate that surface-feature continuity contributes to visual stability across saccades.

Visual attention to emotion in depression: facilitation and withdrawal processes.

Attentional biases for sadness are integral to cognitive theories of depression, but do not emerge under all conditions. Some researchers have argued that depression is associated with delayed withdrawal from, but not facilitated initial allocation of attention toward, sadness. We compared two types of withdrawal processes in clinically depressed and non-depressed individuals: (1) withdrawal requiring overt eye movements during visual search; and (2) covert disengagement of attention in a modified cueing paradigm. We also examined initial allocation of attention towards emotion on the visual search task, allowing comparison of withdrawal and facilitation processes. As predicted, we found no evidence of facilitated attention towards sadness in depressed individuals. However, we also found no evidence of depression-linked differences in withdrawal of attention from sadness on either task, offering no support for the theory that depression is associated with withdrawal rather than initial facilitation of attention.

Real-world object categories and scene contexts conjointly structure statistical learning for the guidance of visual search.

We examined how object categories and scene contexts act in conjunction to structure the acquisition and use of statistical regularities to guide visual search. In an exposure session, participants viewed five object exemplars in each of two colors in each of 42 real-world categories. Objects were presented individually against scene context backgrounds. Exemplars within a category were presented with different contexts as a function of color (e.g., the five red staplers were presented with a classroom scene, and the five blue staplers with an office scene). Participants then completed a visual search task, in which they searched for novel exemplars matching a category label cue among arrays of eight objects superimposed over a scene background. In the context-match condition, the color of the target exemplar was consistent with the color associated with that combination of category and scene context from the exposure phase (e.g., a red stapler in a classroom scene). In the context-mismatch condition, the color of the target was not consistent with that association (e.g., a red stapler in an office scene). In two experiments, search response time was reliably lower in the context-match than in the context-mismatch condition, demonstrating that the learning of category-specific color regularities was itself structured by scene context. The results indicate that categorical templates retrieved from long-term memory are biased toward the properties of recent exemplars and that this learning is organized in a scene-specific manner.

Guidance of attention from visual working memory is feature-based, not object-based: Implications for models of feature binding.

A classic question in visual working memory (VWM) research is whether features from the same object are bound directly in an integrated representation or are maintained separately and bound only indirectly though shared location. Here, we examined this question using a novel method that probed the effects of VWM on the guidance of attention (rather than requiring explicit access to VWM content, as has typically been used). Participants remembered two color-shape conjunction objects. During a retention-interval search task, they searched for a target letter among distractor letters superimposed over color-shape conjunction items. There were two critical conditions. In the same-object-match condition, one search item matched both the color and shape of a single remembered object. In the different-object-match condition, one search item matched the color from one remembered object and the shape from the other. Robust effects of VWM-based guidance were observed, both when probing the incidental guidance of attention (Experiments 1 and 2) and the strategic guidance of attention (Experiment 3). Critically, in none of the experiments was the magnitude of guidance greater for same-object-match than for different-object-match. The results indicate that the representational units of guidance from VWM are individual features rather than integrated objects. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Examining the influence of task set on eye movements and fixations.

The purpose of the present study was to examine the influence of task set on the spatial and temporal characteristics of eye movements during scene perception. In previous work, when strong control was exerted over the viewing task via specification of a target object (as in visual search), task set biased spatial, rather than temporal, parameters of eye movements. Here, we find that more participant-directed tasks (in which the task establishes general goals of viewing rather than specific objects to fixate) affect not only spatial (e.g., saccade amplitude) but also temporal parameters (e.g., fixation duration). Further, task set influenced the rate of change in fixation duration over the course of viewing but not saccade amplitude, suggesting independent mechanisms for control of these parameters.

Categorical cuing: Object categories structure the acquisition of statistical regularities to guide visual search.

Recent statistical regularities have been demonstrated to influence visual search across a wide variety of learning mechanisms and search features. To function in the guidance of real-world search, however, such learning must be contingent on the context in which the search occurs and the object that is the target of search. The former has been studied extensively under the rubric of contextual cuing. Here, we examined, for the first time, categorical cuing: The role of object categories in structuring the acquisition of statistical regularities used to guide visual search. After an exposure session in which participants viewed six exemplars with the same general color in each of 40 different real-world categories, they completed a categorical search task, in which they searched for any member of a category based on a label cue. Targets that matched recent within-category regularities were found faster than targets that did not (Experiment 1). Such categorical cuing was also found to span multiple recent colors within a category (Experiment 2). It was observed to influence both the guidance of search to the target object (Experiment 3) and the basic operation of assigning single exemplars to categories (Experiment 4). Finally, the rapid acquisition of category-specific regularities was also quickly modified, with the benefit rapidly decreasing during the search session as participants were exposed equally to the two possible colors in each category. The results demonstrate that object categories organize the acquisition of perceptual regularities and that this learning exerts strong control over the instantiation of the category representation as a template for visual search. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Visual working memory content influences correspondence processes.

Representing objects as continuous across time requires the establishment of correspondence, whereby current stimuli are represented as deriving from the same object as earlier stimuli. Spatiotemporal continuity and surface-feature similarity play important roles in these correspondence processes. Because objects are often represented across extended periods of time, visual working memory (VWM) content should also play a role in object correspondence. We tested this prediction using Ternus motion. Displays consisted of three-disk arrays that shifted horizontally by one position between frames. Depending on how correspondence is resolved, Ternus displays are perceived as group motion, where all three disks appear to move together, or element motion, where one disk appears to jump across the others. Reports of which motion is perceived provide an index of how correspondence was resolved. Ternus displays were adapted such that the color of some disks biased element motion while the color of others biased group motion. Maintaining one or the other of the colors in VWM for later report systematically biased which type of motion was perceived (Experiments 1 and 2). When color was incidental to the VWM task, however, it did not (Experiment 3). These results confirm that VWM content contributes to object correspondence. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Feature-based guidance of attention by visual working memory is applied independently of remembered object location.

Visual working memory (VWM) has been implicated both in the online representation of object tokens (in the object-file framework) and in the top-down guidance of attention during visual search, implementing a feature template. It is well established that object representations in VWM are structured by location, with access to the content of VWM modulated by position consistency. In the present study, we examined whether this property generalizes to the guidance of attention. Specifically, in two experiments, we probed whether the guidance of spatial attention from features in VWM is modulated by the position of the object from which these features were encoded. Participants remembered an object with an incidental color. Items in a subsequent search array could match either the color of the remembered object, the location, or both. Robust benefits of color match (when the matching item was the target) and costs (when the matching items was a distractor) were observed. Critically, the magnitude of neither effect was influenced by spatial correspondence. The results demonstrate that features in VWM influence attentional priority maps in a manner that does not necessarily inherit the spatial structure of the object representations in which those features are maintained.

The architecture of working memory: Features from multiple remembered objects produce parallel, coactive guidance of attention in visual search.

Theories of working memory (WM) differ in their claims about the number of items that can be maintained in a state that directly interacts with other, ongoing cognitive operations (termed the focus of attention). A similar debate has arisen in the literature on visual working memory (VWM), focused on the number of items that can simultaneously interact with attentional priority. In 3 experiments, we used a redundancy-gain paradigm to provide a comprehensive test of the latter question. Participants searched for 2 cued features (e.g., a color and a shape) within a search array. The cued feature values changed on a trial-by-trial basis, requiring VWM. The target (when present) could match 1 of the cued features (single-target trials) or both cued features (redundant-target trials). We tested whether response time distributions contained a substantial proportion of trials with redundant-target responses that were faster than predicted by 2 independent guidance processes operating in parallel (i.e., violations of the race-model inequality). Violations are consistent with a coactive architecture in which both cued values guide attention in parallel and sum on the priority map. Robust violations were observed in all cases predicted by the hypothesis that multiple items in VWM can guide attention simultaneously, and these results were inconsistent with the hypothesis that guidance is limited to a single item simultaneously. When considered in the larger context of the literature on VWM and attention, the present results are consistent with a model of WM architecture in which the focus of attention can maintain multiple, independent representations. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Novelty is not always the best policy: inhibition of return and facilitation of return as a function of visual task.

We report a study that examined whether inhibition of return (IOR) is specific to visual search or a general characteristic of visual behavior. Participants were shown a series of scenes and were asked to (a) search each scene for a target, (b) memorize each scene, (c) rate how pleasant each scene was, or (d) view each scene freely. An examination of saccadic reaction times to probes provided evidence of IOR during search: Participants were slower to look at probes at previously fixated locations than to look at probes at novel locations. For the other three conditions, however, the opposite pattern of results was observed: Participants were faster to look at probes at previously fixated locations than to look at probes at novel locations, a facilitation-of-return effect that has not been reported previously. These results demonstrate that IOR is a search-specific strategy and not a general characteristic of visual attention.

The comparison of visual working memory representations with perceptual inputs.

The human visual system can notice differences between memories of previous visual inputs and perceptions of new visual inputs, but the comparison process that detects these differences has not been well characterized. In this study, the authors tested the hypothesis that differences between the memory of a stimulus array and the perception of a new array are detected in a manner that is analogous to the detection of simple features in visual search tasks. That is, just as the presence of a task-relevant feature in visual search can be detected in parallel, triggering a rapid shift of attention to the object containing the feature, the presence of a memory-percept difference along a task-relevant dimension can be detected in parallel, triggering a rapid shift of attention to the changed object. Supporting evidence was obtained in a series of experiments in which manual reaction times, saccadic reaction times, and event-related potential latencies were examined. However, these experiments also showed that a slow, limited-capacity process must occur before the observer can make a manual change detection response.

Contrasting episodic and template-based guidance during search through natural scenes.

Visual search through natural scenes can be guided by knowledge of where a target object has been observed previously (episodic guidance) and knowledge of that object's visual properties (template guidance). In the present experiments, we compared the relative contributions of these two sources of guidance. Episodic guidance was implemented in a contextual cuing task: participants searched multiple times through a set of scenes for a target letter that appeared in a consistent location within each scene. Template guidance was implemented by the color match between a critical distractor in each scene and a secondary visual working memory (VWM) load. There were four main findings. First, search time decreased with increasing scene repetition; episodic memory guided search. Second, the critical distractor was fixated more frequently on match compared with mismatch trials, consistent with automatic template guidance. Third, the VWM-match effect persisted in blocks with strong episodic guidance. Finally, VWM-match effects were observed from the first saccade during search, whereas episodic guidance to the target developed only later in the trial. The results support a view of natural search in which template-based mechanisms operate early during search in a manner that is not strongly constrained by scene-based forms of guidance, such as episodic knowledge. (PsycINFO Database Record (c) 2019 APA, all rights reserved).