Goal-Dependent Use of Temporal Regularities to Orient Attention under Spatial and Action Uncertainty.

The temporal regularities in our environments support the proactive dynamic anticipation of relevant events. In visual attention, one important outstanding question is whether temporal predictions must be linked to predictions about spatial locations or motor plans to facilitate behaviour. To test this, we developed a task for manipulating temporal expectations and task relevance of visual stimuli appearing within rapidly presented streams, while stimulus location and responding hand remained uncertain. Differently coloured stimuli appeared in one of two concurrent (left and right) streams with distinct temporal probability structures. Targets were defined by colour on a trial-by-trial basis and appeared equiprobably in either stream, requiring a localisation response. Across two experiments, participants were faster and more accurate at detecting temporally predictable targets compared to temporally unpredictable targets. We conclude that temporal expectations learned incidentally from temporal regularities can be called upon flexibly in a goal-driven manner to guide behaviour. Moreover, we show that visual temporal attention can facilitate performance in the absence of concomitant spatial or motor expectations in dynamically unfolding contexts.

Shifting attention between perception and working memory.

Most everyday tasks require shifting the focus of attention between sensory signals in the external environment and internal contents in working memory. To date, shifts of attention have been investigated within each domain, but shifts between the external and internal domain remain poorly understood. We developed a combined perception and working-memory task to investigate and compare the consequences of shifting spatial attention within and between domains in the service of a common orientation-reproduction task. Participants were sequentially cued to attend to items either in working memory or to an upcoming sensory stimulation. Stay trials provided a baseline condition, while shift trials required participants to shift their attention to another item within the same or different domain. Validating our experimental approach, we found evidence that participants shifted attention effectively in either domain (Experiment 1). In addition, we observed greater costs when transitioning attention between as compared to within domains (Experiments 1, 2). Strikingly, these costs persisted even when participants were given more time to complete the attentional shift (Experiment 2). Biases in fixational gaze behaviour tracked attentional orienting in both domains, but revealed no latency or magnitude difference for within- versus between-domain shifts (Experiment 1). Collectively, the results from Experiments 1 and 2 suggest that shifting between attentional domains might be regulated by a unique control function. Our results break new ground for exploring the ubiquitous act of shifting attention between perception and working memory to guide adaptive behaviour in everyday cognition.

Terms of debate: Consensus definitions to guide the scientific discourse on visual distraction.

Hypothesis-driven research rests on clearly articulated scientific theories. The building blocks for communicating these theories are scientific terms. Obviously, communication - and thus, scientific progress - is hampered if the meaning of these terms varies idiosyncratically across (sub)fields and even across individual researchers within the same subfield. We have formed an international group of experts representing various theoretical stances with the goal to homogenize the use of the terms that are most relevant to fundamental research on visual distraction in visual search. Our discussions revealed striking heterogeneity and we had to invest much time and effort to increase our mutual understanding of each other's use of central terms, which turned out to be strongly related to our respective theoretical positions. We present the outcomes of these discussions in a glossary and provide some context in several essays. Specifically, we explicate how central terms are used in the distraction literature and consensually sharpen their definitions in order to enable communication across theoretical standpoints. Where applicable, we also explain how the respective constructs can be measured. We believe that this novel type of adversarial collaboration can serve as a model for other fields of psychological research that strive to build a solid groundwork for theorizing and communicating by establishing a common language. For the field of visual distraction, the present paper should facilitate communication across theoretical standpoints and may serve as an introduction and reference text for newcomers.

Prospection of Potential Actions during Visual Working Memory Starts Early, Is Flexible, and Predicts Behavior.

For visual working memory to serve upcoming behavior, it is crucial that we prepare for the potential use of working-memory contents ahead of time. Recent studies have demonstrated how the prospection and planning for an upcoming manual action starts early after visual encoding, and occurs alongside visual retention. Here, we address whether such "output planning" in visual working memory flexibly adapts to different visual-motor mappings, and occurs even when an upcoming action will only potentially become relevant for behavior. Human participants (female and male) performed a visual-motor working memory task in which they remembered one or two colored oriented bars for later (potential) use. We linked, and counterbalanced, the tilt of the visual items to specific manual responses. This allowed us to track planning of upcoming behavior through contralateral attenuation of ß band activity, a canonical motor-cortical EEG signature of manual-action planning. The results revealed how action encoding and subsequent planning alongside visual working memory (1) reflect anticipated task demands rather than specific visual-motor mappings, (2) occur even for actions that will only potentially become relevant for behavior, and (3) are associated with faster performance for the encoded item, at the expense of performance to other working-memory content. This reveals how the potential prospective use of visual working memory content is flexibly planned early on, with consequences for the speed of memory-guided behavior.SIGNIFICANCE STATEMENT It is increasingly studied how visual working memory helps us to prepare for the future, in addition to how it helps us to hold onto the past. Recent studies have demonstrated that the planning of prospective actions occurs alongside encoding and retention in working memory. We show that such early "output planning" flexibly adapts to varying visual-motor mappings, occurs both for certain and potential actions, and predicts ensuing working-memory guided behavior. These results highlight the flexible and future-oriented nature of visual working memory, and provide insight into the neural basis of the anticipatory dynamics that translate visual representations into adaptive upcoming behavior.

Good-enough attentional guidance.

Theories of attention posit that attentional guidance operates on information held in a target template within memory. The template is often thought to contain veridical target features, akin to a photograph, and to guide attention to objects that match the exact target features. However, recent evidence suggests that attentional guidance is highly flexible and often guided by non-veridical features, a subset of features, or only associated features. We integrate these findings and propose that attentional guidance maximizes search efficiency based on a 'good-enough' principle to rapidly localize candidate target objects. Candidates are then serially interrogated to make target-match decisions using more precise information. We suggest that good-enough guidance optimizes the speed-accuracy-effort trade-offs inherent in each stage of visual search.

Consequences of predictable temporal structure in multi-task situations.

In everyday life, we often anticipate the timing of an upcoming task or event while actively engaging in another. Here, we investigated the effects of predictable temporal structure within such a multi-task scenario. In a visual working-memory task, we manipulated whether the onset of a working-memory probe could be predicted in time, while also embedding a simple intervening task within the delay period. We first show that working-memory performance benefitted from temporal expectations, even though an intervening task had to be completed in the interim. Moreover, temporal predictions regarding the upcoming working-memory probe additionally affected performance on the intervening task, resulting in faster responses when the memory probe was expected early, and slower responses when the memory probe was expected late, as compared to when it was temporally unpredictable. Because the intervening task always occurred at the same time during the memory delay, differences in performance on this intervening task result from a between-task consequence of temporal expectation. Thus, we show that within multi-task settings, knowing when working-memory contents will be required for an upcoming task not only facilitates performance of the associated working-memory task, but can also influence the performance of other, intervening tasks.

Be there on time: Spatial-temporal regularities guide young children's attention in dynamic environments.

Children's ability to benefit from spatiotemporal regularities to detect goal-relevant targets was tested in a dynamic, extended context. Young adults and children (from a low-deprivation area school in the United Kingdom; N = 80; 5-6 years; 39 female; ethics approval did not permit individual-level race/ethnicity surveying) completed a dynamic visual-search task. Targets and distractors faded in and out of a display over seconds. Half of the targets appeared at predictable times and locations. Search performance in children was poorer overall. Nevertheless, they benefitted equivalently from spatiotemporal regularities, detecting more predictable than unpredictable targets. Children's benefits from predictions correlated positively with their attention. The study brings ecological validity to the study of attentional guidance in children, revealing striking behavioral benefits of dynamic experience-based predictions.

Right place, right time: Spatiotemporal predictions guide attention in dynamic visual search.

Visual search is a fundamental element of human behavior and is predominantly studied in a laboratory setting using static displays. However, real-life search is often an extended process taking place in dynamic environments. We have designed a dynamic-search task in order to incorporate the temporal dimension into visual search. Using this task, we tested how participants learn and utilize spatiotemporal regularities embedded within the environment to guide performance. Participants searched for eight instances of a target that faded in and out of a display containing similarly transient distractors. In each trial, four of the eight targets appeared in a temporally predictable fashion with one target appearing in each of four spatially separated quadrants. The other four targets were spatially and temporally unpredictable. Participants' performance was significantly better for spatiotemporally predictable compared to unpredictable targets (Experiments 1-4). The effects were reliable over different patterns of spatiotemporal predictability (Experiment 2) and primarily reflected long-term learning over trials (Experiments 3, 4), although single-trial priming effects also contributed (Experiment 4). Eye-movement recordings (Experiment 1) revealed that spatiotemporal regularities guide attention proactively and dynamically. Taken together, our results show that regularities across both space and time can guide visual search and this guidance can primarily be attributed to robust long-term representations of these regularities. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Shielding working-memory representations from temporally predictable external interference.

Protecting working-memory content from distracting external sensory inputs and intervening tasks is an ubiquitous demand in daily life. Here, we ask whether and how temporal expectations about external events can help mitigate effects of such interference during working-memory retention. We manipulated the temporal predictability of interfering items that occurred during the retention period of a visual working-memory task and report that temporal expectations reduce the detrimental influence of external interference on subsequent memory performance. Moreover, to determine if the protective effects of temporal expectations rely on distractor suppression or involve shielding of internal representations, we compared effects after irrelevant distractors that could be ignored vs. interrupters that required a response. Whereas distractor suppression may be sufficient to confer protection from predictable distractors, any benefits after interruption are likely to involve memory shielding. We found similar benefits of temporal expectations after both types of interference. We conclude that temporal expectations may play an important role in safeguarding behaviour based on working memory - acting through mechanisms that include the shielding of internal content from external interference.

Output planning at the input stage in visual working memory.

Working memory serves as the buffer between past sensations and future behavior, making it vital to understand not only how we encode and retain sensory information in memory but also how we plan for its upcoming use. We ask when prospective action goals emerge alongside the encoding and retention of visual information in working memory. We show that prospective action plans do not emerge gradually during memory delays but are brought into memory early, in tandem with sensory encoding. This action encoding (i) precedes a second stage of action preparation that adapts to the time of expected memory utilization, (ii) occurs even ahead of an intervening motor task, and (iii) predicts visual memory-guided behavior several seconds later. By bringing prospective action plans into working memory at an early stage, the brain creates a dual (visual-motor) memory code that can make memories more effective and robust for serving ensuing behavior.

Functional biases in attentional templates from associative memory.

In everyday life, attentional templates-which facilitate the perception of task-relevant sensory inputs-are often based on associations in long-term memory. We ask whether templates retrieved from memory are necessarily faithful reproductions of the encoded information or if associative-memory templates can be functionally adapted after retrieval in service of current task demands. Participants learned associations between four shapes and four colored gratings, each with a characteristic combination of color (green or pink) and orientation (left or right tilt). On each trial, observers saw one shape followed by a grating and indicated whether the pair matched the learned shape-grating association. Across experimental blocks, we manipulated the types of nonmatch (lure) gratings most often presented. In some blocks the lures were most likely to differ in color but not tilt, whereas in other blocks this was reversed. If participants functionally adapt the retrieved template such that the distinguishing information between lures and targets is prioritized, then they should overemphasize the most commonly diagnostic feature dimension within the template. We found evidence for this in the behavioral responses to the lures: participants were more accurate and faster when responding to common versus rare lures, as predicted by the functional-but not the strictly veridical-template hypothesis. This shows that templates retrieved from memory can be functionally biased to optimize task performance in a flexible, context-dependent, manner.

One Thing Leads to Another: Anticipating Visual Object Identity Based on Associative-Memory Templates.

Probabilistic associations between stimuli afford memory templates that guide perception through proactive anticipatory mechanisms. A great deal of work has examined the behavioral consequences and human electrophysiological substrates of anticipation following probabilistic memory cues that carry spatial or temporal information to guide perception. However, less is understood about the electrophysiological substrates linked to anticipating the sensory content of events based on recurring associations between successive events. Here, we demonstrate behavioral and electrophysiological signatures of using associative-memory templates to guide perception, while equating spatial and temporal anticipation (experiments 1 and 2), as well as target probability and response demands (experiment 2). By recording the electroencephalogram in the two experiments (N = 55; 24 females), we show that two markers in human electrophysiology implicated in spatial and temporal anticipation also contribute to the anticipation of perceptual identity, as follows: attenuation of alpha-band oscillations and the contingent negative variation (CNV). Together, our results show that memory-guided identity templates proactively impact perception and are associated with anticipatory states of attenuated alpha oscillations and the CNV. Furthermore, by isolating object-identity anticipation from spatial and temporal anticipation, our results suggest a role for alpha attenuation and the CNV in specific visual content anticipation beyond general changes in neural excitability or readiness.SIGNIFICANCE STATEMENT Probabilistic associations between stimuli afford memory templates that guide perception through proactive anticipatory mechanisms. The current work isolates the behavioral benefits and electrophysiological signatures of memory-guided identity-based anticipation, while equating anticipation of space, time, motor responses, and task relevance. Our results show that anticipation of the specific identity of a forthcoming percept impacts performance and is associated with states of attenuated alpha oscillations and the contingent negative variation, extending previous work implicating these neural substrates in spatial and temporal preparatory attention. Together, this work bridges fields of attention, memory, and perception, providing new insights into the neural mechanisms that support complex attentional templates.

Reading scenes: how scene grammar guides attention and aids perception in real-world environments.

Despite many recent technical advances, the human efficacy of naturalistic scene processing is still unparalleled. What guides attention in real world environments? How does scene context affect object search and classification? And how are the many predictions we have with regards to our visual environment structured? Here, we review the latest findings that speak to these questions, ranging from traditional psychophysics, eye movement, and neurocognitive studies in the lab to experiments in virtual reality (VR) and the real world. The growing interest and grasp of a scene's inner workings are enriching our understanding of the efficiency of scene perception and could inspire new technological and clinical advances.

Anchoring visual search in scenes: Assessing the role of anchor objects on eye movements during visual search.

The arrangement of the contents of real-world scenes follows certain spatial rules that allow for extremely efficient visual exploration. What remains underexplored is the role different types of objects hold in a scene. In the current work, we seek to unveil an important building block of scenes-anchor objects. Anchors hold specific spatial predictions regarding the likely position of other objects in an environment. In a series of three eye tracking experiments we tested what role anchor objects occupy during visual search. In all of the experiments, participants searched through scenes for an object that was cued in the beginning of each trial. Critically, in half of the scenes a target relevant anchor was swapped for an irrelevant, albeit semantically consistent, object. We found that relevant anchor objects can guide visual search leading to faster reaction times, less scene coverage, and less time between fixating the anchor and the target. The choice of anchor objects was confirmed through an independent large image database, which allowed us to identify key attributes of anchors. Anchor objects seem to play a unique role in the spatial layout of scenes and need to be considered for understanding the efficiency of visual search in realistic stimuli.

Lost in the supermarket: Quantifying the cost of partitioning memory sets in hybrid search.

The items on a memorized grocery list are not relevant in every aisle; for example, it is useless to search for the cabbage in the cereal aisle. It might be beneficial if one could mentally partition the list so only the relevant subset was active, so that vegetables would be activated in the produce section. In four experiments, we explored observers' abilities to partition memory searches. For example, if observers held 16 items in memory, but only eight of the items were relevant, would response times resemble a search through eight or 16 items? In Experiments 1a and 1b, observers were not faster for the partition set; however, they suffered relatively small deficits when "lures" (items from the irrelevant subset) were presented, indicating that they were aware of the partition. In Experiment 2 the partitions were based on semantic distinctions, and again, observers were unable to restrict search to the relevant items. In Experiments 3a and 3b, observers attempted to remove items from the list one trial at a time but did not speed up over the course of a block, indicating that they also could not limit their memory searches. Finally, Experiments 4a, 4b, 4c, and 4d showed that observers were able to limit their memory searches when a subset was relevant for a run of trials. Overall, observers appear to be unable or unwilling to partition memory sets from trial to trial, yet they are capable of restricting search to a memory subset that remains relevant for several trials. This pattern is consistent with a cost to switching between currently relevant memory items.

One visual search, many memory searches: An eye-tracking investigation of hybrid search.

Suppose you go to the supermarket with a shopping list of 10 items held in memory. Your shopping expedition can be seen as a combination of visual search and memory search. This is known as "hybrid search." There is a growing interest in understanding how hybrid search tasks are accomplished. We used eye tracking to examine how manipulating the number of possible targets (the memory set size [MSS]) changes how observers (Os) search. We found that dwell time on each distractor increased with MSS, suggesting a memory search was being executed each time a new distractor was fixated. Meanwhile, although the rate of refixation increased with MSS, it was not nearly enough to suggest a strategy that involves repeatedly searching visual space for subgroups of the target set. These data provide a clear demonstration that hybrid search tasks are carried out via a "one visual search, many memory searches" heuristic in which Os examine items in the visual array once with a very low rate of refixations. For each item selected, Os activate a memory search that produces logarithmic response time increases with increased MSS. Furthermore, the percentage of distractors fixated was strongly modulated by the MSS: More items in the MSS led to a higher percentage of fixated distractors. Searching for more potential targets appears to significantly alter how Os approach the task, ultimately resulting in more eye movements and longer response times.

Hybrid foraging search: Searching for multiple instances of multiple types of target.

This paper introduces the "hybrid foraging" paradigm. In typical visual search tasks, observers search for one instance of one target among distractors. In hybrid search, observers search through visual displays for one instance of any of several types of target held in memory. In foraging search, observers collect multiple instances of a single target type from visual displays. Combining these paradigms, in hybrid foraging tasks observers search visual displays for multiple instances of any of several types of target (as might be the case in searching the kitchen for dinner ingredients or an X-ray for different pathologies). In the present experiment, observers held 8-64 target objects in memory. They viewed displays of 60-105 randomly moving photographs of objects and used the computer mouse to collect multiple targets before choosing to move to the next display. Rather than selecting at random among available targets, observers tended to collect items in runs of one target type. Reaction time (RT) data indicate searching again for the same item is more efficient than searching for any other targets, held in memory. Observers were trying to maximize collection rate. As a result, and consistent with optimal foraging theory, they tended to leave 25-33% of targets uncollected when moving to the next screen/patch. The pattern of RTs shows that while observers were collecting a target item, they had already begun searching memory and the visual display for additional targets, making the hybrid foraging task a useful way to investigate the interaction of visual and memory search.

Searching while loaded: Visual working memory does not interfere with hybrid search efficiency but hybrid search uses working memory capacity.

In "hybrid search" tasks, such as finding items on a grocery list, one must search the scene for targets while also searching the list in memory. How is the representation of a visual item compared with the representations of items in the memory set? Predominant theories would propose a role for visual working memory (VWM) either as the site of the comparison or as a conduit between visual and memory systems. In seven experiments, we loaded VWM in different ways and found little or no effect on hybrid search performance. However, the presence of a hybrid search task did reduce the measured capacity of VWM by a constant amount regardless of the size of the memory or visual sets. These data are broadly consistent with an account in which VWM must dedicate a fixed amount of its capacity to passing visual representations to long-term memory for comparison to the items in the memory set. The data cast doubt on models in which the search template resides in VWM or where memory set item representations are moved from LTM through VWM to earlier areas for comparison to visual items.

You look familiar, but I don't care: Lure rejection in hybrid visual and memory search is not based on familiarity.

In "hybrid" search tasks, observers hold multiple possible targets in memory while searching for those targets among distractor items in visual displays. Wolfe (2012) found that, if the target set is held constant over a block of trials, reaction times (RTs) in such tasks were a linear function of the number of items in the visual display and a linear function of the log of the number of items held in memory. However, in such tasks, the targets can become far more familiar than the distractors. Does this "familiarity"- operationalized here as the frequency and recency with which an item has appeared-influence performance in hybrid tasks In Experiment 1, we compared searches where distractors appeared with the same frequency as the targets to searches where all distractors were novel. Distractor familiarity did not have any reliable effect on search. In Experiment 2, most distractors were novel but some critical distractors were as common as the targets while others were 4× more common. Familiar distractors did not produce false alarm errors, though they did slightly increase RTs. In Experiment 3, observers successfully searched for the new, unfamiliar item among distractors that, in many cases, had been seen only once before. We conclude that when the memory set is held constant for many trials, item familiarity alone does not cause observers to mistakenly confuse target with distractors.