Effects of Context Changes on Memory Reactivation.

While the influence of context on long-term memory (LTM) is well documented, its effects on the interaction between working memory (WM) and LTM remain less understood. In this study, we explored these interactions using a delayed match-to-sample task, where participants (6 males, 16 females) encountered the same target object across six consecutive trials, facilitating the transition from WM to LTM. During half of these target repetitions, the background color changed. We measured the WM storage of the target using the contralateral delay activity in electroencephalography. Our results reveal that task-irrelevant context changes trigger the reactivation of long-term memories in WM. This reactivation may be attributed to content-context binding in WM and hippocampal pattern separation.

Decoding load or selection in visuospatial working memory?

Flexible updating of information in Visual Working Memory (VWM) is crucial to deal with its limited capacity. Previous research has shown that the removal of no longer relevant information takes some time to complete. Here, we sought to study the time course of such removal by tracking the accompanying drop in load through behavioral and neurophysiological measures. In the first experimental session, participants completed a visuospatial retro-cue task in which the Cue-Target Interval (CTI) was manipulated. The performance revealed that it takes about half a second to make full use of the retro-cue. In a second session, we sought to study the dynamics of load-related electroencephalographic (EEG) signals to track the removal of information. We applied Multivariate Pattern Analysis (MVPA) to EEG data from the same task. Right after encoding, results replicated previous research using MVPA to decode load. However, especially after the retro-cue, results suggested that classifiers were mainly sensitive to a selection component, and not so much to load per se. Additionally, visual cue variations, as well as eye movements that accompany load manipulations can also contribute to decoding. These findings advise caution when using MVPA to decode VWM load, as classifiers may be sensitive to confounding operations.

A matter of availability: sharper tuning for memorized than for perceived stimulus features.

Our visual environment is relatively stable over time. An optimized visual system could capitalize on this by devoting less representational resources to objects that are physically present. The vividness of subjective experience, however, suggests that externally available (perceived) information is more strongly represented in neural signals than memorized information. To distinguish between these opposing predictions, we use EEG multivariate pattern analysis to quantify the representational strength of task-relevant features in anticipation of a change-detection task. Perceptual availability was manipulated between experimental blocks by either keeping the stimulus available on the screen during a 2-s delay period (perception) or removing it shortly after its initial presentation (memory). We find that task-relevant (attended) memorized features are more strongly represented than irrelevant (unattended) features. More importantly, we find that task-relevant features evoke significantly weaker representations when they are perceptually available compared with when they are unavailable. These findings demonstrate that, contrary to what subjective experience suggests, vividly perceived stimuli elicit weaker neural representations (in terms of detectable multivariate information) than the same stimuli maintained in visual working memory. We hypothesize that an efficient visual system spends little of its limited resources on the internal representation of information that is externally available anyway.

Visual Working Memory Adapts to the Nature of Anticipated Interference.

Visual working memory has been proven to be relatively robust against interference. However, little is known on whether such robust coding is obligatory, or can be flexibly recruited depending on its expected usefulness. To address this, participants remembered both the color and orientation of a grating. During the maintenance, we inserted a secondary color/orientation memory task, interfering with the primary task. Crucially, we varied the expectations of the type of interference by varying the probability of the two types of intervening task. Behavioral data indicate that to-be-remembered features for which interference is expected are bolstered, whereas to-be-remembered features for which no interference is expected are left vulnerable. This was further supported by fMRI data obtained from visual cortex. In conclusion, the flexibility of visual working memory allows it to strengthen memories for which it anticipates the highest risk of interference.

Neural Repetition Suppression Modulates Time Perception: Evidence From Electrophysiology and Pupillometry.

Human time perception is malleable and subject to many biases. For example, it has repeatedly been shown that stimuli that are physically intense or that are unexpected seem to last longer. Two competing hypotheses have been proposed to account for such biases: One states that these temporal illusions are the result of increased levels of arousal that speeds up neural clock dynamics, whereas the alternative "magnitude coding" account states that the magnitude of sensory responses causally modulates perceived durations. Common experimental paradigms used to study temporal biases cannot dissociate between these accounts, as arousal and sensory magnitude covary and modulate each other. Here, we present two temporal discrimination experiments where two flashing stimuli demarcated the start and end of a to-be-timed interval. These stimuli could be either in the same or a different location, which led to different sensory responses because of neural repetition suppression. Crucially, changes and repetitions were fully predictable, which allowed us to explore effects of sensory response magnitude without changes in arousal or surprise. Intervals with changing markers were perceived as lasting longer than those with repeating markers. We measured EEG (Experiment 1) and pupil size (Experiment 2) and found that temporal perception was related to changes in ERPs (P2) and pupil constriction, both of which have been related to responses in the sensory cortex. Conversely, correlates of surprise and arousal (P3 amplitude and pupil dilation) were unaffected by stimulus repetitions and changes. These results demonstrate, for the first time, that sensory magnitude affects time perception even under constant levels of arousal.

Memory for Stimulus Duration Is Not Bound to Spatial Information.

Different theories have been proposed to explain how the human brain derives an accurate sense of time. One specific class of theories, intrinsic clock theories, postulate that temporal information of a stimulus is represented much like other features such as color and location, bound together to form a coherent percept. Here, we explored to what extent this holds for temporal information after it has been perceived and is held in working memory for subsequent comparison. We recorded EEG of participants who were asked to time stimuli at lateral positions of the screen followed by comparison stimuli presented in the center. Using well-established markers of working memory maintenance, we investigated whether the usage of temporal information evoked neural signatures that were indicative of the location where the stimuli had been presented, both during maintenance and during comparison. Behavior and neural measures including the contralateral delay activity, lateralized alpha suppression, and decoding analyses through time all supported the same conclusion: The representation of location was strongly involved during perception of temporal information, but when temporal information was to be used for comparison, it no longer showed a relation to spatial information. These results support a model where the initial perception of a stimulus involves intrinsic computations, but that this information is subsequently translated to a stimulus-independent format to be used to further guide behavior.

Flexible Working Memory Through Selective Gating and Attentional Tagging.

Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control.

The dynamics of saliency-driven and goal-driven visual selection as a function of eccentricity.

Both saliency and goal information are important factors in driving visual selection. Saliency-driven selection occurs primarily in early responses, whereas goal-driven selection happens predominantly in later responses. Here, we investigated how eccentricity affects the time courses of saliency-driven and goal-driven visual selection. In three experiments, we asked people to make a speeded eye movement toward a predefined target singleton which was simultaneously presented with a non-target singleton in a background of multiple homogeneously oriented other items. The target singleton could be either more or less salient than the non-target singleton. Both singletons were presented at one of three eccentricities (i.e., near, middle, or far). The results showed that, even though eccentricity had only little effect on overall selection performance, the underlying time courses of saliency-driven and goal-driven selection altered such that saliency effects became protracted and relevance effects became delayed for far eccentricity conditions. The protracted saliency effect was shown to be modulated by expectations as induced by the preceding trial. The results demonstrate the importance of incorporating both time and eccentricity as factors in models of visual selection.

Individual differences in crowding predict visual search performance.

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

Beta and Theta Oscillations Differentially Support Free Versus Forced Control over Multiple-Target Search.

Many important situations require human observers to simultaneously search for more than one object. Despite a long history of research into visual search, the behavioral and neural mechanisms associated with multiple-target search are poorly understood. Here we test the novel theory that the efficiency of looking for multiple targets critically depends on the mode of cognitive control the environment affords to the observer. We used an innovative combination of electroencephalogram (EEG) and eye tracking while participants searched for two targets, within two different contexts: either both targets were present in the search display and observers were free to prioritize either one of them, thus enabling proactive control over selection; or only one of the two targets would be present in each search display, which requires reactive control to reconfigure selection when the wrong target has been prioritized. During proactive control, both univariate and multivariate signals of beta-band (15-35 Hz) power suppression before display onset predicted switches between target selections. This signal originated over midfrontal and sensorimotor regions and has previously been associated with endogenous state changes. In contrast, imposed target selections requiring reactive control elicited prefrontal power enhancements in the delta/theta band (2-8 Hz), but only after display onset. This signal predicted individual differences in associated oculomotor switch costs, reflecting reactive reconfiguration of target selection. The results provide compelling evidence that multiple target representations are differentially prioritized during visual search, and for the first time reveal distinct neural mechanisms underlying proactive and reactive control over multiple-target search.SIGNIFICANCE STATEMENT Searching for more than one object in complex visual scenes can be detrimental for search performance. Although perhaps annoying in daily life, this can have severe consequences in professional settings such as medical and security screening. Previous research has not yet resolved whether multiple-target search involves changing priorities in what people attend to, and how such changes are controlled. We approached these questions by concurrently measuring cortical activity and eye movements using EEG and eye tracking while observers searched for multiple possible targets. Our findings provide the first unequivocal support for the existence of two modes of control during multiple-target search, which are expressed in qualitatively distinct time-frequency signatures of the EEG both before and after visual selection.

Perceptual integration without conscious access.

The visual system has the remarkable ability to integrate fragmentary visual input into a perceptually organized collection of surfaces and objects, a process we refer to as perceptual integration. Despite a long tradition of perception research, it is not known whether access to consciousness is required to complete perceptual integration. To investigate this question, we manipulated access to consciousness using the attentional blink. We show that, behaviorally, the attentional blink impairs conscious decisions about the presence of integrated surface structure from fragmented input. However, despite conscious access being impaired, the ability to decode the presence of integrated percepts remains intact, as shown through multivariate classification analyses of electroencephalogram (EEG) data. In contrast, when disrupting perception through masking, decisions about integrated percepts and decoding of integrated percepts are impaired in tandem, while leaving feedforward representations intact. Together, these data show that access consciousness and perceptual integration can be dissociated.

Oscillatory Mechanisms of Preparing for Visual Distraction.

Evidence shows that observers preactivate a target representation in preparation of a visual selection task. In this study, we addressed the question if and how preparing to ignore an anticipated distractor differs from preparing for an anticipated target. We measured EEG while participants memorized a laterally presented color, which was cued to be either a target or a distractor in two subsequent visual search tasks. Decoding the location of items in the search display from EOG channels revealed that, initially, the anticipated distractor attracted attention and could only be ignored later during the trial. This suggests that distractors could not be suppressed in advance but were represented in an active, attention-guiding format. Consistent with this, lateralized posterior alpha power did not dissociate between target and distractor templates during the delay periods, suggesting similar encoding and maintenance. However, distractor preparation did lead to relatively enhanced nonlateralized posterior alpha power, which appeared to gate sensory processing at search display onset to prevent attentional capture in general. Finally, anticipating distractors also led to enhanced midfrontal theta power during the delay period, a signal that was predictive of how strongly both target and distractor were represented in the search display. Together, our results speak against a distractor-specific advance inhibitory template, thus contrary to the preactivation of specific target templates. Rather, we demonstrate a general selection suppression mechanism, which serves to prevent initial involuntary capture by anticipated distracting input.

Decoding the status of working memory representations in preparation of visual selection.

Daily life is filled with sequences of multiple tasks, each with their own relevant perceptual input. Working memory needs to dissociate representations that drive attention towards currently relevant information from prospective representations that are needed for future tasks, but that until then should be prevented from guiding attention. Yet, little is known about how the brain initiates and controls such sequential prioritization of selection. In the current study we recorded EEG while subjects remembered a color as the target template for one of two sequential search tasks, thus making it either currently relevant (when it was the target for the first search) or prospectively relevant (when it was the target for the second search) prior to the task sequence. Using time-frequency specific linear classifiers, we were able to predict the priority status (current versus prospective) of the memory representation from multivariate patterns of delta (2-4 Hz) and non-lateralized alpha power (8-14 Hz) during both delay periods. The delta band was only transiently involved when initializing the priority status at the start of the first delay, or when switching priority during the second delay, which we interpret as reflecting the momentary top-down control over prioritization. In contrast, alpha power decoding was based on a more stable pattern of activity that generalized across time both within and between delay periods, which we interpret as reflecting a difference in the prioritized memory representations themselves. Taken together, we reveal the involvement of a complex, distributed and dynamic spatiotemporal landscape of frequency-specific oscillatory activity in controlling prioritization of information within working memory.

Frontal cortex differentiates between free and imposed target selection in multiple-target search.

Cognitive control can involve proactive (preparatory) and reactive (corrective) mechanisms. Using a gaze-contingent eye tracking paradigm combined with fMRI, we investigated the involvement of these different modes of control and their underlying neural networks, when switching between different targets in multiple-target search. Participants simultaneously searched for two possible targets presented among distractors, and selected one of them. In one condition, only one of the targets was available in each display, so that the choice was imposed, and reactive control would be required. In the other condition, both targets were present, giving observers free choice over target selection, and allowing for proactive control. Switch costs emerged only when targets were imposed and not when target selection was free. We found differential levels of activity in the frontoparietal control network depending on whether target switches were free or imposed. Furthermore, we observed core regions of the default mode network to be active during target repetitions, indicating reduced control on these trials. Free and imposed switches jointly activated parietal and posterior frontal cortices, while free switches additionally activated anterior frontal cortices. These findings highlight unique contributions of proactive and reactive control during visual search.

Priority Switches in Visual Working Memory are Supported by Frontal Delta and Posterior Alpha Interactions.

Visual working memory (VWM) distinguishes between representations relevant for imminent versus future perceptual goals. We investigated how the brain sequentially prioritizes visual working memory representations that serve consecutive tasks. Observers remembered two targets for a sequence of two visual search tasks, thus making one target currently relevant, and the other prospectively relevant. We show that during the retention interval prior to the first search, lateralized parieto-occipital EEG alpha (8-14 Hz) suppression is stronger for current compared with prospective search targets. Crucially, between the first and second search task, this difference in posterior alpha lateralization reverses, reflecting the change in priority states of the two target representations. Connectivity analyses indicate that this switch in posterior alpha lateralization is driven by frontal delta/low-theta (2-6 Hz) activity. Moreover, this frontal low-frequency signal also predicts task performance after the switch. We thus obtained evidence for large-scale network interactions underlying the flexible shifting between the priority states of multiple memory representations in VWM.

Posterior α EEG Dynamics Dissociate Current from Future Goals in Working Memory-Guided Visual Search.

Current models of visual search assume that search is guided by an active visual working memory representation of what we are currently looking for. This attentional template for currently relevant stimuli can be dissociated from accessory memory representations that are only needed prospectively, for a future task, and that should be prevented from guiding current attention. However, it remains unclear what electrophysiological mechanisms dissociate currently relevant (serving upcoming selection) from prospectively relevant memories (serving future selection). We measured EEG of 20 human subjects while they performed two consecutive visual search tasks. Before the search tasks, a cue instructed observers which item to look for first (current template) and which second (prospective template). During the delay leading up to the first search display, we found clear suppression of α band (8-14 Hz) activity in regions contralateral to remembered items, comprising both local power and interregional phase synchronization within a posterior parietal network. Importantly, these lateralization effects were stronger when the memory item was currently relevant (i.e., for the first search) compared with when it was prospectively relevant (i.e., for the second search), consistent with current templates being prioritized over future templates. In contrast, event-related potential analysis revealed that the contralateral delay activity was similar for all conditions, suggesting no difference in storage. Together, these findings support the idea that posterior α oscillations represent a state of increased processing or excitability in task-relevant cortical regions, and reflect enhanced cortical prioritization of memory representations that serve as a current selection filter.SIGNIFICANCE STATEMENT Our days are filled with looking for relevant objects while ignoring irrelevant visual information. Such visual search activity is thought to be driven by current goals activated in working memory. However, working memory not only serves current goals, but also future goals, with differential impact upon visual selection. Little is known about how the brain differentiates between current and future goals. Here we show, for the first time, that modulations of brain oscillations in the EEG α frequency band in posterior cortex can dissociate current from future search goals in working memory. Moreover, the dynamics of these oscillations uncover how we flexibly switch focus between memory representations. Together, we reveal how the brain assigns priority for selection.

Spatially Selective Alpha Oscillations Reveal Moment-by-Moment Trade-offs between Working Memory and Attention.

Current theories assume a functional role for covert attention in the maintenance of spatial information in working memory. Consistent with this view, both the locus of attention and positions stored in working memory can be decoded based on the topography of oscillatory alpha-band (8-12 Hz) activity on the scalp. Thus far, however, alpha modulation has been studied in isolation for covert attention and working memory tasks. Here, we applied an inverted spatial encoding model in combination with EEG to study the temporal dynamics of spatially specific alpha activity during a task that required observers to visually select a target location while maintaining another independently varying location in working memory. During the memory delay period, alpha-based spatial tuning functions shifted from the position stored in working memory to the covertly attended position and back again after the attention task was completed. The findings provide further evidence for a common oscillatory mechanism in both the selection and the maintenance of relevant spatial visual information and demonstrate the dynamic trade-off in prioritization between two spatial tasks.

Local and interregional alpha EEG dynamics dissociate between memory for search and memory for recognition.

Attention during visual search is thought to be guided by an active visual working memory (VWM) representation of the search target. We tested the hypothesis that a VWM representation used for searching a target among competing information (a "search template") is distinct from VWM representations used for simple recognition tasks, without competition. We analyzed EEG from 20 human participants while they performed three different VWM-based visual detection tasks. All tasks started with identical lateralized VWM cues, but differed with respect to the presence and nature of competing distractors during the target display at test, where participants performed a simple recognition task without distractors, or visual search in pop-out (distinct) and serial (non-distinct) search displays. Performance was worst for non-distinct search, and best for simple recognition. During the one second delay period between cue and test, we observed robust suppression of EEG dynamics in the alpha (8-14Hz) band over parieto-occipital sites contralateral to the relevant VWM item, both in terms of local power as well as interregional phase synchrony within a posterior-parietal network. Importantly, these lateralization dynamics were more strongly expressed prior to search compared to simple recognition. Furthermore, before the VWM cue, alpha phase synchrony between prefrontal and mid-posterior-parietal sites was strongest for non-distinct search, reflecting enhanced anticipatory control prior to VWM encoding. Directional connectivity analyses confirmed this effect to be in an anterior-to-posterior direction. Together, these results provide evidence for frontally mediated top-down control of VWM in preparation of visual search.

Rapid top-down control over template-guided attention shifts to multiple objects.

Previous research has shown that when observers search for targets defined by a particular colour, attention can be directed rapidly and independently to two target objects that appear in close temporal proximity. We investigated how such rapid attention shifts are modulated by task instructions to selectively attend versus ignore one of these objects. Two search displays that both contained a colour-defined target and a distractor in a different colour were presented in rapid succession, with a stimulus onset asynchrony (SOA) of 100ms. In different blocks, participants were instructed to attend and respond to target-colour objects in the first display and to ignore these objects in the second display, or vice versa. N2pc components were measured to track the allocation of spatial attention to target-colour objects in these two displays. When participants responded to the second display, irrelevant target-colour objects in the first display still triggered N2pc components, demonstrating task-set contingent attentional capture while a feature-specific target template is active. Critically, when participants responded to the first display instead, no N2pc was elicited by target-colour items in the second display, indicating that they no longer rapidly captured attention. However, these items still elicited a longer-latency contralateral negativity (SPCN component), suggesting that attention was oriented towards template-matching objects in working memory. This dissociation between N2pc and SPCN components shows that rapid attentional capture and subsequent attentional selection processes within working memory can be independent. We suggest that early attentional orienting mechanisms can be inhibited when task-set matching objects are no longer task-relevant, and that this type of inhibitory control is a rapid but transient process.

Lack of Free Choice Reveals the Cost of Having to Search for More Than One Object.

It is debated whether people can actively search for more than one object or whether this results in switch costs. Using a gaze-contingent eye-tracking paradigm, we revealed a crucial role for cognitive control in multiple-target search. We instructed participants to simultaneously search for two target objects presented among distractors. In one condition, both targets were available, which gave the observer free choice of what to search for and allowed for proactive control. In the other condition, only one of the two targets was available, so that the choice was imposed, and a reactive mechanism would be required. No switch costs emerged when target choice was free, but switch costs emerged reliably when targets were imposed. Bridging contradictory findings, the results are consistent with models of visual selection in which only one attentional template actively drives selection and in which the efficiency of switching targets depends on the type of cognitive control allowed for by the environment.