Shared Representational Formats for Information Maintained in Working Memory and Information Retrieved from Long-Term Memory.

Current theories propose that the short-term retention of information in working memory (WM) and the recall of information from long-term memory (LTM) are supported by overlapping neural mechanisms in occipital and parietal cortex. However, the extent of the shared representations between WM and LTM is unclear. We designed a spatial memory task that allowed us to directly compare the representations of remembered spatial information in WM and LTM with carefully matched behavioral response precision between tasks. Using multivariate pattern analyses on functional magnetic resonance imaging data, we show that visual memories were represented in a sensory-like code in both memory tasks across retinotopic regions in occipital and parietal cortex. Regions in lateral parietal cortex also encoded remembered locations in both tasks, but in a format that differed from sensory-evoked activity. These results suggest a striking correspondence in the format of representations maintained in WM and retrieved from LTM across occipital and parietal cortex. On the other hand, we also show that activity patterns in nearly all parietal regions, but not occipital regions, contained information that could discriminate between WM and LTM trials. Our data provide new evidence for theories of memory systems and the representation of mnemonic content.

Spatially Guided Distractor Suppression during Visual Search.

Past work has demonstrated that active suppression of salient distractors is a critical part of visual selection. Evidence for goal-driven suppression includes below-baseline visual encoding at the position of salient distractors (Gaspelin and Luck, 2018) and neural signals such as the distractor positivity (Pd) that track how many distractors are presented in a given hemifield (Feldmann-Wüstefeld and Vogel, 2019). One basic question regarding distractor suppression is whether it is inherently spatial or nonspatial in character. Indeed, past work has shown that distractors evoke both spatial (Theeuwes, 1992) and nonspatial forms of interference (Folk and Remington, 1998), motivating a direct examination of whether space is integral to goal-driven distractor suppression. Here, we use behavioral and EEG data from adult humans (male and female) to provide clear evidence for a spatial gradient of suppression surrounding salient singleton distractors. Replicating past work, both reaction time and neural indices of target selection improved monotonically as the distance between target and distractor increased. Importantly, these target selection effects were paralleled by a monotonic decline in the amplitude of the Pd, an electrophysiological index of distractor suppression. Moreover, multivariate analyses revealed spatially selective activity in the θ-band that tracked the position of the target and, critically, revealed suppressed activity at spatial channels centered on distractor positions. Thus, goal-driven selection of relevant over irrelevant information benefits from a spatial gradient of suppression surrounding salient distractors.SIGNIFICANCE STATEMENT Past work has shown that distractor suppression is an important part of goal-driven attentional selection, but has not yet revealed whether suppression is spatially directed. Using behavioral data, event-related potentials (ERPs) of the EEG signal [N2pc and distractor positivity (Pd) component], as well as a multivariate model of EEG data [channel tuning functions (CTF)], we show that suppression-related neural activity increases monotonically as the distance between targets and distractors decreases, and that spatially-selective activity in the θ-band reveals depressed activity in spatial channels that index distractor positions. Thus, we provide robust evidence for spatially-guided distractor suppression, a result that has important implications for models of goal-driven attentional control.

Covert Attention Increases the Gain of Stimulus-Evoked Population Codes.

Covert spatial attention has a variety of effects on the responses of individual neurons. However, relatively little is known about the net effect of these changes on sensory population codes, even though perception ultimately depends on population activity. Here, we measured the EEG in human observers (male and female), and isolated stimulus-evoked activity that was phase-locked to the onset of attended and ignored visual stimuli. Using an encoding model, we reconstructed spatially selective population tuning functions from the pattern of stimulus-evoked activity across the scalp. Our EEG-based approach allowed us to measure very early visually evoked responses occurring ∼100 ms after stimulus onset. In Experiment 1, we found that covert attention increased the amplitude of spatially tuned population responses at this early stage of sensory processing. In Experiment 2, we parametrically varied stimulus contrast to test how this effect scaled with stimulus contrast. We found that the effect of attention on the amplitude of spatially tuned responses increased with stimulus contrast, and was well described by an increase in response gain (i.e., a multiplicative scaling of the population response). Together, our results show that attention increases the gain of spatial population codes during the first wave of visual processing.SIGNIFICANCE STATEMENT We know relatively little about how attention improves population codes, even though perception is thought to critically depend on population activity. In this study, we used an encoding-model approach to test how attention modulates the spatial tuning of stimulus-evoked population responses measured with EEG. We found that attention multiplicatively scales the amplitude of spatially tuned population responses. Furthermore, this effect was present within 100 ms of stimulus onset. Thus, our results show that attention improves spatial population codes by increasing their gain at this early stage of processing.

Is There an Activity-silent Working Memory?

Although storage in working memory (WM) can be tracked via measurements of ongoing neural activity, past work has shown that observers can maintain access to that information despite temporary interruptions of those neural patterns. This observation has been regarded as evidence for a neurally silent form of WM storage. Alternatively, however, unattended information could be retrieved from episodic long-term memory (eLTM) rather than being maintained in WM during the activity-silent period. Here, we tested between these possibilities by examining whether WM performance showed evidence of proactive interference (PI)-a hallmark of retrieval from eLTM-following such interruptions. Participants remembered the colors (Experiments 1-3) or locations (Experiment 4) of serially presented objects. We found PI for set sizes larger than 4, but not for smaller set sizes, suggesting that eLTM may have supported performance when WM capacity was exceeded. Critically, performance with small set sizes remained resistant to PI, even following prolonged interruptions by a challenging distractor task. Thus, we found evidence for PI-resistant memories that were maintained across likely interruptions of storage-related neural activity, an empirical pattern that implies activity-silent storage in WM.

Storage in Visual Working Memory Recruits a Content-Independent Pointer System.

Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.

Item-specific delay activity demonstrates concurrent storage of multiple active neural representations in working memory.

Persistent neural activity that encodes online mental representations plays a central role in working memory (WM). However, there has been debate regarding the number of items that can be concurrently represented in this active neural state, which is often called the "focus of attention." Some models propose a strict single-item limit, such that just 1 item can be neurally active at once while other items are relegated to an activity-silent state. Although past studies have decoded multiple items stored in WM, these studies cannot rule out a switching account in which only a single item is actively represented at a time. Here, we directly tested whether multiple representations can be held concurrently in an active state. We tracked spatial representations in WM using alpha-band (8-12 Hz) activity, which encodes spatial positions held in WM. Human observers remembered 1 or 2 positions over a short delay while we recorded electroencephalography (EEG) data. Using a spatial encoding model, we reconstructed active stimulus-specific representations (channel-tuning functions [CTFs]) from the scalp distribution of alpha-band power. Consistent with past work, we found that the selectivity of spatial CTFs was lower when 2 items were stored than when 1 item was stored. Critically, data-driven simulations revealed that the selectivity of spatial representations in the two-item condition could not be explained by models that propose that only a single item can exist in an active state at once. Thus, our findings demonstrate that multiple items can be concurrently represented in an active neural state.

Controlling the Flow of Distracting Information in Working Memory.

Visual working memory (WM) must maintain relevant information, despite the constant influx of both relevant and irrelevant information. Attentional control mechanisms help determine which of this new information gets access to our capacity-limited WM system. Previous work has treated attentional control as a monolithic process-either distractors capture attention or they are suppressed. Here, we provide evidence that attentional capture may instead be broken down into at least two distinct subcomponent processes: (1) Spatial capture, which refers to when spatial attention shifts towards the location of irrelevant stimuli and (2) item-based capture, which refers to when item-based WM representations of irrelevant stimuli are formed. To dissociate these two subcomponent processes of attentional capture, we utilized a series of electroencephalography components that track WM maintenance (contralateral delay activity), suppression (distractor positivity), item individuation (N2pc), and spatial attention (lateralized alpha power). We show that new, relevant information (i.e., a task-relevant distractor) triggers both spatial and item-based capture. Irrelevant distractors, however, only trigger spatial capture from which ongoing WM representations can recover more easily. This fractionation of attentional capture into distinct subcomponent processes provides a refined framework for understanding how distracting stimuli affect attention and WM.

Covert Spatial Attention Speeds Target Individuation.

Covert spatial attention has long been thought to speed visual processing. Psychophysics studies have shown that target information accrues faster at attended locations than at unattended locations. However, with behavioral evidence alone, it is difficult to determine whether attention speeds visual processing of the target or subsequent postperceptual stages of processing (e.g., converting sensory responses into decision signals). Moreover, although many studies have shown that attention can boost the amplitude of visually evoked neural responses, no robust effect has been observed on the latency of those neural responses. Here, we offer new evidence that may reconcile the neural and behavioral findings. We examined whether covert attention influenced the latency of the N2pc component, an electrophysiological marker of visual selection that has been linked with object individuation-the formation of an object representation that is distinct from the background and from other objects in the scene. To this end, we manipulated whether or not human observers (male and female) covertly attended the location of an impending search target. We found that the target evoked N2pc onset ∼20 ms earlier when the target location was cued than when it was not cued. In a second experiment, we provided a direct replication of this effect, confirming that the effect of attention on N2pc latency is robust. Thus, although attention may not speed the earliest stages of sensory processing, attention does speed the critical transition between raw sensory encoding and the formation of individuated object representations.SIGNIFICANCE STATEMENT Covert spatial attention improves processing at attended locations. Past behavioral studies have shown that information about visual targets accrues faster at attended than at unattended locations. However, it has remained unclear whether attention speeds perceptual analysis or subsequent postperceptual stages of processing. Here, we present robust evidence that attention speeds the N2pc, an electrophysiological signal that indexes the formation of individuated object representations. Our findings show that attention speeds a relatively early stage of perceptual processing while also elucidating the specific perceptual process that is speeded.

Perceptual Grouping Reveals Distinct Roles for Sustained Slow Wave Activity and Alpha Oscillations in Working Memory.

Multiple neural signals have been found to track the number of items stored in working memory (WM). These signals include oscillatory activity in the alpha band and slow-wave components in human EEG, both of which vary with storage loads and predict individual differences in WM capacity. However, recent evidence suggests that these two signals play distinct roles in spatial attention and item-based storage in WM. Here, we examine the hypothesis that sustained negative voltage deflections over parieto-occipital electrodes reflect the number of individuated items in WM, whereas oscillatory activity in the alpha frequency band (8-12 Hz) within the same electrodes tracks the attended positions in the visual display. We measured EEG activity while participants stored the orientation of visual elements that were either grouped by collinearity or not. This grouping manipulation altered the number of individuated items perceived while holding constant the number of locations occupied by visual stimuli. The negative slow wave tracked the number of items stored and was reduced in amplitude in the grouped condition. By contrast, oscillatory activity in the alpha frequency band tracked the number of positions occupied by the memoranda and was unaffected by perceptual grouping. Perceptual grouping, then, reduced the number of individuated representations stored in WM as reflected by the negative slow wave, whereas the location of each element was actively maintained as indicated by alpha power. These findings contribute to the emerging idea that distinct classes of EEG signals work in concert to successfully maintain on-line representations in WM.

Sustained Attention and Spatial Attention Distinctly Influence Long-term Memory Encoding.

Our attention is critically important for what we remember. Prior measures of the relationship between attention and memory, however, have largely treated "attention" as a monolith. Here, across three experiments, we provide evidence for two dissociable aspects of attention that influence encoding into long-term memory. Using spatial cues together with a sensitive continuous report procedure, we find that long-term memory response error is affected by both trial-by-trial fluctuations of sustained attention and prioritization via covert spatial attention. Furthermore, using multivariate analyses of EEG, we track both sustained attention and spatial attention before stimulus onset. Intriguingly, even during moments of low sustained attention, there is no decline in the representation of the spatially attended location, showing that these two aspects of attention have robust but independent effects on long-term memory encoding. Finally, sustained and spatial attention predicted distinct variance in long-term memory performance across individuals. That is, the relationship between attention and long-term memory suggests a composite model, wherein distinct attentional subcomponents influence encoding into long-term memory. These results point toward a taxonomy of the distinct attentional processes that constrain our memories.

Multivariate analysis of EEG activity indexes contingent attentional capture.

An extensive body of work has shown that attentional capture is contingent on the goals of the observer: Capture is strongly reduced or even eliminated when an irrelevant singleton stimulus does not match the target-defining properties (Folk et al., 1992). There has been a long-standing debate on whether attentional capture can be explained by goal-driven and/or stimulus-driven accounts. Here, we shed further light on this matter by using EEG activity (raw EEG and alpha power) to provide a time-resolved index of attentional orienting towards salient stimuli that either matched or did not match target-defining properties. A search display containing the target stimulus was preceded by a spatially uninformative singleton cue that either matched the color of the upcoming target (contingent cues), or that appeared in an irrelevant color (non-contingent cues). Multivariate analysis of raw EEG and alpha power revealed preferential tuning to the location of both contingent and non-contingent cues, with a stronger bias towards contingent than non-contingent cues. The time course of these effects, however, depended on the neural signal. Raw EEG data revealed attentional orienting towards the contingent cue early on in the trial (>156 ms), while alpha power revealed sustained spatial selection in the cued locations at a later moment in the trial (>250 ms). Moreover, while raw EEG showed stronger capture by contingent cues during this early time window, an advantage for contingent cues arose during a later time window in alpha band activity. Thus, our findings suggest that raw EEG activity and alpha-band power tap into distinct neural processes that index separate aspects of covert spatial attention.

Alpha-band Activity Tracks the Zoom Lens of Attention.

Voluntary control over spatial attention has been likened to the operation of a zoom lens, such that processing quality declines as the size of the attended region increases, with a gradient of performance that peaks at the center of the selected area. Although concurrent changes in activity in visual regions suggest that zoom lens adjustments influence perceptual stages of processing, extant work has not distinguished between changes in the spatial selectivity of attention-driven neural activity and baseline shift of activity that can increase mean levels of activity without changes in selectivity. Here, we distinguished between these alternatives by measuring EEG activity in humans to track preparatory changes in alpha activity that indexed the precise topography of attention across the possible target positions. We observed increased spatial selectivity in alpha activity when observers voluntarily directed attention toward a narrower region of space, a pattern that was mirrored in target discrimination accuracy. Thus, alpha activity tracks both the centroid and spatial extent of covert spatial attention before the onset of the target display, lending support to the hypothesis that narrowing the zoom lens of attention shapes the initial encoding of sensory information.

Perturbing Neural Representations of Working Memory with Task-irrelevant Interruption.

Working memory maintains information so that it can be used in complex cognitive tasks. A key challenge for this system is to maintain relevant information in the face of task-irrelevant perturbations. Across two experiments, we investigated the impact of task-irrelevant interruptions on neural representations of working memory. We recorded EEG activity in humans while they performed a working memory task. On a subset of trials, we interrupted participants with salient but task-irrelevant objects. To track the impact of these task-irrelevant interruptions on neural representations of working memory, we measured two well-characterized, temporally sensitive EEG markers that reflect active, prioritized working memory representations: the contralateral delay activity and lateralized alpha power (8-12 Hz). After interruption, we found that contralateral delay activity amplitude momentarily sustained but was gone by the end of the trial. Lateralized alpha power was immediately influenced by the interrupters but recovered by the end of the trial. This suggests that dissociable neural processes contribute to the maintenance of working memory information and that brief irrelevant onsets disrupt two distinct online aspects of working memory. In addition, we found that task expectancy modulated the timing and magnitude of how these two neural signals responded to task-irrelevant interruptions, suggesting that the brain's response to task-irrelevant interruption is shaped by task context.

Alpha-band oscillations track the retrieval of precise spatial representations from long-term memory.

A hallmark of episodic memory is the phenomenon of mentally reexperiencing the details of past events, and a well-established concept is that the neuronal activity that mediates encoding is reinstated at retrieval. Evidence for reinstatement has come from multiple modalities, including functional magnetic resonance imaging and electroencephalography (EEG). These EEG studies have shed light on the time course of reinstatement but have been limited to distinguishing between a few categories. The goal of this work was to use recently developed experimental and technical approaches, namely continuous report tasks and inverted encoding models, to determine which frequencies of oscillatory brain activity support the retrieval of precise spatial memories. In experiment 1, we establish that an inverted encoding model applied to multivariate alpha topography tracks the retrieval of precise spatial memories. In experiment 2, we demonstrate that the frequencies and patterns of multivariate activity at study are similar to the frequencies and patterns observed during retrieval. These findings highlight the broad potential for using encoding models to characterize long-term memory retrieval.NEW & NOTEWORTHY Previous EEG work has shown that category-level information observed during encoding is recapitulated during memory retrieval, but studies with this time-resolved method have not demonstrated the reinstatement of feature-specific patterns of neural activity during retrieval. Here we show that EEG alpha-band activity tracks the retrieval of spatial representations from long-term memory. Moreover, we find considerable overlap between the frequencies and patterns of activity that track spatial memories during initial study and at retrieval.

Dissecting the Neural Focus of Attention Reveals Distinct Processes for Spatial Attention and Object-Based Storage in Visual Working Memory.

Complex cognition relies on both on-line representations in working memory (WM), said to reside in the focus of attention, and passive off-line representations of related information. Here, we dissected the focus of attention by showing that distinct neural signals index the on-line storage of objects and sustained spatial attention. We recorded electroencephalogram (EEG) activity during two tasks that employed identical stimulus displays but varied the relative demands for object storage and spatial attention. We found distinct delay-period signatures for an attention task (which required only spatial attention) and a WM task (which invoked both spatial attention and object storage). Although both tasks required active maintenance of spatial information, only the WM task elicited robust contralateral delay activity that was sensitive to mnemonic load. Thus, we argue that the focus of attention is maintained via a collaboration between distinct processes for covert spatial orienting and object-based storage.

Contralateral Delay Activity Indexes Working Memory Storage, Not the Current Focus of Spatial Attention.

Contralateral delay activity (CDA) has long been argued to track the number of items stored in visual working memory (WM). Recently, however, Berggren and Eimer [Berggren, N., & Eimer, M. Does contralateral delay activity reflect working memory storage or the current focus of spatial attention within visual working memory? Journal of Cognitive Neuroscience, 28, 2003-2020, 2016] proposed the alternative hypothesis that the CDA tracks the current focus of spatial attention instead of WM storage. This hypothesis was based on the finding that, when two successive arrays of memoranda were placed in opposite hemifields, CDA amplitude was primarily determined by the position and number of items in the second display, not the total memory load across both displays. Here, we considered the alternative interpretation that participants dropped the first array from WM when they encoded the second array because the format of the probe display was spatially incompatible with the initial sample display. In this case, even if the CDA indexes active storage rather than spatial attention, CDA activity would be determined by the second array. We tested this idea by directly manipulating the spatial compatibility of sample and probe displays. With spatially incompatible displays, we replicated Berggren and Eimer's findings. However, with spatially compatible displays, we found clear evidence that CDA activity tracked the full storage load across both arrays, in line with a WM storage account of CDA activity. We propose that expectations of display compatibility influenced whether participants viewed the arrays as parts of a single extended event or two independent episodes. Thus, these findings raise interesting new questions about how event boundaries may shape the interplay between passive and active representations of task-relevant information.

Spatially Selective Alpha Oscillations Reveal Moment-by-Moment Trade-offs between Working Memory and Attention.

Current theories assume a functional role for covert attention in the maintenance of spatial information in working memory. Consistent with this view, both the locus of attention and positions stored in working memory can be decoded based on the topography of oscillatory alpha-band (8-12 Hz) activity on the scalp. Thus far, however, alpha modulation has been studied in isolation for covert attention and working memory tasks. Here, we applied an inverted spatial encoding model in combination with EEG to study the temporal dynamics of spatially specific alpha activity during a task that required observers to visually select a target location while maintaining another independently varying location in working memory. During the memory delay period, alpha-based spatial tuning functions shifted from the position stored in working memory to the covertly attended position and back again after the attention task was completed. The findings provide further evidence for a common oscillatory mechanism in both the selection and the maintenance of relevant spatial visual information and demonstrate the dynamic trade-off in prioritization between two spatial tasks.

Feature-Selective Attentional Modulations in Human Frontoparietal Cortex.

UNLABELLED: Control over visual selection has long been framed in terms of a dichotomy between "source" and "site," where top-down feedback signals originating in frontoparietal cortical areas modulate or bias sensory processing in posterior visual areas. This distinction is motivated in part by observations that frontoparietal cortical areas encode task-level variables (e.g., what stimulus is currently relevant or what motor outputs are appropriate), while posterior sensory areas encode continuous or analog feature representations. Here, we present evidence that challenges this distinction. We used fMRI, a roving searchlight analysis, and an inverted encoding model to examine representations of an elementary feature property (orientation) across the entire human cortical sheet while participants attended either the orientation or luminance of a peripheral grating. Orientation-selective representations were present in a multitude of visual, parietal, and prefrontal cortical areas, including portions of the medial occipital cortex, the lateral parietal cortex, and the superior precentral sulcus (thought to contain the human homolog of the macaque frontal eye fields). Additionally, representations in many-but not all-of these regions were stronger when participants were instructed to attend orientation relative to luminance. Collectively, these findings challenge models that posit a strict segregation between sources and sites of attentional control on the basis of representational properties by demonstrating that simple feature values are encoded by cortical regions throughout the visual processing hierarchy, and that representations in many of these areas are modulated by attention. SIGNIFICANCE STATEMENT: Influential models of visual attention posit a distinction between top-down control and bottom-up sensory processing networks. These models are motivated in part by demonstrations showing that frontoparietal cortical areas associated with top-down control represent abstract or categorical stimulus information, while visual areas encode parametric feature information. Here, we show that multivariate activity in human visual, parietal, and frontal cortical areas encode representations of a simple feature property (orientation). Moreover, representations in several (though not all) of these areas were modulated by feature-based attention in a similar fashion. These results provide an important challenge to models that posit dissociable top-down control and sensory processing networks on the basis of representational properties.

Alpha-Band Oscillations Enable Spatially and Temporally Resolved Tracking of Covert Spatial Attention.

Covert spatial attention is essential for humans' ability to direct limited processing resources to the relevant aspects of visual scenes. A growing body of evidence suggests that rhythmic neural activity in the alpha frequency band (8-12 Hz) tracks the spatial locus of covert attention, which suggests that alpha activity is integral to spatial attention. However, extant work has not provided a compelling test of another key prediction: that alpha activity tracks the temporal dynamics of covert spatial orienting. In the current study, we examined the time course of spatially specific alpha activity after central cues and during visual search. Critically, the time course of this activity tracked trial-by-trial variations in orienting latency during visual search. These findings provide important new evidence for the link between rhythmic brain activity and covert spatial attention, and they highlight a powerful approach for tracking the spatial and temporal dynamics of this core cognitive process.

Clear evidence for item limits in visual working memory.

There is a consensus that visual working memory (WM) resources are sharply limited, but debate persists regarding the simple question of whether there is a limit to the total number of items that can be stored concurrently. Zhang and Luck (2008) advanced this debate with an analytic procedure that provided strong evidence for random guessing responses, but their findings can also be described by models that deny guessing while asserting a high prevalence of low precision memories. Here, we used a whole report memory procedure in which subjects reported all items in each trial and indicated whether they were guessing with each response. Critically, this procedure allowed us to measure memory performance for all items in each trial. When subjects were asked to remember 6 items, the response error distributions for about 3 out of the 6 items were best fit by a parameter-free guessing model (i.e. a uniform distribution). In addition, subjects' self-reports of guessing precisely tracked the guessing rate estimated with a mixture model. Control experiments determined that guessing behavior was not due to output interference, and that there was still a high prevalence of guessing when subjects were instructed not to guess. Our novel approach yielded evidence that guesses, not low-precision representations, best explain limitations in working memory. These guesses also corroborate a capacity-limited working memory system - we found evidence that subjects are able to report non-zero information for only 3-4 items. Thus, WM capacity is constrained by an item limit that precludes the storage of more than 3-4 individuated feature values.