Encoded and updated spatial working memories share a common representational format in alpha activity.

Working memory (WM) flexibly updates information to adapt to the dynamic environment. Here, we used alpha-band activity in the EEG to reconstruct the content of dynamic WM updates and compared this representational format to static WM content. An inverted encoding model using alpha activity precisely tracked both the initially encoded position and the updated position following an auditory cue signaling mental updating. The timing of the update, as tracked in the EEG, correlated with reaction times and saccade latency. Finally, cross-training analyses revealed a robust generalization of alpha-band reconstruction of WM contents before and after updating. These findings demonstrate that alpha activity tracks the dynamic updates to spatial WM and that the format of this activity is preserved across the encoded and updated representations. Thus, our results highlight a new approach for measuring updates to WM and show common representational formats during dynamic mental updating and static storage.

Object-based encoding constrains storage in visual working memory.

The fundamental unit of visual working memory (WM) has been debated for decades. WM could be object-based, such that capacity is set by the number of individuated objects, or feature-based, such that capacity is determined by the total number of feature values stored. The present work examined whether object- or feature-based models would best explain how multifeature objects (i.e., color/orientation or color/shape) are encoded into visual WM. If maximum capacity is limited by the number of individuated objects, then above-chance performance should be restricted to the same number of items as in a single-feature condition. By contrast, if the capacity is determined by independent storage resources for distinct features-without respect to the objects that contain those features-then successful storage of feature values could be distributed across a larger number of objects than when only a single feature is relevant. We conducted four experiments using a whole-report task in which subjects reported both features from every item in a six-item array. The crucial finding was that above-chance recall-for both single- and multifeatured objects-was restricted to the first three or four responses, while the later responses were best modeled as guesses. Thus, whole-report with multifeature objects reveals a distribution of recalled features that indicates an object-based limit on WM capacity. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Distinguishing guesses from fuzzy memories: Further evidence for item limits in visual working memory.

There is consistent debate over whether capacity in working memory (WM) is subject to an item limit, or whether an unlimited number of items can be held in this online memory system. The item limit hypothesis clearly predicts guessing responses when capacity is exceeded, and proponents of this view have highlighted evidence for guessing in visual working memory tasks. Nevertheless, various models that deny item limits can explain the same empirical patterns by asserting extremely low fidelity representations that cannot be distinguished from guesses. To address this ambiguity, we employed a task for which guess responses elicited a qualitatively distinct pattern from low fidelity memories. Inspired by work from Rouder et al. (2014), we employed an orientation WM task that required subjects to recall the precise orientation of each of six memoranda presented 1 s earlier. The orientation stimuli were created by rotating the position of a "clock hand" inside a circular region that was demarcated by four colored quadrants. Critically, when observers guess with these stimuli, the distribution of responses is biased towards the center of these quadrants, creating a "banded" pattern that cannot be explained by a low precision memory. We confirmed the presence of this guessing pattern using formal model comparisons, and we show that the prevalence of this pattern matches observers' own reports of when they thought they were guessing. Thus, these findings provide further evidence for guessing behaviors predicted by item limit models of WM capacity.

Change localization: A highly reliable and sensitive measure of capacity in visual working memory.

The change detection paradigm has been a widely used approach for measuring capacity in visual working memory (WM). In this task, subjects see an array of visual items, followed by a short blank delay and a single test item. Their task is to indicate whether that test item changed relative to the item in the sample array. This task provides reliable measurements of WM capacity that exhibit robust correlations with many outcome variables of interest. Here, we offer a new variant of this task that we call "change localization." This task is closely modeled after the change detection task described above, except that the test array contains the same number of items as the sample array, and one item has always changed in each trial. The subject's task is to select the changed item in the test array. Using both color and shape stimuli, scores in the change localization task were highly correlated with those in the change detection task, suggesting that change localization taps into the same variance in WM ability. Moreover, the change localization task was far more reliable than change detection, such that only half the number of trials were required to achieve robust reliability. To further validate the approach, we replicated known effects from the literature, demonstrating that they could be detected with far fewer trials than with change detection. Thus, change localization provides a highly reliable and sensitive approach for measuring visual working memory capacity.

Pupillometry signatures of sustained attention and working memory.

There exists an intricate relationship between attention and working memory. Recent work has further established that attention and working memory fluctuate synchronously, by tightly interleaving sustained attention and working memory tasks. This work has raised many open questions about physiological signatures underlying these behavioral fluctuations. Across two experiments, we explore pupil dynamics using real-time triggering in conjunction with an interleaved sustained attention and working memory task. In Experiment 1, we use behavioral real-time triggering and replicate recent findings from our lab (deBettencourt et al., 2019) that sustained attention fluctuates concurrently with the number of items maintained in working memory. Furthermore, highly attentive moments, detected via behavior, also exhibited larger pupil sizes. In Experiment 2, we develop a novel real-time pupil-triggering technique to track pupil size fluctuations in real time and trigger working memory probes. We show that this pupil triggering procedure reveals differences in sustained attention, as indexed by response time. These experiments reflect methodological advances in real-time triggering and further disentangle the relationship among general arousal, sustained attention, and working memory.

Is There an Activity-silent Working Memory?

Although storage in working memory (WM) can be tracked via measurements of ongoing neural activity, past work has shown that observers can maintain access to that information despite temporary interruptions of those neural patterns. This observation has been regarded as evidence for a neurally silent form of WM storage. Alternatively, however, unattended information could be retrieved from episodic long-term memory (eLTM) rather than being maintained in WM during the activity-silent period. Here, we tested between these possibilities by examining whether WM performance showed evidence of proactive interference (PI)-a hallmark of retrieval from eLTM-following such interruptions. Participants remembered the colors (Experiments 1-3) or locations (Experiment 4) of serially presented objects. We found PI for set sizes larger than 4, but not for smaller set sizes, suggesting that eLTM may have supported performance when WM capacity was exceeded. Critically, performance with small set sizes remained resistant to PI, even following prolonged interruptions by a challenging distractor task. Thus, we found evidence for PI-resistant memories that were maintained across likely interruptions of storage-related neural activity, an empirical pattern that implies activity-silent storage in WM.

Storage in Visual Working Memory Recruits a Content-Independent Pointer System.

Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.

Shared Representational Formats for Information Maintained in Working Memory and Information Retrieved from Long-Term Memory.

Current theories propose that the short-term retention of information in working memory (WM) and the recall of information from long-term memory (LTM) are supported by overlapping neural mechanisms in occipital and parietal cortex. However, the extent of the shared representations between WM and LTM is unclear. We designed a spatial memory task that allowed us to directly compare the representations of remembered spatial information in WM and LTM with carefully matched behavioral response precision between tasks. Using multivariate pattern analyses on functional magnetic resonance imaging data, we show that visual memories were represented in a sensory-like code in both memory tasks across retinotopic regions in occipital and parietal cortex. Regions in lateral parietal cortex also encoded remembered locations in both tasks, but in a format that differed from sensory-evoked activity. These results suggest a striking correspondence in the format of representations maintained in WM and retrieved from LTM across occipital and parietal cortex. On the other hand, we also show that activity patterns in nearly all parietal regions, but not occipital regions, contained information that could discriminate between WM and LTM trials. Our data provide new evidence for theories of memory systems and the representation of mnemonic content.

Inter-electrode correlations measured with EEG predict individual differences in cognitive ability.

Human brains share a broadly similar functional organization with consequential individual variation. This duality in brain function has primarily been observed when using techniques that consider the spatial organization of the brain, such as MRI. Here, we ask whether these common and unique signals of cognition are also present in temporally sensitive but spatially insensitive neural signals. To address this question, we compiled electroencephalogram (EEG) data from individuals of both sexes while they performed multiple working memory tasks at two different data-collection sites (n = 171 and 165). Results revealed that trial-averaged EEG activity exhibited inter-electrode correlations that were stable within individuals and unique across individuals. Furthermore, models based on these inter-electrode correlations generalized across datasets to predict participants' working memory capacity and general fluid intelligence. Thus, inter-electrode correlation patterns measured with EEG provide a signature of working memory and fluid intelligence in humans and a new framework for characterizing individual differences in cognitive abilities.

Sustained Attention and Spatial Attention Distinctly Influence Long-term Memory Encoding.

Our attention is critically important for what we remember. Prior measures of the relationship between attention and memory, however, have largely treated "attention" as a monolith. Here, across three experiments, we provide evidence for two dissociable aspects of attention that influence encoding into long-term memory. Using spatial cues together with a sensitive continuous report procedure, we find that long-term memory response error is affected by both trial-by-trial fluctuations of sustained attention and prioritization via covert spatial attention. Furthermore, using multivariate analyses of EEG, we track both sustained attention and spatial attention before stimulus onset. Intriguingly, even during moments of low sustained attention, there is no decline in the representation of the spatially attended location, showing that these two aspects of attention have robust but independent effects on long-term memory encoding. Finally, sustained and spatial attention predicted distinct variance in long-term memory performance across individuals. That is, the relationship between attention and long-term memory suggests a composite model, wherein distinct attentional subcomponents influence encoding into long-term memory. These results point toward a taxonomy of the distinct attentional processes that constrain our memories.

Perceptual Grouping Reveals Distinct Roles for Sustained Slow Wave Activity and Alpha Oscillations in Working Memory.

Multiple neural signals have been found to track the number of items stored in working memory (WM). These signals include oscillatory activity in the alpha band and slow-wave components in human EEG, both of which vary with storage loads and predict individual differences in WM capacity. However, recent evidence suggests that these two signals play distinct roles in spatial attention and item-based storage in WM. Here, we examine the hypothesis that sustained negative voltage deflections over parieto-occipital electrodes reflect the number of individuated items in WM, whereas oscillatory activity in the alpha frequency band (8-12 Hz) within the same electrodes tracks the attended positions in the visual display. We measured EEG activity while participants stored the orientation of visual elements that were either grouped by collinearity or not. This grouping manipulation altered the number of individuated items perceived while holding constant the number of locations occupied by visual stimuli. The negative slow wave tracked the number of items stored and was reduced in amplitude in the grouped condition. By contrast, oscillatory activity in the alpha frequency band tracked the number of positions occupied by the memoranda and was unaffected by perceptual grouping. Perceptual grouping, then, reduced the number of individuated representations stored in WM as reflected by the negative slow wave, whereas the location of each element was actively maintained as indicated by alpha power. These findings contribute to the emerging idea that distinct classes of EEG signals work in concert to successfully maintain on-line representations in WM.

Controlling the Flow of Distracting Information in Working Memory.

Visual working memory (WM) must maintain relevant information, despite the constant influx of both relevant and irrelevant information. Attentional control mechanisms help determine which of this new information gets access to our capacity-limited WM system. Previous work has treated attentional control as a monolithic process-either distractors capture attention or they are suppressed. Here, we provide evidence that attentional capture may instead be broken down into at least two distinct subcomponent processes: (1) Spatial capture, which refers to when spatial attention shifts towards the location of irrelevant stimuli and (2) item-based capture, which refers to when item-based WM representations of irrelevant stimuli are formed. To dissociate these two subcomponent processes of attentional capture, we utilized a series of electroencephalography components that track WM maintenance (contralateral delay activity), suppression (distractor positivity), item individuation (N2pc), and spatial attention (lateralized alpha power). We show that new, relevant information (i.e., a task-relevant distractor) triggers both spatial and item-based capture. Irrelevant distractors, however, only trigger spatial capture from which ongoing WM representations can recover more easily. This fractionation of attentional capture into distinct subcomponent processes provides a refined framework for understanding how distracting stimuli affect attention and WM.

Spatially Guided Distractor Suppression during Visual Search.

Past work has demonstrated that active suppression of salient distractors is a critical part of visual selection. Evidence for goal-driven suppression includes below-baseline visual encoding at the position of salient distractors (Gaspelin and Luck, 2018) and neural signals such as the distractor positivity (Pd) that track how many distractors are presented in a given hemifield (Feldmann-Wüstefeld and Vogel, 2019). One basic question regarding distractor suppression is whether it is inherently spatial or nonspatial in character. Indeed, past work has shown that distractors evoke both spatial (Theeuwes, 1992) and nonspatial forms of interference (Folk and Remington, 1998), motivating a direct examination of whether space is integral to goal-driven distractor suppression. Here, we use behavioral and EEG data from adult humans (male and female) to provide clear evidence for a spatial gradient of suppression surrounding salient singleton distractors. Replicating past work, both reaction time and neural indices of target selection improved monotonically as the distance between target and distractor increased. Importantly, these target selection effects were paralleled by a monotonic decline in the amplitude of the Pd, an electrophysiological index of distractor suppression. Moreover, multivariate analyses revealed spatially selective activity in the θ-band that tracked the position of the target and, critically, revealed suppressed activity at spatial channels centered on distractor positions. Thus, goal-driven selection of relevant over irrelevant information benefits from a spatial gradient of suppression surrounding salient distractors.SIGNIFICANCE STATEMENT Past work has shown that distractor suppression is an important part of goal-driven attentional selection, but has not yet revealed whether suppression is spatially directed. Using behavioral data, event-related potentials (ERPs) of the EEG signal [N2pc and distractor positivity (Pd) component], as well as a multivariate model of EEG data [channel tuning functions (CTF)], we show that suppression-related neural activity increases monotonically as the distance between targets and distractors decreases, and that spatially-selective activity in the θ-band reveals depressed activity in spatial channels that index distractor positions. Thus, we provide robust evidence for spatially-guided distractor suppression, a result that has important implications for models of goal-driven attentional control.

Decoding chromaticity and luminance from patterns of EEG activity.

A long-standing question in the field of vision research is whether scalp-recorded EEG activity contains sufficient information to identify stimulus chromaticity. Recent multivariate work suggests that it is possible to decode which chromaticity an observer is viewing from the multielectrode pattern of EEG activity. There is debate, however, about whether the claimed effects of stimulus chromaticity on visual evoked potentials (VEPs) are instead caused by unequal stimulus luminances, which are achromatic differences. Here, we tested whether stimulus chromaticity could be decoded when potential confounds with luminance were minimized by (1) equating chromatic stimuli in luminance using heterochromatic flicker photometry for each observer and (2) independently varying the chromaticity and luminance of target stimuli, enabling us to test whether the pattern for a given chromaticity generalized across wide variations in luminance. We also tested whether luminance variations can be decoded from the topography of voltage across the scalp. In Experiment 1, we presented two chromaticities (appearing red and green) at three luminance levels during separate trials. In Experiment 2, we presented four chromaticities (appearing red, orange, yellow, and green) at two luminance levels. Using a pattern classifier and the multielectrode pattern of EEG activity, we were able to accurately decode the chromaticity and luminance level of each stimulus. Furthermore, we were able to decode stimulus chromaticity when we trained the classifier on chromaticities presented at one luminance level and tested at a different luminance level. Thus, EEG topography contains robust information regarding stimulus chromaticity, despite large variations in stimulus luminance.

Estimating the statistical power to detect set-size effects in contralateral delay activity.

The contralateral delay activity (CDA) is an event-related potential component commonly used to examine the online processes of visual working memory. Here, we provide a robust analysis of the statistical power that is needed to achieve reliable and reproducible results with the CDA. Using two very large EEG datasets that examined the contrast between CDA amplitude with set sizes 2 and 6 items and set sizes 2 and 4 items, we present a subsampling analysis that estimates the statistical power achieved with varying numbers of subjects and trials based on the proportion of significant tests in 10,000 iterations. We also generated simulated data using Bayesian multilevel modeling to estimate power beyond the bounds of the original datasets. The number of trials and subjects required depends critically on the effect size. Detecting the presence of the CDA-a reliable difference between contralateral and ipsilateral electrodes during the memory period-required only 30-50 clean trials with a sample of 25 subjects to achieve approximately 80% statistical power. However, for detecting a difference in CDA amplitude between two set sizes, a substantially larger number of trials and subjects were required; approximately 400 clean trials with 25 subjects to achieve 80% power. Thus, to achieve robust tests of how CDA activity differs across conditions, it is essential to be mindful of the estimated effect size. We recommend researchers designing experiments to detect set-size differences in the CDA collect substantially more trials per subject.

Covert Attention Increases the Gain of Stimulus-Evoked Population Codes.

Covert spatial attention has a variety of effects on the responses of individual neurons. However, relatively little is known about the net effect of these changes on sensory population codes, even though perception ultimately depends on population activity. Here, we measured the EEG in human observers (male and female), and isolated stimulus-evoked activity that was phase-locked to the onset of attended and ignored visual stimuli. Using an encoding model, we reconstructed spatially selective population tuning functions from the pattern of stimulus-evoked activity across the scalp. Our EEG-based approach allowed us to measure very early visually evoked responses occurring ∼100 ms after stimulus onset. In Experiment 1, we found that covert attention increased the amplitude of spatially tuned population responses at this early stage of sensory processing. In Experiment 2, we parametrically varied stimulus contrast to test how this effect scaled with stimulus contrast. We found that the effect of attention on the amplitude of spatially tuned responses increased with stimulus contrast, and was well described by an increase in response gain (i.e., a multiplicative scaling of the population response). Together, our results show that attention increases the gain of spatial population codes during the first wave of visual processing.SIGNIFICANCE STATEMENT We know relatively little about how attention improves population codes, even though perception is thought to critically depend on population activity. In this study, we used an encoding-model approach to test how attention modulates the spatial tuning of stimulus-evoked population responses measured with EEG. We found that attention multiplicatively scales the amplitude of spatially tuned population responses. Furthermore, this effect was present within 100 ms of stimulus onset. Thus, our results show that attention improves spatial population codes by increasing their gain at this early stage of processing.

Multivariate analysis of EEG activity indexes contingent attentional capture.

An extensive body of work has shown that attentional capture is contingent on the goals of the observer: Capture is strongly reduced or even eliminated when an irrelevant singleton stimulus does not match the target-defining properties (Folk et al., 1992). There has been a long-standing debate on whether attentional capture can be explained by goal-driven and/or stimulus-driven accounts. Here, we shed further light on this matter by using EEG activity (raw EEG and alpha power) to provide a time-resolved index of attentional orienting towards salient stimuli that either matched or did not match target-defining properties. A search display containing the target stimulus was preceded by a spatially uninformative singleton cue that either matched the color of the upcoming target (contingent cues), or that appeared in an irrelevant color (non-contingent cues). Multivariate analysis of raw EEG and alpha power revealed preferential tuning to the location of both contingent and non-contingent cues, with a stronger bias towards contingent than non-contingent cues. The time course of these effects, however, depended on the neural signal. Raw EEG data revealed attentional orienting towards the contingent cue early on in the trial (>156 ms), while alpha power revealed sustained spatial selection in the cued locations at a later moment in the trial (>250 ms). Moreover, while raw EEG showed stronger capture by contingent cues during this early time window, an advantage for contingent cues arose during a later time window in alpha band activity. Thus, our findings suggest that raw EEG activity and alpha-band power tap into distinct neural processes that index separate aspects of covert spatial attention.

Multivariate analysis reveals a generalizable human electrophysiological signature of working memory load.

Working memory (WM) is an online memory system that is critical for holding information in a rapidly accessible state during ongoing cognitive processing. Thus, there is strong value in methods that provide a temporally resolved index of WM load. While univariate EEG signals have been identified that vary with WM load, recent advances in multivariate analytic approaches suggest that there may be rich sources of information that do not generate reliable univariate signatures. Here, using data from four published studies (n = 286 and >250,000 trials), we demonstrate that multivariate analysis of EEG voltage topography provides a sensitive index of the number of items stored in WM that generalizes to novel human observers. Moreover, multivariate load detection ("mvLoad") can provide robust information at the single-trial level, exceeding the sensitivity of extant univariate approaches. We show that this method tracks WM load in a manner that is (1) independent of the spatial position of the memoranda, (2) precise enough to differentiate item-by-item increments in the number of stored items, (3) generalizable across distinct tasks and stimulus displays, and (4) correlated with individual differences in WM behavior. Thus, this approach provides a powerful complement to univariate analytic approaches, enabling temporally resolved tracking of online memory storage in humans.

Attention fluctuations impact ongoing maintenance of information in working memory.

Working memory maintains information in a readily accessible state and has been shown to degrade as the length of the retention interval increases. Previous research has suggested that this decline is attributable to changes in precision as well as sudden loss of item representations. Here, by measuring trial-to-trial variations in performance, we examined an orthogonal distinction between the maximum number of items that an individual can store, and the probability of achieving that maximum. Across two experiments, we replicated the finding that performance declines after long (10 s) retention intervals, as well as past observations that forgetting was due to probabilistic dropping of individual items rather than all-or-none losses of the stored memories. Critically, longer retention intervals did not reduce the maximum amount of information that could be stored in working memory. Instead, lower attentional control accounted for a decreased probability of maintaining the maximum number of items in working memory. Thus, longer retention intervals impact working memory storage via fluctuations in attentional control that lower the probability of achieving a stable maximum storage capacity.

Covert Spatial Attention Speeds Target Individuation.

Covert spatial attention has long been thought to speed visual processing. Psychophysics studies have shown that target information accrues faster at attended locations than at unattended locations. However, with behavioral evidence alone, it is difficult to determine whether attention speeds visual processing of the target or subsequent postperceptual stages of processing (e.g., converting sensory responses into decision signals). Moreover, although many studies have shown that attention can boost the amplitude of visually evoked neural responses, no robust effect has been observed on the latency of those neural responses. Here, we offer new evidence that may reconcile the neural and behavioral findings. We examined whether covert attention influenced the latency of the N2pc component, an electrophysiological marker of visual selection that has been linked with object individuation-the formation of an object representation that is distinct from the background and from other objects in the scene. To this end, we manipulated whether or not human observers (male and female) covertly attended the location of an impending search target. We found that the target evoked N2pc onset ∼20 ms earlier when the target location was cued than when it was not cued. In a second experiment, we provided a direct replication of this effect, confirming that the effect of attention on N2pc latency is robust. Thus, although attention may not speed the earliest stages of sensory processing, attention does speed the critical transition between raw sensory encoding and the formation of individuated object representations.SIGNIFICANCE STATEMENT Covert spatial attention improves processing at attended locations. Past behavioral studies have shown that information about visual targets accrues faster at attended than at unattended locations. However, it has remained unclear whether attention speeds perceptual analysis or subsequent postperceptual stages of processing. Here, we present robust evidence that attention speeds the N2pc, an electrophysiological signal that indexes the formation of individuated object representations. Our findings show that attention speeds a relatively early stage of perceptual processing while also elucidating the specific perceptual process that is speeded.