Behavioral signatures of the rapid recruitment of long-term memory to overcome working memory capacity limits.

Working- and long-term memory are often studied in isolation. To better understand the specific limitations of working memory, effort is made to reduce the potential influence of long-term memory on performance in working memory tasks (e.g., asking participants to remember artificial, abstract items rather than familiar real-world objects). However, in everyday life we use working- and long-term memory in tandem. Here, our goal was to characterize how long-term memory can be recruited to circumvent capacity limits in a typical visual working memory task (i.e., remembering colored squares). Prior work has shown that incidental repetitions of working memory arrays often do not improve visual working memory performance - even after dozens of incidental repetitions, working memory performance often shows no improvement for repeated arrays. Here, we used a whole-report working memory task with explicit rather than incidental repetitions of arrays. In contrast to prior work with incidental repetitions, in two behavioral experiments we found that explicit repetitions of arrays yielded robust improvement to working memory performance, even after a single repetition. Participants performed above chance at recognizing repeated arrays in a later long-term memory test, consistent with the idea that long-term memory was used to rapidly improve performance across array repetitions. Finally, we analyzed inter-item response times and we found a response time signature of chunk formation that only emerged after the array was repeated (inter-response time slowing after two to three items); thus, inter-item response times may be useful for examining the coordinated interaction of visual working and long-term memory in future work.

Distinguishing guesses from fuzzy memories: Further evidence for item limits in visual working memory.

There is consistent debate over whether capacity in working memory (WM) is subject to an item limit, or whether an unlimited number of items can be held in this online memory system. The item limit hypothesis clearly predicts guessing responses when capacity is exceeded, and proponents of this view have highlighted evidence for guessing in visual working memory tasks. Nevertheless, various models that deny item limits can explain the same empirical patterns by asserting extremely low fidelity representations that cannot be distinguished from guesses. To address this ambiguity, we employed a task for which guess responses elicited a qualitatively distinct pattern from low fidelity memories. Inspired by work from Rouder et al. (2014), we employed an orientation WM task that required subjects to recall the precise orientation of each of six memoranda presented 1 s earlier. The orientation stimuli were created by rotating the position of a "clock hand" inside a circular region that was demarcated by four colored quadrants. Critically, when observers guess with these stimuli, the distribution of responses is biased towards the center of these quadrants, creating a "banded" pattern that cannot be explained by a low precision memory. We confirmed the presence of this guessing pattern using formal model comparisons, and we show that the prevalence of this pattern matches observers' own reports of when they thought they were guessing. Thus, these findings provide further evidence for guessing behaviors predicted by item limit models of WM capacity.

Dynamics Are the Only Constant in Working Memory.

In this short perspective, we reflect upon our tendency to use oversimplified and idiosyncratic tasks in a quest to discover general mechanisms of working memory. We discuss how the work of Mark Stokes and collaborators has looked beyond localized, temporally persistent neural activity and shifted focus toward the importance of distributed, dynamic neural codes for working memory. A critical lesson from this work is that using simplified tasks does not automatically simplify the neural computations supporting behavior (even if we wish it were so). Moreover, Stokes' insights about multidimensional dynamics highlight the flexibility of the neural codes underlying cognition and have pushed the field to look beyond static measures of working memory.

Storage in Visual Working Memory Recruits a Content-Independent Pointer System.

Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.

Classic Visual Search Effects in an Additional Singleton Task: An Open Dataset.

Visual search refers to our ability to find what we are looking for among many competing visual inputs. Here, we report the availability of a rich dataset that replicates key visual search effects and shows that these effects are robust to several changes to the experimental design. Experiment 1 replicates classic findings from an additional singleton visual search task. First, participants are captured by a salient but irrelevant color singleton, as indexed by slower response times when a color singleton distractor is present versus absent. Second, attentional capture by a color singleton is reduced when the visual search array contains heterogeneous shapes rather than homogenous shapes. Finally, attentional capture by a color singleton is reduced when the display colors are repeated rather than switched unpredictably from trial to trial. Experiment 2 demonstrates that these classic visual search effects are robust to small procedural changes such as task timing (i.e., a 2-8 second rather than ~1 second inter-trial interval). Experiment 3 demonstrates that these classic effects are likewise robust to changes to the distractor frequency (75% rather than 50%) and to fully blocking versus interleaving blocks of two task conditions. All told, this dataset includes 8 sub-experiments, 190 participants and >210,000 trials, and it will serve as a useful resource for power analyses and exploratory analyses of visual search behaviors.

History Modulates Early Sensory Processing of Salient Distractors.

To find important objects, we must focus on our goals, ignore distractions, and take our changing environment into account. This is formalized in models of visual search whereby goal-driven, stimulus-driven, and history-driven factors are integrated into a priority map that guides attention. Stimulus history robustly influences where attention is allocated even when the physical stimulus is the same: when a salient distractor is repeated over time, it captures attention less effectively. A key open question is how we come to ignore salient distractors when they are repeated. Goal-driven accounts propose that we use an active, expectation-driven mechanism to attenuate the distractor signal (e.g., predictive coding), whereas stimulus-driven accounts propose that the distractor signal is attenuated because of passive changes to neural activity and inter-item competition (e.g., adaptation). To test these competing accounts, we measured item-specific fMRI responses in human visual cortex during a visual search task where trial history was manipulated (colors unpredictably switched or were repeated). Consistent with a stimulus-driven account of history-based distractor suppression, we found that repeated singleton distractors were suppressed starting in V1, and distractor suppression did not increase in later visual areas. In contrast, we observed signatures of goal-driven target enhancement that were absent in V1, increased across visual areas, and were not modulated by stimulus history. Our data suggest that stimulus history does not alter goal-driven expectations, but rather modulates canonically stimulus-driven sensory responses to contribute to a temporally integrated representation of priority.SIGNIFICANCE STATEMENT Visual search refers to our ability to find what we are looking for in a cluttered visual world (e.g., finding your keys). To perform visual search, we must integrate information about our goals (e.g., "find the red keychain"), the environment (e.g., salient items capture your attention), and changes to the environment (i.e., stimulus history). Although stimulus history impacts behavior, the neural mechanisms that mediate history-driven effects remain debated. Here, we leveraged fMRI and multivariate analysis techniques to measure history-driven changes to the neural representation of items during visual search. We found that stimulus history influenced the representation of a salient "pop-out" distractor starting in V1, suggesting that stimulus history operates via modulations of early sensory processing rather than goal-driven expectations.

Estimating the statistical power to detect set-size effects in contralateral delay activity.

The contralateral delay activity (CDA) is an event-related potential component commonly used to examine the online processes of visual working memory. Here, we provide a robust analysis of the statistical power that is needed to achieve reliable and reproducible results with the CDA. Using two very large EEG datasets that examined the contrast between CDA amplitude with set sizes 2 and 6 items and set sizes 2 and 4 items, we present a subsampling analysis that estimates the statistical power achieved with varying numbers of subjects and trials based on the proportion of significant tests in 10,000 iterations. We also generated simulated data using Bayesian multilevel modeling to estimate power beyond the bounds of the original datasets. The number of trials and subjects required depends critically on the effect size. Detecting the presence of the CDA-a reliable difference between contralateral and ipsilateral electrodes during the memory period-required only 30-50 clean trials with a sample of 25 subjects to achieve approximately 80% statistical power. However, for detecting a difference in CDA amplitude between two set sizes, a substantially larger number of trials and subjects were required; approximately 400 clean trials with 25 subjects to achieve 80% power. Thus, to achieve robust tests of how CDA activity differs across conditions, it is essential to be mindful of the estimated effect size. We recommend researchers designing experiments to detect set-size differences in the CDA collect substantially more trials per subject.

Steady-State Visually Evoked Potentials and Feature-based Attention: Preregistered Null Results and a Focused Review of Methodological Considerations.

Feature-based attention is the ability to selectively attend to a particular feature (e.g., attend to red but not green items while looking for the ketchup bottle in your refrigerator), and steady-state visually evoked potentials (SSVEPs) measured from the human EEG signal have been used to track the neural deployment of feature-based attention. Although many published studies suggest that we can use trial-by-trial cues to enhance relevant feature information (i.e., greater SSVEP response to the cued color), there is ongoing debate about whether participants may likewise use trial-by-trial cues to voluntarily ignore a particular feature. Here, we report the results of a preregistered study in which participants either were cued to attend or to ignore a color. Counter to prior work, we found no attention-related modulation of the SSVEP response in either cue condition. However, positive control analyses revealed that participants paid some degree of attention to the cued color (i.e., we observed a greater P300 component to targets in the attended vs. the unattended color). In light of these unexpected null results, we conducted a focused review of methodological considerations for studies of feature-based attention using SSVEPs. In the review, we quantify potentially important stimulus parameters that have been used in the past (e.g., stimulation frequency, trial counts) and we discuss the potential importance of these and other task factors (e.g., feature-based priming) for SSVEP studies.

Multivariate analysis reveals a generalizable human electrophysiological signature of working memory load.

Working memory (WM) is an online memory system that is critical for holding information in a rapidly accessible state during ongoing cognitive processing. Thus, there is strong value in methods that provide a temporally resolved index of WM load. While univariate EEG signals have been identified that vary with WM load, recent advances in multivariate analytic approaches suggest that there may be rich sources of information that do not generate reliable univariate signatures. Here, using data from four published studies (n = 286 and >250,000 trials), we demonstrate that multivariate analysis of EEG voltage topography provides a sensitive index of the number of items stored in WM that generalizes to novel human observers. Moreover, multivariate load detection ("mvLoad") can provide robust information at the single-trial level, exceeding the sensitivity of extant univariate approaches. We show that this method tracks WM load in a manner that is (1) independent of the spatial position of the memoranda, (2) precise enough to differentiate item-by-item increments in the number of stored items, (3) generalizable across distinct tasks and stimulus displays, and (4) correlated with individual differences in WM behavior. Thus, this approach provides a powerful complement to univariate analytic approaches, enabling temporally resolved tracking of online memory storage in humans.

No Evidence for an Object Working Memory Capacity Benefit with Extended Viewing Time.

Visual working memory is the ability to hold visual information temporarily in mind. A key feature of working memory is its starkly limited capacity, such that only a few simple items can be remembered at once. Prior work has shown that this capacity limit cannot be circumvented by providing additional encoding time, whether providing just 200 ms or up to 1300 ms, capacity is still limited to only three to four items. In contrast, Brady et al. (2016) hypothesized that real-world objects, but not simple items used in prior research, benefit from additional encoding time and are not subject to traditional capacity limits. They supported this hypothesis with results from both behavior and the contralateral delay activity (CDA), an EEG marker of working memory storage, and concluded that familiar, complex stimuli are necessary to observe encoding time effects. Here, we conducted three replications of Brady et al.'s key manipulation with a larger number of human participants and more trials per condition. We failed to replicate their primary behavioral result (objects benefit more than colors from additional encoding time) and failed to observe an object-specific increase in the CDA. Instead, we found that encoding time benefitted both simple color items and real-world objects, in contrast to both the findings by Brady et al., and some prior work on this topic. Overall, we observed no support for the hypothesis that real-world objects have a different capacity than colored squares. We discuss the implications of our findings for theories of visual working memory (VWM).

Interplay between persistent activity and activity-silent dynamics in the prefrontal cortex underlies serial biases in working memory.

Persistent neuronal spiking has long been considered the mechanism underlying working memory, but recent proposals argue for alternative 'activity-silent' substrates. Using monkey and human electrophysiology data, we show here that attractor dynamics that control neural spiking during mnemonic periods interact with activity-silent mechanisms in the prefrontal cortex (PFC). This interaction allows memory reactivations, which enhance serial biases in spatial working memory. Stimulus information was not decodable between trials, but remained present in activity-silent traces inferred from spiking synchrony in the PFC. Just before the new stimulus, this latent trace was reignited into activity that recapitulated the previous stimulus representation. Importantly, the reactivation strength correlated with the strength of serial biases in both monkeys and humans, as predicted by a computational model that integrates activity-based and activity-silent mechanisms. Finally, single-pulse transcranial magnetic stimulation applied to the human PFC between successive trials enhanced serial biases, thus demonstrating the causal role of prefrontal reactivations in determining working-memory behavior.

Δ9-Tetrahydrocannabinol (THC) impairs visual working memory performance: a randomized crossover trial.

With the increasing prevalence of legal cannabis use and availability, there is an urgent need to identify cognitive impairments related to its use. It is widely believed that cannabis, or its main psychoactive component Δ9-tetrahydrocannabinol (THC), impairs working memory, i.e., the ability to temporarily hold information in mind. However, our review of the literature yielded surprisingly little empirical support for an effect of THC or cannabis on working memory. We thus conducted a study with three main goals: (1) quantify the effect of THC on visual working memory in a well-powered sample, (2) test the potential role of cognitive effects (mind wandering and metacognition) in disrupting working memory, and (3) demonstrate how insufficient sample size and task duration reduce the likelihood of detecting a drug effect. We conducted two double-blind, randomized crossover experiments in which healthy adults (N = 23, 23) performed a reliable and validated visual working memory task (the "Discrete Whole Report task", 90 trials) after administration of THC (7.5 and/or 15 mg oral) or placebo. We also assessed self-reported "mind wandering" (Exp 1) and metacognitive accuracy about ongoing task performance (Exp 2). THC impaired working memory performance (d = 0.65), increased mind wandering (Exp 1), and decreased metacognitive accuracy about task performance (Exp 2). Thus, our findings indicate that THC does impair visual working memory, and that this impairment may be related to both increased mind wandering and decreased monitoring of task performance. Finally, we used a down-sampling procedure to illustrate the effects of task length and sample size on power to detect the acute effect of THC on working memory.

Distinguishing cognitive effort and working memory load using scale-invariance and alpha suppression in EEG.

Despite being intuitive, cognitive effort has proven difficult to define quantitatively. Here, we proposed to study cognitive effort by investigating the degree to which the brain deviates from its default state, where brain activity is scale-invariant. Specifically, we measured such deviations by examining changes in scale-invariance of brain activity as a function of task difficulty and posited suppression of scale-invariance as a proxy for exertion of cognitive effort. While there is some fMRI evidence supporting this proposition, EEG investigations on the matter are scant, despite the EEG signal being more suitable for analysis of scale invariance (i.e., having a much broader frequency range). In the current study we validated the correspondence between scale-invariance (H) of cortical activity recorded by EEG and task load during two working memory (WM) experiments with varying set sizes. Then, we used this neural signature to disentangle cognitive effort from the number of items stored in WM within participants. Our results showed monotonic decreases in H with increased set size, even after set size exceeded WM capacity. This behavior of H contrasted with behavioral performance and an oscillatory indicator of WM load (i.e., alpha-band desynchronization), both of which showed a plateau at difficulty levels surpassing WM capacity. This is the first reported evidence for the suppression of scale-invariance in EEG due to task difficulty, and our work suggests that H suppression may be used to quantify changes in cognitive effort even when working memory load is at maximum capacity.

Fluctuations of Attention and Working Memory.

Attention and working memory are intricately related, yet there remain ambiguities in how to best characterize this relationship. In his review, Oberauer formalizes several dimensions for the relationship between attention and working memory, focusing especially on the supporting role of attention during working memory maintenance. In this commentary, we highlight how attention and working memory relate on a broader time scale via trial-to-trial fluctuations. Specifically, we briefly describe evidence and implications of these fluctuations of attention and working memory. A strong link has been shown behaviorally (e.g., interleaved sustained attention and working memory tasks) and neurally (e.g., pre-trial predictors of working memory success), yet fluctuations of attention and working memory are also distinct. Thus, we argue that attention and working memory fluctuate synchronously but not synonymously.

Working Memory: Flexible but Finite.

There are inherent trade-offs between the flexibility and the capacity of working memory, or the ability to temporarily hold information "in mind." In a recent issue of Neuron, Bouchacourt and Buschman (2019) present a new model of working memory that demonstrates how coordinated activity between specialized sensory networks and flexible higher-order networks may support these competing constraints.

Item-specific delay activity demonstrates concurrent storage of multiple active neural representations in working memory.

Persistent neural activity that encodes online mental representations plays a central role in working memory (WM). However, there has been debate regarding the number of items that can be concurrently represented in this active neural state, which is often called the "focus of attention." Some models propose a strict single-item limit, such that just 1 item can be neurally active at once while other items are relegated to an activity-silent state. Although past studies have decoded multiple items stored in WM, these studies cannot rule out a switching account in which only a single item is actively represented at a time. Here, we directly tested whether multiple representations can be held concurrently in an active state. We tracked spatial representations in WM using alpha-band (8-12 Hz) activity, which encodes spatial positions held in WM. Human observers remembered 1 or 2 positions over a short delay while we recorded electroencephalography (EEG) data. Using a spatial encoding model, we reconstructed active stimulus-specific representations (channel-tuning functions [CTFs]) from the scalp distribution of alpha-band power. Consistent with past work, we found that the selectivity of spatial CTFs was lower when 2 items were stored than when 1 item was stored. Critically, data-driven simulations revealed that the selectivity of spatial representations in the two-item condition could not be explained by models that propose that only a single item can exist in an active state at once. Thus, our findings demonstrate that multiple items can be concurrently represented in an active neural state.

Dissecting the Neural Focus of Attention Reveals Distinct Processes for Spatial Attention and Object-Based Storage in Visual Working Memory.

Complex cognition relies on both on-line representations in working memory (WM), said to reside in the focus of attention, and passive off-line representations of related information. Here, we dissected the focus of attention by showing that distinct neural signals index the on-line storage of objects and sustained spatial attention. We recorded electroencephalogram (EEG) activity during two tasks that employed identical stimulus displays but varied the relative demands for object storage and spatial attention. We found distinct delay-period signatures for an attention task (which required only spatial attention) and a WM task (which invoked both spatial attention and object storage). Although both tasks required active maintenance of spatial information, only the WM task elicited robust contralateral delay activity that was sensitive to mnemonic load. Thus, we argue that the focus of attention is maintained via a collaboration between distinct processes for covert spatial orienting and object-based storage.

Improvements to visual working memory performance with practice and feedback.

Visual working memory capacity is estimated to be around 3-4 items, but on some trials participants fail to correctly report even a single item from the memory array. Such failures of working memory performance are surprisingly common, and participants have poor self-awareness of them. Previous work has shown that behavioral feedback can reduce the frequency of working memory failures, but the benefits of feedback disappeared immediately after it was taken away. Here, we tested whether extended practice with or without trial-by-trial feedback would lead to persistent improvements in working memory performance. Participants were assigned to one of four groups: (1) Working memory practice with feedback (2) Working memory practice without feedback (3) Crossword puzzle active control (4) No-contact control. Consistent with previous work, simple practice with a visual working memory task robustly improved working memory performance across practice sessions. However, we found only partial support for the efficacy of feedback in improving working memory performance. Practicing with feedback improved working memory performance relative to a no-feedback group for some practice sessions. However, the feedback benefits did not persist across all training sessions and did not transfer to a final test session without the feedback. Thus, the benefits of performance feedback did not persist over time. Further, we found only stimulus-specific transfer of visual working memory practice benefits. We also found that participants' metaknowledge improved with practice, but that receiving feedback about task accuracy actually slightly harmed the accuracy of concurrent metaknowledge ratings. Finally, we discuss important design considerations for future work in this area (e.g. power, expectations, and "spacing effects"). For example, we found that achieved statistical power to detect a between-groups effect declined with practice. This finding has potentially critical implications for any study using a 1-session study to calculate power for a planned multi-session study.