Behavioral signatures of the rapid recruitment of long-term memory to overcome working memory capacity limits.

Working- and long-term memory are often studied in isolation. To better understand the specific limitations of working memory, effort is made to reduce the potential influence of long-term memory on performance in working memory tasks (e.g., asking participants to remember artificial, abstract items rather than familiar real-world objects). However, in everyday life we use working- and long-term memory in tandem. Here, our goal was to characterize how long-term memory can be recruited to circumvent capacity limits in a typical visual working memory task (i.e., remembering colored squares). Prior work has shown that incidental repetitions of working memory arrays often do not improve visual working memory performance - even after dozens of incidental repetitions, working memory performance often shows no improvement for repeated arrays. Here, we used a whole-report working memory task with explicit rather than incidental repetitions of arrays. In contrast to prior work with incidental repetitions, in two behavioral experiments we found that explicit repetitions of arrays yielded robust improvement to working memory performance, even after a single repetition. Participants performed above chance at recognizing repeated arrays in a later long-term memory test, consistent with the idea that long-term memory was used to rapidly improve performance across array repetitions. Finally, we analyzed inter-item response times and we found a response time signature of chunk formation that only emerged after the array was repeated (inter-response time slowing after two to three items); thus, inter-item response times may be useful for examining the coordinated interaction of visual working and long-term memory in future work.

Encoded and updated spatial working memories share a common representational format in alpha activity.

Working memory (WM) flexibly updates information to adapt to the dynamic environment. Here, we used alpha-band activity in the EEG to reconstruct the content of dynamic WM updates and compared this representational format to static WM content. An inverted encoding model using alpha activity precisely tracked both the initially encoded position and the updated position following an auditory cue signaling mental updating. The timing of the update, as tracked in the EEG, correlated with reaction times and saccade latency. Finally, cross-training analyses revealed a robust generalization of alpha-band reconstruction of WM contents before and after updating. These findings demonstrate that alpha activity tracks the dynamic updates to spatial WM and that the format of this activity is preserved across the encoded and updated representations. Thus, our results highlight a new approach for measuring updates to WM and show common representational formats during dynamic mental updating and static storage.

Sequential encoding paradigm reliably captures the individual differences from a simultaneous visual working memory task.

Converging behavioral and neural evidence have suggested that visual stimuli could be attached to existing visual working memory sequentially in time. However, whether individual differences in sequential visual working memory paradigm are similar to those measured by the classical simultaneous change detection paradigm remain unknown. Here, we first show that sequentially presented visual stimuli exhibit similar working memory capacity bottlenecks as previous research using simultaneously presented items. We further reveal that within the same subject, the accuracy and capacity estimates using sequential and simultaneous paradigm were comparable across four different set sizes. Also, we discover that the individual differences measured by the sequential paradigm were highly correlated to those by the simultaneous paradigm within the same subject across all four set sizes of interest. Finally, we find that in a larger sample of subjects (n = 200), the variance and higher-order statistics were similar for sequential and simultaneous paradigms with set size of 6. Collectively, these findings indicate that individual differences measured by the sequential presentation of visual items rely on the similar working memory resources as those by the simultaneous form of presentation.

Pupillometry signatures of sustained attention and working memory.

There exists an intricate relationship between attention and working memory. Recent work has further established that attention and working memory fluctuate synchronously, by tightly interleaving sustained attention and working memory tasks. This work has raised many open questions about physiological signatures underlying these behavioral fluctuations. Across two experiments, we explore pupil dynamics using real-time triggering in conjunction with an interleaved sustained attention and working memory task. In Experiment 1, we use behavioral real-time triggering and replicate recent findings from our lab (deBettencourt et al., 2019) that sustained attention fluctuates concurrently with the number of items maintained in working memory. Furthermore, highly attentive moments, detected via behavior, also exhibited larger pupil sizes. In Experiment 2, we develop a novel real-time pupil-triggering technique to track pupil size fluctuations in real time and trigger working memory probes. We show that this pupil triggering procedure reveals differences in sustained attention, as indexed by response time. These experiments reflect methodological advances in real-time triggering and further disentangle the relationship among general arousal, sustained attention, and working memory.

Storage in Visual Working Memory Recruits a Content-Independent Pointer System.

Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.

Inter-electrode correlations measured with EEG predict individual differences in cognitive ability.

Human brains share a broadly similar functional organization with consequential individual variation. This duality in brain function has primarily been observed when using techniques that consider the spatial organization of the brain, such as MRI. Here, we ask whether these common and unique signals of cognition are also present in temporally sensitive but spatially insensitive neural signals. To address this question, we compiled electroencephalogram (EEG) data from individuals of both sexes while they performed multiple working memory tasks at two different data-collection sites (n = 171 and 165). Results revealed that trial-averaged EEG activity exhibited inter-electrode correlations that were stable within individuals and unique across individuals. Furthermore, models based on these inter-electrode correlations generalized across datasets to predict participants' working memory capacity and general fluid intelligence. Thus, inter-electrode correlation patterns measured with EEG provide a signature of working memory and fluid intelligence in humans and a new framework for characterizing individual differences in cognitive abilities.

The visual arrays task: Visual storage capacity or attention control?

Extant literature suggests that performance on visual arrays tasks reflects limited-capacity storage of visual information. However, there is also evidence to suggest that visual arrays task performance reflects individual differences in controlled processing. The purpose of this study is to empirically evaluate the degree to which visual arrays tasks are more closely related to memory storage capacity or measures of attention control. To this end, we conducted new analyses on a series of large data sets that incorporate various versions of a visual arrays task. Based on these analyses, we suggest that the degree to which the visual arrays is related to memory storage ability or effortful attention control may be task-dependent. Specifically, when versions of the task require participants to ignore elements of the target display, individual differences in controlled attention reliably provide unique predictive value. Therefore, at least some versions of the visual arrays tasks can be used as valid indicators of individual differences in attention control. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Sustained Attention and Spatial Attention Distinctly Influence Long-term Memory Encoding.

Our attention is critically important for what we remember. Prior measures of the relationship between attention and memory, however, have largely treated "attention" as a monolith. Here, across three experiments, we provide evidence for two dissociable aspects of attention that influence encoding into long-term memory. Using spatial cues together with a sensitive continuous report procedure, we find that long-term memory response error is affected by both trial-by-trial fluctuations of sustained attention and prioritization via covert spatial attention. Furthermore, using multivariate analyses of EEG, we track both sustained attention and spatial attention before stimulus onset. Intriguingly, even during moments of low sustained attention, there is no decline in the representation of the spatially attended location, showing that these two aspects of attention have robust but independent effects on long-term memory encoding. Finally, sustained and spatial attention predicted distinct variance in long-term memory performance across individuals. That is, the relationship between attention and long-term memory suggests a composite model, wherein distinct attentional subcomponents influence encoding into long-term memory. These results point toward a taxonomy of the distinct attentional processes that constrain our memories.

Perceptual Grouping Reveals Distinct Roles for Sustained Slow Wave Activity and Alpha Oscillations in Working Memory.

Multiple neural signals have been found to track the number of items stored in working memory (WM). These signals include oscillatory activity in the alpha band and slow-wave components in human EEG, both of which vary with storage loads and predict individual differences in WM capacity. However, recent evidence suggests that these two signals play distinct roles in spatial attention and item-based storage in WM. Here, we examine the hypothesis that sustained negative voltage deflections over parieto-occipital electrodes reflect the number of individuated items in WM, whereas oscillatory activity in the alpha frequency band (8-12 Hz) within the same electrodes tracks the attended positions in the visual display. We measured EEG activity while participants stored the orientation of visual elements that were either grouped by collinearity or not. This grouping manipulation altered the number of individuated items perceived while holding constant the number of locations occupied by visual stimuli. The negative slow wave tracked the number of items stored and was reduced in amplitude in the grouped condition. By contrast, oscillatory activity in the alpha frequency band tracked the number of positions occupied by the memoranda and was unaffected by perceptual grouping. Perceptual grouping, then, reduced the number of individuated representations stored in WM as reflected by the negative slow wave, whereas the location of each element was actively maintained as indicated by alpha power. These findings contribute to the emerging idea that distinct classes of EEG signals work in concert to successfully maintain on-line representations in WM.

Controlling the Flow of Distracting Information in Working Memory.

Visual working memory (WM) must maintain relevant information, despite the constant influx of both relevant and irrelevant information. Attentional control mechanisms help determine which of this new information gets access to our capacity-limited WM system. Previous work has treated attentional control as a monolithic process-either distractors capture attention or they are suppressed. Here, we provide evidence that attentional capture may instead be broken down into at least two distinct subcomponent processes: (1) Spatial capture, which refers to when spatial attention shifts towards the location of irrelevant stimuli and (2) item-based capture, which refers to when item-based WM representations of irrelevant stimuli are formed. To dissociate these two subcomponent processes of attentional capture, we utilized a series of electroencephalography components that track WM maintenance (contralateral delay activity), suppression (distractor positivity), item individuation (N2pc), and spatial attention (lateralized alpha power). We show that new, relevant information (i.e., a task-relevant distractor) triggers both spatial and item-based capture. Irrelevant distractors, however, only trigger spatial capture from which ongoing WM representations can recover more easily. This fractionation of attentional capture into distinct subcomponent processes provides a refined framework for understanding how distracting stimuli affect attention and WM.

Estimating the statistical power to detect set-size effects in contralateral delay activity.

The contralateral delay activity (CDA) is an event-related potential component commonly used to examine the online processes of visual working memory. Here, we provide a robust analysis of the statistical power that is needed to achieve reliable and reproducible results with the CDA. Using two very large EEG datasets that examined the contrast between CDA amplitude with set sizes 2 and 6 items and set sizes 2 and 4 items, we present a subsampling analysis that estimates the statistical power achieved with varying numbers of subjects and trials based on the proportion of significant tests in 10,000 iterations. We also generated simulated data using Bayesian multilevel modeling to estimate power beyond the bounds of the original datasets. The number of trials and subjects required depends critically on the effect size. Detecting the presence of the CDA-a reliable difference between contralateral and ipsilateral electrodes during the memory period-required only 30-50 clean trials with a sample of 25 subjects to achieve approximately 80% statistical power. However, for detecting a difference in CDA amplitude between two set sizes, a substantially larger number of trials and subjects were required; approximately 400 clean trials with 25 subjects to achieve 80% power. Thus, to achieve robust tests of how CDA activity differs across conditions, it is essential to be mindful of the estimated effect size. We recommend researchers designing experiments to detect set-size differences in the CDA collect substantially more trials per subject.

Multivariate analysis reveals a generalizable human electrophysiological signature of working memory load.

Working memory (WM) is an online memory system that is critical for holding information in a rapidly accessible state during ongoing cognitive processing. Thus, there is strong value in methods that provide a temporally resolved index of WM load. While univariate EEG signals have been identified that vary with WM load, recent advances in multivariate analytic approaches suggest that there may be rich sources of information that do not generate reliable univariate signatures. Here, using data from four published studies (n = 286 and >250,000 trials), we demonstrate that multivariate analysis of EEG voltage topography provides a sensitive index of the number of items stored in WM that generalizes to novel human observers. Moreover, multivariate load detection ("mvLoad") can provide robust information at the single-trial level, exceeding the sensitivity of extant univariate approaches. We show that this method tracks WM load in a manner that is (1) independent of the spatial position of the memoranda, (2) precise enough to differentiate item-by-item increments in the number of stored items, (3) generalizable across distinct tasks and stimulus displays, and (4) correlated with individual differences in WM behavior. Thus, this approach provides a powerful complement to univariate analytic approaches, enabling temporally resolved tracking of online memory storage in humans.

No Evidence for an Object Working Memory Capacity Benefit with Extended Viewing Time.

Visual working memory is the ability to hold visual information temporarily in mind. A key feature of working memory is its starkly limited capacity, such that only a few simple items can be remembered at once. Prior work has shown that this capacity limit cannot be circumvented by providing additional encoding time, whether providing just 200 ms or up to 1300 ms, capacity is still limited to only three to four items. In contrast, Brady et al. (2016) hypothesized that real-world objects, but not simple items used in prior research, benefit from additional encoding time and are not subject to traditional capacity limits. They supported this hypothesis with results from both behavior and the contralateral delay activity (CDA), an EEG marker of working memory storage, and concluded that familiar, complex stimuli are necessary to observe encoding time effects. Here, we conducted three replications of Brady et al.'s key manipulation with a larger number of human participants and more trials per condition. We failed to replicate their primary behavioral result (objects benefit more than colors from additional encoding time) and failed to observe an object-specific increase in the CDA. Instead, we found that encoding time benefitted both simple color items and real-world objects, in contrast to both the findings by Brady et al., and some prior work on this topic. Overall, we observed no support for the hypothesis that real-world objects have a different capacity than colored squares. We discuss the implications of our findings for theories of visual working memory (VWM).

Attention fluctuations impact ongoing maintenance of information in working memory.

Working memory maintains information in a readily accessible state and has been shown to degrade as the length of the retention interval increases. Previous research has suggested that this decline is attributable to changes in precision as well as sudden loss of item representations. Here, by measuring trial-to-trial variations in performance, we examined an orthogonal distinction between the maximum number of items that an individual can store, and the probability of achieving that maximum. Across two experiments, we replicated the finding that performance declines after long (10 s) retention intervals, as well as past observations that forgetting was due to probabilistic dropping of individual items rather than all-or-none losses of the stored memories. Critically, longer retention intervals did not reduce the maximum amount of information that could be stored in working memory. Instead, lower attentional control accounted for a decreased probability of maintaining the maximum number of items in working memory. Thus, longer retention intervals impact working memory storage via fluctuations in attentional control that lower the probability of achieving a stable maximum storage capacity.

Unconscious Number Discrimination in the Human Visual System.

How do humans compute approximate number? According to one influential theory, approximate number representations arise in the intraparietal sulcus and are amodal, meaning that they arise independent of any sensory modality. Alternatively, approximate number may be computed initially within sensory systems. Here we tested for sensitivity to approximate number in the visual system using steady state visual evoked potentials. We recorded electroencephalography from humans while they viewed dotclouds presented at 30 Hz, which alternated in numerosity (ranging from 10 to 20 dots) at 15 Hz. At this rate, each dotcloud backward masked the previous dotcloud, disrupting top-down feedback to visual cortex and preventing conscious awareness of the dotclouds' numerosities. Spectral amplitude at 15 Hz measured over the occipital lobe (Oz) correlated positively with the numerical ratio of the stimuli, even when nonnumerical stimulus attributes were controlled, indicating that subjects' visual systems were differentiating dotclouds on the basis of their numerical ratios. Crucially, subjects were unable to discriminate the numerosities of the dotclouds consciously, indicating the backward masking of the stimuli disrupted reentrant feedback to visual cortex. Approximate number appears to be computed within the visual system, independently of higher-order areas, such as the intraparietal sulcus.

Δ9-Tetrahydrocannabinol (THC) impairs visual working memory performance: a randomized crossover trial.

With the increasing prevalence of legal cannabis use and availability, there is an urgent need to identify cognitive impairments related to its use. It is widely believed that cannabis, or its main psychoactive component Δ9-tetrahydrocannabinol (THC), impairs working memory, i.e., the ability to temporarily hold information in mind. However, our review of the literature yielded surprisingly little empirical support for an effect of THC or cannabis on working memory. We thus conducted a study with three main goals: (1) quantify the effect of THC on visual working memory in a well-powered sample, (2) test the potential role of cognitive effects (mind wandering and metacognition) in disrupting working memory, and (3) demonstrate how insufficient sample size and task duration reduce the likelihood of detecting a drug effect. We conducted two double-blind, randomized crossover experiments in which healthy adults (N = 23, 23) performed a reliable and validated visual working memory task (the "Discrete Whole Report task", 90 trials) after administration of THC (7.5 and/or 15 mg oral) or placebo. We also assessed self-reported "mind wandering" (Exp 1) and metacognitive accuracy about ongoing task performance (Exp 2). THC impaired working memory performance (d = 0.65), increased mind wandering (Exp 1), and decreased metacognitive accuracy about task performance (Exp 2). Thus, our findings indicate that THC does impair visual working memory, and that this impairment may be related to both increased mind wandering and decreased monitoring of task performance. Finally, we used a down-sampling procedure to illustrate the effects of task length and sample size on power to detect the acute effect of THC on working memory.

Distinguishing cognitive effort and working memory load using scale-invariance and alpha suppression in EEG.

Despite being intuitive, cognitive effort has proven difficult to define quantitatively. Here, we proposed to study cognitive effort by investigating the degree to which the brain deviates from its default state, where brain activity is scale-invariant. Specifically, we measured such deviations by examining changes in scale-invariance of brain activity as a function of task difficulty and posited suppression of scale-invariance as a proxy for exertion of cognitive effort. While there is some fMRI evidence supporting this proposition, EEG investigations on the matter are scant, despite the EEG signal being more suitable for analysis of scale invariance (i.e., having a much broader frequency range). In the current study we validated the correspondence between scale-invariance (H) of cortical activity recorded by EEG and task load during two working memory (WM) experiments with varying set sizes. Then, we used this neural signature to disentangle cognitive effort from the number of items stored in WM within participants. Our results showed monotonic decreases in H with increased set size, even after set size exceeded WM capacity. This behavior of H contrasted with behavioral performance and an oscillatory indicator of WM load (i.e., alpha-band desynchronization), both of which showed a plateau at difficulty levels surpassing WM capacity. This is the first reported evidence for the suppression of scale-invariance in EEG due to task difficulty, and our work suggests that H suppression may be used to quantify changes in cognitive effort even when working memory load is at maximum capacity.

Perturbing Neural Representations of Working Memory with Task-irrelevant Interruption.

Working memory maintains information so that it can be used in complex cognitive tasks. A key challenge for this system is to maintain relevant information in the face of task-irrelevant perturbations. Across two experiments, we investigated the impact of task-irrelevant interruptions on neural representations of working memory. We recorded EEG activity in humans while they performed a working memory task. On a subset of trials, we interrupted participants with salient but task-irrelevant objects. To track the impact of these task-irrelevant interruptions on neural representations of working memory, we measured two well-characterized, temporally sensitive EEG markers that reflect active, prioritized working memory representations: the contralateral delay activity and lateralized alpha power (8-12 Hz). After interruption, we found that contralateral delay activity amplitude momentarily sustained but was gone by the end of the trial. Lateralized alpha power was immediately influenced by the interrupters but recovered by the end of the trial. This suggests that dissociable neural processes contribute to the maintenance of working memory information and that brief irrelevant onsets disrupt two distinct online aspects of working memory. In addition, we found that task expectancy modulated the timing and magnitude of how these two neural signals responded to task-irrelevant interruptions, suggesting that the brain's response to task-irrelevant interruption is shaped by task context.

Visual short-term memory capacity predicts the "bandwidth" of visual long-term memory encoding.

We are capable of storing a virtually infinite amount of visual information in visual long-term memory (VLTM) storage. At the same time, the amount of visual information we can encode and maintain in visual short-term memory (VSTM) at a given time is severely limited. How do these two memory systems interact to accumulate vast amount of VLTM? In this series of experiments, we exploited interindividual and intraindividual differences VSTM capacity to examine the direct involvement of VSTM in determining the encoding rate (or "bandwidth") of VLTM. Here, we found that the amount of visual information encoded into VSTM at a given moment (i.e., VSTM capacity), but neither the maintenance duration nor the test process, predicts the effective encoding "bandwidth" of VLTM.

Alpha-band oscillations track the retrieval of precise spatial representations from long-term memory.

A hallmark of episodic memory is the phenomenon of mentally reexperiencing the details of past events, and a well-established concept is that the neuronal activity that mediates encoding is reinstated at retrieval. Evidence for reinstatement has come from multiple modalities, including functional magnetic resonance imaging and electroencephalography (EEG). These EEG studies have shed light on the time course of reinstatement but have been limited to distinguishing between a few categories. The goal of this work was to use recently developed experimental and technical approaches, namely continuous report tasks and inverted encoding models, to determine which frequencies of oscillatory brain activity support the retrieval of precise spatial memories. In experiment 1, we establish that an inverted encoding model applied to multivariate alpha topography tracks the retrieval of precise spatial memories. In experiment 2, we demonstrate that the frequencies and patterns of multivariate activity at study are similar to the frequencies and patterns observed during retrieval. These findings highlight the broad potential for using encoding models to characterize long-term memory retrieval.NEW & NOTEWORTHY Previous EEG work has shown that category-level information observed during encoding is recapitulated during memory retrieval, but studies with this time-resolved method have not demonstrated the reinstatement of feature-specific patterns of neural activity during retrieval. Here we show that EEG alpha-band activity tracks the retrieval of spatial representations from long-term memory. Moreover, we find considerable overlap between the frequencies and patterns of activity that track spatial memories during initial study and at retrieval.