Sustained Attention and Spatial Attention Distinctly Influence Long-term Memory Encoding.

Our attention is critically important for what we remember. Prior measures of the relationship between attention and memory, however, have largely treated "attention" as a monolith. Here, across three experiments, we provide evidence for two dissociable aspects of attention that influence encoding into long-term memory. Using spatial cues together with a sensitive continuous report procedure, we find that long-term memory response error is affected by both trial-by-trial fluctuations of sustained attention and prioritization via covert spatial attention. Furthermore, using multivariate analyses of EEG, we track both sustained attention and spatial attention before stimulus onset. Intriguingly, even during moments of low sustained attention, there is no decline in the representation of the spatially attended location, showing that these two aspects of attention have robust but independent effects on long-term memory encoding. Finally, sustained and spatial attention predicted distinct variance in long-term memory performance across individuals. That is, the relationship between attention and long-term memory suggests a composite model, wherein distinct attentional subcomponents influence encoding into long-term memory. These results point toward a taxonomy of the distinct attentional processes that constrain our memories.

Neurofeedback helps to reveal a relationship between context reinstatement and memory retrieval.

Theories of mental context and memory posit that successful mental context reinstatement enables better retrieval of memories from the same context, at the expense of memories from other contexts. To test this hypothesis, we had participants study lists of words, interleaved with task-irrelevant images from one category (e.g., scenes). Following encoding, participants were cued to mentally reinstate the context associated with a particular list, by thinking about the images that had appeared between the words. We measured context reinstatement by applying multivariate pattern classifiers to fMRI, and related this to performance on a free recall test that followed immediately afterwards. To increase sensitivity, we used a closed-loop neurofeedback procedure, whereby higher classifier evidence for the cued category elicited increased visibility of the images from the studied context onscreen. Our goal was to create a positive feedback loop that amplified small fluctuations in mental context reinstatement, making it easier to experimentally detect a relationship between context reinstatement and recall. As predicted, we found that greater amounts of classifier evidence were associated with better recall of words from the reinstated context, and worse recall of words from a different context. In a second experiment, we assessed the role of neurofeedback in identifying this brain-behavior relationship by presenting context images again and manipulating whether their visibility depended on classifier evidence. When neurofeedback was removed, the relationship between classifier evidence and memory retrieval disappeared. Together, these findings demonstrate a clear effect of context reinstatement on memory recall and suggest that neurofeedback can be a useful tool for characterizing brain-behavior relationships.

Decoding the content of working memory in school-aged children.

Developmental improvements in working memory (WM) maintenance predict many real-world outcomes, including educational attainment. It is thus critical to understand which WM mechanisms support these behavioral improvements, and how WM maintenance strategies might change through development. One challenge is that specific WM neural mechanisms cannot easily be measured behaviorally, especially in a child population. However, new multivariate decoding techniques have been designed, primarily in adult populations, that can sensitively decode the contents of WM. The goal of this study was to deploy multivariate decoding techniques known to decode memory representations in adults to decode the contents of WM in children. We created a simple computerized WM game for children, in which children maintained different categories of information (visual, spatial or verbal). We collected electroencephalography (EEG) data from 20 children (7-12-year-olds) while they played the game. Using Multivariate Pattern Analysis (MVPA) on children's EEG signals, we reliably decoded the category of the maintained information during the sensory and maintenance period. Across exploratory reliability and validity analyses, we examined the robustness of these results when trained on less data, and how these patterns generalized within individuals throughout the testing session. Furthermore, these results matched theory-based predictions of WM across individuals and across ages. Our proof-of-concept study proposes a direct and age-appropriate potential alternative to exclusively behavioral WM maintenance measures in children. Our study demonstrates the utility of MVPA to measure and track the uninstructed representational content of children's WM. Future research could use our technique to investigate children's WM maintenance and strategies.

Decoding the content of working memory in school-aged children.

Developmental improvements in working memory (WM) maintenance predict many real-world outcomes, including educational attainment. It is thus critical to understand which WM mechanisms support these behavioral improvements, and how WM maintenance strategies might change through development. One challenge is that specific WM neural mechanisms cannot easily be measured behaviorally, especially in a child population. However, new multivariate decoding techniques have been designed, primarily in adult populations, that can sensitively decode the contents of WM. The goal of this study was to deploy multivariate decoding techniques known to decode memory representations in adults to decode the contents of WM in children. We created a simple computerized WM game for children, in which children maintained different categories of information (visual, spatial or verbal). We collected electroencephalography (EEG) data from 20 children (7-12-year-olds) while they played the game. Using Multivariate Pattern Analysis (MVPA) on children's EEG signals, we reliably decoded the category of the maintained information during the sensory and maintenance period. Across exploratory reliability and validity analyses, we examined the robustness of these results when trained on less data, and how these patterns generalized within individuals throughout the testing session. Furthermore, these results matched theory-based predictions of WM across individuals and across ages. Our proof-of-concept study proposes a direct and age-appropriate potential alternative to exclusively behavioral WM maintenance measures in children. Our study demonstrates the utility of MVPA to measure and track the uninstructed representational content of children's WM. Future research could use our technique to investigate children's WM maintenance and strategies.

Pupillometry signatures of sustained attention and working memory.

There exists an intricate relationship between attention and working memory. Recent work has further established that attention and working memory fluctuate synchronously, by tightly interleaving sustained attention and working memory tasks. This work has raised many open questions about physiological signatures underlying these behavioral fluctuations. Across two experiments, we explore pupil dynamics using real-time triggering in conjunction with an interleaved sustained attention and working memory task. In Experiment 1, we use behavioral real-time triggering and replicate recent findings from our lab (deBettencourt et al., 2019) that sustained attention fluctuates concurrently with the number of items maintained in working memory. Furthermore, highly attentive moments, detected via behavior, also exhibited larger pupil sizes. In Experiment 2, we develop a novel real-time pupil-triggering technique to track pupil size fluctuations in real time and trigger working memory probes. We show that this pupil triggering procedure reveals differences in sustained attention, as indexed by response time. These experiments reflect methodological advances in real-time triggering and further disentangle the relationship among general arousal, sustained attention, and working memory.

Predicting visual memory across images and within individuals.

We only remember a fraction of what we see-including images that are highly memorable and those that we encounter during highly attentive states. However, most models of human memory disregard both an image's memorability and an individual's fluctuating attentional states. Here, we build the first model of memory synthesizing these two disparate factors to predict subsequent image recognition. We combine memorability scores of 1100 images (Experiment 1, n = 706) and attentional state indexed by response time on a continuous performance task (Experiments 2 and 3, n = 57 total). Image memorability and sustained attentional state explained significant variance in image memory, and a joint model of memory including both factors outperformed models including either factor alone. Furthermore, models including both factors successfully predicted memory in an out-of-sample group. Thus, building models based on individual- and image-specific factors allows for directed forecasting of our memories. SIGNIFICANCE STATEMENT: Although memory is a fundamental cognitive process, much of the time memory failures cannot be predicted until it is too late. However, in this study, we show that much of memory is surprisingly pre-determined ahead of time, by factors shared across the population and highly specific to each individual. Specifically, we build a new multidimensional model that predicts memory based just on the images a person sees and when they see them. This research synthesizes findings from disparate domains ranging from computer vision, attention, and memory into a predictive model. These findings have resounding implications for domains such as education, business, and marketing, where it is a top priority to predict (and even manipulate) what information people will remember.

Cloud-Based Functional Magnetic Resonance Imaging Neurofeedback to Reduce the Negative Attentional Bias in Depression: A Proof-of-Concept Study.

Individuals with depression show an attentional bias toward negatively valenced stimuli and thoughts. In this proof-of-concept study, we present a novel closed-loop neurofeedback procedure intended to remediate this bias. Internal attentional states were detected in real time by applying machine learning techniques to functional magnetic resonance imaging data on a cloud server; these attentional states were externalized using a visual stimulus that the participant could learn to control. We trained 15 participants with major depressive disorder and 12 healthy control participants over 3 functional magnetic resonance imaging sessions. Exploratory analysis showed that participants with major depressive disorder were initially more likely than healthy control participants to get stuck in negative attentional states, but this diminished with neurofeedback training relative to controls. Depression severity also decreased from pre- to posttraining. These results demonstrate that our method is sensitive to the negative attentional bias in major depressive disorder and showcase the potential of this novel technique as a treatment that can be evaluated in future clinical trials.

Attention fluctuations impact ongoing maintenance of information in working memory.

Working memory maintains information in a readily accessible state and has been shown to degrade as the length of the retention interval increases. Previous research has suggested that this decline is attributable to changes in precision as well as sudden loss of item representations. Here, by measuring trial-to-trial variations in performance, we examined an orthogonal distinction between the maximum number of items that an individual can store, and the probability of achieving that maximum. Across two experiments, we replicated the finding that performance declines after long (10 s) retention intervals, as well as past observations that forgetting was due to probabilistic dropping of individual items rather than all-or-none losses of the stored memories. Critically, longer retention intervals did not reduce the maximum amount of information that could be stored in working memory. Instead, lower attentional control accounted for a decreased probability of maintaining the maximum number of items in working memory. Thus, longer retention intervals impact working memory storage via fluctuations in attentional control that lower the probability of achieving a stable maximum storage capacity.

Fluctuations of Attention and Working Memory.

Attention and working memory are intricately related, yet there remain ambiguities in how to best characterize this relationship. In his review, Oberauer formalizes several dimensions for the relationship between attention and working memory, focusing especially on the supporting role of attention during working memory maintenance. In this commentary, we highlight how attention and working memory relate on a broader time scale via trial-to-trial fluctuations. Specifically, we briefly describe evidence and implications of these fluctuations of attention and working memory. A strong link has been shown behaviorally (e.g., interleaved sustained attention and working memory tasks) and neurally (e.g., pre-trial predictors of working memory success), yet fluctuations of attention and working memory are also distinct. Thus, we argue that attention and working memory fluctuate synchronously but not synonymously.

Real-time triggering reveals concurrent lapses of attention and working memory.

Attention and working memory are clearly intertwined, as shown by co-variations in individual ability and the recruitment of similar neural substrates. Both processes fluctuate over time1-5, and these fluctuations may be a key determinant of individual variations in ability6,7. If these fluctuations are due to the waxing and waning of a common cognitive resource, attention and working memory should co-vary on a moment-to-moment basis. To test this, we developed a hybrid task that interleaved a sustained attention task and a whole-report working memory task. Experiment 1 established that performance fluctuations on these tasks correlated across and within participants: attention lapses led to worse working memory performance. Experiment 2 extended this finding using a real-time triggering procedure that monitored attention fluctuations to probe working memory during optimal (high-attention) or suboptimal (low-attention) moments. In low-attention moments, participants stored fewer items in working memory. Experiment 3 ruled out task-general fluctuations as an explanation for these co-variations by showing that the precision of colour memory was unaffected by variations in attention state. In summary, we demonstrate that attention and working memory lapse together, providing additional evidence for the tight integration of these cognitive processes.

Forgetting from lapses of sustained attention.

When performing any task for an extended period of time, attention fluctuates between good and bad states. These fluctuations affect performance in the moment, but may also have lasting consequences for what gets encoded into memory. Experiment 1 establishes this relationship between attentional states and memory, by showing that subsequent memory for an item was predicted by a response time index of sustained attention (average response time during the three trials prior to stimulus onset). Experiment 2 strengthens the causal interpretation of this predictive relationship by treating the sustained attention index as an independent variable to trigger the appearance of an encoding trial. Subsequent memory was better when items were triggered from good versus bad attentional states. Together, these findings suggest that sustained attention can have downstream consequences for what we remember, and they highlight the inferential utility of adaptive experimental designs. By continuously monitoring attention, we can influence what will later be remembered.

Neuroscience: Incepting Associations.

A recent study has used real-time fMRI neurofeedback to induce color-specific activity patterns in early visual cortex as participants viewed achromatic gratings. This procedure resulted in an association between the color and the displayed grating orientation, suggesting that early visual cortex can support associative learning of this type.

Closed-loop training of attention with real-time brain imaging.

Lapses of attention can have negative consequences, including accidents and lost productivity. Here we used closed-loop neurofeedback to improve sustained attention abilities and reduce the frequency of lapses. During a sustained attention task, the focus of attention was monitored in real time with multivariate pattern analysis of whole-brain neuroimaging data. When indicators of an attentional lapse were detected in the brain, we gave human participants feedback by making the task more difficult. Behavioral performance improved after one training session, relative to control participants who received feedback from other participants' brains. This improvement was largest when feedback carried information from a frontoparietal attention network. A neural consequence of training was that the basal ganglia and ventral temporal cortex came to represent attentional states more distinctively. These findings suggest that attentional failures do not reflect an upper limit on cognitive potential and that attention can be trained with appropriate feedback about neural signals.

Neurocognitive therapeutics: from concept to application in the treatment of negative attention bias.

There is growing interest in the use of neuroimaging for the direct treatment of mental illness. Here, we present a new framework for such treatment, neurocognitive therapeutics. What distinguishes neurocognitive therapeutics from prior approaches is the use of precise brain-decoding techniques within a real-time feedback system, in order to adapt treatment online and tailor feedback to individuals' needs. We report an initial feasibility study that uses this framework to alter negative attention bias in a small number of patients experiencing significant mood symptoms. The results are consistent with the promise of neurocognitive therapeutics to improve mood symptoms and alter brain networks mediating attentional control. Future work should focus on optimizing the approach, validating its effectiveness, and expanding the scope of targeted disorders.

Environmental influences on neural systems of relational complexity.

Constructivist learning theory contends that we construct knowledge by experience and that environmental context influences learning. To explore this principle, we examined the cognitive process relational complexity (RC), defined as the number of visual dimensions considered during problem solving on a matrix reasoning task and a well-documented measure of mature reasoning capacity. We sought to determine how the visual environment influences RC by examining the influence of color and visual contrast on RC in a neuroimaging task. To specify the contributions of sensory demand and relational integration to reasoning, our participants performed a non-verbal matrix task comprised of color, no-color line, or black-white visual contrast conditions parametrically varied by complexity (relations 0, 1, 2). The use of matrix reasoning is ecologically valid for its psychometric relevance and for its potential to link the processing of psychophysically specific visual properties with various levels of RC during reasoning. The role of these elements is important because matrix tests assess intellectual aptitude based on these seemingly context-less exercises. This experiment is a first step toward examining the psychophysical underpinnings of performance on these types of problems. The importance of this is increased in light of recent evidence that intelligence can be linked to visual discrimination. We submit three main findings. First, color and black-white visual contrast (BWVC) add demand at a basic sensory level, but contributions from color and from BWVC are dissociable in cortex such that color engages a "reasoning heuristic" and BWVC engages a "sensory heuristic." Second, color supports contextual sense-making by boosting salience resulting in faster problem solving. Lastly, when visual complexity reaches 2-relations, color and visual contrast relinquish salience to other dimensions of problem solving.