Benefits of spaced learning are predicted by re-encoding of past experience in ventromedial prefrontal cortex.

More than a century of research shows that spaced learning improves long-term memory. Yet, there remains debate concerning why. A major limitation to resolving theoretical debates is the lack of evidence for how neural representations change as a function of spacing. Here, leveraging a massive-scale 7T human fMRI dataset, we tracked neural representations and behavioral expressions of memory as participants viewed thousands of natural scene images that repeated at lags ranging from seconds to many months. We show that spaced learning increases the similarity of human ventromedial prefrontal cortex representations across stimulus encounters and, critically, these increases parallel and predict the behavioral benefits of spacing. Additionally, we show that these spacing benefits critically depend on remembering and, in turn, 're-encoding' past experience. Collectively, our findings provide fundamental insight into how spaced learning influences neural representations and why spacing is beneficial.

Recall as a Window into Hippocampally Defined Events.

We experience the present as a continuous stream of information, but often experience the past in parcels of unique events or episodes. Decades of research have helped to articulate how we perform this event segmentation in the moment, as well as how events and their boundaries influence what we later remember. More recently, neuroscientific research has suggested that the hippocampus plays a role at critical moments during event formation alongside its established role in enabling subsequent recall. Here, we review and explore the relationship between event processing and recall with the perspective that it can be uniquely characterized by the contributions of the hippocampus and its interactions with the rest of the brain. Specifically, we highlight a growing number of empirical studies suggesting that the hippocampus is important for processing events that have just ended, bridging the gap between the prior and current event, and influencing the contents and trajectories of recalled information. We also catalogue and summarize the multifaceted sets of findings concerning how recall is influenced by event structure. Lastly, we discuss several exciting directions for future research and how our understanding of events might be enriched by characterizing them in terms of the operations of different regions of the brain.

Adding Meaning to Memories: How Parietal Cortex Combines Semantic Content with Episodic Experience.

Neuroimaging studies of human memory have consistently found that univariate responses in parietal cortex track episodic experience with stimuli (whether stimuli are 'old' or 'new'). More recently, pattern-based fMRI studies have shown that parietal cortex also carries information about the semantic content of remembered experiences. However, it is not well understood how memory-based and content-based signals are integrated within parietal cortex. Here, in humans (males and females), we used voxel-wise encoding models and a recognition memory task to predict the fMRI activity patterns evoked by complex natural scene images based on (1) the episodic history and (2) the semantic content of each image. Models were generated and compared across distinct subregions of parietal cortex and for occipitotemporal cortex. We show that parietal and occipitotemporal regions each encode memory and content information, but they differ in how they combine this information. Among parietal subregions, angular gyrus was characterized by robust and overlapping effects of memory and content. Moreover, subject-specific semantic tuning functions revealed that successful recognition shifted the amplitude of tuning functions in angular gyrus but did not change the selectivity of tuning. In other words, effects of memory and content were additive in angular gyrus. This pattern of data contrasted with occipitotemporal cortex where memory and content effects were interactive: memory effects were preferentially expressed by voxels tuned to the content of a remembered image. Collectively, these findings provide unique insight into how parietal cortex combines information about episodic memory and semantic content.SIGNIFICANCE STATEMENT Neuroimaging studies of human memory have identified multiple brain regions that not only carry information about "whether" a visual stimulus is successfully recognized but also "what" the content of that stimulus includes. However, a fundamental and open question concerns how the brain integrates these two types of information (memory and content). Here, using a powerful combination of fMRI analysis methods, we show that parietal cortex, particularly the angular gyrus, robustly combines memory- and content-related information, but these two forms of information are represented via additive, independent signals. In contrast, memory effects in high-level visual cortex critically depend on (and interact with) content representations. Together, these findings reveal multiple and distinct ways in which the brain combines memory- and content-related information.

Re-expression of CA1 and entorhinal activity patterns preserves temporal context memory at long timescales.

Converging, cross-species evidence indicates that memory for time is supported by hippocampal area CA1 and entorhinal cortex. However, limited evidence characterizes how these regions preserve temporal memories over long timescales (e.g., months). At long timescales, memoranda may be encountered in multiple temporal contexts, potentially creating interference. Here, using 7T fMRI, we measured CA1 and entorhinal activity patterns as human participants viewed thousands of natural scene images distributed, and repeated, across many months. We show that memory for an image's original temporal context was predicted by the degree to which CA1/entorhinal activity patterns from the first encounter with an image were re-expressed during re-encounters occurring minutes to months later. Critically, temporal memory signals were dissociable from predictors of recognition confidence, which were carried by distinct medial temporal lobe expressions. These findings suggest that CA1 and entorhinal cortex preserve temporal memories across long timescales by coding for and reinstating temporal context information.

Perception and memory retrieval states are reflected in distributed patterns of background functional connectivity.

The same visual input can serve as the target of perception or as a trigger for memory retrieval depending on whether cognitive processing is externally oriented (perception) or internally oriented (memory retrieval). While numerous human neuroimaging studies have characterized how visual stimuli are differentially processed during perception versus memory retrieval, perception and memory retrieval may also be associated with distinct neural states that are independent of stimulus-evoked neural activity. Here, we combined human fMRI with full correlation matrix analysis (FCMA) to reveal potential differences in "background" functional connectivity across perception and memory retrieval states. We found that perception and retrieval states could be discriminated with high accuracy based on patterns of connectivity across (1) the control network, (2) the default mode network (DMN), and (3) retrosplenial cortex (RSC). In particular, clusters in the control network increased connectivity with each other during the perception state, whereas clusters in the DMN were more strongly coupled during the retrieval state. Interestingly, RSC switched its coupling between networks as the cognitive state shifted from retrieval to perception. Finally, we show that background connectivity (1) was fully independent from stimulus-related variance in the signal and, further, (2) captured distinct aspects of cognitive states compared to traditional classification of stimulus-evoked responses. Together, our results reveal that perception and memory retrieval are associated with sustained cognitive states that manifest as distinct patterns of connectivity among large-scale brain networks.

Correspondence of functional connectivity gradients across human isocortex, cerebellum, and hippocampus.

Gradient mapping is an important technique to summarize high dimensional biological features as low dimensional manifold representations in exploring brain structure-function relationships at various levels of the cerebral cortex. While recent studies have characterized the major gradients of functional connectivity in several brain structures using this technique, very few have systematically examined the correspondence of such gradients across structures under a common systems-level framework. Using resting-state functional magnetic resonance imaging, here we show that the organizing principles of the isocortex, and those of the cerebellum and hippocampus in relation to the isocortex, can be described using two common functional gradients. We suggest that the similarity in functional connectivity gradients across these structures can be meaningfully interpreted within a common computational framework based on the principles of predictive processing. The present results, and the specific hypotheses that they suggest, represent an important step toward an integrative account of brain function.

A Computational Neural Model for Mapping Degenerate Neural Architectures.

Degeneracy in biological systems refers to a many-to-one mapping between physical structures and their functional (including psychological) outcomes. Despite the ubiquity of the phenomenon, traditional analytical tools for modeling degeneracy in neuroscience are extremely limited. In this study, we generated synthetic datasets to describe three situations of degeneracy in fMRI data to demonstrate the limitations of the current univariate approach. We describe a novel computational approach for the analysis referred to as neural topographic factor analysis (NTFA). NTFA is designed to capture variations in neural activity across task conditions and participants. The advantage of this discovery-oriented approach is to reveal whether and how experimental trials and participants cluster into task conditions and participant groups. We applied NTFA on simulated data, revealing the appropriate degeneracy assumption in all three situations and demonstrating NTFA's utility in uncovering degeneracy. Lastly, we discussed the importance of testing degeneracy in fMRI data and the implications of applying NTFA to do so.

A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence.

Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. Here we present the Natural Scenes Dataset (NSD), in which high-resolution functional magnetic resonance imaging responses to tens of thousands of richly annotated natural scenes were measured while participants performed a continuous recognition task. To optimize data quality, we developed and applied novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we used NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality and breadth, NSD opens new avenues of inquiry in cognitive neuroscience and artificial intelligence.

BrainIAK: The Brain Imaging Analysis Kit.

Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEMs), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high-performance compute (HPC) clusters, and the same code can be se amlessly transferred from a laptop to a cluster. For each of the aforementioned techniques, we describe the data analysis problem that the technique is meant to solve and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.

Comparing supervised and unsupervised approaches to emotion categorization in the human brain, body, and subjective experience.

Machine learning methods provide powerful tools to map physical measurements to scientific categories. But are such methods suitable for discovering the ground truth about psychological categories? We use the science of emotion as a test case to explore this question. In studies of emotion, researchers use supervised classifiers, guided by emotion labels, to attempt to discover biomarkers in the brain or body for the corresponding emotion categories. This practice relies on the assumption that the labels refer to objective categories that can be discovered. Here, we critically examine this approach across three distinct datasets collected during emotional episodes-measuring the human brain, body, and subjective experience-and compare supervised classification solutions with those from unsupervised clustering in which no labels are assigned to the data. We conclude with a set of recommendations to guide researchers towards meaningful, data-driven discoveries in the science of emotion and beyond.

The power of predictions: An emerging paradigm for psychological research.

The last two decades of neuroscience research has produced a growing number of studies that suggest the various psychological phenomena are produced by predictive processes in the brain. When considered together, these studies form a coherent, neurobiologically-inspired research program for guiding psychological research about the mind and behavior. In this paper, we briefly consider the common assumptions and hypotheses that unify an emerging framework and discuss its ramifications, both for improving the replicability and robustness of psychological research and for innovating psychological theory by suggesting an alternative ontology of the human mind.

What Is Memory-Guided Attention? How Past Experiences Shape Selective Visuospatial Attention in the Present.

What controls our attention? It is historically thought that there are two primary factors that determine selective attention: the perceptual salience of the stimuli and the goals based on the task at hand. However, this distinction doesn't neatly capture the varied ways our past experience can influence our ongoing mental processing. In this chapter, we aim to describe how past experience can be systematically characterized by different types of memory, and we outline experimental evidence suggesting how attention can then be guided by each of these different memory types. We highlight findings from human behavioral, neuroimaging, and neuropsychological work from the perspective of two related frameworks of human memory: the multiple memory systems (MMS) framework and the neural processing (NP) framework. The MMS framework underscores how memory can be separated based on consciousness (declarative and non-declarative memory), while the NP framework emphasizes different forms of memory as reflective of different brain processing modes (rapid encoding of flexible associations, slow encoding of rigid associations, and rapid encoding of single or unitized items). We describe how memory defined by these frameworks can guide our attention, even when they do not directly relate to perceptual salience or the goals concerning the current task. We close by briefly discussing theoretical implications as well as some interesting avenues for future research.

When past is present: Substitutions of long-term memory for sensory evidence in perceptual judgments.

When perception is underdetermined by current sensory inputs, memories for related experiences in the past might fill in missing detail. To evaluate this possibility, we measured the likelihood of relying on long-term memory versus sensory evidence when judging the appearance of an object near the threshold of awareness. Specifically, we associated colors with shapes in long-term memory and then presented the shapes again later in unrelated colors and had observers judge the appearance of the new colors. We found that responses were well characterized as a bimodal mixture of original and current-color representations (vs. an integrated unimodal representation). That is, although irrelevant to judgments of the current color, observers occasionally anchored their responses on the original colors in memory. Moreover, the likelihood of such memory substitutions increased when sensory input was degraded. In fact, they occurred even in the absence of sensory input when observers falsely reported having seen something. Thus, although perceptual judgments intuitively seem to reflect the current state of the environment, they can also unknowingly be dictated by past experiences.

Biased Competition during Long-term Memory Formation.

A key task for the brain is to determine which pieces of information are worth storing in memory. To build a more complete representation of the environment, memory systems may prioritize new information that has not already been stored. Here, we propose a mechanism that supports this preferential encoding of new information, whereby prior experience attenuates neural activity for old information that is competing for processing. We evaluated this hypothesis with fMRI by presenting a series of novel stimuli concurrently with repeated stimuli at different spatial locations in Experiment 1 and from different visual categories (i.e., faces and scenes) in Experiment 2. Subsequent memory for the novel stimuli could be predicted from the reduction in activity in ventral temporal cortex for the accompanying repeated stimuli. This relationship was eliminated in control conditions where the competition during encoding came from another novel stimulus. These findings reveal how prior experience adaptively guides learning toward new aspects of the environment.

Electrocorticography reveals the temporal dynamics of posterior parietal cortical activity during recognition memory decisions.

Theories of the neurobiology of episodic memory predominantly focus on the contributions of medial temporal lobe structures, based on extensive lesion, electrophysiological, and imaging evidence. Against this backdrop, functional neuroimaging data have unexpectedly implicated left posterior parietal cortex (PPC) in episodic retrieval, revealing distinct activation patterns in PPC subregions as humans make memory-related decisions. To date, theorizing about the functional contributions of PPC has been hampered by the absence of information about the temporal dynamics of PPC activity as retrieval unfolds. Here, we leveraged electrocorticography to examine the temporal profile of high gamma power (HGP) in dorsal PPC subregions as participants made old/new recognition memory decisions. A double dissociation in memory-related HGP was observed, with activity in left intraparietal sulcus (IPS) and left superior parietal lobule (SPL) differing in time and sign for recognized old items (Hits) and correctly rejected novel items (CRs). Specifically, HGP in left IPS increased for Hits 300-700 ms poststimulus onset, and decayed to baseline ∼200 ms preresponse. By contrast, HGP in left SPL increased for CRs early after stimulus onset (200-300 ms) and late in the memory decision (from 700 ms to response). These memory-related effects were unique to left PPC, as they were not observed in right PPC. Finally, memory-related HGP in left IPS and SPL was sufficiently reliable to enable brain-based decoding of the participant's memory state at the single-trial level, using multivariate pattern classification. Collectively, these data provide insights into left PPC temporal dynamics as humans make recognition memory decisions.

Increased functional connectivity between dorsal posterior parietal and ventral occipitotemporal cortex during uncertain memory decisions.

Retrieval of episodic memories is a multi-component act that relies on numerous operations ranging from processing the retrieval cue, evaluating retrieved information, and selecting the appropriate response given the demands of the task. Motivated by a rich functional neuroimaging literature, recent theorizing about various computations at retrieval has focused on the role of posterior parietal cortex (PPC). In a potentially promising line of research, recent neuroimaging findings suggest that different subregions of dorsal PPC respond distinctly to different aspects of retrieval decisions, suggesting that better understanding of their contributions might shed light on the component processes of retrieval. In an attempt to understand the basic operations performed by dorsal PPC, we used functional MRI and functional connectivity analyses to examine how activation in, and connectivity between, dorsal PPC and ventral temporal regions representing retrieval cues varies as a function of retrieval decision uncertainty. Specifically, participants made a five-point recognition confidence judgment for a series of old and new visually presented words. Consistent with prior studies, memory-related activity patterns dissociated across left dorsal PPC subregions, with activity in the lateral IPS tracking the degree to which participants perceived an item to be old, whereas activity in the SPL increased as a function of decision uncertainty. Importantly, whole-brain functional connectivity analyses further revealed that SPL activity was more strongly correlated with that in the visual word-form area during uncertain relative to certain decisions. These data suggest that the involvement of SPL during episodic retrieval reflects, at least in part, the processing of the retrieval cue, perhaps in service of attempts to increase the mnemonic evidence elicited by the cue.

Functional MRI-based lie detection: scientific and societal challenges.

Functional MRI (fMRI)-based lie detection has been marketed as a tool for enhancing personnel selection, strengthening national security and protecting personal reputations, and at least three US courts have been asked to admit the results of lie detection scans as evidence during trials. How well does fMRI-based lie detection perform, and how should the courts, and society more generally, respond? Here, we address various questions ­ some of which are based on a meta-analysis of published studies ­ concerning the scientific state of the art in fMRI-based lie detection and its legal status, and discuss broader ethical and societal implications. We close with three general policy recommendations.

Functional heterogeneity in posterior parietal cortex across attention and episodic memory retrieval.

While attention is critical for event memory, debate has arisen regarding the extent to which posterior parietal cortex (PPC) activation during episodic retrieval reflects engagement of PPC-mediated mechanisms of attention. Here, we directly examined the relationship between attention and memory, within and across subjects, using functional magnetic resonance imaging attention-mapping and episodic retrieval paradigms. During retrieval, 4 functionally dissociable PPC regions were identified. Specifically, 2 PPC regions positively tracked retrieval outcomes: lateral intraparietal sulcus (latIPS) indexed graded item memory strength, whereas angular gyrus (AnG) tracked recollection. By contrast, 2 other PPC regions demonstrated nonmonotonic relationships with retrieval: superior parietal lobule (SPL) tracked retrieval reaction time, consistent with a graded engagement of top-down attention, whereas temporoparietal junction displayed a complex pattern of below-baseline retrieval activity, perhaps reflecting disengagement of bottom-up attention. Analyses of retrieval effects in PPC topographic spatial attention maps (IPS0-IPS5; SPL1) revealed that IPS5 and SPL1 exhibited a nonmonotonic relationship with retrieval outcomes resembling that in the SPL region, further suggesting that SPL activation during retrieval reflects top-down attention. While demands on PPC attention mechanisms vary during retrieval attempts, the present functional parcellation of PPC indicates that 2 additional mechanisms (mediated by latIPS and AnG) positively track retrieval outcomes.

Memory-guided attention: control from multiple memory systems.

Attention is strongly influenced by both external stimuli and internal goals. However, this useful dichotomy does not readily capture the ubiquitous and often automatic contribution of past experience stored in memory. We review recent evidence about how multiple memory systems control attention, consider how such interactions are manifested in the brain, and highlight how this framework for 'memory-guided attention' might help systematize previous findings and guide future research.