Differential Mnemonic Contributions of Cortical Representations during Encoding and Retrieval.

Several recent fMRI studies of episodic and working memory representations converge on the finding that visual information is most strongly represented in occipito-temporal cortex during the encoding phase but in parietal regions during the retrieval phase. It has been suggested that this location shift reflects a change in the content of representations, from predominantly visual during encoding to primarily semantic during retrieval. Yet, direct evidence on the nature of encoding and retrieval representations is lacking. It is also unclear how the representations mediating the encoding-retrieval shift contribute to memory performance. To investigate these two issues, in the current fMRI study, participants encoded pictures (e.g., picture of a cardinal) and later performed a word recognition test (e.g., word "cardinal"). Representational similarity analyses examined how visual (e.g., red color) and semantic representations (e.g., what cardinals eat) support successful encoding and retrieval. These analyses revealed two novel findings. First, successful memory was associated with representational changes in cortical location (from occipito-temporal at encoding to parietal at retrieval) but not with changes in representational content (visual vs. semantic). Thus, the representational encoding-retrieval shift cannot be easily attributed to a change in the nature of representations. Second, in parietal regions, stronger representations predicted encoding failure but retrieval success. This encoding-retrieval "flip" in representations mimics the one previously reported in univariate activation studies. In summary, by answering important questions regarding the content and contributions to the performance of the representations mediating the encoding-retrieval shift, our findings clarify the neural mechanisms of this intriguing phenomenon.

Surprise!-Clarifying the link between insight and prediction error.

The AHA experience, a moment of deep understanding during insightful problem-solving involving feelings of certainty, pleasure, and surprise, has captivated psychologists for more than a century. Recently, a new theoretical framework has proposed a link between the AHA experience and prediction error (PE), a popular concept in decision-making and reinforcement learning. This framework suggests that participants maintain a meta-cognitive prediction about the time it takes to solve a problem and the AHA experience arises when the problem is solved earlier than expected, resulting in a meta-cognitive PE. In our preregistered online study, we delved deeper into this idea, investigating whether prediction errors also pertain to participants' predictions regarding the solvability of the problem itself, and which dimension of the AHA experience aligns with the meta-cognitive PE. Utilizing verbal insight problems, we found a positive association between the AHA experience and the meta-cognitive PE, specifically in regards to problem solvability. Specifically, the element of surprise, a critical AHA dimension, emerged as a key indicator of the meta-cognitive PE, while other dimensions-such as pleasure, certainty, and suddenness-showed no signs for similar relationships, with suddenness exhibiting a negative correlation with meta-cognitive PE. This new finding provides further evidence that aspects of the AHA experience, surprise in particular, correspond to a meta-cognitive PE. The finding also underscores the multifaceted nature of this phenomenon, linking insights with learning theories and enhancing our understanding of this intriguing phenomenon.

Prediction error minimization as a common computational principle for curiosity and creativity.

We propose expanding the authors' shared novelty-seeking basis for creativity and curiosity by emphasizing an underlying computational principle: Minimizing prediction errors (mismatch between predictions and incoming data). Curiosity is tied to the anticipation of minimizing prediction errors through future, novel information, whereas creative AHA moments are connected to the actual minimization of prediction errors through current, novel information.

Connectivity analyses for task-based fMRI.

Functional connectivity is conventionally defined by measuring the similarity between brain signals from two regions. The technique has become widely adopted in the analysis of functional magnetic resonance imaging (fMRI) data, where it has provided cognitive neuroscientists with abundant information on how brain regions interact to support complex cognition. However, in the past decade the notion of "connectivity" has expanded in both the complexity and heterogeneity of its application to cognitive neuroscience, resulting in greater difficulty of interpretation, replication, and cross-study comparisons. In this paper, we begin with the canonical notions of functional connectivity and then introduce recent methodological developments that either estimate some alternative form of connectivity or extend the analytical framework, with the hope of bringing better clarity for cognitive neuroscience researchers.

Visual Recognition Memory of Scenes Is Driven by Categorical, Not Sensory, Visual Representations.

When we perceive a scene, our brain processes various types of visual information simultaneously, ranging from sensory features, such as line orientations and colors, to categorical features, such as objects and their arrangements. Whereas the role of sensory and categorical visual representations in predicting subsequent memory has been studied using isolated objects, their impact on memory for complex scenes remains largely unknown. To address this gap, we conducted an fMRI study in which female and male participants encoded pictures of familiar scenes (e.g., an airport picture) and later recalled them, while rating the vividness of their visual recall. Outside the scanner, participants had to distinguish each seen scene from three similar lures (e.g., three airport pictures). We modeled the sensory and categorical visual features of multiple scenes using both early and late layers of a deep convolutional neural network. Then, we applied representational similarity analysis to determine which brain regions represented stimuli in accordance with the sensory and categorical models. We found that categorical, but not sensory, representations predicted subsequent memory. In line with the previous result, only for the categorical model, the average recognition performance of each scene exhibited a positive correlation with the average visual dissimilarity between the item in question and its respective lures. These results strongly suggest that even in memory tests that ostensibly rely solely on visual cues (such as forced-choice visual recognition with similar distractors), memory decisions for scenes may be primarily influenced by categorical rather than sensory representations.

Hippocampal Functions Modulate Transfer-Appropriate Cortical Representations Supporting Subsequent Memory.

The hippocampus plays a central role as a coordinate system or index of information stored in neocortical loci. Nonetheless, it remains unclear how hippocampal processes integrate with cortical information to facilitate successful memory encoding. Thus, the goal of the current study was to identify specific hippocampal-cortical interactions that support object encoding. We collected fMRI data while 19 human participants (7 female and 12 male) encoded images of real-world objects and tested their memory for object concepts and image exemplars (i.e., conceptual and perceptual memory). Representational similarity analysis revealed robust representations of visual and semantic information in canonical visual (e.g., occipital cortex) and semantic (e.g., angular gyrus) regions in the cortex, but not in the hippocampus. Critically, hippocampal functions modulated the mnemonic impact of cortical representations that are most pertinent to future memory demands, or transfer-appropriate representations Subsequent perceptual memory was best predicted by the strength of visual representations in ventromedial occipital cortex in coordination with hippocampal activity and pattern information during encoding. In parallel, subsequent conceptual memory was best predicted by the strength of semantic representations in left inferior frontal gyrus and angular gyrus in coordination with either hippocampal activity or semantic representational strength during encoding. We found no evidence for transfer-incongruent hippocampal-cortical interactions supporting subsequent memory (i.e., no hippocampal interactions with cortical visual/semantic representations supported conceptual/perceptual memory). Collectively, these results suggest that diverse hippocampal functions flexibly modulate cortical representations of object properties to satisfy distinct future memory demands.Significance Statement The hippocampus is theorized to index pieces of information stored throughout the cortex to support episodic memory. Yet how hippocampal processes integrate with cortical representation of stimulus information remains unclear. Using fMRI, we examined various forms of hippocampal-cortical interactions during object encoding in relation to subsequent performance on conceptual and perceptual memory tests. Our results revealed novel hippocampal-cortical interactions that utilize semantic and visual representations in transfer-appropriate manners: conceptual memory supported by hippocampal modulation of frontoparietal semantic representations, and perceptual memory supported by hippocampal modulation of occipital visual representations. These findings provide important insights into the neural mechanisms underlying the formation of information-rich episodic memory and underscore the value of studying the flexible interplay between brain regions for complex cognition.

Endel Tulving (1927-2023).

Explorer, innovator, and theorist of human memory.

The influence of insight on risky decision making and nucleus accumbens activation.

During insightful problem solving, the solution appears unexpectedly and is accompanied by the feeling of an AHA!. Research suggests that this affective component of insight can have consequences beyond the solution itself by motivating future behavior, such as risky (high reward and high uncertainty) decision making. Here, we investigate the behavioral and neural support for the motivational role of AHA in decision making involving monetary choices. The positive affect of the AHA! experience has been linked to internal reward. Reward in turn has been linked to dopaminergic signal transmission in the Nucleus Accumbens (NAcc) and risky decision making. Therefore, we hypothesized that insight activates reward-related brain areas, modulating risky decision making. We tested this hypothesis in two studies. First, in a pre-registered online study (Study 1), we demonstrated the behavioral effect of insight-related increase in risky decision making using a visual Mooney identification paradigm. Participants were more likely to choose the riskier monetary payout when they had previously solved the Mooney image with high compared to low accompanied AHA!. Second, in an fMRI study (Study 2), we measured the effects of insight on NAcc activity using a similar Mooney identification paradigm to the one of Study 1. Greater NAcc activity was found when participants solved the Mooney image with high vs low AHA!. Taken together, our results link insight to enhanced NAcc activity and a preference for high but uncertain rewards, suggesting that insight enhances reward-related brain areas possibly via dopaminergic signal transmission, promoting risky decision making.

Effects of Aging on Successful Object Encoding: Enhanced Semantic Representations Compensate for Impaired Visual Representations.

Although episodic memory and visual processing decline substantially with healthy aging, semantic knowledge is generally spared. There is evidence that older adults' spared semantic knowledge can support episodic memory. Here, we used functional magnetic resonance imaging (fMRI) combined with representational similarity analyses (RSAs) to examine how novel visual and preexisting semantic representations at encoding predict subjective memory vividness at retrieval. Eighteen young and seventeen older adults (female and male participants) encoded images of objects during fMRI scanning and recalled these images while rating the vividness of their memories. After scanning, participants discriminated between studied images and similar lures. RSA based on a deep convolutional neural network and normative concept feature data were used to link patterns of neural activity during encoding to visual and semantic representations. Relative to young adults, the specificity of activation patterns for visual features was reduced in older adults, consistent with dedifferentiation. However, the specificity of activation patterns for semantic features was enhanced in older adults, consistent with hyperdifferentiation. Despite dedifferentiation, visual representations in early visual cortex (EVC) predicted high memory vividness in both age groups. In contrast, semantic representations in lingual gyrus (LG) and fusiform gyrus (FG) were associated with high memory vividness only in the older adults. Intriguingly, data suggests that older adults with lower specificity of visual representations in combination with higher specificity of semantic representations tended to rate their memories as more vivid. Our findings suggest that memory vividness in aging relies more on semantic representations over anterior regions, potentially compensating for age-related dedifferentiation of visual information in posterior regions.SIGNIFICANCE STATEMENT Normal aging is associated with impaired memory for events while semantic knowledge might even improve. We investigated the effects of aging on the specificity of visual and semantic information in the brain when viewing common objects and how this information enables subsequent memory vividness for these objects. Using functional magnetic resonance imaging (fMRI) combined with modeling of the stimuli we found that visual information was represented with less specificity in older than young adults while still supporting memory vividness. In contrast semantic information supported memory vividness only in older adults and especially in those individuals that had the lowest specificity of visual information. These findings provide evidence for a spared semantic memory system increasingly recruited to compensate for degraded visual representations in older age.

Neural retrieval processes occur more rapidly for visual mental images that were previously encoded with high-vividness.

Visual mental imagery refers to our ability to experience visual images in the absence of sensory stimulation. Studies have shown that visual mental imagery can improve episodic memory. However, we have limited understanding of the neural mechanisms underlying this improvement. Using electroencephalography, we examined the neural processes associated with the retrieval of previously generated visual mental images, focusing on how the vividness at generation can modulate retrieval processes. Participants viewed word stimuli referring to common objects, forming a visual mental image of each word and rating the vividness of the mental image. This was followed by a surprise old/new recognition task. We compared retrieval performance for items rated as high- versus low-vividness at encoding. High-vividness items were retrieved with faster reaction times and higher confidence ratings in the memory judgment. While controlling for confidence, neural measures indicated that high-vividness items produced an earlier decrease in alpha-band activity at retrieval compared with low-vividness items, suggesting an earlier memory reinstatement. Even when low-vividness items were remembered with high confidence, they were not retrieved as quickly as high-vividness items. These results indicate that when highly vivid mental images are encoded, the speed of their retrieval occurs more rapidly, relative to low-vivid items.

A framework for concepts of reserve and resilience in aging.

The study of factors, across species, that allow some individuals to age more successfully than others has important implications for individual wellbeing as well as health education, policy and intervention. Design of studies and communication across investigators in this area has been hampered by a diversity of terminology. The Collaboratory on Research Definitions for Reserve and Resilience in Cognitive Aging and Dementia was funded by the National Institute on Aging and established in 2019 as a 3-year process of developing consensus definitions and research guidelines. The proposed Framework is based on an iterative process including 3 annual Workshops, focused workgroups, and input from numerous international investigators. It suggests the overarching term: resilience, and presents operational definitions for 3 concepts: cognitive reserve, brain maintenance, and brain reserve. Twelve pilot studies that integrate these definitions are presented. The use of a common vocabulary and operational definitions will facilitate even greater progress in understanding the factors that are associated with successful aging.

Assessing creativity independently of language: A language-independent remote associate task (LI-RAT).

Most creativity measures are either complex or language-dependent, hindering cross-cultural creativity assessment. We have therefore developed and tested a simple, language-independent insight task based on pictures in the style of the widely used verbal remote associate task (RAT). We demonstrate that the language-independent RAT (LI-RAT) allows assessment of different aspects of insight across large samples with different languages. It also correlates with other creativity and general problem-solving tasks. The entire stimulus set, including its preliminary normative data, is made freely available. This information can be used to select items based on accuracy, mean solution time, likelihood to produce an insight, or conceptual and perceptual similarity between the pictures per item.

The influence of imagery vividness and internally-directed attention on the neural mechanisms underlying the encoding of visual mental images into episodic memory.

Attention can be directed externally toward sensory information or internally toward self-generated information. Using electroencephalography (EEG), we investigated the attentional processes underlying the formation and encoding of self-generated mental images into episodic memory. Participants viewed flickering words referring to common objects and were tasked with forming visual mental images of the objects and rating their vividness. Subsequent memory for the presented object words was assessed using an old-new recognition task. Internally-directed attention during image generation was indexed as a reduction in steady-state visual evoked potentials (SSVEPs), oscillatory EEG responses at the frequency of a flickering stimulus. The results yielded 3 main findings. First, SSVEP power driven by the flickering word stimuli decreased as subjects directed attention internally to form the corresponding mental image. Second, SSVEP power returned to pre-imagery baseline more slowly for low- than high-vividness later remembered items, suggesting that longer internally-directed attention is required to generate subsequently remembered low-vividness images. Finally, the event-related-potential difference due to memory was more sustained for subsequently remembered low- versus high-vividness items, suggesting that additional conceptual processing may have been needed to remember the low-vividness visual images. Taken together, the results clarify the neural mechanisms supporting the encoding of self-generated information.

A functional neuroimaging investigation of Moral Foundations Theory.

Moral Foundations Theory (MFT) posits that the human mind contains modules (or "foundations") that are functionally specialized to moralize unique dimensions of the social world: Authority, Loyalty, Purity, Harm, Fairness, and Liberty. Despite this strong claim about cognitive architecture, it is unclear whether neural activity during moral reasoning exhibits this modular structure. Here, we use spatiotemporal partial least squares correlation (PLSC) analyses of fMRI data collected during judgments of foundation-specific violations to investigate whether MFT's cognitive modularity claim extends to the neural level. A mean-centered PLSC analysis returned two latent variables that differentiated between social norm and moral foundation violations, functionally segregated Purity, Loyalty, Physical Harm, and Fairness from the other foundations, and suggested that Authority has a different neural basis than other binding foundations. Non-rotated PLSC analyses confirmed that neural activity distinguished social norm from moral foundation violations, and distinguished individualizing and binding moral foundations if Authority is dropped from the binding foundations. Purity violations were persistently associated with amygdala activity, whereas moral foundation violations more broadly tended to engage the default network. Our results constitute partial evidence for neural modularity and motivate further research on the novel groupings identified by the PLSC analyses.

Neural Mechanisms of Perceiving and Subsequently Recollecting Emotional Facial Expressions in Young and Older Adults.

It is known that emotional facial expressions modulate the perception and subsequent recollection of faces and that aging alters these modulatory effects. Yet, the underlying neural mechanisms are not well understood, and they were the focus of the current fMRI study. We scanned healthy young and older adults while perceiving happy, neutral, or angry faces paired with names. Participants were then provided with the names of the faces and asked to recall the facial expression of each face. fMRI analyses focused on the fusiform face area (FFA), the posterior superior temporal sulcus (pSTS), the OFC, the amygdala (AMY), and the hippocampus (HC). Univariate activity, multivariate pattern (MVPA), and functional connectivity analyses were performed. The study yielded two main sets of findings. First, in pSTS and AMY, univariate activity and MVPA discrimination during the processing of facial expressions were similar in young and older adults, whereas in FFA and OFC, MVPA discriminated facial expressions less accurately in older than young adults. These findings suggest that facial expression representations in FFA and OFC reflect age-related dedifferentiation and positivity effect. Second, HC-OFC connectivity showed subsequent memory effects (SMEs) for happy expressions in both age groups, HC-FFA connectivity exhibited SMEs for happy and neutral expressions in young adults, and HC-pSTS interactions displayed SMEs for happy expressions in older adults. These results could be related to compensatory mechanisms and positivity effects in older adults. Taken together, the results clarify the effects of aging on the neural mechanisms in perceiving and encoding facial expressions.

Between automatic and control processes: How relationships between problem elements interact to facilitate or impede insight.

Solving a problem requires relating the pieces of information available to each other and to the solution. We investigated how the strength of these relationships determines the likelihood of solving insight tasks based on remote associates. In these tasks, the solver is provided with several cues (e.g., drop, coat, summer) and has to find the solution that matches those cues (e.g., rain). We measured the semantic similarity between the cues and the solution (cue-solution similarity) as well as between cues (cue-cue similarity). We assume those relationships modulate two basic processes underlying insight problem-solving. First, there is an automatic activation process whereby conceptual activation spreads across a semantic network from each cue node to their associated nodes, potentially reaching the node of the solution. Thus, in general, the higher cue-solution similarity, the more likely the solution will be found (Prediction 1). Second, there is a controlled search process focused on an area in semantic space whose radius depends on competing cue-cue similarity. High cue-cue similarity will bias a search for the solution close to the provided cues because the associated nodes shared by both cues are highly coactivated. Therefore, high cue-cue similarity will have a beneficial effect when the cue-solution similarity is high but a detrimental effect when cue-solution similarity is low (Prediction 2). Our two predictions were confirmed using both verbal and pictorial remote association tasks, supporting the view that insight is dependent on an interaction of meaningful relationships between cues and solutions, and clarify the mechanisms of insight problem solving in remote associates.

Network-based rTMS to modulate working memory: The difficult choice of effective parameters for online interventions.

BACKGROUND: Online repetitive transcranialmagnetic stimulation (rTMS) has been shown to modulate working memory (WM) performance in a site-specific manner, with behavioral improvements due to stimulation of the dorsolateral prefrontal cortex (DLPFC), and impairment from stimulation to the lateral parietal cortex (LPC). Neurobehavioral studies have demonstrated that subprocesses of WM allowing for the maintenance and manipulation of information in the mind involve unique cortical networks. Despite promising evidence of modulatory effects of rTMS on WM, no studies have yet demonstrated distinct modulatory control of these two subprocesses. The current study therefore sought to explore this possibility through site-specific stimulation during an online task invoking both skills. METHODS: Twenty-nine subjects completed a 4-day protocol, in which active or sham 5Hz rTMS was applied over the DLPFC and LPC in separate blocks of trials while participants performed tasks that required either maintenance alone, or both maintenance and manipulation (alphabetization) of information. Stimulation targets were defined individually based on fMRI activation and structural network properties. Stimulation amplitude was adjusted using electric field modeling to equate induced current in the target region across participants. RESULTS: Despite the use of advanced techniques, no significant differences or interactions between active and sham stimulation were found. Exploratory analyses testing stimulation amplitude, fMRI activation, and modal controllability showed nonsignificant but interesting trends with rTMS effects. CONCLUSION: While this study did not reveal any significant behavioral changes in WM, the results may point to parameters that contribute to positive effects, such as stimulation amplitude and functional activation.

Age-related dedifferentiation and hyperdifferentiation of perceptual and mnemonic representations.

Preliminary evidence indicates that occipito-temporal activation patterns for different visual stimuli are less distinct in older (OAs) than younger (YAs) adults, suggesting a dedifferentiation of visual representations with aging. Yet, it is unclear if this deficit (1) affects only sensory or also categorical aspects of representations during visual perception (perceptual representations), and (2) affects only perceptual or also mnemonic representations. To investigate these issues, we fMRI-scanned YAs and OAs viewing and then remembering visual scenes. First, using representational similarity analyses, we distinguished sensory vs. categorical features of perceptual representations. We found that, compared to YAs, sensory features in early visual cortex were less differentiated in OAs (i.e., age-related dedifferentiation), replicating previous research, whereas categorical features in anterior temporal lobe (ATL) were more differentiated in OAs. This is, to our knowledge, the first report of an age-related hyperdifferentiation. Second, we assessed the quality of mnemonic representations by measuring encoding-retrieval similarity (ERS) in activation patterns. We found that aging impaired mnemonic representations in early visual cortex and hippocampus but enhanced mnemonic representations in ATL. Thus, both perceptual and mnemonic representations in ATL were enhanced by aging. In sum, our findings suggest that aging impairs visual and mnemonic representations in posterior brain regions but enhances them in anterior regions.

Memory and Counterfactual Simulations for Past Wrongdoings Foster Moral Learning and Improvement.

In four studies, we investigated the role of remembering, reflecting on, and mutating personal past moral transgressions to learn from those moral mistakes and to form intentions for moral improvement. Participants reported having ruminated on their past wrongdoings, particularly their more severe transgressions, and they reported having frequently thought about morally better ways in which they could have acted instead (i.e., morally upward counterfactuals; Studies 1-3). The more that participants reported having mentally simulated morally better ways in which they could have acted, the stronger their intentions were to improve in the future (Studies 2 and 3). Implementing an experimental manipulation, we then found that making accessible a morally upward counterfactual after committing a moral transgression strengthened reported intentions for moral improvement-relative to resimulating the remembered event and considering morally worse ways in which they could have acted instead (Study 4). We discuss the implications of these results for competing theoretical views on the relationship between memory and morality and for functional theories of counterfactual thinking.