Building and Breaking the Chain: A Model of Reward Prediction Error Integration and Segmentation of Memory.

Prediction errors drive reinforcement learning and organize episodic memory into distinct contexts, but do these effects interact? Here, we review the roles of midbrain dopamine, the locus coeruleus, and the hippocampus in event cognition to propose and simulate the theoretical influence of two prediction error signals in integrating versus segmenting events in memory. We suggest that signed reward prediction errors can build mental models of reward environments, increasing the contextual similarity (integration) of experiences with stronger, more stable reward expectations. On the other hand, unsigned reward prediction errors can signal a new model of the environment, generating a contextual shift (segmentation) between experiences that crossed them. We moreover predicted that these differences in contextual similarity give rise to distinct patterns of temporal-order memory. We combined these ideas in a computational model to account for a seemingly paradoxical pattern of temporal-order memory where greater representational distance helps order memory within context but impairs it across contexts. We found that simulating signed reward prediction error integration and unsigned reward prediction error segmentation differentially enabled the model to perform associative chaining, which involved reactivating items between two tested probes to assist with sequential retrieval. In summary, our simulations provide a unifying explanation for the varied ways that neuromodulatory systems alter event cognition and memory.

Goal Shifts Structure Memories and Prioritize Event-defining Information in Memory.

Every day, we encounter far more information than we could possibly remember. Thus, our memory systems must organize and prioritize the details from an experience that can adaptively guide the storage and retrieval of specific episodic events. Prior work has shown that shifts in internal goal states can function as event boundaries, chunking experiences into distinct and memorable episodes. In addition, at short delays, memory for contextual information at boundaries has been shown to be enhanced compared with items within each event. However, it remains unclear if these memory enhancements are limited to features that signal a meaningful transition between events. To determine how changes in dynamic goal states influence the organization and content of long-term memory, we designed a 2-day experiment in which participants viewed a series of black-and-white objects surrounded by a color border on a two-by-two grid. The location of the object on the grid determined which of two tasks participants performed on a given trial. To examine if distinct types of goal shifts modulate the effects of event segmentation, we changed the border color, the task, or both after every four items in a sequence. We found that goal shifts influenced the temporal memory in a manner consistent with the formation of distinct events. However, for subjective memory representations in particular, these effects differed by the type of event boundary. Furthermore, to examine if goal shifts lead to the prioritization of goal-relevant features in longer lasting memories, we tested the source memory for each object's color and grid location both immediately and after a 24-hr delay. On the immediate test, boundaries enhanced the memory for all concurrent source features compared with nonboundary items, but only if those boundaries involved a goal shift. In contrast, after a delay, the source memory was selectively enhanced for the feature relevant to the goal shift. These findings suggest that goals can adaptively structure memories by prioritizing contextual features that define a unique episode in memory.

Emotional arousal lingers in time to bind discrete episodes in memory.

Temporal stability and change in neutral contexts can transform continuous experiences into distinct and memorable events. However, less is known about how shifting emotional states influence these memory processes, despite ample evidence that emotion impacts non-temporal aspects of memory. Here, we examined if emotional stimuli influence temporal memory for recent event sequences. Participants encoded lists of neutral images while listening to auditory tones. At regular intervals within each list, participants heard emotional positive, negative, or neutral sounds, which served as "emotional event boundaries" that divided each sequence into discrete events. Temporal order memory was tested for neutral item pairs that either spanned an emotional sound or were encountered within the same auditory event. Encountering a highly arousing event boundary led to faster response times for items encoded within the next event. Critically, we found that highly arousing sounds had different effects on binding ongoing versus ensuing sequential representations in memory. Specifically, highly arousing sounds were significantly more likely to enhance temporal order memory for ensuing information compared to information that spanned those boundaries, especially for boundaries with negative valence. These findings suggest that within aversive emotional contexts, fluctuations in arousal help shape the temporal organisation of events in memory.

Survival of the salient: Aversive learning rescues otherwise forgettable memories via neural reactivation and post-encoding hippocampal connectivity.

The effects of aversive events on memory are complex and go beyond the simple enhancement of threatening information. Negative experiences can also rescue related but otherwise forgettable details encoded close in time. Here, we used functional magnetic resonance imaging (fMRI) in healthy young adults to examine the brain mechanisms that support this retrograde memory effect. In a two-phase incidental encoding paradigm, participants viewed different pictures of tools and animals before and during Pavlovian fear conditioning. During Phase 1, these images were intermixed with neutral scenes, which provided a unique 'context tag' for this specific phase of encoding. A few minutes later, during Phase 2, new pictures from one category were paired with a mild shock (threat-conditioned stimulus; CS+), while pictures from the other category were not shocked. FMRI analyses revealed that, across-participants, individuals who showed aversive learning-related retroactive memory benefits for Phase 1 CS+ items were also more likely to exhibit three brain effects: first, greater spontaneous reinstatement of the Phase 1 context when participants viewed conceptually-related CS+ items in Phase 2; second, greater successful encoding-related VTA/SN and LC activation for Phase 2 CS+ items; and third, learning-dependent increases in post-encoding hippocampal functional coupling with CS+ category-selective cortex. These biases in hippocampal-cortical connectivity also mediated the relationship between VTA/SN aversive encoding effects and across-participant variability in the retroactive memory benefit. Collectively, our findings suggest that both online and offline brain mechanisms may enable threatening events to preserve memories that acquire new significance in the future.

Isometric exercise facilitates attention to salient events in women via the noradrenergic system.

The locus coeruleus (LC) regulates attention via the release of norepinephrine (NE), with levels of tonic LC activity constraining the intensity of phasic LC responses. In the current fMRI study, we used isometric handgrip to modulate tonic LC-NE activity in older women and in young women with different hormone statuses during the time period immediately after the handgrip. During this post-handgrip time, an oddball detection task was used to probe how changes in tonic arousal influenced functional coordination between the LC and a right frontoparietal network that supports attentional selectivity. As expected, the frontoparietal network responded more to infrequent target and novel sounds than to frequent sounds. Across participants, greater LC-frontoparietal functional connectivity, pupil dilation, and faster oddball detection were all positively associated with LC MRI structural contrast from a neuromelanin-sensitive scan. Thus, LC structure was related to LC functional dynamics and attentional performance during the oddball task. We also found that handgrip influenced pupil and attentional processing during a subsequent oddball task. Handgrip decreased subsequent tonic pupil size, increased phasic pupil responses to oddball sounds, speeded oddball detection speed, and increased frontoparietal network activation, suggesting that inducing strong LC activity benefits attentional performance in the next few minutes, potentially due to reduced tonic LC activity. In addition, older women showed a similar benefit of handgrip on frontoparietal network activation as younger women, despite showing lower frontoparietal network activation overall. Together these findings suggest that a simple exercise may improve selective attention in healthy aging, at least for several minutes afterwards.

Locus Coeruleus Activity Strengthens Prioritized Memories Under Arousal.

Recent models posit that bursts of locus ceruleus (LC) activity amplify neural gain such that limited attention and encoding resources focus even more on prioritized mental representations under arousal. Here, we tested this hypothesis in human males and females using fMRI, neuromelanin MRI, and pupil dilation, a biomarker of arousal and LC activity. During scanning, participants performed a monetary incentive encoding task in which threat of punishment motivated them to prioritize encoding of scene images over superimposed objects. Threat of punishment elicited arousal and selectively enhanced memory for goal-relevant scenes. Furthermore, trial-level pupil dilations predicted better scene memory under threat, but were not related to object memory outcomes. fMRI analyses revealed that greater threat-evoked pupil dilations were positively associated with greater scene encoding activity in LC and parahippocampal cortex, a region specialized to process scene information. Across participants, this pattern of LC engagement for goal-relevant encoding was correlated with neuromelanin signal intensity, providing the first evidence that LC structure relates to its activation pattern during cognitive processing. Threat also reduced dynamic functional connectivity between high-priority (parahippocampal place area) and lower-priority (lateral occipital cortex) category-selective visual cortex in ways that predicted increased memory selectivity. Together, these findings support the idea that, under arousal, LC activity selectively strengthens prioritized memory representations by modulating local and functional network-level patterns of information processing.SIGNIFICANCE STATEMENT Adaptive behavior relies on the ability to select and store important information amid distraction. Prioritizing encoding of task-relevant inputs is especially critical in threatening or arousing situations, when forming these memories is essential for avoiding danger in the future. However, little is known about the arousal mechanisms that support such memory selectivity. Using fMRI, neuromelanin MRI, and pupil measures, we demonstrate that locus ceruleus (LC) activity amplifies neural gain such that limited encoding resources focus even more on prioritized mental representations under arousal. For the first time, we also show that LC structure relates to its involvement in threat-related encoding processes. These results shed new light on the brain mechanisms by which we process important information when it is most needed.

Transcending time in the brain: How event memories are constructed from experience.

Our daily lives unfold continuously, yet when we reflect on the past, we remember those experiences as distinct and cohesive events. To understand this phenomenon, early investigations focused on how and when individuals perceive natural breakpoints, or boundaries, in ongoing experience. More recent research has examined how these boundaries modulate brain mechanisms that support long-term episodic memory. This work has revealed that a complex interplay between hippocampus and prefrontal cortex promotes the integration and separation of sequential information to help organize our experiences into mnemonic events. Here, we discuss how both temporal stability and change in one's thoughts, goals, and surroundings may provide scaffolding for these neural processes to link and separate memories across time. When learning novel or familiar sequences of information, dynamic hippocampal processes may work both independently from and in concert with other brain regions to bind sequential representations together in memory. The formation and storage of discrete episodic memories may occur both proactively as an experience unfolds. They may also occur retroactively, either during a context shift or when reactivation mechanisms bring the past into the present to allow integration. We also describe conditions and factors that shape the construction and integration of event memories across different timescales. Together these findings shed new light on how the brain transcends time to transform everyday experiences into meaningful memory representations.

Higher locus coeruleus MRI contrast is associated with lower parasympathetic influence over heart rate variability.

The locus coeruleus (LC) is a key node of the sympathetic nervous system and suppresses parasympathetic activity that would otherwise increase heart rate variability. In the current study, we examined whether LC-MRI contrast reflecting neuromelanin accumulation in the LC was associated with high-frequency heart rate variability (HF-HRV), a measure reflecting parasympathetic influences on the heart. Recent evidence indicates that neuromelanin, a byproduct of catecholamine metabolism, accumulates in the LC through young and mid adulthood, suggesting that LC-MRI contrast may be a useful biomarker of individual differences in habitual LC activation. We found that, across younger and older adults, greater LC-MRI contrast was negatively associated with HF-HRV during fear conditioning and spatial detection tasks. This correlation was not accounted for by individual differences in age or anxiety. These findings indicate that individual differences in LC structure relate to key cardiovascular parameters.

Noradrenergic mechanisms of arousal's bidirectional effects on episodic memory.

Arousal's selective effects on cognition go beyond the simple enhancement of emotional stimuli, sometimes enhancing and other times impairing processing of proximal neutral information. Past work shows that arousal impairs encoding of subsequent neutral stimuli regardless of their top-down priority via the engagement of ß-adrenoreceptors. In contrast, retrograde amnesia induced by emotional arousal can flip to enhancement when preceding neutral items are prioritized in top-down attention. Whether ß-adrenoreceptors also contribute to this retrograde memory enhancement of goal-relevant neutral stimuli is unclear. In this pharmacological study, we administered 40mg of propranolol or 40mg of placebo to healthy young adults to examine whether emotional arousal's bidirectional effects on declarative memory relies on ß-adrenoreceptor activation. Following pill intake, participants completed an emotional oddball task in which they were asked to prioritize a neutral object appearing just before an emotional or neutral oddball image within a sequence of 7 neutral objects. Under placebo, emotional oddballs impaired memory for lower priority oddball+1 objects but had no effect on memory for high priority oddball-1 objects. Propranolol blocked this anterograde amnesic effect of arousal. Emotional oddballs also enhanced selective memory trade-offs significantly more in the placebo than drug condition, such that high priority oddball-1 objects were more likely to be remembered at the cost of their corresponding lower priority oddball+1 objects under arousal. Lastly, those who recalled more high priority oddball-1 objects preceding an emotional versus neutral oddball image showed greater increases in salivary alpha-amylase, a biomarker of noradrenergic system activation, across the task. Together these findings suggest that different noradrenergic mechanisms contribute to the anterograde and retrograde mnemonic effects of arousal on proximal neutral memoranda.

Structural foundations of resting-state and task-based functional connectivity in the human brain.

Magnetic resonance imaging enables the noninvasive mapping of both anatomical white matter connectivity and dynamic patterns of neural activity in the human brain. We examine the relationship between the structural properties of white matter streamlines (structural connectivity) and the functional properties of correlations in neural activity (functional connectivity) within 84 healthy human subjects both at rest and during the performance of attention- and memory-demanding tasks. We show that structural properties, including the length, number, and spatial location of white matter streamlines, are indicative of and can be inferred from the strength of resting-state and task-based functional correlations between brain regions. These results, which are both representative of the entire set of subjects and consistently observed within individual subjects, uncover robust links between structural and functional connectivity in the human brain.

GANEing traction: The broad applicability of NE hotspots to diverse cognitive and arousal phenomena.

The GANE (glutamate amplifies noradrenergic effects) model proposes that local glutamate-norepinephrine interactions enable "winner-take-more" effects in perception and memory under arousal. A diverse range of commentaries addressed both the nature of this "hotspot" feedback mechanism and its implications in a variety of psychological domains, inspiring exciting avenues for future research.

Norepinephrine ignites local hotspots of neuronal excitation: How arousal amplifies selectivity in perception and memory.

Emotional arousal enhances perception and memory of high-priority information but impairs processing of other information. Here, we propose that, under arousal, local glutamate levels signal the current strength of a representation and interact with norepinephrine (NE) to enhance high priority representations and out-compete or suppress lower priority representations. In our "glutamate amplifies noradrenergic effects" (GANE) model, high glutamate at the site of prioritized representations increases local NE release from the locus coeruleus (LC) to generate "NE hotspots." At these NE hotspots, local glutamate and NE release are mutually enhancing and amplify activation of prioritized representations. In contrast, arousal-induced LC activity inhibits less active representations via two mechanisms: 1) Where there are hotspots, lateral inhibition is amplified; 2) Where no hotspots emerge, NE levels are only high enough to activate low-threshold inhibitory adrenoreceptors. Thus, LC activation promotes a few hotspots of excitation in the context of widespread suppression, enhancing high priority representations while suppressing the rest. Hotspots also help synchronize oscillations across neural ensembles transmitting high-priority information. Furthermore, brain structures that detect stimulus priority interact with phasic NE release to preferentially route such information through large-scale functional brain networks. A surge of NE before, during, or after encoding enhances synaptic plasticity at NE hotspots, triggering local protein synthesis processes that enhance selective memory consolidation. Together, these noradrenergic mechanisms promote selective attention and memory under arousal. GANE not only reconciles apparently contradictory findings in the emotion-cognition literature but also extends previous influential theories of LC neuromodulation by proposing specific mechanisms for how LC-NE activity increases neural gain.

Increased functional coupling between the left fronto-parietal network and anterior insula predicts steeper delay discounting in smokers.

In previous work, smokers showed steeper devaluation of delayed rewards than non-smokers. While the neural correlates of this link between nicotine dependence and delay of discounting are not established, altered activity in executive networks may relate to impaired delayed gratification. The goal of this study was to examine neural correlates of discounting and their relation to nicotine dependence. Thirty-nine smokers and 33 non-smokers completed a functional magnetic resonance imaging (fMRI) intertemporal choice task in which they made individualized Hard (similarly valued), easy (dissimilarly valued), and control monetary choices. FMRI data were analyzed using a group independent component analysis and dual regression. Smokers discounted more steeply than non-smokers, although this difference was only significant among severely dependent smokers. Intertemporal choices recruited distinct left- and right-lateralized fronto-parietal networks. A group-by-difficulty interaction indicated that smokers, relative to non-smokers, exhibited less difficulty-sensitivity in the right fronto-parietal network. In contrast, smokers showed greater functional connectivity between the left fronto-parietal network and the left fronto-insular cortex. Moreover, the degree of functional connectivity between the left fronto-parietal network and left fronto-insular cortex was significantly correlated with individual differences in discounting. Thus, greater functional coupling between the anterior insula and left fronto-parietal network is a candidate neural substrate linking smoking and impulsivity. Given the anterior insula's role in interfacing cognitive and interoceptive processing, this altered functional connectivity may relate to an addiction-related bias towards immediate rewards.

Locus coeruleus neuromodulation of memories encoded during negative or unexpected action outcomes.

When people experience surprising or sub-optimal performance outcomes, an increase in autonomic arousal helps allocate cognitive resources to adjust behavior accordingly. The locus-coeruleus-norepinephrine (LC-NE) system regulates a central orienting response to behaviorally relevant events, and might therefore signal the need to attend to and learn from performance feedback. Memories of such events also rely on elevated NE, suggesting that LC activity not only responds to salient performance outcomes but also strengthens memory for stimuli associated with their occurrence. In the present study, we used a monetary incentive delay paradigm to determine whether LC functional connectivity during reaction time feedback relates to trial-by-trial memory of preceding photo-objects. We used one psychophysiological interaction (PPI) analysis to examine patterns of LC functional connectivity that were associated with subsequent memory for picture trials in which negative or positive feedback was given, and a second PPI analysis to investigate whether successfully encoded objects from trials with uncertain outcomes were related to distinct patterns of LC functional connectivity across the brain. The PPI results revealed that successfully encoded negative feedback trials (i.e., responses exceeding the response deadline) were uniquely associated with enhanced functional coupling between the LC and left anterior insula. Furthermore, successful memory for objects in low reaction time certainty trials (i.e., responses closest to the response deadline) were linked to positive LC functional coupling with left dorsolateral prefrontal cortex. These findings suggest that noradrenergic influences help facilitate memory encoding during outcome processing via dynamic interactions with regions that process negative or unexpected feedback.

Locus coeruleus activation 'resets' hippocampal event representations and separates adjacent memories.

Memories reflect the ebb and flow of experiences, capturing unique and meaningful events from our lives. Using a combination of functional magnetic resonance imaging (fMRI), neuromelanin imaging, and pupillometry, we show that arousal and locus coeruleus (LC) activation transform otherwise continuous experiences into distinct episodic memories. As sequences unfold, encountering a context shift, or event boundary, triggers arousal and LC processes that predict later memory separation. Boundaries furthermore promote temporal pattern separation within left hippocampal dentate gyrus, which correlates with heightened LC responses to those same transition points. We also find that a neurochemical index of prolonged LC activation correlates with diminished arousal responses at boundaries, suggesting a connection between elevated LC output and impaired event processing. These findings align with the idea that arousal processes initiate a neural and memory 'reset' in response to significant changes, constructing the very episodes that define everyday memory.

Effortful retrieval of semantic memories induces forgetting of related negative and neutral episodic memories.

Retrieval-induced forgetting (RIF) experiments show that the act of retrieving some recently encoded items from a given conceptual category leads to greater forgetting of competing items from that same category. However, RIF studies using emotional stimuli have produced mixed results, perhaps due to the reinstatement of arousal or negative affect during retrieval practice. To induce forgetting of negative episodic memories more indirectly, we examined if retrieving neutral semantic memories leads to RIF of related negative memories. In two experiments, participants studied eight categorized lists comprised of an equal number of negative and neutral words (Experiment 1) or neutral words preceded by neutral or negative images (Experiment 2). To avoid re-exposing individuals to negative material during retrieval practice, participants then performed a semantic memory retrieval task in which they generated (i.e., completed word-stems for) new neutral words from half of the studied categories. We found that semantic retrieval, or word generation, induced forgetting of recently studied words irrespective of their emotional valence or original emotional context. Additionally, across both experiments, less successful word generation was associated with stronger RIF effects. In Experiment 2, the magnitude of RIF was also correlated with higher subjective ratings of retrieval effort during word generation. Together, these results suggest that even when retrieving neutral semantic memories, effortful retrieval may enhance inhibitory processes that lead to forgetting of both neutral and negative episodic memories.

Amygdala functional connectivity is reduced after the cold pressor task.

The amygdala forms a crucial link between central pain and stress systems. Previous research indicates that psychological stress affects amygdala activity, but it is less clear how painful stressors influence subsequent amygdala functional connectivity. In the present study, we used pulsed arterial spin labeling (PASL) to investigate differences in healthy male adults' resting-state amygdala functional connectivity following a cold pressor versus a control task, with the stressor and control conditions being conducted on different days. During the period of peak cortisol response to acute stress (approximately 15-30 min after stressor onset), participants were asked to rest for 6 min with their eyes closed during a PASL scanning sequence. The cold pressor task led to reduced resting-state functional connectivity between the amygdalae and orbitofrontal cortex (OFC) and ventromedial prefrontal cortex, and this occurred irrespective of cortisol release. The stressor also induced greater inverse connectivity between the left amygdala and dorsal anterior cingulate cortex (ACC), a brain region implicated in the down-regulation of amygdala responsivity. Furthermore, the degree of poststressor left amygdala decoupling with the lateral OFC varied according to self-reported pain intensity during the cold pressor task. These findings indicate that the cold pressor task alters amygdala interactions with prefrontal and ACC regions 15-30 min after the stressor, and that these altered functional connectivity patterns are related to pain perception rather than cortisol feedback.

Subliminal backdoors to forgetting emotional memories.

Weakening negative memories often requires first remembering those events. To bypass this distressing process, Zhu et al. elicited forgetting by subliminally reactivating negative memories near in time to retrieval suppression of unrelated memories. Casting an amnesic shadow over harmful, reactivated memories thereby brings new therapeutic possibilities, and questions, to light.

Dynamic emotional states shape the episodic structure of memory.

Human emotions fluctuate over time. However, it is unclear how these shifting emotional states influence the organization of episodic memory. Here, we examine how emotion dynamics transform experiences into memorable events. Using custom musical pieces and a dynamic emotion-tracking tool to elicit and measure temporal fluctuations in felt valence and arousal, our results demonstrate that memory is organized around emotional states. While listening to music, fluctuations between different emotional valences bias temporal encoding process toward memory integration or separation. Whereas a large absolute or negative shift in valence helps segment memories into episodes, a positive emotional shift binds sequential representations together. Both discrete and dynamic shifts in music-evoked valence and arousal also enhance delayed item and temporal source memory for concurrent neutral items, signaling the beginning of new emotional events. These findings are in line with the idea that the rise and fall of emotions can sculpt unfolding experiences into memories of meaningful events.