Schema-driven prediction effects on episodic memory across the lifespan.

The predictive processing framework posits that one of the main functions of the brain is to anticipate the incoming information. Internal models facilitate interactions with the world by predicting future states against which actual evidence is compared. The difference between predicted and actual states, the prediction error (PE), signals novel information. However, how PE affects cognitive processing downstream is not fully understood: one such aspect pertains to how PE influences episodic memories, and whether those effect on memory differ across the lifespan. We examine the relationship between PE and episodic memory in children, young and older adults. We use a novel paradigm whereby rich visual narratives are used to build action schemas that enable probing different mnemonic aspects. To create different levels of PE, we manipulate the story endings to be either expected, neutral or unexpected with respect to the unfolded action. We show that (i) expected endings are better encoded than neutral endings and (ii) unexpected endings improve the encoding of mismatching events and other aspects of the narrative. These effects are differentially modulated across the lifespan with PE-driven encoding being more prominent in children and young adults and with schema integration playing a larger role on memory encoding in older adults. These results highlight the role of predictions by enriching past experiences and informing future anticipations.This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

Mnemonic But Not Contextual Feedback Signals Defy Dedifferentiation in the Aging Early Visual Cortex.

Perception is an intricate interplay between feedforward visual input and internally generated feedback signals that comprise concurrent contextual and time-distant mnemonic (episodic and semantic) information. Yet, an unresolved question is how the composition of feedback signals changes across the lifespan and to what extent feedback signals undergo age-related dedifferentiation, that is, a decline in neural specificity. Previous research on this topic has focused on feedforward perceptual representation and episodic memory reinstatement, suggesting reduced fidelity of neural representations at the item and category levels. In this fMRI study, we combined an occlusion paradigm that filters feedforward input to the visual cortex and multivariate analysis techniques to investigate the information content in cortical feedback, focusing on age-related differences in its composition. We further asked to what extent differentiation in feedback signals (in the occluded region) is correlated to differentiation in feedforward signals. Comparing younger (18-30 years) and older female and male adults (65-75 years), we found that contextual but not mnemonic feedback was prone to age-related dedifferentiation. Semantic feedback signals were even better differentiated in older adults, highlighting the growing importance of generalized knowledge across ages. We also found that differentiation in feedforward signals was correlated with differentiation in episodic but not semantic feedback signals. Our results provide evidence for age-related adjustments in the composition of feedback signals and underscore the importance of examining dedifferentiation in aging for both feedforward and feedback processing.

Parameters of prediction: Multidimensional characterization of top-down influence in visual perception.

Despite the recent popularity of predictive processing models of brain function, the term prediction is often instantiated very differently across studies. These differences in definition can substantially change the type of cognitive or neural operation hypothesised and thus have critical implications for the corresponding behavioural and neural correlates during visual perception. Here, we propose a five-dimensional scheme to characterise different parameters of prediction. Namely, flow of information, mnemonic origin, specificity, complexity, and temporal precision. We describe these dimensions and provide examples of their application to previous work. Such a characterisation not only facilitates the integration of findings across studies, but also helps stimulate new research questions.

Feedback signals in visual cortex during episodic and schematic memory retrieval and their potential implications for aphantasia.

Recent findings indicate that visual feedback derived from episodic memory can be traced down to the earliest stages of visual processing, whereas feedback stemming from schema-related memories only reach intermediate levels in the visual processing hierarchy. In this opinion piece, we examine these differences in light of the 'what' and 'where' streams of visual perception. We build upon this new framework to propose that the memory deficits observed in aphantasics might be better understood as a difference in high-level feedback processing along the 'what' stream, rather than an episodic memory impairment.

The effect of prediction error on episodic memory encoding is modulated by the outcome of the predictions.

Expectations can lead to prediction errors of varying degrees depending on the extent to which the information encountered in the environment conforms with prior knowledge. While there is strong evidence on the computationally specific effects of such prediction errors on learning, relatively less evidence is available regarding their effects on episodic memory. Here, we had participants work on a task in which they learned context/object-category associations of different strengths based on the outcomes of their predictions. We then used a reinforcement learning model to derive subject-specific trial-to-trial estimates of prediction error at encoding and link it to subsequent recognition memory. Results showed that model-derived prediction errors at encoding influenced subsequent memory as a function of the outcome of participants' predictions (correct vs. incorrect). When participants correctly predicted the object category, stronger prediction errors (as a consequence of weak expectations) led to enhanced memory. In contrast, when participants incorrectly predicted the object category, stronger prediction errors (as a consequence of strong expectations) led to impaired memory. These results highlight the important moderating role of choice outcome that may be related to interactions between the hippocampal and striatal dopaminergic systems.

Not what u expect: Effects of prediction errors on item memory.

The characterization of the relationship between predictions and one-shot episodic encoding poses an important challenge for memory research. On the one hand, events that are compatible with our previous knowledge are thought to be remembered better than incompatible ones. On the other hand, unexpected situations, by virtue of their novelty, are known to cause enhanced learning. Several theoretical accounts try to solve this apparent paradox by conceptualizing prediction error (PE) as a continuum ranging from low PE (for expectation-matching events) to high PE (for expectation-mismatching ones). Under such a framework, the relationship between PE and memory encoding would be described by a U-shape function with higher memory performance for extreme levels of PE and lower memory for middle levels of PE. In this study, we tested the framework by using a gradual manipulation of the strength of association between scenes and objects to render different levels of PE and then tested for item memory of the (mis)matching events. In two experiments, in contrast to what was anticipated, recognition memory for object identity followed an inverted U-shape as a function of PE, with higher performance for intermediate levels of PE. Furthermore, in two additional experiments, we showed the relevance of explicit predictions at encoding to reveal such an inverted U pattern, thus providing the boundary conditions of the effect. We discussed our findings in light of existing literature relating PE and episodic memory, pointing out the potential roles of uncertainty in the environment, and the importance of the cognitive operations underlying encoding tasks. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Concurrent contextual and time-distant mnemonic information co-exist as feedback in the human visual cortex.

Efficient processing of the visual environment necessitates the integration of incoming sensory evidence with concurrent contextual inputs and mnemonic content from our past experiences. To examine how this integration takes place in the brain, we isolated different types of feedback signals from the neural patterns of non-stimulated areas of the early visual cortex in humans (i.e., V1 and V2). Using multivariate pattern analysis, we showed that both contextual and time-distant information, coexist in V1 and V2 as feedback signals. In addition, we found that the extent to which mnemonic information is reinstated in V1 and V2 depends on whether the information is retrieved episodically or semantically. Critically, this reinstatement was independent on the retrieval route in the object-selective cortex. These results demonstrate that our early visual processing contains not just direct and indirect information from the visual surrounding, but also memory-based predictions.

The attentional boost effect and perceptual degradation: Assessing the influence of attention on recognition memory.

Researchers have suggested that the recognition memory effects resulting from two separate attentional manipulations-attentional boost and perceptual degradation-may share a common cause; namely a transient up-regulation of attention at the time of encoding that leads to enhanced memory performance at the time of retrieval. Prior research has demonstrated that inducing two similar transient shifts of attention simultaneously produces redundant performance in memory. In the present study, we sought to evaluate the combined influence of the attentional boost and perceptual degradation on recognition memory. If these two effects share a common cause, then we ought to observe a redundancy in memory performance, such that these two factors interact. Yet, across four experiments we fail to observe such a redundancy in recognition memory. We evaluate these results using the limited resource model of attention and speculate on how combining transient shifts of attention may produce redundant memory performance in the one case, but non-redundant performance in the other case.

Increasing control improves further control, but it does not enhance memory for the targets in a face-word Stroop task.

Recent research on the dynamics between attentional and memory processes have outlined the idea that applying control in a conflicting situation directly leads to enhanced episodic memory of the processed information. However, in spite of a small subset of studies supporting this claim, the majority of the evidence in the field seems to support the opposite pattern. In this study, we used a face-word Stroop task to enforce different control modes either from trial to trial or in an item-specific manner. Both manipulations of congruency proved to be effective in making participants' responses to conflicting stimuli more efficient over time by applying a trial-specific control mode. However, these manipulations had no impact on memory performance on a surprise recognition memory test. To our knowledge, this is the first attempt at measuring the memory consequences of the application of specific control modes at the trial level. The results reported here call for caution and possibly reconceptualization of the relationship between cognitive control and memory.

Semantic incongruity attracts attention at a pre-conscious level: Evidence from a TMS study.

Unpredicted objects, i.e., those that do not fit in a specific context, have been shown to quickly attract attention as a mean of extracting more information about potentially relevant items. Whether the required semantic processing triggering the attraction of attention can occur independently of participants' awareness of the object is still a highly debated topic. In the present study we make use of a change detection task in which we manipulate the semantic congruity between the to-be-detected object and the background scene. We applied inhibitory repetitive transcranial magnetic stimulation (rTMS) over the right temporo-parietal junction (right TPJ) and a control location (vertex) to test the causal role of the former in the processing of objects at a pre-conscious level. Our results clearly show that semantic congruity can impact detection and identification processes in opposite ways, even when low-level features are controlled for. Incongruent objects are quickly detected but poorly identified. rTMS over the right TPJ effectively diminishes semantic effects on object detection. These results suggest that at least some high order category processing takes place before conscious detection to direct attention towards the most informative regions of space. Moreover, rTMS over right TPJ also impacts object identification, which calls for a re-evaluation of right TPJ's role on object processing.

A cow on the prairie vs. a cow on the street: long-term consequences of semantic conflict on episodic encoding.

Long-term effects of cognitive conflict on performance are not as well understood as immediate effects. We used a change detection task to explore long-term consequences of cognitive conflict by manipulating the congruity between a changing object and a background scene. According to conflict-based accounts of memory formation, incongruent trials (e.g., a cow on the street), in spite of hindering immediate performance, should promote stronger encoding than congruent trials (e.g., a cow on a prairie). Surprisingly, across three experiments we show that semantic incongruity actually impairs remembering of the information presented during scene processing. This set of results is incompatible with the frequently accepted hypothesis of conflict-triggered learning. Rather, we discuss the present data and other studies previously reported in the literature in the light of two much older hypotheses of memory formation: the desirable difficulty and the levels of processing principles.