Time-dependent memory transformation in hippocampus and neocortex is semantic in nature.

Memories undergo a time-dependent neural reorganization, which is assumed to be accompanied by a transformation from detailed to more gist-like memory. However, the nature of this transformation and its underlying neural mechanisms are largely unknown. Here, we report that the time-dependent transformation of memory is semantic in nature, while we find no credible evidence for a perceptual transformation. Model-based MRI analyses reveal time-dependent increases in semantically transformed representations of events in prefrontal and parietal cortices, while specific pattern representations in the anterior hippocampus decline over time. Posterior hippocampal memory reinstatement, in turn, increases over time and is linked to the semantic gist of the original memory, without a statistically significant link to perceptual details. These findings indicate that qualitative changes in memory over time, associated with distinct representational changes in the neocortex and within the hippocampus, reflect a semantic transformation, which may promote the integration of memories into abstract knowledge structures.

Imagining is not seeing: lower insight-driven memory reconfiguration when imagining the link between separate events.

Gaining insight into the relationship between previously separate events allows us to combine these events into coherent episodes. This insight may occur via observation or imagination. Although much of our reasoning occurs in the absence of direct sensory stimuli, how mnemonic integration is accomplished via imagination has remained completely unknown. Here, we combined fMRI with representational similarity analysis and a real-life-like narrative-insight task (NIT) to elucidate the behavioral and neural effects of insight through imagination (vs. observation). Healthy participants performed the NIT in the MRI scanner and underwent memory testing one week later. Crucially, participants in the observation group gained insight through a video, while participants in the imagination group gained insight through an imagination instruction. Although we show that insight via imagination was weaker than insight via direct observation, the imagination group showed better detail memory. Moreover, the imagination group showed no representational change in the anterior hippocampus or increases in frontal and striatal activity for the linked events, as was the case in the observation group. However, the hippocampus and striatum were more activated during linking via imagination, which might indicate that their increased recruitment during imagination impedes concurrent mnemonic integration but may facilitate long-term memory.

Multivariate functional neuroimaging analyses reveal that strength-dependent face expectations are represented in higher-level face-identity areas.

Perception is an active inference in which prior expectations are combined with sensory input. It is still unclear how the strength of prior expectations is represented in the human brain. The strength, or precision, of a prior could be represented with its content, potentially in higher-level sensory areas. We used multivariate analyses of functional resonance imaging data to test whether expectation strength is represented together with the expected face in high-level face-sensitive regions. Participants were trained to associate images of scenes with subsequently presented images of different faces. Each scene predicted three faces, each with either low, intermediate, or high probability. We found that anticipation enhances the similarity of response patterns in the face-sensitive anterior temporal lobe to response patterns specifically associated with the image of the expected face. In contrast, during face presentation, activity increased for unexpected faces in a typical prediction error network, containing areas such as the caudate and the insula. Our findings show that strength-dependent face expectations are represented in higher-level face-identity areas, supporting hierarchical theories of predictive processing according to which higher-level sensory regions represent weighted priors.

Stress disrupts insight-driven mnemonic reconfiguration in the medial temporal lobe.

Memories are not stored in isolation. Insight into the relationship of initially unrelated events may trigger a flexible reconfiguration of the mnemonic representation of these events. Such representational changes allow the integration of events into coherent episodes and help to build up-to-date-models of the world around us. This process is, however, frequently impaired in stress-related mental disorders resulting in symptoms such as fragmented memories in PTSD. Here, we combined a real life-like narrative-insight task, in which participants learned how initially separate events are linked, with fMRI-based representational similarity analysis to test if and how acute stress interferes with the insight-driven reconfiguration of memories. Our results showed that stress reduced the activity of medial temporal and prefrontal areas when participants gained insight into the link between events. Moreover, stress abolished the insight-related increase in representational dissimilarity for linked events in the anterior part of the hippocampus as well as its association with measures of subsequent memory that we observed in non-stressed controls. However, memory performance, as assessed in a forced-choice recognition test, was even enhanced in the stress group. Our findings suggest that acute stress impedes the neural integration of events into coherent episodes but promotes long-term memory for these integrated narratives and may thus have implications for understanding memory distortions in stress-related mental disorders.

The Assimilation of Novel Information into Schemata and Its Efficient Consolidation.

Schemata enhance memory formation for related novel information. This is true even when this information is neutral with respect to schema-driven expectations. This assimilation of novel information into schemata has been attributed to more effective organizational processing that leads to more referential connections with the activated associative schema network. Animal data suggest that systems consolidation of novel assimilated information is also accelerated. In the current study, we used both multivariate and univariate fMRI analyses to provide further support for these proposals and to elucidate the neural underpinning of these processes. Twenty-eight participants (5 male) overlearned fictitious schemata for 7 weeks and then encoded novel related and control facts in the scanner. These facts were retrieved both immediately and 2 weeks later, also in the scanner. Our results conceptually replicate previous findings with respect to enhanced vmPFC-hippocampus coupling during encoding of novel related information and point to a prior knowledge effect that is distinct from situations where novel information is experienced as congruent or incongruent with a schema. Moreover, the combination of both multivariate and univariate results further specified the proposed contributions of the vmPFC, precuneus and angular gyrus network to the more efficient encoding of schema-related information. In addition, our data provide further evidence for more efficient systems consolidation of such novel schema-related and potentially assimilated information.SIGNIFICANCE STATEMENT Our prior knowledge in a certain domain, often termed schema, heavily influences whether and how we form memories for novel information that can be related to them. The results of the current study show how a ventromedial prefrontal-precuneal-angular network contributes to the more efficient encoding of novel related information. Furthermore, the observed increase in prefrontal-hippocampal coupling during this process points to a critical distinction from the previously described mechanisms supporting the encoding of information that is experienced as congruent with schema-driven expectations. In addition, we find further support for the proposal based on animal data that prior knowledge enhances also the consolidation of schema-related information.

Noradrenergic arousal after encoding reverses the course of systems consolidation in humans.

It is commonly assumed that episodic memories undergo a time-dependent systems consolidation process, during which hippocampus-dependent memories eventually become reliant on neocortical areas. Here we show that systems consolidation dynamics can be experimentally manipulated and even reversed. We combined a single pharmacological elevation of post-encoding noradrenergic activity through the α2-adrenoceptor antagonist yohimbine with fMRI scanning both during encoding and recognition testing either 1 or 28 days later. We show that yohimbine administration, in contrast to placebo, leads to a time-dependent increase in hippocampal activity and multivariate encoding-retrieval pattern similarity, an indicator of episodic reinstatement, between 1 and 28 days. This is accompanied by a time-dependent decrease in neocortical activity. Behaviorally, these neural changes are linked to a reduced memory decline over time after yohimbine intake. These findings indicate that noradrenergic activity shortly after encoding may alter and even reverse systems consolidation in humans, thus maintaining vividness of memories over time.

Topographical and laminar distribution of audiovisual processing within human planum temporale.

The brain is capable of integrating signals from multiple sensory modalities. Such multisensory integration can occur in areas that are commonly considered unisensory, such as planum temporale (PT) representing the auditory association cortex. However, the roles of different afferents (feedforward vs. feedback) to PT in multisensory processing are not well understood. Our study aims to understand that by examining laminar activity patterns in different topographical subfields of human PT under unimodal and multisensory stimuli. To this end, we adopted an advanced mesoscopic (sub-millimeter) fMRI methodology at 7 T by acquiring BOLD (blood-oxygen-level-dependent contrast, which has higher sensitivity) and VAPER (integrated blood volume and perfusion contrast, which has superior laminar specificity) signal concurrently, and performed all analyses in native fMRI space benefiting from an identical acquisition between functional and anatomical images. We found a division of function between visual and auditory processing in PT and distinct feedback mechanisms in different subareas. Specifically, anterior PT was activated more by auditory inputs and received feedback modulation in superficial layers. This feedback depended on task performance and likely arose from top-down influences from higher-order multimodal areas. In contrast, posterior PT was preferentially activated by visual inputs and received visual feedback in both superficial and deep layers, which is likely projected directly from the early visual cortex. Together, these findings provide novel insights into the mechanism of multisensory interaction in human PT at the mesoscopic spatial scale.

Correction: Inferring exemplar discriminability in brain representations.

[This corrects the article DOI: 10.1371/journal.pone.0232551.].

Can expectation suppression be explained by reduced attention to predictable stimuli?

The expectation-suppression effect - reduced stimulus-evoked responses to expected stimuli - is widely considered to be an empirical hallmark of reduced prediction errors in the framework of predictive coding. Here we challenge this notion by proposing that that expectation suppression could be explained by a reduced attention effect. Specifically, we argue that reduced responses to predictable stimuli can also be explained by a reduced saliency-driven allocation of attention. We base our discussion mainly on findings in the visual cortex and propose that resolving this controversy requires the assessment of qualitative differences between the ways in which attention and surprise enhance brain responses.

Clinically relevant autistic traits predict greater reliance on detail for image recognition.

Individuals with an autism spectrum disorder (ASD) diagnosis are often described as having an eye for detail. But it remains to be shown that a detail-focused processing bias is a ubiquitous property of vision in individuals with ASD. To address this question, we investigated whether a greater number of autistic traits in neurotypical subjects is associated with an increased reliance on image details during a natural image recognition task. To this end, we use a novel reverse correlation-based method (feature diagnosticity mapping) for measuring the relative importance of low-level image features for object recognition. The main finding of this study is that image recognition in participants with an above-median number of autistic traits benefited more from the presence of high-spatial frequency image features. Furthermore, we found that this reliance-on-detail effect was best predicted by the presence of the most clinically relevant autistic traits. Therefore, our findings suggest that a greater number of autistic traits in neurotypical individuals is associated with a more detail-oriented visual information processing strategy and that this effect might generalize to a clinical ASD population.

Inferring exemplar discriminability in brain representations.

Representational distinctions within categories are important in all perceptual modalities and also in cognitive and motor representations. Recent pattern-information studies of brain activity have used condition-rich designs to sample the stimulus space more densely. To test whether brain response patterns discriminate among a set of stimuli (e.g. exemplars within a category) with good sensitivity, we can pool statistical evidence over all pairwise comparisons. Here we describe a wide range of statistical tests of exemplar discriminability and assess the validity (specificity) and power (sensitivity) of each test. The tests include previously used and novel, parametric and nonparametric tests, which treat subject as a random or fixed effect, and are based on different dissimilarity measures, different test statistics, and different inference procedures. We use simulated and real data to determine which tests are valid and which are most sensitive. A popular test statistic reflecting exemplar information is the exemplar discriminability index (EDI), which is defined as the average of the pattern dissimilarity estimates between different exemplars minus the average of the pattern dissimilarity estimates between repetitions of identical exemplars. The popular across-subject t test of the EDI (typically using correlation distance as the pattern dissimilarity measure) requires the assumption that the EDI is 0-mean normal under H0. Although this assumption is not strictly true, our simulations suggest that the test controls the false-positives rate at the nominal level, and is thus valid, in practice. However, test statistics based on average Mahalanobis distances or average linear-discriminant t values (both accounting for the multivariate error covariance among responses) are substantially more powerful for both random- and fixed-effects inference. Unlike average cross-validated distances, the EDI is sensitive to differences between the distributions associated with different exemplars (e.g. greater variability for some exemplars than for others), which complicates its interpretation. We suggest preferred procedures for safely and sensitively detecting subtle pattern differences between exemplars.

Valence Encoding Signals in the Human Amygdala and the Willingness to Eat.

One of the strongest drivers of food consumption is pleasure, and with a large variety of palatable food continuously available, there is rarely any necessity to eat something not tasty. The amygdala is involved in hedonic valuation, but its role in valence assignment during food choices is less understood. Given recent evidence for spatially segregated amygdala signatures encoding palatability, we applied a multivariate approach on fMRI data to extract valence-specific signal patterns during an explicit evaluation of food liking. These valence localizers were then used to identify hedonic valuation processes while the same healthy human participants (14 female, 16 male; in overnight fasted state on both scanning days) performed a willingness-to-eat task in a separate fMRI measurement. Valence-specific patterns of amygdala signaling predicted decisions on food consumption significantly. Findings could be validated using the same valence localizers to predict consumption decisions participants made on a separate set of food stimuli that had not been used for localizer identification. Control analyses revealed these findings to be restricted to a multivariate compared with a univariate approach, and to be specific for valence processing in the amygdala. Spatially distributed valuation signals of the amygdala thus appear to modulate appetitive consumption decisions, and may be useful to identify current hedonic valuation processes triggering food choices even when not explicitly instructed.SIGNIFICANCE STATEMENT The expectation of tastiness is a particularly strong driver in everyday decisions on food consumption. The amygdala is important for hedonic valuation processes and involved in valence-related behavior, but the relationship between both processes is less understood. Here, we show that hedonic values of food are represented in spatially distributed activation patterns in the amygdala. The engagement of these patterns during food choices modulates consumption decisions. Findings are stable in a separate stimulus set. These results suggest that valence-specific amygdala signals are integrated into the formation of food choices.

Fixation-pattern similarity analysis reveals adaptive changes in face-viewing strategies following aversive learning.

Animals can effortlessly adapt their behavior by generalizing from past aversive experiences, allowing to avoid harm in novel situations. We studied how visual information was sampled by eye-movements during this process called fear generalization, using faces organized along a circular two-dimensional perceptual continuum. During learning, one face was conditioned to predict a harmful event, whereas the most dissimilar face stayed neutral. This introduced an adversity gradient along one specific dimension, while the other, unspecific dimension was defined solely by perceptual similarity. Aversive learning changed scanning patterns selectively along the adversity-related dimension, but not the orthogonal dimension. This effect was mainly located within the eye region of faces. Our results provide evidence for adaptive changes in viewing strategies of faces following aversive learning. This is compatible with the view that these changes serve to sample information in a way that allows discriminating between safe and adverse for a better threat prediction.

Forward models demonstrate that repetition suppression is best modelled by local neural scaling.

Inferring neural mechanisms from functional magnetic resonance imaging (fMRI) is challenging because the fMRI signal integrates over millions of neurons. One approach is to compare computational models that map neural activity to fMRI responses, to see which best predicts fMRI data. We use this approach to compare four possible neural mechanisms of fMRI adaptation to repeated stimuli (scaling, sharpening, repulsive shifting and attractive shifting), acting across three domains (global, local and remote). Six features of fMRI repetition effects are identified, both univariate and multivariate, from two independent fMRI experiments. After searching over parameter values, only the local scaling model can simultaneously fit all data features from both experiments. Thus fMRI stimulus repetition effects are best captured by down-scaling neuronal tuning curves in proportion to the difference between the stimulus and neuronal preference. These results emphasise the importance of formal modelling for bridging neuronal and fMRI levels of investigation.

Author Correction: Retrieval induces adaptive forgetting of competing memories via cortical pattern suppression.

In the published version of this article, a detail is missing from the Methods section "Experimental procedure." The following sentence is to be inserted at the end of its fourth paragraph: "If participants failed to respond within 3.5 s, we assumed that they were unable to successfully recognize the item and coded the corresponding trial as an error." The critical behavioral forgetting effect is significant irrespective of whether these timeouts are coded as errors (t23 = 4.91, P < 0.001) or as missing data (t23 = 3.31, P < 0.01). The original article has not been corrected.

Prospective motion correction improves the sensitivity of fMRI pattern decoding.

We evaluated the effectiveness of prospective motion correction (PMC) on a simple visual task when no deliberate subject motion was present. The PMC system utilizes an in-bore optical camera to track an external marker attached to the participant via a custom-molded mouthpiece. The study was conducted at two resolutions (1.5 mm vs 3 mm) and under three conditions (PMC On and Mouthpiece On vs PMC Off and Mouthpiece On vs PMC Off and Mouthpiece Off). Multiple data analysis methods were conducted, including univariate and multivariate approaches, and we demonstrated that the benefit of PMC is most apparent for multi-voxel pattern decoding at higher resolutions. Additional testing on two participants showed that our inexpensive, commercially available mouthpiece solution produced comparable results to a dentist-molded mouthpiece. Our results showed that PMC is increasingly important at higher resolutions for analyses that require accurate voxel registration across time.

Local opposite orientation preferences in V1: fMRI sensitivity to fine-grained pattern information.

The orientation of a visual grating can be decoded from human primary visual cortex (V1) using functional magnetic resonance imaging (fMRI) at conventional resolutions (2-3 mm voxel width, 3T scanner). It is unclear to what extent this information originates from different spatial scales of neuronal selectivity, ranging from orientation columns to global areal maps. According to the global-areal-map account, fMRI orientation decoding relies exclusively on fMRI voxels in V1 exhibiting a radial or vertical preference. Here we show, by contrast, that 2-mm isotropic voxels in a small patch of V1 within a quarterfield representation exhibit reliable opposite selectivities. Sets of voxels with opposite selectivities are locally intermingled and each set can support orientation decoding. This indicates that global areal maps cannot fully account for orientation information in fMRI and demonstrates that fMRI also reflects fine-grained patterns of neuronal selectivity.

Reliability of dissimilarity measures for multi-voxel pattern analysis.

Representational similarity analysis of activation patterns has become an increasingly important tool for studying brain representations. The dissimilarity between two patterns is commonly quantified by the correlation distance or the accuracy of a linear classifier. However, there are many different ways to measure pattern dissimilarity and little is known about their relative reliability. Here, we compare the reliability of three classes of dissimilarity measure: classification accuracy, Euclidean/Mahalanobis distance, and Pearson correlation distance. Using simulations and four real functional magnetic resonance imaging (fMRI) datasets, we demonstrate that continuous dissimilarity measures are substantially more reliable than the classification accuracy. The difference in reliability can be explained by two characteristics of classifiers: discretization and susceptibility of the discriminant function to shifts of the pattern ensemble between imaging runs. Reliability can be further improved through multivariate noise normalization for all measures. Finally, unlike conventional distance measures, crossvalidated distances provide unbiased estimates of pattern dissimilarity on a ratio scale, thus providing an interpretable zero point. Overall, our results indicate that the crossvalidated Mahalanobis distance is preferable to both the classification accuracy and the correlation distance for characterizing representational geometries.

Retrieval induces adaptive forgetting of competing memories via cortical pattern suppression.

Remembering a past experience can, surprisingly, cause forgetting. Forgetting arises when other competing traces interfere with retrieval and inhibitory control mechanisms are engaged to suppress the distraction they cause. This form of forgetting is considered to be adaptive because it reduces future interference. The effect of this proposed inhibition process on competing memories has, however, never been observed, as behavioral methods are 'blind' to retrieval dynamics and neuroimaging methods have not isolated retrieval of individual memories. We developed a canonical template tracking method to quantify the activation state of individual target memories and competitors during retrieval. This method revealed that repeatedly retrieving target memories suppressed cortical patterns unique to competitors. Pattern suppression was related to engagement of prefrontal regions that have been implicated in resolving retrieval competition and, critically, predicted later forgetting. Thus, our findings demonstrate a cortical pattern suppression mechanism through which remembering adaptively shapes which aspects of our past remain accessible.