Spontaneous eye movements reflect the representational geometries of conceptual spaces.

Functional neuroimaging studies indicate that the human brain can represent concepts and their relational structure in memory using coding schemes typical of spatial navigation. However, whether we can read out the internal representational geometries of conceptual spaces solely from human behavior remains unclear. Here, we report that the relational structure between concepts in memory might be reflected in spontaneous eye movements during verbal fluency tasks: When we asked participants to randomly generate numbers, their eye movements correlated with distances along the left-to-right one-dimensional geometry of the number space (mental number line), while they scaled with distance along the ring-like two-dimensional geometry of the color space (color wheel) when they randomly generated color names. Moreover, when participants randomly produced animal names, eye movements correlated with low-dimensional similarity in word frequencies. These results suggest that the representational geometries used to internally organize conceptual spaces might be read out from gaze behavior.

Parallel cognitive maps for multiple knowledge structures in the hippocampal formation.

The hippocampal-entorhinal system uses cognitive maps to represent spatial knowledge and other types of relational information. However, objects can often be characterized by different types of relations simultaneously. How does the hippocampal formation handle the embedding of stimuli in multiple relational structures that differ vastly in their mode and timescale of acquisition? Does the hippocampal formation integrate different stimulus dimensions into one conjunctive map or is each dimension represented in a parallel map? Here, we reanalyzed human functional magnetic resonance imaging data from Garvert et al. (2017) that had previously revealed a map in the hippocampal formation coding for a newly learnt transition structure. Using functional magnetic resonance imaging adaptation analysis, we found that the degree of representational similarity in the bilateral hippocampus also decreased as a function of the semantic distance between presented objects. Importantly, while both map-like structures localized to the hippocampal formation, the semantic map was located in more posterior regions of the hippocampal formation than the transition structure and thus anatomically distinct. This finding supports the idea that the hippocampal-entorhinal system forms parallel cognitive maps that reflect the embedding of objects in diverse relational structures.

Task imprinting: Another mechanism of representational change?

Research from several areas suggests that mental representations adapt to the specific tasks we carry out in our environment. In this study, we propose a mechanism of adaptive representational change, task imprinting. Thereby, we introduce a computational model, which portrays task imprinting as an adaptation to specific task goals via selective storage of helpful representations in long-term memory. We test the main qualitative prediction of the model in four behavioral experiments using healthy young adults as participants. In each experiment, we assess participants' baseline representations in the beginning of the experiment, then expose participants to one of two tasks intended to shape representations differently according to our model, and finally assess any potential change in representations. Crucially, the tasks used to measure representations differ in the amount that strategic, judgmental processes play a role. The results of Experiments 1 and 2 allow us to exclude the option that representations used in more perceptual tasks become biased categorically. The results of Experiment 4 make it likely that people strategically decide given the specific task context whether they use categorical information or not. One signature of representational change was however observed: category learning practice increased the perceptual sensitivity over and above mere exposure to the same stimuli.

Imagining is not seeing: lower insight-driven memory reconfiguration when imagining the link between separate events.

Gaining insight into the relationship between previously separate events allows us to combine these events into coherent episodes. This insight may occur via observation or imagination. Although much of our reasoning occurs in the absence of direct sensory stimuli, how mnemonic integration is accomplished via imagination has remained completely unknown. Here, we combined fMRI with representational similarity analysis and a real-life-like narrative-insight task (NIT) to elucidate the behavioral and neural effects of insight through imagination (vs. observation). Healthy participants performed the NIT in the MRI scanner and underwent memory testing one week later. Crucially, participants in the observation group gained insight through a video, while participants in the imagination group gained insight through an imagination instruction. Although we show that insight via imagination was weaker than insight via direct observation, the imagination group showed better detail memory. Moreover, the imagination group showed no representational change in the anterior hippocampus or increases in frontal and striatal activity for the linked events, as was the case in the observation group. However, the hippocampus and striatum were more activated during linking via imagination, which might indicate that their increased recruitment during imagination impedes concurrent mnemonic integration but may facilitate long-term memory.

Stress disrupts insight-driven mnemonic reconfiguration in the medial temporal lobe.

Memories are not stored in isolation. Insight into the relationship of initially unrelated events may trigger a flexible reconfiguration of the mnemonic representation of these events. Such representational changes allow the integration of events into coherent episodes and help to build up-to-date-models of the world around us. This process is, however, frequently impaired in stress-related mental disorders resulting in symptoms such as fragmented memories in PTSD. Here, we combined a real life-like narrative-insight task, in which participants learned how initially separate events are linked, with fMRI-based representational similarity analysis to test if and how acute stress interferes with the insight-driven reconfiguration of memories. Our results showed that stress reduced the activity of medial temporal and prefrontal areas when participants gained insight into the link between events. Moreover, stress abolished the insight-related increase in representational dissimilarity for linked events in the anterior part of the hippocampus as well as its association with measures of subsequent memory that we observed in non-stressed controls. However, memory performance, as assessed in a forced-choice recognition test, was even enhanced in the stress group. Our findings suggest that acute stress impedes the neural integration of events into coherent episodes but promotes long-term memory for these integrated narratives and may thus have implications for understanding memory distortions in stress-related mental disorders.

Learning and Representation of Hierarchical Concepts in Hippocampus and Prefrontal Cortex.

A key aspect of conceptual knowledge is that it can be flexibly applied at different levels of abstraction, implying a hierarchical organization. It is yet unclear how this hierarchical structure is acquired and represented in the brain. Here we investigate the computations underlying the acquisition and representation of the hierarchical structure of conceptual knowledge in the hippocampal-prefrontal system of 32 human participants (22 females). We assessed the hierarchical nature of learning during a novel tree-like categorization task via computational model comparisons. The winning model allowed to extract and quantify estimates for accumulation and updating of hierarchical compared with single-feature-based concepts from behavior. We find that mPFC tracks accumulation of hierarchical conceptual knowledge over time, and mPFC and hippocampus both support trial-to-trial updating. As a function of those learning parameters, mPFC and hippocampus further show connectivity changes to rostro-lateral PFC, which ultimately represented the hierarchical structure of the concept in the final stages of learning. Our results suggest that mPFC and hippocampus support the integration of accumulated evidence and instantaneous updates into hierarchical concept representations in rostro-lateral PFC.SIGNIFICANCE STATEMENT A hallmark of human cognition is the flexible use of conceptual knowledge at different levels of abstraction, ranging from a coarse category level to a fine-grained subcategory level. While previous work probed the representational geometry of long-term category knowledge, it is unclear how this hierarchical structure inherent to conceptual knowledge is acquired and represented. By combining a novel hierarchical concept learning task with computational modeling of categorization behavior and concurrent fMRI, we differentiate the roles of key concept learning regions in hippocampus and PFC in learning computations and the representation of a hierarchical category structure.

The CABB dataset: A multimodal corpus of communicative interactions for behavioural and neural analyses.

We present a dataset of behavioural and fMRI observations acquired in the context of humans involved in multimodal referential communication. The dataset contains audio/video and motion-tracking recordings of face-to-face, task-based communicative interactions in Dutch, as well as behavioural and neural correlates of participants' representations of dialogue referents. Seventy-one pairs of unacquainted participants performed two interleaved interactional tasks in which they described and located 16 novel geometrical objects (i.e., Fribbles) yielding spontaneous interactions of about one hour. We share high-quality video (from three cameras), audio (from head-mounted microphones), and motion-tracking (Kinect) data, as well as speech transcripts of the interactions. Before and after engaging in the face-to-face communicative interactions, participants' individual representations of the 16 Fribbles were estimated. Behaviourally, participants provided a written description (one to three words) for each Fribble and positioned them along 29 independent conceptual dimensions (e.g., rounded, human, audible). Neurally, fMRI signal evoked by each Fribble was measured during a one-back working-memory task. To enable functional hyperalignment across participants, the dataset also includes fMRI measurements obtained during visual presentation of eight animated movies (35 min total). We present analyses for the various types of data demonstrating their quality and consistency with earlier research. Besides high-resolution multimodal interactional data, this dataset includes different correlates of communicative referents, obtained before and after face-to-face dialogue, allowing for novel investigations into the relation between communicative behaviours and the representational space shared by communicators. This unique combination of data can be used for research in neuroscience, psychology, linguistics, and beyond.

Sleep strengthens integration of spatial memory systems.

Spatial memory comprises different representational systems that are sensitive to different environmental cues, like proximal landmarks or local boundaries. Here we examined how sleep affects the formation of a spatial representation integrating landmark-referenced and boundary-referenced representations. To this end, participants (n = 42) were familiarized with an environment featuring both a proximal landmark and a local boundary. After nocturnal periods of sleep or wakefulness and another night of sleep, integration of the two representational systems was tested by testing the participant's flexibility to switch from landmark-based to boundary-based navigation in the environment, and vice versa. Results indicate a distinctly increased flexibility in relying on either landmarks or boundaries for navigation, when familiarization to the environment was followed by sleep rather than by wakefulness. A second control study (n = 45) did not reveal effects of sleep (vs. wakefulness) on navigation in environments featuring only landmarks or only boundaries. Thus, rather than strengthening isolated representational systems per se, sleep presumably through forming an integrative representation, enhances flexible coordination of representational subsystems.

The Hippocampus Maps Concept Space, Not Feature Space.

The hippocampal formation encodes maps of space and a key question in neuroscience is whether its spatial coding principles also provide a universal metric for the organization of nonspatial, conceptual information. Previous work demonstrated directional coding during navigation through a continuous stimulus feature space as well as mapping of distances in a feature space that was relevant for concept learning. Here we provide the first unambiguous evidence for a hippocampal representation of the actual concept space, by showing that the hippocampal distance signal selectively reflects the mapping of specifically conceptually relevant rather than of all feature dimensions. During fMRI scanning of 32 human participants (21 females), we presented everyday objects, which had beforehand been associated with specific values on three continuous feature dimensions. Crucially, only two dimensions were relevant to prior concept learning. We find that hippocampal responses to the objects reflect their relative distances in a space defined along conceptually relevant dimensions compared with distances in a space defined along all feature dimensions. These findings suggest that the hippocampus supports knowledge acquisition by dynamically encoding information in a space spanned along dimensions that are relevant in relation to define concepts.SIGNIFICANCE STATEMENT How are neural representations of conceptual knowledge organized, such that humans are able to infer never experienced relations or categorize new exemplars? Map-like representations as supported by the hippocampal formation to encode physical space during navigation have been suggested as a suitable format. Here we provide the first evidence for a hippocampal representation of a conceptual space compared with a general feature-based space.

Structural connectivity-based segmentation of the human entorhinal cortex.

The medial (MEC) and lateral entorhinal cortex (LEC), widely studied in rodents, are well defined and characterized. In humans, however, the exact locations of their homologues remain uncertain. Previous functional magnetic resonance imaging (fMRI) studies have subdivided the human EC into posteromedial (pmEC) and anterolateral (alEC) parts, but uncertainty remains about the choice of imaging modality and seed regions, in particular in light of a substantial revision of the classical model of EC connectivity based on novel insights from rodent anatomy. Here, we used structural, not functional imaging, namely diffusion tensor imaging (DTI) and probabilistic tractography to segment the human EC based on differential connectivity to other brain regions known to project selectively to MEC or LEC. We defined MEC as more strongly connected with presubiculum and retrosplenial cortex (RSC), and LEC as more strongly connected with distal CA1 and proximal subiculum (dCA1pSub) and lateral orbitofrontal cortex (OFC). Although our DTI segmentation had a larger medial-lateral component than in the previous fMRI studies, our results show that the human MEC and LEC homologues have a border oriented both towards the posterior-anterior and medial-lateral axes, supporting the differentiation between pmEC and alEC.

Sequence Memory in the Hippocampal-Entorhinal Region.

Episodic memories are constructed from sequences of events. When recalling such a memory, we not only recall individual events, but we also retrieve information about how the sequence of events unfolded. Here, we focus on the role of the hippocampal-entorhinal region in processing and remembering sequences of events, which are thought to be stored in relational networks. We summarize evidence that temporal relations are a central organizational principle for memories in the hippocampus. Importantly, we incorporate novel insights from recent studies about the role of the adjacent entorhinal cortex in sequence memory. In rodents, the lateral entorhinal subregion carries temporal information during ongoing behavior. The human homologue is recruited during memory recall where its representations reflect the temporal relationships between events encountered in a sequence. We further introduce the idea that the hippocampal-entorhinal region might enable temporal scaling of sequence representations. Flexible changes of sequence progression speed could underlie the traversal of episodic memories and mental simulations at different paces. In conclusion, we describe how the entorhinal cortex and hippocampus contribute to remembering event sequences-a core component of episodic memory.

The role of hippocampal spatial representations in contextualization and generalization of fear.

Using contextual information to predict aversive events is a critical ability that protects from generalizing fear responses to safe contexts. Animal models have demonstrated the importance of spatial context representations within the hippocampal formation in contextualization of fear learning. The ventromedial prefrontal cortex (vmPFC) is known to play an important role in safety learning, possibly also through the incorporation of context information. However, if contextual representations are related to context-dependent expression of fear memory in humans remains unclear. Twenty-one healthy participants underwent functional MRI combined with a cue-context conditioning paradigm within a self-navigated virtual reality environment. The environment included two buildings (Threat and Safe context), which had distinct features outside but were identical inside. Within each context, participants saw two cues (CS+, CS-). The CS+ was consistently (100% reinforcement rate) paired with an electric shock in the Threat context, but never in the Safe context. The CS- was never paired with a shock. We found robust differential skin conductance responses (SCRs; CS+ â€‹> â€‹CS-) in the Threat context, but also within the Safe context, indicating fear generalization. Within the Safe context, vmPFC responses to the CS+ were larger than those in the Threat context. We furthermore found environment-specific representations for the two contexts in the training paradigm (i.e., before conditioning took place) in the hippocampus to be related to fear expression and generalization. Namely, participants with a weak context representation (z-score < 1.65) showed stronger fear generalization compared to participants with a strong context representation (z-score > 1.65). Thus, a weak neural representation strength of spatial context may explain overgeneralization of memory to safe contexts. In addition, our findings demonstrate that context-dependent regulation of fear expression engages ventromedial prefrontal pathways suggesting this involves a similar mechanism that is known to be involved in retrieval of extinction memory.

Saccades are phase-locked to alpha oscillations in the occipital and medial temporal lobe during successful memory encoding.

Efficient sampling of visual information requires a coordination of eye movements and ongoing brain oscillations. Using intracranial and magnetoencephalography (MEG) recordings, we show that saccades are locked to the phase of visual alpha oscillations and that this coordination is related to successful mnemonic encoding of visual scenes. Furthermore, parahippocampal and retrosplenial cortex involvement in this coordination reflects effective vision-to-memory mapping, highlighting the importance of neural oscillations for the interaction between visual and memory domains.

Functional parcellation using time courses of instantaneous connectivity.

Functional neuroimaging studies have led to understanding the brain as a collection of spatially segregated functional networks. It is thought that each of these networks is in turn composed of a set of distinct sub-regions that together support each network's function. Considering the sub-regions to be an essential part of the brain's functional architecture, several strategies have been put forward that aim at identifying the functional sub-units of the brain by means of functional parcellations. Current parcellation strategies typically employ a bottom-up strategy, creating a parcellation by clustering smaller units. We propose a novel top-down parcellation strategy, using time courses of instantaneous connectivity to subdivide an initial region of interest into sub-regions. We use split-half reproducibility to choose the optimal number of sub-regions. We apply our Instantaneous Connectivity Parcellation (ICP) strategy on high-quality resting-state FMRI data, and demonstrate the ability to generate parcellations for thalamus, entorhinal cortex, motor cortex, and subcortex including brainstem and striatum. We evaluate the subdivisions against available cytoarchitecture maps to show that our parcellation strategy recovers biologically valid subdivisions that adhere to known cytoarchitectural features.

Coding of Event Nodes and Narrative Context in the Hippocampus.

Narratives may provide a general context, unrestricted by space and time, which can be used to organize episodic memories into networks of related events. However, it is not clear how narrative contexts are represented in the brain. Here we test the novel hypothesis that the formation of narrative-based contextual representations in humans relies on the same hippocampal mechanisms that enable formation of spatiotemporal contexts in rodents. Participants watched a movie consisting of two interleaved narratives while we monitored their brain activity using fMRI. We used representational similarity analysis, a type of multivariate pattern analysis, which uses across-voxel correlations as a proxy for neural-pattern similarity, to examine whether the patterns of neural activity can be used to differentiate between narratives and recurring narrative elements, such as people and locations. We demonstrate that the neural activity patterns in the hippocampus differentiate between event nodes (people and locations) and narratives (different stories) and that these narrative-context representations diverge gradually over time akin to remapping-induced spatial maps represented by rodent place cells. SIGNIFICANCE STATEMENT: Narratives, especially in movie format, are very engaging and can be used to investigate neural mechanisms underlying cognitive functions in more naturalistic settings than that of traditional paradigms. Narratives also provide a more general context, unrestricted by space and time, that can be used to organize memories into networks of related events. For this reason, narratives are ideally suited to engage neural mechanisms underlying episodic memory formation. In this study, participants watched a movie with two interleaved narratives while their brain activity was monitored using fMRI. We show that the hippocampus, which is involved in formation of spatiotemporal contexts in episodic memory, also represents gradually diverging narrative contexts as well as narrative elements, such as people and locations.

Updating of visual orientation in a gravity-based reference frame.

The brain can use multiple reference frames to code line orientation, including head-, object-, and gravity-centered references. If these frames change orientation, their representations must be updated to keep register with actual line orientation. We tested this internal updating during head rotation in roll, exploiting the rod-and-frame effect: The illusory tilt of a vertical line surrounded by a tilted visual frame. If line orientation is stored relative to gravity, these distortions should also affect the updating process. Alternatively, if coding is head- or frame-centered, updating errors should be related to the changes in their orientation. Ten subjects were instructed to memorize the orientation of a briefly flashed line, surrounded by a tilted visual frame, then rotate their head, and subsequently judge the orientation of a second line relative to the memorized first while the frame was upright. Results showed that updating errors were mostly related to the amount of subjective distortion of gravity at both the initial and final head orientation, rather than to the amount of intervening head rotation. In some subjects, a smaller part of the updating error was also related to the change of visual frame orientation. We conclude that the brain relies primarily on a gravity-based reference to remember line orientation during head roll.

Hippocampal Volume Reduction in Humans Predicts Impaired Allocentric Spatial Memory in Virtual-Reality Navigation.

The extent to which navigational spatial memory depends on hippocampal integrity in humans is not well documented. We investigated allocentric spatial recall using a virtual environment in a group of patients with severe hippocampal damage (SHD), a group of patients with "moderate" hippocampal damage (MHD), and a normal control group. Through four learning blocks with feedback, participants learned the target locations of four different objects in a circular arena. Distal cues were present throughout the experiment to provide orientation. A circular boundary as well as an intra-arena landmark provided spatial reference frames. During a subsequent test phase, recall of all four objects was tested with only the boundary or the landmark being present. Patients with SHD were impaired in both phases of this task. Across groups, performance on both types of spatial recall was highly correlated with memory quotient (MQ), but not with intelligence quotient (IQ), age, or sex. However, both measures of spatial recall separated experimental groups beyond what would be expected based on MQ, a widely used measure of general memory function. Boundary-based and landmark-based spatial recall were both strongly related to bilateral hippocampal volumes, but not to volumes of the thalamus, putamen, pallidum, nucleus accumbens, or caudate nucleus. The results show that boundary-based and landmark-based allocentric spatial recall are similarly impaired in patients with SHD, that both types of recall are impaired beyond that predicted by MQ, and that recall deficits are best explained by a reduction in bilateral hippocampal volumes. SIGNIFICANCE STATEMENT: In humans, bilateral hippocampal atrophy can lead to profound impairments in episodic memory. Across species, perhaps the most well-established contribution of the hippocampus to memory is not to episodic memory generally but to allocentric spatial memory. However, the extent to which navigational spatial memory depends on hippocampal integrity in humans is not well documented. We investigated spatial recall using a virtual environment in two groups of patients with hippocampal damage (moderate/severe) and a normal control group. The results showed that patients with severe hippocampal damage are impaired in learning and recalling allocentric spatial information. Furthermore, hippocampal volume reduction impaired allocentric navigation beyond what can be predicted by memory quotient as a widely used measure of general memory function.

Evidence for grid cells in a human memory network.

Grid cells in the entorhinal cortex of freely moving rats provide a strikingly periodic representation of self-location which is indicative of very specific computational mechanisms. However, the existence of grid cells in humans and their distribution throughout the brain are unknown. Here we show that the preferred firing directions of directionally modulated grid cells in rat entorhinal cortex are aligned with the grids, and that the spatial organization of grid-cell firing is more strongly apparent at faster than slower running speeds. Because the grids are also aligned with each other, we predicted a macroscopic signal visible to functional magnetic resonance imaging (fMRI) in humans. We then looked for this signal as participants explored a virtual reality environment, mimicking the rats' foraging task: fMRI activation and adaptation showing a speed-modulated six-fold rotational symmetry in running direction. The signal was found in a network of entorhinal/subicular, posterior and medial parietal, lateral temporal and medial prefrontal areas. The effect was strongest in right entorhinal cortex, and the coherence of the directional signal across entorhinal cortex correlated with spatial memory performance. Our study illustrates the potential power of combining single-unit electrophysiology with fMRI in systems neuroscience. Our results provide evidence for grid-cell-like representations in humans, and implicate a specific type of neural representation in a network of regions which supports spatial cognition and also autobiographical memory.

Interaction Between Hippocampus and Cerebellum Crus I in Sequence-Based but not Place-Based Navigation.

To examine the cerebellar contribution to human spatial navigation we used functional magnetic resonance imaging and virtual reality. Our findings show that the sensory-motor requirements of navigation induce activity in cerebellar lobules and cortical areas known to be involved in the motor loop and vestibular processing. By contrast, cognitive aspects of navigation mainly induce activity in a different cerebellar lobule (VIIA Crus I). Our results demonstrate a functional link between cerebellum and hippocampus in humans and identify specific functional circuits linking lobule VIIA Crus I of the cerebellum to medial parietal, medial prefrontal, and hippocampal cortices in nonmotor aspects of navigation. They further suggest that Crus I belongs to 2 nonmotor loops, involved in different strategies: place-based navigation is supported by coherent activity between left cerebellar lobule VIIA Crus I and medial parietal cortex along with right hippocampus activity, while sequence-based navigation is supported by coherent activity between right lobule VIIA Crus I, medial prefrontal cortex, and left hippocampus. These results highlight the prominent role of the human cerebellum in both motor and cognitive aspects of navigation, and specify the cortico-cerebellar circuits by which it acts depending on the requirements of the task.

Reinstatement of associative memories in early visual cortex is signaled by the hippocampus.

The cortical reinstatement hypothesis of memory retrieval posits that content-specific cortical activity at encoding is reinstated at retrieval. Evidence for cortical reinstatement was found in higher-order sensory regions, reflecting reactivation of complex object-based information. However, it remains unclear whether the same detailed sensory, feature-based information perceived during encoding is subsequently reinstated in early sensory cortex and what the role of the hippocampus is in this process. In this study, we used a combination of visual psychophysics, functional neuroimaging, multivoxel pattern analysis, and a well controlled cued recall paradigm to address this issue. We found that the visual information human participants were retrieving could be predicted by the activation patterns in early visual cortex. Importantly, this reinstatement resembled the neural pattern elicited when participants viewed the visual stimuli passively, indicating shared representations between stimulus-driven activity and memory. Furthermore, hippocampal activity covaried with the strength of stimulus-specific cortical reinstatement on a trial-by-trial level during cued recall. These findings provide evidence for reinstatement of unique associative memories in early visual cortex and suggest that the hippocampus modulates the mnemonic strength of this reinstatement.