Imagining is not seeing: lower insight-driven memory reconfiguration when imagining the link between separate events.

Gaining insight into the relationship between previously separate events allows us to combine these events into coherent episodes. This insight may occur via observation or imagination. Although much of our reasoning occurs in the absence of direct sensory stimuli, how mnemonic integration is accomplished via imagination has remained completely unknown. Here, we combined fMRI with representational similarity analysis and a real-life-like narrative-insight task (NIT) to elucidate the behavioral and neural effects of insight through imagination (vs. observation). Healthy participants performed the NIT in the MRI scanner and underwent memory testing one week later. Crucially, participants in the observation group gained insight through a video, while participants in the imagination group gained insight through an imagination instruction. Although we show that insight via imagination was weaker than insight via direct observation, the imagination group showed better detail memory. Moreover, the imagination group showed no representational change in the anterior hippocampus or increases in frontal and striatal activity for the linked events, as was the case in the observation group. However, the hippocampus and striatum were more activated during linking via imagination, which might indicate that their increased recruitment during imagination impedes concurrent mnemonic integration but may facilitate long-term memory.

Stress disrupts insight-driven mnemonic reconfiguration in the medial temporal lobe.

Memories are not stored in isolation. Insight into the relationship of initially unrelated events may trigger a flexible reconfiguration of the mnemonic representation of these events. Such representational changes allow the integration of events into coherent episodes and help to build up-to-date-models of the world around us. This process is, however, frequently impaired in stress-related mental disorders resulting in symptoms such as fragmented memories in PTSD. Here, we combined a real life-like narrative-insight task, in which participants learned how initially separate events are linked, with fMRI-based representational similarity analysis to test if and how acute stress interferes with the insight-driven reconfiguration of memories. Our results showed that stress reduced the activity of medial temporal and prefrontal areas when participants gained insight into the link between events. Moreover, stress abolished the insight-related increase in representational dissimilarity for linked events in the anterior part of the hippocampus as well as its association with measures of subsequent memory that we observed in non-stressed controls. However, memory performance, as assessed in a forced-choice recognition test, was even enhanced in the stress group. Our findings suggest that acute stress impedes the neural integration of events into coherent episodes but promotes long-term memory for these integrated narratives and may thus have implications for understanding memory distortions in stress-related mental disorders.

The CABB dataset: A multimodal corpus of communicative interactions for behavioural and neural analyses.

We present a dataset of behavioural and fMRI observations acquired in the context of humans involved in multimodal referential communication. The dataset contains audio/video and motion-tracking recordings of face-to-face, task-based communicative interactions in Dutch, as well as behavioural and neural correlates of participants' representations of dialogue referents. Seventy-one pairs of unacquainted participants performed two interleaved interactional tasks in which they described and located 16 novel geometrical objects (i.e., Fribbles) yielding spontaneous interactions of about one hour. We share high-quality video (from three cameras), audio (from head-mounted microphones), and motion-tracking (Kinect) data, as well as speech transcripts of the interactions. Before and after engaging in the face-to-face communicative interactions, participants' individual representations of the 16 Fribbles were estimated. Behaviourally, participants provided a written description (one to three words) for each Fribble and positioned them along 29 independent conceptual dimensions (e.g., rounded, human, audible). Neurally, fMRI signal evoked by each Fribble was measured during a one-back working-memory task. To enable functional hyperalignment across participants, the dataset also includes fMRI measurements obtained during visual presentation of eight animated movies (35 min total). We present analyses for the various types of data demonstrating their quality and consistency with earlier research. Besides high-resolution multimodal interactional data, this dataset includes different correlates of communicative referents, obtained before and after face-to-face dialogue, allowing for novel investigations into the relation between communicative behaviours and the representational space shared by communicators. This unique combination of data can be used for research in neuroscience, psychology, linguistics, and beyond.

Coding of Event Nodes and Narrative Context in the Hippocampus.

Narratives may provide a general context, unrestricted by space and time, which can be used to organize episodic memories into networks of related events. However, it is not clear how narrative contexts are represented in the brain. Here we test the novel hypothesis that the formation of narrative-based contextual representations in humans relies on the same hippocampal mechanisms that enable formation of spatiotemporal contexts in rodents. Participants watched a movie consisting of two interleaved narratives while we monitored their brain activity using fMRI. We used representational similarity analysis, a type of multivariate pattern analysis, which uses across-voxel correlations as a proxy for neural-pattern similarity, to examine whether the patterns of neural activity can be used to differentiate between narratives and recurring narrative elements, such as people and locations. We demonstrate that the neural activity patterns in the hippocampus differentiate between event nodes (people and locations) and narratives (different stories) and that these narrative-context representations diverge gradually over time akin to remapping-induced spatial maps represented by rodent place cells. SIGNIFICANCE STATEMENT: Narratives, especially in movie format, are very engaging and can be used to investigate neural mechanisms underlying cognitive functions in more naturalistic settings than that of traditional paradigms. Narratives also provide a more general context, unrestricted by space and time, that can be used to organize memories into networks of related events. For this reason, narratives are ideally suited to engage neural mechanisms underlying episodic memory formation. In this study, participants watched a movie with two interleaved narratives while their brain activity was monitored using fMRI. We show that the hippocampus, which is involved in formation of spatiotemporal contexts in episodic memory, also represents gradually diverging narrative contexts as well as narrative elements, such as people and locations.

Causal role of the angular gyrus in insight-driven memory reconfiguration.

Maintaining an accurate model of the world relies on our ability to update memory representations in light of new information. Previous research on the integration of new information into memory mainly focused on the hippocampus. Here, we hypothesized that the angular gyrus, known to be involved in episodic memory and imagination, plays a pivotal role in the insight-driven reconfiguration of memory representations. To test this hypothesis, participants received continuous theta burst stimulation (cTBS) over the left angular gyrus or sham stimulation before gaining insight into the relationship between previously separate life-like animated events in a narrative-insight task. During this task, participants also underwent EEG recording and their memory for linked and non-linked events was assessed shortly thereafter. Our results show that cTBS to the angular gyrus decreased memory for the linking events and reduced the memory advantage for linked relative to non-linked events. At the neural level, cTBS targeting the angular gyrus reduced centro-temporal coupling with frontal regions and abolished insight-induced neural representational changes for events linked via imagination, indicating impaired memory reconfiguration. Further, the cTBS group showed representational changes for non-linked events that resembled the patterns observed in the sham group for the linked events, suggesting failed pruning of the narrative in memory. Together, our findings demonstrate a causal role of the left angular gyrus in insight-related memory reconfigurations.

Narratives for Neuroscience.

People organize and convey their thoughts according to narratives. However, neuroscientists are often reluctant to incorporate narrative stimuli into their experiments. We argue that narratives deserve wider adoption in human neuroscience because they tap into the brain's native machinery for representing the world and provide rich variability for testing hypotheses.

Orientation sensitivity of the N1 evoked by letters and digits.

Orientation sensitivity of the amplitude and latency of the P1 and the N1 was investigated while participants performed letter-digit category judgments and red-blue color judgments. These two tasks were used in order to ascertain whether the orientation effects reflect access to object identity, which would be necessary for category, but not color, judgments. Character misorientation significantly affected both the latency and the amplitude of the N1, but not the P1, component. The N1 amplitude increased gradually up to 90 degrees , then leveled off up to 150 degrees , and dipped somewhat for 180 degrees . The effect of orientation on N1 latency differed between the hemispheres, with a quadratic function characterizing the effect of orientation on the left, and linear and quadratic trends characterizing the effect on the right. The effects of orientation were attributed to perceptual learning rather than object recognition, and the hemispheric differences in N1 latency suggest feature-based processing in the left hemisphere and holistic processing in the right.

Insight reconfigures hippocampal-prefrontal memories.

Our memories are remarkably dynamic and allow us to reinterpret the past once new information comes to light. Gaining novel insights can lead to mental reorganization of previously unrelated events, thus linking them into narratives. The hippocampus and medial prefrontal cortex (mPFC) support integration of partially overlapping events, but the neural mechanisms underlying the reorganization of memories for the formation of coherent narratives remain elusive. Here, we combine fMRI with The Sims 3 videos of life-like animated events, which could either be integrated into narratives or not. We show that insight triggers the emergence of de novo mnemonic representations of the narratives and is associated with increased neural similarity between linked event representations in the posterior hippocampus, mPFC, and autobiographical-memory network. Simultaneously, events irrelevant to the newly established memory of the narrative were pruned out. This process was accompanied by increased neural dissimilarity between non-linked event representations in the posterior hippocampus and mPFC and was additionally signaled by a mismatch response in the anterior hippocampus. Our results demonstrate that insight leads to neural reconfiguration of representational networks within a memory space and have implications for knowledge acquisition in educational settings.

Memory hierarchies map onto the hippocampal long axis in humans.

Memories, similar to the internal representation of space, can be recalled at different resolutions ranging from detailed events to more comprehensive, multi-event narratives. Single-cell recordings in rodents have suggested that different spatial scales are represented as a gradient along the hippocampal axis. We found that a similar organization holds for human episodic memory: memory representations systematically vary in scale along the hippocampal long axis, which may enable the formation of mnemonic hierarchies.

Mnemonic networks in the hippocampal formation: from spatial maps to temporal and conceptual codes.

The hippocampal formation has been associated with a wide variety of functions including spatial navigation and planning, memory encoding and retrieval, relational processing, novelty detection, and imagination. These functions are dissimilar in terms of their behavioral consequences and modality of representation. Consequently, theoretical standpoints have focused on explaining the role of the hippocampal formation in terms of either its spatial or nonspatial functions. Contrary to this dichotomy, we propose that it is essential to look beyond these traditional boundaries between mnemonic and spatial functions and focus instead on the processes that these functions have in common. In this framework, we use electrophysiology data from the spatial domain to predict effects on the systems level, both in spatial and nonspatial domains. We initially outline the results of studies that have used findings from spatial navigation in rodents to predict the patterns of brain activity observable in people who are exploring virtual environments. We discuss how certain properties of space-defining neurons enable space to be represented as a mental map of interconnected locations, which are expressed at multiple spatial scales in separate modules in the hippocampal formation. We then suggest that memories are also organized in networks, characterized by mnemonic and temporal hierarchies. We finish by discussing how virtual-reality techniques can be used to create novel lifelike episodes allowing us to look at episodic memory processes while multivariate analysis tools can be used to explore the organizational structure of mnemonic networks.

Atypical excitation-inhibition balance in autism captured by the gamma response to contextual modulation.

Atypical visual perception in people with autism spectrum disorders (ASD) is hypothesized to stem from an imbalance in excitatory and inhibitory processes in the brain. We used neuronal oscillations in the gamma frequency range (30-90 Hz), which emerge from a balanced interaction of excitation and inhibition in the brain, to assess contextual modulation processes in early visual perception. Electroencephalography was recorded in 12 high-functioning adults with ASD and 12 age- and IQ-matched control participants. Oscillations in the gamma frequency range were analyzed in response to stimuli consisting of small line-like elements. Orientation-specific contextual modulation was manipulated by parametrically increasing the amount of homogeneously oriented elements in the stimuli. The stimuli elicited a strong steady-state gamma response around the refresh-rate of 60 Hz, which was larger for controls than for participants with ASD. The amount of orientation homogeneity (contextual modulation) influenced the gamma response in control subjects, while for subjects with ASD this was not the case. The atypical steady-state gamma response to contextual modulation in subjects with ASD may capture the link between an imbalance in excitatory and inhibitory neuronal processing and atypical visual processing in ASD.

Social status determines how we monitor and evaluate our performance.

Since people with low status are more likely to experience social evaluative threat and are therefore more inclined to monitor for these threats and inhibit approach behaviour, we expected that low-status subjects would be more engaged in evaluating their own performance, compared with high-status subjects. We created a highly salient social hierarchy based on the performance of a simple time estimation task. Subjects could achieve high, middle or low status while performing this task simultaneously with other two players who were either higher or lower in status. Subjects received feedback on their own performance, as well as on the performance of the other two players simultaneously. Electroencephalography (EEG) was recorded from all three participants. The results showed that medial frontal negativity (an event-related potential reflecting performance evaluation) was significantly enhanced for low-status subjects. Implications for status-related differences in goal-directed behaviour are discussed.

About turn: how object orientation affects categorisation and mental rotation.

High-density ERPs evoked by rotated alphanumeric characters were examined to determine how neural processing is affected by stimulus orientation during letter/digit classifications and during mirror/normal discriminations. The former task typically produces response times that are unaffected by stimulus orientation while the latter is thought to require mental rotation. Sensitivity to orientation was first observed around 100-140 ms and this effect was attributed to differences in low-level features between vertical and oblique orientations. Subsequently, character misorientation amplified the N170, a neural marker of object classification, between 160 and 220 ms. Top-down processing is reflected in the ERPs beginning at 280-320 ms and this time range may reflect binding of ventral and dorsal stream information. In the case of mirror-normal discrimination these top-down processes can lead to mental rotation between 340 and 700 ms. Therefore, although neural processing reflects object orientation, these effects do not translate into increases in reaction-times or impaired accuracy for categorisation, and precede those that do in the mental-rotation task.

Functional neuroanatomy of mental rotation.

Brain regions involved in mental rotation were determined by assessing increases in fMRI activation associated with increases in stimulus rotation during a mirror-normal parity-judgment task with letters and digits. A letter-digit category judgment task was used as a control for orientation-dependent neural processing unrelated to mental rotation per se. Compared to the category judgments, the parity judgments elicited increases in activation in both the dorsal and the ventral visual streams, as well as higher-order premotor areas, inferior frontal gyrus, and anterior insula. Only a subset of these areas, namely, the posterior part of the dorsal intraparietal sulcus, higher-order premotor regions, and the anterior insula showed increased activation as a function of stimulus orientation. Parity judgments elicited greater activation in the right than in the left ventral intraparietal sulcus, but there were no hemispheric differences in orientation-dependent activation, suggesting that neither hemisphere is dominant for mental rotation per se. Hemispheric asymmetries associated with parity-judgment tasks may reflect visuospatial processing other than mental rotation itself, which is subserved by a bilateral fronto-parietal network, rather than regions restricted to the posterior parietal.

Hemispheric dominance for mental rotation: it is a matter of time.

Mental rotation is often said to be a right-hemisphere function. The evidence for this claim, however, remains equivocal. To investigate whether right-hemispheric superiority relates to speed of processing, we measured the latencies and amplitudes of individual participant's mental-rotation event-related potential effects over the parietal electrodes. The results suggest that increases in parietal negativity begin around 400 ms after stimulus onset, and continue until 550 ms over the right hemisphere, and 610 ms over the left hemisphere. The effect of orientation on event-related potential amplitudes during those times do not differ between hemispheres. These results indicate that the lateralization effect is primarily related to timing, rather than the extent of cortical involvement.

Requirements for the localization of p130 Cas to Z-lines in cardiac myocytes.

The vertebrate heart responds to hemodynamic load with the enlargement of postmitotic, terminally differentiated cardiac myocytes. Such hypertrophic changes are characterized by alterations in sarcomeric organization and gene expression. Previously, we established a role for a nonreceptor tyrosine kinase, focal adhesion kinase, in signaling the changes in cytoskeletal organization associated with hypertrophy. Here, we report on data supporting a key role for p130Cas in this process. In neonatal cardiac myocytes FAK, Cas and paxillin are located in sarcomeric Z-lines, suggesting that the Z-line is an important signaling locus in these cells. The expression of different Cas mutants results in a nearly complete loss of sarcomeric organization in these myocytes. Moreover, expression of the C-terminal focal adhesion-targeting domain of FAK both disrupted sarcomeric organization and interfered with the localization of endogenous Cas to Z-lines. These findings suggest that the association of FAK and Cas and the preservation of multiple protein-interaction motifs of Cas are required for the correct assembly of sarcomeres in cardiac myocytes.