The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation.

The hippocampal-entorhinal system is important for spatial and relational memory tasks. We formally link these domains, provide a mechanistic understanding of the hippocampal role in generalization, and offer unifying principles underlying many entorhinal and hippocampal cell types. We propose medial entorhinal cells form a basis describing structural knowledge, and hippocampal cells link this basis with sensory representations. Adopting these principles, we introduce the Tolman-Eichenbaum machine (TEM). After learning, TEM entorhinal cells display diverse properties resembling apparently bespoke spatial responses, such as grid, band, border, and object-vector cells. TEM hippocampal cells include place and landmark cells that remap between environments. Crucially, TEM also aligns with empirically recorded representations in complex non-spatial tasks. TEM also generates predictions that hippocampal remapping is not random as previously believed; rather, structural knowledge is preserved across environments. We confirm this structural transfer over remapping in simultaneously recorded place and grid cells.

Neuronal Computation Underlying Inferential Reasoning in Humans and Mice.

Every day we make decisions critical for adaptation and survival. We repeat actions with known consequences. But we also draw on loosely related events to infer and imagine the outcome of entirely novel choices. These inferential decisions are thought to engage a number of brain regions; however, the underlying neuronal computation remains unknown. Here, we use a multi-day cross-species approach in humans and mice to report the functional anatomy and neuronal computation underlying inferential decisions. We show that during successful inference, the mammalian brain uses a hippocampal prospective code to forecast temporally structured learned associations. Moreover, during resting behavior, coactivation of hippocampal cells in sharp-wave/ripples represent inferred relationships that include reward, thereby "joining-the-dots" between events that have not been observed together but lead to profitable outcomes. Computing mnemonic links in this manner may provide an important mechanism to build a cognitive map that stretches beyond direct experience, thus supporting flexible behavior.

Human Replay Spontaneously Reorganizes Experience.

Knowledge abstracted from previous experiences can be transferred to aid new learning. Here, we asked whether such abstract knowledge immediately guides the replay of new experiences. We first trained participants on a rule defining an ordering of objects and then presented a novel set of objects in a scrambled order. Across two studies, we observed that representations of these novel objects were reactivated during a subsequent rest. As in rodents, human "replay" events occurred in sequences accelerated in time, compared to actual experience, and reversed their direction after a reward. Notably, replay did not simply recapitulate visual experience, but followed instead a sequence implied by learned abstract knowledge. Furthermore, each replay contained more than sensory representations of the relevant objects. A sensory code of object representations was preceded 50 ms by a code factorized into sequence position and sequence identity. We argue that this factorized representation facilitates the generalization of a previously learned structure to new objects.

Decoding cognition from spontaneous neural activity.

In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.

Combined model-free and model-sensitive reinforcement learning in non-human primates.

Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, model-sensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials. We trained rhesus monkeys to perform a two-stage decision task designed to elicit and discriminate the use of MF and MS methods. A descriptive analysis of choice behaviour revealed directly that the structure of the task (of MS importance) and the reward history (of MF and MS importance) significantly influenced both choice and response vigour. A detailed, trial-by-trial computational analysis confirmed that choices were made according to a combination of strategies, with a dominant influence of a particular form of model sensitivity that persisted over weeks of testing. The residuals from this model necessitated development of a new combined RL model which incorporates a particular credit assignment weighting procedure. Finally, response vigor exhibited a subtly different collection of MF and MS influences. These results provide new illumination onto RL behavioural processes in non-human primates.

Inhibitory engrams in perception and memory.

Nervous systems use excitatory cell assemblies to encode and represent sensory percepts. Similarly, synaptically connected cell assemblies or "engrams" are thought to represent memories of past experience. Multiple lines of recent evidence indicate that brain systems create and use inhibitory replicas of excitatory representations for important cognitive functions. Such matched "inhibitory engrams" can form through homeostatic potentiation of inhibition onto postsynaptic cells that show increased levels of excitation. Inhibitory engrams can reduce behavioral responses to familiar stimuli, thereby resulting in behavioral habituation. In addition, by preventing inappropriate activation of excitatory memory engrams, inhibitory engrams can make memories quiescent, stored in a latent form that is available for context-relevant activation. In neural networks with balanced excitatory and inhibitory engrams, the release of innate responses and recall of associative memories can occur through focused disinhibition. Understanding mechanisms that regulate the formation and expression of inhibitory engrams in vivo may help not only to explain key features of cognition but also to provide insight into transdiagnostic traits associated with psychiatric conditions such as autism, schizophrenia, and posttraumatic stress disorder.

Functional Segmentation of the Anterior Limb of the Internal Capsule: Linking White Matter Abnormalities to Specific Connections.

The anterior limb of the internal capsule (ALIC) carries thalamic and brainstem fibers from prefrontal cortical regions that are associated with different aspects of emotion, motivation, cognition processing, and decision-making. This large fiber bundle is abnormal in several psychiatric illnesses and a major target for deep brain stimulation. Yet, we have very little information about where specific prefrontal fibers travel within the bundle. Using a combination of tracing studies and diffusion MRI in male nonhuman primates, as well as diffusion MRI in male and female human subjects, we segmented the human ALIC into five regions based on the positions of axons from different cortical regions within the capsule. Fractional anisotropy (FA) abnormalities in patients with bipolar disorder were detected when FA was averaged in the ALIC segment that carries ventrolateral prefrontal cortical connections. Together, the results set the stage for linking abnormalities within the ALIC to specific connections and demonstrate the utility of applying connectivity profiles of large white matter bundles based on animal anatomic studies to human connections and associating disease abnormalities in those pathways with specific connections. The ability to functionally segment large white matter bundles into their components begins a new era of refining how we think about white matter organization and use that information in understanding abnormalities.SIGNIFICANCE STATEMENT The anterior limb of the internal capsule (ALIC) connects prefrontal cortex with the thalamus and brainstem and is abnormal in psychiatric illnesses. However, we know little about the location of specific prefrontal fibers within the bundle. Using a combination of animal tracing studies and diffusion MRI in animals and human subjects, we segmented the human ALIC into five regions based on the positions of axons from different cortical regions. We then demonstrated that differences in FA values between bipolar disorder patients and healthy control subjects were specific to a given segment. Together, the results set the stage for linking abnormalities within the ALIC to specific connections and for refining how we think about white matter organization in general.

A gyral coordinate system predictive of fibre orientations.

When axonal fibres approach or leave the cortex, their trajectories tend to closely follow the cortical convolutions. To quantify this tendency, we propose a three-dimensional coordinate system based on the gyral geometry. For every voxel in the brain, we define a "radial" axis orthogonal to nearby surfaces, a "sulcal" axis along the sulcal depth gradient that preferentially points from deep white matter to the gyral crown, and a "gyral" axis aligned with the long axis of the gyrus. When compared with high-resolution, in-vivo diffusion MRI data from the Human Connectome Project, we find that in superficial white matter the apparent diffusion coefficient (at b = 1000) along the sulcal axis is on average 16% larger than along the gyral axis and twice as large as along the radial axis. This is reflected in the vast majority of observed fibre orientations lying close to the tangential plane (median angular offset < 7°), with the dominant fibre orientation typically aligning with the sulcal axis. In cortical grey matter, fibre orientations transition to a predominantly radial orientation. We quantify the width and location of this transition and find strong reproducibility in test-retest data, but also a clear dependence on the resolution of the diffusion data. The ratio of radial to tangential diffusion is fairly constant throughout most of the cortex, except for a decrease of the diffusivitiy ratio in the sulcal fundi and the primary somatosensory cortex (Brodmann area 3) and an increase in the primary motor cortex (Brodmann area 4). Although only constrained by cortical folds, the proposed gyral coordinate system provides a simple and intuitive representation of white and grey matter fibre orientations near the cortex, and may be useful for future studies of white matter development and organisation.

Using Diffusion Tractography to Predict Cortical Connection Strength and Distance: A Quantitative Comparison with Tracers in the Monkey.

UNLABELLED: Tractography based on diffusion MRI offers the promise of characterizing many aspects of long-distance connectivity in the brain, but requires quantitative validation to assess its strengths and limitations. Here, we evaluate tractography's ability to estimate the presence and strength of connections between areas of macaque neocortex by comparing its results with published data from retrograde tracer injections. Probabilistic tractography was performed on high-quality postmortem diffusion imaging scans from two Old World monkey brains. Tractography connection weights were estimated using a fractional scaling method based on normalized streamline density. We found a correlation between log-transformed tractography and tracer connection weights of r = 0.59, twice that reported in a recent study on the macaque. Using a novel method to estimate interareal connection lengths from tractography streamlines, we regressed out the distance dependence of connection strength and found that the correlation between tractography and tracers remains positive, albeit substantially reduced. Altogether, these observations provide a valuable, data-driven perspective on both the strengths and limitations of tractography for analyzing interareal corticocortical connectivity in nonhuman primates and a framework for assessing future tractography methodological refinements objectively. SIGNIFICANCE STATEMENT: Tractography based on diffusion MRI has great potential for a variety of applications, including estimation of comprehensive maps of neural connections in the brain ("connectomes"). Here, we describe methods to assess quantitatively tractography's performance in detecting interareal cortical connections and estimating connection strength by comparing it against published results using neuroanatomical tracers. We found the correlation of tractography's estimated connection strengths versus tracer to be twice that of a previous study. Using a novel method for calculating interareal cortical distances, we show that tractography-based estimates of connection strength have useful predictive power beyond just interareal separation. By freely sharing these methods and datasets, we provide a valuable resource for future studies in cortical connectomics.

Improved tractography using asymmetric fibre orientation distributions.

Diffusion MRI allows us to make inferences on the structural organisation of the brain by mapping water diffusion to white matter microstructure. However, such a mapping is generally ill-defined; for instance, diffusion measurements are antipodally symmetric (diffusion along x and -x are equal), whereas the distribution of fibre orientations within a voxel is generally not symmetric. Therefore, different sub-voxel patterns such as crossing, fanning, or sharp bending, cannot be distinguished by fitting a voxel-wise model to the signal. However, asymmetric fibre patterns can potentially be distinguished once spatial information from neighbouring voxels is taken into account. We propose a neighbourhood-constrained spherical deconvolution approach that is capable of inferring asymmetric fibre orientation distributions (A-fods). Importantly, we further design and implement a tractography algorithm that utilises the estimated A-fods, since the commonly used streamline tractography paradigm cannot directly take advantage of the new information. We assess performance using ultra-high resolution histology data where we can compare true orientation distributions against sub-voxel fibre patterns estimated from down-sampled data. Finally, we explore the benefits of A-fods-based tractography using in vivo data by evaluating agreement of tractography predictions with connectivity estimates made using different in-vivo modalities. The proposed approach can reliably estimate complex fibre patterns such as sharp bending and fanning, which voxel-wise approaches cannot estimate. Moreover, histology-based and in-vivo results show that the new framework allows more accurate tractography and reconstruction of maps quantifying (symmetric and asymmetric) fibre complexity.

Structural and functional brain rewiring clarifies preserved interhemispheric transfer in humans born without the corpus callosum.

Why do humans born without the corpus callosum, the major interhemispheric commissure, lack the disconnection syndrome classically described in callosotomized patients? This paradox was discovered by Nobel laureate Roger Sperry in 1968, and has remained unsolved since then. To tackle the hypothesis that alternative neural pathways could explain this puzzle, we investigated patients with callosal dysgenesis using structural and functional neuroimaging, as well as neuropsychological assessments. We identified two anomalous white-matter tracts by deterministic and probabilistic tractography, and provide supporting resting-state functional neuroimaging and neuropsychological evidence for their functional role in preserved interhemispheric transfer of complex tactile information, such as object recognition. These compensatory pathways connect the homotopic posterior parietal cortical areas (Brodmann areas 39 and surroundings) via the posterior and anterior commissures. We propose that anomalous brain circuitry of callosal dysgenesis is determined by long-distance plasticity, a set of hardware changes occurring in the developing brain after pathological interference. So far unknown, these pathological changes somehow divert growing axons away from the dorsal midline, creating alternative tracts through the ventral forebrain and the dorsal midbrain midline, with partial compensatory effects to the interhemispheric transfer of cortical function.

Fusion in diffusion MRI for improved fibre orientation estimation: An application to the 3T and 7T data of the Human Connectome Project.

Determining the acquisition parameters in diffusion magnetic resonance imaging (dMRI) is governed by a series of trade-offs. Images of lower resolution have less spatial specificity but higher signal to noise ratio (SNR). At the same time higher angular contrast, important for resolving complex fibre patterns, also yields lower SNR. Considering these trade-offs, the Human Connectome Project (HCP) acquires high quality dMRI data for the same subjects at different field strengths (3T and 7T), which are publically released. Due to differences in the signal behavior and in the underlying scanner hardware, the HCP 3T and 7T data have complementary features in k- and q-space. The 3T dMRI has higher angular contrast and resolution, while the 7T dMRI has higher spatial resolution. Given the availability of these datasets, we explore the idea of fusing them together with the aim of combining their benefits. We extend a previously proposed data-fusion framework and apply it to integrate both datasets from the same subject into a single joint analysis. We use a generative model for performing parametric spherical deconvolution and estimate fibre orientations by simultaneously using data acquired under different protocols. We illustrate unique features from each dataset and how they are retained after fusion. We further show that this allows us to complement benefits and improve brain connectivity analysis compared to analyzing each of the datasets individually.

Brain systems for probabilistic and dynamic prediction: computational specificity and integration.

A computational approach to functional specialization suggests that brain systems can be characterized in terms of the types of computations they perform, rather than their sensory or behavioral domains. We contrasted the neural systems associated with two computationally distinct forms of predictive model: a reinforcement-learning model of the environment obtained through experience with discrete events, and continuous dynamic forward modeling. By manipulating the precision with which each type of prediction could be used, we caused participants to shift computational strategies within a single spatial prediction task. Hence (using fMRI) we showed that activity in two brain systems (typically associated with reward learning and motor control) could be dissociated in terms of the forms of computations that were performed there, even when both systems were used to make parallel predictions of the same event. A region in parietal cortex, which was sensitive to the divergence between the predictions of the models and anatomically connected to both computational networks, is proposed to mediate integration of the two predictive modes to produce a single behavioral output.

Dissociable effects of surprise and model update in parietal and anterior cingulate cortex.

Brains use predictive models to facilitate the processing of expected stimuli or planned actions. Under a predictive model, surprising (low probability) stimuli or actions necessitate the immediate reallocation of processing resources, but they can also signal the need to update the underlying predictive model to reflect changes in the environment. Surprise and updating are often correlated in experimental paradigms but are, in fact, distinct constructs that can be formally defined as the Shannon information (IS) and Kullback-Leibler divergence (DKL) associated with an observation. In a saccadic planning task, we observed that distinct behaviors and brain regions are associated with surprise/IS and updating/DKL. Although surprise/IS was associated with behavioral reprogramming as indexed by slower reaction times, as well as with activity in the posterior parietal cortex [human lateral intraparietal area (LIP)], the anterior cingulate cortex (ACC) was specifically activated during updating of the predictive model (DKL). A second saccade-sensitive region in the inferior posterior parietal cortex (human 7a), which has connections to both LIP and ACC, was activated by surprise and modulated by updating. Pupillometry revealed a further dissociation between surprise and updating with an early positive effect of surprise and late negative effect of updating on pupil area. These results give a computational account of the roles of the ACC and two parietal saccade regions, LIP and 7a, by which their involvement in diverse tasks can be understood mechanistically. The dissociation of functional roles between regions within the reorienting/reprogramming network may also inform models of neurological phenomena, such as extinction and Balint syndrome, and neglect.

Connectivity-based functional analysis of dopamine release in the striatum using diffusion-weighted MRI and positron emission tomography.

The striatum acts in conjunction with the cortex to control and execute functions that are impaired by abnormal dopamine neurotransmission in disorders such as Parkinson's and schizophrenia. To date, in vivo quantification of striatal dopamine has been restricted to structure-based striatal subdivisions. Here, we present a multimodal imaging approach that quantifies the endogenous dopamine release following the administration of d-amphetamine in the functional subdivisions of the striatum of healthy humans with [(11)C]PHNO and [(11)C]Raclopride positron emission tomography ligands. Using connectivity-based (CB) parcellation, we subdivided the striatum into functional subregions based on striato-cortical anatomical connectivity information derived from diffusion magnetic resonance imaging (MRI) and probabilistic tractography. Our parcellation showed that the functional organization of the striatum was spatially coherent across individuals, congruent with primate data and previous diffusion MRI studies, with distinctive and overlapping networks. d-amphetamine induced the highest dopamine release in the limbic followed by the sensory, motor, and executive areas. The data suggest that the relative regional proportions of D2-like receptors are unlikely to be responsible for this regional dopamine release pattern. Notably, the homogeneity of dopamine release was significantly higher within the CB functional subdivisions in comparison with the structural subdivisions. These results support an association between local levels of dopamine release and cortical connectivity fingerprints.

Human and monkey ventral prefrontal fibers use the same organizational principles to reach their targets: tracing versus tractography.

This article is a comparative study of white matter projections from ventral prefrontal cortex (vPFC) between human and macaque brains. We test whether the organizational rules that vPFC connections follow in macaques are preserved in humans. These rules concern the trajectories of some of the white matter projections from vPFC and how the position of regions in the vPFC dictate the trajectories of their projections in the white matter. To address this question, we present a novel approach that combines direct tracer measurements of entire white matter trajectories in macaque monkeys with diffusion MRI tractography of both macaques and humans. The approach allows us to provide explicit validation of diffusion tractography and transfer tractography strategies across species to test the extent to which inferences from macaques can be applied to human neuroanatomy. Apart from one exception, we found a remarkable overlap between the two techniques in the macaque. Furthermore, the organizational principles followed by vPFC tracts in macaques are preserved in humans.

Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex.

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available-a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.

Trial-type dependent frames of reference for value comparison.

A central question in cognitive neuroscience regards the means by which options are compared and decisions are resolved during value-guided choice. It is clear that several component processes are needed; these include identifying options, a value-based comparison, and implementation of actions to execute the decision. What is less clear is the temporal precedence and functional organisation of these component processes in the brain. Competing models of decision making have proposed that value comparison may occur in the space of alternative actions, or in the space of abstract goods. We hypothesized that the signals observed might in fact depend upon the framing of the decision. We recorded magnetoencephalographic data from humans performing value-guided choices in which two closely related trial types were interleaved. In the first trial type, each option was revealed separately, potentially causing subjects to estimate each action's value as it was revealed and perform comparison in action-space. In the second trial type, both options were presented simultaneously, potentially leading to comparison in abstract goods-space prior to commitment to a specific action. Distinct activity patterns (in distinct brain regions) on the two trial types demonstrated that the observed frame of reference used for decision making indeed differed, despite the information presented being formally identical, between the two trial types. This provides a potential reconciliation of conflicting accounts of value-guided choice.

Associative learning of social value.

Our decisions are guided by information learnt from our environment. This information may come via personal experiences of reward, but also from the behaviour of social partners. Social learning is widely held to be distinct from other forms of learning in its mechanism and neural implementation; it is often assumed to compete with simpler mechanisms, such as reward-based associative learning, to drive behaviour. Recently, neural signals have been observed during social exchange reminiscent of signals seen in studies of associative learning. Here we demonstrate that social information may be acquired using the same associative processes assumed to underlie reward-based learning. We find that key computational variables for learning in the social and reward domains are processed in a similar fashion, but in parallel neural processing streams. Two neighbouring divisions of the anterior cingulate cortex were central to learning about social and reward-based information, and for determining the extent to which each source of information guides behaviour. When making a decision, however, the information learnt using these parallel streams was combined within ventromedial prefrontal cortex. These findings suggest that human social valuation can be realized by means of the same associative processes previously established for learning other, simpler, features of the environment.

The first year of a new era.

What happened when eLife decided to eliminate accept/reject decisions after peer review?