The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation.

The hippocampal-entorhinal system is important for spatial and relational memory tasks. We formally link these domains, provide a mechanistic understanding of the hippocampal role in generalization, and offer unifying principles underlying many entorhinal and hippocampal cell types. We propose medial entorhinal cells form a basis describing structural knowledge, and hippocampal cells link this basis with sensory representations. Adopting these principles, we introduce the Tolman-Eichenbaum machine (TEM). After learning, TEM entorhinal cells display diverse properties resembling apparently bespoke spatial responses, such as grid, band, border, and object-vector cells. TEM hippocampal cells include place and landmark cells that remap between environments. Crucially, TEM also aligns with empirically recorded representations in complex non-spatial tasks. TEM also generates predictions that hippocampal remapping is not random as previously believed; rather, structural knowledge is preserved across environments. We confirm this structural transfer over remapping in simultaneously recorded place and grid cells.

Functionally Distinct Neuronal Ensembles within the Memory Engram.

Memories are believed to be encoded by sparse ensembles of neurons in the brain. However, it remains unclear whether there is functional heterogeneity within individual memory engrams, i.e., if separate neuronal subpopulations encode distinct aspects of the memory and drive memory expression differently. Here, we show that contextual fear memory engrams in the mouse dentate gyrus contain functionally distinct neuronal ensembles, genetically defined by the Fos- or Npas4-dependent transcriptional pathways. The Fos-dependent ensemble promotes memory generalization and receives enhanced excitatory synaptic inputs from the medial entorhinal cortex, which we find itself also mediates generalization. The Npas4-dependent ensemble promotes memory discrimination and receives enhanced inhibitory drive from local cholecystokinin-expressing interneurons, the activity of which is required for discrimination. Our study provides causal evidence for functional heterogeneity within the memory engram and reveals synaptic and circuit mechanisms used by each ensemble to regulate the memory discrimination-generalization balance.

Human Replay Spontaneously Reorganizes Experience.

Knowledge abstracted from previous experiences can be transferred to aid new learning. Here, we asked whether such abstract knowledge immediately guides the replay of new experiences. We first trained participants on a rule defining an ordering of objects and then presented a novel set of objects in a scrambled order. Across two studies, we observed that representations of these novel objects were reactivated during a subsequent rest. As in rodents, human "replay" events occurred in sequences accelerated in time, compared to actual experience, and reversed their direction after a reward. Notably, replay did not simply recapitulate visual experience, but followed instead a sequence implied by learned abstract knowledge. Furthermore, each replay contained more than sensory representations of the relevant objects. A sensory code of object representations was preceded 50 ms by a code factorized into sequence position and sequence identity. We argue that this factorized representation facilitates the generalization of a previously learned structure to new objects.

Humans flexibly integrate social information despite interindividual differences in reward.

There has been much progress in understanding human social learning, including recent studies integrating social information into the reinforcement learning framework. Yet previous studies often assume identical payoffs between observer and demonstrator, overlooking the diversity of social information in real-world interactions. We address this gap by introducing a socially correlated bandit task that accommodates payoff differences among participants, allowing for the study of social learning under more realistic conditions. Our Social Generalization (SG) model, tested through evolutionary simulations and two online experiments, outperforms existing models by incorporating social information into the generalization process, but treating it as noisier than individual observations. Our findings suggest that human social learning is more flexible than previously believed, with the SG model indicating a potential resource-rational trade-off where social learning partially replaces individual exploration. This research highlights the flexibility of humans' social learning, allowing us to integrate social information from others with different preferences, skills, or goals.

A mathematical theory of relational generalization in transitive inference.

Humans and animals routinely infer relations between different items or events and generalize these relations to novel combinations of items. This allows them to respond appropriately to radically novel circumstances and is fundamental to advanced cognition. However, how learning systems (including the brain) can implement the necessary inductive biases has been unclear. We investigated transitive inference (TI), a classic relational task paradigm in which subjects must learn a relation ([Formula: see text] and [Formula: see text]) and generalize it to new combinations of items ([Formula: see text]). Through mathematical analysis, we found that a broad range of biologically relevant learning models (e.g. gradient flow or ridge regression) perform TI successfully and recapitulate signature behavioral patterns long observed in living subjects. First, we found that models with item-wise additive representations automatically encode transitive relations. Second, for more general representations, a single scalar "conjunctivity factor" determines model behavior on TI and, further, the principle of norm minimization (a standard statistical inductive bias) enables models with fixed, partly conjunctive representations to generalize transitively. Finally, neural networks in the "rich regime," which enables representation learning and improves generalization on many tasks, unexpectedly show poor generalization and anomalous behavior on TI. We find that such networks implement a form of norm minimization (over hidden weights) that yields a local encoding mechanism lacking transitivity. Our findings show how minimal statistical learning principles give rise to a classical relational inductive bias (transitivity), explain empirically observed behaviors, and establish a formal approach to understanding the neural basis of relational abstraction.

Perceptual learning across saccades: Feature but not location specific.

Perceptual learning is the ability to enhance perception through practice. The hallmark of perceptual learning is its specificity for the trained location and stimulus features, such as orientation. For example, training in discriminating a grating's orientation improves performance only at the trained location but not in other untrained locations. Perceptual learning has mostly been studied using stimuli presented briefly while observers maintained gaze at one location. However, in everyday life, stimuli are actively explored through eye movements, which results in successive projections of the same stimulus at different retinal locations. Here, we studied perceptual learning of orientation discrimination across saccades. Observers were trained to saccade to a peripheral grating and to discriminate its orientation change that occurred during the saccade. The results showed that training led to transsaccadic perceptual learning (TPL) and performance improvements which did not generalize to an untrained orientation. Remarkably, however, for the trained orientation, we found a complete transfer of TPL to the untrained location in the opposite hemifield suggesting high flexibility of reference frame encoding in TPL. Three control experiments in which participants were trained without saccades did not show such transfer, confirming that the location transfer was contingent upon eye movements. Moreover, performance at the trained location, but not at the untrained location, was also improved in an untrained fixation task. Our results suggest that TPL has both, a location-specific component that occurs before the eye movement and a saccade-related component that involves location generalization.

Human-like scene interpretation by a guided counterstream processing.

In modeling vision, there has been a remarkable progress in recognizing a range of scene components, but the problem of analyzing full scenes, an ultimate goal of visual perception, is still largely open. To deal with complete scenes, recent work focused on the training of models for extracting the full graph-like structure of a scene. In contrast with scene graphs, humans' scene perception focuses on selected structures in the scene, starting with a limited interpretation and evolving sequentially in a goal-directed manner [G. L. Malcolm, I. I. A. Groen, C. I. Baker, Trends. Cogn. Sci. 20, 843-856 (2016)]. Guidance is crucial throughout scene interpretation since the extraction of full scene representation is often infeasible. Here, we present a model that performs human-like guided scene interpretation, using an iterative bottom-up, top-down processing, in a "counterstream" structure motivated by cortical circuitry. The process proceeds by the sequential application of top-down instructions that guide the interpretation process. The results show how scene structures of interest to the viewer are extracted by an automatically selected sequence of top-down instructions. The model shows two further benefits. One is an inherent capability to deal well with the problem of combinatorial generalization-generalizing broadly to unseen scene configurations, which is limited in current network models [B. Lake, M. Baroni, 35th International Conference on Machine Learning, ICML 2018 (2018)]. The second is the ability to combine visual with nonvisual information at each cycle of the interpretation process, which is a key aspect for modeling human perception as well as advancing AI vision systems.

Auditory aversive generalization learning prompts threat-specific changes in alpha-band activity.

Pairing a neutral stimulus with aversive outcomes prompts neurophysiological and autonomic changes in response to the conditioned stimulus (CS+), compared to cues that signal safety (CS-). One of these changes-selective amplitude reduction of parietal alpha-band oscillations-has been reliably linked to processing of visual CS+. It is, however, unclear to what extent auditory conditioned cues prompt similar changes, how these changes evolve as learning progresses, and how alpha reduction in the auditory domain generalizes to similar stimuli. To address these questions, 55 participants listened to three sine wave tones, with either the highest or lowest pitch (CS+) being associated with a noxious white noise burst. A threat-specific (CS+) reduction in occipital-parietal alpha-band power was observed similar to changes expected for visual stimuli. No evidence for aversive generalization to the tone most similar to the CS+ was observed in terms of alpha-band power changes, aversiveness ratings, or pupil dilation. By-trial analyses found that selective alpha-band changes continued to increase as aversive conditioning continued, beyond when participants reported awareness of the contingencies. The results support a theoretical model in which selective alpha power represents a cross-modal index of continuous aversive learning, accompanied by sustained sensory discrimination of conditioned threat from safety cues.

Intolerance of uncertainty affects the behavioral and neural mechanisms of higher generalization.

Intolerance of uncertainty (IU) is associated with several anxiety disorders. In this study, we employed rewards and losses as unconditioned positive and negative stimuli, respectively, to explore the effects of an individual's IU level on positive and negative generalizations using magnetic resonance imaging technology. Following instrumental learning, 48 participants (24 high IU; 24 low IU) were invited to complete positive and negative generalization tasks; their behavioral responses and neural activities were recorded by functional magnetic resonance imaging. The behavior results demonstrated that participants with high IUs exhibited higher generalizations to both positive and negative cues as compared with participants having low IUs. Neuroimaging results demonstrated that they exhibited higher activation levels in the right anterior insula and the default mode network (i.e. precuneus and posterior cingulate gyrus), as well as related reward circuits (i.e. caudate and right putamen). Therefore, higher generalization scores and the related abnormal brain activation may be key markers of IU as a vulnerability factor for anxiety disorders.

Adaptive adjustment after conflict with group opinion: evidence from neural electrophysiology.

Individuals inherently seek social consensus when making decisions or judgments. Previous studies have consistently indicated that dissenting group opinions are perceived as social conflict that demands attitude adjustment. However, the neurocognitive processes of attitude adjustment are unclear. In this electrophysiological study, participants were recruited to perform a face attractiveness judgment task. After forming their own judgment of a face, participants were informed of a purported group judgment (either consistent or inconsistent with their judgment), and then, critically, the same face was presented again. The neural responses to the second presented faces were measured. The second presented faces evoked a larger late positive potential after conflict with group opinions than those that did not conflict, suggesting that more motivated attention was allocated to stimulus. Moreover, faces elicited greater midfrontal theta (4-7 Hz) power after conflict with group opinions than after consistency with group opinions, suggesting that cognitive control was initiated to support attitude adjustment. Furthermore, the mixed-effects model revealed that single-trial theta power predicted behavioral change in the Conflict condition, but not in the No-Conflict condition. These findings provide novel insights into the neurocognitive processes underlying attitude adjustment, which is crucial to behavioral change during conformity.

Adult neurogenesis acts as a neural regularizer.

New neurons are continuously generated in the subgranular zone of the dentate gyrus throughout adulthood. These new neurons gradually integrate into hippocampal circuits, forming new naive synapses. Viewed from this perspective, these new neurons may represent a significant source of "wiring" noise in hippocampal networks. In machine learning, such noise injection is commonly used as a regularization technique. Regularization techniques help prevent overfitting training data and allow models to generalize learning to new, unseen data. Using a computational modeling approach, here we ask whether a neurogenesis-like process similarly acts as a regularizer, facilitating generalization in a category learning task. In a convolutional neural network (CNN) trained on the CIFAR-10 object recognition dataset, we modeled neurogenesis as a replacement/turnover mechanism, where weights for a randomly chosen small subset of hidden layer neurons were reinitialized to new values as the model learned to categorize 10 different classes of objects. We found that neurogenesis enhanced generalization on unseen test data compared to networks with no neurogenesis. Moreover, neurogenic networks either outperformed or performed similarly to networks with conventional noise injection (i.e., dropout, weight decay, and neural noise). These results suggest that neurogenesis can enhance generalization in hippocampal learning through noise injection, expanding on the roles that neurogenesis may have in cognition.

Curriculum learning for human compositional generalization.

Generalization (or transfer) is the ability to repurpose knowledge in novel settings. It is often asserted that generalization is an important ingredient of human intelligence, but its extent, nature, and determinants have proved controversial. Here, we examine this ability with a paradigm that formalizes the transfer learning problem as one of recomposing existing functions to solve unseen problems. We find that people can generalize compositionally in ways that are elusive for standard neural networks and that human generalization benefits from training regimes in which items are axis aligned and temporally correlated. We describe a neural network model based around a Hebbian gating process that can capture how human generalization benefits from different training curricula. We additionally find that adult humans tend to learn composable functions asynchronously, exhibiting discontinuities in learning that resemble those seen in child development.

MFNet: Meta-learning based on frequency-space mix for MRI segmentation in nasopharyngeal carcinoma.

Deep learning techniques have been applied to medical image segmentation and demonstrated expert-level performance. Due to the poor generalization abilities of the models in the deployment in different centres, common solutions, such as transfer learning and domain adaptation techniques, have been proposed to mitigate this issue. However, these solutions necessitate retraining the models with target domain data and annotations, which limits their deployment in clinical settings in unseen domains. We evaluated the performance of domain generalization methods on the task of MRI segmentation of nasopharyngeal carcinoma (NPC) by collecting a new dataset of 321 patients with manually annotated MRIs from two hospitals. We transformed the modalities of MRI, including T1WI, T2WI and CE-T1WI, from the spatial domain to the frequency domain using Fourier transform. To address the bottleneck of domain generalization in MRI segmentation of NPC, we propose a meta-learning approach based on frequency domain feature mixing. We evaluated the performance of MFNet against existing techniques for generalizing NPC segmentation in terms of Dice and MIoU. Our method evidently outperforms the baseline in handling the generalization of NPC segmentation. The MF-Net clearly demonstrates its effectiveness for generalizing NPC MRI segmentation to unseen domains (Dice = 67.59%, MIoU = 75.74% T1W1). MFNet enhances the model's generalization capabilities by incorporating mixed-feature meta-learning. Our approach offers a novel perspective to tackle the domain generalization problem in the field of medical imaging by effectively exploiting the unique characteristics of medical images.

SiMix: A domain generalization method for cross-site brain MRI harmonization via site mixing.

Brain magnetic resonance imaging (MRI) is widely used in clinical practice for disease diagnosis. However, MRI scans acquired at different sites can have different appearances due to the difference in the hardware, pulse sequence, and imaging parameter. It is important to reduce or eliminate such cross-site variations with brain MRI harmonization so that downstream image processing and analysis is performed consistently. Previous works on the harmonization problem require the data acquired from the sites of interest for model training. But in real-world scenarios there can be test data from a new site of interest after the model is trained, and training data from the new site is unavailable when the model is trained. In this case, previous methods cannot optimally handle the test data from the new unseen site. To address the problem, in this work we explore domain generalization for brain MRI harmonization and propose Site Mix (SiMix). We assume that images of travelling subjects are acquired at a few existing sites for model training. To allow the training data to better represent the test data from unseen sites, we first propose to mix the training images belonging to different sites stochastically, which substantially increases the diversity of the training data while preserving the authenticity of the mixed training images. Second, at test time, when a test image from an unseen site is given, we propose a multiview strategy that perturbs the test image with preserved authenticity and ensembles the harmonization results of the perturbed images for improved harmonization quality. To validate SiMix, we performed experiments on the publicly available SRPBS dataset and MUSHAC dataset that comprised brain MRI acquired at nine and two different sites, respectively. The results indicate that SiMix improves brain MRI harmonization for unseen sites, and it is also beneficial to the harmonization of existing sites.

Exaggerated sensitivity to threat and reduced medial prefrontal engagement during threat generalization in reactive aggressive adolescents.

Aggressive adolescents tend to exhibit abnormal fear acquisition and extinction, and reactive aggressive adolescents are often more anxious. However, the relationship between fear generalization and reactive aggression (RA) remains unknown. According to Reactive-Proactive Aggression Questionnaire (RPQ) scores, 61 adolescents were divided into two groups, namely, a high RA group (N = 30) and a low aggression (LA) group (N = 31). All participants underwent three consecutive phases of the Pavlovian conditioning paradigm (i.e., habituation, acquisition, and generalization), and neural activation of the medial prefrontal cortex (mPFC) was assessed by functional near-infrared spectroscopy (fNIRS). The stimuli were ten circles with varying sizes, including two conditioned stimuli (CSs) and eight generalization stimuli (GSs). A scream at 85 dB served as the auditory unconditioned stimulus (US). The US expectancy ratings of both CSs and GSs were higher in the RA group than in the LA group. The fNIRS results showed that CSs and GSs evoked lower mPFC activation in the RA group compared to the LA group during fear generalization. These findings suggest that abnormalities in fear acquisition and generalization are prototypical dysregulations in adolescents with RA. They provide neurocognitive evidence for dysregulated fear learning in the mechanisms underlying adolescents with RA, highlighting the need to develop emotional regulation interventions for these individuals.

Sensorimotor adaptation to a non-uniform formant perturbation generalizes to untrained vowels.

When speakers learn to change the way they produce a speech sound, how much does that learning generalize to other speech sounds? Past studies of speech sensorimotor learning have typically tested the generalization of a single transformation learned in a single context. Here, we investigate the ability of the speech motor system to generalize learning when multiple opposing sensorimotor transformations are learned in separate regions of the vowel space. We find that speakers adapt to a non-uniform "centralization" perturbation, learning to produce vowels with greater acoustic contrast, and that this adaptation generalizes to untrained vowels, which pattern like neighboring trained vowels and show increased contrast of a similar magnitude.

Motion state-dependent motor learning based on explicit visual feedback has limited spatiotemporal properties compared with adaptation to physical perturbations.

We recently showed that subjects can learn motion state-dependent changes to motor output (temporal force patterns) based on explicit visual feedback of the equivalent force field (i.e., without the physical perturbation). Here, we examined the spatiotemporal properties of this learning compared with learning based on physical perturbations. There were two human subject groups and two experimental paradigms. One group (n = 40) experienced physical perturbations (i.e., a velocity-dependent force field, vFF), whereas the second (n = 40) was given explicit visual feedback (EVF) of the force-velocity relationship. In the latter, subjects moved in force channels and we provided visual feedback of the lateral force exerted during the movement, as well as the required force pattern based on movement velocity. In the first paradigm (spatial generalization), following vFF or EVF training, generalization of learning was tested by requiring subjects to move to 14 untrained target locations (0° to ±135° around the trained location). In the second paradigm (temporal stability), following training, we examined the decay of learning over eight delay periods (0 to 90 s). Results showed that learning based on EVF did not generalize to untrained directions, whereas the generalization for the vFF was significant for targets ≤ 45° away. In addition, the decay of learning for the EVF group was significantly faster than the FF group (a time constant of 2.72 ± 1.74 s vs. 12.53 ± 11.83 s). Collectively, our results suggest that recalibrating motor output based on explicit motion state information, in contrast to physical disturbances, uses learning mechanisms with limited spatiotemporal properties.NEW & NOTEWORTHY Adjustment of motor output based on limb motion state information can be achieved based on explicit information or from physical perturbations. Here, we investigated the spatiotemporal characteristics of short-term motor learning to determine the properties of the respective learning mechanisms. Our results suggest that adjustments based on physical perturbations are more temporally stable and applied over a greater spatial range than the learning based on explicit visual feedback, suggesting largely separate learning mechanisms.

GABAergic interneurons in the hippocampal CA1 mediate contextual fear generalization in PTSD rats.

Fear overgeneralization is widely accepted as a pathogenic marker of post-traumatic stress disorder (PTSD). Recently, GABAergic interneurons have been regarded as key players in the regulation of fear memory. The role of hippocampal GABAergic interneurons in contextual fear generalization of PTSD remains incompletely understood. In the present study, we established a rat model of PTSD with inescapable foot shocks (IFS) and observed the loss of GABAergic interneuron phenotype in the hippocampal cornu ammonis-1 (CA1) subfield. To determine whether the loss of GABAergic interneuron phenotype was associated with fear generalization in PTSD rats, we used adeno-associated virus (AAV) to reduce the expression of GAD67 in CA1 and observed its effect on fear generalization. The results showed that the reduction of GAD67 in CA1 enhanced contextual fear generalization in rats. We investigated whether the PERK pathway was involved in the GABAergic interneuron injury. Increased expression of p-PERK, CHOP, and Caspase12 in GABAergic interneurons of PTSD rats was observed. Then, we used salubrinal, an endoplasmic reticulum stress inhibitor, to modulate the PERK pathway. The salubrinal treatment significantly protected the GABAergic interneurons and relieved fear generalization in PTSD rats. In addition, the results showed that salubrinal down-regulated the expression of CHOP and Caspase12 in GABAergic interneurons of PTSD rats. In conclusion, this study provided evidence that the loss of GABAergic interneuron phenotype in CA1 may contribute to contextual fear generalization in PTSD. The PERK pathway is involved in the GABAergic interneuron injury of PTSD rats and modulating it can protect GABAergic interneurons and constrain contextual fear generalization.

Disease-driven domain generalization for neuroimaging-based assessment of Alzheimer's disease.

Development of deep learning models to evaluate structural brain changes caused by cognitive impairment in MRI scans holds significant translational value. The efficacy of these models often encounters challenges due to variabilities arising from different data generation protocols, imaging equipment, radiological artifacts, and shifts in demographic distributions. Domain generalization (DG) techniques show promise in addressing these challenges by enabling the model to learn from one or more source domains and apply this knowledge to new, unseen target domains. Here we present a framework that utilizes model interpretability to enhance the generalizability of classification models across various cohorts. We used MRI scans and clinical diagnoses from four independent cohorts: Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 1821), the Framingham Heart Study (FHS, n = 304), the Australian Imaging Biomarkers & Lifestyle Study of Ageing (AIBL, n = 661), and the National Alzheimer's Coordinating Center (NACC, n = 4647). With this data, we trained a deep neural network to focus on areas of the brain identified as relevant to the disease for model training. Our approach involved training a classifier to differentiate between structural neurodegeneration in individuals with normal cognition (NC), mild cognitive impairment (MCI), and dementia due to Alzheimer's disease (AD). This was achieved by aligning class-wise attention with a unified visual saliency prior, which was computed offline for each class using all the training data. Our method not only competes with state-of-the-art approaches but also shows improved correlation with postmortem histology. This alignment with the gold standard evidence is a significant step towards validating the effectiveness of DG frameworks, paving the way for their broader application in the field.

AMPA receptor potentiation alleviates NLRP3 knockout-induced fear generalization in mice.

Genetic knockout and pharmaceutical inhibition of the NLRP3 inflammasome enhances the extinction of contextual fear memory, which is attributed to its role in neuronal and synaptic dysregulation, concurrent with neurotransmitter function disturbances. This study aimed to determine whether NLRP3 plays a role in generalizing fear via the inflammatory axis. We established the NLRP3 KO mice model, followed by behavioral and biochemical analyses. The NLRP3 KO mice displayed impaired fear generalization, lower neuroinflammation levels, and dysregulated neurotransmitter function. Additionally, α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptors, but not the inhibition of NMDA or 5-HT2C receptors, resulted in fear generalization in NLRP3 KO mice because TAT-GluA2 3Y, but not SB242084 and D-cycloserine, treated blocked NLRP3 deprivation effects on fear generalization. Thus, global knockout of NLRP3 is associated with aberrant fear generalization, possibly through AMPA receptor signaling.