Cell-type-specific population dynamics of diverse reward computations.

Computational analysis of cellular activity has developed largely independently of modern transcriptomic cell typology, but integrating these approaches may be essential for full insight into cellular-level mechanisms underlying brain function and dysfunction. Applying this approach to the habenula (a structure with diverse, intermingled molecular, anatomical, and computational features), we identified encoding of reward-predictive cues and reward outcomes in distinct genetically defined neural populations, including TH+ cells and Tac1+ cells. Data from genetically targeted recordings were used to train an optimized nonlinear dynamical systems model and revealed activity dynamics consistent with a line attractor. High-density, cell-type-specific electrophysiological recordings and optogenetic perturbation provided supporting evidence for this model. Reverse-engineering predicted how Tac1+ cells might integrate reward history, which was complemented by in vivo experimentation. This integrated approach describes a process by which data-driven computational models of population activity can generate and frame actionable hypotheses for cell-type-specific investigation in biological systems.

Event Integration and Temporal Differentiation: How Hierarchical Knowledge Emerges in Hippocampal Subfields through Learning.

Everyday life is composed of events organized by changes in contexts, with each event containing an unfolding sequence of occurrences. A major challenge facing our memory systems is how to integrate sequential occurrences within events while also maintaining their details and avoiding over-integration across different contexts. We asked if and how distinct hippocampal subfields come to hierarchically and, in parallel, represent both event context and subevent occurrences with learning. Female and male human participants viewed sequential events defined as sequences of objects superimposed on shared color frames while undergoing high-resolution fMRI. Importantly, these events were repeated to induce learning. Event segmentation, as indexed by increased reaction times at event boundaries, was observed in all repetitions. Temporal memory decisions were quicker for items from the same event compared to across different events, indicating that events shaped memory. With learning, hippocampal CA3 multivoxel activation patterns clustered to reflect the event context, with more clustering correlated with behavioral facilitation during event transitions. In contrast, in the dentate gyrus (DG), temporally proximal items that belonged to the same event became associated with more differentiated neural patterns. A computational model explained these results by dynamic inhibition in the DG. Additional similarity measures support the notion that CA3 clustered representations reflect shared voxel populations, while DG's distinct item representations reflect different voxel populations. These findings suggest an interplay between temporal differentiation in the DG and attractor dynamics in CA3. They advance our understanding of how knowledge is structured through integration and separation across time and context.

Flexible integration of continuous sensory evidence in perceptual estimation tasks.

Temporal accumulation of evidence is crucial for making accurate judgments based on noisy or ambiguous sensory input. The integration process leading to categorical decisions is thought to rely on competition between neural populations, each encoding a discrete categorical choice. How recurrent neural circuits integrate evidence for continuous perceptual judgments is unknown. Here, we show that a continuous bump attractor network can integrate a circular feature, such as stimulus direction, nearly optimally. As required by optimal integration, the population activity of the network unfolds on a two-dimensional manifold, in which the position of the network's activity bump tracks the stimulus average, and, simultaneously, the bump amplitude tracks stimulus uncertainty. Moreover, the temporal weighting of sensory evidence by the network depends on the relative strength of the stimulus compared to the internally generated bump dynamics, yielding either early (primacy), uniform, or late (recency) weighting. The model can flexibly switch between these regimes by changing a single control parameter, the global excitatory drive. We show that this mechanism can quantitatively explain individual temporal weighting profiles of human observers, and we validate the model prediction that temporal weighting impacts reaction times. Our findings point to continuous attractor dynamics as a plausible neural mechanism underlying stimulus integration in perceptual estimation tasks.

Microcircuit Synchronization and Heavy-Tailed Synaptic Weight Distribution Augment preBötzinger Complex Bursting Dynamics.

The preBötzinger Complex (preBötC) encodes inspiratory time as rhythmic bursts of activity underlying each breath. Spike synchronization throughout a sparsely connected preBötC microcircuit initiates bursts that ultimately drive the inspiratory motor patterns. Using minimal microcircuit models to explore burst initiation dynamics, we examined the variability in probability and latency to burst following exogenous stimulation of a small subset of neurons, mimicking experiments. Among various physiologically plausible graphs of 1000 excitatory neurons constructed using experimentally determined synaptic and connectivity parameters, directed Erdos-Rényi graphs with a broad (lognormal) distribution of synaptic weights best captured the experimentally observed dynamics. preBötC synchronization leading to bursts was regulated by the efferent connectivity of spiking neurons that are optimally tuned to amplify modest preinspiratory activity through input convergence. Using graph-theoretic and machine learning-based analyses, we found that input convergence of efferent connectivity at the next-nearest neighbor order was a strong predictor of incipient synchronization. Our analyses revealed a crucial role of synaptic heterogeneity in imparting exceptionally robust yet flexible preBötC attractor dynamics. Given the pervasiveness of lognormally distributed synaptic strengths throughout the nervous system, we postulate that these mechanisms represent a ubiquitous template for temporal processing and decision-making computational motifs.SIGNIFICANCE STATEMENT Mammalian breathing is robust, virtually continuous throughout life, yet is inherently labile: to adapt to rapid metabolic shifts (e.g., fleeing a predator or chasing prey); for airway reflexes; and to enable nonventilatory behaviors (e.g., vocalization, breathholding, laughing). Canonical theoretical frameworks-based on pacemakers and intrinsic bursting-cannot account for the observed robustness and flexibility of the preBötzinger Complex rhythm. Experiments reveal that network synchronization is the key to initiate inspiratory bursts in each breathing cycle. We investigated preBötC synchronization dynamics using network models constructed with experimentally determined neuronal and synaptic parameters. We discovered that a fat-tailed (non-Gaussian) synaptic weight distribution-a manifestation of synaptic heterogeneity-augments neuronal synchronization and attractor dynamics in this vital rhythmogenic network, contributing to its extraordinary reliability and responsiveness.

Attractor-like Dynamics in the Subicular Complex.

Distinct computations are performed at multiple brain regions during the encoding of spatial environments. Neural representations in the hippocampal, entorhinal, and head direction (HD) networks during spatial navigation have been clearly documented, while the representational properties of the subicular complex (SC) are relatively underexplored, although it has extensive anatomic connections with various brain regions involved in spatial information processing. We simultaneously recorded single units from different subregions of the SC in male rats while they ran clockwise on a centrally placed textured circular track (four different textures, each covering a quadrant), surrounded by six distal cues. The neural activity was monitored in standard sessions by maintaining the same configuration between the cues, while in cue manipulation sessions, the distal and local cues were either rotated in opposite directions to create a mismatch between them or the distal cues were removed. We report a highly coherent neural representation of the environment and a robust coupling between the HD cells and the spatial cells in the SC, strikingly different from previous reports of coupling between cells from co-recorded sites. Neural representations were (1) originally governed by the distal cues under local-distal cue-conflict conditions, (2) controlled by the local cues in the absence of distal cues, and (3) governed by the cues that are perceived to be stable. We propose that such attractor-like dynamics in the SC might play a critical role in the orientation of spatial representations, thus providing a "reference map" of the environment for further processing by other networks.SIGNIFICANCE STATEMENT The subicular complex (SC) receives major inputs from the entorhinal cortex and the hippocampus, and head direction (HD) information directly from the HD system. Using cue-conflict experiments, we studied the hierarchical representation of the local and distal cues in the SC to understand its role in the cognitive map, and report a highly coherent neural representation with robust coupling between the HD cells and the spatial cells in different subregions of the SC exhibiting attractor-like dynamics unaffected by the cue manipulations, strikingly different from previous reports of coupling between cells from co-recorded sites. This unique feature may allow the SC to function as a single computational unit during the representation of space, which may serve as a reference map of the environment.

Global inhibition in head-direction neural circuits: a systematic comparison between connectome-based spiking neural circuit models.

The recent discovery of the head-direction (HD) system in fruit flies has provided unprecedented insights into the neural mechanisms of spatial orientation. Despite the progress, the neural substance of global inhibition, an essential component of the HD circuits, remains controversial. Some studies suggested that the ring neurons provide global inhibition, while others suggested the Δ7 neurons. In the present study, we provide evaluations from the theoretical perspective by performing systematic analyses on the computational models based on the ring-neuron (R models) and Δ7-neurons (Delta models) hypotheses with modifications according to the latest connectomic data. We conducted four tests: robustness, persistency, speed, and dynamical characteristics. We discovered that the two models led to a comparable performance in general, but each excelled in different tests. The R Models were more robust, while the Delta models were better in the persistency test. We also tested a hybrid model that combines both inhibitory mechanisms. While the performances of the R and Delta models in each test are highly parameter-dependent, the Hybrid model performed well in all tests with the same set of parameters. Our results suggest the possibility of combined inhibitory mechanisms in the HD circuits of fruit flies.

Strength training as a dynamical model of motor learning.

This paper outlines a framework for strength training as a dynamical model of perceptual-motor learning. We show, with emphasis on fixed-point attractor dynamics, that strength training can be mapped to the general dynamical principles of motor learning that arise from the constraints on action, including the distribution of practice/training. The time scales of the respective dynamics of performance change (increment and decrement) in discrete strength training and motor learning tasks reveal superposition of exponential functions in fixed-point dynamics, but distinctive attractor and parameter dynamics in oscillatory limit cycle and more continuous tasks, together with unique timescales to process influences (including practice, learning, strength, fitness, fatigue, warm-up decrement). Increments and decrements of strength can be viewed within a dynamical model of change in motor performance that reflects the integration of practice and training processes at multiple levels of learning and skill development.

Sequential Attractors in Combinatorial Threshold-Linear Networks.

Sequences of neural activity arise in many brain areas, including cortex, hippocampus, and central pattern generator circuits that underlie rhythmic behaviors like locomotion. While network architectures supporting sequence generation vary considerably, a common feature is an abundance of inhibition. In this work, we focus on architectures that support sequential activity in recurrently connected networks with inhibition-dominated dynamics. Specifically, we study emergent sequences in a special family of threshold-linear networks, called combinatorial threshold-linear networks (CTLNs), whose connectivity matrices are defined from directed graphs. Such networks naturally give rise to an abundance of sequences whose dynamics are tightly connected to the underlying graph. We find that architectures based on generalizations of cycle graphs produce limit cycle attractors that can be activated to generate transient or persistent (repeating) sequences. Each architecture type gives rise to an infinite family of graphs that can be built from arbitrary component subgraphs. Moreover, we prove a number of graph rules for the corresponding CTLNs in each family. The graph rules allow us to strongly constrain, and in some cases fully determine, the fixed points of the network in terms of the fixed points of the component subnetworks. Finally, we also show how the structure of certain architectures gives insight into the sequential dynamics of the corresponding attractor.

Emergent elasticity in the neural code for space.

Upon encountering a novel environment, an animal must construct a consistent environmental map, as well as an internal estimate of its position within that map, by combining information from two distinct sources: self-motion cues and sensory landmark cues. How do known aspects of neural circuit dynamics and synaptic plasticity conspire to accomplish this feat? Here we show analytically how a neural attractor model that combines path integration of self-motion cues with Hebbian plasticity in synaptic weights from landmark cells can self-organize a consistent map of space as the animal explores an environment. Intriguingly, the emergence of this map can be understood as an elastic relaxation process between landmark cells mediated by the attractor network. Moreover, our model makes several experimentally testable predictions, including (i) systematic path-dependent shifts in the firing fields of grid cells toward the most recently encountered landmark, even in a fully learned environment; (ii) systematic deformations in the firing fields of grid cells in irregular environments, akin to elastic deformations of solids forced into irregular containers; and (iii) the creation of topological defects in grid cell firing patterns through specific environmental manipulations. Taken together, our results conceptually link known aspects of neurons and synapses to an emergent solution of a fundamental computational problem in navigation, while providing a unified account of disparate experimental observations.

A speed-accurate self-sustaining head direction cell path integration model without recurrent excitation.

The head direction (HD) system signals HD in an allocentric frame of reference. The system is able to update firing based on internally derived information about self-motion, a process known as path integration. Of particular interest is how path integration might maintain concordance between true HD and internally represented HD. Here we present a self-sustaining two-layer model, capable of self-organizing, which produces extremely accurate path integration. The implications of this work for future investigations of HD system path integration are discussed.

A tweaking principle for executive control: neuronal circuit mechanism for rule-based task switching and conflict resolution.

A hallmark of executive control is the brain's agility to shift between different tasks depending on the behavioral rule currently in play. In this work, we propose a "tweaking hypothesis" for task switching: a weak rule signal provides a small bias that is dramatically amplified by reverberating attractor dynamics in neural circuits for stimulus categorization and action selection, leading to an all-or-none reconfiguration of sensory-motor mapping. Based on this principle, we developed a biologically realistic model with multiple modules for task switching. We found that the model quantitatively accounts for complex task switching behavior: switch cost, congruency effect, and task-response interaction; as well as monkey's single-neuron activity associated with task switching. The model yields several testable predictions, in particular, that category-selective neurons play a key role in resolving sensory-motor conflict. This work represents a neural circuit model for task switching and sheds insights in the brain mechanism of a fundamental cognitive capability.

The GARD Prebiotic Reproduction Model Described in Order and Complexity.

Early steps in the origin of life were necessarily connected to the unlikely formation of self-reproducing structures from chaotic chemistry. Simulations of chemical kinetics based on the graded autocatalysis replication domain (GARD) model demonstrate the ability of a micellar system to become self-reproducing units away from equilibrium. Even though they may be very rare in the initial state of the system, the property of their endogenous mutually catalytic networks being dynamic attractors greatly enhanced reproduction propensity, revealing their potential for selection and Darwinian evolution processes. In parallel, order and complexity have been shown to be crucial parameters in successful evolution. Here, we probe these parameters in the dynamics of GARD-governed entities in an attempt to identify characteristic mechanisms of their development in non-covalent molecular assemblies. Using a virtual random walk perspective, a value for consecutive order is defined based on statistical thermodynamics. The complexity, on the other hand, is determined by the size of a minimal algorithm fully describing the statistical properties of the random walk. By referring to a previously published diagonal line in an order/complexity diagram that represents the progression of evolution, it is shown that the GARD model has the potential to advance in this direction. These results can serve as a solid foundation for identifying general criteria for future analyses of evolving systems.

Organization of hippocampal CA3 into correlated cell assemblies supports a stable spatial code.

Hippocampal subfield CA3 is thought to stably store memories in assemblies of recurrently connected cells functioning as a collective. However, the collective hippocampal coding properties that are unique to CA3 and how such properties facilitate the stability or precision of the neural code remain unclear. Here, we performed large-scale Ca2+ imaging in hippocampal CA1 and CA3 of freely behaving mice that repeatedly explored the same, initially novel environments over weeks. CA3 place cells have more precise and more stable tuning and show a higher statistical dependence with their peers compared with CA1 place cells, uncovering a cell assembly organization in CA3. Surprisingly, although tuning precision and long-term stability are correlated, cells with stronger peer dependence exhibit higher stability but not higher precision. Overall, our results expose the three-way relationship between tuning precision, long-term stability, and peer dependence, suggesting that a cell assembly organization underlies long-term storage of information in the hippocampus.

Understanding cingulotomy's therapeutic effect in OCD through computer models.

Cingulotomy is therapeutic in OCD, but what are the possible mechanisms? Computer models that formalize cortical OCD abnormalities and anterior cingulate cortex (ACC) function can help answer this. At the neural dynamics level, cortical dynamics in OCD have been modeled using attractor networks, where activity patterns resistant to change denote the inability to switch to new patterns, which can reflect inflexible thinking patterns or behaviors. From that perspective, cingulotomy might reduce the influence of difficult-to-escape ACC attractor dynamics on other cortical areas. At the functional level, computer formulations based on model-free reinforcement learning (RL) have been used to describe the multitude of phenomena ACC is involved in, such as tracking the timing of expected outcomes and estimating the cost of exerting cognitive control and effort. Different elements of model-free RL models of ACC could be affected by the inflexible cortical dynamics, making it challenging to update their values. An agent can also use a world model, a representation of how the states of the world change, to plan its actions, through model-based RL. OCD has been hypothesized to be driven by reduced certainty of how the brain's world model describes changes. Cingulotomy might improve such uncertainties about the world and one's actions, making it possible to trust the outcomes of these actions more and thus reduce the urge to collect more sensory information in the form of compulsions. Connecting the neural dynamics models with the functional formulations can provide new ways of understanding the role of ACC in OCD, with potential therapeutic insights.

Selective connectivity enhances storage capacity in attractor models of memory function.

Autoassociative neural networks provide a simple model of how memories can be stored through Hebbian synaptic plasticity as retrievable patterns of neural activity. Although progress has been made along the last decades in understanding the biological implementation of autoassociative networks, their modest theoretical storage capacity has remained a major constraint. While most previous approaches utilize randomly connected networks, here we explore the possibility of optimizing network performance by selective connectivity between neurons, that could be implemented in the brain through creation and pruning of synaptic connections. We show through numerical simulations that a reconfiguration of the connectivity matrix can improve the storage capacity of autoassociative networks up to one order of magnitude compared to randomly connected networks, either by reducing the noise or by making it reinforce the signal. Our results indicate that the signal-reinforcement scenario is not only the best performing but also the most adequate for brain-like highly diluted connectivity. In this scenario, the optimized network tends to select synapses characterized by a high consensus across stored patterns. We also introduced an online algorithm in which the network modifies its connectivity while learning new patterns. We observed that, similarly to what happens in the human brain, creation of connections dominated in an initial stage, followed by a stage characterized by pruning, leading to an equilibrium state that was independent of the initial connectivity of the network. Our results suggest that selective connectivity could be a key component to make attractor networks in the brain viable in terms of storage capacity.

Flexible rerouting of hippocampal replay sequences around changing barriers in the absence of global place field remapping.

Flexibility is a hallmark of memories that depend on the hippocampus. For navigating animals, flexibility is necessitated by environmental changes such as blocked paths and extinguished food sources. To better understand the neural basis of this flexibility, we recorded hippocampal replays in a spatial memory task where barriers as well as goals were moved between sessions to see whether replays could adapt to new spatial and reward contingencies. Strikingly, replays consistently depicted new goal-directed trajectories around each new barrier configuration and largely avoided barrier violations. Barrier-respecting replays were learned rapidly and did not rely on place cell remapping. These data distinguish sharply between place field responses, which were largely stable and remained tied to sensory cues, and replays, which changed flexibly to reflect the learned contingencies in the environment and suggest sequenced activations such as replay to be an important link between the hippocampus and flexible memory.

To stay or not to stay: The stability of choice perseveration in value-based decision making.

In real life, decisions are often naturally embedded in decision sequences. In contrast, in the laboratory, decisions are oftentimes analysed in isolation. Here, we investigated the influence of decision sequences in value-based decision making and whether the stability of such effects can be modulated. In our decision task, participants needed to collect rewards in a virtual two-dimensional world. We presented a series of two reward options that were either quick to collect but were smaller in value or took longer to collect but were larger in value. The subjective value of each option was driven by the options' value and how quickly they could be reached. We manipulated the subjective values of the options so that one option became gradually less valuable over the course of a sequence, which allowed us to measure choice perseveration (i.e., how long participants stick to this option). In two experiments, we further manipulated the time interval between two trials (inter-trial interval), and the time delay between the onsets of both reward options (stimulus onset asynchrony). We predicted how these manipulations would affect choice perseveration using a computational attractor model. Our results indicate that both the inter-trial interval and the stimulus onset asynchrony modulate choice perseveration as predicted by the model. We discuss how our findings extend to research on cognitive stability and flexibility.

Compositional memory in attractor neural networks with one-step learning.

Compositionality refers to the ability of an intelligent system to construct models out of reusable parts. This is critical for the productivity and generalization of human reasoning, and is considered a necessary ingredient for human-level artificial intelligence. While traditional symbolic methods have proven effective for modeling compositionality, artificial neural networks struggle to learn systematic rules for encoding generalizable structured models. We suggest that this is due in part to short-term memory that is based on persistent maintenance of activity patterns without fast weight changes. We present a recurrent neural network that encodes structured representations as systems of contextually-gated dynamical attractors called attractor graphs. This network implements a functionally compositional working memory that is manipulated using top-down gating and fast local learning. We evaluate this approach with empirical experiments on storage and retrieval of graph-based data structures, as well as an automated hierarchical planning task. Our results demonstrate that compositional structures can be stored in and retrieved from neural working memory without persistent maintenance of multiple activity patterns. Further, memory capacity is improved by the use of a fast store-erase learning rule that permits controlled erasure and mutation of previously learned associations. We conclude that the combination of top-down gating and fast associative learning provides recurrent neural networks with a robust functional mechanism for compositional working memory.

Bad Timing for Epileptic Networks: Role of Temporal Dynamics in Seizures and Cognitive Deficits.

The precise coordination of neuronal activity is critical for optimal brain function. When such coordination fails, this can lead to dire consequences. In this review, I will present evidence that in epilepsy, failed coordination leads not only to seizures but also to alterations of the rhythmical patterns observed in the electroencephalogram and cognitive deficits. Restoring the dynamic coordination of epileptic networks could therefore both improve seizures and cognitive outcomes.

Interaction between neuronal encoding and population dynamics during categorization task switching in parietal cortex.

Primates excel at categorization, a cognitive process for assigning stimuli into behaviorally relevant groups. Categories are encoded in multiple brain areas and tasks, yet it remains unclear how neural encoding and dynamics support cognitive tasks with different demands. We recorded from parietal cortex during flexible switching between categorization tasks with distinct cognitive and motor demands and also studied recurrent neural networks (RNNs) trained on the same tasks. In the one-interval categorization task (OIC), monkeys rapidly reported their decisions with a saccade. In the delayed match-to-category (DMC) task, monkeys decided whether sequentially presented stimuli were categorical matches. Neuronal category encoding generalized across tasks, but categorical encoding was more binary-like in the DMC task and more graded in the OIC task. Furthermore, analysis of trained RNNs supports the hypothesis that binary-like encoding in DMC arises through compression of graded feature encoding by attractor dynamics underlying stimulus maintenance and/or comparison in working memory.