Goal-oriented inference of environment from redundant observations.

The agent learns to organize decision behavior to achieve a behavioral goal, such as reward maximization, and reinforcement learning is often used for this optimization. Learning an optimal behavioral strategy is difficult under the uncertainty that events necessary for learning are only partially observable, called as Partially Observable Markov Decision Process (POMDP). However, the real-world environment also gives many events irrelevant to reward delivery and an optimal behavioral strategy. The conventional methods in POMDP, which attempt to infer transition rules among the entire observations, including irrelevant states, are ineffective in such an environment. Supposing Redundantly Observable Markov Decision Process (ROMDP), here we propose a method for goal-oriented reinforcement learning to efficiently learn state transition rules among reward-related "core states" from redundant observations. Starting with a small number of initial core states, our model gradually adds new core states to the transition diagram until it achieves an optimal behavioral strategy consistent with the Bellman equation. We demonstrate that the resultant inference model outperforms the conventional method for POMDP. We emphasize that our model only containing the core states has high explainability. Furthermore, the proposed method suits online learning as it suppresses memory consumption and improves learning speed.

Competition on presynaptic resources enhances the discrimination of interfering memories.

Evidence suggests that hippocampal adult neurogenesis is critical for discriminating considerably interfering memories. During adult neurogenesis, synaptic competition modifies the weights of synaptic connections nonlocally across neurons, thus providing a different form of unsupervised learning from Hebb's local plasticity rule. However, how synaptic competition achieves separating similar memories largely remains unknown. Here, we aim to link synaptic competition with such pattern separation. In synaptic competition, adult-born neurons are integrated into the existing neuronal pool by competing with mature neurons for synaptic connections from the entorhinal cortex. We show that synaptic competition and neuronal maturation play distinct roles in separating interfering memory patterns. Furthermore, we demonstrate that a feedforward neural network trained by a competition-based learning rule can outperform a multilayer perceptron trained by the backpropagation algorithm when only a small number of samples are available. Our results unveil the functional implications and potential applications of synaptic competition in neural computation.

Computational models of Idling brain activity for memory processing.

Studying the underlying neural mechanisms of cognitive functions of the brain is one of the central questions in modern biology. Moreover, it has significantly impacted the development of novel technologies in artificial intelligence. Spontaneous activity is a unique feature of the brain and is currently lacking in many artificially constructed intelligent machines. Spontaneous activity may represent the brain's idling states, which are internally driven by neuronal networks and possibly participate in offline processing during awake, sleep, and resting states. Evidence is accumulating that the brain's spontaneous activity is not mere noise but part of the mechanisms to process information about previous experiences. A bunch of literature has shown how previous sensory and behavioral experiences influence the subsequent patterns of brain activity with various methods in various animals. It seems, however, that the patterns of neural activity and their computational roles differ significantly from area to area and from function to function. In this article, I review the various forms of the brain's spontaneous activity, especially those observed during memory processing, and some attempts to model the generation mechanisms and computational roles of such activities.

A microcircuit model involving parvalbumin, somatostatin, and vasoactive intestinal polypeptide inhibitory interneurons for the modulation of neuronal oscillation during visual processing.

Various subtypes of inhibitory interneurons contact one another to organize cortical networks. Most cortical inhibitory interneurons express 1 of 3 genes: parvalbumin (PV), somatostatin (SOM), or vasoactive intestinal polypeptide (VIP). This diversity of inhibition allows the flexible regulation of neuronal responses within and between cortical areas. However, the exact roles of these interneuron subtypes and of excitatory pyramidal (Pyr) neurons in regulating neuronal network activity and establishing perception (via interactions between feedforward sensory and feedback attentional signals) remain largely unknown. To explore the regulatory roles of distinct neuronal types in cortical computation, we developed a computational microcircuit model with biologically plausible visual cortex layers 2/3 that combined Pyr neurons and the 3 inhibitory interneuron subtypes to generate network activity. In simulations with our model, inhibitory signals from PV and SOM neurons preferentially induced neuronal firing at gamma (30-80 Hz) and beta (20-30 Hz) frequencies, respectively, in agreement with observed physiological results. Furthermore, our model indicated that rapid inhibition from VIP to SOM subtypes underlies marked attentional modulation for low-gamma frequency (30-50 Hz) in Pyr neuron responses. Our results suggest the distinct but cooperative roles of inhibitory interneuron subtypes in the establishment of visual perception.

Amplitude-mediated chimera states in nonlocally coupled Stuart-Landau oscillators.

Chimera states achieve the coexistence of coherent and incoherent subgroups through symmetry breaking and emerge in physical, chemical, and biological systems. We show the presence of amplitude-mediated multicluster chimera states in nonlocally coupled Stuart-Landau oscillators. We clarify the prerequisites for having different types of chimera states by analytically and numerically studying how phase transitions occur between these states. Our results demonstrate how the oscillation amplitudes interact with the phase degrees of freedom in chimera states and significantly advance our understanding of the generation mechanisms of such states in coupled oscillator systems.

Neural circuit mechanisms of hierarchical sequence learning tested on large-scale recording data.

The brain performs various cognitive functions by learning the spatiotemporal salient features of the environment. This learning requires unsupervised segmentation of hierarchically organized spike sequences, but the underlying neural mechanism is only poorly understood. Here, we show that a recurrent gated network of neurons with dendrites can efficiently solve difficult segmentation tasks. In this model, multiplicative recurrent connections learn a context-dependent gating of dendro-somatic information transfers to minimize error in the prediction of somatic responses by the dendrites. Consequently, these connections filter the redundant input features represented by the dendrites but unnecessary in the given context. The model was tested on both synthetic and real neural data. In particular, the model was successful for segmenting multiple cell assemblies repeating in large-scale calcium imaging data containing thousands of cortical neurons. Our results suggest that recurrent gating of dendro-somatic signal transfers is crucial for cortical learning of context-dependent segmentation tasks.

Modeling the Repetition-Based Recovering of Acoustic and Visual Sources With Dendritic Neurons.

In natural auditory environments, acoustic signals originate from the temporal superimposition of different sound sources. The problem of inferring individual sources from ambiguous mixtures of sounds is known as blind source decomposition. Experiments on humans have demonstrated that the auditory system can identify sound sources as repeating patterns embedded in the acoustic input. Source repetition produces temporal regularities that can be detected and used for segregation. Specifically, listeners can identify sounds occurring more than once across different mixtures, but not sounds heard only in a single mixture. However, whether such a behavior can be computationally modeled has not yet been explored. Here, we propose a biologically inspired computational model to perform blind source separation on sequences of mixtures of acoustic stimuli. Our method relies on a somatodendritic neuron model trained with a Hebbian-like learning rule which was originally conceived to detect spatio-temporal patterns recurring in synaptic inputs. We show that the segregation capabilities of our model are reminiscent of the features of human performance in a variety of experimental settings involving synthesized sounds with naturalistic properties. Furthermore, we extend the study to investigate the properties of segregation on task settings not yet explored with human subjects, namely natural sounds and images. Overall, our work suggests that somatodendritic neuron models offer a promising neuro-inspired learning strategy to account for the characteristics of the brain segregation capabilities as well as to make predictions on yet untested experimental settings.

Intrinsic bursts facilitate learning of Lévy flight movements in recurrent neural network models.

Isolated spikes and bursts of spikes are thought to provide the two major modes of information coding by neurons. Bursts are known to be crucial for fundamental processes between neuron pairs, such as neuronal communications and synaptic plasticity. Neuronal bursting also has implications in neurodegenerative diseases and mental disorders. Despite these findings on the roles of bursts, whether and how bursts have an advantage over isolated spikes in the network-level computation remains elusive. Here, we demonstrate in a computational model that not isolated spikes, but intrinsic bursts can greatly facilitate learning of Lévy flight random walk trajectories by synchronizing burst onsets across a neural population. Lévy flight is a hallmark of optimal search strategies and appears in cognitive behaviors such as saccadic eye movements and memory retrieval. Our results suggest that bursting is crucial for sequence learning by recurrent neural networks when sequences comprise long-tailed distributed discrete jumps.

Neural mechanisms for learning hierarchical structures of information.

Spatial and temporal information from the environment is often hierarchically organized, so is our knowledge formed about the environment. Identifying the meaningful segments embedded in hierarchically structured information is crucial for cognitive functions, including visual, auditory, motor, memory, and language processing. Segmentation enables the grasping of the links between isolated entities, offering the basis for reasoning and thinking. Importantly, the brain learns such segmentation without external instructions. Here, we review the underlying computational mechanisms implemented at the single-cell and network levels. The network-level mechanism has an interesting similarity to machine-learning methods for graph segmentation. The brain possibly implements methods for the analysis of the hierarchical structures of the environment at multiple levels of its processing hierarchy.

Astrocyte GluN2C NMDA receptors control basal synaptic strengths of hippocampal CA1 pyramidal neurons in the stratum radiatum.

Experience-dependent plasticity is a key feature of brain synapses for which neuronal N-Methyl-D-Aspartate receptors (NMDARs) play a major role, from developmental circuit refinement to learning and memory. Astrocytes also express NMDARs, although their exact function has remained controversial. Here, we identify in mouse hippocampus, a circuit function for GluN2C NMDAR, a subtype highly expressed in astrocytes, in layer-specific tuning of synaptic strengths in CA1 pyramidal neurons. Interfering with astrocyte NMDAR or GluN2C NMDAR activity reduces the range of presynaptic strength distribution specifically in the stratum radiatum inputs without an appreciable change in the mean presynaptic strength. Mathematical modeling shows that narrowing of the width of presynaptic release probability distribution compromises the expression of long-term synaptic plasticity. Our findings suggest a novel feedback signaling system that uses astrocyte GluN2C NMDARs to adjust basal synaptic weight distribution of Schaffer collateral inputs, which in turn impacts computations performed by the CA1 pyramidal neuron.

Zebrafish capable of generating future state prediction error show improved active avoidance behavior in virtual reality.

Animals make decisions under the principle of reward value maximization and surprise minimization. It is still unclear how these principles are represented in the brain and are reflected in behavior. We addressed this question using a closed-loop virtual reality system to train adult zebrafish for active avoidance. Analysis of the neural activity of the dorsal pallium during training revealed neural ensembles assigning rules to the colors of the surrounding walls. Additionally, one third of fish generated another ensemble that becomes activated only when the real perceived scenery shows discrepancy from the predicted favorable scenery. The fish with the latter ensemble escape more efficiently than the fish with the former ensembles alone, even though both fish have successfully learned to escape, consistent with the hypothesis that the latter ensemble guides zebrafish to take action to minimize this prediction error. Our results suggest that zebrafish can use both principles of goal-directed behavior, but with different behavioral consequences depending on the repertoire of the adopted principles.

Multiscale representations of community structures in attractor neural networks.

Our cognition relies on the ability of the brain to segment hierarchically structured events on multiple scales. Recent evidence suggests that the brain performs this event segmentation based on the structure of state-transition graphs behind sequential experiences. However, the underlying circuit mechanisms are poorly understood. In this paper we propose an extended attractor network model for graph-based hierarchical computation which we call the Laplacian associative memory. This model generates multiscale representations for communities (clusters) of associative links between memory items, and the scale is regulated by the heterogenous modulation of inhibitory circuits. We analytically and numerically show that these representations correspond to graph Laplacian eigenvectors, a popular method for graph segmentation and dimensionality reduction. Finally, we demonstrate that our model exhibits chunked sequential activity patterns resembling hippocampal theta sequences. Our model connects graph theory and attractor dynamics to provide a biologically plausible mechanism for abstraction in the brain.

Concomitant Processing of Choice and Outcome in Frontal Corticostriatal Ensembles Correlates with Performance of Rats.

The frontal cortex-basal ganglia network plays a pivotal role in adaptive goal-directed behaviors. Medial frontal cortex (MFC) encodes information about choices and outcomes into sequential activation of neural population, or neural trajectory. While MFC projects to the dorsal striatum (DS), whether DS also displays temporally coordinated activity remains unknown. We studied this question by simultaneously recording neural ensembles in the MFC and DS of rodents performing an outcome-based alternative choice task. We found that the two regions exhibited highly parallel evolution of neural trajectories, transforming choice information into outcome-related information. When the two trajectories were highly correlated, spike synchrony was task-dependently modulated in some MFC-DS neuron pairs. Our results suggest that neural trajectories concomitantly process decision-relevant information in MFC and DS with increased spike synchrony between these regions.

Oscillation-Driven Memory Encoding, Maintenance, and Recall in an Entorhinal-Hippocampal Circuit Model.

During the execution of working memory tasks, task-relevant information is processed by local circuits across multiple brain regions. How this multiarea computation is conducted by the brain remains largely unknown. To explore such mechanisms in spatial working memory, we constructed a neural network model involving parvalbumin-positive, somatostatin-positive, and vasoactive intestinal polypeptide-positive interneurons in the hippocampal CA1 and the superficial and deep layers of medial entorhinal cortex (MEC). Our model is based on a hypothesis that cholinergic modulations differently regulate information flows across CA1 and MEC at memory encoding, maintenance, and recall during delayed nonmatching-to-place tasks. In the model, theta oscillation coordinates the proper timing of interactions between these regions. Furthermore, the model predicts that MEC is engaged in decoding as well as encoding spatial memory, which we confirmed by experimental data analysis. Thus, our model accounts for the neurobiological characteristics of the cross-area information routing underlying working memory tasks.

Distinct Mechanisms of Over-Representation of Landmarks and Rewards in the Hippocampus.

In the hippocampus, locations associated with salient features are represented by a disproportionately large number of neurons, but the cellular and molecular mechanisms underlying this over-representation remain elusive. Using longitudinal calcium imaging in mice learning to navigate in virtual reality, we find that the over-representation of reward and landmark locations are mediated by persistent and separable subsets of neurons, with distinct time courses of emergence and differing underlying molecular mechanisms. Strikingly, we find that in mice lacking Shank2, an autism spectrum disorder (ASD)-linked gene encoding an excitatory postsynaptic scaffold protein, the learning-induced over-representation of landmarks was absent whereas the over-representation of rewards was substantially increased, as was goal-directed behavior. These findings demonstrate that multiple hippocampal coding processes for unique types of salient features are distinguished by a Shank2-dependent mechanism and suggest that abnormally distorted hippocampal salience mapping may underlie cognitive and behavioral abnormalities in a subset of ASDs.

Somatodendritic consistency check for temporal feature segmentation.

The brain identifies potentially salient features within continuous information streams to process hierarchical temporal events. This requires the compression of information streams, for which effective computational principles are yet to be explored. Backpropagating action potentials can induce synaptic plasticity in the dendrites of cortical pyramidal neurons. By analogy with this effect, we model a self-supervising process that increases the similarity between dendritic and somatic activities where the somatic activity is normalized by a running average. We further show that a family of networks composed of the two-compartment neurons performs a surprisingly wide variety of complex unsupervised learning tasks, including chunking of temporal sequences and the source separation of mixed correlated signals. Common methods applicable to these temporal feature analyses were previously unknown. Our results suggest the powerful ability of neural networks with dendrites to analyze temporal features. This simple neuron model may also be potentially useful in neural engineering applications.

Extended Temporal Association Memory by Modulations of Inhibitory Circuits.

Hebbian learning of excitatory synapses plays a central role in storing activity patterns in associative memory models. Interstimulus Hebbian learning associates multiple items by converting temporal correlation to spatial correlation between attractors. Growing evidence suggests the importance of inhibitory plasticity in memory processing, but the consequence of such regulation in associative memory has not been understood. Noting that Hebbian learning of inhibitory synapses yields an anti-Hebbian effect, we show that the combination of Hebbian and anti-Hebbian learning can significantly increase the span of temporal association between correlated attractors as well as the sensitivity of these states to external input. Furthermore, these effects are regulated by changing the ratio of local and global recurrent inhibition after learning weights for excitation-inhibition balance. Our results suggest a nontrivial role of plasticity and modulation of inhibitory circuits in associative memory.

Unsupervised Detection of Cell-Assembly Sequences by Similarity-Based Clustering.

Neurons which fire in a fixed temporal pattern (i.e., "cell assemblies") are hypothesized to be a fundamental unit of neural information processing. Several methods are available for the detection of cell assemblies without a time structure. However, the systematic detection of cell assemblies with time structure has been challenging, especially in large datasets, due to the lack of efficient methods for handling the time structure. Here, we show a method to detect a variety of cell-assembly activity patterns, recurring in noisy neural population activities at multiple timescales. The key innovation is the use of a computer science method to comparing strings ("edit similarity"), to group spikes into assemblies. We validated the method using artificial data and experimental data, which were previously recorded from the hippocampus of male Long-Evans rats and the prefrontal cortex of male Brown Norway/Fisher hybrid rats. From the hippocampus, we could simultaneously extract place-cell sequences occurring on different timescales during navigation and awake replay. From the prefrontal cortex, we could discover multiple spike sequences of neurons encoding different segments of a goal-directed task. Unlike conventional event-driven statistical approaches, our method detects cell assemblies without creating event-locked averages. Thus, the method offers a novel analytical tool for deciphering the neural code during arbitrary behavioral and mental processes.

Orchestrated ensemble activities constitute a hippocampal memory engram.

The brain stores and recalls memories through a set of neurons, termed engram cells. However, it is unclear how these cells are organized to constitute a corresponding memory trace. We established a unique imaging system that combines Ca2+ imaging and engram identification to extract the characteristics of engram activity by visualizing and discriminating between engram and non-engram cells. Here, we show that engram cells detected in the hippocampus display higher repetitive activity than non-engram cells during novel context learning. The total activity pattern of the engram cells during learning is stable across post-learning memory processing. Within a single engram population, we detected several sub-ensembles composed of neurons collectively activated during learning. Some sub-ensembles preferentially reappear during post-learning sleep, and these replayed sub-ensembles are more likely to be reactivated during retrieval. These results indicate that sub-ensembles represent distinct pieces of information, which are then orchestrated to constitute an entire memory.

Discrete-Attractor-like Tracking in Continuous Attractor Neural Networks.

Continuous attractor neural networks generate a set of smoothly connected attractor states. In memory systems of the brain, these attractor states may represent continuous pieces of information such as spatial locations and head directions of animals. However, during the replay of previous experiences, hippocampal neurons show a discontinuous sequence in which discrete transitions of the neural state are phase locked with the slow-gamma (∼30-50 Hz) oscillation. Here, we explore the underlying mechanisms of the discontinuous sequence generation. We find that a continuous attractor neural network has several phases depending on the interactions between external input and local inhibitory feedback. The discrete-attractor-like behavior naturally emerges in one of these phases without any discreteness assumption. We propose that the dynamics of continuous attractor neural networks is the key to generate discontinuous state changes phase locked to the brain rhythm.