All-optical physiology resolves a synaptic basis for behavioral timescale plasticity.

Learning has been associated with modifications of synaptic and circuit properties, but the precise changes storing information in mammals have remained largely unclear. We combined genetically targeted voltage imaging with targeted optogenetic activation and silencing of pre- and post-synaptic neurons to study the mechanisms underlying hippocampal behavioral timescale plasticity. In mice navigating a virtual-reality environment, targeted optogenetic activation of individual CA1 cells at specific places induced stable representations of these places in the targeted cells. Optical elicitation, recording, and modulation of synaptic transmission in behaving mice revealed that activity in presynaptic CA2/3 cells was required for the induction of plasticity in CA1 and, furthermore, that during induction of these place fields in single CA1 cells, synaptic input from CA2/3 onto these same cells was potentiated. These results reveal synaptic implementation of hippocampal behavioral timescale plasticity and define a methodology to resolve synaptic plasticity during learning and memory in behaving mammals.

A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement.

The striosome compartment within the dorsal striatum has been implicated in reinforcement learning and regulation of motivation, but how striosomal neurons contribute to these functions remains elusive. Here, we show that a genetically identified striosomal population, which expresses the Teashirt family zinc finger 1 (Tshz1) and belongs to the direct pathway, drives negative reinforcement and is essential for aversive learning in mice. Contrasting a "conventional" striosomal direct pathway, the Tshz1 neurons cause aversion, movement suppression, and negative reinforcement once activated, and they receive a distinct set of synaptic inputs. These neurons are predominantly excited by punishment rather than reward and represent the anticipation of punishment or the motivation for avoidance. Furthermore, inhibiting these neurons impairs punishment-based learning without affecting reward learning or movement. These results establish a major role of striosomal neurons in behaviors reinforced by punishment and moreover uncover functions of the direct pathway unaccounted for in classic models.

Primate Amygdala Neurons Simulate Decision Processes of Social Partners.

By observing their social partners, primates learn about reward values of objects. Here, we show that monkeys' amygdala neurons derive object values from observation and use these values to simulate a partner monkey's decision process. While monkeys alternated making reward-based choices, amygdala neurons encoded object-specific values learned from observation. Dynamic activities converted these values to representations of the recorded monkey's own choices. Surprisingly, the same activity patterns unfolded spontaneously before partner's choices in separate neurons, as if these neurons simulated the partner's decision-making. These "simulation neurons" encoded signatures of mutual-inhibitory decision computation, including value comparisons and value-to-choice conversions, resulting in accurate predictions of partner's choices. Population decoding identified differential contributions of amygdala subnuclei. Biophysical modeling of amygdala circuits showed that simulation neurons emerge naturally from convergence between object-value neurons and self-other neurons. By simulating decision computations during observation, these neurons could allow primates to reconstruct their social partners' mental states.

Area-Specificity and Plasticity of History-Dependent Value Coding During Learning.

Decision making is often driven by the subjective value of available options, a value which is formed through experience. To support this fundamental behavior, the brain must encode and maintain the subjective value. To investigate the area specificity and plasticity of value coding, we trained mice in a value-based decision task and imaged neural activity in 6 cortical areas with cellular resolution. History- and value-related signals were widespread across areas, but their strength and temporal patterns differed. In expert mice, the retrosplenial cortex (RSC) uniquely encoded history- and value-related signals with persistent population activity patterns across trials. This unique encoding of RSC emerged during task learning with a strong increase in more distant history signals. Acute inactivation of RSC selectively impaired the reward-history-based behavioral strategy. Our results indicate that RSC flexibly changes its history coding and persistently encodes value-related signals to support adaptive behaviors.

Corticoamygdala Transfer of Socially Derived Information Gates Observational Learning.

Observational learning is a powerful survival tool allowing individuals to learn about threat-predictive stimuli without directly experiencing the pairing of the predictive cue and punishment. This ability has been linked to the anterior cingulate cortex (ACC) and the basolateral amygdala (BLA). To investigate how information is encoded and transmitted through this circuit, we performed electrophysiological recordings in mice observing a demonstrator mouse undergo associative fear conditioning and found that BLA-projecting ACC (ACC→BLA) neurons preferentially encode socially derived aversive cue information. Inhibition of ACC→BLA alters real-time amygdala representation of the aversive cue during observational conditioning. Selective inhibition of the ACC→BLA projection impaired acquisition, but not expression, of observational fear conditioning. We show that information derived from observation about the aversive value of the cue is transmitted from the ACC to the BLA and that this routing of information is critically instructive for observational fear conditioning. VIDEO ABSTRACT.

Fast-Spiking Interneurons Supply Feedforward Control of Bursting, Calcium, and Plasticity for Efficient Learning.

Fast-spiking interneurons (FSIs) are a prominent class of forebrain GABAergic cells implicated in two seemingly independent network functions: gain control and network plasticity. Little is known, however, about how these roles interact. Here, we use a combination of cell-type-specific ablation, optogenetics, electrophysiology, imaging, and behavior to describe a unified mechanism by which striatal FSIs control burst firing, calcium influx, and synaptic plasticity in neighboring medium spiny projection neurons (MSNs). In vivo silencing of FSIs increased bursting, calcium transients, and AMPA/NMDA ratios in MSNs. In a motor sequence task, FSI silencing increased the frequency of calcium transients but reduced the specificity with which transients aligned to individual task events. Consistent with this, ablation of FSIs disrupted the acquisition of striatum-dependent egocentric learning strategies. Together, our data support a model in which feedforward inhibition from FSIs temporally restricts MSN bursting and calcium-dependent synaptic plasticity to facilitate striatum-dependent sequence learning.

Light Affects Mood and Learning through Distinct Retina-Brain Pathways.

Light exerts a range of powerful biological effects beyond image vision, including mood and learning regulation. While the source of photic information affecting mood and cognitive functions is well established, viz. intrinsically photosensitive retinal ganglion cells (ipRGCs), the central mediators are unknown. Here, we reveal that the direct effects of light on learning and mood utilize distinct ipRGC output streams. ipRGCs that project to the suprachiasmatic nucleus (SCN) mediate the effects of light on learning, independently of the SCN's pacemaker function. Mood regulation by light, on the other hand, requires an SCN-independent pathway linking ipRGCs to a previously unrecognized thalamic region, termed perihabenular nucleus (PHb). The PHb is integrated in a distinctive circuitry with mood-regulating centers and is both necessary and sufficient for driving the effects of light on affective behavior. Together, these results provide new insights into the neural basis required for light to influence mood and learning.

Cerebellar Functions Beyond Movement and Learning.

The cerebellum has a well-established role in controlling motor functions, including coordination, posture, and the learning of skilled movements. The mechanisms for how it carries out motor behavior remain under intense investigation. Interestingly though, in recent years the mechanisms of cerebellar function have faced additional scrutiny since nonmotor behaviors may also be controlled by the cerebellum. With such complexity arising, there is now a pressing need to better understand how cerebellar structure, function, and behavior intersect to influence behaviors that are dynamically called upon as an animal experiences its environment. Here, we discuss recent experimental work that frames possible neural mechanisms for how the cerebellum shapes disparate behaviors and why its dysfunction is catastrophic in hereditary and acquired conditions-both motor and nonmotor. For these reasons, the cerebellum might be the ideal therapeutic target.

Learning, Fast and Slow: Single- and Many-Shot Learning in the Hippocampus.

The hippocampus is critical for memory and spatial navigation. The ability to map novel environments, as well as more abstract conceptual relationships, is fundamental to the cognitive flexibility that humans and other animals require to survive in a dynamic world. In this review, we survey recent advances in our understanding of how this flexibility is implemented anatomically and functionally by hippocampal circuitry, during both active exploration (online) and rest (offline). We discuss the advantages and limitations of spike timing-dependent plasticity and the more recently discovered behavioral timescale synaptic plasticity in supporting distinct learning modes in the hippocampus. Finally, we suggest complementary roles for these plasticity types in explaining many-shot and single-shot learning in the hippocampus and discuss how these rules could work together to support the learning of cognitive maps.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making.

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics.

The Cerebellar Cortex.

The cerebellar cortex is an important system for relating neural circuits and learning. Its promise reflects the longstanding idea that it contains simple, repeated circuit modules with only a few cell types and a single plasticity mechanism that mediates learning according to classical Marr-Albus models. However, emerging data have revealed surprising diversity in neuron types, synaptic connections, and plasticity mechanisms, both locally and regionally within the cerebellar cortex. In light of these findings, it is not surprising that attempts to generate a holistic model of cerebellar learning across different behaviors have not been successful. While the cerebellum remains an ideal system for linking neuronal function with behavior, it is necessary to update the cerebellar circuit framework to achieve its great promise. In this review, we highlight recent advances in our understanding of cerebellar-cortical cell types, synaptic connections, signaling mechanisms, and forms of plasticity that enrich cerebellar processing.

The hidden fitness of the male zebra finch courtship song.

Vocal learning in songbirds is thought to have evolved through sexual selection, with female preference driving males to develop large and varied song repertoires1-3. However, many songbird species learn only a single song in their lifetime4. How sexual selection drives the evolution of single-song repertoires is not known. Here, by applying dimensionality-reduction techniques to the singing behaviour of zebra finches (Taeniopygia guttata), we show that syllable spread in low-dimensional feature space explains how single songs function as honest indicators of fitness. We find that this Gestalt measure of behaviour captures the spectrotemporal distinctiveness of song syllables in zebra finches; that females strongly prefer songs that occupy more latent space; and that matching path lengths in low-dimensional space is difficult for young males. Our findings clarify how simple vocal repertoires may have evolved in songbirds and indicate divergent strategies for how sexual selection can shape vocal learning.

Dynamic behaviour restructuring mediates dopamine-dependent credit assignment.

Animals exhibit a diverse behavioural repertoire when exploring new environments and can learn which actions or action sequences produce positive outcomes. Dopamine release after encountering a reward is critical for reinforcing reward-producing actions1-3. However, it has been challenging to understand how credit is assigned to the exact action that produced the dopamine release during continuous behaviour. Here we investigated this problem in mice using a self-stimulation paradigm in which specific spontaneous movements triggered optogenetic stimulation of dopaminergic neurons. Dopamine self-stimulation rapidly and dynamically changes the structure of the entire behavioural repertoire. Initial stimulations reinforced not only the stimulation-producing target action, but also actions similar to the target action and actions that occurred a few seconds before stimulation. Repeated pairings led to a gradual refinement of the behavioural repertoire to home in on the target action. Reinforcement of action sequences revealed further temporal dependencies of refinement. Action pairs spontaneously separated by long time intervals promoted a stepwise credit assignment, with early refinement of actions most proximal to stimulation and subsequent refinement of more distal actions. Thus, a retrospective reinforcement mechanism promotes not only reinforcement, but also gradual refinement of the entire behavioural repertoire to assign credit to specific actions and action sequences that lead to dopamine release.

Mental navigation in the primate entorhinal cortex.

A cognitive map is a suitably structured representation that enables novel computations using previous experience; for example, planning a new route in a familiar space1. Work in mammals has found direct evidence for such representations in the presence of exogenous sensory inputs in both spatial2,3 and non-spatial domains4-10. Here we tested a foundational postulate of the original cognitive map theory1,11: that cognitive maps support endogenous computations without external input. We recorded from the entorhinal cortex of monkeys in a mental navigation task that required the monkeys to use a joystick to produce one-dimensional vectors between pairs of visual landmarks without seeing the intermediate landmarks. The ability of the monkeys to perform the task and generalize to new pairs indicated that they relied on a structured representation of the landmarks. Task-modulated neurons exhibited periodicity and ramping that matched the temporal structure of the landmarks and showed signatures of continuous attractor networks12,13. A continuous attractor network model of path integration14 augmented with a Hebbian-like learning mechanism provided an explanation of how the system could endogenously recall landmarks. The model also made an unexpected prediction that endogenous landmarks transiently slow path integration, reset the dynamics and thereby reduce variability. This prediction was borne out in a reanalysis of firing rate variability and behaviour. Our findings link the structured patterns of activity in the entorhinal cortex to the endogenous recruitment of a cognitive map during mental navigation.

Synaptic wiring motifs in posterior parietal cortex support decision-making.

The posterior parietal cortex exhibits choice-selective activity during perceptual decision-making tasks1-10. However, it is not known how this selective activity arises from the underlying synaptic connectivity. Here we combined virtual-reality behaviour, two-photon calcium imaging, high-throughput electron microscopy and circuit modelling to analyse how synaptic connectivity between neurons in the posterior parietal cortex relates to their selective activity. We found that excitatory pyramidal neurons preferentially target inhibitory interneurons with the same selectivity. In turn, inhibitory interneurons preferentially target pyramidal neurons with opposite selectivity, forming an opponent inhibition motif. This motif was present even between neurons with activity peaks in different task epochs. We developed neural-circuit models of the computations performed by these motifs, and found that opponent inhibition between neural populations with opposite selectivity amplifies selective inputs, thereby improving the encoding of trial-type information. The models also predict that opponent inhibition between neurons with activity peaks in different task epochs contributes to creating choice-specific sequential activity. These results provide evidence for how synaptic connectivity in cortical circuits supports a learned decision-making task.

How development sculpts hippocampal circuits and function.

In mammals, the selective transformation of transient experience into stored memory occurs in the hippocampus, which develops representations of specific events in the context in which they occur. In this review, we focus on the development of hippocampal circuits and the self-organized dynamics embedded within them since the latter critically support the role of the hippocampus in learning and memory. We first discuss evidence that adult hippocampal cells and circuits are sculpted by development as early as during embryonic neurogenesis. We argue that these primary developmental programs provide a scaffold onto which later experience of the external world can be grafted. Next, we review the different sequences in the development of hippocampal cells and circuits at anatomical and functional levels. We cover a period extending from neurogenesis and migration to the appearance of phenotypic diversity within hippocampal cells and their wiring into functional networks. We describe the progressive emergence of network dynamics in the hippocampus, from sensorimotor-driven early sharp waves to sequences of place cells tracking relational information. We outline the critical turn points and discontinuities in that developmental journey, and close by formulating open questions. We propose that rewinding the process of hippocampal development helps understand the main organization principles of memory circuits.

Microcircuits for spatial coding in the medial entorhinal cortex.

The hippocampal formation is critically involved in learning and memory and contains a large proportion of neurons encoding aspects of the organism's spatial surroundings. In the medial entorhinal cortex (MEC), this includes grid cells with their distinctive hexagonal firing fields as well as a host of other functionally defined cell types including head direction cells, speed cells, border cells, and object-vector cells. Such spatial coding emerges from the processing of external inputs by local microcircuits. However, it remains unclear exactly how local microcircuits and their dynamics within the MEC contribute to spatial discharge patterns. In this review we focus on recent investigations of intrinsic MEC connectivity, which have started to describe and quantify both excitatory and inhibitory wiring in the superficial layers of the MEC. Although the picture is far from complete, it appears that these layers contain robust recurrent connectivity that could sustain the attractor dynamics posited to underlie grid pattern formation. These findings pave the way to a deeper understanding of the mechanisms underlying spatial navigation and memory.

The generative grammar of the brain: a critique of internally generated representations.

The past decade of progress in neurobiology has uncovered important organizational principles for network preconfiguration and neuronal selection that suggest a generative grammar exists in the brain. In this Perspective, I discuss the competence of the hippocampal neural network to generically express temporally compressed sequences of neuronal firing that represent novel experiences, which is envisioned as a form of generative neural syntax supporting a neurobiological perspective on brain function. I compare this neural competence with the hippocampal network performance that represents specific experiences with higher fidelity after new learning during replay, which is envisioned as a form of neural semantic that supports a complementary neuropsychological perspective. I also demonstrate how the syntax of network competence emerges a priori during early postnatal life and is followed by the later development of network performance that enables rapid encoding and memory consolidation. Thus, I propose that this generative grammar of the brain is essential for internally generated representations, which are crucial for the cognitive processes underlying learning and memory, prospection, and inference, which ultimately underlie our reason and representation of the world.

Bistable parvalbumin circuits pivotal for brain plasticity.

Experience shapes brain function throughout life to varying degrees. In a recent issue of Nature, Donato et al. identify reversible shifts in focal parvalbumin cell state during adult learning, placing it on a mechanistic continuum with developmental critical periods. A disinhibitory microcircuit controls the plasticity switch to modulate memory formation.

Dopaminergic systems create reward seeking despite adverse consequences.

Resource-seeking behaviours are ordinarily constrained by physiological needs and threats of danger, and the loss of these controls is associated with pathological reward seeking1. Although dysfunction of the dopaminergic valuation system of the brain is known to contribute towards unconstrained reward seeking2,3, the underlying reasons for this behaviour are unclear. Here we describe dopaminergic neural mechanisms that produce reward seeking despite adverse consequences in Drosophila melanogaster. Odours paired with optogenetic activation of a defined subset of reward-encoding dopaminergic neurons become cues that starved flies seek while neglecting food and enduring electric shock punishment. Unconstrained seeking of reward is not observed after learning with sugar or synthetic engagement of other dopaminergic neuron populations. Antagonism between reward-encoding and punishment-encoding dopaminergic neurons accounts for the perseverance of reward seeking despite punishment, whereas synthetic engagement of the reward-encoding dopaminergic neurons also impairs the ordinary need-dependent dopaminergic valuation of available food. Connectome analyses reveal that the population of reward-encoding dopaminergic neurons receives highly heterogeneous input, consistent with parallel representation of diverse rewards, and recordings demonstrate state-specific gating and satiety-related signals. We propose that a similar dopaminergic valuation system dysfunction is likely to contribute to maladaptive seeking of rewards by mammals.