A feature-specific prediction error model explains dopaminergic heterogeneity.

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

Prioritizing replay when future goals are unknown.

Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, and it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Our work extends the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses.

Independent inhibitory control mechanisms for aggressive motivation and action.

Social behaviors often consist of a motivational phase followed by action. Here we show that neurons in the ventromedial hypothalamus ventrolateral area (VMHvl) of mice encode the temporal sequence of aggressive motivation to action. The VMHvl receives local inhibitory input (VMHvl shell) and long-range input from the medial preoptic area (MPO) with functional coupling to neurons with specific temporal profiles. Encoding models reveal that during aggression, VMHvl shellvgat+ activity peaks at the start of an attack, whereas activity from the MPO-VMHvlvgat+ input peaks at specific interaction endpoints. Activation of the MPO-VMHvlvgat+ input promotes and prolongs a low motivation state, whereas activation of VMHvl shellvgat+ results in action-related deficits, acutely terminating attack. Moreover, stimulation of MPO-VMHvlvgat+ input is positively valenced and anxiolytic. Together, these data demonstrate how distinct inhibitory inputs to the hypothalamus can independently gate the motivational and action phases of aggression through a single locus of control.

A neural mechanism for learning from delayed postingestive feedback.

Animals learn the value of foods based on their postingestive effects and thereby develop aversions to foods that are toxic1-6 and preferences to those that are nutritious7-14. However, it remains unclear how the brain is able to assign credit to flavors experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here, we reveal an unexpected role for postingestive reactivation of neural flavor representations in this temporal credit assignment process. To begin, we leverage the fact that mice learn to associate novel15-18, but not familiar, flavors with delayed gastric malaise signals to investigate how the brain represents flavors that support aversive postingestive learning. Surveying cellular resolution brainwide activation patterns reveals that a network of amygdala regions is unique in being preferentially activated by novel flavors across every stage of the learning process: the initial meal, delayed malaise, and memory retrieval. By combining high-density recordings in the amygdala with optogenetic stimulation of genetically defined hindbrain malaise cells, we find that postingestive malaise signals potently and specifically reactivate amygdalar novel flavor representations from a recent meal. The degree of malaise-driven reactivation of individual neurons predicts strengthening of flavor responses upon memory retrieval, leading to stabilization of the population-level representation of the recently consumed flavor. In contrast, meals without postingestive consequences degrade neural flavor representations as flavors become familiar and safe. Thus, our findings demonstrate that interoceptive reactivation of amygdalar flavor representations provides a neural mechanism to resolve the temporal credit assignment problem inherent to postingestive learning.

Overlapping representations of food and social stimuli in mouse VTA dopamine neurons.

Dopamine neurons of the ventral tegmental area (VTADA) respond to food and social stimuli and contribute to both forms of motivation. However, it is unclear whether the same or different VTADA neurons encode these different stimuli. To address this question, we performed two-photon calcium imaging in mice presented with food and conspecifics and found statistically significant overlap in the populations responsive to both stimuli. Both hunger and opposite-sex social experience further increased the proportion of neurons that respond to both stimuli, implying that increasing motivation for one stimulus increases overlap. In addition, single-nucleus RNA sequencing revealed significant co-expression of feeding- and social-hormone-related genes in individual VTADA neurons. Taken together, our functional and transcriptional data suggest overlapping VTADA populations underlie food and social motivation.

Overlapping representations of food and social stimuli in VTA dopamine neurons.

Dopamine neurons of the ventral tegmental area (VTA DA ) respond to food and social stimuli and contribute to both forms of motivation. However, it is unclear if the same or different VTA DA neurons encode these different stimuli. To address this question, we performed 2-photon calcium imaging in mice presented with food and conspecifics, and found statistically significant overlap in the populations responsive to both stimuli. Both hunger and opposite-sex social experience further increased the proportion of neurons that respond to both stimuli, implying that modifying motivation for one stimulus affects responses to both stimuli. In addition, single-nucleus RNA sequencing revealed significant co-expression of feeding- and social-hormone related genes in individual VTA DA neurons. Taken together, our functional and transcriptional data suggest overlapping VTA DA populations underlie food and social motivation.

A neural substrate of sex-dependent modulation of motivation.

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.

Neural circuit models for evidence accumulation through choice-selective sequences.

Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity. Encoded evidence is then faithfully transferred between neuronal populations representing different positions or times. Neural recordings from four different brain regions during a decision-making task showed that, during the evidence accumulation period, different brain regions displayed tuning curves consistent with different candidate models for evidence accumulation. This work provides mechanistic models and potential neural substrates for how graded-value information may be precisely accumulated within and transferred between neural populations, a set of computations fundamental to many cognitive operations.

Corrigendum: An accumulation-of-evidence task using visual pulses for mice navigating in virtual reality.

[This corrects the article DOI: 10.3389/fnbeh.2018.00036.].

Perineuronal Nets in the Dorsomedial Striatum Contribute to Behavioral Dysfunction in Mouse Models of Excessive Repetitive Behavior.

Background: Excessive repetitive behavior is a debilitating symptom of several neuropsychiatric disorders. Parvalbumin-positive inhibitory interneurons in the dorsal striatum have been linked to repetitive behavior, and a sizable portion of these cells are surrounded by perineuronal nets (PNNs), specialized extracellular matrix structures. Although PNNs have been associated with plasticity and neuropsychiatric disease, no previous studies have investigated their involvement in excessive repetitive behavior. Methods: We used histochemistry and confocal imaging to investigate PNNs surrounding parvalbumin-positive cells in the dorsal striatum of 4 mouse models of excessive repetitive behavior (BTBR, Cntnap2, Shank3, prenatal valproate treatment). We then investigated one of these models, the BTBR mouse, in detail, with DiI labeling, in vivo and in vitro recordings, and behavioral analyses. We next degraded PNNs in the dorsomedial striatum (DMS) using the enzyme chondroitinase ABC and assessed dendritic spine density, electrophysiology, and repetitive behavior. Results: We found a greater percentage of parvalbumin-positive interneurons with PNNs in the DMS of all 4 mouse models of excessive repetitive behavior compared with control mice. In BTBR mice, we found fewer dendritic spines on medium spiny neurons (targets of parvalbumin-positive interneurons) and differences in neuronal oscillations as well as inhibitory postsynaptic potentials compared with control mice. Reduction of DMS PNNs in BTBR mice altered dendritic spine density and inhibitory responses and normalized repetitive behavior. Conclusions: These findings suggest that cellular abnormalities in the DMS are associated with maladaptive repetitive behaviors and that manipulating PNNs can restore normal levels of repetitive behavior while altering DMS dendritic spines and inhibitory signaling.

Behavioural and dopaminergic signatures of resilience.

Chronic stress can have lasting adverse consequences in some individuals, yet others are resilient to the same stressor1,2. Susceptible and resilient individuals exhibit differences in the intrinsic properties of mesolimbic dopamine (DA) neurons after the stressful experience is over3-8. However, the causal links between DA, behaviour during stress and individual differences in resilience are unknown. Here we recorded behaviour in mice simultaneously with DA neuron activity in projections to the nucleus accumbens (NAc) (which signals reward9-12) and the tail striatum (TS) (which signals threat13-16) during social defeat. Supervised and unsupervised behavioural quantification revealed that during stress, resilient and susceptible mice use different behavioural strategies and have distinct activity patterns in DA terminals in the NAc (but not the TS). Neurally, resilient mice have greater activity near the aggressor, including at the onset of fighting back. Conversely, susceptible mice have greater activity at the offset of attacks and onset of fleeing. We also performed optogenetic stimulation of NAc-projecting DA neurons in open loop (randomly timed) during defeat or timed to specific behaviours using real-time behavioural classification. Both open-loop and fighting-back-timed activation promoted resilience and reorganized behaviour during defeat towards resilience-associated patterns. Together, these data provide a link between DA neural activity, resilience and resilience-associated behaviour during the experience of stress.

Cholinergic interneurons mediate cocaine extinction in male mice through plasticity across medium spiny neuron subtypes.

Cholinergic interneurons (ChINs) in the nucleus accumbens (NAc) have been implicated in the extinction of drug associations, as well as related plasticity in medium spiny neurons (MSNs). However, since most previous work relied on artificial manipulations, whether endogenous acetylcholine signaling relates to drug associations is unclear. Moreover, despite great interest in the opposing effects of dopamine on MSN subtypes, whether ChIN-mediated effects vary by MSN subtype is also unclear. Here, we find that high endogenous acetylcholine event frequency correlates with greater extinction of cocaine-context associations across male mice. Additionally, extinction is associated with a weakening of glutamatergic synapses across MSN subtypes. Manipulating ChIN activity bidirectionally controls both the rate of extinction and the associated plasticity at MSNs. Our findings indicate that NAc ChINs mediate drug-context extinction by reducing glutamatergic synaptic strength across MSN subtypes, and that natural variation in acetylcholine signaling may contribute to individual differences in extinction.

Choice-selective sequences dominate in cortical relative to thalamic inputs to NAc to support reinforcement learning.

How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens, which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex and midline regions of the thalamus. However, little is known about whether and how representations differ across these input pathways. By comparing these inputs during a reinforcement learning task in mice, we discovered that prelimbic cortical inputs preferentially represent actions and choices, whereas midline thalamic inputs preferentially represent cues. Choice-selective activity in the prelimbic cortical inputs is organized in sequences that persist beyond the outcome. Through computational modeling, we demonstrate that these sequences can support the neural implementation of reinforcement-learning algorithms, in both a circuit model based on synaptic plasticity and one based on neural dynamics. Finally, we test and confirm a prediction of our circuit models by direct manipulation of nucleus accumbens input neurons.

Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state.

A classic view of the striatum holds that activity in direct and indirect pathways oppositely modulates motor output. Whether this involves direct control of movement, or reflects a cognitive process underlying movement, remains unresolved. Here we find that strong, opponent control of behavior by the two pathways of the dorsomedial striatum depends on the cognitive requirements of a task. Furthermore, a latent state model (a hidden Markov model with generalized linear model observations) reveals that-even within a single task-the contribution of the two pathways to behavior is state dependent. Specifically, the two pathways have large contributions in one of two states associated with a strategy of evidence accumulation, compared to a state associated with a strategy of repeating previous choices. Thus, both the demands imposed by a task, as well as the internal state of mice when performing a task, determine whether dorsomedial striatum pathways provide strong and opponent control of behavior.

Inferring spikes from calcium imaging in dopamine neurons.

Calcium imaging has led to discoveries about neural correlates of behavior in subcortical neurons, including dopamine (DA) neurons. However, spike inference methods have not been tested in most populations of subcortical neurons. To address this gap, we simultaneously performed calcium imaging and electrophysiology in DA neurons in brain slices and applied a recently developed spike inference algorithm to the GCaMP fluorescence. This revealed that individual spikes can be inferred accurately in this population. Next, we inferred spikes in vivo from calcium imaging from these neurons during Pavlovian conditioning, as well as during navigation in virtual reality. In both cases, we quantitatively recapitulated previous in vivo electrophysiological observations. Our work provides a validated approach to infer spikes from calcium imaging in DA neurons and implies that aspects of both tonic and phasic spike patterns can be recovered.

Standardized and reproducible measurement of decision-making in mice.

Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.

In science, it is of vital importance that multiple studies corroborate the same result. Researchers therefore need to know all the details of previous experiments in order to implement the procedures as exactly as possible. However, this is becoming a major problem in neuroscience, as animal studies of behavior have proven to be hard to reproduce, and most experiments are never replicated by other laboratories. Mice are increasingly being used to study the neural mechanisms of decision making, taking advantage of the genetic, imaging and physiological tools that are available for mouse brains. Yet, the lack of standardized behavioral assays is leading to inconsistent results between laboratories. This makes it challenging to carry out large-scale collaborations which have led to massive breakthroughs in other fields such as physics and genetics. To help make these studies more reproducible, the International Brain Laboratory (a collaborative research group) et al. developed a standardized approach for investigating decision making in mice that incorporates every step of the process; from the training protocol to the software used to analyze the data. In the experiment, mice were shown images with different contrast and had to indicate, using a steering wheel, whether it appeared on their right or left. The mice then received a drop of sugar water for every correction decision. When the image contrast was high, mice could rely on their vision. However, when the image contrast was very low or zero, they needed to consider the information of previous trials and choose the side that had recently appeared more frequently. This method was used to train 140 mice in seven laboratories from three different countries. The results showed that learning speed was different across mice and laboratories, but once training was complete the mice behaved consistently, relying on visual stimuli or experiences to guide their choices in a similar way. These results show that complex behaviors in mice can be reproduced across multiple laboratories, providing an unprecedented dataset and open-access tools for studying decision making. This work could serve as a foundation for other groups, paving the way to a more collaborative approach in the field of neuroscience that could help to tackle complex research challenges.

A Comparison of Dopaminergic and Cholinergic Populations Reveals Unique Contributions of VTA Dopamine Neurons to Short-Term Memory.

We systematically compare the contributions of two dopaminergic and two cholinergic ascending populations to a spatial short-term memory task in rats. In ventral tegmental area dopamine (VTA-DA) and nucleus basalis cholinergic (NB-ChAT) populations, trial-by-trial fluctuations in activity during the delay period relate to performance with an inverted-U, despite the fact that both populations have low activity during that time. Transient manipulations reveal that only VTA-DA neurons, and not the other three populations we examine, contribute causally and selectively to short-term memory. This contribution is most significant during the delay period, when both increases and decreases in VTA-DA activity impair short-term memory. Our results reveal a surprising dissociation between when VTA-DA neurons are most active and when they have the biggest causal contribution to short-term memory, and they also provide support for classic ideas about an inverted-U relationship between neuromodulation and cognition.

Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times.

Dopamine (DA) neurons are to encode reward prediction error (RPE), in addition to other signals, such as salience. While RPE is known to support learning, the role of salience in learning remains less clear. To address this, we recorded and manipulated VTA DA neurons in mice during fear extinction. We applied deep learning to classify mouse freezing behavior, eliminating the need for human scoring. Our fiber photometry recordings showed DA neurons in medial and lateral VTA have distinct activity profiles during fear extinction: medial VTA activity more closely reflected RPE, while lateral VTA activity more closely reflected a salience-like signal. Optogenetic inhibition of DA neurons in either region slowed fear extinction, with the relevant time period for inhibition differing across regions. Our results indicate salience-like signals can have similar downstream consequences to RPE-like signals, although with different temporal dependencies.

Striatal circuits for reward learning and decision-making.

The striatum is essential for learning which actions lead to reward and for implementing those actions. Decades of experimental and theoretical work have led to several influential theories and hypotheses about how the striatal circuit mediates these functions. However, owing to technical limitations, testing these hypotheses rigorously has been difficult. In this Review, we briefly describe some of the classic ideas of striatal function. We then review recent studies in rodents that take advantage of optical and genetic methods to test these classic ideas by recording and manipulating identified cell types within the circuit. This new body of work has provided experimental support of some longstanding ideas about the striatal circuit and has uncovered critical aspects of the classic view that are incorrect or incomplete.