Learning attentional templates for value-based decision-making.

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed across tasks.

A neural circuit for male sexual behavior and reward.

Male sexual behavior is innate and rewarding. Despite its centrality to reproduction, a molecularly specified neural circuit governing innate male sexual behavior and reward remains to be characterized. We have discovered a developmentally wired neural circuit necessary and sufficient for male mating. This circuit connects chemosensory input to BNSTprTac1 neurons, which innervate POATacr1 neurons that project to centers regulating motor output and reward. Epistasis studies demonstrate that BNSTprTac1 neurons are upstream of POATacr1 neurons, and BNSTprTac1-released substance P following mate recognition potentiates activation of POATacr1 neurons through Tacr1 to initiate mating. Experimental activation of POATacr1 neurons triggers mating, even in sexually satiated males, and it is rewarding, eliciting dopamine release and self-stimulation of these cells. Together, we have uncovered a neural circuit that governs the key aspects of innate male sexual behavior: motor displays, drive, and reward.

Neural mechanism underlying depressive-like state associated with social status loss.

Downward social mobility is a well-known mental risk factor for depression, but its neural mechanism remains elusive. Here, by forcing mice to lose against their subordinates in a non-violent social contest, we lower their social ranks stably and induce depressive-like behaviors. These rank-decline-associated depressive-like behaviors can be reversed by regaining social status. In vivo fiber photometry and single-unit electrophysiological recording show that forced loss, but not natural loss, generates negative reward prediction error (RPE). Through the lateral hypothalamus, the RPE strongly activates the brain's anti-reward center, the lateral habenula (LHb). LHb activation inhibits the medial prefrontal cortex (mPFC) that controls social competitiveness and reinforces retreats in contests. These results reveal the core neural mechanisms mutually promoting social status loss and depressive behaviors. The intertwined neuronal signaling controlling mPFC and LHb activities provides a mechanistic foundation for the crosstalk between social mobility and psychological disorder, unveiling a promising target for intervention.

Touch neurons underlying dopaminergic pleasurable touch and sexual receptivity.

Pleasurable touch is paramount during social behavior, including sexual encounters. However, the identity and precise role of sensory neurons that transduce sexual touch remain unknown. A population of sensory neurons labeled by developmental expression of the G protein-coupled receptor Mrgprb4 detects mechanical stimulation in mice. Here, we study the social relevance of Mrgprb4-lineage neurons and reveal that these neurons are required for sexual receptivity and sufficient to induce dopamine release in the brain. Even in social isolation, optogenetic stimulation of Mrgprb4-lineage neurons through the back skin is sufficient to induce a conditioned place preference and a striking dorsiflexion resembling the lordotic copulatory posture. In the absence of Mrgprb4-lineage neurons, female mice no longer find male mounts rewarding: sexual receptivity is supplanted by aggression and a coincident decline in dopamine release in the nucleus accumbens. Together, these findings establish that Mrgprb4-lineage neurons initiate a skin-to-brain circuit encoding the rewarding quality of social touch.

Cell-type-specific population dynamics of diverse reward computations.

Computational analysis of cellular activity has developed largely independently of modern transcriptomic cell typology, but integrating these approaches may be essential for full insight into cellular-level mechanisms underlying brain function and dysfunction. Applying this approach to the habenula (a structure with diverse, intermingled molecular, anatomical, and computational features), we identified encoding of reward-predictive cues and reward outcomes in distinct genetically defined neural populations, including TH+ cells and Tac1+ cells. Data from genetically targeted recordings were used to train an optimized nonlinear dynamical systems model and revealed activity dynamics consistent with a line attractor. High-density, cell-type-specific electrophysiological recordings and optogenetic perturbation provided supporting evidence for this model. Reverse-engineering predicted how Tac1+ cells might integrate reward history, which was complemented by in vivo experimentation. This integrated approach describes a process by which data-driven computational models of population activity can generate and frame actionable hypotheses for cell-type-specific investigation in biological systems.

Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment.

Significant evidence supports the view that dopamine shapes learning by encoding reward prediction errors. However, it is unknown whether striatal targets receive tailored dopamine dynamics based on regional functional specialization. Here, we report wave-like spatiotemporal activity patterns in dopamine axons and release across the dorsal striatum. These waves switch between activational motifs and organize dopamine transients into localized clusters within functionally related striatal subregions. Notably, wave trajectories were tailored to task demands, propagating from dorsomedial to dorsolateral striatum when rewards are contingent on animal behavior and in the opponent direction when rewards are independent of behavioral responses. We propose a computational architecture in which striatal dopamine waves are sculpted by inference about agency and provide a mechanism to direct credit assignment to specialized striatal subregions. Supporting model predictions, dorsomedial dopamine activity during reward-pursuit signaled the extent of instrumental control and interacted with reward waves to predict future behavioral adjustments.

Differential encoding in prefrontal cortex projection neuron classes across cognitive tasks.

Single-cell transcriptomics has been widely applied to classify neurons in the mammalian brain, while systems neuroscience has historically analyzed the encoding properties of cortical neurons without considering cell types. Here we examine how specific transcriptomic types of mouse prefrontal cortex (PFC) projection neurons relate to axonal projections and encoding properties across multiple cognitive tasks. We found that most types projected to multiple targets, and most targets received projections from multiple types, except PFC→PAG (periaqueductal gray). By comparing Ca2+ activity of the molecularly homogeneous PFC→PAG type against two heterogeneous classes in several two-alternative choice tasks in freely moving mice, we found that all task-related signals assayed were qualitatively present in all examined classes. However, PAG-projecting neurons most potently encoded choice in cued tasks, whereas contralateral PFC-projecting neurons most potently encoded reward context in an uncued task. Thus, task signals are organized redundantly, but with clear quantitative biases across cells of specific molecular-anatomical characteristics.

The Statistical Structure of the Hippocampal Code for Space as a Function of Time, Context, and Value.

Hippocampal activity represents many behaviorally important variables, including context, an animal's location within a given environmental context, time, and reward. Using longitudinal calcium imaging in mice, multiple large virtual environments, and differing reward contingencies, we derived a unified probabilistic model of CA1 representations centered on a single feature-the field propensity. Each cell's propensity governs how many place fields it has per unit space, predicts its reward-related activity, and is preserved across distinct environments and over months. Propensity is broadly distributed-with many low, and some very high, propensity cells-and thus strongly shapes hippocampal representations. This results in a range of spatial codes, from sparse to dense. Propensity varied ∼10-fold between adjacent cells in salt-and-pepper fashion, indicating substantial functional differences within a presumed cell type. Intracellular recordings linked propensity to cell excitability. The stability of each cell's propensity across conditions suggests this fundamental property has anatomical, transcriptional, and/or developmental origins.

Targeted Activation of Hippocampal Place Cells Drives Memory-Guided Spatial Behavior.

The hippocampus is crucial for spatial navigation and episodic memory formation. Hippocampal place cells exhibit spatially selective activity within an environment and have been proposed to form the neural basis of a cognitive map of space that supports these mnemonic functions. However, the direct influence of place cell activity on spatial navigation behavior has not yet been demonstrated. Using an 'all-optical' combination of simultaneous two-photon calcium imaging and two-photon optogenetics, we identified and selectively activated place cells that encoded behaviorally relevant locations in a virtual reality environment. Targeted stimulation of a small number of place cells was sufficient to bias the behavior of animals during a spatial memory task, providing causal evidence that hippocampal place cells actively support spatial navigation and memory.

A Unified Framework for Dopamine Signals across Timescales.

Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.

Striosomes Mediate Value-Based Learning Vulnerable in Age and a Huntington's Disease Model.

Learning valence-based responses to favorable and unfavorable options requires judgments of the relative value of the options, a process necessary for species survival. We found, using engineered mice, that circuit connectivity and function of the striosome compartment of the striatum are critical for this type of learning. Calcium imaging during valence-based learning exhibited a selective correlation between learning and striosomal but not matrix signals. This striosomal activity encoded discrimination learning and was correlated with task engagement, which, in turn, could be regulated by chemogenetic excitation and inhibition. Striosomal function during discrimination learning was disturbed with aging and severely so in a mouse model of Huntington's disease. Anatomical and functional connectivity of parvalbumin-positive, putative fast-spiking interneurons (FSIs) to striatal projection neurons was enhanced in striosomes compared with matrix in mice that learned. Computational modeling of these findings suggests that FSIs can modulate the striosomal signal-to-noise ratio, crucial for discrimination and learning.

Hindbrain Double-Negative Feedback Mediates Palatability-Guided Food and Water Consumption.

Hunger and thirst have distinct goals but control similar ingestive behaviors, and little is known about neural processes that are shared between these behavioral states. We identify glutamatergic neurons in the peri-locus coeruleus (periLCVGLUT2 neurons) as a polysynaptic convergence node from separate energy-sensitive and hydration-sensitive cell populations. We develop methods for stable hindbrain calcium imaging in free-moving mice, which show that periLCVGLUT2 neurons are tuned to ingestive behaviors and respond similarly to food or water consumption. PeriLCVGLUT2 neurons are scalably inhibited by palatability and homeostatic need during consumption. Inhibition of periLCVGLUT2 neurons is rewarding and increases consumption by enhancing palatability and prolonging ingestion duration. These properties comprise a double-negative feedback relationship that sustains food or water consumption without affecting food- or water-seeking. PeriLCVGLUT2 neurons are a hub between hunger and thirst that specifically controls motivation for food and water ingestion, which is a factor that contributes to hedonic overeating and obesity.

Human Replay Spontaneously Reorganizes Experience.

Knowledge abstracted from previous experiences can be transferred to aid new learning. Here, we asked whether such abstract knowledge immediately guides the replay of new experiences. We first trained participants on a rule defining an ordering of objects and then presented a novel set of objects in a scrambled order. Across two studies, we observed that representations of these novel objects were reactivated during a subsequent rest. As in rodents, human "replay" events occurred in sequences accelerated in time, compared to actual experience, and reversed their direction after a reward. Notably, replay did not simply recapitulate visual experience, but followed instead a sequence implied by learned abstract knowledge. Furthermore, each replay contained more than sensory representations of the relevant objects. A sensory code of object representations was preceded 50 ms by a code factorized into sequence position and sequence identity. We argue that this factorized representation facilitates the generalization of a previously learned structure to new objects.

Primate Amygdala Neurons Simulate Decision Processes of Social Partners.

By observing their social partners, primates learn about reward values of objects. Here, we show that monkeys' amygdala neurons derive object values from observation and use these values to simulate a partner monkey's decision process. While monkeys alternated making reward-based choices, amygdala neurons encoded object-specific values learned from observation. Dynamic activities converted these values to representations of the recorded monkey's own choices. Surprisingly, the same activity patterns unfolded spontaneously before partner's choices in separate neurons, as if these neurons simulated the partner's decision-making. These "simulation neurons" encoded signatures of mutual-inhibitory decision computation, including value comparisons and value-to-choice conversions, resulting in accurate predictions of partner's choices. Population decoding identified differential contributions of amygdala subnuclei. Biophysical modeling of amygdala circuits showed that simulation neurons emerge naturally from convergence between object-value neurons and self-other neurons. By simulating decision computations during observation, these neurons could allow primates to reconstruct their social partners' mental states.

A Hippocampus-Accumbens Tripartite Neuronal Motif Guides Appetitive Memory in Space.

Retrieving and acting on memories of food-predicting environments are fundamental processes for animal survival. Hippocampal pyramidal cells (PYRs) of the mammalian brain provide mnemonic representations of space. Yet the substrates by which these hippocampal representations support memory-guided behavior remain unknown. Here, we uncover a direct connection from dorsal CA1 (dCA1) hippocampus to nucleus accumbens (NAc) that enables the behavioral manifestation of place-reward memories. By monitoring neuronal ensembles in mouse dCA1→NAc pathway, combined with cell-type selective optogenetic manipulations of input-defined postsynaptic neurons, we show that dCA1 PYRs drive NAc medium spiny neurons and orchestrate their spiking activity using feedforward inhibition mediated by dCA1-connected parvalbumin-expressing fast-spiking interneurons. This tripartite cross-circuit motif supports spatial appetitive memory and associated NAc assemblies, being independent of dorsal subiculum and dispensable for both spatial novelty detection and reward seeking. Our findings demonstrate that the dCA1→NAc pathway instantiates a limbic-motor interface for neuronal representations of space to promote effective appetitive behavior.

Distinct Dopamine Receptor Pathways Underlie the Temporal Sensitivity of Associative Learning.

Animals rely on the relative timing of events in their environment to form and update predictive associations, but the molecular and circuit mechanisms for this temporal sensitivity remain incompletely understood. Here, we show that olfactory associations in Drosophila can be written and reversed on a trial-by-trial basis depending on the temporal relationship between an odor cue and dopaminergic reinforcement. Through the synchronous recording of neural activity and behavior, we show that reversals in learned odor attraction correlate with bidirectional neural plasticity in the mushroom body, the associative olfactory center of the fly. Two dopamine receptors, DopR1 and DopR2, contribute to this temporal sensitivity by coupling to distinct second messengers and directing either synaptic depression or potentiation. Our results reveal how dopamine-receptor signaling pathways can detect the order of events to instruct opposing forms of synaptic and behavioral plasticity, allowing animals to flexibly update their associations in a dynamic environment.

A Paranigral VTA Nociceptin Circuit that Constrains Motivation for Reward.

Nociceptin and its receptor are widely distributed throughout the brain in regions associated with reward behavior, yet how and when they act is unknown. Here, we dissected the role of a nociceptin peptide circuit in reward seeking. We generated a prepronociceptin (Pnoc)-Cre mouse line that revealed a unique subpopulation of paranigral ventral tegmental area (pnVTA) neurons enriched in prepronociceptin. Fiber photometry recordings during progressive ratio operant behavior revealed pnVTAPnoc neurons become most active when mice stop seeking natural rewards. Selective pnVTAPnoc neuron ablation, inhibition, and conditional VTA nociceptin receptor (NOPR) deletion increased operant responding, revealing that the pnVTAPnoc nucleus and VTA NOPR signaling are necessary for regulating reward motivation. Additionally, optogenetic and chemogenetic activation of this pnVTAPnoc nucleus caused avoidance and decreased motivation for rewards. These findings provide insight into neuromodulatory circuits that regulate motivated behaviors through identification of a previously unknown neuropeptide-containing pnVTA nucleus that limits motivation for rewards.

Fear Extinction Requires Reward.

Learning theorists long hypothesized that appetitive and aversive motivational states influence one another antagonistically. Here, Felsenberg et al. show that the activity of neurons in Drosophila known to be important in appetitive conditioning is necessary for the extinction of aversive conditioning, thereby uncovering biological evidence for this opponent-process.

A Neural Circuit for Gut-Induced Reward.

The gut is now recognized as a major regulator of motivational and emotional states. However, the relevant gut-brain neuronal circuitry remains unknown. We show that optical activation of gut-innervating vagal sensory neurons recapitulates the hallmark effects of stimulating brain reward neurons. Specifically, right, but not left, vagal sensory ganglion activation sustained self-stimulation behavior, conditioned both flavor and place preferences, and induced dopamine release from Substantia nigra. Cell-specific transneuronal tracing revealed asymmetric ascending pathways of vagal origin throughout the CNS. In particular, transneuronal labeling identified the glutamatergic neurons of the dorsolateral parabrachial region as the obligatory relay linking the right vagal sensory ganglion to dopamine cells in Substantia nigra. Consistently, optical activation of parabrachio-nigral projections replicated the rewarding effects of right vagus excitation. Our findings establish the vagal gut-to-brain axis as an integral component of the neuronal reward pathway. They also suggest novel vagal stimulation approaches to affective disorders.

Molecular and Circuit-Dynamical Identification of Top-Down Neural Mechanisms for Restraint of Reward Seeking.

Reward-seeking behavior is fundamental to survival, but suppression of this behavior can be essential as well, even for rewards of high value. In humans and rodents, the medial prefrontal cortex (mPFC) has been implicated in suppressing reward seeking; however, despite vital significance in health and disease, the neural circuitry through which mPFC regulates reward seeking remains incompletely understood. Here, we show that a specific subset of superficial mPFC projections to a subfield of nucleus accumbens (NAc) neurons naturally encodes the decision to initiate or suppress reward seeking when faced with risk of punishment. A highly resolved subpopulation of these top-down projecting neurons, identified by 2-photon Ca2+ imaging and activity-dependent labeling to recruit the relevant neurons, was found capable of suppressing reward seeking. This natural activity-resolved mPFC-to-NAc projection displayed unique molecular-genetic and microcircuit-level features concordant with a conserved role in the regulation of reward-seeking behavior, providing cellular and anatomical identifiers of behavioral and possible therapeutic significance.