Learning attentional templates for value-based decision-making.

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed across tasks.

Touch neurons underlying dopaminergic pleasurable touch and sexual receptivity.

Pleasurable touch is paramount during social behavior, including sexual encounters. However, the identity and precise role of sensory neurons that transduce sexual touch remain unknown. A population of sensory neurons labeled by developmental expression of the G protein-coupled receptor Mrgprb4 detects mechanical stimulation in mice. Here, we study the social relevance of Mrgprb4-lineage neurons and reveal that these neurons are required for sexual receptivity and sufficient to induce dopamine release in the brain. Even in social isolation, optogenetic stimulation of Mrgprb4-lineage neurons through the back skin is sufficient to induce a conditioned place preference and a striking dorsiflexion resembling the lordotic copulatory posture. In the absence of Mrgprb4-lineage neurons, female mice no longer find male mounts rewarding: sexual receptivity is supplanted by aggression and a coincident decline in dopamine release in the nucleus accumbens. Together, these findings establish that Mrgprb4-lineage neurons initiate a skin-to-brain circuit encoding the rewarding quality of social touch.

A neural circuit for male sexual behavior and reward.

Male sexual behavior is innate and rewarding. Despite its centrality to reproduction, a molecularly specified neural circuit governing innate male sexual behavior and reward remains to be characterized. We have discovered a developmentally wired neural circuit necessary and sufficient for male mating. This circuit connects chemosensory input to BNSTprTac1 neurons, which innervate POATacr1 neurons that project to centers regulating motor output and reward. Epistasis studies demonstrate that BNSTprTac1 neurons are upstream of POATacr1 neurons, and BNSTprTac1-released substance P following mate recognition potentiates activation of POATacr1 neurons through Tacr1 to initiate mating. Experimental activation of POATacr1 neurons triggers mating, even in sexually satiated males, and it is rewarding, eliciting dopamine release and self-stimulation of these cells. Together, we have uncovered a neural circuit that governs the key aspects of innate male sexual behavior: motor displays, drive, and reward.

Neural mechanism underlying depressive-like state associated with social status loss.

Downward social mobility is a well-known mental risk factor for depression, but its neural mechanism remains elusive. Here, by forcing mice to lose against their subordinates in a non-violent social contest, we lower their social ranks stably and induce depressive-like behaviors. These rank-decline-associated depressive-like behaviors can be reversed by regaining social status. In vivo fiber photometry and single-unit electrophysiological recording show that forced loss, but not natural loss, generates negative reward prediction error (RPE). Through the lateral hypothalamus, the RPE strongly activates the brain's anti-reward center, the lateral habenula (LHb). LHb activation inhibits the medial prefrontal cortex (mPFC) that controls social competitiveness and reinforces retreats in contests. These results reveal the core neural mechanisms mutually promoting social status loss and depressive behaviors. The intertwined neuronal signaling controlling mPFC and LHb activities provides a mechanistic foundation for the crosstalk between social mobility and psychological disorder, unveiling a promising target for intervention.

Cell-type-specific population dynamics of diverse reward computations.

Computational analysis of cellular activity has developed largely independently of modern transcriptomic cell typology, but integrating these approaches may be essential for full insight into cellular-level mechanisms underlying brain function and dysfunction. Applying this approach to the habenula (a structure with diverse, intermingled molecular, anatomical, and computational features), we identified encoding of reward-predictive cues and reward outcomes in distinct genetically defined neural populations, including TH+ cells and Tac1+ cells. Data from genetically targeted recordings were used to train an optimized nonlinear dynamical systems model and revealed activity dynamics consistent with a line attractor. High-density, cell-type-specific electrophysiological recordings and optogenetic perturbation provided supporting evidence for this model. Reverse-engineering predicted how Tac1+ cells might integrate reward history, which was complemented by in vivo experimentation. This integrated approach describes a process by which data-driven computational models of population activity can generate and frame actionable hypotheses for cell-type-specific investigation in biological systems.

Hippocampal neurons construct a map of an abstract value space.

The hippocampus is thought to encode a "cognitive map," a structural organization of knowledge about relationships in the world. Place cells, spatially selective hippocampal neurons that have been extensively studied in rodents, are one component of this map, describing the relative position of environmental features. However, whether this map extends to abstract, cognitive information remains unknown. Using the relative reward value of cues to define continuous "paths" through an abstract value space, we show that single neurons in primate hippocampus encode this space through value place fields, much like a rodent's place neurons encode paths through physical space. Value place fields remapped when cues changed but also became increasingly correlated across contexts, allowing maps to become generalized. Our findings help explain the critical contribution of the hippocampus to value-based decision-making, providing a mechanism by which knowledge of relationships in the world can be incorporated into reward predictions for guiding decisions.

A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement.

The striosome compartment within the dorsal striatum has been implicated in reinforcement learning and regulation of motivation, but how striosomal neurons contribute to these functions remains elusive. Here, we show that a genetically identified striosomal population, which expresses the Teashirt family zinc finger 1 (Tshz1) and belongs to the direct pathway, drives negative reinforcement and is essential for aversive learning in mice. Contrasting a "conventional" striosomal direct pathway, the Tshz1 neurons cause aversion, movement suppression, and negative reinforcement once activated, and they receive a distinct set of synaptic inputs. These neurons are predominantly excited by punishment rather than reward and represent the anticipation of punishment or the motivation for avoidance. Furthermore, inhibiting these neurons impairs punishment-based learning without affecting reward learning or movement. These results establish a major role of striosomal neurons in behaviors reinforced by punishment and moreover uncover functions of the direct pathway unaccounted for in classic models.

Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex.

Every decision we make is accompanied by a sense of confidence about its likely outcome. This sense informs subsequent behavior, such as investing more-whether time, effort, or money-when reward is more certain. A neural representation of confidence should originate from a statistical computation and predict confidence-guided behavior. An additional requirement for confidence representations to support metacognition is abstraction: they should emerge irrespective of the source of information and inform multiple confidence-guided behaviors. It is unknown whether neural confidence signals meet these criteria. Here, we show that single orbitofrontal cortex neurons in rats encode statistical decision confidence irrespective of the sensory modality, olfactory or auditory, used to make a choice. The activity of these neurons also predicts two confidence-guided behaviors: trial-by-trial time investment and cross-trial choice strategy updating. Orbitofrontal cortex thus represents decision confidence consistent with a metacognitive process that is useful for mediating confidence-guided economic decisions.

A Hippocampus-Accumbens Tripartite Neuronal Motif Guides Appetitive Memory in Space.

Retrieving and acting on memories of food-predicting environments are fundamental processes for animal survival. Hippocampal pyramidal cells (PYRs) of the mammalian brain provide mnemonic representations of space. Yet the substrates by which these hippocampal representations support memory-guided behavior remain unknown. Here, we uncover a direct connection from dorsal CA1 (dCA1) hippocampus to nucleus accumbens (NAc) that enables the behavioral manifestation of place-reward memories. By monitoring neuronal ensembles in mouse dCA1→NAc pathway, combined with cell-type selective optogenetic manipulations of input-defined postsynaptic neurons, we show that dCA1 PYRs drive NAc medium spiny neurons and orchestrate their spiking activity using feedforward inhibition mediated by dCA1-connected parvalbumin-expressing fast-spiking interneurons. This tripartite cross-circuit motif supports spatial appetitive memory and associated NAc assemblies, being independent of dorsal subiculum and dispensable for both spatial novelty detection and reward seeking. Our findings demonstrate that the dCA1→NAc pathway instantiates a limbic-motor interface for neuronal representations of space to promote effective appetitive behavior.

A Paranigral VTA Nociceptin Circuit that Constrains Motivation for Reward.

Nociceptin and its receptor are widely distributed throughout the brain in regions associated with reward behavior, yet how and when they act is unknown. Here, we dissected the role of a nociceptin peptide circuit in reward seeking. We generated a prepronociceptin (Pnoc)-Cre mouse line that revealed a unique subpopulation of paranigral ventral tegmental area (pnVTA) neurons enriched in prepronociceptin. Fiber photometry recordings during progressive ratio operant behavior revealed pnVTAPnoc neurons become most active when mice stop seeking natural rewards. Selective pnVTAPnoc neuron ablation, inhibition, and conditional VTA nociceptin receptor (NOPR) deletion increased operant responding, revealing that the pnVTAPnoc nucleus and VTA NOPR signaling are necessary for regulating reward motivation. Additionally, optogenetic and chemogenetic activation of this pnVTAPnoc nucleus caused avoidance and decreased motivation for rewards. These findings provide insight into neuromodulatory circuits that regulate motivated behaviors through identification of a previously unknown neuropeptide-containing pnVTA nucleus that limits motivation for rewards.

Shared Cortex-Cerebellum Dynamics in the Execution and Learning of a Motor Task.

Throughout mammalian neocortex, layer 5 pyramidal (L5) cells project via the pons to a vast number of cerebellar granule cells (GrCs), forming a fundamental pathway. Yet, it is unknown how neuronal dynamics are transformed through the L5→GrC pathway. Here, by directly comparing premotor L5 and GrC activity during a forelimb movement task using dual-site two-photon Ca2+ imaging, we found that in expert mice, L5 and GrC dynamics were highly similar. L5 cells and GrCs shared a common set of task-encoding activity patterns, possessed similar diversity of responses, and exhibited high correlations comparable to local correlations among L5 cells. Chronic imaging revealed that these dynamics co-emerged in cortex and cerebellum over learning: as behavioral performance improved, initially dissimilar L5 cells and GrCs converged onto a shared, low-dimensional, task-encoding set of neural activity patterns. Thus, a key function of cortico-cerebellar communication is the propagation of shared dynamics that emerge during learning.

Primate Amygdala Neurons Simulate Decision Processes of Social Partners.

By observing their social partners, primates learn about reward values of objects. Here, we show that monkeys' amygdala neurons derive object values from observation and use these values to simulate a partner monkey's decision process. While monkeys alternated making reward-based choices, amygdala neurons encoded object-specific values learned from observation. Dynamic activities converted these values to representations of the recorded monkey's own choices. Surprisingly, the same activity patterns unfolded spontaneously before partner's choices in separate neurons, as if these neurons simulated the partner's decision-making. These "simulation neurons" encoded signatures of mutual-inhibitory decision computation, including value comparisons and value-to-choice conversions, resulting in accurate predictions of partner's choices. Population decoding identified differential contributions of amygdala subnuclei. Biophysical modeling of amygdala circuits showed that simulation neurons emerge naturally from convergence between object-value neurons and self-other neurons. By simulating decision computations during observation, these neurons could allow primates to reconstruct their social partners' mental states.

A Neural Circuit for Gut-Induced Reward.

The gut is now recognized as a major regulator of motivational and emotional states. However, the relevant gut-brain neuronal circuitry remains unknown. We show that optical activation of gut-innervating vagal sensory neurons recapitulates the hallmark effects of stimulating brain reward neurons. Specifically, right, but not left, vagal sensory ganglion activation sustained self-stimulation behavior, conditioned both flavor and place preferences, and induced dopamine release from Substantia nigra. Cell-specific transneuronal tracing revealed asymmetric ascending pathways of vagal origin throughout the CNS. In particular, transneuronal labeling identified the glutamatergic neurons of the dorsolateral parabrachial region as the obligatory relay linking the right vagal sensory ganglion to dopamine cells in Substantia nigra. Consistently, optical activation of parabrachio-nigral projections replicated the rewarding effects of right vagus excitation. Our findings establish the vagal gut-to-brain axis as an integral component of the neuronal reward pathway. They also suggest novel vagal stimulation approaches to affective disorders.

Molecular and Circuit-Dynamical Identification of Top-Down Neural Mechanisms for Restraint of Reward Seeking.

Reward-seeking behavior is fundamental to survival, but suppression of this behavior can be essential as well, even for rewards of high value. In humans and rodents, the medial prefrontal cortex (mPFC) has been implicated in suppressing reward seeking; however, despite vital significance in health and disease, the neural circuitry through which mPFC regulates reward seeking remains incompletely understood. Here, we show that a specific subset of superficial mPFC projections to a subfield of nucleus accumbens (NAc) neurons naturally encodes the decision to initiate or suppress reward seeking when faced with risk of punishment. A highly resolved subpopulation of these top-down projecting neurons, identified by 2-photon Ca2+ imaging and activity-dependent labeling to recruit the relevant neurons, was found capable of suppressing reward seeking. This natural activity-resolved mPFC-to-NAc projection displayed unique molecular-genetic and microcircuit-level features concordant with a conserved role in the regulation of reward-seeking behavior, providing cellular and anatomical identifiers of behavioral and possible therapeutic significance.

Mesoaccumbal Dopamine Heterogeneity: What Do Dopamine Firing and Release Have to Do with It?

Ventral tegmental area (VTA) dopamine (DA) neurons are often thought to uniformly encode reward prediction errors. Conversely, DA release in the nucleus accumbens (NAc), the prominent projection target of these neurons, has been implicated in reinforcement learning, motivation, aversion, and incentive salience. This contrast between heterogeneous functions of DA release versus a homogeneous role for DA neuron activity raises numerous questions regarding how VTA DA activity translates into NAc DA release. Further complicating this issue is increasing evidence that distinct VTA DA projections into defined NAc subregions mediate diverse behavioral functions. Here, we evaluate evidence for heterogeneity within the mesoaccumbal DA system and argue that frameworks of DA function must incorporate the precise topographic organization of VTA DA neurons to clarify their contribution to health and disease.

The central renin-angiotensin system: A genetic pathway, functional decoding, and selective target engagement characterization in humans.

Accumulating evidence suggests that the brain renin angiotensin system (RAS) plays a pivotal role in the regulation of cognition and behavior as well as in the neuropathology of neurological and mental disorders. The angiotensin II type 1 receptor (AT1R) mediates most functional and neuropathology-relevant actions associated with the central RAS. However, an overarching comprehension to guide translation and utilize the therapeutic potential of the central RAS in humans is currently lacking. We conducted a comprehensive characterization of the RAS using an innovative combination of transcriptomic gene expression mapping, image-based behavioral decoding, and pre-registered randomized controlled discovery-replication pharmacological resting-state functional magnetic resonance imaging (fMRI) trials (N = 132) with a selective AT1R antagonist. The AT1R exhibited a particular dense expression in a subcortical network encompassing the thalamus, striatum, and amygdalo-hippocampal formation. Behavioral decoding of the AT1R gene expression brain map showed an association with memory, stress, reward, and motivational processes. Transient pharmacological blockade of the AT1R further decreased neural activity in subcortical systems characterized by a high AT1R expression, while increasing functional connectivity in the cortico-basal ganglia-thalamo-cortical circuitry. Effects of AT1R blockade on the network level were specifically associated with the transcriptomic signatures of the dopaminergic, opioid, acetylcholine, and corticotropin-releasing hormone signaling systems. The robustness of the results was supported in an independent pharmacological fMRI trial. These findings present a biologically informed comprehensive characterization of the central AT1R pathways and their functional relevance on the neural and behavioral level in humans.

A dopamine mechanism for reward maximization.

Individual survival and evolutionary selection require biological organisms to maximize reward. Economic choice theories define the necessary and sufficient conditions, and neuronal signals of decision variables provide mechanistic explanations. Reinforcement learning (RL) formalisms use predictions, actions, and policies to maximize reward. Midbrain dopamine neurons code reward prediction errors (RPE) of subjective reward value suitable for RL. Electrical and optogenetic self-stimulation experiments demonstrate that monkeys and rodents repeat behaviors that result in dopamine excitation. Dopamine excitations reflect positive RPEs that increase reward predictions via RL; against increasing predictions, obtaining similar dopamine RPE signals again requires better rewards than before. The positive RPEs drive predictions higher again and thus advance a recursive reward-RPE-prediction iteration toward better and better rewards. Agents also avoid dopamine inhibitions that lower reward prediction via RL, which allows smaller rewards than before to elicit positive dopamine RPE signals and resume the iteration toward better rewards. In this way, dopamine RPE signals serve a causal mechanism that attracts agents via RL to the best rewards. The mechanism improves daily life and benefits evolutionary selection but may also induce restlessness and greed.

The gut-brain axis and cognitive control: A role for the vagus nerve.

Survival requires the integration of external information and interoceptive cues to effectively guide advantageous behaviors, particularly foraging and other behaviors that promote energy acquisition and consumption. The vagus nerve acts as a critical relay between the abdominal viscera and the brain to convey metabolic signals. This review synthesizes recent findings from rodent models and humans revealing the impact of vagus nerve signaling from the gut on the control of higher-order neurocognitive domains, including anxiety, depression, reward motivation, and learning and memory. We propose a framework where meal consumption engages gastrointestinal tract-originating vagal afferent signaling that functions to alleviate anxiety and depressive-like states, while also promoting motivational and memory functions. These concurrent processes serve to favor the encoding of meal-relevant information into memory storage, thus facilitating future foraging behaviors. Modulation of these neurocognitive domains by vagal tone is also discussed in the context of pathological conditions, including the use of transcutaneous vagus nerve stimulation for the treatment of anxiety disorders, major depressive disorder, and dementia-associated memory impairments. Collectively, these findings highlight the contributions of gastrointestinal vagus nerve signaling to the regulation of neurocognitive processes that shape various adaptive behavioral responses.

Anxiety, Depression, and Decision Making: A Computational Perspective.

In everyday life, the outcomes of our actions are rarely certain. Further, we often lack the information needed to precisely estimate the probability and value of potential outcomes as well as how much effort will be required by the courses of action under consideration. Under such conditions of uncertainty, individual differences in the estimation and weighting of these variables, and in reliance on model-free versus model-based decision making, have the potential to strongly influence our behavior. Both anxiety and depression are associated with difficulties in decision making. Further, anxiety is linked to increased engagement in threat-avoidance behaviors and depression is linked to reduced engagement in reward-seeking behaviors. The precise deficits, or biases, in decision making associated with these common forms of psychopathology remain to be fully specified. In this article, we review evidence for which of the computations supporting decision making are altered in anxiety and depression and consider the potential consequences for action selection. In addition, we provide a schematic framework that integrates the findings reviewed and will hopefully be of value to future studies.

Memory Allocation: Mechanisms and Function.

Memories for events are thought to be represented in sparse, distributed neuronal ensembles (or engrams). In this article, we review how neurons are chosen to become part of a particular engram, via a process of neuronal allocation. Experiments in rodents indicate that eligible neurons compete for allocation to a given engram, with more excitable neurons winning this competition. Moreover, fluctuations in neuronal excitability determine how engrams interact, promoting either memory integration (via coallocation to overlapping engrams) or separation (via disallocation to nonoverlapping engrams). In parallel with rodent studies, recent findings in humans verify the importance of this memory integration process for linking memories that occur close in time or share related content. A deeper understanding of allocation promises to provide insights into the logic underlying how knowledge is normally organized in the brain and the disorders in which this process has gone awry.