A neural circuit for male sexual behavior and reward.

Male sexual behavior is innate and rewarding. Despite its centrality to reproduction, a molecularly specified neural circuit governing innate male sexual behavior and reward remains to be characterized. We have discovered a developmentally wired neural circuit necessary and sufficient for male mating. This circuit connects chemosensory input to BNSTprTac1 neurons, which innervate POATacr1 neurons that project to centers regulating motor output and reward. Epistasis studies demonstrate that BNSTprTac1 neurons are upstream of POATacr1 neurons, and BNSTprTac1-released substance P following mate recognition potentiates activation of POATacr1 neurons through Tacr1 to initiate mating. Experimental activation of POATacr1 neurons triggers mating, even in sexually satiated males, and it is rewarding, eliciting dopamine release and self-stimulation of these cells. Together, we have uncovered a neural circuit that governs the key aspects of innate male sexual behavior: motor displays, drive, and reward.

Cell-type-specific population dynamics of diverse reward computations.

Computational analysis of cellular activity has developed largely independently of modern transcriptomic cell typology, but integrating these approaches may be essential for full insight into cellular-level mechanisms underlying brain function and dysfunction. Applying this approach to the habenula (a structure with diverse, intermingled molecular, anatomical, and computational features), we identified encoding of reward-predictive cues and reward outcomes in distinct genetically defined neural populations, including TH+ cells and Tac1+ cells. Data from genetically targeted recordings were used to train an optimized nonlinear dynamical systems model and revealed activity dynamics consistent with a line attractor. High-density, cell-type-specific electrophysiological recordings and optogenetic perturbation provided supporting evidence for this model. Reverse-engineering predicted how Tac1+ cells might integrate reward history, which was complemented by in vivo experimentation. This integrated approach describes a process by which data-driven computational models of population activity can generate and frame actionable hypotheses for cell-type-specific investigation in biological systems.

A genetically defined insula-brainstem circuit selectively controls motivational vigor.

The anterior insular cortex (aIC) plays a critical role in cognitive and motivational control of behavior, but the underlying neural mechanism remains elusive. Here, we show that aIC neurons expressing Fezf2 (aICFezf2), which are the pyramidal tract neurons, signal motivational vigor and invigorate need-seeking behavior through projections to the brainstem nucleus tractus solitarii (NTS). aICFezf2 neurons and their postsynaptic NTS neurons acquire anticipatory activity through learning, which encodes the perceived value and the vigor of actions to pursue homeostatic needs. Correspondingly, aIC → NTS circuit activity controls vigor, effort, and striatal dopamine release but only if the action is learned and the outcome is needed. Notably, aICFezf2 neurons do not represent taste or valence. Moreover, aIC → NTS activity neither drives reinforcement nor influences total consumption. These results pinpoint specific functions of aIC → NTS circuit for selectively controlling motivational vigor and suggest that motivation is subserved, in part, by aIC's top-down regulation of dopamine signaling.

Striosomes Mediate Value-Based Learning Vulnerable in Age and a Huntington's Disease Model.

Learning valence-based responses to favorable and unfavorable options requires judgments of the relative value of the options, a process necessary for species survival. We found, using engineered mice, that circuit connectivity and function of the striosome compartment of the striatum are critical for this type of learning. Calcium imaging during valence-based learning exhibited a selective correlation between learning and striosomal but not matrix signals. This striosomal activity encoded discrimination learning and was correlated with task engagement, which, in turn, could be regulated by chemogenetic excitation and inhibition. Striosomal function during discrimination learning was disturbed with aging and severely so in a mouse model of Huntington's disease. Anatomical and functional connectivity of parvalbumin-positive, putative fast-spiking interneurons (FSIs) to striatal projection neurons was enhanced in striosomes compared with matrix in mice that learned. Computational modeling of these findings suggests that FSIs can modulate the striosomal signal-to-noise ratio, crucial for discrimination and learning.

A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement.

The striosome compartment within the dorsal striatum has been implicated in reinforcement learning and regulation of motivation, but how striosomal neurons contribute to these functions remains elusive. Here, we show that a genetically identified striosomal population, which expresses the Teashirt family zinc finger 1 (Tshz1) and belongs to the direct pathway, drives negative reinforcement and is essential for aversive learning in mice. Contrasting a "conventional" striosomal direct pathway, the Tshz1 neurons cause aversion, movement suppression, and negative reinforcement once activated, and they receive a distinct set of synaptic inputs. These neurons are predominantly excited by punishment rather than reward and represent the anticipation of punishment or the motivation for avoidance. Furthermore, inhibiting these neurons impairs punishment-based learning without affecting reward learning or movement. These results establish a major role of striosomal neurons in behaviors reinforced by punishment and moreover uncover functions of the direct pathway unaccounted for in classic models.

A Paranigral VTA Nociceptin Circuit that Constrains Motivation for Reward.

Nociceptin and its receptor are widely distributed throughout the brain in regions associated with reward behavior, yet how and when they act is unknown. Here, we dissected the role of a nociceptin peptide circuit in reward seeking. We generated a prepronociceptin (Pnoc)-Cre mouse line that revealed a unique subpopulation of paranigral ventral tegmental area (pnVTA) neurons enriched in prepronociceptin. Fiber photometry recordings during progressive ratio operant behavior revealed pnVTAPnoc neurons become most active when mice stop seeking natural rewards. Selective pnVTAPnoc neuron ablation, inhibition, and conditional VTA nociceptin receptor (NOPR) deletion increased operant responding, revealing that the pnVTAPnoc nucleus and VTA NOPR signaling are necessary for regulating reward motivation. Additionally, optogenetic and chemogenetic activation of this pnVTAPnoc nucleus caused avoidance and decreased motivation for rewards. These findings provide insight into neuromodulatory circuits that regulate motivated behaviors through identification of a previously unknown neuropeptide-containing pnVTA nucleus that limits motivation for rewards.

Regulated Intron Removal Integrates Motivational State and Experience.

Myriad experiences produce transient memory, yet, contingent on the internal state of the organism and the saliency of the experience, only some memories persist over time. How experience and internal state influence the duration of memory at the molecular level remains unknown. A self-assembled aggregated state of Drosophila Orb2A protein is required specifically for long-lasting memory. We report that in the adult fly brain the mRNA encoding Orb2A protein exists in an unspliced non-protein-coding form. The convergence of experience and internal drive transiently increases the spliced protein-coding Orb2A mRNA. A screen identified pasilla, the fly ortholog of mammalian Nova-1/2, as a mediator of Orb2A mRNA processing. A single-nucleotide substitution in the intronic region that reduces Pasilla binding and intron removal selectively impairs long-term memory. We posit that pasilla-mediated processing of unspliced Orb2A mRNA integrates experience and internal state to control Orb2A protein abundance and long-term memory formation.

Mesoaccumbal Dopamine Heterogeneity: What Do Dopamine Firing and Release Have to Do with It?

Ventral tegmental area (VTA) dopamine (DA) neurons are often thought to uniformly encode reward prediction errors. Conversely, DA release in the nucleus accumbens (NAc), the prominent projection target of these neurons, has been implicated in reinforcement learning, motivation, aversion, and incentive salience. This contrast between heterogeneous functions of DA release versus a homogeneous role for DA neuron activity raises numerous questions regarding how VTA DA activity translates into NAc DA release. Further complicating this issue is increasing evidence that distinct VTA DA projections into defined NAc subregions mediate diverse behavioral functions. Here, we evaluate evidence for heterogeneity within the mesoaccumbal DA system and argue that frameworks of DA function must incorporate the precise topographic organization of VTA DA neurons to clarify their contribution to health and disease.

Biochemical evidence accumulates across neurons to drive a network-level eruption.

Neural network computations are usually assumed to emerge from patterns of fast electrical activity. Challenging this view, we show that a male fly's decision to persist in mating hinges on a biochemical computation that enables processing over minutes to hours. Each neuron in a recurrent network contains slightly different internal molecular estimates of mating progress. Protein kinase A (PKA) activity contrasts this internal measurement with input from the other neurons to represent accumulated evidence that the goal of the network has been achieved. When consensus is reached, PKA pushes the network toward a large-scale and synchronized burst of calcium influx that we call an eruption. Eruptions transform continuous deliberation within the network into an all-or-nothing output, after which the male will no longer sacrifice his life to continue mating. Here, biochemical activity, invisible to most large-scale recording techniques, is the key computational currency directing behavior and motivational state.

Reward Contributions to Serotonergic Functions.

The brain serotonin systems participate in numerous aspects of reward processing, although it remains elusive how exactly serotonin signals regulate neural computation and reward-related behavior. The application of optogenetics and imaging techniques during the last decade has provided many insights. Here, we review recent progress on the organization and physiology of the dorsal raphe serotonin neurons and the relationships between their activity and behavioral functions in the context of reward processing. We also discuss several interesting theories on serotonin's function and how these theories may be reconciled by the possibility that serotonin, acting in synergy with coreleased glutamate, tracks and calculates the so-called beneficialness of the current state to guide an animal's behavior in dynamic environments.

Apathy and Motivation: Biological Basis and Drug Treatment.

Apathy is a disabling syndrome associated with poor functional outcomes that is common across a broad range of neurological and psychiatric conditions. Currently, there are no established therapies specifically for the condition, and safe and effective treatments are urgently needed. Advances in the understanding of motivation and goal-directed behavior in humans and animals have shed light on the cognitive and neurobiological mechanisms contributing to apathy, providing an important foundation for the development of new treatments. Here, we review the cognitive components, neural circuitry, and pharmacology of apathy and motivation, highlighting converging evidence of shared transdiagnostic mechanisms. Though no pharmacological treatments have yet been licensed, we summarize trials of existing and novel compounds to date, identifying several promising candidates for clinical use and avenues of future drug development.

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making.

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we report an exploratory investigation into this with chronic intracranial recordings from the prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12 to 20 Hz) oscillations tracking effort on a single-trial basis and PFC theta (4 to 7 Hz) signaling previous trial reward, with no effects of net subjective value. Stimulation of PFC increased overall acceptance of offers and sensitivity to reward while decreasing the impact of effort on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, supports a causal role of PFC for such choices, and seeds hypotheses for future studies.

The gut-brain axis and cognitive control: A role for the vagus nerve.

Survival requires the integration of external information and interoceptive cues to effectively guide advantageous behaviors, particularly foraging and other behaviors that promote energy acquisition and consumption. The vagus nerve acts as a critical relay between the abdominal viscera and the brain to convey metabolic signals. This review synthesizes recent findings from rodent models and humans revealing the impact of vagus nerve signaling from the gut on the control of higher-order neurocognitive domains, including anxiety, depression, reward motivation, and learning and memory. We propose a framework where meal consumption engages gastrointestinal tract-originating vagal afferent signaling that functions to alleviate anxiety and depressive-like states, while also promoting motivational and memory functions. These concurrent processes serve to favor the encoding of meal-relevant information into memory storage, thus facilitating future foraging behaviors. Modulation of these neurocognitive domains by vagal tone is also discussed in the context of pathological conditions, including the use of transcutaneous vagus nerve stimulation for the treatment of anxiety disorders, major depressive disorder, and dementia-associated memory impairments. Collectively, these findings highlight the contributions of gastrointestinal vagus nerve signaling to the regulation of neurocognitive processes that shape various adaptive behavioral responses.

A unified theoretical framework underlying the regulation of motivated behavior.

To orchestrate behaviors for survival, multiple psychological components have evolved. The current theories do not clearly distinguish the distinct components. In this article, we provide a unified theoretical framework. To optimize survival, there should be four components; (1) "need", an alarm based on a predicted deficiency. (2) "motivation", a direct behavior driver. (3) "pleasure", a teacher based on immediate outcomes. (4) "utility", a teacher based on final delayed outcomes. For behavior stability, need should be accumulated into motivation to drive behavior. Based on the immediate outcome of the behavior, the pleasure should teach whether to continue the current behavior. Based on the final delay outcome, the utility should teach whether to increase future behavior by reshaping the other three components. We provide several neural substrate candidates in the food context. The proposed theoretical framework, in combination with appropriate experiments, will unravel the neural components responsible for each theoretical component.

Payments for ecosystem services did not crowd out pro-environmental behavior: Long-term experimental evidence from Uganda.

Payments for ecosystem services (PES) are increasingly being implemented worldwide as conservation instruments that provide conditional economic incentives to landowners for a prespecified duration. However, in the psychological and economic literature, critics have raised concerns that PES can undermine the recipient's intrinsic motivation to engage in pro-environmental behavior. Such "crowding out" may reduce the effectiveness of PES and may even worsen conservation outcomes once programs are terminated. In this study, we harnessed a randomized controlled trial that provided PES to land users in Western Uganda and evaluated whether these incentives had a persistent effect on pro-environmental behavior and its underlying behavioral drivers 6 y after the last payments were made. We elicited pro-environmental behavior with an incentivized, experimental measure that consisted of a choice for respondents between more and less environmentally friendly tree seedlings. In addition to this main outcome, survey-based measures for underlying behavioral drivers captured self-efficacy beliefs, intrinsic motivation, and perceived forest benefits. Overall, we found no indications that PES led to the crowding out of pro-environmental behavior. That is, respondents from the treatment villages were as likely as respondents from the control villages to choose environmentally friendly tree seedlings. We also found no systematic differences between these two groups in their underlying behavioral drivers, and nor did we find evidence for crowding effects when focusing on self-reported tree planting behavior as an alternative outcome measure.

Instructed motivational states bias reinforcement learning and memory formation.

Motivation influences goals, decisions, and memory formation. Imperative motivation links urgent goals to actions, narrowing the focus of attention and memory. Conversely, interrogative motivation integrates goals over time and space, supporting rich memory encoding for flexible future use. We manipulated motivational states via cover stories for a reinforcement learning task: The imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded. In an initial sample and a preregistered replication, we demonstrated that imperative motivation increased exploitation during reinforcement learning. Conversely, interrogative motivation increased directed (but not random) exploration, despite the cost to participants' earnings. At test, the interrogative group was more accurate at recognizing paintings and recalling associated values. In the interrogative group, higher value paintings were more likely to be remembered; imperative motivation disrupted this effect of reward modulating memory. Overall, we demonstrate that a prelearning motivational manipulation can bias learning and memory, bearing implications for education, behavior change, clinical interventions, and communication.

Effort Foraging Task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans.

Effort-based decisions, in which people weigh potential future rewards against effort costs required to achieve those rewards involve both cognitive and physical effort, though the mechanistic relationship between them is not yet understood. Here, we use an individual differences approach to isolate and measure the computational processes underlying effort-based decisions and test the association between cognitive and physical domains. Patch foraging is an ecologically valid reward rate maximization problem with well-developed theoretical tools. We developed the Effort Foraging Task, which embedded cognitive or physical effort into patch foraging, to quantify the cost of both cognitive and physical effort indirectly, by their effects on foraging choices. Participants chose between harvesting a depleting patch, or traveling to a new patch that was costly in time and effort. Participants' exit thresholds (reflecting the reward they expected to receive by harvesting when they chose to travel to a new patch) were sensitive to cognitive and physical effort demands, allowing us to quantify the perceived effort cost in monetary terms. The indirect sequential choice style revealed effort-seeking behavior in a minority of participants (preferring high over low effort) that has apparently been missed by many previous approaches. Individual differences in cognitive and physical effort costs were positively correlated, suggesting that these are perceived and processed in common. We used canonical correlation analysis to probe the relationship of task measures to self-reported affect and motivation, and found correlations of cognitive effort with anxiety, cognitive function, behavioral activation, and self-efficacy, but no similar correlations with physical effort.

Neural Representation of Valenced and Generic Probability and Uncertainty.

Representing the probability and uncertainty of outcomes facilitates adaptive behavior by allowing organisms to prepare in advance and devote attention to relevant events. Probability and uncertainty are often studied only for valenced (appetitive or aversive) outcomes, raising the question of whether the identified neural machinery also processes the probability and uncertainty of motivationally neutral outcomes. Here, we aimed to dissociate valenced from valence-independent (i.e., generic) probability (p; maximum at p = 1) and uncertainty (maximum at p = 0.5) signals using human neuroimaging. In a Pavlovian task (n = 41; 19 females), different cues predicted appetitive, aversive, or neutral liquids with different probabilities (p = 0, p = 0.5, p = 1). Cue-elicited motor responses accelerated, and pupil sizes increased primarily for cues that predicted valenced liquids with higher probability. For neutral liquids, uncertainty rather than probability tended to accelerate cue-induced responding and decrease pupil size. At the neural level, generic uncertainty signals were limited to the occipital cortex, while generic probability also activated the anterior ventromedial prefrontal cortex. These generic probability and uncertainty signals contrasted with cue-induced responses that only encoded the probability and uncertainty of valenced liquids in medial prefrontal, insular, and occipital cortices. Our findings show a behavioral and neural dissociation of generic and valenced signals. Thus, some parts of the brain keep track of motivational charge while others do not, highlighting the need and usefulness of characterizing the exact nature of learned representations.

Drawbacks to Strengthening Neural Salience Encoding: A Link Between Cortisol and Risky Drinking.

Emotionally salient experiences are encoded and remembered more strongly, an effect that can be amplified by hormones like cortisol. Such memories can in turn profoundly influence later behavior. However, little is known about the link between amplified salience encoding and subsequent behavior. This pathway may be particularly important for risky alcohol drinking, which has been linked to sensitized salience responses, memory, and cortisol. To test this possibility, we integrated pharmacology using a double-blind cross-over design with fMRI, cognitive, and motivation assays across a range of healthy male and female social drinkers. As anticipated, cortisol enhanced memory for salient alcohol-related events; critically, this bias was in turn associated with later alcohol motivation. Increased alcohol motivation was particularly pronounced in more susceptible risky drinkers, for whom cortisol enhanced brain salience responses to alcohol. These sensitized salience responses predicted both memory biases and alcohol motivation. Together, these findings reveal maladaptive consequences of enhanced salience encoding.

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers.

Reinforcement learning is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g., money, points) that are later exchanged for primary reinforcers (e.g., food, drink). Although symbolic reinforcers are ubiquitous in our daily lives, widely used in laboratory tasks because they can be motivating, mechanisms by which they become motivating are less understood. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g., the current number of accumulated tokens, choice options, task epoch, trials since the last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to then correlate with the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n = 5 monkeys, three males and two females). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in the state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.