Estrogenic control of reward prediction errors and reinforcement learning.

Gonadal hormones act throughout the brain 1 , and nearly all neuropsychiatric disorders vary in symptom severity with hormonal fluctuations over the reproductive cycle, gestation, and perimenopause 2-4 . Yet the mechanisms by which hormones influence mental and cognitive processes are unclear. Exogenous estrogenic hormones modulate dopamine signaling in the nucleus accumbens core (NAcc) 5,6 , which instantiates reward prediction errors (RPEs) for reinforcement learning 7-16 . Here we show that endogenous estrogenic hormones enhance RPEs and sensitivity to previous rewards by regulating expression of dopamine reuptake proteins in the NAcc. We trained rats to perform a temporal wagering task with different reward states; rats adjusted how quickly they initiated trials across states, balancing effort against expected rewards. Dopamine release in the NAcc reflected RPEs that predicted and causally in-fluenced subsequent initiation times. When fertile, females more quickly adjusted their initiation times to match reward states due to enhanced dopaminergic RPEs in the NAcc. Proteomics revealed reduced expression of dopamine transporters in fertile stages of the reproductive cycle. Finally, genetic suppression of midbrain estrogen receptors eliminated hormonal modulation of behavior. Estrogenic hormones therefore control the rate of reinforcement learning by regulating RPEs via dopamine reuptake, providing a mechanism by which hormones influence neural dynamics for motivation and learning.

Neural dynamics in the limbic system during male social behaviors.

Sexual and aggressive behaviors are vital for species survival and individual reproductive success. Although many limbic regions have been found relevant to these behaviors, how social cues are represented across regions and how the network activity generates each behavior remains elusive. To answer these questions, we utilize multi-fiber photometry (MFP) to simultaneously record Ca2+ signals of estrogen receptor alpha (Esr1)-expressing cells from 13 limbic regions in male mice during mating and fighting. We find that conspecific sensory information and social action signals are widely distributed in the limbic system and can be decoded from the network activity. Cross-region correlation analysis reveals striking increases in the network functional connectivity during the social action initiation phase, whereas late copulation is accompanied by a "dissociated" network state. Based on the response patterns, we propose a mating-biased network (MBN) and an aggression-biased network (ABN) for mediating male sexual and aggressive behaviors, respectively.

A hypothalamic pathway that suppresses aggression toward superior opponents.

Aggression is costly and requires tight regulation. Here we identify the projection from estrogen receptor alpha-expressing cells in the caudal part of the medial preoptic area (cMPOAEsr1) to the ventrolateral part of the ventromedial hypothalamus (VMHvl) as an essential pathway for modulating aggression in male mice. cMPOAEsr1 cells increase activity mainly during male-male interaction, which differs from the female-biased response pattern of rostral MPOAEsr1 (rMPOAEsr1) cells. Notably, cMPOAEsr1 cell responses to male opponents correlated with the opponents' fighting capability, which mice could estimate based on physical traits or learn through physical combats. Inactivating the cMPOAEsr1-VMHvl pathway increased aggression, whereas activating the pathway suppressed natural intermale aggression. Thus, cMPOAEsr1 is a key population for encoding opponents' fighting capability-information that could be used to prevent animals from engaging in disadvantageous conflicts with superior opponents by suppressing the activity of VMHvl cells essential for attack behaviors.

Posterior amygdala regulates sexual and aggressive behaviors in male mice.

Sexual and aggressive behaviors are fundamental to animal survival and reproduction. The medial preoptic nucleus (MPN) and ventrolateral part of the ventromedial hypothalamus (VMHvl) are essential regions for male sexual and aggressive behaviors, respectively. While key inhibitory inputs to the VMHvl and MPN have been identified, the extrahypothalamic excitatory inputs essential for social behaviors remain elusive. Here we identify estrogen receptor alpha (Esr1)-expressing cells in the posterior amygdala (PA) as a main source of excitatory inputs to the hypothalamus and key mediators for mating and fighting in male mice. We find two largely distinct PA subpopulations that differ in connectivity, gene expression, in vivo responses and social behavior relevance. MPN-projecting PAEsr1+ cells are activated during mating and are necessary and sufficient for male sexual behaviors, while VMHvl-projecting PAEsr1+ cells are excited during intermale aggression and promote attacks. These findings place the PA as a key node in both male aggression and reproduction circuits.

Hypothalamic Control of Conspecific Self-Defense.

Active defense against a conspecific aggressor is essential for survival. Previous studies revealed strong c-Fos expression in the ventrolateral part of the ventromedial hypothalamus (VMHvl) in defeated animals. Here, we examined the functional relevance and in vivo responses of the VMHvl during conspecific defense. We found that VMHvl cells expressing estrogen receptor α (Esr1) are acutely excited during active conspecific defense. Optogenetic inhibition of the cells compromised an animal's ability to actively defend against an aggressor, whereas activating the cells elicited defense-like behaviors. Furthermore, the VMHvl is known for its role in aggression. In vivo recording and c-Fos mapping revealed differential organization of the defense and aggression-responsive cells in the VMHvl. Specifically, defense-activated cells are concentrated in the anterior part of the VMHvl, which preferentially targets the periaqueductal gray (PAG). Thus, our study identified an essential neural substrate for active conspecific defense and expanded the function of the VMHvl.

A Hypothalamic Midbrain Pathway Essential for Driving Maternal Behaviors.

Maternal behaviors are essential for the survival of the young. Previous studies implicated the medial preoptic area (MPOA) as an important region for maternal behaviors, but details of the maternal circuit remain incompletely understood. Here we identify estrogen receptor alpha (Esr1)-expressing cells in the MPOA as key mediators of pup approach and retrieval. Reversible inactivation of MPOAEsr1+ cells impairs those behaviors, whereas optogenetic activation induces immediate pup retrieval. In vivo recordings demonstrate preferential activation of MPOAEsr1+ cells during maternal behaviors and changes in MPOA cell responses across reproductive states. Furthermore, channelrhodopsin-assisted circuit mapping reveals a strong inhibitory projection from MPOAEsr1+ cells to ventral tegmental area (VTA) non-dopaminergic cells. Pathway-specific manipulations reveal that this projection is essential for driving pup approach and retrieval and that VTA dopaminergic cells are reliably activated during those behaviors. Altogether, this study provides new insight into the neural circuit that generates maternal behaviors.

Esr1+ cells in the ventromedial hypothalamus control female aggression.

As an essential means of resolving conflicts, aggression is expressed by both sexes but often at a higher level in males than in females. Recent studies suggest that cells in the ventrolateral part of the ventromedial hypothalamus (VMHvl) that express estrogen receptor-α (Esr1) and progesterone receptor are essential for male but not female mouse aggression. In contrast, here we show that VMHvlEsr1+ cells are indispensable for female aggression. This population was active when females attacked naturally. Inactivation of these cells reduced female aggression whereas their activation elicited attack. Additionally, we found that female VMHvl contains two anatomically distinguishable subdivisions that showed differential gene expression, projection and activation patterns after mating and fighting. These results support an essential role of the VMHvl in both male and female aggression and reveal the existence of two previously unappreciated subdivisions in the female VMHvl that are involved in distinct social behaviors.

Recent advances in understanding the role of the hypothalamic circuit during aggression.

The hypothalamus was first implicated in the classic "fight or flight" response nearly a century ago, and since then, many important strides have been made in understanding both the circuitry and the neural dynamics underlying the generation of these behaviors. In this review, we will focus on the role of the hypothalamus in aggression, paying particular attention to recent advances in the field that have allowed for functional identification of relevant hypothalamic subnuclei. Recent progress in this field has been aided by the development of new techniques for functional manipulation including optogenetics and pharmacogenetics, as well as advances in technology used for chronic in vivo recordings during complex social behaviors. We will examine the role of the hypothalamus through the complimentary lenses of (1) loss of function studies, including pharmacology and pharmacogenetics; (2) gain of function studies, including specific comparisons between results from classic electrical stimulation studies and more recent work using optogenetics; and (3) neural activity, including both immediate early gene and awake-behaving recordings. Lastly, we will outline current approaches to identifying the precise role of the hypothalamus in promoting aggressive motivation and aggressive action.

Functional identification of an aggression locus in the mouse hypothalamus.

Electrical stimulation of certain hypothalamic regions in cats and rodents can elicit attack behaviour, but the exact location of relevant cells within these regions, their requirement for naturally occurring aggression and their relationship to mating circuits have not been clear. Genetic methods for neural circuit manipulation in mice provide a potentially powerful approach to this problem, but brain-stimulation-evoked aggression has never been demonstrated in this species. Here we show that optogenetic, but not electrical, stimulation of neurons in the ventromedial hypothalamus, ventrolateral subdivision (VMHvl) causes male mice to attack both females and inanimate objects, as well as males. Pharmacogenetic silencing of VMHvl reversibly inhibits inter-male aggression. Immediate early gene analysis and single unit recordings from VMHvl during social interactions reveal overlapping but distinct neuronal subpopulations involved in fighting and mating. Neurons activated during attack are inhibited during mating, suggesting a potential neural substrate for competition between these opponent social behaviours.