Adolescent Dopamine Neurons Represent Reward Differently during Action and State Guided Learning.

Neuronal underpinning of learning cause-and-effect associations in the adolescent brain remains poorly understood. Two fundamental forms of associative learning are Pavlovian (classical) conditioning, where a stimulus is followed by an outcome, and operant (instrumental) conditioning, where outcome is contingent on action execution. Both forms of learning, when associated with a rewarding outcome, rely on midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN). We find that, in adolescent male rats, reward-guided associative learning is encoded differently by midbrain dopamine neurons in each conditioning paradigm. Whereas simultaneously recorded VTA and SN adult neurons have a similar phasic response to reward delivery during both forms of conditioning, adolescent neurons display a muted reward response during operant but a profoundly larger reward response during Pavlovian conditioning. These results suggest that adolescent neurons assign a different value to reward when it is not gated by action. The learning rate of adolescents and adults during both forms of conditioning was similar, supporting the notion that differences in reward response in each paradigm may be because of differences in motivation and independent of state versus action value learning. Static characteristics of dopamine neurons, such as dopamine cell number and size, were similar in the VTA and SN of both ages, but there were age-related differences in stimulated dopamine release and correlated spike activity, suggesting that differences in reward responsiveness by adolescent dopamine neurons are not because of differences in intrinsic properties of these neurons but engagement of different dopaminergic networks.SIGNIFICANCE STATEMENT Reckless behavior and impulsive decision-making by adolescents suggest that motivated behavioral states are encoded differently by the adolescent brain. Motivated behavior, which is dependent on the function of the dopamine system, follows learning of cause-and-effect associations in the environment. We find that dopamine neurons in adolescents encode reward differently depending on the cause-and-effect relationship of the means to receive that reward. Compared with adults, reward contingent on action led to a muted response, whereas reward that followed a cue but was not gated by action produced an augmented phasic response. These data demonstrate an age-related difference in dopamine neuron response to reward that is not uniform and is guided by processes that differentiate between state and action values.

Sex and strain differences in dynamic and static properties of the mesolimbic dopamine system.

Sex is a biological variable that contributes to the incidence, clinical course, and treatment outcome of brain disorders. Chief among these are disorders associated with the dopamine system. These include Parkinson's disease, ADHD, schizophrenia, and mood disorders, which show stark differences in prevalence and outcome between men and women. In order to reveal the influence of biological sex as a risk factor in these disorders, there is a critical need to collect fundamental information about basic properties of the dopamine system in males and females. In Long Evans rats, we measured dynamic and static properties related to the mesolimbic dopamine system. Static measures included assessing ventral tegmental area (VTA) dopamine cell number and volume and expression of tyrosine hydroxylase and dopamine transporter. Dynamic measures in behaving animals included assessing (1) VTA neuronal encoding during learning of a cue-action-reward instrumental task and (2) dopamine release in the nucleus accumbens in response to electrical stimulation of the VTA, vesicular depletion of dopamine, and amphetamine. We found little or no sex difference in these measures, suggesting sexual congruency in fundamental static and dynamic properties of dopamine neurons. Thus, dopamine related sex-differences are likely mediated by secondary mechanisms that flexibly influence the function of the dopamine cells and circuits. Finally, we noted that most behavioral sex differences had been reported in Sprague-Dawley rats and repeated some of the above measures in that strain. We found some sex differences in those animals highlighting the importance of considering strain differences in experimental design and result interpretation.

Sex differences in reward- and punishment-guided actions.

Differences in the prevalence and presentation of psychiatric illnesses in men and women suggest that neurobiological sex differences confer vulnerability or resilience in these disorders. Rodent behavioral models are critical for understanding the mechanisms of these differences. Reward processing and punishment avoidance are fundamental dimensions of the symptoms of psychiatric disorders. Here we explored sex differences along these dimensions using multiple and distinct behavioral paradigms. We found no sex difference in reward-guided associative learning but a faster punishment-avoidance learning in females. After learning, females were more sensitive than males to probabilistic punishment but less sensitive when punishment could be avoided with certainty. No sex differences were found in reward-guided cognitive flexibility. Thus, sex differences in goal-directed behaviors emerged selectively when there was an aversive context. These differences were critically sensitive to whether the punishment was certain or unpredictable. Our findings with these new paradigms provide conceptual and practical tools for investigating brain mechanisms that account for sex differences in susceptibility to anxiety and impulsivity. They may also provide insight for understanding the evolution of sex-specific optimal behavioral strategies in dynamic environments.

Androgen Regulation of the Mesocorticolimbic System and Executive Function.

Multiple lines of evidence indicate that androgens, such as testosterone, modulate the mesocorticolimbic system and executive function. This review integrates neuroanatomical, molecular biological, neurochemical, and behavioral studies to highlight how endogenous and exogenous androgens alter behaviors, such as behavioral flexibility, decision making, and risk taking. First, we briefly review the neuroanatomy of the mesocorticolimbic system, which mediates executive function, with a focus on the ventral tegmental area (VTA), nucleus accumbens (NAc), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC). Second, we present evidence that androgen receptors (AR) and other steroid receptors are expressed in the mesocorticolimbic system. Using sensitive immunohistochemistry and quantitative polymerase chain reaction (qPCR) techniques, ARs are detected in the VTA, NAc, mPFC, and OFC. Third, we describe recent evidence for local androgens ("neuroandrogens") in the mesocorticolimbic system. Steroidogenic enzymes are expressed in mesocorticolimbic regions. Furthermore, following long-term gonadectomy, testosterone is nondetectable in the blood but detectable in the mesocorticolimbic system, using liquid chromatography tandem mass spectrometry. However, the physiological relevance of neuroandrogens remains unknown. Fourth, we review how anabolic-androgenic steroids (AAS) influence the mesocorticolimbic system. Fifth, we describe how androgens modulate the neurochemistry and structure of the mesocorticolimbic system, particularly with regard to dopaminergic signaling. Finally, we discuss evidence that androgens influence executive functions, including the effects of androgen deprivation therapy and AAS. Taken together, the evidence indicates that androgens are critical modulators of executive function. Similar to dopamine signaling, there might be optimal levels of androgen signaling within the mesocorticolimbic system for executive functioning. Future studies should examine the regulation and functions of neurosteroids in the mesocorticolimbic system, as well as the potential deleterious and enduring effects of AAS use.

Anabolic-androgenic steroids (AAS) increase sensitivity to uncertainty by inhibition of dopamine D1 and D2 receptors.

BACKGROUND: Anabolic-androgenic steroid abuse is implicated in maladaptive behaviors such as impaired cognition in humans. In a rat model, our lab has shown that testosterone decreases preference for a large/uncertain reward in probability discounting. Other studies have shown that androgens decrease dopamine D1 and D2 receptors in the nucleus accumbens shell, a region important for decision-making behavior in probability discounting. Thus, we attempted to restore selection of the large/uncertain reward in testosterone-treated rats by administering the D2 receptor agonist quinpirole or the D1 receptor agonist SKF81297 and testing probability discounting. METHODS: Adolescent male Long-Evans rats were treated chronically with high-dose testosterone (7.5 mg/kg) or vehicle (13% cyclodextrin in water), and tested for probability discounting after injections of saline, 0.1 and 0.5 mg/kg of quinpirole or SKF81297. Rats chose between a small/certain reward (1 sugar pellet, 100% probability) and a large/uncertain reward (4 pellets, decreasing probability: 100, 75, 50, 25, 0%). RESULTS: Testosterone-treated rats selected the large/uncertain reward significantly less than vehicle-treated controls after saline injection. However, acute injection with 0.1 mg/kg quinpirole increased large/uncertain reward preference in testosterone-treated rats only, indicated by a testosterone × quinpirole interaction. At 0.5 mg/kg, quinpirole increased large/uncertain reward preference in all rats. Acute injection with SKF81297 at 0.1 or 0.5 mg/kg rescued large/uncertain reward preference in testosterone-treated rats by eliminating the difference between groups. CONCLUSIONS: It appears that altered probability discounting behavior in testosterone-treated rats is due to both decreased D1 and D2 receptor function.

Sex differences and hormonal modulation of ethanol-enhanced risk taking in rats.

BACKGROUND: Ethanol (EtOH) intake correlates with increased risk-taking, and sex differences exist in both EtOH use and risk-taking in humans and rats. However, the interaction of sex and gonadal hormones to affect risk-taking under the influence of EtOH has not been determined. This was the focus of the current study. METHODS: Adult Long-Evans rats (n=18 males and females) were gonadectomized and received hormone replacement at physiologic levels or blank implants (n=7-9/group). Risk-taking was assessed with probability discounting, requiring rats to choose between a small/certain reward and a large/uncertain reward delivered with decreasing probability throughout each daily session. Before testing, rats received saline or EtOH (0.5 or 1.0g/kg) ip. RESULTS: In males, EtOH increased preference for the large/uncertain reward lever (F2,28=10.462, p<0.05). However, there was no effect of EtOH on lever preference in females (F1,30=0.914, p>0.05). At baseline, ORCHX+T males showed a greater preference for the large/uncertain reward lever then ORCHX males (F1,14=13.805, p<0.05). In females only, EtOH decreased choice latency relative to baseline (F1,10=7.25, p<0.05). EtOH decreased loss sensitivity in both sexes, with all rats exhibiting decreased lose-shift ratios (males: F2,18=5.10, p<0.05; females F2,10=4.37, p<0.05). CONCLUSIONS: These results show that EtOH, sex, and hormones interact to influence decision making. EtOH increases risk taking in males, but not in females. However, EtOH selectively decreases choice latency in females, and decreases loss sensitivity in both sexes. These findings are relevant to understanding human behavior, particularly in adolescents who experience increased hormone levels and often drink EtOH and engage in risky behavior.