Sex-dependent effects of chronic intermittent voluntary alcohol consumption on attentional, not motivational, measures during probabilistic learning and reversal.

BACKGROUND: Forced alcohol (ethanol, EtOH) exposure has been shown to cause significant impairments on reversal learning, a widely-used assay of cognitive flexibility, specifically on fully-predictive, deterministic versions of this task. However, previous studies have not adequately considered voluntary EtOH consumption and sex effects on probabilistic reversal learning. The present study aimed to fill this gap in the literature. METHODS: Male and female Long-Evans rats underwent either 10 weeks of voluntary intermittent 20% EtOH access or water only (H2O) access. Rats were then pretrained to initiate trials and learn stimulus-reward associations via touchscreen response, and subsequently required to select between two visual stimuli, rewarded with probability 0.70 or 0.30. In the final phase, reinforcement contingencies were reversed. RESULTS: We found significant sex differences on several EtOH-drinking variables, with females reaching a higher maximum EtOH consumption, exhibiting more high-drinking days, and escalating their EtOH at a quicker rate compared to males. During early abstinence, EtOH drinkers (and particularly EtOH-drinking females) made more initiation omissions and were slower to initiate trials than H2O drinking controls, especially during pretraining. A similar pattern in trial initiations was also observed in discrimination, but not in reversal learning. EtOH drinking rats were unaffected in their reward collection and stimulus response times, indicating intact motivation and motor responding. Although there were sex differences in discrimination and reversal phases, performance improved over time. We also observed sex-independent drinking group differences in win-stay and lose-shift strategies specific to the reversal phase. CONCLUSIONS: Females exhibit increased vulnerability to EtOH effects in early learning: there were sex-dependent EtOH effects on attentional measures during pretraining and discrimination phases. We also found sex-independent EtOH effects on exploration strategies during reversal. Future studies should aim to uncover the neural mechanisms for changes in attention and exploration in both acute and prolonged EtOH withdrawal.

Can the VTA Come Out to Play? Only When the mPFC's Predictions Go Astray!

Confidence in perceptual decisions scales neural responses to violations in reward expectation. In this issue of Neuron, Lak et al. (2020) show that the medial prefrontal cortex in mice computes a confidence-dependent expectation signal that influences how dopamine neurons convey reward prediction errors to guide learning.

Rodent Models of Adaptive Value Learning and Decision-Making.

Real-world decisions are rarely as straightforward as choosing between clearly "good" vs. "bad" options. More often, options must be evaluated carefully because they differ in relative value. For example, we typically learn about (and make decisions between) options in comparison, where one outcome may be more costly or risky than the other. Several neuropsychiatric conditions are characterized by atypical evaluation of effort and risk costs, including major depression, schizophrenia, autism, obsessive-compulsive disorder, and substance use disorders. Aberrant value learning and decision-making have long been considered a cognitive-behavioral endophenotype of these disorders and can be modeled in rodents. This chapter presents two general methodological domains that the experimenter can manipulate in animal decision-making tasks: risk and effort. Here, we present detailed methods of rodent tasks frequently employed within these domains: probabilistic reversal learning (PRL) and effort choice. These tasks recruit regions within rodent frontal cortex, the amygdala, and the striatum, and performance is heavily modulated by dopamine, making these assays highly valid measures in the study of behavioral and substance addictions, in particular.

Solving the Credit Assignment Problem With the Prefrontal Cortex.

In naturalistic multi-cue and multi-step learning tasks, where outcomes of behavior are delayed in time, discovering which choices are responsible for rewards can present a challenge, known as the credit assignment problem. In this review, I summarize recent work that highlighted a critical role for the prefrontal cortex (PFC) in assigning credit where it is due in tasks where only a few of the multitude of cues or choices are relevant to the final outcome of behavior. Collectively, these investigations have provided compelling support for specialized roles of the orbitofrontal (OFC), anterior cingulate (ACC), and dorsolateral prefrontal (dlPFC) cortices in contingent learning. However, recent work has similarly revealed shared contributions and emphasized rich and heterogeneous response properties of neurons in these brain regions. Such functional overlap is not surprising given the complexity of reciprocal projections spanning the PFC. In the concluding section, I overview the evidence suggesting that the OFC, ACC and dlPFC communicate extensively, sharing the information about presented options, executed decisions and received rewards, which enables them to assign credit for outcomes to choices on which they are contingent. This account suggests that lesion or inactivation/inhibition experiments targeting a localized PFC subregion will be insufficient to gain a fine-grained understanding of credit assignment during learning and instead poses refined questions for future research, shifting the focus from focal manipulations to experimental techniques targeting cortico-cortical projections.

Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty.

We make choices based on the values of expected outcomes, informed by previous experience in similar settings. When the outcomes of our decisions consistently violate expectations, new learning is needed to maximize rewards. Yet not every surprising event indicates a meaningful change in the environment. Even when conditions are stable overall, outcomes of a single experience can still be unpredictable due to small fluctuations (i.e., expected uncertainty) in reward or costs. In the present work, we investigate causal contributions of the basolateral amygdala (BLA) and orbitofrontal cortex (OFC) in rats to learning under expected outcome uncertainty in a novel delay-based task that incorporates both predictable fluctuations and directional shifts in outcome values. We demonstrate that OFC is required to accurately represent the distribution of wait times to stabilize choice preferences despite trial-by-trial fluctuations in outcomes, whereas BLA is necessary for the facilitation of learning in response to surprising events.

Steep effort discounting of a preferred reward over a freely-available option in prolonged methamphetamine withdrawal in male rats.

RATIONALE: Drug addiction can be described as aberrant allocation of effort toward acquiring drug, despite associated costs. It is unclear if this behavioral pattern results from an overvaluation of reward or to an altered sensitivity to costs. OBJECTIVE: Present experiments assessed reward sensitivity and effortful choice in rats following 1 week of withdrawal from methamphetamine (mAMPH). METHODS: Rats were treated with either saline or an escalating dose mAMPH regimen, then tested after a week without the drug. In experiment 1, rats were given a free choice between water and various concentrations of sucrose solution to assess general reward sensitivity. In experiment 2, rats were presented with a choice between lever-pressing for sucrose pellets on a progressive ratio schedule or consuming freely-available chow. RESULTS: In experiment 1, we found no differences in sucrose preference between mAMPH- and saline-pretreated rats. In experiment 2, when selecting between two options, mAMPH-pretreated rats engaged in less lever-pressing for sucrose pellets (p < 0.01) and switched from this preferred reward to the chow sooner than saline-pretreated rats (p < 0.05). This effect was not consistent with general reward devaluation or loss of motivation. CONCLUSIONS: These findings demonstrate that mAMPH exposure and withdrawal lead to steeper discounting of reward value by effort, an effect that is consistent with the effect of mAMPH on discounting by delay, and which may reflect an underlying shared mechanism.

Rigid patterns of effortful choice behavior after acute stress in rats.

Physical effort is a common cost of acquiring rewards, and decreased effort is a feature of many neuropsychiatric disorders. Stress affects performance on several tests of cognition and decision making in both humans and nonhumans. Only a few recent reports show impairing effects of stress in operant tasks involving effort and cognitive flexibility. Brain regions affected by stress, such as the medial prefrontal cortex and amygdala, are also implicated in mediating effortful choices. Here, we assessed effort-based decision making after an acute stress procedure known to induce persistent impairment in shuttle escape and elevated plasma corticosterone. In these animals, we also probed levels of polysialyted neural cell adhesion molecule (PSA-NCAM), a marker of structural plasticity, in medial frontal cortex and amygdala. We found that animals that consistently worked for high magnitude rewards continued to do so, even after acute shock stress. We also found that PSA-NCAM was increased in both regions after effortful choice experience but not after shock stress alone. These findings are discussed with reference to the existing broad literature on cognitive effects of stress and in the context of how acute stress may bias effortful decisions to a rigid pattern of responding.

Sex differences, learning flexibility, and striatal dopamine D1 and D2 following adolescent drug exposure in rats.

Corticostriatal circuitry supports flexible reward learning and emotional behavior from the critical neurodevelopmental stage of adolescence through adulthood. It is still poorly understood how prescription drug exposure in adolescence may impact these outcomes in the long-term. We studied adolescent methylphenidate (MPH) and fluoxetine (FLX) exposure in rats and their impact on learning and emotion in adulthood. In Experiment 1, male and female rats were administered MPH, FLX, or saline (SAL), and compared with methamphetamine (mAMPH) treatment beginning in postnatal day (PND) 37. The rats were then tested on discrimination and reversal learning in adulthood. In Experiment 2, animals were administered MPH or SAL also beginning in PND 37 and later tested in adulthood for anxiety levels. In Experiment 3, we analyzed striatal dopamine D1 and D2 receptor expression in adulthood following either extensive learning (after Experiment 1) or more brief emotional measures (after Experiment 2). We found sex differences in discrimination learning and attenuated reversal learning after MPH and only sex differences in adulthood anxiety. In learners, there was enhanced striatal D1, but not D2, after either adolescent MPH or mAMPH. Lastly, also in learners, there was a sex x treatment group interaction for D2, but not D1, driven by the MPH-pretreated females, who expressed significantly higher D2 levels compared to SAL. These results show enduring effects of adolescent MPH on reversal learning in rats. Developmental psychostimulant exposure may interact with learning to enhance D1 expression in adulthood, and affect D2 expression in a sex-dependent manner.

Methamphetamine blocks exercise effects on Bdnf and Drd2 gene expression in frontal cortex and striatum.

Exposure to drugs of abuse can produce many neurobiological changes which may lead to increased valuation of rewards and decreased sensitivity to their costs. Many of these behavioral alterations are associated with activity of D2-expressing medium spiny neurons in the striatum. Additionally, Bdnf in the striatum has been shown to play a role in flexible reward-seeking behavior. Given that voluntary aerobic exercise can affect the expression of these proteins in healthy subjects, and that exercise has shown promise as an anti-addictive therapy, we set out to quantify changes in D2 and Bdnf expression in methamphetamine-exposed rats given access to running wheels. Sixty-four rats were treated for two weeks with an escalating dose of methamphetamine or saline, then either sacrificed, housed in standard cages, or given free access to a running wheel for 6 weeks prior to sacrifice. Rats treated with methamphetamine ran significantly greater distances than saline-treated rats, suggesting an augmentation in the reinforcement value of voluntary wheel running. Transcription of Drd2 and Bdnf was assessed via RT-qPCR. Protein expression levels of D2 and phosphorylation of the TrkB receptor were measured via western blot. Drd2 and Bdnf mRNA levels were impacted independently by exercise and methamphetamine, but exposure to methamphetamine prior to the initiation of exercise blocked the exercise-induced changes seen in rats treated with saline. Expression levels of both proteins were elevated immediately after methamphetamine, but returned to baseline after six weeks, regardless of exercise status.

Post-training depletions of basolateral amygdala serotonin fail to disrupt discrimination, retention, or reversal learning.

In goal-directed pursuits, the basolateral amygdala (BLA) is critical in learning about changes in the value of rewards. BLA-lesioned rats show enhanced reversal learning, a task employed to measure the flexibility of response to changes in reward. Similarly, there is a trend for enhanced discrimination learning, suggesting that BLA may modulate formation of stimulus-reward associations. There is a parallel literature on the importance of serotonin (5HT) in new stimulus-reward and reversal learning. Recent postulations implicate 5HT in learning from punishment. Whereas, dopaminergic involvement is critical in behavioral activation and reinforcement, 5HT may be most critical for aversive processing and behavioral inhibition, complementary cognitive processes. Given these findings, a 5HT-mediated mechanism in BLA may mediate the facilitated learning observed previously. The present study investigated the effects of selective 5HT lesions in BLA using 5,7-dihydroxytryptamine (5,7-DHT) vs. infusions of saline (Sham) on discrimination, retention, and deterministic reversal learning. Rats were required to reach an 85% correct pairwise discrimination and single reversal criterion prior to surgery. Postoperatively, rats were then tested on the (1) retention of the pretreatment discrimination pair, (2) discrimination of a novel pair, and (3) reversal learning performance. We found statistically comparable preoperative learning rates between groups, intact postoperative retention, and unaltered novel discrimination and reversal learning in 5,7-DHT rats. These findings suggest that 5HT in BLA is not required for formation and flexible adjustment of new stimulus-reward associations when the strategy to efficiently solve the task has already been learned. Given the complementary role of orbitofrontal cortex in reward learning and its interconnectivity with BLA, these findings add to the list of dissociable mechanisms for BLA and orbitofrontal cortex in reward learning.

Distinct patterns of outcome valuation and amygdala-prefrontal cortex synaptic remodeling in adolescence and adulthood.

Adolescent behavior is typified by increased risk-taking, reward- and novelty-seeking, as well as an augmented need for social and environmental stimulation. This behavioral phenotype may result from alterations in outcome valuation or reward learning. In the present set of experiments, we directly compared adult and adolescent animals on tasks measuring both of these processes. Additionally, we examined developmental differences in dopamine D1-like receptor (D1R), dopamine D2-like receptor (D2R), and polysialylated neural cell adhesion molecule (PSA-NCAM) expression in animals that were trained on an effortful reward valuation task, given that these proteins play an important role in the functional development of the amygdala-prefrontocortical (PFC) circuit and mesocorticolimbic dopamine system. We found that adolescent animals were not different from adults in appetitive associative learning, but exhibited distinct pattern of responses to differences in outcome values, which was paralleled by an enhanced motivation to invest effort to obtain larger rewards. There were no differences in D2 receptor expression, but D1 receptor expression was significantly reduced in the striatum of animals that had experiences with reward learning during adolescence compared to animals that went through the same experiences in adulthood. We observed increased levels of PSA-NCAM expression in both PFC and amygdala of late adolescents compared to adults that were previously trained on an effortful reward valuation task. PSA-NCAM levels in PFC were strongly and positively associated with high effort/reward (HER) choices in adolescents, but not in adult animals. Increased levels of PSA-NCAM expression in adolescents may index increased structural plasticity and represent a neural correlate of a reward sensitive endophenotype.

Reductions in frontocortical cytokine levels are associated with long-lasting alterations in reward valuation after methamphetamine.

Alterations in reward valuation are thought to have a central role at all stages of the addiction process. We previously reported work aversion in an effortful T-maze task following a binge exposure to methamphetamine, and no such changes in effort following escalating doses. Limitations of the T-maze task include its two available options, with an effort requirement, in the form of increasing barrier height, varying incrementally as a function of time, and reward magnitudes held constant. Reward preferences and choices, however, are likely affected by the number of options available and the manner in which alternatives are presented. In the present experiment, we investigated the long-lasting, off-drug effects of methamphetamine on reward choices in a novel effortful maze task with three possible courses of action, each associated with different effort requirements and reward magnitudes. Neuroinflammatory responses associated with drug exposure, proposed as one of the mechanisms contributing to suboptimal choices on effort-based tasks, were also examined. We investigated region-specific changes in pro- and anti-inflammatory markers in the mesocorticolimbic pathway after methamphetamine, and their relationship with animals' reward choices. We observed long-lasting, increased sensitivity to differences in reward magnitude in the methamphetamine group: animals were more likely to overcome greater effort costs to obtain larger rewards on our novel effortful maze task. These behavioral changes were strongly predicted by pronounced decreases in frontocortical cytokines, but not amygdalar or striatal markers. The present results provide the first evidence that neuroinflammatory processes are associated with alterations in reward valuation during protracted drug withdrawal.

Positive and negative feedback learning and associated dopamine and serotonin transporter binding after methamphetamine.

Learning from mistakes and prospectively adjusting behavior in response to reward feedback is an important facet of performance monitoring. Dopamine (DA) pathways play an important role in feedback learning and a growing literature has also emerged on the importance of serotonin (5HT) in reward learning, particularly during punishment or reward omission (negative feedback). Cognitive impairments resulting from psychostimulant exposure may arise from altered patterns in feedback learning, which in turn may be modulated by DA and 5HT transmission. We analyzed long-term, off-drug changes in learning from positive and negative feedback and associated striatal DA transporter (DAT) and frontocortical 5HT transporter (SERT) binding in rats pretreated with methamphetamine (mAMPH). Specifically, we assessed the reversal phase of pairwise visual discrimination learning in rats receiving single dose- (mAMPHsingle) vs. escalating-dose exposure (mAMPHescal). Using fine-grained trial-by-trial analyses, we found increased sensitivity to and reliance on positive feedback in mAMPH-pretreated animals, with the mAMPHsingle group showing more pronounced use of this type of feedback. In contrast, overall negative feedback sensitivity was not altered following any mAMPH treatment. In addition to validating the enduring effects of mAMPH on early reversal learning, we found more consecutive error commissions before the first correct response in mAMPH-pretreated rats. This behavioral rigidity was negatively correlated with subregional frontocortical SERT whereas positive feedback sensitivity negatively correlated with striatal DAT binding. These results provide new evidence for the overlapping, yet dissociable roles of DA and 5HT systems in overcoming perseveration and in learning new reward rules.