Cooperative synaptic and intrinsic plasticity in a disynaptic limbic circuit drive stress-induced anhedonia and passive coping in mice.

Stress promotes negative affective states, which include anhedonia and passive coping. While these features are in part mediated by neuroadaptations in brain reward circuitry, a comprehensive framework of how stress-induced negative affect may be encoded within key nodes of this circuit is lacking. Here, we show in a mouse model for stress-induced anhedonia and passive coping that these phenomena are associated with increased synaptic strength of ventral hippocampus (VH) excitatory synapses onto D1 medium spiny neurons (D1-MSNs) in the nucleus accumbens medial shell (NAcmSh), and with lateral hypothalamus (LH)-projecting D1-MSN hyperexcitability mediated by decreased inwardly rectifying potassium channel (IRK) function. Stress-induced negative affective states are prevented by depotentiation of VH to NAcmSh synapses, restoring Kir2.1 function in D1R-MSNs, or disrupting co-participation of these synaptic and intrinsic adaptations in D1-MSNs. In conclusion, our data provide strong evidence for a disynaptic pathway controlling maladaptive emotional behavior.

Neural Estimates of Imagined Outcomes in Basolateral Amygdala Depend on Orbitofrontal Cortex.

Reciprocal connections between the orbitofrontal cortex (OFC) and the basolateral nucleus of the amygdala (BLA) provide a critical circuit for guiding normal behavior when information about expected outcomes is required. Recently, we reported that outcome signaling by OFC neurons is also necessary for learning in the face of unexpected outcomes during a Pavlovian over-expectation task. Key to learning in this task is the ability to build on prior learning to infer or estimate an amount of reward never previously received. OFC was critical to this process. Notably, in parallel work, we found that BLA was not necessary for learning in this setting. This suggested a dissociation in which the BLA might be critical for acquiring information about the outcomes but not for subsequently using it to make novel predictions. Here we evaluated this hypothesis by recording single-unit activity from BLA in rats during the same Pavlovian over-expectation task used previously. We found that spiking activity recorded in BLA in control rats did reflect novel outcome estimates derived from the integration of prior learning, however consistent with a model in which this process occurs in the OFC, these correlates were entirely abolished by ipsilateral OFC lesions. These data indicate that this information about these novel predictions is represented in the BLA, supported via direct or indirect input from the OFC, even though it does not appear to be necessary for learning. SIGNIFICANCE STATEMENT: The basolateral nucleus of the amygdala (BLA) and the orbitofrontal cortex (OFC) are involved in behavior that depends on knowledge of impending outcomes. Recently, we found that only the OFC was necessary for using such information for learning in a Pavlovian over-expectation task. The current experiment was designed to search for neural correlates of this process in the BLA and, if present, to ask whether they would still be dependent on OFC input. We found that although spiking activity in BLA in control rats did reflect the novel outcome estimates underlying learning, these correlates were entirely abolished by OFC lesions.

Ketamine rescues anhedonia by cell-type and input specific adaptations in the Nucleus Accumbens.

Ketamine's role in providing a rapid and sustained antidepressant response, particularly for patients unresponsive to conventional treatments, is increasingly recognized. A core symptom of depression, anhedonia, or the loss of enjoyment or interest in previously pleasurable activities, is known to be significantly alleviated by ketamine. While several hypotheses have been proposed regarding the mechanisms by which ketamine alleviates anhedonia, the specific circuits and synaptic changes responsible for its sustained therapeutic effects are not yet understood. Here, we show that the nucleus accumbens (NAc), a major hub of the reward circuitry, is essential for ketamine's effect in rescuing anhedonia in mice subjected to chronic stress, a critical risk factor in the genesis of depression in humans. Specifically, a single exposure to ketamine rescues stress-induced decreased strength of excitatory synapses on NAc D1 dopamine receptor-expressing medium spiny neurons (D1-MSNs). By using a novel cell-specific pharmacology method, we demonstrate that this cell-type specific neuroadaptation is necessary for the sustained therapeutic effects of ketamine. To test for causal sufficiency, we artificially mimicked ketamine-induced increase in excitatory strength on D1-MSNs and found that this recapitulates the behavioral amelioration induced by ketamine. Finally, to determine the presynaptic origin of the relevant glutamatergic inputs for ketamine-elicited synaptic and behavioral effects, we used a combination of opto- and chemogenetics. We found that ketamine rescues stress-induced reduction in excitatory strength at medial prefrontal cortex and ventral hippocampus inputs to NAc D1-MSNs. Chemogenetically preventing ketamine-evoked plasticity at those unique inputs to the NAc reveals a ketamine-operated input-specific control of hedonic behavior. These results establish that ketamine rescues stress-induced anhedonia via cell-type-specific adaptations as well as information integration in the NAc via discrete excitatory synapses.

Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning.

In many cases, learning is thought to be driven by differences between the value of rewards we expect and rewards we actually receive. Yet learning can also occur when the identity of the reward we receive is not as expected, even if its value remains unchanged. Learning from changes in reward identity implies access to an internal model of the environment, from which information about the identity of the expected reward can be derived. As a result, such learning is not easily accounted for by model-free reinforcement learning theories such as temporal difference reinforcement learning (TDRL), which predicate learning on changes in reward value, but not identity. Here, we used unblocking procedures to assess learning driven by value- versus identity-based prediction errors. Rats were trained to associate distinct visual cues with different food quantities and identities. These cues were subsequently presented in compound with novel auditory cues and the reward quantity or identity was selectively changed. Unblocking was assessed by presenting the auditory cues alone in a probe test. Consistent with neural implementations of TDRL models, we found that the ventral striatum was necessary for learning in response to changes in reward value. However, this area, along with orbitofrontal cortex, was also required for learning driven by changes in reward identity. This observation requires that existing models of TDRL in the ventral striatum be modified to include information about the specific features of expected outcomes derived from model-based representations, and that the role of orbitofrontal cortex in these models be clearly delineated.

The role of the nucleus accumbens in knowing when to respond.

While knowing what to expect is important, it is equally important to know when to expect it and to respond accordingly. This is apparent even in simple Pavlovian training situations in which animals learn to respond more strongly closer to reward delivery. Here we report that the nucleus accumbens core, an area well-positioned to represent information about the timing of impending rewards, plays a critical role in this timing function.

Aversive stimuli bias corticothalamic responses to motivationally significant cues.

Making predictions about future rewards or punishments is fundamental to adaptive behavior. These processes are influenced by prior experience. For example, prior exposure to aversive stimuli or stressors changes behavioral responses to negative- and positive-value predictive cues. Here, we demonstrate a role for medial prefrontal cortex (mPFC) neurons projecting to the paraventricular nucleus of the thalamus (PVT; mPFC→PVT) in this process. We found that a history of aversive stimuli negatively biased behavioral responses to motivationally relevant cues in mice and that this negative bias was associated with hyperactivity in mPFC→PVT neurons during exposure to those cues. Furthermore, artificially mimicking this hyperactive response with selective optogenetic excitation of the same pathway recapitulated the negative behavioral bias induced by aversive stimuli, whereas optogenetic inactivation of mPFC→PVT neurons prevented the development of the negative bias. Together, our results highlight how information flow within the mPFC→PVT circuit is critical for making predictions about motivationally-relevant outcomes as a function of prior experience.

Opposite environmental regulation of heroin and amphetamine self-administration in the rat.

RATIONALE: The circumstances of drug taking are thought to play a role in drug abuse but the evidence of it is anecdotal. Previous studies have shown that the intravenous self-administration of cocaine is facilitated in rats non-residing in the test chambers relative to rats that live in the test chambers at all times. We investigated here whether environmental context could exert its modulatory influence on heroin and amphetamine self-administration as well. MATERIALS AND METHODS: Independent groups of rats were given the possibility to self-administer different doses of heroin or amphetamine (12.5, 25.0, or 50.0 microg/kg). Some animals were housed in the self-administration chambers (resident groups) whereas other rats were transported to the self-administration chambers only for the test sessions (non-resident groups). RESULTS: Amphetamine-reinforcing effects were more pronounced in non-resident rats than in resident rats, as previously reported for cocaine. Quite unexpectedly, the opposite was found for heroin. Because of this surprising dissociation, some of the rats trained to self-administer amphetamine were later given the opportunity to self-administer heroin. Also in this case, resident rats took more heroin than non-resident rats. CONCLUSIONS: These findings suggest an unforeseen dissociation between opioid and psychostimulant reward and demonstrate that even in the laboratory rat some contexts are associated with the propensity to self-administer more opioid than psychostimulant drugs and vice versa, thus indicating that drug taking is influenced not only by economical or cultural factors but also can be modulated at a much more basic level by the setting in which drugs are experienced.

Thinking Outside the Box: Orbitofrontal Cortex, Imagination, and How We Can Treat Addiction.

Addiction involves an inability to control drug-seeking behavior. While this may be thought of as secondary to an overwhelming desire for drugs, it could equally well reflect a failure of the brain mechanisms that allow addicts to learn about and mentally simulate non-drug consequences. Importantly, this process of mental simulation draws upon, but is not normally bound by, our past experiences. Rather we have the ability to think outside the box of our past, integrating knowledge gained from a variety of similar and not-so-similar life experiences to derive estimates or imagine what might happen next. These estimates influence our current behavior directly and also affect future behavior by serving as the background against which outcomes are evaluated to support learning. Here we will review evidence, from our own work using a Pavlovian over-expectation task as well as from other sources, that the orbitofrontal cortex is a critical node in the neural circuit that generates these estimates. Further we will offer the specific hypothesis that degradation of this function secondary to drug-induced changes is a critical and likely addressable part of addiction.

Effects of prior cocaine versus morphine or heroin self-administration on extinction learning driven by overexpectation versus omission of reward.

BACKGROUND: Addiction is characterized by an inability to stop using drugs, despite adverse consequences. One contributing factor to this compulsive drug taking could be the impact of drug use on the ability to extinguish drug seeking after changes in expected outcomes. Here, we compared effects of cocaine, morphine, and heroin self-administration on two forms of extinction learning: standard extinction driven by reward omission and extinction driven by reward overexpectation. METHODS: In experiment 1, we trained rats to self-administer cocaine, morphine, or sucrose for 3 hours per day (limited access). In experiment 2, we trained rats to self-administer heroin or sucrose for 12 hours per day (extended access). Three weeks later, we trained the rats to associate several cues with palatable food reward, after which we assessed extinction of the learned Pavlovian response, first by pairing two cues together in the overexpectation procedure and later by omitting the food reward. RESULTS: Rats trained under limited access conditions to self-administer sucrose or morphine demonstrated normal extinction in response to both overexpectation and reward omission, whereas cocaine-experienced rats or rats trained to self-administer heroin under extended access conditions exhibited normal extinction in response to reward omission but failed to show extinction in response to overexpectation. CONCLUSIONS: Here we show that cocaine and heroin can induce long-lasting deficits in the ability to extinguish reward seeking. These deficits were not observed in a standard extinction procedure but instead only affected extinction learning driven by a more complex phenomenon of overexpectation.

Effect of the Novel Positive Allosteric Modulator of Metabotropic Glutamate Receptor 2 AZD8529 on Incubation of Methamphetamine Craving After Prolonged Voluntary Abstinence in a Rat Model.

BACKGROUND: Cue-induced methamphetamine craving increases after prolonged forced (experimenter-imposed) abstinence from the drug (incubation of methamphetamine craving). Here, we determined whether this incubation phenomenon would occur under conditions that promote voluntary (self-imposed) abstinence. We also determined the effect of the novel metabotropic glutamate receptor 2 positive allosteric modulator, AZD8529, on incubation of methamphetamine craving after forced or voluntary abstinence. METHODS: We trained rats to self-administer palatable food (6 sessions) and then to self-administer methamphetamine under two conditions: 12 sessions (9 hours/day) or 50 sessions (3 hours/day). We then assessed cue-induced methamphetamine seeking in extinction tests after 1 or 21 abstinence days. Between tests, the rats underwent either forced abstinence (no access to the food- or drug-paired levers) or voluntary abstinence (achieved via a discrete choice procedure between methamphetamine and palatable food; 20 trials per day) for 19 days. We also determined the effect of subcutaneous injections of AZD8529 (20 and 40 mg/kg) on cue-induced methamphetamine seeking 1 day or 21 days after forced or voluntary abstinence. RESULTS: Under both training and abstinence conditions, cue-induced methamphetamine seeking in the extinction tests was higher after 21 abstinence days than after 1 day (incubation of methamphetamine craving). AZD8529 decreased cue-induced methamphetamine seeking on day 21 but not day 1 of forced or voluntary abstinence. CONCLUSIONS: We introduce a novel animal model to study incubation of drug craving and cue-induced drug seeking after prolonged voluntary abstinence, mimicking the human condition of relapse after successful contingency management treatment. Our data suggest that positive allosteric modulators of metabotropic glutamate receptor 2 should be considered for relapse prevention.

Transition from 'model-based' to 'model-free' behavioral control in addiction: Involvement of the orbitofrontal cortex and dorsolateral striatum.

Cocaine addiction is a complex and multidimensional process involving a number of behavioral and neural forms of plasticity. The behavioral transition from voluntary drug use to compulsive drug taking may be explained at the neural level by drug-induced changes in function or interaction between a flexible planning system, associated with prefrontal cortical regions, and a rigid habit system, associated with the striatum. The dichotomy between these two systems is operationalized in computational theory by positing model-based and model-free learning mechanisms, the former relying on an "internal model" of the environment and the latter on pre-computed or cached values to control behavior. In this review, we will suggest that model-free and model-based learning mechanisms appear to be differentially affected, at least in the case of psychostimulants such as cocaine, with the former being enhanced while the latter are disrupted. As a result, the behavior of long-term drug users becomes less flexible and responsive to the desirability of expected outcomes and more habitual, based on the long history of reinforcement. To support our specific proposal, we will review recent neural and behavioral evidence on the effect of psychostimulant exposure on orbitofrontal and dorsolateral striatum structure and function. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.

Orbitofrontal activation restores insight lost after cocaine use.

Addiction is characterized by a lack of insight into the likely outcomes of one's behavior. Insight, or the ability to imagine outcomes, is evident when outcomes have not been directly experienced. Using this concept, work in both rats and humans has recently identified neural correlates of insight in the medial and orbital prefrontal cortices. We found that these correlates were selectively abolished in rats by cocaine self-administration. Their abolition was associated with behavioral deficits and reduced synaptic efficacy in orbitofrontal cortex, the reversal of which by optogenetic activation restored normal behavior. These results provide a link between cocaine use and problems with insight. Deficits in these functions are likely to be particularly important for problems such as drug relapse, in which behavior fails to account for likely adverse outcomes. As such, our data provide a neural target for therapeutic approaches to address these defining long-term effects of drug use.

Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning.

Imagination, defined as the ability to interpret reality in ways that diverge from past experience, is fundamental to adaptive behavior. This can be seen at a simple level in our capacity to predict novel outcomes in new situations. The ability to anticipate outcomes never before received can also influence learning if those imagined outcomes are not received. The orbitofrontal cortex is a key candidate for where the process of imagining likely outcomes occurs; however, its precise role in generating these estimates and applying them to learning remain open questions. Here we address these questions by showing that single-unit activity in the orbitofrontal cortex reflects novel outcome estimates. The strength of these neural correlates predicted both behavior and learning, learning that was abolished by temporally specific inhibition of orbitofrontal neurons. These results are consistent with the proposal that the orbitofrontal cortex is critical for integrating information to imagine future outcomes.

The impact of orbitofrontal dysfunction on cocaine addiction.

Cocaine addiction is characterized by poor judgment and maladaptive decision-making. Here we review evidence implicating the orbitofrontal cortex in such behavior. This evidence suggests that cocaine-induced changes in orbitofrontal cortex disrupt the representation of states and transition functions that form the basis of flexible and adaptive 'model-based' behavioral control. By impairing this function, cocaine exposure leads to an overemphasis on less flexible, maladaptive 'model-free' control systems. We propose that such an effect accounts for the complex pattern of maladaptive behaviors associated with cocaine addiction.

Ambience and drug choice: cocaine- and heroin-taking as a function of environmental context in humans and rats.

BACKGROUND: We have recently observed an unforeseen dissociation in the effect of environmental context on heroin versus cocaine self-administration in rats. Rats housed in the self-administration chambers (Residents) took more heroin than rats that were transferred to the self-administration chambers only for the test sessions (Nonresidents). The contrary was found for cocaine. The twofold aim of the present study was to investigate: 1) drug choice as a function of ambience in rats given access to both cocaine and heroin, and 2) ambience of choice for cocaine- versus heroin-taking in human addicts. METHODS: Resident and Nonresident rats with double-lumen intrajugular catheters were trained to self-administer cocaine (400 microg/kg/infusion) and heroin (25 microg/kg/infusion) on alternate days and then given the opportunity to choose between the two drugs during seven daily sessions. In the human study, we asked heroin and cocaine abusers where they preferred to take these drugs. RESULTS: Approximately 46.7% of Resident rats exhibited a preference for heroin over cocaine; 33.3% preferred cocaine, and 20% expressed no preference. In contrast, only 8.3% of Nonresident rats preferred heroin, whereas 66.7% preferred cocaine, and 25% expressed no preference. In the human study, 73% of co-abusers reported that they used heroin exclusively or mostly at home (22% used it outside the home), whereas only 25% reported using cocaine at home (67% took it outside their homes). CONCLUSIONS: Environmental context plays an important role in drug choice in both humans and rats self-administering heroin and cocaine.