Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related.

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Positive modulation of N-methyl-D-aspartate receptors in the mPFC reduces the spontaneous recovery of fear.

N-methyl-D-aspartate receptor (NMDAR) modulators have recently received increased attention as potential therapeutics for posttraumatic stress disorder (PTSD). Here, we tested a novel NMDAR-positive modulator, NYX-783, in the following two rodent models of PTSD: an auditory fear-conditioning model and a single-prolonged stress (SPS) model. We examined the ability of NYX-783 to reduce subsequent fear-based behaviors by measuring enhanced fear extinction and reduced spontaneous recovery (spontaneous return of fear) in male mice. NYX-783 administration significantly reduced spontaneous recovery in both PTSD models and enhanced fear extinction in the SPS model. Furthermore, NYX-783 increased the NMDA-induced inward currents of excitatory and inhibitory neurons in the infralimbic medial prefrontal cortex (IL mPFC) and that the GluN2B subunit of NMDARs on pyramidal neurons in the IL mPFC is required for its effect on spontaneous recovery. The downstream expression of brain-derived neurotrophic factor was required for NYX-783 to achieve its behavioral effect. These results elucidate the cellular targets of NYX-783 and the molecular mechanisms underlying the inhibition of spontaneous recovery. These preclinical findings support the hypothesis that NYX-783 may have therapeutic potential for PTSD treatment and may be particularly useful for inhibiting spontaneous recovery.

The effects of fatty acid amide hydrolase inhibition and monoacylglycerol lipase inhibition on habit formation in mice.

Emerging data indicate that endocannabinoid signaling is critical to the formation of habitual behavior. Previous work demonstrated that antagonism of cannabinoid receptor type 1 (CB1R) with AM251 during operant training impairs habit formation, but it is not known if this behavioral effect is specific to disrupted signaling of the endocannabinoid ligands anandamide or 2-arachidonoyl glycerol (2-AG). Here, we used selective pharmacological compounds during operant training to determine the impact of fatty acid amide hydrolase (FAAH) inhibition to increase anandamide (and other n-acylethanolamines) or monoacylglycerol lipase (MAGL) inhibition to increase 2-AG levels on the formation of habitual behaviors in mice using a food-reinforced contingency degradation procedure. We found, contrary to our hypothesis, that inhibition of FAAH and of MAGL disrupted the formation of habits. Next, AM251 was administered during training to verify that impaired habit formation could be assessed using contingency degradation. AM251-exposed mice responded at lower rates during training and at higher rates in the test. To understand the inconsistency with published data, we performed a proof-of-principle dose-response experiment to compare AM251 in our vehicle-solution to the published vehicle-suspension on response rates. We found consistent reductions in response rate with increasing doses of AM251 in solution and an inconsistent dose-response relationship with AM251 in suspension. Together, our data suggest that further characterization of the role of CB1R signaling in the formation of habitual responding is warranted and that augmenting endocannabinoids may have clinical utility for prophylactically preventing aberrant habit formation such as that hypothesized to occur in substance use disorders.

Reinforcement Learning during Adolescence in Rats.

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.

Medial Nucleus Accumbens Projections to the Ventral Tegmental Area Control Food Consumption.

Decades of research have shown that the NAc is a critical region influencing addiction, mood, and food consumption through its effects on reinforcement learning, motivation, and hedonic experience. Pharmacological studies have demonstrated that inhibition of the NAc shell induces voracious feeding, leading to the hypothesis that the inhibitory projections that emerge from the NAc normally act to restrict feeding. While much of this work has focused on projections to the lateral hypothalamus, the role of NAc projections to the VTA in the control food intake has been largely unexplored. Using a retrograde viral labeling technique and real-time monitoring of neural activity with fiber photometry, we find that medial NAc shell projections to the VTA (mNAc→VTA) are inhibited during food-seeking and food consumption in male mice. We also demonstrate that this circuit bidirectionally controls feeding: optogenetic activation of NAc projections to the VTA inhibits food-seeking and food intake (in both sexes), while optogenetic inhibition of this circuit potentiates food-seeking behavior. Additionally, we show that activity of the NAc to VTA pathway is necessary for adaptive inhibition of food intake in response to external cues. These data provide new insight into NAc control over feeding in mice, and contribute to an emerging literature elucidating the role of inhibitory midbrain feedback within the mesolimbic circuit.SIGNIFICANCE STATEMENT The medial NAc has long been known to control consummatory behavior, with particular focus on accumbens projections to the lateral hypothalamus. Conversely, NAc projections to the VTA have mainly been studied in the context of drug reward. We show that NAc projections to the VTA bidirectionally control food intake, consistent with a permissive role in feeding. Additionally, we show that this circuit is normally inactivated during consumption and food-seeking. Together, these findings elucidate how mesolimbic circuits control food consumption.

MPEP Lowers Binge Drinking in Male and Female C57BL/6 Mice: Relationship with mGlu5/Homer2/Erk2 Signaling.

BACKGROUND: Metabotropic glutamate receptor 5 (mGlu5) plays an important role in excessive alcohol use and the mGlu5/Homer2/Erk2 signaling pathway has been implicated in binge drinking. The mGlu5 negative allosteric modulator (NAM) 2-methyl-6-(phenylethynyl)pyridine hydrochloride (MPEP) has been shown to reduce binge drinking in male mice, but less is known about its effect on female mice. Here, we sought to determine whether sex differences exists in the effects of MPEP on binge drinking and whether they relate to changes in the MPEP mGlu5/Homer2/Erk2 signaling. METHODS: We measured the dose-response effect of MPEP on alcohol consumption in male and female mice using the Drinking in the Dark (DID) paradigm to assess potential sex differences. To rule out possible confounds of MPEP on locomotion, we measured the effects of MPEP on locomotor activity and drinking simultaneously during DID. Lastly, to test whether MPEP-induced changes in alcohol consumption were related to changes in Homer2 or Erk2 expression, we performed qPCR using brain tissue acquired from mice that had undergone 7 days of DID. RESULTS: 30 mg/kg MPEP reduced binge alcohol consumption across female and male mice, with no sex differences in the dose-response relationship. Locomotor activity did not mediate the effects of MPEP on alcohol intake, but activity correlated with alcohol intake independent of MPEP. MPEP did not change the expression of Homer2 and Erk2 mRNA in the bed nucleus of the stria terminalis (BNST) or nucleus accumbens in mice whose drinking was reduced by MPEP, relative to saline. There was a positive relationship between alcohol intake and Homer2 expression in the BNST. CONCLUSIONS: MPEP reduced alcohol consumption during DID in male and female C57BL/6 mice but did not change Homer2/Erk2 expression. Locomotor activity did not mediate the effects of MPEP on alcohol intake, though it correlated with alcohol intake. Alcohol intake during DID predicted BNST Homer2 expression. These data provide support for the regulation of alcohol consumption by mGlu5 across sexes.

Neurochemical and Behavioral Dissections of Decision-Making in a Rodent Multistage Task.

Flexible decision-making in dynamic environments requires both retrospective appraisal of reinforced actions and prospective reasoning about the consequences of actions. These complementary reinforcement-learning systems can be characterized computationally with model-free and model-based algorithms, but how these processes interact at a neurobehavioral level in normal and pathological states is unknown. Here, we developed a translationally analogous multistage decision-making (MSDM) task to independently quantify model-free and model-based behavioral mechanisms in rats. We provide the first direct evidence that male rats, similar to humans, use both model-free and model-based learning when making value-based choices in the MSDM task and provide novel analytic approaches for independently quantifying these reinforcement-learning strategies. Furthermore, we report that ex vivo dopamine tone in the ventral striatum and orbitofrontal cortex correlate with model-based, but not model-free, strategies, indicating that the biological mechanisms mediating decision-making in the multistage task are conserved in rats and humans. This new multistage task provides a unique behavioral platform for conducting systems-level analyses of decision-making in normal and pathological states.SIGNIFICANCE STATEMENT Decision-making is influenced by both a retrospective "model-free" system and a prospective "model-based" system in humans, but the biobehavioral mechanisms mediating these learning systems in normal and disease states are unknown. Here, we describe a translationally analogous multistage decision-making task to provide a behavioral platform for conducting neuroscience studies of decision-making in rats. We provide the first evidence that choice behavior in rats is influenced by model-free and model-based systems and demonstrate that model-based, but not model-free, learning is associated with corticostriatal dopamine tone. This novel behavioral paradigm has the potential to yield critical insights into the mechanisms mediating decision-making alterations in mental disorders.

Endocannabinoid contributions to alcohol habits and motivation: Relevance to treatment.

Individuals with alcohol use disorder exhibit compulsive habitual behaviors that are thought to be, in part, a consequence of chronic and persistent use of alcohol. The endocannabinoid system plays a critical role in habit learning and in ethanol self-administration, but the role of this neuromodulatory system in the expression of habitual alcohol seeking is unknown. Here, we investigated the role of the endocannabinoid system in established alcohol habits using contingency degradation in male C57BL/6 mice. We found that administration of the novel diacyl glycerol lipase inhibitor DO34, which decreases the biosynthesis of the endocannabinoid 2-arachidonoyl glycerol (2-AG), reduced habitual responding for ethanol and ethanol approach behaviors. Moreover, administration of the endocannabinoid transport inhibitor AM404 or the cannabinoid receptor type 1 antagonist AM251 produced similar reductions in habitual responding for ethanol and ethanol approach behaviors. Notably, AM404 was also able to reduce ethanol seeking and consumption in mice that were insensitive to lithium chloride-induced devaluation of ethanol. Conversely, administration of JZL184, a monoacyl glycerol lipase inhibitor that increases levels of 2-AG, increased motivation to respond for ethanol on a progressive ratio schedule of reinforcement. These results demonstrate an important role for endocannabinoid signaling in the motivation to seek ethanol, in ethanol-motivated habits, and suggest that pharmacological manipulations of endocannabinoid signaling could be effective therapeutics for treating alcohol use disorder.

Bi-directional modulation of food habit expression by the endocannabinoid system.

The compulsive, habitual behaviors that have been observed in individuals diagnosed with substance use disorders may be due to disruptions in the neural circuits that mediate goal-directed actions. The endocannabinoid system has been shown to play a critical role in habit learning, but the role of this neuromodulatory system in habit expression is unclear. Here, we investigated the role of the endocannabinoid system in established habitual actions using contingency degradation in male C57BL/6 mice. We found that administration of the endocannabinoid transport inhibitor AM404 reduced habitual responding for food and that antagonism of cannabinoid receptor type 1 (CB1), but not transient receptor potential cation subfamily V (TRPV1), receptors produced a similar reduction in habitual responding. Moreover, pharmacological stimulation of CB1 receptors increased habitual responding for food. Co-administration of an enzyme inhibitor that selectively increases the endocannabinoid 2-arachidonoyl glycerol (2-AG) with AM404 partially restored habitual responding for food. Together, these findings demonstrate an important role for the endocannabinoid system in the expression of habits and provide novel insights into potential pharmacological strategies for reducing habitual behaviors in mental disorders.

Circuit and Synaptic Plasticity Mechanisms of Drug Relapse.

High rates of relapse to drug use during abstinence is a defining feature of human drug addiction. This clinical scenario has been studied at the preclinical level using different animal models in which relapse to drug seeking is assessed after cessation of operant drug self-administration in rodents and monkeys. In our Society for Neuroscience (SFN) session entitled "Circuit and Synaptic Plasticity Mechanisms of Drug Relapse," we will discuss new developments of our understanding of circuits and synaptic plasticity mechanisms of drug relapse from studies combining established and novel animal models with state-of-the-art cellular, electrophysiology, anatomical, chemogenetic, and optogenetic methods. We will also discuss the translational implications of these new developments. In the mini-review that introduces our SFN session, we summarize results from our laboratories on behavioral, cellular, and circuit mechanisms of drug relapse within the context of our session.

Regulation of Alcohol Extinction and Cue-Induced Reinstatement by Specific Projections among Medial Prefrontal Cortex, Nucleus Accumbens, and Basolateral Amygdala.

The ability to inhibit drinking is a significant challenge for recovering alcoholics, especially in the presence of alcohol-associated cues. Previous studies have demonstrated that the regulation of cue-guided alcohol seeking is mediated by the basolateral amygdala (BLA), nucleus accumbens (NAc), and medial prefrontal cortex (mPFC). However, given the high interconnectivity between these structures, it is unclear how mPFC projections to each subcortical structure, as well as projections between BLA and NAc, mediate alcohol-seeking behaviors. Here, we evaluate how cortico-striatal, cortico-amygdalar, and amygdalo-striatal projections control extinction and relapse in a rat model of alcohol seeking. Specifically, we used a combinatorial viral technique to express diphtheria toxin receptors in specific neuron populations based on their projection targets. We then used this strategy to create directionally selective ablations of three distinct pathways after acquisition of ethanol self-administration but before extinction and reinstatement. We demonstrate that ablation of mPFC neurons projecting to NAc, but not BLA, blocks cue-induced reinstatement of alcohol seeking and neither pathway is necessary for extinction of responding. Further, we show that ablating BLA neurons that project to NAc disrupts extinction of alcohol approach behaviors and attenuates reinstatement. Together, these data provide evidence that the mPFC→NAc pathway is necessary for cue-induced reinstatement of alcohol seeking, expand our understanding of how the BLA→NAc pathway regulates alcohol behavior, and introduce a new methodology for the manipulation of target-specific neural projections.SIGNIFICANCE STATEMENT The vast majority of recovering alcoholics will relapse at least once and understanding how the brain regulates relapse will be key to developing more effective behavior and pharmacological therapies for alcoholism. Given the high interconnectivity of cortical, striatal, and limbic structures that regulate alcohol intake, it has been difficult to disentangle how separate projections between them may control different aspects of these complex behaviors. Here, we demonstrate a new approach for noninvasively ablating each of these pathways and testing their necessity for both extinction and relapse. We show that inputs to the nucleus accumbens from medial prefrontal cortex and amygdala regulate alcohol-seeking behaviors differentially, adding to our understanding of the neural control of alcoholism.

ARPP-16 Is a Striatal-Enriched Inhibitor of Protein Phosphatase 2A Regulated by Microtubule-Associated Serine/Threonine Kinase 3 (Mast 3 Kinase).

ARPP-16 (cAMP-regulated phospho-protein of molecular weight 16 kDa) is one of several small acid-soluble proteins highly expressed in medium spiny neurons of striatum that are phosphorylated in response to dopamine acting via D1 receptor/protein kinase A (PKA) signaling. We show here that ARPP-16 is also phosphorylated in vitro and in vivo by microtubule-associated serine/threonine kinase 3 (MAST3 kinase), an enzyme of previously unknown function that is enriched in striatum. We find that ARPP-16 interacts directly with the scaffolding A subunit of the serine/threonine protein phosphatase, PP2A, and that phosphorylation of ARPP-16 at Ser46 by MAST3 kinase converts the protein into a selective inhibitor of B55α- and B56δ-containing heterotrimeric forms of PP2A. Ser46 of ARPP-16 is phosphorylated to a high basal stoichiometry in striatum, suggestive of basal inhibition of PP2A in striatal neurons. In support of this hypothesis, conditional knock-out of ARPP-16 in CaMKIIα::cre/floxed ARPP-16/19 mice results in dephosphorylation of a subset of PP2A substrates including phospho-Thr75-DARPP-32, phospho-T308-Akt, and phospho-T202/Y204-ERK. Conditional knock-out of ARPP-16/19 is associated with increased motivation measured on a progressive ratio schedule of food reinforcement, yet an attenuated locomotor response to acute cocaine. Our previous studies have shown that ARPP-16 is phosphorylated at Ser88 by PKA. Activation of PKA in striatal slices leads to phosphorylation of Ser88, and this is accompanied by marked dephosphorylation of Ser46. Together, these studies suggest that phospho-Ser46-ARPP-16 acts to basally control PP2A in striatal medium spiny neurons but that dopamine acting via PKA inactivates ARPP-16 leading to selective potentiation of PP2A signaling.SIGNIFICANCE STATEMENT We describe a novel mechanism of signal transduction enriched in medium spiny neurons of striatum that likely mediates effects of the neurotransmitter dopamine acting on these cells. We find that the protein ARPP-16, which is highly expressed in striatal medium spiny neurons, acts as a selective inhibitor of certain forms of the serine/threonine protein phosphatase, PP2A, when phosphorylated by the kinase, MAST3. Under basal conditions, ARPP-16 is phosphorylated by MAST3 to a very high stoichiometry. However, the actions of MAST3 are antagonized by dopamine and cAMP-regulated signaling leading to disinhibition of ARPP-16 and increased PP2A action.

The Medial Orbitofrontal Cortex Regulates Sensitivity to Outcome Value.

An essential component of goal-directed decision-making is the ability to maintain flexible responding based on the value of a given reward, or "reinforcer." The medial orbitofrontal cortex (mOFC), a subregion of the ventromedial prefrontal cortex, is uniquely positioned to regulate this process. We trained mice to nose poke for food reinforcers and then stimulated this region using CaMKII-driven Gs-coupled designer receptors exclusively activated by designer drugs (DREADDs). In other mice, we silenced the neuroplasticity-associated neurotrophin brain-derived neurotrophic factor (BDNF). Activation of Gs-DREADDs increased behavioral sensitivity to reinforcer devaluation, whereas Bdnf knockdown blocked sensitivity. These changes were accompanied by modifications in breakpoint ratios in a progressive ratio task, and they were recapitulated in Bdnf(+/-)mice. Replacement of BDNF selectively in the mOFC in Bdnf(+/-)mice rescued behavioral deficiencies, as well as phosphorylation of extracellular-signal regulated kinase 1/2 (ERK1/2). Thus, BDNF expression in the mOFC is both necessary and sufficient for the expression of typical effort allocation relative to an anticipated reinforcer. Additional experiments indicated that expression of the immediate-early gene c-fos was aberrantly elevated in the Bdnf(+/-)dorsal striatum, and BDNF replacement in the mOFC normalized expression. Also, systemic administration of an MAP kinase kinase inhibitor increased breakpoint ratios, whereas the addition of discrete cues bridging the response-outcome contingency rescued breakpoints in Bdnf(+/-)mice. We argue that BDNF-ERK1/2 in the mOFC is a key regulator of "online" goal-directed action selection. SIGNIFICANCE STATEMENT: Goal-directed response selection often involves predicting the consequences of one's actions and the value of potential payoffs. Lesions or chemogenetic inactivation of the medial orbitofrontal cortex (mOFC) in rats induces failures in retrieving outcome identity memories (Bradfield et al., 2015), suggesting that the healthy mOFC serves to access outcome value information when it is not immediately observable and thereby guide goal-directed decision-making. Our findings suggest that the mOFC also bidirectionally regulates effort allocation for a given reward and that expression of the neurotrophin BDNF in the mOFC is both necessary and sufficient for mice to sustain stable representations of reinforcer value.

Phosphoproteomic Analysis Reveals a Novel Mechanism of CaMKIIα Regulation Inversely Induced by Cocaine Memory Extinction versus Reconsolidation.

UNLABELLED: Successful addiction treatment depends on maintaining long-term abstinence, making relapse prevention an essential therapeutic goal. However, exposure to environmental cues associated with drug use often thwarts abstinence efforts by triggering drug using memories that drive craving and relapse. We sought to develop a dual approach for weakening cocaine memories through phosphoproteomic identification of targets regulated in opposite directions by memory extinction compared with reconsolidation in male Sprague-Dawley rats that had been trained to self-administer cocaine paired with an audiovisual cue. We discovered a novel, inversely regulated, memory-dependent phosphorylation event on calcium-calmodulin-dependent kinase II α (CaMKIIα) at serine (S)331. Correspondingly, extinction-associated S331 phosphorylation inhibited CaMKIIα activity. Intra-basolateral amygdala inhibition of CaMKII promoted memory extinction and disrupted reconsolidation, leading to a reduction in subsequent cue-induced reinstatement. CaMKII inhibition had no effect if the memory was neither retrieved nor extinguished. Therefore, inhibition of CaMKII represents a novel mechanism for memory-based addiction treatment that leverages both extinction enhancement and reconsolidation disruption to reduce relapse-like behavior. SIGNIFICANCE STATEMENT: Preventing relapse to drug use is an important goal for the successful treatment of addictive disorders. Relapse-prevention therapies attempt to interfere with drug-associated memories, but are often hindered by unintentional memory strengthening. In this study, we identify phosphorylation events that are bidirectionally regulated by the reconsolidation versus extinction of a cocaine-associated memory, including a novel site on CaMKIIα. Additionally, using a rodent model of addiction, we show that CaMKII inhibition in the amygdala can reduce relapse-like behavior. Together, our data supports the existence of mechanisms that can be used to enhance current strategies for addiction treatment.

Dopamine D3 Receptor Availability Is Associated with Inflexible Decision Making.

UNLABELLED: Dopamine D2/3 receptor signaling is critical for flexible adaptive behavior; however, it is unclear whether D2, D3, or both receptor subtypes modulate precise signals of feedback and reward history that underlie optimal decision making. Here, PET with the radioligand [(11)C]-(+)-PHNO was used to quantify individual differences in putative D3 receptor availability in rodents trained on a novel three-choice spatial acquisition and reversal-learning task with probabilistic reinforcement. Binding of [(11)C]-(+)-PHNO in the midbrain was negatively related to the ability of rats to adapt to changes in rewarded locations, but not to the initial learning. Computational modeling of choice behavior in the reversal phase indicated that [(11)C]-(+)-PHNO binding in the midbrain was related to the learning rate and sensitivity to positive, but not negative, feedback. Administration of a D3-preferring agonist likewise impaired reversal performance by reducing the learning rate and sensitivity to positive feedback. These results demonstrate a previously unrecognized role for D3 receptors in select aspects of reinforcement learning and suggest that individual variation in midbrain D3 receptors influences flexible behavior. Our combined neuroimaging, behavioral, pharmacological, and computational approach implicates the dopamine D3 receptor in decision-making processes that are altered in psychiatric disorders. SIGNIFICANCE STATEMENT: Flexible decision-making behavior is dependent upon dopamine D2/3 signaling in corticostriatal brain regions. However, the role of D3 receptors in adaptive, goal-directed behavior has not been thoroughly investigated. By combining PET imaging with the D3-preferring radioligand [(11)C]-(+)-PHNO, pharmacology, a novel three-choice probabilistic discrimination and reversal task and computational modeling of behavior in rats, we report that naturally occurring variation in [(11)C]-(+)-PHNO receptor availability relates to specific aspects of flexible decision making. We confirm these relationships using a D3-preferring agonist, thus identifying a unique role of midbrain D3 receptors in decision-making processes.

Ketamine accelerates fear extinction via mTORC1 signaling.

Impaired fear extinction contributes to the persistence of post-traumatic stress disorder (PTSD), and can be utilized for the study of novel therapeutic agents. Glutamate plays an important role in the formation of traumatic memories, and in the pathophysiology and treatment of PTSD, highlighting several possible drug targets. Recent clinical studies demonstrate that infusion of ketamine, a glutamate NMDA receptor antagonist, rapidly and significantly reduces symptom severity in PTSD patients. In the present study, we examine the mechanisms underlying the actions of ketamine in a rodent model of fear conditioning, extinction, and renewal. Rats received ketamine or saline 24h after fear conditioning and were then subjected to extinction-training on each of the following three days. Ketamine administration enhanced extinction on the second day of training (i.e., reduced freezing behavior to cue) and produced a long-lasting reduction in freezing on exposure to cue plus context 8days later. Additionally, ketamine and extinction exposure increased levels of mTORC1 in the medial prefrontal cortex (mPFC), a region involved in the acquisition and retrieval of extinction, and infusion of the selective mTORC1 inhibitor rapamycin into the mPFC blocked the effects of ketamine on extinction. Ketamine plus extinction also increased cFos in the mPFC and administration of a glutamate-AMPA receptor antagonist blocked the effects of ketamine. These results support the hypothesis that ketamine produces long-lasting mTORC1/protein synthesis and activity dependent effects on neuronal circuits that enhance the expression of extinction and could represent a novel approach for the treatment of PTSD.

Reconsolidation and psychopathology: Moving towards reconsolidation-based treatments.

Interfering with memory reconsolidation has valuable potential to be used as a treatment for maladaptive memories and psychiatric disorders. Numerous studies suggest that reconsolidation-based therapies may benefit psychiatric populations, but much remains unanswered. After reviewing the literature in clinical and healthy human populations, we discuss some of the major limitations to reconsolidation studies and clinical application. Finally, we provide recommendations for developing improved reconsolidation-based treatments, namely exploiting known boundary conditions and focusing on a novel unconditioned stimulus-retrieval paradigm.

Inhibition of protein synthesis but not ß-adrenergic receptors blocks reconsolidation of a cocaine-associated cue memory.

Previously consolidated memories have the potential to enter a state of lability upon memory recall, during which time the memory can be altered before undergoing an additional consolidation-like process and being stored again as a long-term memory. Blocking reconsolidation of aberrant memories has been proposed as a potential treatment for psychiatric disorders including addiction. Here we investigated of the effect of systemically administering the protein synthesis inhibitor cycloheximide or the ß-adrenergic antagonist propranolol on reconsolidation. Rats were trained to self-administer cocaine, during which each lever press resulted in the presentation of a cue paired with an intravenous infusion of cocaine. After undergoing lever press extinction to reduce operant responding, the cue memory was reactivated and rats were administered systemic injections of propranolol, cycloheximide, or vehicle. Post-reactivation cycloheximide, but not propranolol, resulted in a reactivation-dependent decrease in cue-induced reinstatement, indicative of reconsolidation blockade by protein synthesis inhibition. The present data indicate that systemically targeting protein synthesis as opposed to the ß-adrenergic system may more effectively attenuate the reconsolidation of a drug-related memory and decrease drug-seeking behavior.

Prefrontal cortical BDNF: A regulatory key in cocaine- and food-reinforced behaviors.

Brain-derived neurotrophic factor (BDNF) affects synaptic plasticity and neural structure and plays key roles in learning and memory processes. Recent evidence also points to important, yet complex, roles for BDNF in rodent models of cocaine abuse and addiction. Here we examine the role of prefrontal cortical (PFC) BDNF in reward-related decision making and behavioral sensitivity to, and responding for, cocaine. We focus on BDNF within the medial and orbital PFC, its regulation by cocaine during early postnatal development and in adulthood, and how BDNF in turn influences responding for drug reinforcement, including in reinstatement models. When relevant, we draw comparisons and contrasts with experiments using natural (food) reinforcers. We also summarize findings supporting, or refuting, the possibility that BDNF in the medial and orbital PFC regulate the development and maintenance of stimulus-response habits. Further investigation could assist in the development of novel treatment approaches for cocaine use disorders.

Infralimbic prefrontal cortex interacts with nucleus accumbens shell to unmask expression of outcome-selective Pavlovian-to-instrumental transfer.

Although several studies have examined the subcortical circuitry underlying Pavlovian-to-instrumental transfer (PIT), the role of medial prefrontal cortex in this behavior is largely unknown. Elucidating the cortical contributions to PIT will be key for understanding how reward-paired cues control behavior in both adaptive and maladaptive context (i.e., addiction). Here we use bilateral lesions in a rat model to show that infralimbic prefrontal cortex (ilPFC) is necessary for appropriate expression of PIT. Further, we show that ilPFC mediates this effect via functional connectivity with nucleus accumbens shell (NAcS). Together, these data provide the first demonstration that a specific cortico-striatal circuit is necessary for cue-invigorated reward seeking during specific PIT.