Optogenetic stimulation of the locus coeruleus enhances appetitive extinction in rats.

Extinction is a specific example of learning where a previously reinforced stimulus or response is no longer reinforced, and the previously learned behaviour is no longer necessary and must be modified. Current theories suggest extinction is not the erasure of the original learning but involves new learning that acts to suppress the original behaviour. Evidence for this can be found when the original behaviour recovers following the passage of time (spontaneous recovery) or reintroduction of the reinforcement (i.e. reinstatement). Recent studies have shown that pharmacological manipulation of noradrenaline (NA) or its receptors can influence appetitive extinction; however, the role and source of endogenous NA in these effects are unknown. Here, we examined the role of the locus coeruleus (LC) in appetitive extinction. Specifically, we tested whether optogenetic stimulation of LC neurons during extinction of a food-seeking behaviour would enhance extinction evidenced by reduced spontaneous recovery in future tests. LC stimulation during extinction trials did not change the rate of extinction but did serve to reduce subsequent spontaneous recovery, suggesting that stimulation of the LC can augment reward-related extinction. Optogenetic inhibition of the LC during extinction trials reduced responding during the trials where it was applied, but no long-lasting changes in the retention of extinction were observed. Since not all LC cells expressed halorhodopsin, it is possible that more complete LC inhibition or pathway-specific targeting would be more effective at suppressing extinction learning. These results provide further insight into the neural basis of appetitive extinction, and in particular the role of the LC. A deeper understanding of the physiological bases of extinction can aid development of more effective extinction-based therapies.

Ghrelin receptor antagonism and satiety attenuate Pavlovian-instrumental transfer.

Animals rely on learned cues to guide their behaviour for rewards such as food. The Pavlovian-instrumental transfer (PIT) task can be used to investigate the influence of Pavlovian stimuli on instrumental responding. Ghrelin, an orexigenic peptide, and its receptor, growth hormone secretagogue receptor 1A (GHS-R1A), has received growing interest for its role in reward-motivated learning and behaviours. A significant population of GHS-R1A have been identified within the ventral tegmental area (VTA), a critical node in the mesolimbic reward circuit that is necessary for the expression of PIT. As ghrelin has been found to increase dopaminergic activity in the VTA, we predicted that GHS-R1A antagonism with JMV-2959 would attenuate PIT. Further, given the relationship between hunger levels and changes in ghrelin signalling, we sought to compare the effects GHS-R1A antagonism with those of satiety, hypothesizing parallel effects, with each attenuating PIT. Rats received daily sessions of Pavlovian and then instrumental training over 3 weeks. Across three experiments, we examined the effects of a shift to satiety, or treatment with the GHS-R1A antagonist JMV-2959, either peripherally or directly into the VTA. We found that presentations of a stimulus paired with food reward enhanced responding for food across all conditions, thus demonstrating the expected PIT effect. Further, GHS-R1A antagonism, both peripherally and within the VTA, as well as satiety significantly reduced the magnitude of the PIT effect compared to control conditions. These results clarify our understanding of ghrelin signalling in PIT and begin to elucidate the role of feeding-related peptides in the modulation of reward-related responding.

Serotonin and Dopamine Show Different Response Profiles to Acute Stress in the Nucleus Accumbens and Medial Prefrontal Cortex of Rats with Neuropathic Pain.

The ability to adaptively guide behaviour requires the integration of external information with internal motivational factors. Decision-making capabilities can be impaired by acute stress and is often exacerbated by chronic pain. Chronic neuropathic pain patients often present with cognitive dysfunction, including impaired decision-making. The mechanisms underlying these changes are not well understood but may include altered monoaminergic transmission in the brain. In this study we investigated the relationships between dopamine, serotonin, and their metabolites in key brain regions that regulate motivated behaviour and decision-making. The neurochemical profiles of the medial prefrontal cortex, orbital prefrontal cortex, and nucleus accumbens were analysed using HPLC in rats that received a chronic constriction injury (CCI) of the right sciatic nerve and an acute stress (15-min restraint), prior to an outcome devaluation task. CCI alone significantly decreased dopamine but not serotonin concentrations in the medial prefrontal cortex. By contrast, restraint stress acutely increased dopamine in the medial prefrontal cortex, and the nucleus accumbens; and increased serotonin in the medial prefrontal cortex 2 h later. The sustained dopaminergic and serotonergic responses to acute stress highlight the importance of an animal's ability to mount an effective coping response. In addition, these data suggest that the impact of nerve injury and acute stress on outcome-devaluation occurs independently of dopaminergic and serotonergic transmission in the medial prefrontal cortex, orbital prefrontal cortex and nucleus accumbens of rats.

The role of dorsomedial striatum adenosine 2A receptors in the loss of goal-directed behaviour.

RATIONALE: Adenosine A2A receptors (A2AR) in the dorsal striatum have been implicated in goal-directed behaviour. While activation of these receptors with several methods has resulted in an insensitivity to outcome devaluation, particular explanations for how they disrupt behaviour have not been explored. We both confirm a role for A2A receptors in goal-directed responding and evaluate additional behavioural aspects of goal-directed control to more fully understand the role of A2A receptors in instrumental behaviour. OBJECTIVES: To examine the effects of the adenosine A2A agonist CGS-21680 in the DMS on response-outcome encoding, updating representations of outcome value and on the ability to inhibit behaviour when reward is not available. METHODS: Male rats were trained to lever press for food reward. The A2AR agonist CGS-21680 was infused into the dorsomedial striatum either before an outcome devaluation test, prior to training with two distinct response-outcome associations or prior to a test of discriminative stimulus control over instrumental performance. RESULTS: Intra-DMS administration of CGS-21680 impaired sensitivity to outcome devaluation. CGS-21680 treatment did not impair acquisition of specific response-outcome associations, selective control of responding based on the presence of stimuli that signaled when reward was or was not available, discrimination between stimuli or lever choices nor did it influence the effect of devaluation on the amounts of food eaten in a consumption test. CONCLUSIONS: CGS-21680 impairs the ability to modulate responding based on recent changes to outcome value, an effect that is not accounted for by impairments in behavioural inhibition, discrimination, encoding the specific outcome of a response or the effectiveness of specific satiety.

Food for thought: diet-induced impairments to decision-making and amelioration by N-acetylcysteine in male rats.

RATIONALE: Attempts to lose weight often fail despite knowledge of the health risks associated with obesity and determined efforts. We previously showed that rodents fed an obesogenic diet displayed premature habitual behavioural control and weakened flexible decision-making based on the current value of outcomes produced by their behaviour. Thus, habitual control may contribute to failed attempts to modify eating behaviours. OBJECTIVES: To examine the effects of an obesogenic diet on behavioural control and glutamate transmission in dorsal striatum regions and to assess the ability of N-acetylcysteine (NAC) to reverse deficits. METHODS: Here, we examined diet-induced changes to decision-making and used in vitro electrophysiology to investigate the effects of diet on glutamate transmission within the dorsomedial (DMS) and dorsolateral (DLS) striatum, areas that control goal-directed and habitual behaviours, respectively. We administered NAC in order to normalize glutamate release and tested whether this would restore goal-directed performance following an obesogenic diet. RESULTS: We found that an obesogenic diet reduced sensitivity to outcome devaluation and increased glutamate release in the DMS, but not DLS. Administration of NAC restored goal-directed control and normalized mEPSCs in the DMS. Finally, NAC administered directly to the DMS was sufficient to reinstate sensitivity to outcome devaluation following an obesogenic diet. CONCLUSIONS: These data indicate that obesogenic diets alter neural activity in the basal ganglia circuit responsible for goal-directed learning and control which leads to premature habitual control. While the effects of diet are numerous and widespread, normalization of glutamatergic activity in this circuit is sufficient for restoring goal-directed behaviour.

Diet-induced deficits in goal-directed control are rescued by agonism of group II metabotropic glutamate receptors in the dorsomedial striatum.

Many overweight or obese people struggle to sustain the behavioural changes necessary to achieve and maintain weight loss. In rodents, obesogenic diet can disrupt goal-directed control of responding for food reinforcers, which may indicate that diet can disrupt brain regions associated with behavioural control. We investigated a potential glutamatergic mechanism to return goal-directed control to rats who had been given an obesogenic diet prior to operant training. We found that an obesogenic diet reduced goal-directed control and that systemic injection of LY379268, a Group II metabotropic glutamate receptor (mGluR2/3) agonist, returned goal-directed responding in these rats. Further, we found that direct infusion of LY379268 into the dorsomedial striatum, a region associated with goal-directed control, also restored goal-directed responding in the obesogenic-diet group. This indicates that one mechanism through which obesogenic diet disrupts goal-directed control is glutamatergic, and infusion of a mGluR2/3 agonist into the DMS is sufficient to ameliorate deficits in goal-directed control.

Engram Size Varies with Learning and Reflects Memory Content and Precision.

Memories are rarely acquired under ideal conditions, rendering them vulnerable to profound omissions, errors, and ambiguities. Consistent with this, recent work using context fear conditioning has shown that memories formed after inadequate learning time display a variety of maladaptive properties, including overgeneralization to similar contexts. However, the neuronal basis of such poor learning and memory imprecision remains unknown. Using c-fos to track neuronal activity in male mice, we examined how these learning-dependent changes in context fear memory precision are encoded in hippocampal ensembles. We found that the total number of c-fos-encoding cells did not correspond with learning history but instead more closely reflected the length of the session immediately preceding c-fos measurement. However, using a c-fos-driven tagging method (TRAP2 mouse line), we found that the degree of learning and memory specificity corresponded with neuronal activity in a subset of dentate gyrus cells that were active during both learning and recall. Comprehensive memories acquired after longer learning intervals were associated with more double-labeled cells. These were preferentially reactivated in the conditioning context compared with a similar context, paralleling behavioral discrimination. Conversely, impoverished memories acquired after shorter learning intervals were associated with fewer double-labeled cells. These were reactivated equally in both contexts, corresponding with overgeneralization. Together, these findings provide two surprising conclusions. First, engram size varies with learning. Second, larger engrams support better neuronal and behavioral discrimination. These findings are incorporated into a model that describes how neuronal activity is influenced by previous learning and present experience, thus driving behavior.SIGNIFICANCE STATEMENT Memories are not always formed under ideal circumstances. This is especially true in traumatic situations, such as car accidents, where individuals have insufficient time to process what happened around them. Such memories have the potential to overgeneralize to irrelevant situations, producing inappropriate fear and contributing to disorders, such as post-traumatic stress disorder. However, it is unknown how such poorly formed fear memories are encoded within the brain. We find that restricting learning time results in fear memories that are encoded by fewer hippocampal cells. Moreover, these fewer cells are inappropriately reactivated in both dangerous and safe contexts. These findings suggest that fear memories formed at brief periods overgeneralize because they lack the detail-rich information necessary to support neuronal discrimination.

Problematic eating as an issue of habitual control.

Obesity has reached alarming rates worldwide. Although many people attempt to control weight by modifying their food-related behaviours, this typically only has short-term effects and most dieters regain the weight that was lost. Why do so many people struggle to regulate their food-related behaviours? One possible explanation is that these behaviours have become habits that are not immediately sensitive to their consequences. Here we review experimental evidence for a shift to habitual control over food-related behaviours and the neural systems that control them and how this relates to difficulty changing ones' eating behavior.

Neural substrates of appetitive and aversive prediction error.

Prediction error, defined by the discrepancy between real and expected outcomes, lies at the core of associative learning. Behavioural investigations have provided evidence that prediction error up- and down-regulates associative relationships, and allocates attention to stimuli to enable learning. These behavioural advances have recently been followed by investigations into the neurobiological substrates of prediction error. In the present paper, we review neuroscience data obtained using causal and recording neural methods from a variety of key behavioural designs. We explore the neurobiology of both appetitive (reward) and aversive (fear) prediction error with a focus on the mesolimbic dopamine system, the amygdala, ventrolateral periaqueductal gray, hippocampus, cortex and locus coeruleus noradrenaline. New questions and avenues for research are considered.

The role of the response-outcome association in the nature of inhibitory Pavlovian-instrumental transfer in rats.

Inhibitory stimuli can reduce animals' reward seeking in an outcome-specific manner or outcome-general manner. However, we do not understand the factors that determine which of these effects are produced. To address this, we carried out three experiments which examined whether instrumental training with one or multiple outcomes determined the nature of subsequently observed Pavlovian-instrumental transfer (PIT). Rats underwent Pavlovian training to produce inhibitors and excitors for two outcomes using a feature-negative procedure. In Experiment 1, these stimuli were tested for their effects on a single response trained with one of those outcomes in a PIT procedure. Here, stimuli trained as inhibitors and excitors were found to produce outcome-general effects on reward seeking (in addition to an outcome-specific effect for excitors). In Experiment 2, we trained two responses, one for each of the Pavlovian outcomes, and tested the effect of the stimuli on each response individually. This design also produced outcome-general inhibitory and excitatory PIT effects. Experiment 3 followed the procedure of Experiment 2, except for implementation of a shorter Pavlovian training phase and an additional choice test, where both responses were concurrently available. This procedure produced putative inhibitory effects that were also outcome-general. However, outcome-specific excitatory effects were observed, indicating that the general inhibitory results may not be attributable to the duration of Pavlovian training. Overall, this study suggests that variations in the number of response-outcome contingencies experienced by animals do not readily determine the specificity of putative inhibitors.

Maladaptive Properties of Context-Impoverished Memories.

The context in which sudden fearful events occur can be poorly encoded into memory. Yet, the consequences of the resulting context-impoverished memories remain unknown. We demonstrate that restricting the time available for context encoding during contextual fear conditioning causes maladaptively overgeneralized and inextinguishable fear. However, post-conditioning context exposure enables further context encoding through hippocampal reconsolidation-dependent memory updating. Updating in the conditioning context alleviates overgeneralization and restores capacity for extinction. However, updating in a similar safe context erroneously shifts fear from the dangerous to the safe context. We argue that these phenomena can be explained by uncertainty about where events occurred. Moreover, we show that a hippocampal-neocortical neurocomputational model based on this assumption successfully simulates and explains our observations. These findings reveal that context-impoverished memories are maladaptive and can be improved or distorted after recall, with implications for basic memory theory, memory distortion, and treatment of disorders like post-traumatic stress disorder.

Altered monoamine levels in the dorsal striatum of the rat are associated with alterations in behavioural selection and motivation following peripheral nerve injury and acute stress.

Chronic neuropathic pain and psychological stress interact to compromise goal-directed control over behaviour following mild psychological stress. The dorsomedial (DMS) and dorsolateral (DLS) striatum in the rat are crucial for the expression of goal-directed and habitual behaviours, respectively. This study investigated whether changes in monoamine levels in the DMS and DLS following nerve injury and psychological stress reflect these behavioural differences. Neuropathic pain was induced by a chronic constriction injury (CCI) of the sciatic nerve in Sprague-Dawley rats. Acute stress was induced using a 15-min restraint. Behavioural flexibility was assessed using the outcome devaluation paradigm. Noradrenaline, serotonin, dopamine and associated metabolites were measured bilaterally from the DLS and DMS. In uninjured rats, restraint increased dopaminergic markers in the left and serotonergic markers in the right of both the DMS and DLS, indicating a possible left hemisphere-mediated dominance. CCI led to a slightly different lateralised effect, with a larger effect in the DMS than in the DLS. Individual differences in behavioural flexibility following CCI negatively correlated with dopaminergic markers in the right DLS, but positively correlated with these markers in the left DMS. A combination of CCI and restraint reduced behavioural flexibility, which was associated with the loss of the left/DMS dominance. These data suggest that behavioural flexibility following psychological stress or pain is associated with a left hemisphere dominance within the dorsal striatum. The loss of behavioural flexibility following the combined stressors is then associated with a transition from left to right, and DMS to DLS dominance.

Noradrenergic ß-receptor antagonism in the basolateral amygdala impairs reconsolidation, but not extinction, of alcohol self-administration: Intra-BLA propranolol impairs reconsolidation of alcohol self-administration.

A critical barrier to recovery from alcohol addiction is relapse propensity. Alcohol cues can trigger relapse, and pharmacologically facilitating processes such as extinction, which decreases cue associations, may help prevent relapse. The noradrenergic system mediates extinction learning for alcohol; however, the neural locus of this effect is unknown. This study sought to determine whether the basolateral amygdala (BLA), a region critical for fear extinction, also mediates extinction of alcohol seeking. Hooded Wistar rats (N = 12-15 per experiment) were implanted with bilateral cannula targeting the BLA and trained to lever press for 10% ethanol during auditory or visual cues. Infusions of the ß-receptor antagonist propranolol (2 µg/side) were administered prior to extinction (Experiment 1), and rats assessed for relapse-like behaviour two weeks later, thus allowing for spontaneous recovery. We expected intra-BLA propranolol to impair extinction learning; however, propranolol-treated rats exhibited reduced responding in the test of spontaneous recovery, suggesting enhanced extinction. We investigated this unexpected result by determining if propranolol treatment affected memory processes other than extinction. In a subsequent experiment, rats were infused with propranolol immediately after extinction to target consolidation of extinction (Experiment 2a), and assessed for spontaneous recovery. Propranolol was also infused after self-administration to target reconsolidation of the original learning (Experiment 2b). Propranolol treatment had no effect on consolidation of extinction learning, but impaired reconsolidation of self-administration. Propranolol administered prior to a self-administration session did not affect reinforced responding (Experiment 2c). Extinction and reconsolidation are opposing processes triggered by specific test conditions. We suggest our test conditions induced reconsolidation of self-administration memory by propranolol, rather than modulation of extinction. Thus, our data implicates intra-BLA noradrenergic ß-receptors in reconsolidation of alcohol self-administration memory.

Variety overcomes the specificity of cue-potentiated feeding in rats.

Cue-potentiated feeding (CPF) describes the stimulation of food consumption by cues that have become associated with food. Determining under what Conditions CPF occurs is important to better understand how exposure to food cues contributes to overeating. CPF is typically found to be specific: cues enhance consumption only of the food they have signaled. Further, previous research has focused largely on discrete cues rather than multimodal cues such as a feeding environment. The present experiments paired a "Plus" context with highly palatable food and a "Minus" context with no food or chow in adult female rats. Experiment 1 confirmed that the Plus context enhanced consumption of the paired food (Froot Loops) but not a different food (banana bread). Experiments 2 and 3 tested whether pairing a variety of foods with the Plus context would overcome this specificity. In Experiment 2 the Plus context either contained bland chow (Chow group), 1 (Single group), or 3 palatable foods (Variety group). The test food, Froot Loops, was familiar but never paired with the Plus context. The Variety group exhibited CPF by eating more Froot Loops in the Plus than in the Minus context, while Single and Chow groups ate equivalently in the 2 contexts. Experiment 3 replicated this effect when the Minus context contained chow during training and when a novel food was tested. These findings have important implications for overeating given that modern food environments are typified by variety and that food consumption often occurs outside the home. (PsycINFO Database Record

Methamphetamine promotes habitual action and alters the density of striatal glutamate receptor and vesicular proteins in dorsal striatum.

Goal-directed actions are controlled by the value of the consequences they produce and so increase when what they produce is valuable and decrease when it is not. With continued invariant practice, however, goal-directed actions can become habits, controlled not by their consequences but by antecedent, reward-related states and stimuli. Here, we show that pre-exposure to methamphetamine (METH) caused abnormally rapid development of habitual control. Furthermore, these drug-induced habits differed strikingly from conventional habits; we found that they were insensitive both to changes in reward value and to the effects of negative feedback. In addition to these behavioral changes, METH exposure produced bidirectional changes to synaptic proteins in the dorsal striatum. In the dorsomedial striatum, a structure critical for goal-directed action, METH exposure was associated with a reduction in glutamate receptor and glutamate vesicular proteins, whereas in the dorsolateral striatum, a region that has previously been implicated in habit learning, there was an increase in these proteins. Together, these results indicate that METH exposure promotes habitual control of action that appears to be the result of bidirectional changes in glutamatergic transmission in the circuits underlying goal-directed and habit-based learning.

Peripheral nerve injury impairs the ability to maintain behavioural flexibility following acute stress in the rat.

Chronic neuropathic pain often leads to impaired cognition and reduced behavioural flexibility. This study used a rat model to investigate if a peripheral nerve injury, with or without an additional acute psychological stress, alters behavioural flexibility and goal directed behaviour as measured by sensitivity to devaluation. Neuropathic pain was induced by a chronic constriction injury (CCI) of the sciatic nerve. CCI, sham-injury and naïve rats were trained to press two levers for two rewards. In outcome devaluation tests, one of the rewards was devalued by pre-feeding it to satiety, immediately prior to an extinction test measuring responding on the two levers. The ability to preferentially direct responding toward the action earning the currently-valued reward was taken as evidence of goal-directed behaviour. To test the impact of acute stress, rats were subjected to 15min restraint following pre-feeding and prior to the devaluation test. Neither CCI surgery nor acute stress alone altered sensitivity to devaluation, but in combination CCI and acute stress significantly reduced sensitivity to devaluation. This Study demonstrates that relatively mild stressors that are without effect in uninjured populations can markedly impair cognition under conditions of chronic pain. It further suggests that overlapping neural substrates regulated by nerve injury and/or acute stress are having a cumulative effect on behavioural flexibility.

Extinction of alcohol seeking is enhanced by compound extinction and the noradrenaline reuptake inhibitor atomoxetine.

Alcohol-related stimuli can trigger relapse of alcohol-seeking behaviors even after extended periods of abstinence. Extinction of such stimuli provides a means for reducing their impact on relapse. However, the expression of extinction can be disrupted by exposure to the previous reinforcer as well as the simple passage of time. We investigated whether augmentation of prediction error or of noradrenaline neurotransmission by the reuptake inhibitor atomoxetine would enhance long-term extinction of alcohol-seeking behavior. Rats received Pavlovian conditioning of multiple stimuli signaling the delivery of an alcohol reward before individual extinction was given to each of these stimuli. Further extinction was then given to a target stimulus presented in compound with another alcohol-predictive stimulus intended to augment prediction error (Experiment 1) or after a systemic injection of atomoxetine (1.0 mg/kg; Experiment 2). Experiment 3 examined whether the compound stimulus effect relied on noradrenergic activity by testing the effects of the ß-adrenergic antagonist propranolol, given prior to compound stimulus trials. Long-term retention of extinction learning was assessed a week later. Compound stimulus presentations enhanced long-term extinction as the stimulus extinguished in compound elicited less responding than a stimulus receiving equal extinction alone when tested a week later. This effect was mimicked by atomoxetine and blocked by propranolol given during extinction training. Thus, extinction of alcohol-seeking behavior can be improved by extinguishing multiple alcohol-predictive stimuli or enhancing noradrenaline neurotransmission during extinction training. Both behavioral and neurobiological processes could be exploited to enhance the outcome of extinction-based treatments for alcohol use disorders.

Pulling habits out of rats: adenosine 2A receptor antagonism in dorsomedial striatum rescues meth-amphetamine-induced deficits in goal-directed action.

Addiction is characterized by a persistent loss of behavioral control resulting in insensitivity to negative feedback and abnormal decision-making. Here, we investigated the influence of methamphetamine (METH)-paired contextual cues on decision-making in rats. Choice between goal-directed actions was sensitive to outcome devaluation in a saline-paired context but was impaired in the METH-paired context, a deficit that was also found when negative feedback was provided. Reductions in c-Fos-related immunoreactivity were found in dorsomedial striatum (DMS) but not dorsolateral striatum after exposure to the METH context suggesting this effect reflected a loss specifically in goal-directed control in the METH context. This reduction in c-Fos was localized to non-enkephalin-expressing neurons in the DMS, likely dopamine D1-expressing direct pathway neurons, suggesting a relative change in control by the D1-direct versus D2-indirect pathways originating in the DMS may have been induced by METH-context exposure. To test this suggestion, we infused the adenosine 2A receptor antagonist ZM241385 into the DMS prior to test to reduce activity in D2 neurons relative to D1 neurons in the hope of reducing the inhibitory output from this region of the striatum. We found that this treatment fully restored sensitivity to negative feedback in a test conducted in the METH-paired context. These results suggest that drug exposure alters decision-making by downregulation of the circuitry mediating goal-directed action, an effect that can be ameliorated by acute A2A receptor inhibition in this circuit.