The Neural Bases of Action-Outcome Learning in Humans.

From an associative perspective the acquisition of new goal-directed actions requires the encoding of specific action-outcome (AO) associations and, therefore, sensitivity to the validity of an action as a predictor of a specific outcome relative to other events. Although competitive architectures have been proposed within associative learning theory to achieve this kind of identity-based selection, whether and how these architectures are implemented by the brain is still a matter of conjecture. To investigate this issue, we trained human participants to encode various AO associations while undergoing functional neuroimaging (fMRI). We then degraded one AO contingency by increasing the probability of the outcome in the absence of its associated action while keeping other AO contingencies intact. We found that this treatment selectively reduced performance of the degraded action. Furthermore, when a signal predicted the unpaired outcome, performance of the action was restored, suggesting that the degradation effect reflects competition between the action and the context for prediction of the specific outcome. We used a Kalman filter to model the contribution of different causal variables to AO learning and found that activity in the medial prefrontal cortex (mPFC) and the dorsal anterior cingulate cortex (dACC) tracked changes in the association of the action and context, respectively, with regard to the specific outcome. Furthermore, we found the mPFC participated in a network with the striatum and posterior parietal cortex to segregate the influence of the various competing predictors to establish specific AO associations.SIGNIFICANCE STATEMENT Humans and other animals learn the consequences of their actions, allowing them to control their environment in a goal-directed manner. Nevertheless, it is unknown how we parse environmental causes from the effects of our own actions to establish these specific action-outcome (AO) relationships. Here, we show that the brain learns the causal structure of the environment by segregating the unique influence of actions from other causes in the medial prefrontal and anterior cingulate cortices and, through a network of structures, including the caudate nucleus and posterior parietal cortex, establishes the distinct causal relationships from which specific AO associations are formed.

What does dopamine release reveal about latent inhibition?

A recent paper by Kutlu et al. (2022) argues that changes in dopamine release during stimulus pre-exposure reflect non-associative changes in attention to the conditioned stimulus that are causally related to latent inhibition effects. Associative accounts of pre-exposure-induced changes in associability suggest, however, that such conclusions may be premature.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer.

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Inhibition of vascular adhesion protein 1 protects dopamine neurons from the effects of acute inflammation and restores habit learning in the striatum.

BACKGROUND: Changes in dopaminergic neural function can be induced by an acute inflammatory state that, by altering the integrity of the neurovasculature, induces neuronal stress, cell death and causes functional deficits. Effectively blocking these effects of inflammation could, therefore, reduce both neuronal and functional decline. To test this hypothesis, we inhibited vascular adhesion protein 1 (VAP-1), a membrane-bound protein expressed on the endothelial cell surface, that mediates leukocyte extravasation and induces oxidative stress. METHOD: We induced dopaminergic neuronal loss by infusing lipopolysaccharide (LPS) directly into the substantia nigra (SN) in rats and administered the VAP-1 inhibitor, PXS-4681A, daily. RESULTS: LPS produced: an acute inflammatory response, the loss of dopaminergic neurons in the SN, reduced the dopaminergic projection to SN target regions, particularly the dorsolateral striatum (DLS), and a deficit in habit learning, a key function of the DLS. In an attempt to protect SN neurons from this inflammatory response we found that VAP-1 inhibition not only reduced neutrophil infiltration in the SN and striatum, but also reduced the associated striatal microglia and astrocyte response. We found VAP-1 inhibition protected dopamine neurons in the SN, their projections to the striatum and promoted the functional recovery of habit learning. Thus, we reversed the loss of habitual actions, a function usually dependent on dopamine release in DLS and sensitive to striatal dysfunction. CONCLUSIONS: We establish, therefore, that VAP-1 inhibition has an anti-inflammatory profile that may be beneficial in the treatment of dopamine neuron dysfunction caused by an acute inflammatory state in the brain.

General Pavlovian-instrumental transfer tests reveal selective inhibition of the response type - whether Pavlovian or instrumental - performed during extinction.

The present experiments examined whether extinction of a stimulus predicting food affects the ability of that stimulus to energize instrumental performance to obtain food. We first used a general Pavlovian instrumental transfer (PIT) paradigm in which rats were first given Pavlovian conditioning with a stimulus predicting one type of food outcome and were then trained to lever press for a different food outcome. We found that the Pavlovian stimulus enhanced performance of the lever press response and that this enhancement was preserved after extinction of that stimulus (Experiment 1) even when the context was manipulated to favor the expression of extinction (Experiment 2). Next, we assessed whether extinction influenced the excitatory effect of a stimulus when it was trained as a discriminative stimulus. Extinction of this stimulus alone had no effect on its ability to control instrumental performance; however, when extinguished with its associated lever press response, discriminative control was lost (Experiments 3 and 4). Finally, after instrumental and Pavlovian training, we extinguished a Pavlovian stimulus predicting one food outcome with a lever press response that delivered a different outcome. In a general PIT test, we found this extinction abolished the ability of the Pavlovian stimulus to elevate responding on a lever trained with a different outcome, revealing for the first time that extinction can abolish the general PIT effect. We conclude that extinction can produce an inhibitory association between the stimulus and the general response type, whether Pavlovian or instrumental, performed during the extinction training.

Learning the structure of the world: The adaptive nature of state-space and action representations in multi-stage decision-making.

State-space and action representations form the building blocks of decision-making processes in the brain; states map external cues to the current situation of the agent whereas actions provide the set of motor commands from which the agent can choose to achieve specific goals. Although these factors differ across environments, it is currently unknown whether or how accurately state and action representations are acquired by the agent because previous experiments have typically provided this information a priori through instruction or pre-training. Here we studied how state and action representations adapt to reflect the structure of the world when such a priori knowledge is not available. We used a sequential decision-making task in rats in which they were required to pass through multiple states before reaching the goal, and for which the number of states and how they map onto external cues were unknown a priori. We found that, early in training, animals selected actions as if the task was not sequential and outcomes were the immediate consequence of the most proximal action. During the course of training, however, rats recovered the true structure of the environment and made decisions based on the expanded state-space, reflecting the multiple stages of the task. Similarly, we found that the set of actions expanded with training, although the emergence of new action sequences was sensitive to the experimental parameters and specifics of the training procedure. We conclude that the profile of choices shows a gradual shift from simple representations to more complex structures compatible with the structure of the world.

Models that learn how humans learn: The case of decision-making and its disorders.

Popular computational models of decision-making make specific assumptions about learning processes that may cause them to underfit observed behaviours. Here we suggest an alternative method using recurrent neural networks (RNNs) to generate a flexible family of models that have sufficient capacity to represent the complex learning and decision- making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task and, in this way, learns to imitate the processes underlying subjects' choices and their learning abilities. We demonstrate the benefits of this approach using a new dataset drawn from patients with either unipolar (n = 34) or bipolar (n = 33) depression and matched healthy controls (n = 34) making decisions on a two-armed bandit task. The results indicate that this new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects' choices. We show that the model can be interpreted using off-policy simulations and thereby provides a novel clustering of subjects' learning processes-something that often eludes traditional approaches to modelling and behavioural analysis.

Prefrontal Corticostriatal Disconnection Blocks the Acquisition of Goal-Directed Action.

The acquisition of goal-directed action requires encoding of the association between an action and its specific consequences or outcome. At a neural level, this encoding has been hypothesized to involve a prefrontal corticostriatal circuit involving the projection from the prelimbic cortex (PL) to the posterior dorsomedial striatum (pDMS); however, no direct evidence for this claim has been reported. In a series of experiments, we performed functional disconnection of this pathway using targeted lesions of the anterior corpus callosum to disrupt contralateral corticostriatal projections with asymmetrical lesions of the PL and/or pDMS to block plasticity in this circuit in rats. We first demonstrated that unilaterally blocking the PL input to the pDMS prevented the phosphorylation of extracellular signal-related kinase/mitogen activated protein kinase (pERK/pMAPK) induced by instrumental training. Next, we used a full bilateral disconnection of the PL from the pDMS and assessed goal-directed action using an outcome-devaluation test. Importantly, we found evidence that rats maintaining an ipsilateral and/or contralateral connection between the PL and the pDMS were able to acquire goal-directed actions. In contrast, bilateral PL-pDMS disconnection abolished the acquisition of goal-directed actions. Finally, we used a temporary pharmacological disconnection to disrupt PL inputs to the pDMS by infusing the NMDA antagonist dl-2-amino-5-phosphonopentanoic acid into the pDMS during instrumental training and found that this manipulation also disrupted goal-directed learning. These results establish that, in rats, the acquisition of new goal-directed actions depends on a prefrontal-corticostriatal circuit involving a connection between the PL and the pDMS.SIGNIFICANCE STATEMENT It has been hypothesized that the prelimbic cortex (PL) and posterior dorsomedial striatum (pDMS) in rodents interact in a corticostriatal circuit to mediate goal-directed learning. However, no direct evidence supporting this claim has been reported. Using targeted lesions, we performed functional disconnection of the PL-pDMS pathway to assess its role in goal-directed learning. In the first experiment, we demonstrated that PL input to the pDMS is necessary for instrumental training-induced neuronal activity. Next, we disrupted ipsilateral, contralateral, or bilateral PL-pDMS connections and found that only bilateral PL-pDMS disconnection disrupted the acquisition of goal-directed actions, a finding we replicated in our final study using a pharmacological disconnection procedure.

Open-field PET: Simultaneous brain functional imaging and behavioural response measurements in freely moving small animals.

A comprehensive understanding of how the brain responds to a changing environment requires techniques capable of recording functional outputs at the whole-brain level in response to external stimuli. Positron emission tomography (PET) is an exquisitely sensitive technique for imaging brain function but the need for anaesthesia to avoid motion artefacts precludes concurrent behavioural response studies. Here, we report a technique that combines motion-compensated PET with a robotically-controlled animal enclosure to enable simultaneous brain imaging and behavioural recordings in unrestrained small animals. The technique was used to measure in vivo displacement of [11C]raclopride from dopamine D2 receptors (D2R) concurrently with changes in the behaviour of awake, freely moving rats following administration of unlabelled raclopride or amphetamine. The timing and magnitude of [11C]raclopride displacement from D2R were reliably estimated and, in the case of amphetamine, these changes coincided with a marked increase in stereotyped behaviours and hyper-locomotion. The technique, therefore, allows simultaneous measurement of changes in brain function and behavioural responses to external stimuli in conscious unrestrained animals, giving rise to important applications in behavioural neuroscience.

From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning.

Considerable evidence suggests that the learning and performance of instrumental actions depend on activity in basal ganglia circuitry; however, these two functions have generally been considered independently. Whereas research investigating the associative mechanisms underlying instrumental conditioning has identified critical cortical and limbic input pathways to the dorsal striatum, the performance of instrumental actions has largely been attributed to activity in the dorsal striatal output pathways, with direct and indirect pathway projection neurons mediating action initiation, perseveration and cessation. Here, we discuss evidence that the dorsal striatal input and basal ganglia output pathways mediate the learning and performance of instrumental actions, respectively, with the dorsal striatum functioning as a transition point. From this perspective, the issue of how multiple striatal inputs are integrated at the level of the dorsal striatum and converted into relatively restricted outputs becomes one of critical significance for understanding how learning is translated into action. So too does the question of how learning signals are modulated by recent experience. We propose that this occurs through recurrent corticostriatothalamic feedback circuits that serve to integrate performance signals by updating ongoing action-related learning.

A new framework for conceptualizing symptoms in frontotemporal dementia: from animal models to the clinic.

Behavioural-variant frontotemporal dementia is characterized by a number of ostensibly disparate clinical features, which have largely been considered independently. This update proposes an integrated conceptual framework for these symptoms, by bringing together findings from animal studies, functional neuroimaging and behavioural neurology. The combined evidence indicates that many of the clinical symptoms--such as altered eating behaviour; overspending and susceptibility to scams; reduced empathy and socially inappropriate behaviour; apathy and stereotyped/ritualistic behaviour--can be conceptualized as a common underlying deficiency in goal-directed behaviour and the concomitant emergence of habits. This view is supported by similarities between the characteristic patterns of frontostriatal and insular atrophy in behavioural-variant frontotemporal dementia and the circuitry of homologous brain regions responsible for goal-directed and habitual behaviour in animals. Appreciating the impact of disturbance in goal-directed behaviour provides a new, integrated understanding of the common mechanisms underpinning prototypical clinical symptoms in behavioural-variant frontotemporal dementia. Furthermore, by drawing parallels between animal and clinical research, this translational approach has important implications for the development and evaluation of novel therapeutic treatments, from animal models through to behavioural interventions and clinical trials in humans.10.1093/brain/awy123_video1awy123media15796485557001.

Thalamic Control of Dorsomedial Striatum Regulates Internal State to Guide Goal-Directed Action Selection.

We (Bradfield et al., 2013) have demonstrated previously that parafascicular thalamic nucleus (PF)-controlled neurons in the posterior dorsomedial striatum (pDMS) are critical for interlacing new and existing action-outcome contingencies to control goal-directed action. Based on these findings, it was suggested that animals with a dysfunctional PF-pDMS pathway might suffer a deficit in creating or retrieving internal contexts or "states" on which such information could become conditional. To assess this hypothesis more directly, rats were given a disconnection treatment using contralateral cytotoxic lesions of the PF and pDMS (Group CONTRA) or ipsilateral control lesions (Group IPSI) and trained to press a right and left lever for sucrose and pellet outcomes, after which these contingencies were reversed. The rats were then given an outcome devaluation test (all experiments) and a test of outcome-specific reinstatement (Experiments 1 and 3). We found that devaluation performance was intact for both groups after training of initial contingencies, but impaired for Group CONTRA after reversal. However, performance was restored by additional reversal training. Furthermore, when tested a second time after reversal training, rats in both groups demonstrated responding in accordance with the original contingencies, providing direct evidence of modulation of action selection by state. Finally, we found that external context could substitute for internal state and so could rescue responding in Group CONTRA, but only in the reinstatement test. Together, these findings suggest that animals use internal state information to guide action selection and that this information is modulated by the PF-pDMS pathway.SIGNIFICANCE STATEMENT Individuals with Parkinson's disease dementia often suffer a characteristic deficit in "cognitive flexibility." It has been suggested that neurodegeneration in the pathway between the centromedian/parafascicular thalalmic nucleus (PF) and striatum might underlie such deficits (Smith et al., 2014). In rats, we have similarly observed that a functional disconnection of the PF-posterior dorsomedial striatal pathway produces a specific impairment in the ability to alter goal-directed actions (Bradfield et al., 2013). It was suggested that this impairment could be a result of a deficit in state modulation. Here, we present four experiments that provide evidence for this hypothesis and suggest several ways (e.g., extended practice, providing external cues) in which this state modulation can be rescued.

The Lateral Habenula and Its Input to the Rostromedial Tegmental Nucleus Mediates Outcome-Specific Conditioned Inhibition.

Animals can readily learn that stimuli predict the absence of specific appetitive outcomes; however, the neural substrates underlying such outcome-specific conditioned inhibition remain largely unexplored. Here, using female and male rats as subjects, we examined the involvement of the lateral habenula (LHb) and of its inputs onto the rostromedial tegmental nucleus (RMTg) in inhibitory learning. In these experiments, we used backward conditioning and contingency reversal to establish outcome-specific conditioned inhibitors for two distinct appetitive outcomes. Then, using the Pavlovian-instrumental transfer paradigm, we assessed the effects of manipulations of the LHb and the LHb-RMTg pathway on that inhibitory encoding. In control animals, we found that an outcome-specific conditioned inhibitor biased choice away from actions delivering that outcome and toward actions earning other outcomes. Importantly, this bias was abolished by both electrolytic lesions of the LHb and selective ablation of LHb neurons using Cre-dependent Caspase3 expression in Cre-expressing neurons projecting to the RMTg. This deficit was specific to conditioned inhibition; an excitatory predictor of a specific outcome-biased choice toward actions delivering the same outcome to a similar degree whether the LHb or the LHb-RMTg network was intact or not. LHb lesions also disrupted the ability of animals to inhibit previously encoded stimulus-outcome contingencies after their reversal, pointing to a critical role of the LHb and of its inputs onto the RMTg in outcome-specific conditioned inhibition in appetitive settings. These findings are consistent with the developing view that the LHb promotes a negative reward prediction error in Pavlovian conditioning.SIGNIFICANCE STATEMENT Stimuli that positively or negatively predict rewarding outcomes influence choice between actions that deliver those outcomes. Previous studies have found that a positive predictor of a specific outcome biases choice toward actions delivering that outcome. In contrast, a negative predictor of an outcome biases choice away from actions earning that outcome and toward other actions. Here we reveal that the lateral habenula is critical for negative predictors, but not positive predictors, to affect choice. Furthermore, these effects were found to require activation of lateral habenula inputs to the rostromedial tegmental nucleus. These results are consistent with the view that the lateral habenula establishes inhibitory relationships between stimuli and food outcomes and computes a negative prediction error in Pavlovian conditioning.

Substance P and dopamine interact to modulate the distribution of delta-opioid receptors on cholinergic interneurons in the striatum.

It has been recently demonstrated that predictive learning induces a persistent accumulation of delta-opioid receptors (DOPrs) at the somatic membrane of cholinergic interneurons (CINs) in the nucleus accumbens shell (Nac-S). This accumulation is required for predictive learning to influence subsequent choice between goal-directed actions. The current experiments investigated the local neurochemical events responsible for this translocation. We found that (1) local administration of substance P into multiple striatal sub-territories induced DOPr translocation and (2) that this effect was mediated by the NK1 receptor, likely through its expression on CINs. Interestingly, whereas intrastriatal infusion of the D1 agonist chloro-APB reduced the DOPr translocation on CINs and infusion of the D2 agonist quinpirole had no effect, co-administration of both agonists again generated DOPr translocation, suggesting the effect of the D1 agonist alone was due to receptor internalisation. In support of this, local administration of cocaine was found to increase DOPr translocation as was chloro-APB when co-administered with the DOPr antagonist naltrindole. These studies provide the first evidence of delta-opioid receptor translocation in striatal cholinergic interneurons outside of the accumbens shell and suggest that, despite differences in local striatal neurochemical microenvironments, a similar molecular mechanism - involving an interaction between dopamine and SP signalling via NK1R - regulates DOPr translocation in multiple striatal regions. To our knowledge, this represents a novel mechanism by which DOPr distribution is regulated that may be particularly relevant to learning-induced DOPr trafficking.

Inferring action-dependent outcome representations depends on anterior but not posterior medial orbitofrontal cortex.

Although studies examining orbitofrontal cortex (OFC) often treat it as though it were functionally homogeneous, recent evidence has questioned this assumption. Not only are the various subregions of OFC (lateral, ventral, and medial) hetereogeneous, but there is further evidence of heterogeneity within those subregions. For example, several studies in both humans and monkeys have revealed a functional subdivision along the anterior-posterior gradient of the medial OFC (mOFC). Given our previous findings suggesting that, in rats, the mOFC is responsible for inferring the likelihood of unobservable action outcomes (Bradfield, Dezfouli, van Holstein, Chieng, & Balleine, 2015), and given the anterior nature of the placements of our prior manipulations, we decided to assess whether the rat mOFC also differs in connection and function along its anteroposterior axis. We first used retrograde tracing to compare the density of efferents from mOFC to several structures known to contribute to goal-directed action: the mediodorsal thalamus, basolateral amygdala, posterior dorsomedial striatum, nucleus accumbens core and ventral tegmental area. We then compared the functional effects of anterior versus posterior mOFC excitotoxic lesions on tests of Pavlovian-instrumental transfer, instrumental outcome devaluation and outcome-specific reinstatement. We found evidence that the anterior mOFC had greater connectivity with the accumbens core and greater functional involvement in goal-directed action than the posterior mOFC. Consistent with previous findings across species, therefore, these results suggest that the anterior and posterior mOFC of the rat are indeed functionally distinct, and that it is the anterior mOFC that is particularly critical for inferring unobservable action outcomes.

Motivational state controls the prediction error in Pavlovian appetitive-aversive interactions.

Contemporary theories of learning emphasize the role of a prediction error signal in driving learning, but the nature of this signal remains hotly debated. Here, we used Pavlovian conditioning in rats to investigate whether primary motivational and emotional states interact to control prediction error. We initially generated cues that positively or negatively predicted an appetitive food outcome. We then assessed how these cues modulated aversive conditioning when a novel cue was paired with a foot shock. We found that a positive predictor of food enhances, whereas a negative predictor of that same food impairs, aversive conditioning. Critically, we also showed that the enhancement produced by the positive predictor is removed by reducing the value of its associated food. In contrast, the impairment triggered by the negative predictor remains insensitive to devaluation of its associated food. These findings provide compelling evidence that the motivational value attributed to a predicted food outcome can directly control appetitive-aversive interactions and, therefore, that motivational processes can modulate emotional processes to generate the final error term on which subsequent learning is based.

Methamphetamine promotes habitual action and alters the density of striatal glutamate receptor and vesicular proteins in dorsal striatum.

Goal-directed actions are controlled by the value of the consequences they produce and so increase when what they produce is valuable and decrease when it is not. With continued invariant practice, however, goal-directed actions can become habits, controlled not by their consequences but by antecedent, reward-related states and stimuli. Here, we show that pre-exposure to methamphetamine (METH) caused abnormally rapid development of habitual control. Furthermore, these drug-induced habits differed strikingly from conventional habits; we found that they were insensitive both to changes in reward value and to the effects of negative feedback. In addition to these behavioral changes, METH exposure produced bidirectional changes to synaptic proteins in the dorsal striatum. In the dorsomedial striatum, a structure critical for goal-directed action, METH exposure was associated with a reduction in glutamate receptor and glutamate vesicular proteins, whereas in the dorsolateral striatum, a region that has previously been implicated in habit learning, there was an increase in these proteins. Together, these results indicate that METH exposure promotes habitual control of action that appears to be the result of bidirectional changes in glutamatergic transmission in the circuits underlying goal-directed and habit-based learning.

Consolidation of Goal-Directed Action Depends on MAPK/ERK Signaling in Rodent Prelimbic Cortex.

The prelimbic prefrontal cortex (PL) has consistently been found to be necessary for the acquisition of goal-directed actions in rodents. Nevertheless, the specific cellular processes underlying this learning remain unknown. We assessed changes in learning-related expression of mitogen-activated protein kinase/extracellular signal-related kinase (MAPK/ERK1/2) phosphorylation (pERK) in layers 2-3 and 5-6 of the anterior and posterior PL and in the population of neurons projecting to posterior dorsomedial striatum (pDMS), also implicated in goal-directed learning. Rats were given either a single session of training to press a lever for a pellet reward or yoked reward deliveries without instrumental training and assessed 5 or 60 min after training for pERK expression. Relative to yoked training, instrumental training produced an increase in pERK expression in all regions of the PL both at 5 and 60 min, and this was accompanied by an increase in nuclear pERK expression in the posterior PL in rats given instrumental training. pDMS-projecting neurons showed a transient increase in pERK expression in posterior layer 5-6 projection neurons after 5 min, and a delayed increase in anterior layer 2-3 neurons after 60 min, suggesting that ERK expression in the PL is necessary for the consolidation of goal-directed learning. Consistent with this claim, we found that rats trained on two lever press actions for distinct outcomes and then infused with the MEK inhibitor PD98059 into the PL immediately after training failed to acquire specific action-outcome associations, suggesting that persistent pERK signaling in the PL is necessary for goal-directed learning. SIGNIFICANCE STATEMENT: The prelimbic cortex is implicated in goal-directed learning in rodents; however, it is unclear whether it is involved in the consolidation of this learning, and what cellular processes are involved. We used pERK as a marker of activity-related synaptic plasticity to assess learning-induced changes in distinct layers and neuronal populations of the prelimbic prefrontal cortex (PL). Training produced long-lasting upregulation of pERK throughout the PL and specifically within neurons that project to the pDMS, another region critical for goal-directed learning. Next, we demonstrated that pERK signaling in the PL was necessary for the consolidation of goal-directed learning. Together, these results indicate that instrumental training induces ERK signaling in distinct layers and populations in the PL and this signaling underlies the consolidation of goal-directed learning.

Pulling habits out of rats: adenosine 2A receptor antagonism in dorsomedial striatum rescues meth-amphetamine-induced deficits in goal-directed action.

Addiction is characterized by a persistent loss of behavioral control resulting in insensitivity to negative feedback and abnormal decision-making. Here, we investigated the influence of methamphetamine (METH)-paired contextual cues on decision-making in rats. Choice between goal-directed actions was sensitive to outcome devaluation in a saline-paired context but was impaired in the METH-paired context, a deficit that was also found when negative feedback was provided. Reductions in c-Fos-related immunoreactivity were found in dorsomedial striatum (DMS) but not dorsolateral striatum after exposure to the METH context suggesting this effect reflected a loss specifically in goal-directed control in the METH context. This reduction in c-Fos was localized to non-enkephalin-expressing neurons in the DMS, likely dopamine D1-expressing direct pathway neurons, suggesting a relative change in control by the D1-direct versus D2-indirect pathways originating in the DMS may have been induced by METH-context exposure. To test this suggestion, we infused the adenosine 2A receptor antagonist ZM241385 into the DMS prior to test to reduce activity in D2 neurons relative to D1 neurons in the hope of reducing the inhibitory output from this region of the striatum. We found that this treatment fully restored sensitivity to negative feedback in a test conducted in the METH-paired context. These results suggest that drug exposure alters decision-making by downregulation of the circuitry mediating goal-directed action, an effect that can be ameliorated by acute A2A receptor inhibition in this circuit.

Intermittent feeding alters sensitivity to changes in reward value.

The influence of binge-like feeding schedules on subsequent food-related behavior is not well understood. We investigated the effect of repeated cycles of restriction and refeeding on two food-related behaviors; goal-directed responding for a palatable food reward and sensory-specific satiety. Hungry rats were trained to perform two instrumental actions for two distinct food outcomes and were then subjected to repeated cycles of restricted and unrestricted access to their maintenance chow for 30-days or were maintained on food restriction. Goal-directed control was then assessed using specific satiety-induced outcome devaluation. Rats were given 1 h access to one of theoutcomes and were then immediately given a choice between the two actions. Rats maintained on restriction responded more for the valued than the devalued reward but rats with a history of restriction and refeeding failed to show this effect. Importantly, all rats showed sensory-specific satiety when offered a choice between the two foods, indicating that pre-feeding selectively reduced the value of the pre-fed food. By contrast, sensory-specific satiety was not observed in rats with a history of intermittent feeding when the foods were offered sequentially. These results indicate that, similar to calorically dense diets, intermittent feeding patterns can impair the performance of goal-directed actions as well as the ability to reject a pre-fed food when it is offered alone.