From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning.

Considerable evidence suggests that the learning and performance of instrumental actions depend on activity in basal ganglia circuitry; however, these two functions have generally been considered independently. Whereas research investigating the associative mechanisms underlying instrumental conditioning has identified critical cortical and limbic input pathways to the dorsal striatum, the performance of instrumental actions has largely been attributed to activity in the dorsal striatal output pathways, with direct and indirect pathway projection neurons mediating action initiation, perseveration and cessation. Here, we discuss evidence that the dorsal striatal input and basal ganglia output pathways mediate the learning and performance of instrumental actions, respectively, with the dorsal striatum functioning as a transition point. From this perspective, the issue of how multiple striatal inputs are integrated at the level of the dorsal striatum and converted into relatively restricted outputs becomes one of critical significance for understanding how learning is translated into action. So too does the question of how learning signals are modulated by recent experience. We propose that this occurs through recurrent corticostriatothalamic feedback circuits that serve to integrate performance signals by updating ongoing action-related learning.

Thalamic Control of Dorsomedial Striatum Regulates Internal State to Guide Goal-Directed Action Selection.

We (Bradfield et al., 2013) have demonstrated previously that parafascicular thalamic nucleus (PF)-controlled neurons in the posterior dorsomedial striatum (pDMS) are critical for interlacing new and existing action-outcome contingencies to control goal-directed action. Based on these findings, it was suggested that animals with a dysfunctional PF-pDMS pathway might suffer a deficit in creating or retrieving internal contexts or "states" on which such information could become conditional. To assess this hypothesis more directly, rats were given a disconnection treatment using contralateral cytotoxic lesions of the PF and pDMS (Group CONTRA) or ipsilateral control lesions (Group IPSI) and trained to press a right and left lever for sucrose and pellet outcomes, after which these contingencies were reversed. The rats were then given an outcome devaluation test (all experiments) and a test of outcome-specific reinstatement (Experiments 1 and 3). We found that devaluation performance was intact for both groups after training of initial contingencies, but impaired for Group CONTRA after reversal. However, performance was restored by additional reversal training. Furthermore, when tested a second time after reversal training, rats in both groups demonstrated responding in accordance with the original contingencies, providing direct evidence of modulation of action selection by state. Finally, we found that external context could substitute for internal state and so could rescue responding in Group CONTRA, but only in the reinstatement test. Together, these findings suggest that animals use internal state information to guide action selection and that this information is modulated by the PF-pDMS pathway.SIGNIFICANCE STATEMENT Individuals with Parkinson's disease dementia often suffer a characteristic deficit in "cognitive flexibility." It has been suggested that neurodegeneration in the pathway between the centromedian/parafascicular thalalmic nucleus (PF) and striatum might underlie such deficits (Smith et al., 2014). In rats, we have similarly observed that a functional disconnection of the PF-posterior dorsomedial striatal pathway produces a specific impairment in the ability to alter goal-directed actions (Bradfield et al., 2013). It was suggested that this impairment could be a result of a deficit in state modulation. Here, we present four experiments that provide evidence for this hypothesis and suggest several ways (e.g., extended practice, providing external cues) in which this state modulation can be rescued.

Aging-Related Dysfunction of Striatal Cholinergic Interneurons Produces Conflict in Action Selection.

For goal-directed action to remain adaptive, new strategies are required to accommodate environmental changes, a process for which parafascicular thalamic modulation of cholinergic interneurons in the striatum (PF-to-CIN) appears critical. In the elderly, however, previously acquired experience frequently interferes with new learning, yet the source of this effect has remained unexplored. Here, combining sophisticated behavioral designs, cell-specific manipulation, and extensive neuronal imaging, we investigated the involvement of the PF-to-CIN pathway in this process. We found functional alterations of this circuit in aged mice that were consistent with their incapacity to update initial goal-directed learning, resulting in faulty activation of projection neurons in the striatum. Toxicogenetic ablation of CINs in young mice reproduced these behavioral and neuronal defects, suggesting that age-related deficits in PF-to-CIN function reduce the ability of older individuals to resolve conflict between actions, likely contributing to impairments in adaptive goal-directed action and executive control in aging. VIDEO ABSTRACT.

Ventral pallidal projections to mediodorsal thalamus and ventral tegmental area play distinct roles in outcome-specific Pavlovian-instrumental transfer.

Outcome-specific Pavlovian-instrumental transfer (PIT) demonstrates the way that reward-related cues influence choice between instrumental actions. The nucleus accumbens shell (NAc-S) contributes critically to this effect, particularly through its output to the rostral medial ventral pallidum (VP-m). Using rats, we investigated in two experiments the role in the PIT effect of the two major outputs of this VP-m region innervated by the NAc-S, the mediodorsal thalamus (MD) and the ventral tegmental area (VTA). First, two retrograde tracers were injected into the MD and VTA to compare the neuronal activity of the two populations of projection neurons in the VP-m during PIT relative to controls. Second, the functional role of the connection between the VP-m and the MD or VTA was assessed using asymmetrical pharmacological manipulations before a PIT test. It was found that, whereas neurons in the VP-m projecting to the MD showed significantly more neuronal activation during PIT than those projecting to the VTA, neuronal activation of these latter neurons correlated with the size of the PIT effect. Disconnection of the two pathways during PIT also revealed different deficits in performance: disrupting the VP-m to MD pathway removed the response biasing effects of reward-related cues, whereas disrupting the VP-m to VTA pathway preserved the response bias but altered the overall rate of responding. The current results therefore suggest that the VP-m exerts distinct effects on the VTA and MD and that these latter structures mediate the motivational and cognitive components of specific PIT, respectively.

Thalamocortical integration of instrumental learning and performance and their disintegration in addiction.

A recent focus of addiction research has been on the effect of drug exposure on the neural processes that mediate the acquisition and performance of goal-directed instrumental actions. Deficits in goal-directed control and a consequent dysregulation of habit learning processes have been described as resulting in compulsive drug seeking. Similarly, considerable research has focussed on the motivational and emotional changes that drugs produce and that result in changes in the incentive processes that modulate goal-directed performance. Although these areas have developed independently, we argue that the effects they described are likely not independent. Here we hypothesize that these changes result from a core deficit in the way the learning and performance factors that support goal-directed action are integrated at a neural level to maintain behavioural control. A dorsal basal ganglia stream mediating goal-directed learning and a ventral stream mediating various performance factors find several points of integration in the cortical basal ganglia system, most notably in the thalamocortical network linking basal ganglia output to a variety of cortical control centres. Recent research in humans and other animals is reviewed suggesting that learning and performance factors are integrated in a network centred on the mediodorsal thalamus and that disintegration in this network may provide the basis for a 'switch' from recreational to dysregulated drug seeking resulting in the well documented changes associated with addiction.

The thalamostriatal pathway and cholinergic control of goal-directed action: interlacing new with existing learning in the striatum.

The capacity for goal-directed action depends on encoding specific action-outcome associations, a learning process mediated by the posterior dorsomedial striatum (pDMS). In a changing environment, plasticity has to remain flexible, requiring interference between new and existing learning to be minimized, yet it is not known how new and existing learning are interlaced in this way. Here we investigated the role of the thalamostriatal pathway linking the parafascicular thalamus (Pf) with cholinergic interneurons (CINs) in the pDMS in this process. Removing the excitatory input from Pf to the CINs was found to reduce the firing rate and intrinsic activity of these neurons and produced an enduring deficit in goal-directed learning after changes in the action-outcome contingency. Disconnection of the Pf-pDMS pathway produced similar behavioral effects. These data suggest that CINs reduce interference between new and existing learning, consistent with claims that the thalamostriatal pathway exerts state control over learning-related plasticity.

Resolution of conflict between goal-directed actions: outcome encoding and neural control processes.

According to O-R theory of instrumental learning, incongruent biconditional discriminations should be impossible to solve in a goal-directed manner because the event acting as the outcome of one response also acts as a discriminative stimulus for an opposite response. Each event should therefore be associated with two competing responses. However, Dickinson and de Wit (2003) have presented evidence that rats can learn incongruent discriminations. The present study investigated whether rats were able to engage additional processes to solve incongruent discriminations in a goal-directed manner. Experiment 1 provides evidence that rats resolve the response conflict that arises in the incongruent discrimination by differentially encoding events in their roles as discriminative stimulus and as outcome. Furthermore, Experiment 2 shows that once goal-directed control has been established the dorsomedial prefrontal cortex is not directly involved in its maintenance but rather plays a central role in conflict resolution processes.

Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning.

Although there is consensus that instrumental conditioning depends on the encoding of action-outcome associations, it is not known where this learning process is localized in the brain. Recent research suggests that the posterior dorsomedial striatum (pDMS) may be the critical locus of these associations. We tested this hypothesis by examining the contribution of N-methyl-D-aspartate receptors (NMDARs) in the pDMS to action-outcome learning. Rats with bilateral cannulae in the pDMS were first trained to perform two actions (left and right lever presses), for sucrose solution. After the pre-training phase, they were given an infusion of the NMDA antagonist 2-amino-5-phosphonopentanoic acid (APV, 1 mg/mL) or artificial cerebral spinal fluid (ACSF) before a 30-min session in which pressing one lever delivered food pellets and pressing the other delivered fruit punch. Learning during this session was tested the next day by sating the animals on either the pellets or fruit punch before assessing their performance on the two levers in extinction. The ACSF group selectively reduced responding on the lever that, in training, had earned the now devalued outcome, whereas the APV group did not. Experiment 2 replicated the effect of APV during the critical training session but found no effect of APV given after acquisition and before test. Furthermore, Experiment 3 showed that the effect of APV on instrumental learning was restricted to the pDMS; infusion into the dorsolateral striatum did not prevent learning. These experiments provide the first direct evidence that, in instrumental conditioning, NMDARs in the dorsomedial striatum are involved in encoding action-outcome associations.

Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning.

Habits are controlled by antecedent stimuli rather than by goal expectancy. Interval schedules of feedback have been shown to generate habits, as revealed by the insensitivity of behaviour acquired under this schedule to outcome devaluation treatments. Two experiments were conducted to assess the role of the dorsolateral striatum in habit learning. In Experiment 1, sham operated controls and rats with dorsolateral striatum lesions were trained to press a lever for sucrose under interval schedules. After training, the sucrose was devalued by inducing taste aversion to it using lithium chloride, whereas saline injections were given to the controls. Only rats given the devaluation treatment reduced their consumption of sucrose and this reduction was similar in both the sham and the lesioned groups. All rats were then returned to the instrumental chamber for an extinction test, in which the lever was extended but no sucrose was delivered. In contrast to sham operated controls, rats with dorsolateral striatum lesions refrained from pressing the lever if the outcome was devalued. To assess the specificity of the role of dorsolateral striatum in this effect a second experiment was conducted in which a group with lesions of dorsomedial striatum was added. In relation now to both the sham and the dorsomedial lesioned groups, only rats with lesions of dorsolateral striatum significantly reduced responding after outcome devaluation. In conclusion, this study provides direct evidence that the dorsolateral striatum is necessary for habit formation. Furthermore, it suggests that, when the habit system is disrupted, control over instrumental performance reverts to the system controlling the performance of goal-directed instrumental actions.

Helplessness and escape performance: glutamate-adenosine interactions in the frontal cortex.

Adenosine has been implicated as a proximate mediator of escape deficits in the learned helplessness paradigm, suggesting that neuronal overactivation-a typical precursor to adenosine release-precedes the inescapable shock-induced impairment (T. R. Minor, W. C. Chang, & J. L. Winslow, 1994). In the present experiments, glutamate (100 microg) injection into the rat frontal cortex produced a deficit in escape performance. Pretest treatment with the adenosine receptor antagonist caffeine (7 mg/kg ip) reversed the effect of glutamate when infused 1 hr. but not 72 hr, after glutamate injection. Finally, microinjection of 2-amino-5-phosphonovaleric acid (5 ng) into the frontal cortex prior to inescapable shock prevented the escape deficit. These findings are consistent with the involvement of N-methyl-D-aspartate receptor activation in the frontal cortex in the helplessness effect.