Reward versus motoric activations in nucleus accumbens core of rats during Pavlovian conditioning.

The nucleus accumbens (NAc) core plays an important role in processing of events related to food reward, such as conditioned cues, approach or consumption. Nonetheless, there is lack of clarity regarding whether NAc core processes these separable events differently. We used the high temporal and spatial resolution of single unit recording with trial-by-trial video analysis to examine firing during three distinct categories termed cue, approach and consumption in a Pavlovian task. We had three goals. First, we sought to precisely define task-related behaviour in terms of distinct phases, in order to compare neural activity between motorically matched behaviours. We found that cue-evoked firing did not distinguish between trials on which animals initiated an approach versus ones on which they did not. Firing associated with consumption was greater than firing associated with motorically similar uncued head entry, indicating that previously reported decreases in NAc core firing during consumption relative to approach or baseline may reflect differences in motor behaviour. Secondly, we assessed changes in firing over the course of training. We found that NAc core neurons acquired a response to the tone cue within three sessions but did not change further across 10 total sessions. Thirdly, we correlated individual neuron firing during a given event with its firing during the same event on subsequent sessions. We found substantial variation in processing of cue and approach but not consumption, indicating that a given neuron may process certain events differently from session to session, while maintaining more stable processing of appetitive reward.

NMDA receptor-dependent plasticity in the nucleus accumbens connects reward-predictive cues to approach responses.

Learning associations between environmental cues and rewards is a fundamental adaptive function. Via such learning, reward-predictive cues come to activate approach to locations where reward is available. The nucleus accumbens (NAc) is essential for cued approach behavior in trained subjects, and cue-evoked excitations in NAc neurons are critical for the expression of this behavior. Excitatory synapses within the NAc undergo synaptic plasticity that presumably contributes to cued approach acquisition, but a direct link between synaptic plasticity within the NAc and the development of cue-evoked neural activity during learning has not been established. Here we show that, with repeated cue-reward pairings, cue-evoked excitations in the NAc emerge and grow in the trials prior to the detectable expression of cued approach behavior. We demonstrate that the growth of these signals requires NMDA receptor-dependent plasticity within the NAc, revealing a neural mechanism by which the NAc participates in learning of conditioned reward-seeking behaviors.

The role of the nucleus accumbens in learned approach behavior diminishes with training.

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Expanding the role of striatal cholinergic interneurons and the midbrain dopamine system in appetitive instrumental conditioning.

The basal ganglia are a collection of subcortical nuclei thought to underlie a wide variety of vertebrate behavior. Although a great deal is known about the functional and physiological properties of the basal ganglia, relatively few models have been formally developed that have been tested against both behavioral and physiological data. Our previous work (Ashby FG, Crossley MJ. J Cogn Neurosci 23: 1549-1566, 2011) showed that a model grounded in the neurobiology of the basal ganglia could account for basic single-neuron recording data, as well as behavioral phenomena such as fast reacquisition that constrain models of conditioning. In this article we show that this same model accounts for a variety of appetitive instrumental conditioning phenomena, including the partial reinforcement extinction (PRE) effect, rapid and slowed reacquisition following extinction, and renewal of previously extinguished instrumental responses by environmental context cues.

Effects of varying reinforcement probability on pavlovian approach behavior and ultrasonic vocalizations in rats.

Long Evans rats (n=32) were trained for 2 weeks to respond to an auditory conditioned stimulus (CS) which signaled the delivery of a 20% sucrose unconditioned stimulus (US) with varying probabilities. Animals were randomly assigned to 1 of 4 groups. In the control groups, the CS signaled sucrose delivery with equal probabilities across two weeks, at 100% (Group 100-100) and 25% (Group 25-25) respectively. In the experimental groups (Group 100-25) and (Group 25-100), sucrose probabilities were switched between weeks 1 and 2. Three behavioral measures were recorded: latency to enter the sucrose port upon CS presentation, head entries throughout the session and ultrasonic vocalizations. The results suggest that all groups formed associations between the CS and US, as evidenced by a decrease in latency to respond to the CS across days. The experimental groups were also able to detect when sucrose probability changed, as evidenced by Group 25-100's increase in head entries, to the level of Group 100-100 in week 2, and Group 100-25's decrease in head entries, to the level of Group 25-25 in week 2. Group 100-25 also produced an increase in "22 kHz" ultrasonic vocalizations following the downshift on the first day of week 2. The increase in this ultrasonic frequency range, which is associated with negative affect in rats, preceded both the decrease in head entries and the increase in missed trials, consistent with a multistage model of behaviors resulting from US probability reduction.

Effects of cortical and striatal dopamine D1 receptor blockade on cued versus noncued behavioral responses.

Various lines of evidence suggest that disruptions in brain dopamine (DA) transmission produce behavioral impairments that can be overcome by salient response-eliciting environmental stimuli. We examined here whether D1 receptor blockade within striatal or frontal cortical DA target regions would differentially affect head entry responses elicited by an auditory cue compared with those occurring during noncued intertrial intervals. Rats received 2 drug-free 28-trial daily sessions in which an auditory cue was immediately followed by food delivery. On the following day, separate groups of rats received bilateral infusions of D1 antagonist SCH23390 to the dorsomedial striatum (DMS), nucleus accumbens (NAcc) core, or the medial prefrontal cortex (mPFC). SCH23390 infused into the DMS and NAcc core suppressed noncued head entries but had no effect on head entries in response to the auditory cue. SCH23390 infused to the mPFC did not reduce either cued or noncued approach responses. Systemic administration of the drug, in contrast, reduced the frequency of both cued and noncued approaches. The results are consistent with the notion that has emerged from the Parkinson's literature that reduced DA transmission produces behavioral suppression that can be overcome by salient environmental response elicitors, and extends this notion by showing that D1 receptor transmission within the striatum strongly suppresses noncued responses while leaving the identical behavior intact when cued by an environmental stimulus.

Cortical and basal ganglia contributions to habit learning and automaticity.

In the 20th century it was thought that novel behaviors are mediated primarily in cortex and that the development of automaticity is a process of transferring control to subcortical structures. However, evidence supports the view that subcortical structures, such as the striatum, make significant contributions to initial learning. More recently, there has been increasing evidence that neurons in the associative striatum are selectively activated during early learning, whereas those in the sensorimotor striatum are more active after automaticity has developed. At the same time, other recent reports indicate that automatic behaviors are striatum- and dopamine-independent, and might be mediated entirely within cortex. Resolving this apparent conflict should be a major goal of future research.

Dopamine D1 and D2 antagonist effects on response likelihood and duration.

Experimentally induced and parkinsonian disruptions in dopamine (DA) transmission are associated with motor abnormalities that include a reduced likelihood of behavioral response initiation and an increased duration of executed responses. Here we investigated the dopamine receptor subtypes involved in regulating these two aspects of behavior. We examined the effects of D1 family (D1/D5) antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390; 0, 0.04, 0.08, or 0.16 mg/kg) and D2/D3 antagonist 3,5-dichloro-N-(1-ethylpyrrolidin-2-ylmethyl)-2-hydroxy-6-methoxybenzamide (+)-tartrate salt (raclopride; 0, 0.2, or 0.4 mg/kg) on the likelihood and duration of a cued Pavlovian approach and a cued operant lever-press response. While the high doses of the D1 and D2 antagonists produced similar levels of overall locomotor suppression, only the D2 antagonist increased the duration of time that animals' heads remained in the food compartment during both Pavlovian and operant task performance. In contrast, D1 antagonist SCH23390 decreased the proportion of trials in which animals executed both the Pavlovian approach and operant lever-press, while raclopride did not. The results suggest that D2 receptor blockade preferentially increases response duration, and, under the simple discrete-trial procedures employed here, D1 receptor blockade preferential reduces Pavlovian and operant response likelihood.

Stimulus-response and response-outcome learning mechanisms in the striatum.

While midbrain DA neurons show phasic activations in response to both reward-predicting and salient non-reward events, activation responses to primary and conditioned rewards are sustained for several hundreds of milliseconds beyond those elicited by salient non-reward-related stimuli. The longer-duration DA reward response and corresponding elevated DA release in striatal target sites may selectively strengthen currently-active corticostriatal synapses, i.e., those associated with the successful reward-procuring behavior. This paper describes how similar models of DA-mediated plasticity of corticostriatal synapses may describe both stimulus-response and response-outcome learning. DA-mediated strengthening of corticostriatal synapses in regions of the dorsolateral striatum receiving afferents from primary sensorimotor cortex is likely to bind corticostriatal inputs representing the previously-emitted movement to striatal outputs contributing to the selection of the next movement segment in a behavioral sequence. Within the striatum, more generally, inputs from distinct regions of the frontal cortex that code independently for movement direction and reward expectation send convergent projections to striatal output cells. DA-mediated strengthening of active corticostriatal synapses promotes the future output of the striatal cell under similar input conditions. This is postulated to promote persistence of neuronal activity in the very cortical cells that drive corticostriatal input, leading to the establishment of sustained reverberatory loops that permit cortical movement-related cells to maintain activity until the appropriate time of movement initiation.

Dopaminergic mechanisms in actions and habits.

Recent studies suggest new ways to interpret dopaminergic actions in goal-directed performance and habitual responding. In the early stages of learning dopamine plays an essential role, but with extended training dopamine appears to play a decreasing role in response expression. Experimental manipulation of dopamine levels alters the correlation of cortical and striatal neural activity in behaving animals, and these dopamine-dependent changes in corticostriatal correlations may be reflected in changes in action selection in the basal ganglia. Consistent with this hypothesis, changes in dopamine signaling brought about by sensitization with amphetamine mimic the transition from goal-directed to habit-based instrumental performance. At the cellular level, dopamine-dependent synaptic plasticity may be important initially, and subsequently lead to more persistent changes that no longer require dopamine. The locus of these actions within the cortical and corticostriatal circuitry is a focus on ongoing research.

A "good parent" function of dopamine: transient modulation of learning and performance during early stages of training.

While extracellular dopamine (DA) concentrations are increased by a wide category of salient stimuli, there is evidence to suggest that DA responses to primary and conditioned rewards may be distinct from those elicited by other types of salient events. A reward-specific mode of neuronal responding would be necessary if DA acts to strengthen behavioral response tendencies under particular environmental conditions or to set current environmental inputs as goals that direct approach responses. As described in this review, DA critically mediates both the acquisition and expression of learned behaviors during early stages of training, however, during later stages, at least some forms of learned behavior become independent of (or less dependent upon) DA transmission for their expression.

Amphetamine affects the start of responding in the peak interval timing task.

In this paper we investigate how amphetamine affects performance in a PI task by comparing two analyses of responding during peak trials. After training on 24 s fixed interval (FI-24) with 96 s peak trials, rats were given amphetamine for 4 consecutive days at doses of .5 and 1.0 mg/kg. Responses during peak trials were fitted with a Gaussian distribution to estimate the expected time of reinforcement from the peak time. A single trials analysis was also performed to determine the start time and stop time of the transition into and out of a high rate of responding on each peak trial. Amphetamine significantly decreased peak times as measured with the Gaussian curve fitting. However, in the single trials analysis, animals initiated responding significantly earlier, but did not stop responding earlier. Thus, fitting a Gaussian to the average performance across trials sometimes provides a different characterization of the timing process than does analyzing the start and stop of responding on individual trials. In the current experiment, the latter approach provided a more precise characterization of the effects of amphetamine on response timing.

Dopamine receptor blockade and extinction differentially affect behavioral variability.

Dopamine (DA) neurons respond to unexpected food delivery and are inhibited during the omission of expected reward. DA receptor blockade mimics some, but not all, aspects of non-reward (extinction) conditions. It was therefore of interest to ask whether DA receptor blockade produces extinction-like increases in behavioral variability in addition to its well-known operant response-suppressing effects. In the current experiment, rats were trained drug-free on an operant task in which they pressed on a keyboard. Two of the keys led to food on a continuous reinforcement schedule. Both response rates and behavioral variability were measured. Test day administration of D(1) and D(2) antagonists SCH23390 and raclopride, like extinction, suppressed responding but, unlike extinction, did not lead to an increase in variability.

Visual search deficits in Parkinson's disease are attenuated by bottom-up target salience and top-down information.

Patients with Parkinson's disease (PD), a degenerative disorder primarily affecting the nigrostriatal dopamine system, exhibit deficits in selecting task-relevant stimuli in the presence of irrelevant stimuli, such as in visual search tasks. However, results from previous studies suggest that these deficits may vary as a function of whether selection must rely primarily on the "bottom-up" salience of the target relative to background stimuli, or whether "top-down" information about the identity of the target is available to bias selection. In the present study, moderate-to-severe medicated PD patients and age-matched controls were tested on six visual search tasks that systematically varied the relationship between bottom-up target salience (feature search, noisy feature search, conjunction search) and top-down target knowledge (Target Known versus Target Unknown). Comparison of slope and intercepts of the RT x set size function provided information about the efficiency of search and non-search (e.g., decision, response) components, respectively. Patients exhibited higher intercepts than controls as bottom-up target salience decreased, however these deficits were disproportionately larger under Target Unknown compared to Target Known conditions. Slope differences between PD and controls were limited to the Target Unknown Conjunction condition, where patients exhibited a shallower slope in the target absent condition, indicating that they terminated search earlier. These results suggest that under conditions of high background noise, medicated PD patients were primarily impaired in decision and/or response processes downstream from the target search itself, and that the deficit was attenuated when top-down information was available to guide selection of the target signal.

Extended habit training reduces dopamine mediation of appetitive response expression.

A wide range of behaviors is impaired after disruption of dopamine (DA) transmission, yet behaviors that are reflexive, automatic, or elicited by salient cues often remain intact. Responses triggered by strong external cues appear to be DA independent. Here, we examined the possibility that a single behavior may become DA independent as a result of extended training. Rats were trained to execute a head-entry response to a cue signaling food delivery. Vulnerability of the response to D1 or D2 receptor blockade was assessed on day 3, 7, or 17 of 28-trial-per-day training. During the early stages of training, the D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH 23390) increased response latencies; however, the same behavior was unaffected by SCH 23390 in animals tested during the later stages of training. Other aspects of behavior such as locomotion and head-entry responses during the uncued intertrial interval remained vulnerable to SCH 23390 throughout the experiment. This D1-mediated response was unaffected by the D2 antagonist raclopride, even at a dose that strongly suppressed locomotion. The results provide strong evidence that a D1-dependent behavior becomes less dependent on DA with extended training. A number of fundamental neurobiological changes occur as behaviors become learned habits; at least for some responses, this change involves a shift from D1-mediated to D1-independent responding.

Maintenance of response readiness in patients with Parkinson's disease: evidence from a simple reaction time task.

The authors explored the effect of Parkinson's disease (PD) on the generation and maintenance of response readiness in a simple reaction time task. They compared performance of idiopathic PD patients without dementia, age-matched controls, and younger controls over short (1-, 3-, and 6-s) and long (12- and 18-s) foreperiod intervals. After each trial, the authors probed memory for visual information that also had to be maintained during the trial interval. Patients and controls did not differ overall in their ability to maintain readiness over long delays. However, within the PD group only, errors in maintaining visual information were correlated with difficulty in maintaining readiness, suggesting that systems impaired in PD may facilitate the maintenance of processing in both motor and cognitive domains.

Effects of dopamine antagonists on the timing of two intervals.

Rats were trained on a two-interval (12 and 36 s) temporal production task (the peak procedure). Test sessions were conducted in which either the D(1) antagonist SCH-23390 (SCH; 0.02, 0.04, 0.06 mg/kg) or the D(2) antagonist haloperidol (HAL; 0.05, 0.1, 0.2 mg/kg) were injected prior to testing. Both drugs affected the amount of responding, but only HAL affected timing. Under HAL, both intervals were overestimated, consistent with a HAL-induced decrease in clock speed. Drug-induced decreases in response output were more profound for the long interval than the short. In addition, there was evidence of HAL- and SCH-induced delays in response initiation that were more severe for the long interval, perhaps owing to its status as a weaker conditioned stimulus.

Opposing roles of D1 and D2 receptors in appetitive conditioning.

Previous studies have shown that D(1) receptor blockade disrupts and D(2) receptor blockade enhances long-term potentiation. These data lead to the prediction that D(1) antagonists will attenuate and D(2) antagonists will potentiate at least some types of learning. The prediction is difficult to test, however, because disruptions in either D(1) or D(2) transmission lead to reduced locomotion, exploration, and response execution and are therefore likely to impair learning that requires behavioral responding (including exploration of an environment) during the learning episode. Under a paradigm that minimizes motor requirements, rats were trained to enter a food compartment during pellet presentation. Animals then received tone-food pairings under the influence of D(1) antagonist SCH23390 (0, 0.4, 0.8, and 0.16 mg/kg) or D(2) antagonist raclopride (0, 0.2, 0.4, and 0.8 mg/kg). An additional group received unpaired presentations of tone and food. On a drug-free test day 24 hr later, animals that had been under the influence of SCH23390 (like animals that had received unpaired presentations of tone and food) showed reduced head entries in response to the tone, whereas animals that had been under the influence of raclopride showed increased head entries in response to the tone compared with vehicle controls. These data demonstrate that, under a conditioned approach paradigm, D(1) and D(2) family receptor antagonists disrupt and promote learning, respectively, as predicted by the effects of D(1) and D(2) receptor blockade on neuronal plasticity.

Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum.

Dopamine (DA) neurons of the substantia nigra (SN) and ventral tegmental area (VTA) respond to a wide category of salient stimuli. Activation of SN and VTA DA neurons, and consequent release of nigrostriatal and mesolimbic DA, modulates the processing of concurrent glutamate inputs to dorsal and ventral striatal target regions. According to the view described here, this occurs under conditions of unexpected environmental change regardless of whether that change is rewarding or aversive. Nigrostriatal and mesolimbic DA activity gates the input of sensory, motor, and incentive motivational (e.g. reward) signals to the striatum. In light of recent single-unit and brain imaging data, it is suggested that the striatal reward signals originate in the orbitofrontal cortex and basolateral amygdala (BLA), regions that project strongly to the striatum. A DA signal of salient unexpected event occurrence, from this framework, gates the throughput of the orbitofrontal glutamate reward input to the striatum just as it gates the throughput of corticostriatal sensory and motor signals needed for normal response execution. Processing of these incoming signals is enhanced when synaptic DA levels are high, because DA enhances the synaptic efficacy of strong concurrent glutamate inputs while reducing the efficacy of weak glutamate inputs. The impairments in motor performance and incentive motivational processes that follow from nigrostriatal and mesolimbic DA loss can be understood in terms of a single mechanism: abnormal processing of sensorimotor and incentive motivation-related glutamate input signals to the striatum.