Learning critically drives parkinsonian motor deficits through imbalanced striatal pathway recruitment.

Dopamine (DA) loss in Parkinson's disease (PD) causes debilitating motor deficits. However, dopamine is also widely linked to reward prediction and learning, and the contribution of dopamine-dependent learning to movements that are impaired in PD-which often do not lead to explicit rewards-is unclear. Here, we used two distinct motor tasks to dissociate dopamine's acute motoric effects vs. its long-lasting, learning-mediated effects. In dopamine-depleted mice, motor task performance gradually worsened with task exposure. Task experience was critical, as mice that remained in the home cage during the same period were relatively unimpaired when subsequently probed on the task. Repeated dopamine replacement treatments acutely rescued deficits and gradually induced long-term rescue that persisted despite treatment withdrawal. Surprisingly, both long-term rescue and parkinsonian performance decline were task specific, implicating dopamine-dependent learning. D1R activation potently induced acute rescue that gradually consolidated into long-term rescue. Conversely, reduced D2R activation potently induced parkinsonian decline. In dopamine-depleted mice, either D1R activation or D2R activation prevented parkinsonian decline, and both restored balanced activation of direct vs. indirect striatal pathways. These findings suggest that reinforcement and maintenance of movements-even movements not leading to explicit rewards-are fundamental functions of dopamine and provide potential mechanisms for the hitherto unexplained "long-duration response" by dopaminergic therapies in PD.

Social context has differential effects on acquisition of nicotine self-administration in male and female rats.

RATIONAL: Smoking typically begins during adolescence or early adulthood in a social context, yet the role of social context in animal models is poorly understood. OBJECTIVES: The present study examined the effect of social context on acquisition of nicotine self-administration. METHODS: Sixty-day-old male and female Sprague-Dawley rats were trained to press a lever for nicotine (0.015 mg/kg, IV) or saline infusions (males only) on a fixed ratio (FR1) schedule of reinforcement across nine sessions in duplex chambers that were conjoined with either a solid wall or a wall containing wire mesh creating a social context between rat dyads (social visual, auditory, and olfactory cues). In a subsequent experiment, sex differences and dose-dependent effects of nicotine [0 (saline), 0.015 or 0.03 mg/kg, IV] were directly compared in rats trained in the isolated or social context on a schedule progressing from FR1 to FR3. These rats were given 20 sessions followed by 3 extinction sessions. RESULTS: We consistently found transient social facilitation of low-dose nicotine self-administration in males during the first session. However, across training overall, we found social suppression of nicotine intake that was most prominent in females during later sessions. CONCLUSIONS: Collectively, these findings suggest that at the age of transition from adolescence to adulthood, a social context enhances the initial reinforcing effects of nicotine in males, but protects against nicotine intake during later sessions especially in females. These findings highlight the importance of sex and social context in studying neural mechanisms involved in initiation of nicotine use.

A bout analysis of operant response disruption.

Operant behavior appears to be organized in bouts of responses, whose parameters are differentially sensitive to various manipulations. This study investigated potential differential effects of three forms of operant response disruption-extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)-on response bouts. In Experiment 1, Wistar Kyoto rats (WKY) were trained on a tandem variable-time (VT) 120s fixed-ratio (FR) 5 schedule of reinforcement; after stability was established, their responding was disrupted for three sessions with one of the three disrupters (EXT, NCR, or PRE). In Experiment 2, Long Evans (LE) rats were trained on a tandem VT 240s FR 5 to stability, and their responding disrupted with EXT or NCR. In EXT and NCR, response rates declined significantly and progressively over the course of the session, primarily due to a declining bout-initiation rate in EXT, and to fewer responses per bout in NCR. In contrast, a session-wide drop in response rate was observed in PRE, primarily due to a reduction in bout-initiation rate at the start of the session. These findings suggest that each form of disruption differentially impacts dissociable aspects of behavior. Theories of behavioral persistence should account for these functional relations, which appear to be obscured in response rate measures.

Dopamine D3 and 5-HT1B receptor dysregulation as a result of psychostimulant intake and forced abstinence: Implications for medications development.

Addiction to psychostimulants, including cocaine and amphetamine, is associated with dysregulation of dopamine and serotonin (5-HT) neurotransmitter systems. Neuroadaptations in these systems vary depending on the stage of the drug taking-abstinence-relapse cycle. Consequently, the effects of potential treatments that target these systems may vary depending on whether they are given during abstinence or relapse. In this review, we discuss evidence that dopamine D3 receptors (D3Rs) and 5-HT1B receptors (5-HT1BRs) are dysregulated in response to both chronic psychostimulant use and subsequent abstinence. We then review findings from preclinical self-administration models which support targeting D3Rs and 5-HT1BRs as potential medications for psychostimulant dependence. Potential side effects of the treatments are discussed and attention is given to studies reporting positive treatment outcomes that depend on: 1) whether testing occurs during self-administration versus abstinence, 2) whether escalation of drug self-administration has occurred, 3) whether the treatments are given repeatedly, and 4) whether social factors influence treatment outcomes. We conclude that D3/D2 agonists may decrease psychostimulant intake; however, side effects of D3/D2R full agonists may limit their therapeutic potential, whereas D3/D2R partial agonists have fewer undesirable side effects. D3-selective antagonists may not reduce psychostimulant intake during relapse, but nonetheless, may decrease motivation for seeking psychostimulants with relatively few side-effects. 5-HT1BR agonists provide a striking example of treatment outcomes that are dependent on the stage of the addiction cycle. Specifically, these agonists initially increase cocaine's reinforcing effects during maintenance of self-administration, but after a period of abstinence they reduce psychostimulant seeking and the resumption of self-administration. In conclusion, we suggest that factors contributing to dysregulation of monoamine systems, including drug history, abstinence, and social context, should be considered when evaluating potential treatments to better model treatment effects in humans. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.

Reduction of cocaine self-administration and D3 receptor-mediated behavior by two novel dopamine D3 receptor-selective partial agonists, OS-3-106 and WW-III-55.

Dopamine D3 receptor (D3R)-selective compounds may be useful medications for cocaine dependence. In this study, we identified two novel arylamide phenylpiperazines, OS-3-106 and WW-III-55, as partial agonists at the D3R in the adenylyl cyclase inhibition assay. OS-3-106 and WW-III-55 have 115- and 862-fold D3R:D2 receptor (D2R) binding selectivity, respectively. We investigated their effects (0, 3, 5.6, or 10 mg/kg) on operant responding by using a multiple variable-interval (VI) 60-second schedule that alternated components with sucrose reinforcement and components with intravenous cocaine reinforcement (0.375 mg/kg). Additionally, we evaluated the effect of OS-3-106 (10 mg/kg) on the dose-response function of cocaine self-administration and the effect of WW-III-55 (0-5.6 mg/kg) on a progressive ratio schedule with either cocaine or sucrose reinforcement. Both compounds were also examined for effects on locomotion and yawning induced by a D3R agonist. OS-3-106 decreased cocaine and sucrose reinforcement rates, increased latency to first response for cocaine but not sucrose, and downshifted the cocaine self-administration dose-response function. WW-III-55 did not affect cocaine self-administration on the multiple-variable interval schedule, but it reduced cocaine and sucrose intake on the progressive ratio schedule. Both compounds reduced locomotion at doses that reduced responding, and both compounds attenuated yawning induced by low doses of 7-OH-DPAT (a D3R-mediated behavior), but neither affected yawning on the descending limb of the 7-OH-DPAT dose-response function (a D2R-mediated behavior). Therefore, both compounds blocked a D3R-mediated behavior. However, OS-3-106 was more effective in reducing cocaine self-administration. These findings support D3Rs, and possibly D2Rs, as targets for medications aimed at reducing the motivation to seek cocaine.

Fos expression in response to dopamine D3-preferring phenylpiperazine drugs given with and without cocaine.

WC 44 and WC 10 are phenylpiperazines with low (23 fold) to moderate (42 fold) selectivity for dopamine D3 receptors (D3Rs) over D2Rs, respectively. WC 44 is a full D3R agonist in the forskolin-stimulated adenylyl cyclase (AC) assay, whereas WC 10 has little efficacy. In contrast to their opposite effects in the AC assay, these drugs often produce similar behavioral effects, suggesting that the AC assay does not predict the efficacy of these drugs in vivo. Here, we examined whether Fos protein expression induced by these drugs would be more consistent with their behavioral effects in vivo. Rats received either vehicle, WC 10 (5.6 mg/kg, i.p.), WC 44 (10.0 mg/kg, i.p), cocaine (10.0 mg/kg, i.p.), or cocaine with WC 10 (5.6 mg/kg, i.p.) or with WC 44 (10.0 mg/kg, i.p). Locomotion was monitored for 90 min and the brains were harvested for immunohistochemistry. Both WC 10 and WC 44 decreased spontaneous and cocaine-induced locomotion. Both compounds also increased Fos expression relative to saline in the dorsal striatum and nucleus accumbens core and shell, and relative to cocaine alone in the nucleus accumbens shell. The findings suggest that even though these compounds have different efficacy in the AC bioassy, they produce similar brain activation and attenuation of cocaine hyperlocomotion. Together with our previous research demonstrating that these compounds down-shift the cocaine self-administration dose-effect function, the findings support the idea that D3R-selective compounds may be useful for cocaine dependence medications development.

Extinction learning deficit in a rodent model of attention-deficit hyperactivity disorder.

BACKGROUND: Deficient operant extinction has been hypothesized to be constitutive of ADHD dysfunction. In order to elucidate the behavioral mechanisms underlying this deficit, the performance of an animal model of ADHD, the spontaneously hypertensive rat (SHR), was compared against the performance of a control strain, the Wistar-Kyoto rat (WKY) during extinction. METHOD: Following extensive training of lever pressing under variable interval schedules of food reinforcement (reported previously), SHR and WKY rats were exposed to two sessions of extinction training. Extinction data was analyzed using the Dynamic Bi-Exponential Refractory Model (DBERM) of operant performance. DBERM assumes that operant responses are organized in bouts separated by pauses; during extinction, bouts may decline across multiple dimensions, including frequency and length. DBERM parameters were estimated using hierarchical Bayesian modeling. RESULTS: SHR responded more than WKY during the first extinction session. DBERM parameter estimates revealed that, at the onset of extinction, SHR produced more response bouts than WKY. Over the course of extinction, response bouts progressively shortened for WKY but not for SHR. CONCLUSIONS: Based on prior findings on the sensitivity of DBERM parameters to motivational and schedule manipulations, present data suggests that (1) more frequent response bouts in SHR are likely related to greater incentive motivation, and (2) the persistent length of bouts in SHR are likely related to a slower updating of the response-outcome association. Overall, these findings suggest specific motivational and learning deficits that may explain ADHD-related impairments in operant performance.

Extinction under a behavioral microscope: isolating the sources of decline in operant response rate.

Extinction performance is often used to assess underlying psychological processes without the interference of reinforcement. For example, in the extinction/reinstatement paradigm, motivation to seek drug is assessed by measuring responding elicited by drug-associated cues without drug reinforcement. However, extinction performance is governed by several psychological processes that involve motivation, memory, learning, and motoric functions. These processes are confounded when overall response rate is used to measure performance. Based on evidence that operant responding occurs in bouts, this paper proposes an analytic procedure that separates extinction performance into several behavioral components: (1-3) the baseline bout initiation rate, within-bout response rate, and bout length at the onset of extinction; (4-6) their rates of decay during extinction; (7) the time between extinction onset and the decline of responding; (8) the asymptotic response rate at the end of extinction; (9) the refractory period after each response. Data that illustrate the goodness of fit of this analytic model are presented. This paper also describes procedures to isolate behavioral components contributing to extinction performance and make inferences about experimental effects on these components. This microscopic behavioral analysis allows the mapping of different psychological processes to distinct behavioral components implicated in extinction performance, which may further our understanding of the psychological effects of neurobiological treatments.

A new criterion for acquisition of nicotine self-administration in rats.

BACKGROUND: Acquisition of nicotine self-administration in rodents is relatively difficult to establish and measures of acquisition rate are sometimes confounded by manipulations used to facilitate the process. This study examined acquisition of nicotine self-administration without such manipulations and used mathematical modeling to define the criterion for acquisition. METHODS: Rats were given 20 daily 2-h sessions occurring 6 days/week in chambers equipped with active and inactive levers. Each active lever press resulted in nicotine reinforcement (0-0.06 mg/kg, IV) and retraction of both levers for a 20-s time out, whereas inactive lever presses had no consequences. Acquisition was defined for individual rats by the higher likelihood of reinforcers obtained across sessions fitting a logistic over a constant function according to the corrected Akaike Information Criterion (AICc). RESULTS: For rats that acquired self-administration, an AICc-based multi-model comparison demonstrated that the asymptote (highest number of reinforcers/session) and mid-point of the acquisition curve (h; the number of sessions necessary to reach half the asymptote) varied by nicotine dose, with both exhibiting a negative relationship (the higher the dose, the lower number of reinforcers and the lower h). CONCLUSIONS: The modeling approach used in this study provides a way of defining acquisition of nicotine self-administration that takes advantage of all data from individual subjects and the procedure used is sensitive to dose differences in the absence of manipulations that influence acquisition (e.g., food restriction, prior food reinforcement, conditioned reinforcers).

The isolation of motivational, motoric, and schedule effects on operant performance: a modeling approach.

Dissociating motoric and motivational effects of pharmacological manipulations on operant behavior is a substantial challenge. To address this problem, we applied a response-bout analysis to data from rats trained to lever press for sucrose on variable-interval (VI) schedules of reinforcement. Motoric, motivational, and schedule factors (effort requirement, deprivation level, and schedule requirements, respectively) were manipulated. Bout analysis found that interresponse times (IRTs) were described by a mixture of two exponential distributions, one characterizing IRTs within response bouts, another characterizing intervals between bouts. Increasing effort requirement lengthened the shortest IRT (the refractory period between responses). Adding a ratio requirement increased the length and density of response bouts. Both manipulations also decreased the bout-initiation rate. In contrast, food deprivation only increased the bout-initiation rate. Changes in the distribution of IRTs over time showed that responses during extinction were also emitted in bouts, and that the decrease in response rate was primarily due to progressively longer intervals between bouts. Taken together, these results suggest that changes in the refractory period indicate motoric effects, whereas selective alterations in bout initiation rate indicate incentive-motivational effects. These findings support the use of response-bout analyses to identify the influence of pharmacological manipulations on processes underlying operant performance.

Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats.

BACKGROUND: Animals must frequently act to influence the world even when the reinforcing outcomes of their actions are delayed. Learning with action-outcome delays is a complex problem, and little is known of the neural mechanisms that bridge such delays. When outcomes are delayed, they may be attributed to (or associated with) the action that caused them, or mistakenly attributed to other stimuli, such as the environmental context. Consequently, animals that are poor at forming context-outcome associations might learn action-outcome associations better with delayed reinforcement than normal animals. The hippocampus contributes to the representation of environmental context, being required for aspects of contextual conditioning. We therefore hypothesized that animals with hippocampal lesions would be better than normal animals at learning to act on the basis of delayed reinforcement. We tested the ability of hippocampal-lesioned rats to learn a free-operant instrumental response using delayed reinforcement, and what is potentially a related ability -- the ability to exhibit self-controlled choice, or to sacrifice an immediate, small reward in order to obtain a delayed but larger reward. RESULTS: Rats with sham or excitotoxic hippocampal lesions acquired an instrumental response with different delays (0, 10, or 20 s) between the response and reinforcer delivery. These delays retarded learning in normal rats. Hippocampal-lesioned rats responded slightly less than sham-operated controls in the absence of delays, but they became better at learning (relative to shams) as the delays increased; delays impaired learning less in hippocampal-lesioned rats than in shams. In contrast, lesioned rats exhibited impulsive choice, preferring an immediate, small reward to a delayed, larger reward, even though they preferred the large reward when it was not delayed. CONCLUSION: These results support the view that the hippocampus hinders action-outcome learning with delayed outcomes, perhaps because it promotes the formation of context-outcome associations instead. However, although lesioned rats were better at learning with delayed reinforcement, they were worse at choosing it, suggesting that self-controlled choice and learning with delayed reinforcement tax different psychological processes.

Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat.

BACKGROUND: Delays between actions and their outcomes severely hinder reinforcement learning systems, but little is known of the neural mechanism by which animals overcome this problem and bridge such delays. The nucleus accumbens core (AcbC), part of the ventral striatum, is required for normal preference for a large, delayed reward over a small, immediate reward (self-controlled choice) in rats, but the reason for this is unclear. We investigated the role of the AcbC in learning a free-operant instrumental response using delayed reinforcement, performance of a previously-learned response for delayed reinforcement, and assessment of the relative magnitudes of two different rewards. RESULTS: Groups of rats with excitotoxic or sham lesions of the AcbC acquired an instrumental response with different delays (0, 10, or 20 s) between the lever-press response and reinforcer delivery. A second (inactive) lever was also present, but responding on it was never reinforced. As expected, the delays retarded learning in normal rats. AcbC lesions did not hinder learning in the absence of delays, but AcbC-lesioned rats were impaired in learning when there was a delay, relative to sham-operated controls. All groups eventually acquired the response and discriminated the active lever from the inactive lever to some degree. Rats were subsequently trained to discriminate reinforcers of different magnitudes. AcbC-lesioned rats were more sensitive to differences in reinforcer magnitude than sham-operated controls, suggesting that the deficit in self-controlled choice previously observed in such rats was a consequence of reduced preference for delayed rewards relative to immediate rewards, not of reduced preference for large rewards relative to small rewards. AcbC lesions also impaired the performance of a previously-learned instrumental response in a delay-dependent fashion. CONCLUSIONS: These results demonstrate that the AcbC contributes to instrumental learning and performance by bridging delays between subjects' actions and the ensuing outcomes that reinforce behaviour.