Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.

A Novel Assay Allowing Drug Self-Administration, Extinction, and Reinstatement Testing in Head-Restrained Mice.

Multiphoton microscopy is one of several new technologies providing unprecedented insight into the activity dynamics and function of neural circuits. Unfortunately, some of these technologies require experimentation in head-restrained animals, limiting the behavioral repertoire that can be integrated and studied. This issue is especially evident in drug addiction research, as no laboratories have coupled multiphoton microscopy with simultaneous intravenous drug self-administration, a behavioral paradigm that has predictive validity for treatment outcomes and abuse liability. Here, we describe a new experimental assay wherein head-restrained mice will press an active lever, but not inactive lever, for intravenous delivery of heroin or cocaine. Similar to freely moving animals, we find that lever pressing is suppressed through daily extinction training and subsequently reinstated through the presentation of relapse-provoking triggers (drug-associative cues, the drug itself, and stressors). Finally, we show that head-restrained mice will show similar patterns of behavior for oral delivery of a sucrose reward, a common control used for drug self-administration experiments. Overall, these data demonstrate the feasibility of combining drug self-administration experiments with technologies that require head-restraint, such as multiphoton imaging. The assay described could be replicated by interested labs with readily available materials to aid in identifying the neural underpinnings of substance use disorder.

Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate.

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.

Dissociable mesolimbic dopamine circuits control responding triggered by alcohol-predictive discrete cues and contexts.

Context can influence reactions to environmental cues and this elemental process has implications for substance use disorder. Using an animal model, we show that an alcohol-associated context elevates entry into a fluid port triggered by a conditioned stimulus (CS) that predicted alcohol (CS-triggered alcohol-seeking). This effect persists across multiple sessions and, after it diminishes in extinction, the alcohol context retains the capacity to augment reinstatement. Systemically administered eticlopride and chemogenetic inhibition of ventral tegmental area (VTA) dopamine neurons reduce CS-triggered alcohol-seeking. Chemogenetically silencing VTA dopamine terminals in the nucleus accumbens (NAc) core reduces CS-triggered alcohol-seeking, irrespective of context, whereas silencing VTA dopamine terminals in the NAc shell selectively reduces the elevation of CS-triggered alcohol-seeking in an alcohol context. This dissociation reveals new roles for divergent mesolimbic dopamine circuits in the control of responding to a discrete cue for alcohol and in the amplification of this behaviour in an alcohol context.

Dopamine neurons do not constitute an obligatory stage in the final common path for the evaluation and pursuit of brain stimulation reward.

The neurobiological study of reward was launched by the discovery of intracranial self-stimulation (ICSS). Subsequent investigation of this phenomenon provided the initial link between reward-seeking behavior and dopaminergic neurotransmission. We re-evaluated this relationship by psychophysical, pharmacological, optogenetic, and computational means. In rats working for direct, optical activation of midbrain dopamine neurons, we varied the strength and opportunity cost of the stimulation and measured time allocation, the proportion of trial time devoted to reward pursuit. We found that the dependence of time allocation on the strength and cost of stimulation was similar formally to that observed when electrical stimulation of the medial forebrain bundle served as the reward. When the stimulation is strong and cheap, the rats devote almost all their time to reward pursuit; time allocation falls off as stimulation strength is decreased and/or its opportunity cost is increased. A 3D plot of time allocation versus stimulation strength and cost produces a surface resembling the corner of a plateau (the "reward mountain"). We show that dopamine-transporter blockade shifts the mountain along both the strength and cost axes in rats working for optical activation of midbrain dopamine neurons. In contrast, the same drug shifted the mountain uniquely along the opportunity-cost axis when rats worked for electrical MFB stimulation in a prior study. Dopamine neurons are an obligatory stage in the dominant model of ICSS, which positions them at a key nexus in the final common path for reward seeking. This model fails to provide a cogent account for the differential effect of dopamine transporter blockade on the reward mountain. Instead, we propose that midbrain dopamine neurons and neurons with non-dopaminergic, MFB axons constitute parallel limbs of brain-reward circuitry that ultimately converge on the final-common path for the evaluation and pursuit of rewards.

Paraventricular Thalamus Projection Neurons Integrate Cortical and Hypothalamic Signals for Cue-Reward Processing.

The paraventricular thalamus (PVT) is an interface for brain reward circuits, with input signals arising from structures, such as prefrontal cortex and hypothalamus, that are broadcast to downstream limbic targets. However, the precise synaptic connectivity, activity, and function of PVT circuitry for reward processing are unclear. Here, using in vivo two-photon calcium imaging, we find that PVT neurons projecting to the nucleus accumbens (PVT-NAc) develop inhibitory responses to reward-predictive cues coding for both cue-reward associative information and behavior. The multiplexed activity in PVT-NAc neurons is directed by opposing activity patterns in prefrontal and lateral hypothalamic afferent axons. Further, we find that prefrontal cue encoding may maintain accurate cue-reward processing, as optogenetic disruption of this encoding induced long-lasting effects on downstream PVT-NAc cue responses and behavioral cue discrimination. Together, these data reveal that PVT-NAc neurons act as an interface for reward processing by integrating relevant inputs to accurately inform reward-seeking behavior.

The Effects of Electrical and Optical Stimulation of Midbrain Dopaminergic Neurons on Rat 50-kHz Ultrasonic Vocalizations.

RATIONALE: Adult rats emit ultrasonic vocalizations (USVs) at around 50-kHz; these commonly occur in contexts that putatively engender positive affect. While several reports indicate that dopaminergic (DAergic) transmission plays a role in the emission of 50-kHz calls, the pharmacological evidence is mixed. Different modes of dopamine (DA) release (i.e., tonic and phasic) could potentially explain this discrepancy. OBJECTIVE: To investigate the potential role of phasic DA release in 50-kHz call emission. METHODS: In Experiment 1, USVs were recorded in adult male rats following unexpected electrical stimulation of the medial forebrain bundle (MFB). In parallel, phasic DA release in the nucleus accumbens (NAcc) was recorded using fast-scan cyclic voltammetry. In Experiment 2, USVs were recorded following response-contingent or non-contingent optogenetic stimulation of midbrain DAergic neurons. Four 20-s schedules of optogenetic stimulation were used: fixed-interval, fixed-time, variable-interval, and variable-time. RESULTS: Brief electrical stimulation of the MFB increased both 50-kHz call rate and phasic DA release in the NAcc. During optogenetic stimulation sessions, rats initially called at a high rate comparable to that observed following reinforcers such as psychostimulants. Although optogenetic stimulation maintained reinforced responding throughout the 2-h session, the call rate declined to near zero within the first 30 min. The trill call subtype predominated following both electrical and optical stimulation. CONCLUSION: The occurrence of electrically-evoked 50-kHz calls, time-locked to phasic DA (Experiment 1), provides correlational evidence supporting a role for phasic DA in USV production. However, in Experiment 2, the temporal dissociation between calling and optogenetic stimulation of midbrain DAergic neurons suggests that phasic mesolimbic DA release is not sufficient to produce 50-kHz calls. The emission of the trill subtype of 50-kHz calls potentially provides a marker distinguishing positive affect from positive reinforcement.

Robust optical fiber patch-cords for in vivo optogenetic experiments in rats.

In vivo optogenetic experiments commonly employ long lengths of optical fiber to connect the light source (commonly a laser) to the optical fiber implants in the brain. Commercially available patch cords are expensive and break easily. Researchers have developed methods to build these cables in house for in vivo experiments with rodents [1-4]. However, the half-life of those patch cords is greatly reduced when they are used with behaving rats, which are strong enough to break the delicate cable tip and to bite through the optical fiber and furcation tubing. Based on [3] we have strengthened the patch-cord tip that connects to the optical implant, and we have incorporated multiple layers of shielding to produce more robust and resistant cladding. Here, we illustrate how to build these patch cords with FC or M3 connectors. However, the design can be adapted for use with other common optical-fiber connectors. We have saved time and money by using this design in our optical self-stimulation experiments with rats, which are commonly several months long and last four to eleven hours per session. The main advantages are: •Long half-life.•Resistant to moderate rodent bites.•Suitable for long in vivo optogenetic experiments with large rodents.

Role of dopamine tone in the pursuit of brain stimulation reward.

Dopaminergic neurons contribute to intracranial self-stimulation (ICSS) and other reward-seeking behaviors, but it is not yet known where dopaminergic neurons intervene in the neural circuitry underlying reward pursuit or which psychological processes are involved. In rats working for electrical stimulation of the medial forebrain bundle, we assessed the effect of GBR-12909 (1-[2-[bis(4-fluorophenyl)-methoxy]ethyl]-4-[3- phenylpropyl]piperazine), a specific blocker of the dopamine transporter. Operant performance was measured as a function of the strength and cost of electrical stimulation. GBR-12909 increased the opportunity cost most subjects were willing to pay for a reward of a given intensity. However, this effect was smaller than that produced by a regimen of cocaine administration that drove similar increases in nucleus accumbens (NAc) dopamine levels in unstimulated rats. Delivery of rewarding stimulation to drug-treated rats caused an additional increase in dopamine concentration in the NAc shell in cocaine-treated, but not GBR-12909-treated, rats. These behavioral and neurochemical differences may reflect blockade of the norepinephrine transporter by cocaine but not by GBR-12909. Whereas the effect of psychomotor stimulants on ICSS has long been attributed to dopaminergic action at early stages of the reward pathway, the results reported here imply that increased dopamine tone boosts reward pursuit by acting at or beyond the output of the circuitry that temporally and spatially summates the output of the directly stimulated neurons underlying ICSS. The observed enhancement of reward seeking could be attributable to a decrease in the value of competing behaviors, a decrease in subjective effort costs, or an increase in reward-system gain.

Cannabinoid receptor blockade reduces the opportunity cost at which rats maintain operant performance for rewarding brain stimulation.

There is ample evidence that blockade of CB(1) receptors reduces reward seeking. However, the reported effects of CB(1) blockade on performance for rewarding electrical brain stimulation stand out as an exception. By applying a novel method for conceptualizing and measuring reward seeking, we show that AM-251, a CB(1) receptor antagonist, does indeed decrease performance for rewarding electrical stimulation of the medial forebrain bundle in rats. Reward seeking depends on multiple sets of variables, including the intensity of the reward, its cost, and the value of competing rewards. In turn, reward intensity depends both on the sensitivity and gain of brain reward circuitry. We show that drug-induced changes in sensitivity cannot account for the suppressive effect of AM-251 on reward seeking. Therefore, the role of CB(1) receptors must be sought among the remaining determinants of performance. Our analysis provides an explanation of the inconsistencies between prior reports, which likely arose from the following: (1) the averaging of data across subjects showing heterogeneous effects and (2) the use of methods that cannot distinguish between the different determinants of reward pursuit. By means of microdialysis, we demonstrate that blockade of CB(1) receptors attenuates nucleus accumbens dopamine release in response to rewarding medial forebrain bundle stimulation, and we propose that this action is responsible for the ability of the drug to decrease performance for the electrical reward.