Encoding and context-dependent control of reward consumption within the central nucleus of the amygdala.

Dysregulation of the central amygdala is thought to underlie aberrant choice in alcohol use disorder, but the role of central amygdala neural activity during reward choice and consumption is unclear. We recorded central amygdala neurons in male rats as they consumed alcohol or sucrose. We observed activity changes at the time of reward approach, as well as lick-entrained activity during ongoing consumption of both rewards. In choice scenarios where rats could drink sucrose, alcohol, or quinine-adulterated alcohol with or without central amygdala optogenetic stimulation, rats drank more of stimulation-paired options when the two bottles contained identical options. Given a choice among different options, central amygdala stimulation usually enhanced consumption of stimulation-paired rewards. However, optogenetic stimulation during consumption of the less-preferred option, alcohol, was unable to enhance alcohol intake while sucrose was available. These findings indicate that the central amygdala contributes to refining motivated pursuit toward the preferred available option.

State and rate-of-change encoding in parallel mesoaccumbal dopamine pathways.

The nervous system uses fast- and slow-adapting sensory detectors in parallel to enable neuronal representations of external states and their temporal dynamics. It is unknown whether this dichotomy also applies to internal representations that have no direct correlation in the physical world. Here we find that two distinct dopamine (DA) neuron subtypes encode either a state or its rate-of-change. In mice performing a reward-seeking task, we found that the animal's behavioral state and rate-of-change were encoded by the sustained activity of DA neurons in medial ventral tegmental area (VTA) DA neurons and transient activity in lateral VTA DA neurons, respectively. The neural activity patterns of VTA DA cell bodies matched DA release patterns within anatomically defined mesoaccumbal pathways. Based on these results, we propose a model in which the DA system uses two parallel lines for proportional-differential encoding of a state variable and its temporal dynamics.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance.

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Contexts facilitate dynamic value encoding in the mesolimbic dopamine system.

Adaptive behavior in a dynamic environment often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical test to theories of dopamine's function in learning, motivation, and motor control. Yet how dopamine neurons are involved in the revaluation of cues when the world changes to alter our behavior remains unclear. Here we make use of pharmacology, in vivo electrophysiology, fiber photometry, and optogenetics to resolve the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward-seeking. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by sucrose reward by exploiting the prior, non-overlapping presentation of a separate discrete cue - an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery. As a result, in this task we were able to dissociate the average value of the conditioned stimulus from its immediate expected value on a trial-to-trial basis. Both the activity of ventral tegmental area dopamine neurons and dopamine signaling in the nucleus accumbens were essential for rats to successfully update behavioral responding in response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward. Downstream of dopamine release, we found that single neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together these results reveal a novel mechanism within the mesolimbic dopamine system for the rapid revaluation of motivation.

Encoding and context-dependent control of reward consumption within the central nucleus of the amygdala.

The ability to evaluate and select a preferred option among a variety of available offers is an essential aspect of goal-directed behavior. Dysregulation of this valuation process is characteristic of alcohol use disorder, with the central amygdala being implicated in persistent alcohol pursuit. However, the mechanism by which the central amygdala encodes and promotes the motivation to seek and consume alcohol remains unclear. We recorded single-unit activity in male Long-Evans rats as they consumed 10% ethanol or 14.2% sucrose. We observed significant activity at the time of approach to alcohol or sucrose, as well as lick-entrained activity during the ongoing consumption of both alcohol and sucrose. We then evaluated the ability of central amygdala optogenetic manipulation time-locked to consumption to alter ongoing intake of alcohol or sucrose, a preferred non-drug reward. In closed two-choice scenarios where rats could drink only sucrose, alcohol, or quinine-adulterated alcohol with or without central amygdala stimulation, rats drank more of stimulation-paired options. Microstructural analysis of licking patterns suggests these effects were mediated by changes in motivation, not palatability. Given a choice among different options, central amygdala stimulation enhanced consumption if the stimulation was associated with the preferred reward while closed-loop inhibition only decreased consumption if the options were equally valued. However, optogenetic stimulation during consumption of the less-preferred option, alcohol, was unable to enhance overall alcohol intake while sucrose was available. Collectively, these findings indicate that the central amygdala processes the motivational value of available offers to promote pursuit of the most preferred available option.

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement.

Reward seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement, and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations in male and female rats, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit, whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is nonscalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies.SIGNIFICANCE STATEMENT Dopamine neurons in the midbrain are essential for learning, motivation, and movement. Here we describe in detail the ability of VTA and SNc dopamine neurons to generate instrumental reinforcement, a process where an agent learns about actions they can emit to earn reward. While rats will avidly work and learn to respond for activation of VTA and SNc dopamine neurons, we find that only VTA dopamine neurons imbue actions and their associated cues with motivational value that spur continued pursuit of reward. Our data support a hypothesis that VTA and SNc dopamine neurons engage distinct psychological processes that have consequences for our understanding of these neurons in health and disease.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters.

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Mesoaccumbal Dopamine Heterogeneity: What Do Dopamine Firing and Release Have to Do with It?

Ventral tegmental area (VTA) dopamine (DA) neurons are often thought to uniformly encode reward prediction errors. Conversely, DA release in the nucleus accumbens (NAc), the prominent projection target of these neurons, has been implicated in reinforcement learning, motivation, aversion, and incentive salience. This contrast between heterogeneous functions of DA release versus a homogeneous role for DA neuron activity raises numerous questions regarding how VTA DA activity translates into NAc DA release. Further complicating this issue is increasing evidence that distinct VTA DA projections into defined NAc subregions mediate diverse behavioral functions. Here, we evaluate evidence for heterogeneity within the mesoaccumbal DA system and argue that frameworks of DA function must incorporate the precise topographic organization of VTA DA neurons to clarify their contribution to health and disease.

Reward activity in ventral pallidum tracks satiety-sensitive preference and drives choice behavior.

A key function of the nervous system is producing adaptive behavior across changing conditions, like physiological state. Although states like thirst and hunger are known to impact decision-making, the neurobiology of this phenomenon has been studied minimally. Here, we tracked evolving preference for sucrose and water as rats proceeded from a thirsty to sated state. As rats shifted from water choices to sucrose choices across the session, the activity of a majority of neurons in the ventral pallidum, a region crucial for reward-related behaviors, closely matched the evolving behavioral preference. The timing of this signal followed the pattern of a reward prediction error, occurring at the cue or the reward depending on when reward identity was revealed. Additionally, optogenetic stimulation of ventral pallidum neurons at the time of reward was able to reverse behavioral preference. Our results suggest that ventral pallidum neurons guide reward-related decisions across changing physiological states.

A quantitative reward prediction error signal in the ventral pallidum.

The nervous system is hypothesized to compute reward prediction errors (RPEs) to promote adaptive behavior. Correlates of RPEs have been observed in the midbrain dopamine system, but the extent to which RPE signals exist in other reward-processing regions is less well understood. In the present study, we quantified outcome history-based RPE signals in the ventral pallidum (VP), a basal ganglia region functionally linked to reward-seeking behavior. We trained rats to respond to reward-predicting cues, and we fit computational models to predict the firing rates of individual neurons at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did so more robustly than the nucleus accumbens, an input to the VP. VP RPEs predicted changes in task engagement, and optogenetic manipulation of the VP during reward delivery bidirectionally altered rats' subsequent reward-seeking behavior. Our data suggest a pivotal role for the VP in computing teaching signals that influence adaptive reward seeking.

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking.

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event-an occasion setter (OS)-which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

Occasion setting.

Occasion setting refers to the ability of 1 stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking.

The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core (NAcC). These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined whether the diminishing dopamine signal in the NAcC has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the NAcC or the dorsal lateral striatum (DLS). Dopamine antagonism within the NAcC attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the DLS affected sign-tracking behavior. Critically, extended training did not alter these effects. Although extended experience with an incentive stimulus may reduce cue-evoked dopamine in the NAcC, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training.

A food-predictive cue attributed with incentive salience engages subcortical afferents and efferents of the paraventricular nucleus of the thalamus.

The paraventricular nucleus of the thalamus (PVT) has been implicated in behavioral responses to reward-associated cues. However, the precise role of the PVT in these behaviors has been difficult to ascertain since Pavlovian-conditioned cues can act as both predictive and incentive stimuli. The "sign-tracker/goal-tracker" rat model has allowed us to further elucidate the role of the PVT in cue-motivated behaviors, identifying this structure as a critical component of the neural circuitry underlying individual variation in the propensity to attribute incentive salience to reward cues. The current study assessed differences in the engagement of specific PVT afferents and efferents in response to presentation of a food-cue that had been attributed with only predictive value or with both predictive and incentive value. The retrograde tracer fluorogold (FG) was injected into the PVT or the nucleus accumbens (NAc) of rats, and cue-induced c-Fos in FG-labeled cells was quantified. Presentation of a predictive stimulus that had been attributed with incentive value elicited c-Fos in PVT afferents from the lateral hypothalamus, medial amygdala (MeA), and the prelimbic cortex (PrL), as well as posterior PVT efferents to the NAc. PVT afferents from the PrL also showed elevated c-Fos levels following presentation of a predictive stimulus alone. Thus, presentation of an incentive stimulus results in engagement of subcortical brain regions; supporting a role for the hypothalamic-thalamic-striatal axis, as well as the MeA, in mediating responses to incentive stimuli; whereas activity in the PrL to PVT pathway appears to play a role in processing the predictive qualities of reward-paired stimuli.

Examining the role of dopamine D2 and D3 receptors in Pavlovian conditioned approach behaviors.

Elucidating the neurobiological mechanisms underlying individual differences in the extent to which reward cues acquire the ability to act as incentive stimuli may contribute to the development of successful treatments for addiction and related disorders. We used the sign-tracker/goal-tracker animal model to examine the role of dopamine D2 and D3 receptors in the propensity to attribute incentive salience to reward cues. Following Pavlovian training, wherein a discrete lever-cue was paired with food reward, rats were classified as sign- or goal-trackers based on the resultant conditioned response. We examined the effects of D2/D3 agonists, 7-OH-DPAT (0.01-0.32mg/kg) or pramipexole (0.032-0.32mg/kg), the D2/D3 antagonist raclopride (0.1mg/kg), and the selective D3 antagonist, SB-277011A (6 or 24mg/kg), on the expression of sign- and goal-tracking conditioned responses. The lever-cue acquired predictive value and elicited a conditioned response for sign- and goal-trackers, but only for sign-trackers did it also acquire incentive value. Following administration of either 7-OH-DPAT, pramipexole, or raclopride, the performance of the previously acquired conditioned response was attenuated for both sign- and goal-trackers. For sign-trackers, the D2/D3 agonist, 7-OH-DPAT, also attenuated the conditioned reinforcing properties of the lever-cue. The selective D3 antagonist did not affect either conditioned response. Alterations in D2/D3 receptor signaling, but not D3 signaling alone, transiently attenuate a previously acquired Pavlovian conditioned response, regardless of whether the response is a result of incentive motivational processes. These findings suggest activity at the dopamine D2 receptor is critical for a reward cue to maintain either its incentive or predictive qualities.

Lesions of the paraventricular nucleus of the thalamus differentially affect sign- and goal-tracking conditioned responses.

Recently, evidence has emerged suggesting a role for the paraventricular nucleus of the thalamus (PVT) in the processing of reward-associated cues. However, the specific role of the PVT in these processes has yet to be elucidated. Here we use an animal model that captures individual variation in response to discrete reward-associated cues to further assess the role of the PVT in stimulus-reward learning. When rats are exposed to a Pavlovian conditioning paradigm, wherein a discrete cue predicts food reward, two distinct conditioned responses emerge. Some rats, termed sign-trackers, approach and manipulate the cue, whereas others, termed goal-trackers, approach the location of reward delivery upon cue presentation. For both sign- and goal-trackers the cue is a predictor, but only for sign-trackers is it also an incentive stimulus. We investigated the role of the PVT in the acquisition and expression of these conditioned responses using an excitotoxic lesion. Results indicate that PVT lesions prior to acquisition amplify the differences between phenotypes - increasing sign-tracking and attenuating goal-tracking behavior. Lesions of the PVT after rats had acquired their respective conditioned responses also attenuated the expression of the goal-tracking response, and increased the sign-tracking response, but did so selectively in goal-trackers. These results suggest that the PVT acts to suppress the attribution of incentive salience to reward cues, as disruption of the functional activity within this structure enhances the tendency to sign-track.