Ventral Pallidal GABAergic Neuron Calcium Activity Encodes Cue-Driven Reward Seeking and Persists in the Absence of Reward Delivery.

Reward-seeking behavior is often initiated by environmental cues that signal reward availability. This is a necessary behavioral response; however, cue reactivity and reward-seeking behavior can become maladaptive. To better understand how cue-elicited reward seeking becomes maladaptive, it is important to understand the neural circuits involved in assigning appetitive value to rewarding cues and actions. Ventral pallidum (VP) neurons are known to contribute to cue-elicited reward-seeking behavior and have heterogeneous responses in a discriminative stimulus (DS) task. The VP neuronal subtypes and output pathways that encode distinct aspects of the DS task remain unknown. Here, we used an intersectional viral approach with fiber photometry to record bulk calcium activity in VP GABAergic (VP GABA) neurons in male and female rats as they learned and performed the DS task. We found that VP GABA neurons are excited by reward-predictive cues but not neutral cues and that this response develops over time. We also found that this cue-evoked response predicts reward-seeking behavior and that inhibiting this VP GABA activity during cue presentation decreases reward-seeking behavior. Additionally, we found increased VP GABA calcium activity at the time of expected reward delivery, which occurred even on trials when reward was omitted. Together, these findings suggest that VP GABA neurons encode reward expectation, and calcium activity in these neurons encodes the vigor of cue-elicited reward seeking.SIGNIFICANCE STATEMENT VP circuitry is a major driver of cue-evoked behaviors. Previous work has found that VP neurons have heterogenous responses and contributions to reward-seeking behavior. This functional heterogeneity is because of differences of neurochemical subtypes and projections of VP neurons. Understanding the heterogenous responses among and within VP neuronal cell types is a necessary step in further understanding how cue-evoked behavior becomes maladaptive. Our work explores the canonical GABAergic VP neuron and how the calcium activity of these cells encodes components of cue-evoked reward seeking, including the vigor and persistence of reward seeking.

Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking.

A critical area of inquiry in the neurobiology of alcohol abuse is the mechanism by which cues gain the ability to elicit alcohol use. Previously, we found that cue-evoked activity in rat ventral pallidum robustly encodes the value of sucrose cues trained under both Pavlovian and instrumental contingencies, despite a stronger relationship between cue-evoked activity and behavioral latency after instrumental training (Richard et al., 2018, Elife, 7, e33107). Here, we assessed: (a) ventral pallidal representations of Pavlovian versus instrumental cues trained with alcohol reward, and (b) the impact of non-associative alcohol exposure on ventral pallidal representations of sucrose cues. Decoding of cue identity based on ventral pallidum firing was blunted for the Pavlovian alcohol cue in comparison to both the instrumental cue trained with alcohol and either cue type trained with sucrose. Further, non-associative alcohol exposure had opposing effects on ventral pallidal encoding of sucrose cues trained on instrumental versus Pavlovian associations, enhancing decoding accuracy for an instrumental discriminative stimulus and reducing decoding accuracy for a Pavlovian conditioned stimulus. These findings suggest that alcohol exposure can drive biased engagement of specific reward-related signals in the ventral pallidum.

New insights into the specificity and plasticity of reward and aversion encoding in the mesolimbic system.

The mesocorticolimbic system, consisting, at its core, of the ventral tegmental area, the nucleus accumbens, and medial prefrontal cortex, has historically been investigated primarily for its role in positively motivated behaviors and reinforcement learning, and its dysfunction in addiction, schizophrenia, depression, and other mood disorders. Recently, researchers have undertaken a more comprehensive analysis of this system, including its role in not only reward but also punishment, as well as in both positive and negative reinforcement. This focus has been facilitated by new anatomical, physiological, and behavioral approaches to delineate functional circuits underlying behaviors and to determine how this system flexibly encodes and responds to positive and negative states and events, beyond simple associative learning. This review is a summary of topics covered in a mini-symposium at the 2013 Society for Neuroscience annual meeting.

Pavlovian cue-evoked alcohol seeking is disrupted by ventral pallidal inhibition.

Cues paired with alcohol can be potent drivers of craving, alcohol-seeking, consumption, and relapse. While the ventral pallidum is implicated in appetitive and consummatory responses across several reward classes and types of behaviors, its role in behavioral responses to Pavlovian alcohol cues has not previously been established. Here, we tested the impact of optogenetic inhibition of ventral pallidum on Pavlovian-conditioned alcohol-seeking in male Long Evans rats. Rats underwent Pavlovian conditioning with an auditory cue predicting alcohol delivery to a reward port and a control cue predicting no alcohol delivery, until they consistently entered the reward port more during the alcohol cue than the control cue. We then tested the within-session effects of optogenetic inhibition during 50% of cue presentations. We found that optogenetic inhibition of ventral pallidum during the alcohol cue reduced port entry likelihood and time spent in the port, and increased port entry latency. Overall, these results suggest that normal ventral pallidum activity is necessary for Pavlovian alcohol-seeking.

Ventral pallidal GABAergic neurons drive consumption in male, but not female rats.

Food intake is controlled by multiple converging signals: hormonal signals that provide information about energy homeostasis, but also hedonic and motivational aspects of food and food cues that can drive non-homeostatic or "hedonic "feeding. The ventral pallidum (VP) is a brain region implicated in the hedonic and motivational impact of food and foods cues, as well as consumption of rewards. Disinhibition of VP neurons has been shown to generate intense hyperphagia, or overconsumption. While VP gamma-Aminobutyric acidergic (GABA) neurons have been implicated in cue-elicited reward seeking and motivation, the role of these neurons in the hyperphagia resulting from VP activation remains unclear. Here, we used Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) to activate or inhibit VP GABA neurons in sated male and female rats during chow and sucrose consumption. We found that activation of VP GABA neurons increases consumption of chow and sucrose in male rats, but not female rats. We also found that, while inhibition of VP GABA neurons tended to decrease sucrose consumption, this effect was not statistically significant. Together, these findings suggest that activation of VP GABA neurons can stimulate consumption of routine or highly palatable rewards selectively in male rats.

Ventral pallidum neurons projecting to the ventral tegmental area reinforce but do not invigorate reward-seeking behavior.

Reward-predictive cues acquire motivating and reinforcing properties that contribute to the escalation and relapse of drug use in addiction. The ventral pallidum (VP) and ventral tegmental area (VTA) are two key nodes in brain reward circuitry implicated in addiction and cue-driven behavior. In the current study, we use in vivo fiber photometry and optogenetics to record from and manipulate VP→VTA in rats performing a discriminative stimulus task to determine the role these neurons play in invigoration and reinforcement by reward cues. We find that VP→VTA neurons are active during reward consumption and that optogenetic stimulation of these neurons biases choice behavior and is reinforcing. Critically, we find no encoding of reward-seeking vigor, and optogenetic stimulation does not enhance the probability or vigor of reward seeking in response to cues. Our results suggest that VP→VTA activity is more important for reinforcement than for invigoration of reward seeking by cues.

Nucleus accumbens GABAergic inhibition generates intense eating and fear that resists environmental retuning and needs no local dopamine.

Intense fearful behavior and/or intense appetitive eating behavior can be generated by localized amino acid inhibitions along a rostrocaudal anatomical gradient within medial shell of nucleus accumbens of the rat. This can be produced by microinjections in medial shell of either the γ-aminobutyric acid (GABA)A agonist muscimol (mimicking intrinsic GABAergic inputs) or the AMPA (α-amino-3-hydroxy-5-methylisoxazole-4-propionic acid) antagonist DNQX (6,7-dinitroquinoxaline-2,3-dione), disrupting corticolimbic glutamate inputs). At rostral sites in medial shell, each drug robustly stimulates appetitive eating and food intake, whereas at more caudal sites the same drugs instead produce increasingly fearful behaviors such as escape, distress vocalizations and defensive treading (an antipredator behavior rodents emit to snakes and scorpions). Previously we showed that intense motivated behaviors generated by glutamate blockade require local endogenous dopamine and can be modulated in valence by environmental ambience. Here we investigated whether GABAergic generation of intense appetitive and fearful motivations similarly depends on local dopamine signals, and whether the valence of motivations generated by GABAergic inhibition can also be retuned by changes in environmental ambience. We report that the answer to both questions is 'no'. Eating and fear generated by GABAergic inhibition of accumbens shell does not need endogenous dopamine. Also, the appetitive/fearful valence generated by GABAergic muscimol microinjections resists environmental retuning and is determined almost purely by rostrocaudal anatomical placement. These results suggest that nucleus accumbens GABAergic release of fear and eating are relatively independent of modulatory dopamine signals, and more anatomically pre-determined in valence balance than release of the same intense behaviors by glutamate disruptions.

Outcome devaluation by sensory-specific satiety alters Pavlovian-conditioned behavior in male and female rats.

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Sex-biased effects of outcome devaluation by sensory-specific satiety on Pavlovian-conditioned behavior.

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Reinstatement of Pavlovian responses to alcohol cues by stress.

RATIONALE: Stress may contribute to relapse to alcohol use in part by enhancing reactivity to cues previously paired with alcohol. Yet, standard models of stress-induced reinstatement generally use contingent presentations of alcohol-paired cues to reinforce instrumental behaviors, making it difficult to isolate the ability of cues to invigorate alcohol-seeking. OBJECTIVE: Here we sought to test the impact of stress on behavioral responses to alcohol-paired cues, using a model of stress-induced reinstatement of Pavlovian conditioned approach, inspired by Nadia Chaudhri's work on context-induced reinstatement. METHODS: Long Evans rats were trained to associate one auditory cue with delivery of alcohol or sucrose and an alternative auditory cue with no reward. Following extinction training, rats were exposed to a stressor prior to being re-exposed to the cues under extinction conditions. We assessed the effects of yohimbine, intermittent footshock and olfactory cues paired with social defeat on responses to alcohol-paired cues and the effects of yohimbine on responses to sucrose-paired cues. RESULTS: The pharmacological stressor, yohimbine, enhanced alcohol seeking in a Pavlovian setting, but not in a cue-selective manner. Intermittent footshock and social defeat cues did not enhance alcohol seeking in this paradigm. CONCLUSIONS: While yohimbine elicited reinstatement of reward-seeking in a Pavlovian setting, these effects may be unrelated to activation of stress systems or to interactions with specific cues.

Nucleus accumbens dopamine/glutamate interaction switches modes to generate desire versus dread: D(1) alone for appetitive eating but D(1) and D(2) together for fear.

The medial shell of nucleus accumbens (NAc) and its mesolimbic dopamine inputs mediate forms of fearful as well as of incentive motivation. For example, either appetitive and/or actively fearful behaviors are generated in a keyboard pattern by localized glutamate disruptions in NAc (via microinjection of the AMPA receptor antagonist DNQX) at different anatomical locations along a rostrocaudal gradient within the medial shell of rats. Rostral glutamate disruptions produce intense increases in eating, but more caudally placed disruptions produce increasingly fearful behaviors: distress vocalizations and escape attempts to human touch, and a spontaneous and directed antipredator response called defensive treading/burying. Local endogenous dopamine is required for either intense motivation to be generated by AMPA disruptions. Here we report that only endogenous local signaling at D(1) dopamine receptors is needed for rostral generation of excessive eating, potentially implicating a direct output pathway contribution. In contrast, fear generation at caudal sites requires both D(1) and D(2) signaling simultaneously, potentially implicating an indirect output pathway contribution. Finally, when motivation valence generated by AMPA disruptions at intermediate sites was flipped by manipulating environmental ambience, from mostly appetitive in a comfortable home environment to mostly fearful in a stressful environment, the roles of local D(1) and D(2) signaling in dopamine/glutamate interaction at microinjection sites also switched dynamically to match the motivation valence generated at the moment. Thus, NAc D(1) and D(2) receptors, and their associated neuronal circuits, play different and dynamic roles in enabling desire and dread to be generated by localized NAc glutamate disruptions in medial shell.

Alcohol availability during withdrawal gates the impact of alcohol vapor exposure on responses to alcohol cues.

RATIONALE: Chronic intermittent ethanol (CIE) vapor inhalation is a widely used model of alcohol dependence, but the impact of CIE on cue-elicited alcohol seeking is poorly understood. OBJECTIVE: Here, we assessed the effects of CIE on alcohol-seeking elicited by cues paired with alcohol before or after CIE vapor inhalation. METHODS: In experiment 1, male and female Long-Evans rats were trained in a discriminative stimulus (DS) task, in which one auditory cue (the DS) predicts the availability of 15% ethanol and a control cue (the NS) predicts no ethanol. Rats then underwent CIE or served as controls. Subsets of each group received access to oral ethanol twice a week during acute withdrawal. After CIE, rats were presented with the DS and NS cues under extinction and retraining conditions to determine whether they would alter their responses to these cues. In experiment 2, rats underwent CIE prior to training in the DS task. RESULTS: CIE enhanced behavioral responses to cues previously paired with alcohol, but only in rats that received access to alcohol during acute withdrawal. When CIE occurred before task training, male rats were slower to develop cue responses and less likely to enter the alcohol port, even though they had received alcohol during acute withdrawal. CONCLUSIONS: These results suggest that CIE vapor inhalation alone does not potentiate the motivational value of alcohol cues but that an increase in cue responses requires alcohol experience during acute withdrawal. Furthermore, under some conditions, CIE may disrupt responses to alcohol-paired cues.

Metabotropic glutamate receptor blockade in nucleus accumbens shell shifts affective valence towards fear and disgust.

Glutamatergic inputs to the nucleus accumbens (NAc) modulate both appetitive and fearful motivation. It has been suggested that pathological disturbances of glutamate signaling in NAc contribute to motivation disorders, ranging from excessive desire in drug addiction to paranoia in schizophrenia. Metabotropic glutamate receptors are of special interest, as metabotropic Group II receptor (mglu2/3) agonists have been proposed as potential treatments for both addiction and schizophrenia. Here we tested whether local mglu2/3 receptor blockade in the medial shell of the rat NAc can generate intense distortions of motivation or affect, which might model clinical dysfunction. We found that microinjection of the mglu2/3 antagonist LY341495 at sites throughout medial shell suppressed appetitive motivation to eat and drink. Simultaneously, LY341495 microinjections generated fearful motivation in the form of defensive treading or burying. To assess whether the valence shift extended into a parallel hedonic shift from affective 'liking' to 'disliking' we employed the taste reactivity test, which measures affective orofacial reactions to the sensory pleasure or displeasure of tastes. We found that LY341495 microinjections reduced positive 'liking' reactions to sucrose and enhanced 'disliking' reactions. Overall, mglu2/3 antagonism at most shell sites produced a similar valence shift from positive to negative. This pattern comprised (i) generation of fearful behaviors, and (ii) induction of aversive affective reactions, together with (iii) loss of appetitive ingestion and (iv) loss of 'liking' for rewards. These results are discussed in terms of implications for clinical disorders and the influence of corticolimbic glutamate inputs to NAc in the generation of motivation and affect.

Reward activity in ventral pallidum tracks satiety-sensitive preference and drives choice behavior.

A key function of the nervous system is producing adaptive behavior across changing conditions, like physiological state. Although states like thirst and hunger are known to impact decision-making, the neurobiology of this phenomenon has been studied minimally. Here, we tracked evolving preference for sucrose and water as rats proceeded from a thirsty to sated state. As rats shifted from water choices to sucrose choices across the session, the activity of a majority of neurons in the ventral pallidum, a region crucial for reward-related behaviors, closely matched the evolving behavioral preference. The timing of this signal followed the pattern of a reward prediction error, occurring at the cue or the reward depending on when reward identity was revealed. Additionally, optogenetic stimulation of ventral pallidum neurons at the time of reward was able to reverse behavioral preference. Our results suggest that ventral pallidum neurons guide reward-related decisions across changing physiological states.

A quantitative reward prediction error signal in the ventral pallidum.

The nervous system is hypothesized to compute reward prediction errors (RPEs) to promote adaptive behavior. Correlates of RPEs have been observed in the midbrain dopamine system, but the extent to which RPE signals exist in other reward-processing regions is less well understood. In the present study, we quantified outcome history-based RPE signals in the ventral pallidum (VP), a basal ganglia region functionally linked to reward-seeking behavior. We trained rats to respond to reward-predicting cues, and we fit computational models to predict the firing rates of individual neurons at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did so more robustly than the nucleus accumbens, an input to the VP. VP RPEs predicted changes in task engagement, and optogenetic manipulation of the VP during reward delivery bidirectionally altered rats' subsequent reward-seeking behavior. Our data suggest a pivotal role for the VP in computing teaching signals that influence adaptive reward seeking.

Mesolimbic dopamine in desire and dread: enabling motivation to be generated by localized glutamate disruptions in nucleus accumbens.

An important issue in affective neuroscience concerns the role of mesocorticolimbic dopamine systems in positive-valenced motivation (e.g., reward) versus negative-valenced motivation (e.g., fear). Here, we assessed whether endogenous dopamine receptor stimulation in nucleus accumbens contributes to both appetitive behavior and fearful behavior that is generated in keyboard manner by local glutamate disruptions at different sites in medial shell. 6,7-Dinitroquinoxaline-2,3(1H,4H)-dione (DNQX) microinjections (450 ng) locally disrupt glutamate signals in <4 mm(3) of nucleus accumbens, and generate either desire or fear (or both) depending on precise rostrocaudal location in medial shell. At rostral shell sites, local AMPA/kainate blockade generates positive ingestive behavior, but the elicited motivated behavior becomes incrementally more fearful as the same microinjection is moved caudally. A dopamine-blocking mixture of D(1) and D(2) antagonists (raclopride and SCH-23390 [R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5,-tetrahydro-1H-3-benzazepine hydrochloride]) was combined here in the same microinjection with DNQX to assess the role of endogenous local dopamine in mediating the DNQX-motivated behaviors. We report that local dopamine blockade prevented DNQX microinjections from generating appetitive behavior (eating) in rostral shell, and equally prevented DNQX from generating fearful behavior (defensive treading) in caudal shell. We conclude that local dopamine is needed to enable disruptions of corticolimbic glutamate signals in shell to generate either positive incentive salience or negative fearful salience (valence depending on site and other conditions). Thus, dopamine interacts with localization of valence-biased glutamate circuits in medial shell to facilitate keyboard stimulation of both appetitive and fearful motivations.

Distinct recruitment of dorsomedial and dorsolateral striatum erodes with extended training.

Hypotheses of striatal orchestration of behavior ascribe distinct functions to striatal subregions, with the dorsolateral striatum (DLS) especially implicated in habitual and skilled performance. Thus neural activity patterns recorded from the DLS, but not the dorsomedial striatum (DMS), should be correlated with habitual and automatized performance. Here, we recorded DMS and DLS neural activity in rats during training in a task promoting habitual lever pressing. Despite improving performance across sessions, clear changes in corresponding neural activity patterns were not evident in DMS or DLS during early training. Although DMS and DLS activity patterns were distinct during early training, their activity was similar following extended training. Finally, performance after extended training was not associated with DMS disengagement, as would be predicted from prior work. These results suggest that behavioral sequences may continue to engage both striatal regions long after initial acquisition, when skilled performance is consolidated.

Ventral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens.

The ventral striatopallidal system, a basal ganglia network thought to convert limbic information into behavioral action, includes the nucleus accumbens (NAc) and the ventral pallidum (VP), typically described as a major output of NAc. Here, to investigate how reward-related information is transformed across this circuit, we measure the activity of neurons in NAc and VP when rats receive two highly palatable but differentially preferred rewards, allowing us to track the reward-specific information contained within the neural activity of each region. In VP, we find a prominent preference-related signal that flexibly reports the relative value of reward outcomes across multiple conditions. This reward-specific firing in VP is present in a greater proportion of the population and arises sooner following reward delivery than in NAc. Our findings establish VP as a preeminent value signaler and challenge the existing model of information flow in the ventral basal ganglia.

Ventral pallidal encoding of reward-seeking behavior depends on the underlying associative structure.

Despite its being historically conceptualized as a motor expression site, emerging evidence suggests the ventral pallidum (VP) plays a more active role in integrating information to generate motivation. Here, we investigated whether rat VP cue responses would encode and contribute similarly to the vigor of reward-seeking behaviors trained under Pavlovian versus instrumental contingencies, when these behavioral responses consist of superficially similar locomotor response patterns but may reflect distinct underlying decision-making processes. We find that cue-elicited activity in many VP neurons predicts the latency of instrumental reward seeking, but not of Pavlovian response latency. Further, disruption of VP signaling increases the latency of instrumental but not Pavlovian reward seeking. This suggests that VP encoding of and contributions to response vigor are specific to the ability of incentive cues to invigorate reward-seeking behaviors upon which reward delivery is contingent.