Consolidating the Circuit Model for Addiction.

Addiction is a disease characterized by compulsive drug seeking and consumption observed in 20-30% of users. An addicted individual will favor drug reward over natural rewards, despite major negative consequences. Mechanistic research on rodents modeling core components of the disease has identified altered synaptic transmission as the functional substrate of pathological behavior. While the initial version of a circuit model for addiction focused on early drug adaptive behaviors observed in all individuals, it fell short of accounting for the stochastic nature of the transition to compulsion. The model builds on the initial pharmacological effect common to all addictive drugs-an increase in dopamine levels in the mesolimbic system. Here, we consolidate this early model by integrating circuits underlying compulsion and negative reinforcement. We discuss the genetic and epigenetic correlates of individual vulnerability. Many recent data converge on a gain-of-function explanation for circuit remodeling, revealing blueprints for novel addiction therapies.

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement.

Reward seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement, and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations in male and female rats, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit, whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is nonscalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies.SIGNIFICANCE STATEMENT Dopamine neurons in the midbrain are essential for learning, motivation, and movement. Here we describe in detail the ability of VTA and SNc dopamine neurons to generate instrumental reinforcement, a process where an agent learns about actions they can emit to earn reward. While rats will avidly work and learn to respond for activation of VTA and SNc dopamine neurons, we find that only VTA dopamine neurons imbue actions and their associated cues with motivational value that spur continued pursuit of reward. Our data support a hypothesis that VTA and SNc dopamine neurons engage distinct psychological processes that have consequences for our understanding of these neurons in health and disease.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance.

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Maintained goal-directed control with overtraining on ratio schedules.

It is thought that goal-directed control of actions weakens or becomes masked by habits over time. We tested the opposing hypothesis that goal-directed control becomes stronger over time, and that this growth is modulated by the overall action-outcome contiguity. Despite group differences in action-outcome contiguity early in training, rats trained under random and fixed ratio schedules showed equivalent goal-directed control of lever pressing that appeared to grow over time. We confirmed that goal-directed control was maintained after extended training under another type of ratio schedule-continuous reinforcement-using specific satiety and taste aversion devaluation methods. These results add to the growing literature showing that extensive training does not reliably weaken goal-directed control and that it may strengthen it, or at least maintain it.

From circuits to behaviour in the amygdala.

The amygdala has long been associated with emotion and motivation, playing an essential part in processing both fearful and rewarding environmental stimuli. How can a single structure be crucial for such different functions? With recent technological advances that allow for causal investigations of specific neural circuit elements, we can now begin to map the complex anatomical connections of the amygdala onto behavioural function. Understanding how the amygdala contributes to a wide array of behaviours requires the study of distinct amygdala circuits.

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking.

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event-an occasion setter (OS)-which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking.

A critical area of inquiry in the neurobiology of alcohol abuse is the mechanism by which cues gain the ability to elicit alcohol use. Previously, we found that cue-evoked activity in rat ventral pallidum robustly encodes the value of sucrose cues trained under both Pavlovian and instrumental contingencies, despite a stronger relationship between cue-evoked activity and behavioral latency after instrumental training (Richard et al., 2018, Elife, 7, e33107). Here, we assessed: (a) ventral pallidal representations of Pavlovian versus instrumental cues trained with alcohol reward, and (b) the impact of non-associative alcohol exposure on ventral pallidal representations of sucrose cues. Decoding of cue identity based on ventral pallidum firing was blunted for the Pavlovian alcohol cue in comparison to both the instrumental cue trained with alcohol and either cue type trained with sucrose. Further, non-associative alcohol exposure had opposing effects on ventral pallidal encoding of sucrose cues trained on instrumental versus Pavlovian associations, enhancing decoding accuracy for an instrumental discriminative stimulus and reducing decoding accuracy for a Pavlovian conditioned stimulus. These findings suggest that alcohol exposure can drive biased engagement of specific reward-related signals in the ventral pallidum.

Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking.

The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core (NAcC). These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined whether the diminishing dopamine signal in the NAcC has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the NAcC or the dorsal lateral striatum (DLS). Dopamine antagonism within the NAcC attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the DLS affected sign-tracking behavior. Critically, extended training did not alter these effects. Although extended experience with an incentive stimulus may reduce cue-evoked dopamine in the NAcC, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training.

Nucleus accumbens core and shell are differentially involved in general and outcome-specific forms of Pavlovian-instrumental transfer with alcohol and sucrose rewards.

Alcohol-associated stimuli contribute to relapse risk. Therefore, understanding the behavioural and neural mechanisms underlying the ability of such stimuli to promote alcohol-seeking is important for developing effective treatments for alcohol-use disorders. The Pavlovian-instrumental transfer (PIT) paradigm can be used to study the influence of Pavlovian cues on independently-trained instrumental responses earning reward. The effects can be either general, increasing the vigour of reward-related behaviours, or specific to responses that earn a common outcome. These different forms of PIT are mediated by distinct neural circuits involving the nucleus accumbens (NAC) core and shell, respectively. Here we examined the effects of pharmacological inactivation of either the NAC core or shell on PIT generated by alcohol-predictive and sucrose-predictive stimuli in rats. We found that presentations of a stimulus predicting sucrose enhanced responding for sucrose but not alcohol, suggesting an outcome-specific effect. In contrast, presentations of an alcohol-predictive stimulus enhanced responding for both alcohol and sucrose, suggesting a generally arousing effect. Inactivation of the NAC core reduced PIT and, in particular, the effect of the alcohol stimulus. Inactivation of the NAC shell reduced the specificity of the stimulus effects but left the ability of the stimuli to non-specifically invigorate responding intact, consistent with a role in mediating the specificity of PIT. Together, these results suggest that the NAC core plays a particularly important role in mediating the influence of alcohol-predictive cues on reward-seeking behaviours.

Habitual Alcohol Seeking: Neural Bases and Possible Relations to Alcohol Use Disorders.

Loss of flexible control over alcohol use may contribute to the development of alcohol use disorders. An increased contribution of response habits to alcohol-related behaviors may help explain this loss of control. Focusing on data from outcome devaluation and Pavlovian-instrumental transfer procedures, we review evidence for loss of goal-directed control over alcohol seeking and consumption drawing from both preclinical findings and clinical data where they exist. Over the course of extended alcohol self-administration and exposure, the performance of alcohol-seeking responses becomes less sensitive to reduction in the value of alcohol and more vulnerable to the influences of alcohol-predictive stimuli. These behavioral changes are accompanied by a shift in the corticostriatal circuits that control responding from circuits centered on the dorsomedial to those centered on the dorsolateral striatum. These changes in behavioral and neural control could help explain failures to abstain from alcohol despite intention to do so. Understanding and ultimately ameliorating these changes will aid development of more effective treatment interventions.

Safety encoding in the basal amygdala.

Learning to fear and avoid life-threatening stimuli are critical survival skills but are maladaptive when they persist in the absence of a direct threat. Thus, it is important to detect when a situation is safe and to increase behaviors leading to naturally rewarding actions, such as feeding and mating. It is unclear how the brain distinguishes between dangerous and safe situations. Here, we present a novel protocol designed to investigate the processing of cues that predict danger, safety, or reward (sucrose). In vivo single unit recordings were obtained in the basal amygdala of freely behaving rats undergoing simultaneous reward, fear, and safety conditioning. We observed a population of neurons that did not respond to a Fear Cue but did change their firing rate during the combined presentation of a fear cue simultaneous with a second, safety, cue; this combination of Fear + Safety Cues signified "no shock." This neural population consisted of two subpopulations: neurons that responded to the Fear + Safety Cue but not the Fear or Reward Cue ("safety" neurons), and neurons that responded to the Fear + Safety and Reward Cue but not the Fear Cue ("safety + reward" neurons). These data demonstrate the presence of neurons in the basal amygdala that are selectively responsive to Safety Cues. Furthermore, these data suggest that safety and reward learning use overlapping mechanisms in the basal amygdala.

The orbitofrontal cortex as part of a hierarchical neural system mediating choice between two good options.

Animals rely on environmental cues to identify potential rewards and select the best reward available. The orbitofrontal cortex (OFC) is proposed to encode sensory-specific representations of expected outcome. However, its contribution to the selection of a preferred outcome among different reward options is still unclear. We investigated the effect of transient OFC inactivation (achieved by presession injection of muscimol and baclofen) in a novel two-reward choice task. In discrete trials, rats could choose between a solution of polycose and an equally caloric, but highly preferred, solution of sucrose by visiting one of two liquid dispensers after the presentation of a specific cue signaling the availability of one or both of the solutions. We found that OFC inactivation did not affect outcome preference: rats maintained high preference for sucrose and adapted their behavioral responding when the cue-outcome contingencies were reversed. However, when rats were tested drug-free 24 h after OFC inactivation and reversal learning, memory for the newly learned contingencies was poor. These results suggest a potential conflict between OFC (encoding pre-reversal contingencies) and other brain circuits (encoding the new contingencies). Remarkably, repeating the OFC inactivation before the reversal memory test restored normal behavior, confirming the hypothesis of a dominant impact of OFC on other decision-making circuits. These results indicate that the representations encoded in the OFC, while not essential to the expression of outcome preference, exert hierarchical control on downstream decision-making circuits.

Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning.

What neural changes underlie individual differences in goal-directed learning? The lateral amygdala (LA) is important for assigning emotional and motivational significance to discrete environmental cues, including those that signal rewarding events. Recognizing that a cue predicts a reward enhances an animal's ability to acquire that reward; however, the cellular and synaptic mechanisms that underlie cue-reward learning are unclear. Here we show that marked changes in both cue-induced neuronal firing and input-specific synaptic strength occur with the successful acquisition of a cue-reward association within a single training session. We performed both in vivo and ex vivo electrophysiological recordings in the LA of rats trained to self-administer sucrose. We observed that reward-learning success increased in proportion to the number of amygdala neurons that responded phasically to a reward-predictive cue. Furthermore, cue-reward learning induced an AMPA (alpha-amino-3-hydroxy-5-methyl-isoxazole propionic acid)-receptor-mediated increase in the strength of thalamic, but not cortical, synapses in the LA that was apparent immediately after the first training session. The level of learning attained by individual subjects was highly correlated with the degree of synaptic strength enhancement. Importantly, intra-LA NMDA (N-methyl-d-aspartate)-receptor blockade impaired reward-learning performance and attenuated the associated increase in synaptic strength. These findings provide evidence of a connection between LA synaptic plasticity and cue-reward learning, potentially representing a key mechanism underlying goal-directed behaviour.

Extrasynaptic delta-containing GABAA receptors in the nucleus accumbens dorsomedial shell contribute to alcohol intake.

Recent findings suggest that extrasynaptic δ-subunit-containing GABA(A) receptors are sensitive to low-to-moderate concentrations of alcohol, raising the possibility that these receptors mediate the reinforcing effects of alcohol after consumption of one or a few drinks. We used the technique of viral-mediated RNAi to reduce expression of the GABA(A) receptor δ-subunit in adult rats in localized regions of the nucleus accumbens (NAc) to test the hypothesis that δ-subunit-containing GABA(A) receptors in the NAc are necessary for oral alcohol consumption. We found that knockdown of the δ-subunit in the medial shell region of the NAc, but not in the ventral or lateral shell or in the core, reduced alcohol intake. In contrast, δ-subunit knockdown in the medial shell did not affect intake of a 2% sucrose solution, suggesting that the effects of GABA(A) receptor δ-subunit reduction are specific to alcohol. These results provide strong evidence that extrasynaptic δ-subunit-containing GABA(A) receptors in the medial shell of the NAc are critical for the reinforcing effects of oral ethanol.

Chronic Ethanol Exposure Produces Persistent Impairment in Cognitive Flexibility and Decision Signals in the Striatum.

Lack of cognitive flexibility is a hallmark of substance use disorders and has been associated with drug-induced synaptic plasticity in the dorsomedial striatum (DMS). Yet the possible impact of altered plasticity on real-time striatal neural dynamics during decision-making is unclear. Here, we identified persistent impairments induced by chronic ethanol (EtOH) exposure on cognitive flexibility and striatal decision signals. After a substantial withdrawal period from prior EtOH vapor exposure, male, but not female, rats exhibited reduced adaptability and exploratory behavior during a dynamic decision-making task. Reinforcement learning models showed that prior EtOH exposure enhanced learning from rewards over omissions. Notably, neural signals in the DMS related to the decision outcome were enhanced, while those related to choice and choice-outcome conjunction were reduced, in EtOH-treated rats compared to the controls. These findings highlight the profound impact of chronic EtOH exposure on adaptive decision-making, pinpointing specific changes in striatal representations of actions and outcomes as underlying mechanisms for cognitive deficits.

Basolateral amygdala population coding of a cued reward seeking state depends on orbitofrontal cortex.

Basolateral amygdala (BLA) neuronal responses to conditioned stimuli are closely linked to the expression of conditioned behavior. An area of increasing interest is how the dynamics of BLA neurons relate to evolving behavior. Here, we recorded the activity of individual BLA neurons across the acquisition and extinction of conditioned reward seeking and employed population-level analyses to assess ongoing neural dynamics. We found that, with training, sustained cue-evoked activity emerged that discriminated between the CS+ and CS- and correlated with conditioned responding. This sustained population activity continued until reward receipt and rapidly extinguished along with conditioned behavior during extinction. To assess the contribution of orbitofrontal cortex (OFC), a major reciprocal partner to BLA, to this component of BLA neural activity, we inactivated OFC while recording in BLA and found blunted sustained cue-evoked activity in BLA that accompanied reduced reward seeking. Optogenetic disruption of BLA activity and OFC terminals in BLA also reduced reward seeking. Our data suggest that sustained cue-driven activity in BLA, which in part depends on OFC input, underlies conditioned reward-seeking states.

Pavlovian cue-evoked alcohol seeking is disrupted by ventral pallidal inhibition.

Cues paired with alcohol can be potent drivers of craving, alcohol-seeking, consumption, and relapse. While the ventral pallidum is implicated in appetitive and consummatory responses across several reward classes and types of behaviors, its role in behavioral responses to Pavlovian alcohol cues has not previously been established. Here, we tested the impact of optogenetic inhibition of ventral pallidum on Pavlovian-conditioned alcohol-seeking in male Long Evans rats. Rats underwent Pavlovian conditioning with an auditory cue predicting alcohol delivery to a reward port and a control cue predicting no alcohol delivery, until they consistently entered the reward port more during the alcohol cue than the control cue. We then tested the within-session effects of optogenetic inhibition during 50% of cue presentations. We found that optogenetic inhibition of ventral pallidum during the alcohol cue reduced port entry likelihood and time spent in the port, and increased port entry latency. Overall, these results suggest that normal ventral pallidum activity is necessary for Pavlovian alcohol-seeking.

Pramipexole restores behavioral inhibition in highly impulsive rats through a paradoxical modulation of frontostriatal networks.

Impulse control disorders (ICDs), a wide spectrum of maladaptive behaviors which includes pathological gambling, hypersexuality and compulsive buying, have been recently suggested to be triggered or aggravated by treatments with dopamine D2/3 receptor agonists, such as pramipexole (PPX). Despite evidence showing that impulsivity is associated with functional alterations in corticostriatal networks, the neural basis of the exacerbation of impulsivity by PPX has not been elucidated. Here we used a hotspot analysis to assess the functional recruitment of several corticostriatal structures by PPX in male rats identified as highly (HI), moderately impulsive (MI) or with low levels of impulsivity (LI) in the 5-choice serial reaction time task (5-CSRTT). PPX dramatically reduced impulsivity in HI rats. Assessment of the expression pattern of the two immediate early genes C-fos and Zif268 by in situ hybridization subsequently revealed that PPX resulted in a decrease in Zif268 mRNA levels in different striatal regions of both LI and HI rats accompanied by a high impulsivity specific reduction of Zif268 mRNA levels in prelimbic and cingulate cortices. PPX also decreased C-fos mRNA levels in all striatal regions of LI rats, but only in the dorsolateral striatum and nucleus accumbens core (NAc Core) of HI rats. Structural equation modeling further suggested that the anti-impulsive effect of PPX was mainly attributable to the specific downregulation of Zif268 mRNA in the NAc Core. Altogether, our results show that PPX restores impulse control in highly impulsive rats by modulation of limbic frontostriatal circuits.

Encoding and context-dependent control of reward consumption within the central nucleus of the amygdala.

Dysregulation of the central amygdala is thought to underlie aberrant choice in alcohol use disorder, but the role of central amygdala neural activity during reward choice and consumption is unclear. We recorded central amygdala neurons in male rats as they consumed alcohol or sucrose. We observed activity changes at the time of reward approach, as well as lick-entrained activity during ongoing consumption of both rewards. In choice scenarios where rats could drink sucrose, alcohol, or quinine-adulterated alcohol with or without central amygdala optogenetic stimulation, rats drank more of stimulation-paired options when the two bottles contained identical options. Given a choice among different options, central amygdala stimulation usually enhanced consumption of stimulation-paired rewards. However, optogenetic stimulation during consumption of the less-preferred option, alcohol, was unable to enhance alcohol intake while sucrose was available. These findings indicate that the central amygdala contributes to refining motivated pursuit toward the preferred available option.

Mesostriatal dopamine is sensitive to changes in specific cue-reward contingencies.

Learning causal relationships relies on understanding how often one event precedes another. To investigate how dopamine neuron activity and neurotransmitter release change when a retrospective relationship is degraded for a specific pair of events, we used outcome-selective Pavlovian contingency degradation in rats. Conditioned responding was attenuated for the cue-reward contingency that was degraded, as was dopamine neuron activity in the midbrain and dopamine release in the ventral striatum in response to the cue and subsequent reward. Contingency degradation also abolished the trial-by-trial history dependence of the dopamine responses at the time of trial outcome. This profile of changes in cue- and reward-evoked responding is not easily explained by a standard reinforcement learning model. An alternative model based on learning causal relationships was better able to capture dopamine responses during contingency degradation, as well as conditioned behavior following optogenetic manipulations of dopamine during noncontingent rewards. Our results suggest that mesostriatal dopamine encodes the contingencies between meaningful events during learning.