Consolidating the Circuit Model for Addiction.

Addiction is a disease characterized by compulsive drug seeking and consumption observed in 20-30% of users. An addicted individual will favor drug reward over natural rewards, despite major negative consequences. Mechanistic research on rodents modeling core components of the disease has identified altered synaptic transmission as the functional substrate of pathological behavior. While the initial version of a circuit model for addiction focused on early drug adaptive behaviors observed in all individuals, it fell short of accounting for the stochastic nature of the transition to compulsion. The model builds on the initial pharmacological effect common to all addictive drugs-an increase in dopamine levels in the mesolimbic system. Here, we consolidate this early model by integrating circuits underlying compulsion and negative reinforcement. We discuss the genetic and epigenetic correlates of individual vulnerability. Many recent data converge on a gain-of-function explanation for circuit remodeling, revealing blueprints for novel addiction therapies.

Orbitofrontal cortex mediates sustained basolateral amygdala encoding of cued reward seeking states.

Basolateral amygdala (BLA) neurons are engaged by emotionally salient stimuli. An area of increasing interest is how BLA dynamics relate to evolving reward-seeking behavior, especially under situations of uncertainty or ambiguity. Here, we recorded the activity of individual BLA neurons in male rats across the acquisition and extinction of conditioned reward seeking. We assessed ongoing neural dynamics in a task where long reward cue presentations preceded an unpredictable, variably-time reward delivery. We found that, with training, BLA neurons discriminated the CS+ and CS- cues with sustained cue-evoked activity that correlated with behavior and terminated only after reward receipt. BLA neurons were bidirectionally modulated, with a majority showing prolonged inhibition during cued reward seeking. Strikingly, population-level analyses revealed that neurons showing cue-evoked inhibitions and those showing excitations similarly represented the CS+ and behavioral state. This sustained population code rapidly extinguished in parallel with conditioned behavior. We next assessed the contribution of orbitofrontal cortex (OFC), a major reciprocal partner to BLA. Inactivation of OFC while simultaneously recording in BLA revealed a blunting of sustained cue-evoked activity in BLA that accompanied reduced reward seeking. Optogenetic disruption of BLA activity and OFC terminals in BLA also reduced reward seeking. Our data indicate that the BLA represents reward seeking states via sustained, bidirectional cue-driven neural encoding. This code is regulated by cortical input and is important for the maintenance of vigilant reward seeking behavior.Significance Statement Appropriate representation of the current need for motivated reward seeking, especially under situations of uncertainty or ambiguity, is critical for adaptive behavior. Here, we recorded activity of neurons in the basolateral amygdala (BLA) in rats during conditioned reward seeking, finding a sustained cue-evoked population-level code, which terminated once reward was received. Inactivation of a major BLA input, the orbitofrontal cortex (OFC), blunted sustained cue-evoked activity in BLA and reduced reward seeking. Optogenetic disruption of BLA activity and OFC terminals in BLA also reduced reward seeking. Together, these results show that the BLA represents conditioned motivational states with sustained neural activity - this signal is critical for cue-invigorated reward seeking and depends on functional input from the orbitofrontal cortex.

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement.

Reward seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement, and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations in male and female rats, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit, whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is nonscalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies.SIGNIFICANCE STATEMENT Dopamine neurons in the midbrain are essential for learning, motivation, and movement. Here we describe in detail the ability of VTA and SNc dopamine neurons to generate instrumental reinforcement, a process where an agent learns about actions they can emit to earn reward. While rats will avidly work and learn to respond for activation of VTA and SNc dopamine neurons, we find that only VTA dopamine neurons imbue actions and their associated cues with motivational value that spur continued pursuit of reward. Our data support a hypothesis that VTA and SNc dopamine neurons engage distinct psychological processes that have consequences for our understanding of these neurons in health and disease.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance.

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Maintained goal-directed control with overtraining on ratio schedules.

It is thought that goal-directed control of actions weakens or becomes masked by habits over time. We tested the opposing hypothesis that goal-directed control becomes stronger over time, and that this growth is modulated by the overall action-outcome contiguity. Despite group differences in action-outcome contiguity early in training, rats trained under random and fixed ratio schedules showed equivalent goal-directed control of lever pressing that appeared to grow over time. We confirmed that goal-directed control was maintained after extended training under another type of ratio schedule-continuous reinforcement-using specific satiety and taste aversion devaluation methods. These results add to the growing literature showing that extensive training does not reliably weaken goal-directed control and that it may strengthen it, or at least maintain it.

From circuits to behaviour in the amygdala.

The amygdala has long been associated with emotion and motivation, playing an essential part in processing both fearful and rewarding environmental stimuli. How can a single structure be crucial for such different functions? With recent technological advances that allow for causal investigations of specific neural circuit elements, we can now begin to map the complex anatomical connections of the amygdala onto behavioural function. Understanding how the amygdala contributes to a wide array of behaviours requires the study of distinct amygdala circuits.

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking.

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event-an occasion setter (OS)-which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking.

A critical area of inquiry in the neurobiology of alcohol abuse is the mechanism by which cues gain the ability to elicit alcohol use. Previously, we found that cue-evoked activity in rat ventral pallidum robustly encodes the value of sucrose cues trained under both Pavlovian and instrumental contingencies, despite a stronger relationship between cue-evoked activity and behavioral latency after instrumental training (Richard et al., 2018, Elife, 7, e33107). Here, we assessed: (a) ventral pallidal representations of Pavlovian versus instrumental cues trained with alcohol reward, and (b) the impact of non-associative alcohol exposure on ventral pallidal representations of sucrose cues. Decoding of cue identity based on ventral pallidum firing was blunted for the Pavlovian alcohol cue in comparison to both the instrumental cue trained with alcohol and either cue type trained with sucrose. Further, non-associative alcohol exposure had opposing effects on ventral pallidal encoding of sucrose cues trained on instrumental versus Pavlovian associations, enhancing decoding accuracy for an instrumental discriminative stimulus and reducing decoding accuracy for a Pavlovian conditioned stimulus. These findings suggest that alcohol exposure can drive biased engagement of specific reward-related signals in the ventral pallidum.

Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking.

The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core (NAcC). These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined whether the diminishing dopamine signal in the NAcC has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the NAcC or the dorsal lateral striatum (DLS). Dopamine antagonism within the NAcC attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the DLS affected sign-tracking behavior. Critically, extended training did not alter these effects. Although extended experience with an incentive stimulus may reduce cue-evoked dopamine in the NAcC, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training.

Nucleus accumbens core and shell are differentially involved in general and outcome-specific forms of Pavlovian-instrumental transfer with alcohol and sucrose rewards.

Alcohol-associated stimuli contribute to relapse risk. Therefore, understanding the behavioural and neural mechanisms underlying the ability of such stimuli to promote alcohol-seeking is important for developing effective treatments for alcohol-use disorders. The Pavlovian-instrumental transfer (PIT) paradigm can be used to study the influence of Pavlovian cues on independently-trained instrumental responses earning reward. The effects can be either general, increasing the vigour of reward-related behaviours, or specific to responses that earn a common outcome. These different forms of PIT are mediated by distinct neural circuits involving the nucleus accumbens (NAC) core and shell, respectively. Here we examined the effects of pharmacological inactivation of either the NAC core or shell on PIT generated by alcohol-predictive and sucrose-predictive stimuli in rats. We found that presentations of a stimulus predicting sucrose enhanced responding for sucrose but not alcohol, suggesting an outcome-specific effect. In contrast, presentations of an alcohol-predictive stimulus enhanced responding for both alcohol and sucrose, suggesting a generally arousing effect. Inactivation of the NAC core reduced PIT and, in particular, the effect of the alcohol stimulus. Inactivation of the NAC shell reduced the specificity of the stimulus effects but left the ability of the stimuli to non-specifically invigorate responding intact, consistent with a role in mediating the specificity of PIT. Together, these results suggest that the NAC core plays a particularly important role in mediating the influence of alcohol-predictive cues on reward-seeking behaviours.

Habitual Alcohol Seeking: Neural Bases and Possible Relations to Alcohol Use Disorders.

Loss of flexible control over alcohol use may contribute to the development of alcohol use disorders. An increased contribution of response habits to alcohol-related behaviors may help explain this loss of control. Focusing on data from outcome devaluation and Pavlovian-instrumental transfer procedures, we review evidence for loss of goal-directed control over alcohol seeking and consumption drawing from both preclinical findings and clinical data where they exist. Over the course of extended alcohol self-administration and exposure, the performance of alcohol-seeking responses becomes less sensitive to reduction in the value of alcohol and more vulnerable to the influences of alcohol-predictive stimuli. These behavioral changes are accompanied by a shift in the corticostriatal circuits that control responding from circuits centered on the dorsomedial to those centered on the dorsolateral striatum. These changes in behavioral and neural control could help explain failures to abstain from alcohol despite intention to do so. Understanding and ultimately ameliorating these changes will aid development of more effective treatment interventions.

Safety encoding in the basal amygdala.

Learning to fear and avoid life-threatening stimuli are critical survival skills but are maladaptive when they persist in the absence of a direct threat. Thus, it is important to detect when a situation is safe and to increase behaviors leading to naturally rewarding actions, such as feeding and mating. It is unclear how the brain distinguishes between dangerous and safe situations. Here, we present a novel protocol designed to investigate the processing of cues that predict danger, safety, or reward (sucrose). In vivo single unit recordings were obtained in the basal amygdala of freely behaving rats undergoing simultaneous reward, fear, and safety conditioning. We observed a population of neurons that did not respond to a Fear Cue but did change their firing rate during the combined presentation of a fear cue simultaneous with a second, safety, cue; this combination of Fear + Safety Cues signified "no shock." This neural population consisted of two subpopulations: neurons that responded to the Fear + Safety Cue but not the Fear or Reward Cue ("safety" neurons), and neurons that responded to the Fear + Safety and Reward Cue but not the Fear Cue ("safety + reward" neurons). These data demonstrate the presence of neurons in the basal amygdala that are selectively responsive to Safety Cues. Furthermore, these data suggest that safety and reward learning use overlapping mechanisms in the basal amygdala.

The orbitofrontal cortex as part of a hierarchical neural system mediating choice between two good options.

Animals rely on environmental cues to identify potential rewards and select the best reward available. The orbitofrontal cortex (OFC) is proposed to encode sensory-specific representations of expected outcome. However, its contribution to the selection of a preferred outcome among different reward options is still unclear. We investigated the effect of transient OFC inactivation (achieved by presession injection of muscimol and baclofen) in a novel two-reward choice task. In discrete trials, rats could choose between a solution of polycose and an equally caloric, but highly preferred, solution of sucrose by visiting one of two liquid dispensers after the presentation of a specific cue signaling the availability of one or both of the solutions. We found that OFC inactivation did not affect outcome preference: rats maintained high preference for sucrose and adapted their behavioral responding when the cue-outcome contingencies were reversed. However, when rats were tested drug-free 24 h after OFC inactivation and reversal learning, memory for the newly learned contingencies was poor. These results suggest a potential conflict between OFC (encoding pre-reversal contingencies) and other brain circuits (encoding the new contingencies). Remarkably, repeating the OFC inactivation before the reversal memory test restored normal behavior, confirming the hypothesis of a dominant impact of OFC on other decision-making circuits. These results indicate that the representations encoded in the OFC, while not essential to the expression of outcome preference, exert hierarchical control on downstream decision-making circuits.

Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning.

What neural changes underlie individual differences in goal-directed learning? The lateral amygdala (LA) is important for assigning emotional and motivational significance to discrete environmental cues, including those that signal rewarding events. Recognizing that a cue predicts a reward enhances an animal's ability to acquire that reward; however, the cellular and synaptic mechanisms that underlie cue-reward learning are unclear. Here we show that marked changes in both cue-induced neuronal firing and input-specific synaptic strength occur with the successful acquisition of a cue-reward association within a single training session. We performed both in vivo and ex vivo electrophysiological recordings in the LA of rats trained to self-administer sucrose. We observed that reward-learning success increased in proportion to the number of amygdala neurons that responded phasically to a reward-predictive cue. Furthermore, cue-reward learning induced an AMPA (alpha-amino-3-hydroxy-5-methyl-isoxazole propionic acid)-receptor-mediated increase in the strength of thalamic, but not cortical, synapses in the LA that was apparent immediately after the first training session. The level of learning attained by individual subjects was highly correlated with the degree of synaptic strength enhancement. Importantly, intra-LA NMDA (N-methyl-d-aspartate)-receptor blockade impaired reward-learning performance and attenuated the associated increase in synaptic strength. These findings provide evidence of a connection between LA synaptic plasticity and cue-reward learning, potentially representing a key mechanism underlying goal-directed behaviour.

Extrasynaptic delta-containing GABAA receptors in the nucleus accumbens dorsomedial shell contribute to alcohol intake.

Recent findings suggest that extrasynaptic δ-subunit-containing GABA(A) receptors are sensitive to low-to-moderate concentrations of alcohol, raising the possibility that these receptors mediate the reinforcing effects of alcohol after consumption of one or a few drinks. We used the technique of viral-mediated RNAi to reduce expression of the GABA(A) receptor δ-subunit in adult rats in localized regions of the nucleus accumbens (NAc) to test the hypothesis that δ-subunit-containing GABA(A) receptors in the NAc are necessary for oral alcohol consumption. We found that knockdown of the δ-subunit in the medial shell region of the NAc, but not in the ventral or lateral shell or in the core, reduced alcohol intake. In contrast, δ-subunit knockdown in the medial shell did not affect intake of a 2% sucrose solution, suggesting that the effects of GABA(A) receptor δ-subunit reduction are specific to alcohol. These results provide strong evidence that extrasynaptic δ-subunit-containing GABA(A) receptors in the medial shell of the NAc are critical for the reinforcing effects of oral ethanol.

Chronic Ethanol Exposure Produces Persistent Impairment in Cognitive Flexibility and Decision Signals in the Striatum.

Value-based decision-making relies on the striatum, where neural plasticity can be altered by chronic ethanol (EtOH) exposure, but the effects of such plasticity on striatal neural dynamics during decision-making remain unclear. This study investigated the long-term impacts of EtOH on reward-driven decision-making and striatal neurocomputations in male and female rats using a dynamic probabilistic reversal learning task. Following a prolonged withdrawal period, EtOH-exposed male rats exhibited deficits in adaptability and exploratory behavior, with a preference for value updating based on rewards rather than omissions. These behavioral changes were linked to altered neural encoding in the dorsomedial striatum (DMS), where EtOH increased outcome-related signals and decreased choice-related signals. In contrast, female rats showed minimal behavioral changes with distinct EtOH-evoked alterations of neural signals, revealing significant sex differences in the impact of chronic EtOH. Our findings underscore the profound impact of chronic EtOH exposure on adaptive decision-making, revealing enduring changes in neurocomputational processes in the striatum underlying cognitive deficits that differ by sex.

Role of dopamine in reward expectation and predictability during execution of action sequences.

Reward-associated cues serve different functions depending on whether they precede or terminate action sequences. Cues that precede action sequences and signal opportunity for reward could serve as GO signals to initiate the sequence, whereas sequence termination cues could serve as response feedback by signaling reward delivery. Reward expectation during sequence execution depends on these cues and might condition whether behavior is habitual or goal-directed. However, it remains unknown how sequence initiation and termination cues differentially affect reward expectation and contribute to habit learning. Further, while mesolimbic dopamine plays a key role in cue-induced reward expectation and sequence learning, how dynamic changes in dopamine signals differ depending on the response strategy is unclear. Here, we determined how mesolimbic DA signals change over training during cue-mediated sequence learning, depending on the type of cue and the nature of behavioral control. We found sequence initiation and termination cues differentially affect reward expectation during action sequences, with the termination cue contributing to habit and automaticity. Distinct response strategies induced by sequence initiation and termination cues induced differential changes in mesolimbic DA signals that captured variations in reward expectation along sequence execution. Notably, habit-like behavior induced by the sequence termination cue was associated with a rapid shift in DA signals from reward retrieval to the cue. This habit-like behavior was reflected in behavioral inflexibility and attenuated DA reward prediction error signals. Finally, using optogenetics, we provide evidence that phasic DA activity elicited by the sequence termination cue is critical for the development of habit-like behavior.

Pavlovian cue-evoked alcohol seeking is disrupted by ventral pallidal inhibition.

Cues paired with alcohol can be potent drivers of craving, alcohol-seeking, consumption, and relapse. While the ventral pallidum is implicated in appetitive and consummatory responses across several reward classes and types of behaviors, its role in behavioral responses to Pavlovian alcohol cues has not previously been established. Here, we tested the impact of optogenetic inhibition of ventral pallidum on Pavlovian-conditioned alcohol-seeking in male Long Evans rats. Rats underwent Pavlovian conditioning with an auditory cue predicting alcohol delivery to a reward port and a control cue predicting no alcohol delivery, until they consistently entered the reward port more during the alcohol cue than the control cue. We then tested the within-session effects of optogenetic inhibition during 50% of cue presentations. We found that optogenetic inhibition of ventral pallidum during the alcohol cue reduced port entry likelihood and time spent in the port, and increased port entry latency. Overall, these results suggest that normal ventral pallidum activity is necessary for Pavlovian alcohol-seeking.

Basolateral amygdala population coding of a cued reward seeking state depends on orbitofrontal cortex.

Basolateral amygdala (BLA) neuronal responses to conditioned stimuli are closely linked to the expression of conditioned behavior. An area of increasing interest is how the dynamics of BLA neurons relate to evolving behavior. Here, we recorded the activity of individual BLA neurons across the acquisition and extinction of conditioned reward seeking and employed population-level analyses to assess ongoing neural dynamics. We found that, with training, sustained cue-evoked activity emerged that discriminated between the CS+ and CS- and correlated with conditioned responding. This sustained population activity continued until reward receipt and rapidly extinguished along with conditioned behavior during extinction. To assess the contribution of orbitofrontal cortex (OFC), a major reciprocal partner to BLA, to this component of BLA neural activity, we inactivated OFC while recording in BLA and found blunted sustained cue-evoked activity in BLA that accompanied reduced reward seeking. Optogenetic disruption of BLA activity and OFC terminals in BLA also reduced reward seeking. Our data suggest that sustained cue-driven activity in BLA, which in part depends on OFC input, underlies conditioned reward-seeking states.

Pramipexole restores behavioral inhibition in highly impulsive rats through a paradoxical modulation of frontostriatal networks.

Impulse control disorders (ICDs), a wide spectrum of maladaptive behaviors which includes pathological gambling, hypersexuality and compulsive buying, have been recently suggested to be triggered or aggravated by treatments with dopamine D2/3 receptor agonists, such as pramipexole (PPX). Despite evidence showing that impulsivity is associated with functional alterations in corticostriatal networks, the neural basis of the exacerbation of impulsivity by PPX has not been elucidated. Here we used a hotspot analysis to assess the functional recruitment of several corticostriatal structures by PPX in male rats identified as highly (HI), moderately impulsive (MI) or with low levels of impulsivity (LI) in the 5-choice serial reaction time task (5-CSRTT). PPX dramatically reduced impulsivity in HI rats. Assessment of the expression pattern of the two immediate early genes C-fos and Zif268 by in situ hybridization subsequently revealed that PPX resulted in a decrease in Zif268 mRNA levels in different striatal regions of both LI and HI rats accompanied by a high impulsivity specific reduction of Zif268 mRNA levels in prelimbic and cingulate cortices. PPX also decreased C-fos mRNA levels in all striatal regions of LI rats, but only in the dorsolateral striatum and nucleus accumbens core (NAc Core) of HI rats. Structural equation modeling further suggested that the anti-impulsive effect of PPX was mainly attributable to the specific downregulation of Zif268 mRNA in the NAc Core. Altogether, our results show that PPX restores impulse control in highly impulsive rats by modulation of limbic frontostriatal circuits.