Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release.

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Neural Activity in Ventral Medial Prefrontal Cortex Is Modulated More Before Approach Than Avoidance During Reinforced and Extinction Trial Blocks.

Ventromedial prefrontal cortex (vmPFC) is thought to provide regulatory control over Pavlovian fear responses and has recently been implicated in appetitive approach behavior, but much less is known about its role in contexts in which appetitive and aversive outcomes can be obtained and avoided, respectively. To address this issue, we recorded from single neurons in vmPFC while male rats performed our combined approach and avoidance task under reinforced and non-reinforced (extinction) conditions. Surprisingly, we found that cues predicting reward modulated cell firing in vmPFC more often and more robustly than cues preceding avoidable shock; in addition, firing of vmPFC neurons was both response (press or no-press) and outcome (reinforced or extinction) selective. These results suggest a complex role for vmPFC in regulating behavior and support its role in appetitive contexts during both reinforced and non-reinforced conditions.SIGNIFICANCE STATEMENT Selecting context-appropriate behaviors to gain reward or avoid punishment is critical for survival. Although the role of ventromedial prefrontal cortex (vmPFC) in mediating fear responses is well established, vmPFC has also been implicated in the regulation of reward-guided approach and extinction. Many studies have used indirect methods and simple behavioral procedures to study vmPFC, which leaves the literature incomplete. We recorded vmFPC neural activity during a complex cue-driven combined approach and avoidance task and during extinction. Surprisingly, we found very little vmPFC modulation to cues predicting avoidable shock, whereas cues predicting reward approach robustly modulated vmPFC firing in a response- and outcome-selective manner. This suggests a more complex role for vmPFC than current theories suggest, specifically regarding context-specific behavioral optimization.

Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance.

The mesolimbic dopamine system is believed to be a pathway that processes rewarding information. While previous studies have also implicated a general role for dopamine in punishment and its avoidance, the precise nature of subsecond dopamine release during these phenomena remains unknown. Here, we used fast-scan cyclic voltammetry to investigate whether subsecond dopamine release events in the nucleus accumbens encode cues predicting the avoidance of punishment during behavior maintained in a signaled footshock avoidance procedure. In this task, rats could initiate an avoidance response by pressing a lever within a warning period, preventing footshock. Alternatively, once footshocks commenced, animals could initiate an escape response by pressing the lever, terminating footshock. This design allowed us to assess subsecond dopamine release events during the presentation of a warning signal, safety periods, and two distinct behavioral responses. We found that release consistently increased upon presentation of the warning signal in a manner that reliably predicted successful punishment avoidance. We also observed subsecond dopamine release during the safety period, as occurs following the receipt of reward. Conversely, we observed a decrease in release at the warning signal during escape responses. Because of this finding, we next assessed dopamine release in a conditioned fear model. As seen during escape responses, we observed a time-locked decrease in dopamine release upon presentation of a cue conditioned to inescapable footshock. Together, these data show that subsecond fluctuations in mesolimbic dopamine release predict when rats will successfully avoid punishment and differentially encode cues related to aversive outcomes.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability.

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

Phasic dopamine release in the rat nucleus accumbens predicts approach and avoidance performance.

Dopamine (DA) is critical for reward processing, but significantly less is known about its role in punishment avoidance. Using a combined approach-avoidance task, we measured phasic DA release in the nucleus accumbens (NAc) of rats during presentation of cues that predicted reward, punishment or neutral outcomes and investigated individual differences based on avoidance performance. Here we show that DA release within a single microenvironment is higher for reward and avoidance cues compared with neutral cues and positively correlated with poor avoidance behaviour. We found that DA release delineates trial-type during sessions with good avoidance but is non-selective during poor avoidance, with high release correlating with poor performance. These data demonstrate that phasic DA is released during cued approach and avoidance within the same microenvironment and abnormal processing of value signals is correlated with poor performance.

Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures.

Decision-making is motivated by the possibility of obtaining reward and/or avoiding punishment. Though many have investigated behavior associated with appetitive or aversive outcomes, few have examined behaviors that rely on both. Fewer still have addressed questions related to how anticipated appetitive and aversive outcomes interact to alter neural signals related to expected value, motivation, and salience. Here we review recent rodent, monkey, and human research that address these issues. Further development of this area will be fundamental to understanding the etiology behind human psychiatric diseases and cultivating more effective treatments.

Endocannabinoids promote cocaine-induced impulsivity and its rapid dopaminergic correlates.

BACKGROUND: Impaired decision making, a hallmark of addiction, is hypothesized to arise from maladaptive plasticity in the mesolimbic dopamine pathway. The endocannabinoid system modulates dopamine activity through activation of cannabinoid type 1 receptors (CB1Rs). Here, we investigated whether impulsive behavior observed following cocaine exposure requires CB1R activation. METHODS: We trained rats in a delay-discounting task. Following acquisition of stable performance, rats were exposed to cocaine (10 mg/kg, intraperitoneal) every other day for 14 days and locomotor activity was measured. Two days later, delay-discounting performance was re-evaluated. To assess reversal of impulsivity, injections of a CB1R antagonist (1.5 mg/kg, intraperitoneal) or vehicle were given 30 minutes before the task. During the second experiment, aimed at preventing impulsivity rather than reversing it, CB1Rs were antagonized before each cocaine injection. In this experiment, subsecond dopamine release was measured in the nucleus accumbens during delay-discounting sessions before and after cocaine treatment. RESULTS: Blockade of CB1Rs reversed and prevented cocaine-induced impulsivity. Electrochemical results showed that during baseline and following disruption of endocannabinoid signaling, there was a robust increase in dopamine for immediate large rewards compared with immediate small rewards, but this effect reversed when the delay for the large reward was 10 seconds. In contrast, dopamine release always increased for one-pellet options at minimal or moderate delays in vehicle-treated rats. CONCLUSIONS: Endocannabinoids play a critical role in changes associated with cocaine exposure. Cannabinoid type 1 receptor blockade may thus counteract maladaptive alterations in afferents to dopamine neurons, thereby preventing changes in dopaminergic activity underlying a loss of self-control.

Separate populations of neurons in ventral striatum encode value and motivation.

Neurons in the ventral striatum (VS) fire to cues that predict differently valued rewards. It is unclear whether this activity represents the value associated with the expected reward or the level of motivation induced by reward anticipation. To distinguish between the two, we trained rats on a task in which we varied value independently from motivation by manipulating the size of the reward expected on correct trials and the threat of punishment expected upon errors. We found that separate populations of neurons in VS encode expected value and motivation.