Generalized cue reactivity in dopamine neurons after opioids.

Cue reactivity is the maladaptive neurobiological and behavioral response upon exposure to drug cues and is a major driver of relapse. The leading hypothesis is that dopamine release by addictive drugs represents a persistently positive reward prediction error that causes runaway enhancement of dopamine responses to drug cues, leading to their pathological overvaluation compared to non-drug reward alternatives. However, this hypothesis has not been directly tested. Here we developed Pavlovian and operant procedures to measure firing responses, within the same dopamine neurons, to drug versus natural reward cues, which we found to be similarly enhanced compared to cues predicting natural rewards in drug-naïve controls. This enhancement was associated with increased behavioral reactivity to the drug cue, suggesting that dopamine release is still critical to cue reactivity, albeit not as previously hypothesized. These results challenge the prevailing hypothesis of cue reactivity, warranting new models of dopaminergic function in drug addiction, and provide critical insights into the neurobiology of cue reactivity with potential implications for relapse prevention.

Dopamine and acetylcholine correlations in the nucleus accumbens depend on behavioral task states.

Dopamine in the nucleus accumbens ramps up as animals approach desired goals. These ramps have received intense scrutiny because they seem to violate long-held hypotheses on dopamine function. Furthermore, it has been proposed that they are driven by local acetylcholine release, i.e., that they are mechanistically separate from dopamine signals related to reward prediction errors. Here, we tested this hypothesis by simultaneously recording accumbal dopamine and acetylcholine signals in rats executing a task involving motivated approach. Contrary to recent reports, we found that dopamine ramps were not coincidental with changes in acetylcholine. Instead, we found that acetylcholine could be positively, negatively, or uncorrelated with dopamine depending on whether the task phase was determined by a salient cue, reward prediction error, or active approach, respectively. Our results suggest that accumbal dopamine and acetylcholine are largely independent but may combine to engage different postsynaptic mechanisms depending on the behavioral task states.

The selective D3Receptor antagonist VK4-116 reverses loss of insight caused by self-administration of cocaine in rats.

Chronic psychostimulant use causes long-lasting changes to neural and cognitive function that persist after long periods of abstinence. As cocaine users transition from drug use to abstinence, a parallel transition from hyperactivity to hypoactivity has been found in orbitofrontal-striatal glucose metabolism and striatal D2/D3-receptor activity. Targeting these changes pharmacologically, using highly selective dopamine D3-receptor (D3R) antagonists and partial agonists, has shown promise in reducing drug-taking, and attenuating relapse in animal models of cocaine and opioid use disorder. However, much less attention has been paid to treating the loss of insight, operationalized as the inability to infer likely outcomes, associated with chronic psychostimulant use. Here we tested the selective D3R antagonist VK4-116 as a treatment for this loss in rats with a prior history of cocaine use. Male and female rats were first trained to self-administer cocaine or a sucrose liquid for 2 weeks. After 4 weeks of abstinence, performance was assessed using a sensory preconditioning (SPC) learning paradigm. Rats were given VK4-116 (15 mg/kg, i.p.) or vehicle 30 min prior to each SPC training session, thus creating four drug-treatment groups: sucrose-vehicle, sucrose-VK4-116, cocaine-vehicle, cocaine-VK4-116. The control groups (sucrose-vehicle, sucrose-VK4-116) showed normal sensory preconditioning, whereas cocaine use (cocaine-vehicle) selectively disrupted responding to the preconditioned cue, an effect that was reversed in the cocaine-VK4-116 group, which demonstrating responding to the preconditioned cue at levels comparable to controls. These preclinical findings demonstrate that highly selective dopamine D3R antagonists, particularly VK4-116, can reverse the long-term negative behavioral consequences of cocaine use.

Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks.

Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.

Different Effects of Peer Sex on Operant Responding for Social Interaction and Striatal Dopamine Activity.

When rats are given discrete choices between social interactions with a peer and opioid or psychostimulant drugs, they choose social interaction, even after extensive drug self-administration experience. Studies show that like drug and nondrug food reinforcers, social interaction is an operant reinforcer and induces dopamine release. However, these studies were conducted with same-sex peers. We examined if peer sex influences operant social interaction and the role of estrous cycle and striatal dopamine in same- versus opposite-sex social interaction. We trained male and female rats (n = 13 responders/12 peers) to lever-press (fixed-ratio 1 [FR1] schedule) for 15 s access to a same- or opposite-sex peer for 16 d (8 d/sex) while tracking females' estrous cycle. Next, we transfected GRAB-DA2m and implanted optic fibers into nucleus accumbens (NAc) core and dorsomedial striatum (DMS). We then retrained the rats for 15 s social interaction (FR1 schedule) for 16 d (8 d/sex) and recorded striatal dopamine during operant responding for a peer for 8 d (4 d/sex). Finally, we assessed economic demand by manipulating FR requirements for a peer (10 d/sex). In male, but not female rats, operant responding was higher for the opposite-sex peer. Female's estrous cycle fluctuations had no effect on operant social interaction. Striatal dopamine signals for operant social interaction were dependent on the peer's sex and striatal region (NAc core vs DMS). Results indicate that estrous cycle fluctuations did not influence operant social interaction and that NAc core and DMS dopamine activity reflect sex-dependent features of volitional social interaction.

OFC neurons do not represent the negative value of a conditioned inhibitor.

The orbitofrontal cortex (OFC) is often proposed to function as a value integrator; however, alternative accounts focus on its role in representing associative structures that specify the probability and sensory identity of future outcomes. These two accounts make different predictions about how this area should respond to conditioned inhibitors of reward, since in the former, neural activity should reflect the negative value of the inhibitor, whereas in the latter, it should track the estimated probability of a future reward based on all cues present. Here, we assessed these predictions by recording from small groups of neurons in the lateral OFC of rats during training in a conditioned inhibition design. Rats showed negative summation when the inhibitor was compounded with a novel excitor, suggesting that they learned to respond to the conditioned inhibitor appropriately. Against this backdrop, we found unit and population responses that scaled with expected reward value on excitor + inhibitor compound trials. However, the responses of these neurons did not differentiate between the conditioned inhibitor and a neutral cue when both were presented in isolation. Further, when the ensemble patterns were analyzed, activity to the conditioned inhibitor did not classify according to putative negative value. Instead, it classified with a same-modality neutral cue when presented alone and as a unique item when presented in compound with a novel excitor. This pattern of results supports the notion that OFC encodes a model of the causal structure of the environment rather than either the modality or the value of cues.

Expectancy-related changes in firing of dopamine neurons depend on hippocampus.

The orbitofrontal cortex (OFC) and hippocampus (HC) are both implicated in forming the cognitive or task maps that support flexible behavior. Previously, we used the dopamine neurons as a sensor or tool to measure the functional effects of OFC lesions (Takahashi et al., 2011). We recorded midbrain dopamine neurons as rats performed an odor-based choice task, in which errors in the prediction of reward were induced by manipulating the number or timing of the expected rewards across blocks of trials. We found that OFC lesions ipsilateral to the recording electrodes caused prediction errors to be degraded consistent with a loss in the resolution of the task states, particularly under conditions where hidden information was critical to sharpening the predictions. Here we have repeated this experiment, along with computational modeling of the results, in rats with ipsilateral HC lesions. The results show HC also shapes the map of our task, however unlike OFC, which provides information local to the trial, the HC appears to be necessary for estimating the upper-level hidden states based on the information that is discontinuous or separated by longer timescales. The results contrast the respective roles of the OFC and HC in cognitive mapping and add to evidence that the dopamine neurons access a rich information set from distributed regions regarding the predictive structure of the environment, potentially enabling this powerful teaching signal to support complex learning and behavior.

Lateral orbitofrontal cortex integrates predictive information across multiple cues to guide behavior.

Individuals are often faced with multiple cues that concurrently predict the same outcome, and combining these predictions may benefit behavior. Previous work has studied the neural basis of decision-making, predominantly using isolated sensory stimuli, and so the mechanisms that allow us to leverage multiple cues remain unclear. In two experiments, we used neuroimaging and network-targeted brain stimulation to probe how the brain integrates outcome predictions to guide adaptive behavior. We identified neural signatures of outcome integration in the lateral orbitofrontal cortex (OFC), where concurrently presented cues evoke stronger pattern-based representations of expected outcomes. Moreover, perturbing lateral OFC network activity impairs subjects' ability to leverage predictions from multiple cues to facilitate responding. Intriguingly, we found similar behavioral and brain mechanisms for reward-predicting cues and for cues predicting the absence of reward. These findings highlight a causal role for the lateral OFC in utilizing outcome predictions from multiple cues to guide behavior.

The selective D3-Receptor antagonist VK4-116 effectively treats behavioral inflexibility in rats caused by self-administration and withdrawal from cocaine.

Chronic psychostimulant use can cause long lasting changes to neural and cognitive function that persist even after long periods of abstinence. As cocaine users transition from drug use to abstinence, a parallel transition from hyperactivity to hypoactivity has been found in orbitofrontal-striatal glucose metabolism, and striatal D2/D3 receptor activity. Targeting these changes pharmacologically, using highly selective dopamine D3 receptor (D3R) antagonists and partial agonists, has shown significant promise in reducing drug-taking, and attenuating relapse in animal models of cocaine and opioid use disorder. However, much less attention has been focused on treating inflexible and potentially maladaptive non-drug behaviors following chronic psychostimulant use. Here we tested the selective D3R antagonist VK4-116 as a treatment for the long-term behavioral inflexibility in abstinent male and female rats with a prior history of chronic cocaine use. Rats were first trained to self-administer cocaine (0.75 mg/kg/reinforcer) or a sucrose liquid (10%, .04 mL/reinforcer) for 2 weeks (FR1 schedule, max 60 reinforcers in 3 hrs/ day), followed by 4 weeks of abstinence. Cognitive and behavioral flexibilities were then assessed using a sensory preconditioning (SPC) learning paradigm. Rats were given an VK4-116 (15 mg/kg, i.p.) or vehicle 30 mins prior to each SPC training session, thus creating four drug-treatment groups: sucrose-vehicle, sucrose-VK4-116, cocaine-vehicle, cocaine-VK4-116. The control groups (sucrose-vehicle, sucrose-VK4-116) demonstrated significant evidence of flexible SPC behavior, whereas cocaine use (cocaine-vehicle) disrupted SPC behavior. Remarkably, the D3R antagonist VK4-116 mitigated this cocaine deficit in the cocaine-VK4-116 group, demonstrating flexible SPC to levels comparable to the control groups. These preclinical findings demonstrate that highly selective dopamine D3R antagonists, particularly VK4-116, show significant promise as a pharmacological treatment for the long-term negative behavioral consequences of cocaine use disorder.

Dopaminergic prediction errors in the ventral tegmental area reflect a multithreaded predictive model.

Dopamine neuron activity is tied to the prediction error in temporal difference reinforcement learning models. These models make significant simplifying assumptions, particularly with regard to the structure of the predictions fed into the dopamine neurons, which consist of a single chain of timepoint states. Although this predictive structure can explain error signals observed in many studies, it cannot cope with settings where subjects might infer multiple independent events and outcomes. In the present study, we recorded dopamine neurons in the ventral tegmental area in such a setting to test the validity of the single-stream assumption. Rats were trained in an odor-based choice task, in which the timing and identity of one of several rewards delivered in each trial changed across trial blocks. This design revealed an error signaling pattern that requires the dopamine neurons to access and update multiple independent predictive streams reflecting the subject's belief about timing and potentially unique identities of expected rewards.

The role of the lateral orbitofrontal cortex in creating cognitive maps.

We use mental models of the world-cognitive maps-to guide behavior. The lateral orbitofrontal cortex (lOFC) is typically thought to support behavior by deploying these maps to simulate outcomes, but recent evidence suggests that it may instead support behavior by underlying map creation. We tested between these two alternatives using outcome-specific devaluation and a high-potency chemogenetic approach. Selectively inactivating lOFC principal neurons when male rats learned distinct cue-outcome associations, but before outcome devaluation, disrupted subsequent inference, confirming a role for the lOFC in creating new maps. However, lOFC inactivation surprisingly led to generalized devaluation, a result that is inconsistent with a complete mapping failure. Using a reinforcement learning framework, we show that this effect is best explained by a circumscribed deficit in credit assignment precision during map construction, suggesting that the lOFC has a selective role in defining the specificity of associations that comprise cognitive maps.

Calcium activity is a degraded estimate of spikes.

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.

Dopamine.

Dopamine was first described by George Barger, James Ewens, and Henry Dale in 1910 as an epinephrine-like monoamine compound. Initially believed to be a mere precursor of norepinephrine, it was mostly ignored for the next four decades (Figure 1A). However, in the 1950s Kathleen Montagu showed that dopamine occurred in the brain by itself, and a series of studies by Arvid Carlsson and collaborators demonstrated that dopamine is a bona fide neurotransmitter, a finding that would earn Carlsson the 2000 Nobel Prize in Physiology and Medicine. In a landmark experiment, he pharmacologically blocked all dopamine neurotransmission in rabbits, which rendered them completely paralyzed, and then fully recovered their behavior with an injection of the dopamine precursor L-DOPA, demonstrating that dopamine was essential for self-initiated movement (Figure 1B). A similar effect was quickly reproduced by Oleg Hornykiewicz and collaborators in human Parkinsonian patients. Within a few years, dopamine jumped from relative obscurity to being critical for life as we know it.

Dopamine signaling in the nucleus accumbens core mediates latent inhibition.

Studies investigating the neural mechanisms by which associations between cues and predicted outcomes control behavior often use associative learning frameworks to understand the neural control of behavior. These frameworks do not always account for the full range of effects that novelty can have on behavior and future associative learning. Here, in mice, we show that dopamine in the nucleus accumbens core is evoked by novel, neutral stimuli, and the trajectory of this response over time tracked habituation to these stimuli. Habituation to novel cues before associative learning reduced future associative learning, a phenomenon known as latent inhibition. Crucially, trial-by-trial dopamine response patterns tracked this phenomenon. Optogenetic manipulation of dopamine responses to the cue during the habituation period bidirectionally influenced future associative learning. Thus, dopamine signaling in the nucleus accumbens core has a causal role in novelty-based learning in a way that cannot be predicted based on purely associative factors.

Minimal cross-trial generalization in learning the representation of an odor-guided choice task.

There is no single way to represent a task. Indeed, despite experiencing the same task events and contingencies, different subjects may form distinct task representations. As experimenters, we often assume that subjects represent the task as we envision it. However, such a representation cannot be taken for granted, especially in animal experiments where we cannot deliver explicit instruction regarding the structure of the task. Here, we tested how rats represent an odor-guided choice task in which two odor cues indicated which of two responses would lead to reward, whereas a third odor indicated free choice among the two responses. A parsimonious task representation would allow animals to learn from the forced trials what is the better option to choose in the free-choice trials. However, animals may not necessarily generalize across odors in this way. We fit reinforcement-learning models that use different task representations to trial-by-trial choice behavior of individual rats performing this task, and quantified the degree to which each animal used the more parsimonious representation, generalizing across trial types. Model comparison revealed that most rats did not acquire this representation despite extensive experience. Our results demonstrate the importance of formally testing possible task representations that can afford the observed behavior, rather than assuming that animals' task representations abide by the generative task structure that governs the experimental design.

Anterior cingulate neurons signal neutral cue pairings during sensory preconditioning.

Of all frontocortical subregions, the anterior cingulate cortex (ACC) has perhaps the most overlapping theories of function.1-3 Recording studies in rats, humans, and other primates have reported diverse neural responses that support many theories,4-12 yet nearly all these studies have in common tasks in which one event reliably predicts another. This leaves open the possibility that ACC represents associative pairing of events, independent of their overt biological significance. Sensory preconditioning13 provides an opportunity to test this. In the first phase, preconditioning, value-neutral sensory stimuli are paired (A→B). To test whether this was learned, subjects are given standard conditioning during which one of the previously neutral sensory cues is paired with a biologically meaningful outcome (B→outcome). During the final probe test, the neutral cue which was never paired with a biologically meaningful outcome is presented alone (A→) and will elicit a conditional response, suggesting that subjects had learned the associative structure during preconditioning and use that knowledge to infer presentation of the biologically relevant outcome (A→B→outcome). Inference-based responding demonstrates a fundamental property of model-based reasoning14,15 and requires learning of the associations between neutral stimuli before rewards are introduced.16-19 ACC neurons developed firing patterns that reflected the learning of sensory associations during preconditioning, even though no rewards were present. The strength of these correlates predicted rats' ability to later mobilize and use that associative information during the probe test. These results demonstrate that clear biological significance is not necessary to produce correlates of learning in ACC.

Spatial Representations in Rat Orbitofrontal Cortex.

The orbitofrontal cortex (OFC) and hippocampus share striking cognitive and functional similarities. As a result, both structures have been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. However, while this function has been exemplified by spatial coding in neurons of hippocampal regions-particularly place and grid cells-spatial representations in the OFC have been investigated far less. Here we sought to address this by recording OFC neurons from male rats engaged in an open-field foraging task like that originally developed to characterize place fields in rodent hippocampal neurons. Single-unit activity was recorded as rats searched for food pellets scattered randomly throughout a large enclosure. In some sessions, particular flavors of food occurred more frequently in particular parts of the enclosure; in others, only a single flavor was used. OFC neurons showed spatially localized firing fields in both conditions, and representations changed between flavored and unflavored foraging periods in a manner reminiscent of remapping in the hippocampus. Compared with hippocampal recordings taken under similar behavioral conditions, OFC spatial representations were less temporally reliable, and there was no significant evidence of grid tuning in OFC neurons. These data confirm that OFC neurons show spatial firing fields in a large, two-dimensional environment in a manner similar to hippocampus. Consistent with the focus of the OFC on biological meaning and goals, spatial coding was weaker than in hippocampus and influenced by outcome identity.SIGNIFICANCE STATEMENT The orbitofrontal cortex (OFC) and hippocampus have both been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. This function is exemplified by place and grid cells identified in hippocampus, the activity of which maps spatial environments. The current study directly demonstrates very similar, though not identical, spatial representatives in OFC neurons, confirming that OFC-like hippocampus-can represent a spatial map under the appropriate experimental conditions.

Orbitofrontal cortex and learning predictions of state transitions.

The orbitofrontal cortex (OFC) has been implicated in goal-directed planning and model-based decision-making. One key prerequisite for model-based decision-making is learning the transition structure of the environment-the probabilities of transitioning from one environmental state to another. In this work, we investigated how the OFC might be involved in learning this transition structure, by using fMRI to assess OFC activity while humans experienced probabilistic cue-outcome transitions. We found that OFC activity was indeed correlated with behavioral measures of learning about transition structure. On a trial-by-trial basis, OFC activity was associated with subsequently increased expectation of the more probable outcome; that is, with subsequently more optimal cue-outcome predictions. Interestingly, this relationship was observed no matter what outcome occurred at the time of the OFC activity, and thus is inconsistent with an interpretation of the OFC activity as representing a "state prediction error" that would facilitate learning transitions via error-correcting mechanisms. Finally, OFC activity was related to more optimal predictions only for subsequent trials involving the same cue that was observed at the time of OFC activity-this relationship was not observed for subsequent trials involving a different cue. All together, these results indicate that the OFC is involved in updating or reinforcing a learned transition model on a trial-by-trial basis, specifically for the currently observed cue-outcome associations. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Neuroscience: What, where, and how wonderful?

Hippocampal place cells represent spatial locations, but it is unclear how they incorporate associations between locations and specific outcomes. A recent study illuminates this issue by showing that place cells in intermediate hippocampus remap their fields following changes in reward.