Value-Driven Adaptations of Mesolimbic Dopamine Release Are Governed by Both Model-Based and Model-Free Mechanisms.

The magnitude of dopamine signals elicited by rewarding events and their predictors is updated when reward value changes. It is actively debated how readily these dopamine signals adapt and whether adaptation aligns with model-free or model-based reinforcement-learning principles. To investigate this, we trained male rats in a pavlovian-conditioning paradigm and measured dopamine release in the nucleus accumbens core in response to food reward (unconditioned stimulus) and reward-predictive conditioned stimuli (CS), both before and after reward devaluation, induced via either sensory-specific or nonspecific satiety. We demonstrate that (1) such devaluation reduces CS-induced dopamine release rapidly, without additional pairing of CS with devalued reward and irrespective of whether the devaluation was sensory-specific or nonspecific. In contrast, (2) reward devaluation did not decrease food reward-induced dopamine release. Surprisingly, (3) postdevaluation reconditioning, by additional pairing of CS with devalued reward, rapidly reinstated CS-induced dopamine signals to predevaluation levels. Taken together, we identify distinct, divergent adaptations in dopamine-signal magnitude when reward value is decreased: CS dopamine diminishes but reinstates fast, whereas reward dopamine is resistant to change. This implies that, respective to abovementioned findings, (1) CS dopamine may be governed by a model-based mechanism and (2) reward dopamine by a model-free one, where (3) the latter may contribute to swift reinstatement of the former. However, changes in CS dopamine were not selective for sensory specificity of reward devaluation, which is inconsistent with model-based processes. Thus, mesolimbic dopamine signaling incorporates both model-free and model-based mechanisms and is not exclusively governed by either.

Anticipation of Appetitive Operant Action Induces Sustained Dopamine Release in the Nucleus Accumbens.

The mesolimbic dopamine system is implicated in signaling reward-related information as well as in actions that generate rewarding outcomes. These implications are commonly investigated in either pavlovian or operant reinforcement paradigms, where only the latter requires instrumental action. To parse contributions of reward- and action-related information to dopamine signals, we directly compared the two paradigms: male rats underwent either pavlovian or operant conditioning while dopamine release was measured in the nucleus accumbens, a brain region central for processing this information. Task conditions were identical with the exception of the operant-lever response requirement. Rats in both groups released the same quantity of dopamine at the onset of the reward-predictive cue. However, only the operant-conditioning group showed a subsequent, sustained plateau in dopamine concentration throughout the entire 5 s cue presentation (preceding the required action). This dopamine ramp was unaffected by probabilistic reward delivery, occurred exclusively before operant actions, and was not related to task performance or task acquisition as it persisted throughout the 2 week daily behavioral training. Instead, the ramp flexibly increased in duration with longer cue presentation, seemingly modulating the initial cue-onset-triggered dopamine release, that is, the reward prediction error (RPE) signal, as both signal amplitude and sustainment diminished when reward timing was made more predictable. Thus, our findings suggest that RPE and action components of dopamine release can be differentiated temporally into phasic and ramping/sustained signals, respectively, where the latter depends on the former and presumably reflects the anticipation or incentivization of appetitive action, conceptually akin to motivation.SIGNIFICANCE STATEMENT It is unclear whether the components of dopamine signals that are related to reward-associated information and reward-driven approach behavior can be separated. Most studies investigating the dopamine system use either pavlovian or operant conditioning, which both involve the delivery of reward and necessitate appetitive approach behavior. Thus, used exclusively, neither paradigm can disentangle the contributions of these components to dopamine release. However, by combining both paradigms in the same study, we find that anticipation of a reward-driven operant action induces a modulation of reward-prediction-associated dopamine release, producing so-called dopamine ramps. Therefore, our findings provide new insight into dopamine ramps and suggest that dopamine signals integrate reward and appetitive action in a temporally distinguishable, yet dependent, manner.

Nucleus accumbens dopamine tracks aversive stimulus duration and prediction but not value or prediction error.

There is active debate on the role of dopamine in processing aversive stimuli, where inferred roles range from no involvement at all, to signaling an aversive prediction error (APE). Here, we systematically investigate dopamine release in the nucleus accumbens core (NAC), which is closely linked to reward prediction errors, in rats exposed to white noise (WN, a versatile, underutilized, aversive stimulus) and its predictive cues. Both induced a negative dopamine ramp, followed by slow signal recovery upon stimulus cessation. In contrast to reward conditioning, this dopamine signal was unaffected by WN value, context valence, or probabilistic contingencies, and the WN dopamine response shifted only partially toward its predictive cue. However, unpredicted WN provoked slower post-stimulus signal recovery than predicted WN. Despite differing signal qualities, dopamine responses to simultaneous presentation of rewarding and aversive stimuli were additive. Together, our findings demonstrate that instead of an APE, NAC dopamine primarily tracks prediction and duration of aversive events.

A unidirectional but not uniform striatal landscape of dopamine signaling for motivational stimuli.

Dopamine signals in the striatum are critical for motivated behavior. However, their regional specificity and precise information content are actively debated. Dopaminergic projections to the striatum are topographically organized. Thus, we quantified dopamine release in response to motivational stimuli and associated predictive cues in six principal striatal regions of unrestrained, behaving rats. Absolute signal size and its modulation by stimulus value and by subjective state of the animal were interregionally heterogeneous on a medial to lateral gradient. In contrast, dopamine-concentration direction of change was homogeneous across all regions: appetitive stimuli increased and aversive stimuli decreased dopamine concentration. Although cues predictive of such motivational stimuli acquired the same influence over dopamine homogeneously across all regions, dopamine-mediated prediction-error signals were restricted to the ventromedial, limbic striatum. Together, our findings demonstrate a nuanced striatal landscape of unidirectional but not uniform dopamine signals, topographically encoding distinct aspects of motivational stimuli and their prediction.

A Novel Automated Approach for Improving Standardization of the Marble Burying Test Enables Quantification of Burying Bouts and Activity Characteristics.

The marble burying test is a commonly used paradigm to describe phenotypes in mouse models of neurodevelopmental and psychiatric disorders. The current methodological approach relies predominantly on reporting the number of buried marbles at the end of the test. By measuring the proxy of the behavior (buried marbles), many important characteristics regarding the temporal aspect of this assay are lost. Here, we introduce a novel, automated method to quantify mouse behavior during the marble burying test with the focus on the burying bouts and movement dynamics. Using open-source software packages, we trained a supervised machine learning algorithm (the "classifier") to distinguish burying behavior in freely moving mice. In order to confirm the classifier's accuracy and characterize burying events in high detail, we performed the marble burying test in three mouse models: Ube3am-/p+ [Angelman syndrome (AS) model], Shank2-/- (autism model), and Sapap3-/- [obsessive-compulsive disorder (OCD) model] mice. The classifier scored burying behavior accurately and consistent with the previously reported phenotype of the Ube3am-/p+ mice, which showed decreased levels of burying compared with controls. Shank2-/- mice showed a similar pattern of decreased burying behavior, which was not found in Sapap3-/- mice. Tracking mouse behavior throughout the test revealed hypoactivity in Ube3am-/p+ and hyperactivity in the Shank2-/- mice, indicating that mouse activity is unrelated to burying behavior. Reducing activity with midazolam in Shank2-/- mice did not alter the burying behavior. Together, we demonstrate that our classifier is an accurate method for the analysis of the marble burying test, providing more information than currently used methods.

Striatal dopamine signals are region specific and temporally stable across action-sequence habit formation.

Habits are automatic, inflexible behaviors that develop slowly with repeated performance. Striatal dopamine signaling instantiates this habit-formation process, presumably region specifically and via ventral-to-dorsal and medial-to-lateral signal shifts. Here, we quantify dopamine release in regions implicated in these presumed shifts (ventromedial striatum [VMS], dorsomedial striatum [DMS], and dorsolateral striatum [DLS]) in rats performing an action-sequence task and characterize habit development throughout a 10-week training. Surprisingly, all regions exhibited stable dopamine dynamics throughout habit development. VMS and DLS signals did not differ between habitual and non-habitual animals, but DMS dopamine release increased during action-sequence initiation and decreased during action-sequence completion in habitual rats, whereas non-habitual rats showed opposite effects. Consistently, optogenetic stimulation of DMS dopamine release accelerated habit formation. Thus, we demonstrate that dopamine signals do not shift regionally during habit formation and that dopamine in DMS, but not VMS or DLS, determines habit bias, attributing "habit functions" to a region previously associated exclusively with non-habitual behavior.

Invasive and Non-invasive Neurostimulation for OCD.

It becomes increasingly clear that (non-)invasive neurostimulation is an effective treatment for obsessive-compulsive disorder (OCD). In this chapter we review the available evidence on techniques and targets, clinical results including a meta-analysis, mechanisms of action, and animal research. We focus on deep brain stimulation (DBS), but also cover non-invasive neurostimulation including transcranial magnetic stimulation (TMS). Data shows that most DBS studies target the ventral capsule/ventral striatum (VC/VS), with an overall 76% response rate in treatment-refractory OCD. Also TMS holds clinical promise. Increased insight in the normalizing effects of neurostimulation on cortico-striatal-thalamic-cortical (CSTC) loops - through neuroimaging and animal research - provides novel opportunities to further optimize treatment strategies. Advancing clinical implementation of neurostimulation techniques is essential to ameliorate the lives of the many treatment-refractory OCD patients.

Impaired hippocampal representation of place in the Fmr1-knockout mouse model of fragile X syndrome.

Fragile X syndrome (FXS) is an X-chromosome linked intellectual disability and the most common known inherited single gene cause of autism spectrum disorder (ASD). Building upon demonstrated deficits in neuronal plasticity and spatial memory in FXS, we investigated how spatial information processing is affected in vivo in an FXS mouse model (Fmr1-KO). Healthy hippocampal neurons (so-called place cells) exhibit place-related activity during spatial exploration, and their firing fields tend to remain stable over time. In contrast, we find impaired stability and reduced specificity of Fmr1-KO spatial representations. This is a potential biomarker for the cognitive dysfunction observed in FXS, informative on the ability to integrate sensory information into an abstract representation and successfully retain this conceptual memory. Our results provide key insight into the biological mechanisms underlying cognitive disabilities in FXS and ASD, paving the way for a targeted approach to remedy these.

Abnormal hippocampal theta and gamma hypersynchrony produces network and spike timing disturbances in the Fmr1-KO mouse model of Fragile X syndrome.

Neuronal networks can synchronize their activity through excitatory and inhibitory connections, which is conducive to synaptic plasticity. This synchronization is reflected in rhythmic fluctuations of the extracellular field. In the hippocampus, theta and gamma band LFP oscillations are a hallmark of the processing of spatial information and memory. Fragile X syndrome (FXS) is an intellectual disability and the most common genetic cause of autism spectrum disorder (Belmonte and Bourgeron, 2006). Here, we investigated how neuronal network synchronization in the mouse hippocampus is compromised by the Fmr1 mutation that causes FXS (Santos et al., 2014), relating recently observed single-cell level impairments (Arbab et al., 2017) to neuronal network aberrations. We implanted tetrodes in hippocampus of freely moving Fmr1-KO and littermate wildtype (WT) mice (Mientjes et al., 2006), to record spike trains from multiple, isolated neurons as well as LFPs in a spatial exploration paradigm. Compared to wild type mice, Fmr1-KO mice displayed greater power of hippocampal theta oscillations, and higher coherence in the slow gamma band. Additionally, spike trains of Fmr1-KO interneurons show decreased spike-count correlations and they are hypersynchronized with theta and slow gamma oscillations. The hypersynchronization of Fmr1-KO oscillations and spike timing reflects functional deficits in local networks. This network hypersynchronization pathologically decreases the heterogeneity of spike-LFP phase coupling, compromising information processing within the hippocampal circuit. These findings may reflect a pathophysiological mechanism explaining cognitive impairments in FXS and autism, in which there is anomalous processing of social and environmental cues and associated deficits in memory and cognition.