Striatal Dopamine Signals and Reward Learning.

We are constantly bombarded by sensory information and constantly making decisions on how to act. In order to optimally adapt behavior, we must judge which sequences of sensory inputs and actions lead to successful outcomes in specific circumstances. Neuronal circuits of the basal ganglia have been strongly implicated in action selection, as well as the learning and execution of goal-directed behaviors, with accumulating evidence supporting the hypothesis that midbrain dopamine neurons might encode a reward signal useful for learning. Here, we review evidence suggesting that midbrain dopaminergic neurons signal reward prediction error, driving synaptic plasticity in the striatum underlying learning. We focus on phasic increases in action potential firing of midbrain dopamine neurons in response to unexpected rewards. These dopamine neurons prominently innervate the dorsal and ventral striatum. In the striatum, the released dopamine binds to dopamine receptors, where it regulates the plasticity of glutamatergic synapses. The increase of striatal dopamine accompanying an unexpected reward activates dopamine type 1 receptors (D1Rs) initiating a signaling cascade that promotes long-term potentiation of recently active glutamatergic input onto striatonigral neurons. Sensorimotor-evoked glutamatergic input, which is active immediately before reward delivery will thus be strengthened onto neurons in the striatum expressing D1Rs. In turn, these neurons cause disinhibition of brainstem motor centers and disinhibition of the motor thalamus, thus promoting motor output to reinforce rewarded stimulus-action outcomes. Although many details of the hypothesis need further investigation, altogether, it seems likely that dopamine signals in the striatum might underlie important aspects of goal-directed reward-based learning.

A Unified Framework for Dopamine Signals across Timescales.

Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.

Differential Effects of Deep Brain Stimulation of the Internal Capsule and the Striatum on Excessive Grooming in Sapap3 Mutant Mice.

BACKGROUND: Deep brain stimulation (DBS) is an effective treatment for patients with obsessive-compulsive disorder (OCD) that do not respond to conventional therapies. Although the precise mechanism of action of DBS remains unknown, modulation of activity in corticofugal fibers originating in the prefrontal cortex is thought to underlie its beneficial effects in OCD. METHODS: To gain more mechanistic insight into DBS in OCD, we used Sapap3 mutant mice. These mice display excessive self-grooming and increased anxiety, both of which are responsive to therapeutic drugs used in OCD patients. We selected two clinically relevant DBS targets through which activity in prefronto-corticofugal fibers may be modulated: the internal capsule (IC) and the dorsal part of the ventral striatum (dVS). RESULTS: IC-DBS robustly decreased excessive grooming, whereas dVS-DBS was on average less effective. Grooming was reduced rapidly after IC-DBS onset and reinstated upon DBS offset. Only IC-DBS was associated with increased locomotion. DBS in both targets induced c-Fos expression around the electrode tip and in different regions of the prefrontal cortex. This prefronto-cortical activation was more extensive after IC-DBS, but not associated with behavioral effects. Furthermore, we found that the decline in grooming cannot be attributed to altered locomotor activity and that anxiety, measured on the elevated plus maze, was not affected by DBS. CONCLUSIONS: DBS in both the IC and dVS reduces compulsive grooming in Sapap3 mutant mice. However, IC stimulation was more effective, but also produced motor activation, even though both DBS targets modulated activity in a similar set of prefrontal cortical fibers.