Phasic excitation of ventral tegmental dopamine neurons potentiates the initiation of conditioned approach behavior: parametric and reinforcement-schedule analyses.

Midbrain dopamine neurons are implicated in motivation and learning. However, it is unclear how phasic excitation of dopamine neurons, which is implicated in learning, is involved in motivation. Here we used a self-stimulation procedure to examine how mice seek for optogenetically-induced phasic excitation of dopamine neurons, with an emphasis on the temporal dimension. TH-Cre transgenic mice received adeno-associated viral vectors encoding channelrhodopsin-2 into the ventral tegmental area, resulting in selective expression of the opsin in dopamine neurons. These mice were trained to press on a lever for photo-pulse trains that phasically excited dopamine neurons. They learned to self-stimulate in a fast, constant manner, and rapidly reduced pressing during extinction. We first determined effective parameters of photo-pulse trains in self-stimulation. Lever-press rates changed as a function of the manipulation of pulse number, duration, intensity, and frequency. We then examined effects of interval and ratio schedules of reinforcement on photo-pulse train reinforcement, which was contrasted with food reinforcement. Reinforcement with food inhibited lever pressing for a few seconds, after which pressing was robustly regulated in a goal-directed manner. In contrast, phasic excitation of dopamine neurons robustly potentiated the initiation of lever pressing; however, this effect did not last more than 1 s and quickly diminished. Indeed, response rates markedly decreased when lever pressing was reinforced with inter-reinforcement interval schedules of 3 or 10 s or ratio schedules requiring multiple responses per reinforcement. Thus, phasic excitation of dopamine neurons briefly potentiates the initiation of approach behavior with apparent lack of long-term motivational regulation.

Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion.

Dopamine neurons in the ventral tegmental area (VTA) are implicated in affective functions. However, it is unclear to what extent dopamine neurons in substantia nigra pars compacta (SNc) play such roles. TH-Cre transgenic mice received adeno-associated viral vectors encoding channelrhodopsin2 (ChR2), halorhodopsin (NpHR), or control vector into the VTA or SNc, resulting in selective expression of these opsins in dopamine neurons. Mice with ChR2 learned instrumental responding to deliver photostimulation into the VTA or SNc and also sought for the compartment where they received photostimulation (i.e., operant place preference). Operant place preference scores were highly correlated with self-stimulation responses. In contrast, mice with NpHR avoided the compartment where they received photostimulation into the VTA, SNc, or dorsal striatum, whereas control mice did not. These observations suggest that the excitation and inhibition of SNc dopamine neurons elicit positive and negative affective effects, respectively, similar to those of VTA dopamine neurons.

Electrical stimulation of lateral habenula during learning: frequency-dependent effects on acquisition but not retrieval of a two-way active avoidance response.

The lateral habenula (LHb) is an epithalamic structure involved in signaling reward omission and aversive stimuli, and it inhibits dopaminergic neurons during motivated behavior. Less is known about LHb involvement in the acquisition and retrieval of avoidance learning. Our previous studies indicated that brief electrical stimulation of the LHb, time-locked to the avoidance of aversive footshock (presumably during the positive affective "relief" state that occurs when an aversive outcome is averted), inhibited the acquisition of avoidance learning. In the present study, we used the same paradigm to investigate different frequencies of LHb stimulation. The effect of 20 Hz vs. 50 Hz vs. 100 Hz stimulation was investigated during two phases, either during acquisition or retrieval in Mongolian gerbils. The results indicated that 50 Hz, but not 20 Hz, was sufficient to produce a long-term impairment in avoidance learning, and was somewhat more effective than 100 Hz in this regard. None of the stimulation parameters led to any effects on retrieval of avoidance learning, nor did they affect general motor activity. This suggests that, at frequencies in excess of the observed tonic firing rates of LHb neurons (>1-20 Hz), LHb stimulation may serve to interrupt the consolidation of new avoidance memories. However, these stimulation parameters are not capable of modifying avoidance memories that have already undergone extensive consolidation.

The role of dopamine in the context of aversive stimuli with particular reference to acoustically signaled avoidance learning.

Learning from punishment is a powerful means for behavioral adaptation with high relevance for various mechanisms of self-protection. Several studies have explored the contribution of released dopamine (DA) or responses of DA neurons on reward seeking using rewards such as food, water, and sex. Phasic DA signals evoked by rewards or conditioned reward predictors are well documented, as are modulations of these signals by such parameters as reward magnitude, probability, and deviation of actually occurring from expected rewards. Less attention has been paid to DA neuron firing and DA release in response to aversive stimuli, and the prediction and avoidance of punishment. In this review, we first focus on DA changes in response to aversive stimuli as measured by microdialysis and voltammetry followed by the change in electrophysiological signatures by aversive stimuli and fearful events. We subsequently focus on the role of DA and effect of DA manipulations on signaled avoidance learning, which consists of learning the significance of a warning cue through Pavlovian associations and the execution of an instrumental avoidance response. We present a coherent framework utilizing the data on microdialysis, voltammetry, electrophysiological recording, electrical brain stimulation, and behavioral analysis. We end by outlining current gaps in the literature and proposing future directions aimed at incorporating technical and conceptual progress to understand the involvement of reward circuit on punishment based decisions.

Effects of ventral tegmental area stimulation on the acquisition and long-term retention of active avoidance learning.

The development of avoidance learning depends on dopamine release in forebrain regions. Previous studies indicated that rewarding brain stimulation facilitated two-way active avoidance learning. However, it is not clear whether the temporal relationship of brain stimulation to the training session (before, during or after) is important. To investigate the role of stimulation condition (no stimulation, self-stimulation only, or self-stimulation plus avoidance stimulation) and sequence of self-stimulation training (before or after avoidance training), we used a 3×2 factorial design, in which every level of stimulation was paired with every level of sequence for a total of 6 different groups. The results suggest that self-stimulation either before or after avoidance learning improved acquisition performance, but acquisition was maximal when stimulation was also given during acquisition trials. Importantly, the sequence of self-stimulation (before or after each acquisition session) was irrelevant to this beneficial effect. However, stimulation had no apparent effect on long-term retention when tested 10 days later under conditions of no stimulation, except that the performance of the group that had previously received avoidance-contingent stimulation deteriorated over the course of 60 trials. This may reflect frustration from the omission of expected reward. These results are relevant for optimizing brain stimulation to improve learning.

Differential neuromodulation of acquisition and retrieval of avoidance learning by the lateral habenula and ventral tegmental area.

Several studies suggest an opponent functional relationship between the lateral habenula (LHb) and the ventral tegmental area (VTA). Previous work has linked LHb activation to the inhibition of dopaminergic neurons during loss of reward, as well as to deficits in escape and avoidance learning. We hypothesized that a dopamine signal might underlie the negative reinforcement of avoidance responses and that LHb activation could block this signal and thereby cause avoidance deficits. To test this idea, we implanted stimulating electrodes in either the VTA or LHb of gerbils engaged in two-way active avoidance learning, a task that shows learning-associated dopamine changes and that is acquired faster following LHb lesions. We delivered brief electrical brain stimulation whenever the animal performed a correct response, i.e., when the successful avoidance of foot shock was hypothesized to trigger an intrinsic reward signal. During the acquisition phase, VTA stimulation improved avoidance performance, while LHb stimulation impaired it. VTA stimulation appeared to improve both acquisition and asymptotic performance of the avoidance response, as VTA-stimulated animals reached above-normal performance but reverted to normal responding when stimulation was discontinued. The effects of LHb stimulation during avoidance acquisition were long lasting and persisted even after stimulation was discontinued. However, when given after successful acquisition of avoidance behavior, LHb stimulation had no effect, indicating that LHb stimulation specifically impaired avoidance acquisition without affecting memory retrieval or motivation or ability to perform the avoidance response. These results demonstrate opponent roles of LHb and VTA during acquisition but not during retrieval of avoidance learning.

High-resolution mapping of neuronal activity using the lipophilic thallium chelate complex TlDDC: protocol and validation of the method.

In neurons the rate of K(+)-uptake increases with increasing activity. K(+)-analogues like the heavy metal ion thallium (Tl(+)) can be used, therefore, as tracers for imaging neuronal activity. However, when water-soluble Tl(+)-salts are injected systemically only minute amounts of the tracer enter the brain and the Tl(+)-uptake patterns are influenced by regional differences in blood-brain barrier (BBB) K(+)-permeability. We here show that the BBB-related limitations in using Tl(+) for imaging neuronal activity are no longer present when the lipophilic Tl(+) chelate complex thallium diethyldithiocarbamate (TlDDC) is applied. We systemically injected rodents with TlDDC and mapped the Tl(+)-distribution in the brain using an autometallographic (AMG) technique, a histochemical method for detecting heavy metals. We find that Tl(+)-doses for optimum AMG staining could be substantially reduced, and regional differences attributable to differences in BBB K(+)-permeability were no longer detectable, indicating that TlDDC crosses the BBB. At the cellular level, however, the Tl(+)-distribution was essentially the same as after injection of water-soluble Tl(+)-salts, indicating Tl(+)-release from TlDDC prior to neuronal or glial uptake. Upon sensory stimulation or intracortical microstimulation neuronal Tl(+)-uptake increased after TlDDC injection, upon muscimol treatment neuronal Tl(+)-uptake decreased. We present a protocol for mapping neuronal activity with cellular resolution, which is based on intravenous TlDDC injections during ongoing activity in unrestrained behaving animals and short stimulation times of 5 min.