Glutamate inputs send prediction error of reward, but not negative value of aversive stimuli, to dopamine neurons.

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation. Notably, whereas glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli into more positive responses, whereas excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

Prediction error in dopamine neurons during associative learning.

Dopamine neurons have long been thought to facilitate learning by broadcasting reward prediction error (RPE), a teaching signal used in machine learning, but more recent work has advanced alternative models of dopamine's computational role. Here, I revisit this critical issue and review new experimental evidences that tighten the link between dopamine activity and RPE. First, I introduce the recent observation of a gradual backward shift of dopamine activity that had eluded researchers for over a decade. I also discuss several other findings, such as dopamine ramping, that were initially interpreted to conflict but later found to be consistent with RPE. These findings improve our understanding of neural computation in dopamine neurons.

Glutamate inputs send prediction error of reward but not negative value of aversive stimuli to dopamine neurons.

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs) but the mechanisms underlying RPE computation, particularly contributions of different neurotransmitters, remain poorly understood. Here we used a genetically-encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons. We found that glutamate inputs exhibit virtually all of the characteristics of RPE, rather than conveying a specific component of RPE computation such as reward or expectation. Notably, while glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli toward more positive responses, while excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning.

A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.

Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli.

Midbrain dopamine neurons are well known for their role in reward-based reinforcement learning. We found that the activity of dopamine axons in the posterior tail of the striatum (TS) scaled with the novelty and intensity of external stimuli, but did not encode reward value. We demonstrated that the ablation of TS-projecting dopamine neurons specifically inhibited avoidance of novel or high-intensity stimuli without affecting animals' initial avoidance responses, suggesting a role in reinforcement rather than simply in avoidance itself. Furthermore, we found that animals avoided optogenetic activation of dopamine axons in TS during a choice task and that this stimulation could partially reinstate avoidance of a familiar object. These results suggest that TS-projecting dopamine neurons reinforce avoidance of threatening stimuli. More generally, our results indicate that there are at least two axes of reinforcement learning using dopamine in the striatum: one based on value and one based on external threat.

Optical measurement of neuronal activity in the developing cerebellum of zebrafish using voltage-sensitive dye imaging.

Voltage-sensitive dye (VSD) imaging enables fast, direct, and simultaneous detection of membrane potentials from a population of neurons forming neuronal circuits. This enables the detection of hyperpolarization together with depolarization, whose balance plays a pivotal role in the function of many brain regions. Among these is the cerebellum, which contains a significant number of inhibitory neurons. However, the mechanism underlying the functional development remains unclear. In this study, we used a model system ideal to study neurogenesis by applying VSD imaging to the cerebellum of zebrafish larvae to analyze the neuronal activity of the developing cerebellum, focusing on both excitation and inhibition. We performed in-vivo high-speed imaging of the entire cerebellum of the zebrafish, which was stained using Di-4-ANEPPS, a widely used VSD. To examine whether neuronal activity in the zebrafish cerebellum could be detected by this VSD, we applied electrical stimulation during VSD imaging, which showed that depolarization was detected widely in the cerebellum upon stimulation. These responses mostly disappeared following treatment with tetrodotoxin, indicating that Di-4-ANEPPS enabled optical measurement of neuronal activity in the developing cerebellum of zebrafish. Moreover, hyperpolarizing signals were also detected upon stimulation, but these were significantly reduced by treatment with picrotoxin, a GABAA receptor inhibitor, indicating that these responses represent inhibitory signals. This approach will enable a detailed analysis of the spatiotemporal dynamics of the excitation and inhibition in the cerebellum along its developmental stages, leading to a deeper understanding of the functional development of the cerebellum in vertebrates.

Optical interrogation of neuronal circuitry in zebrafish using genetically encoded voltage indicators.

Optical measurement of membrane potentials enables fast, direct and simultaneous detection of membrane potentials from a population of neurons, providing a desirable approach for functional analysis of neuronal circuits. Here, we applied recently developed genetically encoded voltage indicators, ASAP1 (Accelerated Sensor of Action Potentials 1) and QuasAr2 (Quality superior to Arch 2), to zebrafish, an ideal model system for studying neurogenesis. To achieve this, we established transgenic lines which express the voltage sensors, and showed that ASAP1 is expressed in zebrafish neurons. To examine whether neuronal activity could be detected by ASAP1, we performed whole-cerebellum imaging, showing that depolarization was detected widely in the cerebellum and optic tectum upon electrical stimulation. Spontaneous activity in the spinal cord was also detected by ASAP1 imaging at single-cell resolution as well as at the neuronal population level. These responses mostly disappeared following treatment with tetrodotoxin, indicating that ASAP1 enabled optical measurement of neuronal activity in the zebrafish brain. Combining this method with other approaches, such as optogenetics and behavioural analysis may facilitate a deeper understanding of the functional organization of brain circuitry and its development.

Social conflict resolution regulated by two dorsal habenular subregions in zebrafish.

When animals encounter conflict they initiate and escalate aggression to establish and maintain a social hierarchy. The neural mechanisms by which animals resolve fighting behaviors to determine such social hierarchies remain unknown. We identified two subregions of the dorsal habenula (dHb) in zebrafish that antagonistically regulate the outcome of conflict. The losing experience reduced neural transmission in the lateral subregion of dHb (dHbL)-dorsal/intermediate interpeduncular nucleus (d/iIPN) circuit. Silencing of the dHbL or medial subregion of dHb (dHbM) caused a stronger predisposition to lose or win a fight, respectively. These results demonstrate that the dHbL and dHbM comprise a dual control system for conflict resolution of social aggression.

The habenulo-raphe serotonergic circuit encodes an aversive expectation value essential for adaptive active avoidance of danger.

Anticipation of danger at first elicits panic in animals, but later it helps them to avoid the real threat adaptively. In zebrafish, as fish experience more and more danger, neurons in the ventral habenula (vHb) showed tonic increase in the activity to the presented cue and activated serotonergic neurons in the median raphe (MR). This neuronal activity could represent the expectation of a dangerous outcome and be used for comparison with a real outcome when the fish is learning how to escape from a dangerous to a safer environment. Indeed, inhibiting synaptic transmission from vHb to MR impaired adaptive avoidance learning, while panic behavior induced by classical fear conditioning remained intact. Furthermore, artificially triggering this negative outcome expectation signal by optogenetic stimulation of vHb neurons evoked place avoidance behavior. Thus, vHb-MR circuit is essential for representing the level of expected danger and behavioral programming to adaptively avoid potential hazard.

Imaging of neural ensemble for the retrieval of a learned behavioral program.

The encoding of long-term associative memories for learned behaviors is a fundamental brain function. Yet, how behavior is stably consolidated and retrieved in the vertebrate cortex is poorly understood. We trained zebrafish in aversive reinforcement learning and measured calcium signals across their entire brain during retrieval of the learned response. A discrete area of dorsal telencephalon that was inactive immediately after training became active the next day. Analysis of the identified area indicated that it was specific and essential for long-term memory retrieval and contained electrophysiological responses entrained to the learning stimulus. When the behavioral rule changed, a rapid spatial shift in the functional map across the telencephalon was observed. These results demonstrate that the retrieval of long-term memories for learned behaviors can be studied at the whole-brain scale in behaving zebrafish in vivo. Moreover, the findings indicate that consolidated memory traces can be rapidly modified during reinforcement learning.

Phylogeny and ontogeny of the habenular structure.

Habenula is an epithalamic nucleus connecting the forebrain with the ventral midbrain and hindbrain that plays a pivotal role in decision making by regulating dopaminergic and serotonergic activities. Intriguingly, habenula has also attracted interest as a model for brain asymmetry, since many vertebrates show left-right differences in habenula size and neural circuitry. Despite the functional significance of this nucleus, few studies have addressed the molecular mechanisms underlying habenular development. Mammalian habenula consists of the medial and lateral habenulae, which have distinct neural connectivity. The habenula shows phylogenetic conservation from fish to human, and studies using genetically accessible model animals have provided molecular insights into the developmental mechanisms of the habenula. The results suggest that development of the habenular asymmetry is mediated by differential regulation of the neurogenetic period for generating specific neuronal subtypes. Since the orientation and size ratio of the medial and lateral habenulae differ across species, the evolution of those subregions within the habenula may also reflect changes in neurogenesis duration for each habenular subdivision according to the evolutionary process.

The habenula is crucial for experience-dependent modification of fear responses in zebrafish.

The zebrafish dorsal habenula (dHb) shows conspicuous asymmetry in its connection with the interpeduncular nucleus (IPN) and is equivalent to the mammalian medial habenula. Genetic inactivation of the lateral subnucleus of dHb (dHbL) biased fish towards freezing rather than the normal flight response to a conditioned fear stimulus, suggesting that the dHbL-IPN pathway is important for controlling experience-dependent modification of fear responses.

Identification of the zebrafish ventral habenula as a homolog of the mammalian lateral habenula.

The mammalian habenula consists of the medial and lateral habenulae. Recent behavioral and electrophysiological studies suggested that the lateral habenula plays a pivotal role in controlling motor and cognitive behaviors by influencing the activity of dopaminergic and serotonergic neurons. Despite the functional significance, manipulating neural activity in this pathway remains difficult because of the absence of a genetically accessible animal model such as zebrafish. To address the level of lateral habenula conservation in zebrafish, we applied the tract-tracing technique to GFP (green fluorescent protein)-expressing transgenic zebrafish to identify habenular neurons that project to the raphe nuclei, a major target of the mammalian lateral habenula. Axonal tracing in live and fixed fish showed projection of zebrafish ventral habenula axons to the ventral part of the median raphe, but not to the interpeduncular nucleus where the dorsal habenula projected. The ventral habenula expressed protocadherin 10a, a specific marker of the rat lateral habenula, whereas the dorsal habenula showed no such expression. Gene expression analyses revealed that the ventromedially positioned ventral habenula in the adult originated from the region of primordium lateral to the dorsal habenula during development. This suggested that zebrafish habenulae emerge during development with mediolateral orientation similar to that of the mammalian medial and lateral habenulae. These findings indicated that the lateral habenular pathways are evolutionarily conserved pathways and might control adaptive behaviors in vertebrates through the regulation of monoaminergic activities.

Germline gain-of-function mutations in RAF1 cause Noonan syndrome.

Noonan syndrome is characterized by short stature, facial dysmorphia and a wide spectrum of congenital heart defects. Mutations of PTPN11, KRAS and SOS1 in the RAS-MAPK pathway cause approximately 60% of cases of Noonan syndrome. However, the gene(s) responsible for the remainder are unknown. We have identified five different mutations in RAF1 in ten individuals with Noonan syndrome; those with any of four mutations causing changes in the CR2 domain of RAF1 had hypertrophic cardiomyopathy (HCM), whereas affected individuals with mutations leading to changes in the CR3 domain did not. Cells transfected with constructs containing Noonan syndrome-associated RAF1 mutations showed increased in vitro kinase and ERK activation, and zebrafish embryos with morpholino knockdown of raf1 demonstrated the need for raf1 for the development of normal myocardial structure and function. Thus, our findings implicate RAF1 gain-of-function mutations as a causative agent of a human developmental disorder, representing a new genetic mechanism for the activation of the MAPK pathway.