RESUMEN
Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.
Asunto(s)
Dopamina/metabolismo , Transducción de Señal , Potenciales de Acción/fisiología , Animales , Axones/metabolismo , Calcio/metabolismo , Señalización del Calcio , Cuerpo Celular/metabolismo , Señales (Psicología) , Neuronas Dopaminérgicas/fisiología , Fluorometría , Masculino , Ratones Endogámicos C57BL , Modelos Neurológicos , Estimulación Luminosa , Recompensa , Sensación , Factores de Tiempo , Área Tegmental Ventral/metabolismo , Realidad VirtualRESUMEN
Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a large network of dopaminergic projections. To dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent G-protein-coupled receptor activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity and signal-to-noise ratio with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.
Asunto(s)
Dopamina , Núcleo Accumbens , Ratones , Animales , Núcleo Accumbens/fisiología , Receptores Dopaminérgicos , Encéfalo , Receptores Acoplados a Proteínas GRESUMEN
Dopamine neurons facilitate learning by calculating reward prediction error, or the difference between expected and actual reward. Despite two decades of research, it remains unclear how dopamine neurons make this calculation. Here we review studies that tackle this problem from a diverse set of approaches, from anatomy to electrophysiology to computational modeling and behavior. Several patterns emerge from this synthesis: that dopamine neurons themselves calculate reward prediction error, rather than inherit it passively from upstream regions; that they combine multiple separate and redundant inputs, which are themselves interconnected in a dense recurrent network; and that despite the complexity of inputs, the output from dopamine neurons is remarkably homogeneous and robust. The more we study this simple arithmetic computation, the knottier it appears to be, suggesting a daunting (but stimulating) path ahead for neuroscience more generally.
Asunto(s)
Encéfalo/fisiología , Dopamina/fisiología , Aprendizaje/fisiología , Red Nerviosa/fisiología , Recompensa , Animales , Humanos , Vías Nerviosas/fisiologíaRESUMEN
Mice display robust, stereotyped behaviours towards pups: virgin males typically attack pups, whereas virgin females and sexually experienced males and females display parental care. Here we show that virgin males genetically impaired in vomeronasal sensing do not attack pups and are parental. Furthermore, we uncover a subset of galanin-expressing neurons in the medial preoptic area (MPOA) that are specifically activated during male and female parenting, and a different subpopulation that is activated during mating. Genetic ablation of MPOA galanin neurons results in marked impairment of parental responses in males and females and affects male mating. Optogenetic activation of these neurons in virgin males suppresses inter-male and pup-directed aggression and induces pup grooming. Thus, MPOA galanin neurons emerge as an essential regulatory node of male and female parenting behaviour and other social responses. These results provide an entry point to a circuit-level dissection of parental behaviour and its modulation by social experience.
Asunto(s)
Galanina/metabolismo , Conducta Materna/fisiología , Neuronas/metabolismo , Conducta Paterna/fisiología , Área Preóptica/citología , Agresión/fisiología , Animales , Copulación , Femenino , Galanina/deficiencia , Galanina/genética , Aseo Animal/fisiología , Masculino , Ratones , Optogenética , Feromonas/análisis , Área Preóptica/metabolismo , Canales Catiónicos TRPC/deficiencia , Canales Catiónicos TRPC/genética , Órgano Vomeronasal/fisiologíaRESUMEN
Hunger is a hard-wired motivational state essential for survival. Agouti-related peptide (AgRP)-expressing neurons in the arcuate nucleus (ARC) at the base of the hypothalamus are crucial to the control of hunger. They are activated by caloric deficiency and, when naturally or artificially stimulated, they potently induce intense hunger and subsequent food intake. Consistent with their obligatory role in regulating appetite, genetic ablation or chemogenetic inhibition of AgRP neurons decreases feeding. Excitatory input to AgRP neurons is important in caloric-deficiency-induced activation, and is notable for its remarkable degree of caloric-state-dependent synaptic plasticity. Despite the important role of excitatory input, its source(s) has been unknown. Here, through the use of Cre-recombinase-enabled, cell-specific neuron mapping techniques in mice, we have discovered strong excitatory drive that, unexpectedly, emanates from the hypothalamic paraventricular nucleus, specifically from subsets of neurons expressing thyrotropin-releasing hormone (TRH) and pituitary adenylate cyclase-activating polypeptide (PACAP, also known as ADCYAP1). Chemogenetic stimulation of these afferent neurons in sated mice markedly activates AgRP neurons and induces intense feeding. Conversely, acute inhibition in mice with caloric-deficiency-induced hunger decreases feeding. Discovery of these afferent neurons capable of triggering hunger advances understanding of how this intense motivational state is regulated.
Asunto(s)
Proteína Relacionada con Agouti/metabolismo , Hambre/fisiología , Vías Nerviosas/fisiología , Neuronas/metabolismo , Núcleo Hipotalámico Paraventricular/fisiología , Proteína Relacionada con Agouti/deficiencia , Animales , Apetito/efectos de los fármacos , Apetito/fisiología , Núcleo Arqueado del Hipotálamo/citología , Núcleo Arqueado del Hipotálamo/metabolismo , Mapeo Encefálico , Rastreo Celular , Clozapina/análogos & derivados , Clozapina/farmacología , Dependovirus/genética , Ingestión de Alimentos/efectos de los fármacos , Ingestión de Alimentos/fisiología , Femenino , Privación de Alimentos , Hambre/efectos de los fármacos , Integrasas/metabolismo , Masculino , Ratones , Vías Nerviosas/efectos de los fármacos , Plasticidad Neuronal/efectos de los fármacos , Plasticidad Neuronal/fisiología , Neuronas/efectos de los fármacos , Neuronas Aferentes/efectos de los fármacos , Neuronas Aferentes/metabolismo , Núcleo Hipotalámico Paraventricular/citología , Fragmentos de Péptidos/deficiencia , Fragmentos de Péptidos/metabolismo , Polipéptido Hipofisario Activador de la Adenilato-Ciclasa/metabolismo , Virus de la Rabia/genética , Respuesta de Saciedad/fisiología , Hormona Liberadora de Tirotropina/metabolismoRESUMEN
Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation. Notably, whereas glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli into more positive responses, whereas excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.
Asunto(s)
Neuronas Dopaminérgicas , Ácido Glutámico , Ratones , Animales , Neuronas Dopaminérgicas/fisiología , Dopamina/fisiología , Recompensa , Mesencéfalo , Área Tegmental Ventral/fisiologíaRESUMEN
Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs) but the mechanisms underlying RPE computation, particularly contributions of different neurotransmitters, remain poorly understood. Here we used a genetically-encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons. We found that glutamate inputs exhibit virtually all of the characteristics of RPE, rather than conveying a specific component of RPE computation such as reward or expectation. Notably, while glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli toward more positive responses, while excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.
RESUMEN
Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a vast network of dopaminergic projections. To fully dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent GPCR activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity, and signal-to-noise properties with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in freely moving mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala, and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.
RESUMEN
Social grouping increases survival in many species, including humans1,2. By contrast, social isolation generates an aversive state (loneliness) that motivates social seeking and heightens social interaction upon reunion3-5. The observed rebound in social interaction triggered by isolation suggests a homeostatic process underlying the control of social drive, similar to that observed for physiological needs such as hunger, thirst or sleep3,6. In this study, we assessed social responses in multiple mouse strains and identified the FVB/NJ line as exquisitely sensitive to social isolation. Using FVB/NJ mice, we uncovered two previously uncharacterized neuronal populations in the hypothalamic preoptic nucleus that are activated during social isolation and social rebound and that orchestrate the behavior display of social need and social satiety, respectively. We identified direct connectivity between these two populations of opposite function and with brain areas associated with social behavior, emotional state, reward, and physiological needs, and showed that animals require touch to assess the presence of others and fulfill their social need, thus revealing a brain-wide neural system underlying social homeostasis. These findings offer mechanistic insight into the nature and function of circuits controlling instinctive social need and for the understanding of healthy and diseased brain states associated with social context.
RESUMEN
A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.
Asunto(s)
Dopamina , Recompensa , Animales , Señales (Psicología) , Dopamina/fisiología , Neuronas Dopaminérgicas/fisiología , Aprendizaje Automático , Mesencéfalo , RatonesRESUMEN
Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dynamics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward- and threat-related dopamine sub-systems.
Asunto(s)
Cuerpo Estriado , Dopamina , Animales , Ratones , Dopamina/fisiología , Cuerpo Estriado/fisiología , Refuerzo en Psicología , Recompensa , Aprendizaje/fisiologíaRESUMEN
The polarization of a neuron generally results in the formation of one axon and multiple dendrites, allowing for the establishment of neuronal circuitry. The molecular mechanisms involved in priming one neurite to become the axon, particularly those regulating the microtubule network, remain elusive. Here we report the identification of DOCK7, a member of the DOCK180-related protein superfamily, as a Rac GTPase activator that is asymmetrically distributed in unpolarized hippocampal neurons and selectively expressed in the axon. Knockdown of DOCK7 expression prevents axon formation, whereas overexpression induces formation of multiple axons. We further demonstrate that DOCK7 and Rac activation lead to phosphorylation and inactivation of the microtubule destabilizing protein stathmin/Op18 in the nascent axon and that this event is important for axon development. Our findings unveil a pathway linking the Rac activator DOCK7 to a microtubule regulatory protein and highlight the contribution of microtubule network regulation to axon development.
Asunto(s)
Proteínas Activadoras de GTPasa/metabolismo , Neuronas/metabolismo , Estatmina/metabolismo , Proteínas de Unión al GTP rac/metabolismo , Animales , Axones/metabolismo , Western Blotting , Encéfalo/crecimiento & desarrollo , Encéfalo/metabolismo , Línea Celular , Línea Celular Tumoral , Polaridad Celular , Células Cultivadas , Proteínas Activadoras de GTPasa/genética , Proteínas Fluorescentes Verdes/genética , Proteínas Fluorescentes Verdes/metabolismo , Factores de Intercambio de Guanina Nucleótido , Hipocampo/citología , Hipocampo/metabolismo , Humanos , Microscopía Confocal , Microscopía Fluorescente , Neuronas/citología , Fosforilación , Unión Proteica , Ratas , Proteínas Recombinantes de Fusión/genética , Proteínas Recombinantes de Fusión/metabolismo , Estatmina/genética , Factores de Tiempo , Transfección , Técnicas del Sistema de Dos Híbridos , Proteínas de Unión al GTP rac/genéticaRESUMEN
Different regions of the striatum regulate different types of behavior. However, how dopamine signals differ across striatal regions and how dopamine regulates different behaviors remain unclear. Here, we compared dopamine axon activity in the ventral, dorsomedial, and dorsolateral striatum, while mice performed a perceptual and value-based decision task. Surprisingly, dopamine axon activity was similar across all three areas. At a glance, the activity multiplexed different variables such as stimulus-associated values, confidence, and reward feedback at different phases of the task. Our modeling demonstrates, however, that these modulations can be inclusively explained by moment-by-moment changes in the expected reward, that is the temporal difference error. A major difference between areas was the overall activity level of reward responses: reward responses in dorsolateral striatum were positively shifted, lacking inhibitory responses to negative prediction errors. The differences in dopamine signals put specific constraints on the properties of behaviors controlled by dopamine in these regions.
Asunto(s)
Axones/fisiología , Cuerpo Estriado/fisiología , Toma de Decisiones/fisiología , Neuronas Dopaminérgicas/fisiología , Animales , Femenino , Masculino , Ratones , Ratones Endogámicos C57BL , Odorantes , Refuerzo en Psicología , Recompensa , OlfatoRESUMEN
The prefrontal cortex modifies the sensory system to focus attention. In this issue of Neuron, Nakajima et al. (2019) fill the gap between the prefrontal cortex and the sensory system with an overlooked basal ganglia pathway.
Asunto(s)
Ganglios Basales , Corteza Prefrontal , Atención , Filtrado Sensorial , TálamoRESUMEN
Dopamine and serotonin play critical roles in flexible behaviors and are related to various psychiatric and motor disorders. This paper reviews the global organization of dopamine and serotonin systems through recent findings using a modified rabies virus. We first introduce methods for comprehensive mapping of monosynaptic inputs. We then describe quantitative comparisons across the data regarding monosynaptic inputs to dopamine neurons versus serotonin neurons. There is surprising similarity between the input to dopamine neurons in the ventral tegmental area (VTA) and the input to serotonin neurons in the dorsal raphe (DR), suggesting functional interactions between these systems. We next introduce studies of mapping monosynaptic inputs to subpopulations of dopamine neurons specified by their projection targets. It was found that the population of dopamine neurons that project to the tail of the striatum (TS) forms an anatomically distinct outlier, suggesting a unique function. From these series of anatomical studies, we propose that there are three information flows that regulate these neuromodulatory systems: the midline stream to serotonin neurons in median raphe (MR) and B6, the central stream to value-coding dopamine neurons and serotonin neurons in rostral DR, and the lateral stream to TS-projecting dopamine neurons. Finally we introduce a new approach to investigate firing patterns of monosynaptic inputs to dopamine neurons in behaving animals. Combining anatomical and physiological findings, we propose that within the central stream, dopamine neurons broadcast a central teaching signal rather than personal teaching signals to multiple brain areas, which are computed in a redundant way in multi-layered neural circuits. Examination of global organization of the dopamine and serotonin circuits not only revealed the complexity of the systems but also revealed some principles of their organization. We will also discuss limitations, practical issues and the possibility of future improvements of the rabies virus-mediated tracing system.
Asunto(s)
Neuronas Dopaminérgicas/fisiología , Vectores Genéticos , Virus de la Rabia/genética , Neuronas Serotoninérgicas/fisiología , Sinapsis/fisiología , Animales , Encéfalo/fisiología , Vías Nerviosas/fisiologíaRESUMEN
The ability to predict future outcomes increases the fitness of the animal. Decades of research have shown that dopamine neurons broadcast reward prediction error (RPE) signals-the discrepancy between actual and predicted reward-to drive learning to predict future outcomes. Recent studies have begun to show, however, that dopamine neurons are more diverse than previously thought. In this review, we will summarize a series of our studies that have shown unique properties of dopamine neurons projecting to the posterior "tail" of the striatum (TS) in terms of anatomy, activity, and function. Specifically, TS-projecting dopamine neurons are activated by a subset of negative events including threats from a novel object, send prediction errors for external threats, and reinforce avoidance behaviors. These results indicate that there are at least two axes of dopamine-mediated reinforcement learning in the brain-one learning from canonical RPEs and another learning from threat prediction errors. We argue that the existence of multiple learning systems is an adaptive strategy that makes possible each system optimized for its own needs. The compartmental organization in the mammalian striatum resembles that of a dopamine-recipient area in insects (mushroom body), pointing to a principle of dopamine function conserved across phyla.
RESUMEN
Midbrain dopamine neurons are well known for their role in reward-based reinforcement learning. We found that the activity of dopamine axons in the posterior tail of the striatum (TS) scaled with the novelty and intensity of external stimuli, but did not encode reward value. We demonstrated that the ablation of TS-projecting dopamine neurons specifically inhibited avoidance of novel or high-intensity stimuli without affecting animals' initial avoidance responses, suggesting a role in reinforcement rather than simply in avoidance itself. Furthermore, we found that animals avoided optogenetic activation of dopamine axons in TS during a choice task and that this stimulation could partially reinstate avoidance of a familiar object. These results suggest that TS-projecting dopamine neurons reinforce avoidance of threatening stimuli. More generally, our results indicate that there are at least two axes of reinforcement learning using dopamine in the striatum: one based on value and one based on external threat.
Asunto(s)
Reacción de Prevención/fisiología , Cuerpo Estriado/citología , Neuronas Dopaminérgicas/fisiología , Vías Nerviosas/fisiología , Refuerzo en Psicología , Animales , Benzazepinas/farmacología , Encéfalo/citología , Encéfalo/efectos de los fármacos , Encéfalo/fisiología , Channelrhodopsins/genética , Channelrhodopsins/metabolismo , Fosfodiesterasas de Nucleótidos Cíclicos Tipo 1/genética , Fosfodiesterasas de Nucleótidos Cíclicos Tipo 1/metabolismo , Dopaminérgicos/farmacología , Proteínas de Transporte de Dopamina a través de la Membrana Plasmática/genética , Proteínas de Transporte de Dopamina a través de la Membrana Plasmática/metabolismo , Neuronas Dopaminérgicas/efectos de los fármacos , Proteínas Luminiscentes/genética , Proteínas Luminiscentes/metabolismo , Ratones , Ratones Endogámicos C57BL , Ratones Transgénicos , Oxidopamina/farmacología , Unión Proteica/efectos de los fármacos , Reconocimiento en Psicología/efectos de los fármacos , Proteína 2 de Transporte Vesicular de Glutamato/genética , Proteína 2 de Transporte Vesicular de Glutamato/metabolismoRESUMEN
The formation and elaboration of axonal and dendritic morphologies are fundamental aspects of neuronal polarization critical for information processing. In general, developing CNS neurons elaborate one axon and multiple dendrites in response to intracellular and extracellular cues, so as to transmit and receive information, respectively. The molecular mechanisms underlying axon-dendrite polarity are complex and involve the integration of numerous signaling pathways that impinge on the cytoskeleton. One group of proteins, the Rho GTPases, has emerged as key integrators of environmental cues to regulate the underlying axonal and dendritic cytoskeletons. Here, we discuss the role of regulators of the Rac1 GTPase in axon development and highlight the importance of both actin and microtubule remodeling in this process.
Asunto(s)
Regulación del Desarrollo de la Expresión Génica/fisiología , Neuronas/citología , Neuronas/enzimología , Proteínas de Unión al GTP rho/metabolismo , Animales , Humanos , Neuronas/fisiología , Transducción de Señal/fisiologíaRESUMEN
Although modified rabies viruses have emerged as a powerful tool for tracing the inputs to genetically defined populations of neurons, the toxicity of the virus has limited its utility. A recent study employed a self-inactivating rabies (SiR) virus that enables recording or manipulation of targeted neurons for months.
Asunto(s)
ADN , Virus de la Rabia/genética , NeuronasRESUMEN
Dopamine neurons are thought to encode novelty in addition to reward prediction error (the discrepancy between actual and predicted values). In this study, we compared dopamine activity across the striatum using fiber fluorometry in mice. During classical conditioning, we observed opposite dynamics in dopamine axon signals in the ventral striatum ('VS dopamine') and the posterior tail of the striatum ('TS dopamine'). TS dopamine showed strong excitation to novel cues, whereas VS dopamine showed no responses to novel cues until they had been paired with a reward. TS dopamine cue responses decreased over time, depending on what the cue predicted. Additionally, TS dopamine showed excitation to several types of stimuli including rewarding, aversive, and neutral stimuli whereas VS dopamine showed excitation only to reward or reward-predicting cues. Together, these results demonstrate that dopamine novelty signals are localized in TS along with general salience signals, while VS dopamine reliably encodes reward prediction error.