Opposing Contributions of GABAergic and Glutamatergic Ventral Pallidal Neurons to Motivational Behaviors.

The ventral pallidum (VP) is critical for invigorating reward seeking and is also involved in punishment avoidance, but how it contributes to such opposing behavioral actions remains unclear. Here, we show that GABAergic and glutamatergic VP neurons selectively control behavior in opposing motivational contexts. In vivo recording combined with optogenetics in mice revealed that these two populations oppositely encode positive and negative motivational value, are differentially modulated by animal's internal state, and determine the behavioral response during motivational conflict. Furthermore, GABAergic VP neurons are essential for movements toward reward in a positive motivational context but suppress movements in an aversive context. In contrast, glutamatergic VP neurons are essential for movements to avoid a threat but suppress movements in an appetitive context. Our results indicate that GABAergic and glutamatergic VP neurons encode the drive for approach and avoidance, respectively, with the balance between their activities determining the type of motivational behavior.

Neural Networks With Motivation.

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent's dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain.

The Lateral Habenula Circuitry: Reward Processing and Cognitive Control.

There has been a growing interest in understanding the role of the lateral habenula (LHb) in reward processing, affect regulation, and goal-directed behaviors. The LHb gets major inputs from the habenula-projecting globus pallidus and the mPFC, sending its efferents to the dopaminergic VTA and SNc, serotonergic dorsal raphe nuclei, and the GABAergic rostromedial tegmental nucleus. Recent studies have made advances in our understanding of the LHb circuit organization, yet the precise mechanisms of its involvement in complex behaviors are largely unknown. To begin to address this unresolved question, we present here emerging cross-species perspectives with a goal to provide a more refined understanding of the role of the LHb circuits in reward and cognition. We begin by highlighting recent findings from rodent experiments using optogenetics, electrophysiology, molecular, pharmacology, and tracing techniques that reveal diverse neural phenotypes in the LHb circuits that may underlie previously undescribed behavioral functions. We then discuss results from electrophysiological studies in macaques that suggest that the LHb cooperates with the anterior cingulate cortex to monitor action outcomes and signal behavioral adjustment. Finally, we provide an integrated summary of cross-species findings and discuss how further research on the connectivity, neural signaling, and physiology of the LHb circuits can deepen our understanding of the role of the LHb in normal and maladaptive behaviors associated with mental illnesses and drug abuse.

A basal ganglia circuit for evaluating action outcomes.

The basal ganglia, a group of subcortical nuclei, play a crucial role in decision-making by selecting actions and evaluating their outcomes. While much is known about the function of the basal ganglia circuitry in selection, how these nuclei contribute to outcome evaluation is less clear. Here we show that neurons in the habenula-projecting globus pallidus (GPh) in mice are essential for evaluating action outcomes and are regulated by a specific set of inputs from the basal ganglia. We find in a classical conditioning task that individual mouse GPh neurons bidirectionally encode whether an outcome is better or worse than expected. Mimicking these evaluation signals with optogenetic inhibition or excitation is sufficient to reinforce or discourage actions in a decision-making task. Moreover, cell-type-specific synaptic manipulations reveal that the inhibitory and excitatory inputs to the GPh are necessary for mice to appropriately evaluate positive and negative feedback, respectively. Finally, using rabies-virus-assisted monosynaptic tracing, we show that the GPh is embedded in a basal ganglia circuit wherein it receives inhibitory input from both striosomal and matrix compartments of the striatum, and excitatory input from the 'limbic' regions of the subthalamic nucleus. Our results provide evidence that information about the selection and evaluation of actions is channelled through distinct sets of basal ganglia circuits, with the GPh representing a key locus in which information of opposing valence is integrated to determine whether action outcomes are better or worse than expected.

The lamprey blueprint of the mammalian nervous system.

The basic features of the vertebrate nervous system are conserved throughout vertebrate phylogeny to a much higher degree than previously thought. In this mini-review, we show that not only the organization of the different motor programs underlying eye, orienting, locomotor, and respiratory movements are similarly organized, but also that the basic structure of the forebrain engaged in the control of movement is conserved. In the lamprey, which diverged already 560 million years ago from the vertebrate line of evolution leading up to primates, the basic components of the basal ganglia are similar to those of mammals in considerable detail. Moreover, the properties of the synaptic input are similar as well as transmitters/peptides in the direct and indirect pathway throughout the basal ganglia. The membrane properties of the striatal projection neurons with D1 and D2 receptors, respectively, are also similar, as are those of the pallidal output neurons. Our evidence suggests that the basal ganglia can be subdivided into functional modules controlling different motor programs, like locomotion and eye movements. What has happened during evolution is that the number of modules has increased in parallel with a progressively more complex behavioral repertoire. For value-based decisions, the circuitry through the lateral habenulae to the dopaminergic modulator neurons is also conserved, as well as the relay inhibitory interneurons involved. The habenular input is from a pallidal glutamatergic nucleus in lamprey as well as mammals, and this nucleus in turn receives input from the striosomal compartment within striatum and also from pallium (cortex in mammals).

Evolutionarily conserved organization of the dopaminergic system in lamprey: SNc/VTA afferent and efferent connectivity and D2 receptor expression.

The dopaminergic system influences motor behavior, signals reward and novelty, and is an essential component of the basal ganglia in all vertebrates including the lamprey, one of the phylogenetically oldest vertebrates. The intrinsic organization and function of the lamprey basal ganglia is highly conserved. For instance, the direct and indirect pathways are modulated through dopamine D1 and D2 receptors in lamprey and in mammals. The nucleus of the tuberculum posterior, a homologue of the substantia nigra pars compacta (SNc)/ventral tegmental area (VTA) is present in lamprey, but only scarce data exist about its connectivity. Likewise, the D2 receptor is expressed in the striatum, but little is known about its localization in other brain areas. We used in situ hybridization and tracer injections, both in combination with tyrosine hydroxylase immunohistochemistry, to characterize the SNc/VTA efferent and afferent connectivity, and to relate its projection pattern with D2 receptor expression in particular. We show that most features of the dopaminergic system are highly conserved. As in mammals, the direct pallial (cortex in mammals) input and the basal ganglia connectivity with the SNc/VTA are present as part of the evaluation system, as well as input from the tectum as the evolutionary basis for salience/novelty detection. Moreover, the SNc/VTA receives sensory information from the olfactory bulbs, optic tectum, octavolateral area, and dorsal column nucleus, and it innervates, apart from the nigrostriatal pathway, several motor-related areas. This suggests that the dopaminergic system also contributes to the control of different motor centers at the brainstem level.

Independent circuits in the basal ganglia for the evaluation and selection of actions.

The basal ganglia are critical for selecting actions and evaluating their outcome. Although the circuitry for selection is well understood, how these nuclei evaluate the outcome of actions is unknown. Here, we show in lamprey that a separate evaluation circuit, which regulates the habenula-projecting globus pallidus (GPh) neurons, exists within the basal ganglia. The GPh neurons are glutamatergic and can drive the activity of the lateral habenula, which, in turn, provides an indirect inhibitory influence on midbrain dopamine neurons. We show that GPh neurons receive inhibitory input from the striosomal compartment of the striatum. The striosomal input can reduce the excitatory drive to the lateral habenula and, consequently, decrease the inhibition onto the dopaminergic system. Dopaminergic neurons, in turn, provide feedback that inhibits the GPh. In addition, GPh neurons receive direct projections from the pallium (cortex in mammals), which can increase the GPh activity to drive the lateral habenula to increase the inhibition of the neuromodulatory systems. This circuitry, thus, differs markedly from the "direct" and "indirect" pathways that regulate the pallidal (e.g., globus pallidus) output nuclei involved in the control of motion. Our results show that a distinct reward-evaluation circuit exists within the basal ganglia, in parallel to the direct and indirect pathways, which select actions. Our results suggest that these circuits are part of the fundamental blueprint that all vertebrates use to select actions and evaluate their outcome.

Dopamine differentially modulates the excitability of striatal neurons of the direct and indirect pathways in lamprey.

The functions of the basal ganglia are critically dependent on dopamine. In mammals, dopamine differentially modulates the excitability of the direct and indirect striatal projection neurons, and these populations selectively express dopamine D1 and D2 receptors, respectively. Although the detailed organization of the basal ganglia is conserved throughout the vertebrate phylum, it was unknown whether the differential dopamine modulation of the direct and indirect pathways is present in non-mammalian species. We aim here to determine whether the receptor expression and opposing dopaminergic modulation of the direct and indirect pathways is present in one of the phylogenetically oldest vertebrates, the river lamprey. Using in situ hybridization and patch-clamp recordings, we show that D1 receptors are almost exclusively expressed in the striatal neurons projecting directly to the homolog of the substantia nigra pars reticulata. In addition, the majority of striatal neurons projecting to the homolog of the globus pallidus interna/globus pallidus externa express D1 or D2 receptors. As in mammals, application of dopamine receptor agonists differentially modulates the excitability of these neurons, increasing the excitability of the D1-expressing neurons and decreasing the excitability of D2-expressing neurons. Our results suggest that the segregated expression of the D1 and D2 receptors in the direct and indirect striatal projection neurons has been conserved across the vertebrate phylum. Because dopamine receptor agonists differentially modulate these pathways, increasing the excitability of the direct pathway and decreasing the excitability of the indirect pathway, this organization may be conserved as a mechanism that biases the networks toward action selection.

The evolutionary origin of the vertebrate basal ganglia and its role in action selection.

The group of nuclei within the basal ganglia of the forebrain is central to the control of movement. We present data showing that the structure and function of the basal ganglia have been conserved throughout vertebrate evolution over some 560 million years. The interaction between the different nuclei within the basal ganglia is conserved as well as the cellular and synaptic properties and transmitters. We consider the role of the conserved basal ganglia circuitry for basic patterns of motor behaviour controlled via brainstem circuits. The output of the basal ganglia consists of tonically active GABAergic neurones, which target brainstem motor centres responsible for different patterns of behaviour, such as eye and locomotor movements, posture, and feeding. A prerequisite for activating or releasing a motor programme is that this GABAergic inhibition is temporarily reduced. This can be achieved through activation of GABAergic projection neurons from striatum, the input level of the basal ganglia, given an appropriate synaptic drive from cortex, thalamus and the dopamine system. The tonic inhibition of the motor centres at rest most likely serves to prevent the different motor programmes from becoming active when not intended. Striatal projection neurones are subdivided into one group with dopamine 1 receptors that provides increased excitability of the direct pathway that can initiate movements, while inhibitory dopamine 2 receptors are expressed on neurones that instead inhibit movements and are part of the 'indirect loop' in mammals as well as lamprey. We review the evidence showing that all basic features of the basal ganglia have been conserved throughout vertebrate phylogeny, and discuss these findings in relation to the role of the basal ganglia in selection of behaviour.

Evolutionarily conserved differences in pallial and thalamic short-term synaptic plasticity in striatum.

The striatum of the basal ganglia is conserved throughout the vertebrate phylum. Tracing studies in lamprey have shown that its afferent inputs are organized in a manner similar to that of mammals. The main inputs arise from the thalamus (Th) and lateral pallium (LPal; the homologue of cortex) that represents the two principal excitatory glutamatergic inputs in mammals. The aim here was to characterize the pharmacology and synaptic dynamics of afferent fibres from the LPal and Th onto identified striatal neurons to understand the processing taking place in the lamprey striatum. We used whole-cell current-clamp recordings in acute slices of striatum with preserved fibres from the Th and LPal, as well as tract tracing and immunohistochemistry. We show that the Th and LPal produce monosynaptic excitatory glutamatergic input through NMDA and AMPA receptors. The synaptic input from the LPal displayed short-term facilitation, unlike the Th input that instead displayed strong short-term synaptic depression. There was also an activity-dependent recruitment of intrastriatal oligosynaptic inhibition from both inputs. These results indicate that the two principal inputs undergo different activity-dependent short-term synaptic plasticity in the lamprey striatum. The difference observed between Th and LPal (cortical) input is also observed in mammals, suggesting a conserved trait throughout vertebrate evolution.

The dopamine D2 receptor gene in lamprey, its expression in the striatum and cellular effects of D2 receptor activation.

All basal ganglia subnuclei have recently been identified in lampreys, the phylogenetically oldest group of vertebrates. Furthermore, the interconnectivity of these nuclei is similar to mammals and tyrosine hydroxylase-positive (dopaminergic) fibers have been detected within the input layer, the striatum. Striatal processing is critically dependent on the interplay with the dopamine system, and we explore here whether D2 receptors are expressed in the lamprey striatum and their potential role. We have identified a cDNA encoding the dopamine D2 receptor from the lamprey brain and the deduced protein sequence showed close phylogenetic relationship with other vertebrate D2 receptors, and an almost 100% identity within the transmembrane domains containing the amino acids essential for dopamine binding. There was a strong and distinct expression of D2 receptor mRNA in a subpopulation of striatal neurons, and in the same region tyrosine hydroxylase-immunoreactive synaptic terminals were identified at the ultrastructural level. The synaptic incidence of tyrosine hydroxylase-immunoreactive boutons was highest in a region ventrolateral to the compact layer of striatal neurons, a region where most striatal dendrites arborise. Application of a D2 receptor agonist modulates striatal neurons by causing a reduced spike discharge and a diminished post-inhibitory rebound. We conclude that the D2 receptor gene had already evolved in the earliest group of vertebrates, cyclostomes, when they diverged from the main vertebrate line of evolution (560 mya), and that it is expressed in striatum where it exerts similar cellular effects to that in other vertebrates. These results together with our previous published data (Stephenson-Jones et al. 2011, 2012) further emphasize the high degree of conservation of the basal ganglia, also with regard to the indirect loop, and its role as a basic mechanism for action selection in all vertebrates.

Evolution of the basal ganglia: dual-output pathways conserved throughout vertebrate phylogeny.

The basal ganglia, including the striatum, globus pallidus interna and externa (GPe), subthalamic nucleus (STN), and substantia nigra pars compacta, are conserved throughout vertebrate phylogeny and have been suggested to form a common vertebrate mechanism for action selection. In mammals, this circuitry is further elaborated by the presence of a dual-output nucleus, the substantia nigra pars reticulata (SNr), and the presence of modulatory input from the cholinergic pedunculopontine nucleus (PPN). We sought to determine whether these additional components of the mammalian basal ganglia are also present in one of the phylogenetically oldest vertebrates, the lamprey. We show, by using immunohistochemistry, tract tracing, and whole-cell recordings, that homologs of the SNr and PPN are present in the lamprey. Thus the SNr receives direct projections from inwardly rectifying γ-aminobutyric acid (GABA)-ergic striatal neurons expressing substance P, but it is also influenced by indirect basal ganglia projections from the STN and potentially the GPe. Moreover, GABAergic SNr projection neurons are tonically active and project to the thalamus and brainstem motor areas. The homolog of the PPN contains both cholinergic and GABAergic neurons and is connected with all the nuclei of the basal ganglia, supporting its proposed role as part of an extended basal ganglia. A separate group of cholinergic neurons dorsal to the PPN corresponds to the descending mesencephalic locomotor region. Our results suggest that dual-output nuclei are part of the ancestral basal ganglia and that the PPN appears to have coevolved as part of a mechanism for action selection common to all vertebrates.

Evolutionary conservation of the habenular nuclei and their circuitry controlling the dopamine and 5-hydroxytryptophan (5-HT) systems.

The medial (MHb) and lateral (LHb) habenulae are a small group of nuclei that regulate the activity of monoaminergic neurons. Disruptions to these nuclei lead to deficits in a range of cognitive and motor functions from sleep to decision making. Interestingly, the habenular nuclei are present in all vertebrates, suggesting that they provide a common neural mechanism to influence these diverse functions. To unravel conserved habenula circuitry and approach an understanding of their basic function, we investigated the organization of these nuclei in the lamprey, one of the phylogenetically oldest vertebrates. Based on connectivity and molecular expression, we show that the MHb and LHb circuitry is conserved in the lamprey. As in mammals, separate populations of neurons in the LHb homolog project directly or indirectly to dopamine and serotonin neurons through a nucleus homologous to the GABAergic rostromedial mesopontine tegmental nucleus and directly to histamine neurons. The pallidal and hypothalamic inputs to the LHb homolog are also conserved. In contrast to other species, the habenula projecting pallidal nucleus is topographically distinct from the dorsal pallidum, the homolog of the globus pallidus interna. The efferents of the MHb homolog selectively target the interpeduncular nucleus. The MHb afferents arise from sensory (medial olfactory bulb, parapineal, and pretectum) and not limbic areas, as they do in mammals; consequently, the "context" in which this circuitry is recruited may have changed during evolution. Our results indicate that the habenular nuclei provide a common vertebrate circuitry to adapt behavior in response to rewards, stress, and other motivating factors.

Evolutionary conservation of the basal ganglia as a common vertebrate mechanism for action selection.

BACKGROUND: Although the basal ganglia are thought to play a key role in action selection in mammals, it is unknown whether this mammalian circuitry is present in lower vertebrates as a conserved selection mechanism. We aim here, using lamprey, to elucidate the basal ganglia circuitry in the phylogenetically oldest group of vertebrates (cyclostomes) and determine how this selection architecture evolved to accommodate the increased behavioral repertoires of advanced vertebrates. RESULTS: We show, using immunohistochemistry, tract tracing, and whole-cell recordings, that all parts of the mammalian basal ganglia (striatum, globus pallidus interna [GPi] and externa [GPe], and subthalamic nucleus [STN]) are present in the lamprey forebrain. In addition, the circuit features, molecular markers, and physiological activity patterns are conserved. Thus, GABAergic striatal neurons expressing substance P project directly to the pallidal output layer, whereas enkephalin-expressing striatal neurons project indirectly via nuclei homologous to the GPe and STN. Moreover, pallidal output neurons tonically inhibit tectum, mesencephalic, and diencephalic motor regions. CONCLUSIONS: These results show that the detailed basal ganglia circuitry is present in the phylogenetically oldest vertebrates and has been conserved, most likely as a mechanism for action selection used by all vertebrates, for over 560 million years. Our data also suggest that the mammalian basal ganglia evolved through a process of exaptation, where the ancestral core unit has been co-opted for multiple functions, allowing them to process cognitive, emotional, and motor information in parallel and control a broader range of behaviors.