Multiple neuronal circuits for variable object-action choices based on short- and long-term memories.

At each time in our life, we choose one or few behaviors, while suppressing many other behaviors. This is the basic mechanism in the basal ganglia, which is done by tonic inhibition and selective disinhibition. Dysfunctions of the basal ganglia then cause 2 types of disorders (difficulty in initiating necessary actions and difficulty in suppressing unnecessary actions) that occur in Parkinson's disease. The basal ganglia generate such opposite outcomes through parallel circuits: The direct pathway for initiation and indirect pathway for suppression. Importantly, the direct pathway processes good information and the indirect pathway processes bad information, which enables the choice of good behavior and the rejection of bad behavior. This is mainly enabled by dopaminergic inputs to these circuits. However, the value judgment is complex because the world is complex. Sometimes, the value must be based on recent events, thus is based on short-term memories. Or, the value must be based on historical events, thus is based on long-term memories. Such memory-based value judgment is generated by another parallel circuit originating from the caudate head and caudate tail. These circuit-information mechanisms allow other brain areas (e.g., prefrontal cortex) to contribute to decisions by sending information to these basal ganglia circuits. Moreover, the basal ganglia mechanisms (i.e., what to choose) are associated with cerebellum mechanisms (i.e., when to choose). Overall, multiple levels of parallel circuits in and around the basal ganglia are essential for coordinated behaviors. Understanding these circuits is useful for creating clinical treatments of disorders resulting from the failure of these circuits.

Neuronal connections of direct and indirect pathways for stable value memory in caudal basal ganglia.

Direct and indirect pathways in the basal ganglia work together for controlling behavior. However, it is still a controversial topic whether these pathways are segregated or merged with each other. To address this issue, we studied the connections of these two pathways in the caudal parts of the basal ganglia of rhesus monkeys using anatomical tracers. Our previous studies showed that the caudal basal ganglia control saccades by conveying long-term values (stable values) of many visual objects toward the superior colliculus. In experiment 1, we injected a tracer in the caudate tail (CDt), and found local dense plexuses of axon terminals in the caudal-dorsal-lateral part of substantia nigra pars reticulata (cdlSNr) and the caudal-ventral part of globus pallidus externus (cvGPe). These anterograde projections may correspond to the direct and indirect pathways, respectively. To verify this in experiment 2, we injected different tracers into cdlSNr and cvGPe, and found many retrogradely labeled neurons in CDt and, in addition, the caudal-ventral part of the putamen (cvPut). These cdlSNr-projecting and cvGPe-projecting neurons were found intermingled in both CDt and cvPut (which we call "striatum tail"). A small but significant proportion of neurons (<15%) were double-labeled, indicating that they projected to both cdlSNr and cvGPe. These anatomical results suggest that stable value signals (good vs. bad) are sent from the striatum tail to cdlSNr and cvGPe in a biased (but not exclusive) manner. These connections may play an important role in biasing saccades toward higher valued objects and away from lower valued objects.

Direct and indirect pathways for choosing objects and actions.

A prominent target of the basal ganglia is the superior colliculus (SC) which controls gaze orientation (saccadic eye movement in primates) to an important object. This 'object choice' is crucial for choosing an action on the object. SC is innervated by the substantia nigra pars reticulata (SNr) which is controlled mainly by the caudate nucleus (CD). This CD-SNr-SC circuit is sensitive to the values of individual objects and facilitates saccades to good objects. The object values are processed differently in two parallel circuits: flexibly by the caudate head (CDh) and stably by the caudate tail (CDt). To choose good objects, we need to reject bad objects. In fact, these contrasting functions are accomplished by the circuit originating from CDt: The direct pathway focuses on good objects and facilitates saccades to them; the indirect pathway focuses on bad objects and suppresses saccades to them. Inactivation of CDt deteriorated the object choice, because saccades to bad objects were no longer suppressed. This suggests that the indirect pathway is important for object choice. However, the direct and indirect pathways for 'object choice', which aim at the same action (i.e., saccade), may not work for 'action choice'. One possibility is that circuits controlling different actions are connected through the indirect pathway. Additional connections of the indirect pathway with brain areas outside the basal ganglia may also provide a wider range of behavioral choice. In conclusion, basal ganglia circuits are composed of the basic direct/indirect pathways and additional connections and thus have acquired multiple functions.

Complex social ecology needs complex machineries of foraging.

Uncertainty is caused not only by environmental changes, but also by social interference resulting from competition over food resources. Actually, foraging effort is socially facilitated, which, however, does not require incentive control by the dopamine system; Zajonc's "drive" theory is thus questionable. Instead, social adjustments may be pre-embedded in the limbic network responsible for decisions of appropriate effort-cost investment.

Parallel basal ganglia circuits for decision making.

The basal ganglia control body movements, mainly, based on their values. Critical for this mechanism is dopamine neurons, which sends unpredicted value signals, mainly, to the striatum. This mechanism enables animals to change their behaviors flexibly, eventually choosing a valuable behavior. However, this may not be the best behavior, because the flexible choice is focused on recent, and, therefore, limited, experiences (i.e., short-term memories). Our old and recent studies suggest that the basal ganglia contain separate circuits that process value signals in a completely different manner. They are insensitive to recent changes in value, yet gradually accumulate the value of each behavior (i.e., movement or object choice). These stable circuits eventually encode values of many behaviors and then retain the value signals for a long time (i.e., long-term memories). They are innervated by a separate group of dopamine neurons that retain value signals, even when no reward is predicted. Importantly, the stable circuits can control motor behaviors (e.g., hand or eye) quickly and precisely, which allows animals to automatically acquire valuable outcomes based on historical life experiences. These behaviors would be called 'skills', which are crucial for survival. The stable circuits are localized in the posterior part of the basal ganglia, separately from the flexible circuits located in the anterior part. To summarize, the flexible and stable circuits in the basal ganglia, working together but independently, enable animals (and humans) to reach valuable goals in various contexts.

Neuronal response of the primate striatum tail to face of socially familiar persons.

Recent studies have suggested that the basal ganglia, the center of stimulus-reward associative learning, are involved in social behavior. However, the role of the basal ganglia in social information processing remains unclear. Here, we demonstrate that the striatum tail (STRt) in macaque monkeys, which is sensitive to visual objects with long-term reward history (i.e., stable object value), is also sensitive to socially familiar persons. Many STRt neurons responded to face images of persons, especially those who took daily care of the subject monkeys. These face-responsive neurons also encoded stable object value. The strength of the neuronal modulation of social familiarity and stable object value biases were positively correlated. These results suggest that both social familiarity and stable object value information are mediated by a common neuronal mechanism. Thus, the representation of social information is linked to reward information in the STRt, not in the dedicated social information circuit.

Neuronal mechanism of the encoding of socially familiar faces in the striatum tail.

Although we can quickly locate a familiar person even in a crowd, the underlying neuronal mechanism remains unclear. Recently, we found that the striatum tail (STRt), which is part of the basal ganglia, is sensitive to long-term reward history. Here, we show that long-term value-coding neurons are involved in the detection of socially familiar faces. Many STRt neurons respond to facial images, especially to those of socially familiar persons. Additionally, we found that these face-responsive neurons also encode the stable values of many objects based on long-term reward experiences. Interestingly, the strength of neuronal modulation of social familiarity bias (familiar or unfamiliar) and object value bias (high-valued or low-valued) were positively correlated. These results suggest that both social familiarity and stable object-value information are mediated by a common neuronal mechanism. This mechanism may contribute to the rapid detection of familiar faces in real-world contexts.

Social influences of competition on impulsive choices in domestic chicks.

Social factors involved in the control of impulsiveness were examined in domestic chicks. In binary choices between a large/long-delay option (LL) and a small/short-delay alternative (SS), chicks that had been competitively trained in groups of three individuals showed fewer choices of LL than did those trained in isolation (experiment 1), suggesting that competition causes impulsive choice. In experiment 2, in order to identify the critical factor involved, we tested the effects of perceived competition (coincident feeding without interruption) and scrounging (gaining food without pecking bead) separately. To examine the effects of risk/noise that individual chicks experienced in competition, the food amount varied randomly in trials according to a binomial distribution around the expected mean. Perceived competition primarily contributed to the influence on the impulsive choice, whereas the contribution of scrounging was weaker. Collection risk did not explain the social influences since the perceived competition was not accompanied by actual interruption of the delayed food reward. The risk owing to variable food per se did not cause impulsive choices. Coincident foraging during competition is thought to play a critical role.

Optogenetic manipulation of a value-coding pathway from the primate caudate tail facilitates saccadic gaze shift.

In the primate basal ganglia, the caudate tail (CDt) encodes the historical values (good or bad) of visual objects (i.e., stable values), and electrical stimulation of CDt evokes saccadic eye movements. However, it is still unknown how output from CDt conveys stable value signals to govern behavior. Here, we apply a pathway-selective optogenetic manipulation to elucidate how such value information modulates saccades. We express channelrhodopsin-2 in CDt delivered by viral vector injections. Selective optical activation of CDt-derived terminals in the substantia nigra pars reticulata (SNr) inhibits SNr neurons. Notably, these SNr neurons show inhibitory responses to good objects. Furthermore, the optical stimulation causes prolonged excitation of visual-saccadic neurons in the superior colliculus (SC), and induces contralateral saccades. These SC neurons respond more strongly to good than to bad objects in the contralateral hemifield. The present results demonstrate that CDt facilitates saccades toward good objects by serial inhibitory pathways through SNr.

An Open Resource for Non-human Primate Optogenetics.

Optogenetics has revolutionized neuroscience in small laboratory animals, but its effect on animal models more closely related to humans, such as non-human primates (NHPs), has been mixed. To make evidence-based decisions in primate optogenetics, the scientific community would benefit from a centralized database listing all attempts, successful and unsuccessful, of using optogenetics in the primate brain. We contacted members of the community to ask for their contributions to an open science initiative. As of this writing, 45 laboratories around the world contributed more than 1,000 injection experiments, including precise details regarding their methods and outcomes. Of those entries, more than half had not been published. The resource is free for everyone to consult and contribute to on the Open Science Framework website. Here we review some of the insights from this initial release of the database and discuss methodological considerations to improve the success of optogenetic experiments in NHPs.

Indirect pathway from caudate tail mediates rejection of bad objects in periphery.

The essential everyday task of making appropriate choices is a process controlled mainly by the basal ganglia. To this end, subjects need not only to find "good" objects in their environment but also to reject "bad" objects. To reveal this rejection mechanism, we created a sequential saccade choice task for monkeys and studied the role of the indirect pathway from the CDt (tail of the caudate nucleus) mediated by cvGPe (caudal-ventral globus pallidus externus). Neurons in cvGPe were typically inhibited by the appearance of bad objects; however, this inhibition was reduced on trials when the monkeys made undesired saccades to the bad objects. Moreover, disrupting the inhibitory influence of CDt on cvGPe by local injection of bicuculline (GABAA receptor antagonist) impaired the monkeys' ability to suppress saccades to bad objects. Thus, the indirect pathway mediates the rejection of bad choices, a crucial component of goal-directed behavior.

Visual Neurons in the Superior Colliculus Discriminate Many Objects by Their Historical Values.

The superior colliculus (SC) is an important structure in the mammalian brain that orients the animal toward distinct visual events. Visually responsive neurons in SC are modulated by visual object features, including size, motion, and color. However, it remains unclear whether SC activity is modulated by non-visual object features, such as the reward value associated with the object. To address this question, three monkeys were trained (>10 days) to saccade to multiple fractal objects, half of which were consistently associated with large rewards while other half were associated with small rewards. This created historically high-valued ('good') and low-valued ('bad') objects. During the neuronal recordings from the SC, the monkeys maintained fixation at the center while the objects were flashed in the receptive field of the neuron without any reward. We found that approximately half of the visual neurons responded more strongly to the good than bad objects. In some neurons, this value-coding remained intact for a long time (>1 year) after the last object-reward association learning. Notably, the neuronal discrimination of reward values started about 100 ms after the appearance of visual objects and lasted for more than 100 ms. These results provide evidence that SC neurons can discriminate objects by their historical (long-term) values. This object value information may be provided by the basal ganglia, especially the circuit originating from the tail of the caudate nucleus. The information may be used by the neural circuits inside SC for motor (saccade) output or may be sent to the circuits outside SC for future behavior.

Indirect Pathway of Caudal Basal Ganglia for Rejection of Valueless Visual Objects.

The striatum controls behavior in two ways: facilitation and suppression through the direct and indirect pathways, respectively. However, it is still unclear what information is processed in these pathways. To address this question, we studied two pathways originating from the primate caudate tail (CDt). We found that the CDt innervated the caudal-dorsal-lateral part of the substantia nigra pars reticulata (cdlSNr), directly or indirectly through the caudal-ventral part of the globus pallidus externus (cvGPe). Notably, cvGPe neurons receiving inputs from the CDt were mostly visual neurons that encoded stable reward values of visual objects based on long-past experiences. Their dominant response was inhibition by valueless objects, which generated disinhibition of cdlSNr neurons and inhibition of superior colliculus neurons. Our data suggest that low-value signals are sent by the CDt-indirect pathway to suppress saccades to valueless objects, whereas high-value signals are sent by the CDt-direct pathway to facilitate saccades to valuable objects.

Neuronal codes for the inhibitory control of impulsive actions in the rat infralimbic cortex.

Poor impulse control is a debilitating condition observed in various psychiatric disorders and could be a risk factor for drug addiction, criminal involvement, and suicide. The rat infralimbic cortex (IL), located in the ventral portion of the medial prefrontal cortex, has been implicated in impulse control. To elucidate the neurophysiological basis of impulse control, we recorded single unit activity in the IL of a rat performing a 3-choiceserial reaction time task (3-CSRTT) and 2-choice task (2-CT), which are animal models for impulsivity. The inactivation of IL neuronal activity with an injection of muscimol (0.1 µg /side) disrupted impulse control in the 3-CSRTT. More than 60% (38/56) of isolated IL units were linked to impulse control, while approximately 30% of all units were linked to attentional function in the 3-CSRTT. To avoid confounding motor-related units with the impulse control-related units, we further conducted the 2-CT in which the animals' motor activities were restricted during recording window. More than 30% (14/44) of recorded IL units were linked to impulse control in the 2-CT. Several types of impulse control-related units were identified. Only 16% of all units were compatible with the results of the muscimol experiment, which showed a transient decline in the firing rate immediately before the release of behavioral inhibition. This is the first study to elucidate the neurophysiological basis of impulse control in the IL and to propose that IL neurons control impulsive actions in a more complex manner than previously considered.

Competitor suppresses neuronal representation of food reward in the nucleus accumbens/medial striatum of domestic chicks.

To investigate the role of social contexts in controlling the neuronal representation of food reward, we recorded single neuron activity in the medial striatum/nucleus accumbens of domestic chicks and examined whether activities differed between two blocks with different contexts. Chicks were trained in an operant task to associate light-emitting diode color cues with three trial types that differed in the type of food reward: no reward (S-), a small reward/short-delay option (SS), and a large reward/long-delay alternative (LL). Amount and duration of reward were set such that both of SS and LL were chosen roughly equally. Neurons showing distinct cue-period activity in rewarding trials (SS and LL) were identified during an isolation block, and activity patterns were compared with those recorded from the same neuron during a subsequent pseudo-competition block in which another chick was allowed to forage in the same area, but was separated by a transparent window. In some neurons, cue-period activity was lower in the pseudo-competition block, and the difference was not ascribed to the number of repeated trials. Comparison at neuronal population level revealed statistically significant suppression in the pseudo-competition block in both SS and LL trials, suggesting that perceived competition generally suppressed the representation of cue-associated food reward. The delay- and reward-period activities, however, did not significantly different between blocks. These results demonstrate that visual perception of a competitive forager per se weakens the neuronal representation of predicted food reward. Possible functional links to impulse control are discussed.

Behavioural and pharmacological effects of fluvoxamine on decision-making in food patches and the inter-temporal choices of domestic chicks.

Behavioural effects of fluvoxamine (FLV, selective serotonin reuptake inhibitor) were examined in 1-2 week old domestic chicks. Chicks were tested in an I-shaped maze equipped with a feeder (ON feeder) that served 1 or 2 grains of millet at gradually increasing intervals, so that a depleting food patch was mimicked. By leaving the feeder, the food delivery program was reset, and chicks gained food at short intervals only after a travel to a dummy feeder (OFF feeder) placed on the opposite side of the maze. Chicks quickly learned to actively shuttle between the ON and the OFF feeders. FLV (intra-peritoneal injection, 20 mg/kg BW) acutely caused chicks to stay longer at the gradually depleting ON feeder. Inter-temporal choices were also tested, whereby two coloured beads were simultaneously presented, each associated with a small/short-delay reward or a large/long-delay alternative. FLV suppressed the choice of the short-delay option. It is suggested that an enhanced level of serotonin (5-HT) makes chicks more tolerant of the delayed food item in both behavioural paradigms. Furthermore, the decision to leave a depleting patch cannot be equated to choosing the long-delay option of the choice paradigm. Furthermore, FLV suppressed work efforts (velocity and running distance) in uncued shuttle and number of distress calls. In vivo microdialysis experiments revealed that FLV enhanced the extracellular concentration of 5-HT as well as dopamine (DA) locally in the medial striatum/nucleus accumbens. Underlying neuromodulatory mechanisms of behavioural control are examined in relation to locomotion, behavioural tolerance and interval timing.

Instantaneous and cumulative influences of competition on impulsive choices in domestic chicks.

This study examined instantaneous and cumulative effects of competitive interactions on impulsiveness in the inter-temporal choices in domestic chicks. Chicks were trained to peck colored beads to gain delayed food rewards (1 or 6 grains of millet delivered after a delay ranging between 0 and 4.5 s), and were tested in binary choices between a small-short delay option (SS) and a large-long delay alternative (LL). To examine whether competitive foraging instantaneously changes impulsiveness, we intraindividually compared choices between two consecutive tests in different contexts, one with competitors and another without. We found that (1) the number of the choice of LL was not influenced by competition in the tests, but (2) the operant peck latency was shortened by competition, suggesting a socially enhanced incentive for food. To further examine the lasting changes, two groups of chicks were consecutively trained and tested daily for 2 weeks according to a "behavioral titration" procedure, one with competitors and another without. Inter-group comparisons of the choices revealed that (3) choice impulsiveness gradually decreased along development, while (4) the chicks trained in competition maintained a higher level of impulsiveness. These results suggest that competitive foraging causes impulsive choices not by direct/contextual modification. Causal link between the instantaneous enhancement of incentive and the gradual effects on impulsiveness remains to be examined. Some (yet unspecified) factors may be indirectly involved.