A Unified Framework for Dopamine Signals across Timescales.

Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.

Mice with reduced glutamate transporter GLT1 expression exhibit behaviors related to attention-deficit/hyperactivity disorder.

Attention-deficit/hyperactivity disorder (ADHD) is a common neuropsychiatric disorder in children. Although animal models and human brain imaging studies indicate a significant role for glutamatergic dysfunction in ADHD, there is no direct evidence that glutamatergic dysfunction is sufficient to induce ADHD-like symptoms. The glial glutamate transporter GLT1 plays a critical role in glutamatergic neurotransmission. We report here the generation of mice expressing only 20% of normal levels of the GLT1. Unlike conventional GLT1 knockout mice, these mice survive to adulthood and exhibit ADHD-like phenotypes, including hyperactivity, impulsivity and impaired memory. These findings indicate that glutamatergic dysfunction due to GLT1 deficiency, a mechanism distinct from the dopaminergic deficit hypothesis of ADHD, underlies ADHD-like symptoms.

Ventrolateral Striatal Medium Spiny Neurons Positively Regulate Food-Incentive, Goal-Directed Behavior Independently of D1 and D2 Selectivity.

The ventral striatum is involved in motivated behavior. Akin to the dorsal striatum, the ventral striatum contains two parallel pathways: the striatomesencephalic pathway consisting of dopamine receptor Type 1-expressing medium spiny neurons (D1-MSNs) and the striatopallidal pathway consisting of D2-MSNs. These two genetically identified pathways are thought to encode opposing functions in motivated behavior. It has also been reported that D1/D2 genetic selectivity is not attributed to the anatomical discrimination of two pathways. We wanted to determine whether D1- and D2-MSNs in the ventral striatum functioned in an opposing manner as previous observations claimed, and whether D1/D2 selectivity corresponded to a functional segregation in motivated behavior of mice. To address this question, we focused on the lateral portion of ventral striatum as a region implicated in food-incentive, goal-directed behavior, and recorded D1 or D2-MSN activity by using a gene-encoded ratiometric Ca2+ indicator and by constructing a fiberphotometry system, and manipulated their activities via optogenetic inhibition during ongoing behaviors. We observed concurrent event-related compound Ca2+ elevations in ventrolateral D1- and D2-MSNs, especially at trial start cue-related and first lever press-related times. D1 or D2 selective optogenetic inhibition just after the trial start cue resulted in a reduction of goal-directed behavior, indicating a shared coding of motivated behavior by both populations at this time. Only D1-selective inhibition just after the first lever press resulted in the reduction of behavior, indicating D1-MSN-specific coding at that specific time. Our data did not support opposing encoding by both populations in food-incentive, goal-directed behavior.SIGNIFICANCE STATEMENT An opposing role of dopamine receptor Type 1 or Type 2-expressing medium spiny neurons (D1-MSNs or D2-MSNs) on striatum-mediated behaviors has been widely accepted. However, this idea has been questioned by recent reports. In the present study, we measured concurrent Ca2+ activity patterns of D1- and D2-MSNs in the ventrolateral striatum during food-incentive, goal-directed behavior in mice. According to Ca2+ activity patterns, we conducted timing-specific optogenetic inhibition of each type of MSN. We demonstrated that both D1- and D2-MSNs in the ventrolateral striatum commonly and positively encoded action initiation, whereas only D1-MSNs positively encoded sustained motivated behavior. These findings led us to reconsider the prevailing notion of a functional segregation of MSN activity in the ventral striatum.

A New Paradigm for Evaluating Avoidance/Escape Motivation.

Background: Organisms have evolved to approach pleasurable opportunities and to avoid or escape from aversive experiences. These 2 distinct motivations are referred to as approach and avoidance/escape motivations and are both considered vital for survival. Despite several recent advances in understanding the neurobiology of motivation, most studies addressed approach but not avoidance/escape motivation. Here we develop a new experimental paradigm to quantify avoidance/escape motivation and examine the pharmacological validity. Methods: We set up an avoidance variable ratio 5 task in which mice were required to press a lever for variable times to avoid an upcoming aversive stimulus (foot shock) or to escape the ongoing aversive event if they failed to avoid it. We i.p. injected ketamine (0, 1, or 5 mg/kg) or buspirone (0, 5, or 10 mg/kg) 20 or 30 minutes before the behavioral task to see if ketamine enhanced avoidance/escape behavior and buspirone diminished it as previously reported. Results: We found that the performance on the avoidance variable ratio 5 task was sensitive to the intensity of the aversive stimulus. Treatment with ketamine increased while that with buspirone decreased the probability of avoidance from an aversive stimulus in the variable ratio 5 task, being consistent with previous reports. Conclusion: Our new paradigm will prove useful for quantifying avoidance/escape motivation and will contribute to a more comprehensive understanding of motivation.

Milnacipran affects mouse impulsive, aggressive, and depressive-like behaviors in a distinct dose-dependent manner.

Serotonin/noradrenaline reuptake inhibitors (SNRIs) are widely used for the treatment for major depressive disorder, but these drugs induce several side effects including increased aggression and impulsivity, which are risk factors for substance abuse, criminal involvement, and suicide. To address this issue, milnacipran (0, 3, 10, or 30 mg/kg), an SNRI and antidepressant, was intraperitoneally administered to mice prior to the 3-choice serial reaction time task, resident-intruder test, and forced swimming test to measure impulsive, aggressive, and depressive-like behaviors, respectively. A milnacipran dose of 10 mg/kg suppressed all behaviors, which was accompanied by increased dopamine and serotonin levels in the medial prefrontal cortex (mPFC) but not in the nucleus accumbens (NAc). Although the most effective dose for depressive-like behavior was 30 mg/kg, the highest dose increased aggressive behavior and unaffected impulsive behavior. Increased dopamine levels in the NAc could be responsible for the effects. In addition, the mice basal impulsivity was negatively correlated with the latency to the first agonistic behavior. Thus, the optimal dose range of milnacipran is narrower than previously thought. Finding drugs that increase serotonin and dopamine levels in the mPFC without affecting dopamine levels in the NAc is a potential strategy for developing novel antidepressants.

Yokukansankachimpihange increased body weight but not food-incentive motivation in wild-type mice.

Yokukansankachimpihange (YKSCH), a traditional Japanese medicine, is widely used for the amelioration of the behavioral and psychological symptoms of dementia with digestive dysfunction. Regardless of its successful use for digestive dysfunction, the effect of YKSCH on body weight was unknown. Furthermore, if YKSCH increased body weight, it might increase motivation according to Kampo medicine theory. Therefore, we investigated whether YKSCH had the potential to increase body weight and enhance motivation in mice. To address this, C57BL/6J mice were used to evaluate the long-term effect of YKSCH on body weight and food-incentive motivation. As part of the evaluation, we optimized an operant test for use over the long-term. We found that feeding mice YKSCH-containing chow increased body weight, but did not increase their motivation to food reward. We propose that YKSCH may be a good treatment option for preventing decrease in body weight in patients with dementia.

Milnacipran remediates impulsive deficits in rats with lesions of the ventromedial prefrontal cortex.

BACKGROUND: Deficits in impulse control are often observed in psychiatric disorders in which abnormalities of the prefrontal cortex are observed, including attention-deficit/hyperactivity disorder and bipolar disorder. We recently found that milnacipran, a serotonin/noradrenaline reuptake inhibitor, could suppress impulsive action in normal rats. However, whether milnacipran could suppress elevated impulsive action in rats with lesions of the ventromedial prefrontal cortex, which is functionally comparable with the human prefrontal cortex, remains unknown. METHODS: Selective lesions of the ventromedial prefrontal cortex were made using quinolinic acid in rats previously trained on a 3-choice serial reaction time task. Sham rats received phosphate buffered saline. Following a period of recovery, milnacipran (0 or 10mg/kg/d × 14 days) was orally administered 60 minutes prior to testing on the 3-choice task. After 7 days of drug cessation, Western blotting, immunohistochemistry, electrophysiological analysis, and morphological analysis were conducted. RESULTS: Lesions of the ventromedial prefrontal cortex induced impulsive deficits, and repeated milnacipran ameliorated the impulsive deficit both during the dosing period and after the cessation of the drug. Repeated milnacipran remediated the protein levels of mature brain-derived neurotrophic factor and postsynaptic density-95, dendritic spine density, and excitatory currents in the few surviving neurons in the ventromedial prefrontal cortex of ventromedial prefrontal cortex-lesioned rats. CONCLUSIONS: The findings of this study suggest that milnacipran treatment could be a novel strategy for the treatment of psychiatric disorders that are associated with a lack of impulse control.

[Anti-impulsivity drugs and their mechanisms of action].

Higher impulsivity could be a risk factor for drug addiction, criminal involvement, and suicide. Moreover, poor inhibitory control is observed in several psychiatric disorders such as attention-deficit/hyperactivity disorder, schizophrenia, and bipolar disorder. Thus it is preferred that clinical drugs have anti-impulsive effects in addition to the therapeutic effects on the primary disease. At least it is better to use clinical drugs that do not increase impulsivity. We have developed a 3-choice serial reaction time task and examined the effects of clinical drugs on impulsivity in rats using the task. We have found several anti-impulsive drugs (lithium, tandospirone, and milnacipran) and elucidated the mechanism of action in some of these drugs. For example, we demonstrated that milnacipran enhanced the control of impulsive action by activating D1-like receptors in the infralimbic cortex. In this review, we introduce recent advances in this field and suggest future directions to develop anti-impulsive drugs.

Tandospirone suppresses impulsive action by possible blockade of the 5-HT1A receptor.

Higher impulsivity is observed in several psychiatric disorders and could be a risk factor for drug addiction, criminal involvement, and suicide. Although the involvement of the 5-HT1A receptor in impulsive behavior has been indicated, the effects of clinically relevant drugs have been rarely tested. In the present study, we examined whether (3aR,4S,7R,7aS)-rel-hexahydro-2-[4-[4-(2-pyrimidinyl)-1-piperazinyl]butyl]-4,7-methano-1H-isoindole-1,3(2H)-dione hydrochloride (tandospirone), an anxiolytic and a partial agonist of the 5-HT1A receptor, could affect impulsive action in the 3-choice serial reaction time task. Rats were acutely administered tandospirone (0, 0.1, and 1 mg/kg, i.p.). Tandospirone decreased the number of premature responses, an index of impulsive action, in a dose-dependent manner. N-[2-[4-(2-Methoxyphenyl)-1-piperazinyl]ethyl]-N-2-pyridinylcyclohexanecarboxamide maleate salt (WAY100635; 0.3 mg/kg, s.c.), a 5-HT1A receptor antagonist, did not reverse the suppressing effects of tandospirone on impulsive action. Moreover, a higher dose of WAY100635 (1 mg/kg, s.c.) suppressed impulsive action without tandospirone. Thus the effects of tandospirone on impulsivity might be due to the antagonistic action. Tandospirone could be a therapeutic candidate for impulsivity-related disorders.

Inhibition of the dorsomedial striatal direct pathway is essential for the execution of action sequences.

Contrary to the previous notion that the dorsomedial striatum (DMS) is crucial for acquiring new learning, accumulated evidence has suggested that the DMS also plays a role in the execution of already learned action sequences. Here, we examined how the direct and indirect pathways in the DMS regulate action sequences using a task that requires animals to press a lever consecutively. Cell-type-specific bulk Ca2+ recording revealed that the direct pathway was inhibited at the time of sequence execution. The sequence-related response was blunted in trials where the sequential behaviors were disrupted. Optogenetic activation at the sequence start caused distraction of action sequences without affecting motor function or memory of the task structure. By contrast with the direct pathway, the indirect pathway was slightly activated at the start of the sequence, but the optogenetic suppression of such sequence-related signaling did not impact the behaviors. These results suggest that the inhibition of the DMS direct pathway promotes sequence execution potentially by suppressing the formation of a new association.

A Hypothalamic Circuit Underlying the Dynamic Control of Social Homeostasis.

Social grouping increases survival in many species, including humans1,2. By contrast, social isolation generates an aversive state (loneliness) that motivates social seeking and heightens social interaction upon reunion3-5. The observed rebound in social interaction triggered by isolation suggests a homeostatic process underlying the control of social drive, similar to that observed for physiological needs such as hunger, thirst or sleep3,6. In this study, we assessed social responses in multiple mouse strains and identified the FVB/NJ line as exquisitely sensitive to social isolation. Using FVB/NJ mice, we uncovered two previously uncharacterized neuronal populations in the hypothalamic preoptic nucleus that are activated during social isolation and social rebound and that orchestrate the behavior display of social need and social satiety, respectively. We identified direct connectivity between these two populations of opposite function and with brain areas associated with social behavior, emotional state, reward, and physiological needs, and showed that animals require touch to assess the presence of others and fulfill their social need, thus revealing a brain-wide neural system underlying social homeostasis. These findings offer mechanistic insight into the nature and function of circuits controlling instinctive social need and for the understanding of healthy and diseased brain states associated with social context.

Impulsive behavior and nicotinic acetylcholine receptors.

Higher impulsivity is thought to be a risk factor for drug addiction, criminal involvement, and suicide. Excessive levels of impulsivity are often observed in several psychiatric disorders including attention-deficit/hyperactivity disorder and schizophrenia. Previous studies have demonstrated that nicotinic acetylcholine receptors (nAChRs) are involved in impulsive behavior. Here, we introduce recent advances in this field and describe the role of the following nAChR-related brain mechanisms in modulating impulsive behavior: dopamine release in the ventral striatum; α4ß2 nAChRs in the infralimbic cortex, which is a ventral part of the medial prefrontal cortex (mPFC); and dopamine release in the mPFC. We also suggest several potential therapeutic drugs to address these mechanisms in impulsivity-related disorders and explore future directions to further elucidate the roles of central nAChRs in impulsive behavior.

How does risk preference change under the stress of COVID-19? Evidence from Japan.

In this study, we investigated whether the risk preference systematically changed during the spread of COVID-19 in Japan. Traditionally, risk preference is assumed to be stable over one's life, though it differs among individuals. While recent studies have reported that it changes with a large event like natural disasters and financial crisis, they have not reached a consensus on its direction, risk aversion, or tolerance. We collected panel data of Japanese individuals in five waves from March to June 2020, which covered the period of the first cycle when COVID-19 spread rapidly and then dwindled. We measured risk preference through questions on the willingness to pay for insurance. The main results are as follows: First, people became more risk tolerant throughout the period; and second, people were more averse to mega risk than moderate risk, with the former correlating more strongly with the individual's perception of COVID-19. The first result may be interpreted as "habituation" to repeated stress, as is understood in neuroscience. Supplementary information: The online version contains supplementary material available at 10.1007/s11166-022-09374-z.

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction.

Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dynamics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward- and threat-related dopamine sub-systems.

Spatial inference without a cognitive map: the role of higher-order path integration.

The cognitive map has been taken as the standard model for how agents infer the most efficient route to a goal location. Alternatively, path integration - maintaining a homing vector during navigation - constitutes a primitive and presumably less-flexible strategy than cognitive mapping because path integration relies primarily on vestibular stimuli and pace counting. The historical debate as to whether complex spatial navigation is ruled by associative learning or cognitive map mechanisms has been challenged by experimental difficulties in successfully neutralizing path integration. To our knowledge, there are only three studies that have succeeded in resolving this issue, all showing clear evidence of novel route taking, a behaviour outside the scope of traditional associative learning accounts. Nevertheless, there is no mechanistic explanation as to how animals perform novel route taking. We propose here a new model of spatial learning that combines path integration with higher-order associative learning, and demonstrate how it can account for novel route taking without a cognitive map, thus resolving this long-standing debate. We show how our higher-order path integration (HOPI) model can explain spatial inferences, such as novel detours and shortcuts. Our analysis suggests that a phylogenetically ancient, vector-based navigational strategy utilizing associative processes is powerful enough to support complex spatial inferences.

Distinct temporal difference error signals in dopamine axons in three regions of the striatum in a decision-making task.

Different regions of the striatum regulate different types of behavior. However, how dopamine signals differ across striatal regions and how dopamine regulates different behaviors remain unclear. Here, we compared dopamine axon activity in the ventral, dorsomedial, and dorsolateral striatum, while mice performed a perceptual and value-based decision task. Surprisingly, dopamine axon activity was similar across all three areas. At a glance, the activity multiplexed different variables such as stimulus-associated values, confidence, and reward feedback at different phases of the task. Our modeling demonstrates, however, that these modulations can be inclusively explained by moment-by-moment changes in the expected reward, that is the temporal difference error. A major difference between areas was the overall activity level of reward responses: reward responses in dorsolateral striatum were positively shifted, lacking inhibitory responses to negative prediction errors. The differences in dopamine signals put specific constraints on the properties of behaviors controlled by dopamine in these regions.

Different roles of distinct serotonergic pathways in anxiety-like behavior, antidepressant-like, and anti-impulsive effects.

Serotonergic agents have been widely used for treatment of psychiatric disorders, but the therapeutic effects are insufficient and these drugs often induce severe side effects. We need to specify the distinct serotonergic pathways underlying each mental function to overcome these problems. Preclinical studies have demonstrated that the central serotonergic system is involved in several emotional/cognitive functions including anxiety, depression, and impulse control, but it remains unclear whether each function is regulated by a different serotonergic system. We used optogenetic strategy to increase central serotonergic activity in mice and evaluated the behavioral consequences. Pharmacological and genetic tools were used to determine the subtype of 5-HT receptors responsible for the observed effects. We demonstrated that the serotonergic activation in the median raphe nucleus enhanced anxiety-like behavior, the serotonergic activation in the dorsal raphe nucleus exerted antidepressant-like effects, and the serotonergic activation in the median or dorsal raphe nucleus suppressed impulsive action. We also found that different serotonergic terminals, ventral hippocampus, ventral tegmental area/substantia nigra, and subthalamic/parasubthalamic nucleus, are involved in regulating anxiety-like behavior, antidepressant-like, and anti-impulsive effects, respectively. Furthermore, we found, using triple-transgenic mice, that the stimulation of the 5-HT2C receptor is required to evoke anxiety-like behavior, but not to exert anti-impulsive effects. These results suggest the need for pathway-specific treatments and provide important insights that will help the development of more effective and safer therapeutics. This article is part of the special issue entitled 'Serotonin Research: Crossing Scales and Boundaries'.

Opposing Ventral Striatal Medium Spiny Neuron Activities Shaped by Striatal Parvalbumin-Expressing Interneurons during Goal-Directed Behaviors.

Medium spiny neurons (MSNs) of mice show opposing activities upon the initiation of a food-seeking lever press task. Ventromedial striatal (VMS)-MSNs are inhibited but ventrolateral striatal (VLS)-MSNs are activated; these activities mediate action selection and action initiation, respectively. To understand what input shapes the opposing MSN activities, here, we monitor cortical input activities at the cell population level and artificially reverse them. We demonstrate that the ventral hippocampus (vHP) and the insular cortex (IC) are major inputs to the VMS and VLS, both projections show silencing at the trial start time, and the vHP-VMS and IC-VLS pathways form functionally coupled input-output units during the task. Of note, the upstream IC silencing is converted to the downstream VLS-MSN activation. We find biased localization of striatal parvalbumin-expressing interneurons (PV INs) and verify PV IN-dependent feedforward architecture in the VLS. Our results reveal a distinct mode of cortico-striatal signal conveyance via feedforward disinhibition in behaving animals.

The effects of serotonin and/or noradrenaline reuptake inhibitors on impulsive-like action assessed by the three-choice serial reaction time task: a simple and valid model of impulsive action using rats.

Impulsivity is a pathological symptom in several psychiatric disorders, underscoring the need for animal models of impulsive action to develop a brief screening method for novel therapeutic agents of impulsive action. The aims of this study were (i) to evaluate whether the three-choice serial reaction time task (3-CSRTT), a simple version of the five-choice serial reaction time task (5-CSRTT), is appropriate for brief assessment of impulsive-like action and (ii) to examine the effects of fluvoxamine, a selective serotonin reuptake inhibitor, and milnacipran, a serotonin/noradrenaline reuptake inhibitor, on impulsive-like action using the 3-CSRTT. After training in the 3-CSRTT, rats were administered nicotine (0, 0.1, 0.2, and 0.4 mg/kg, salt, subcutaneously), atomoxetine [0, 0.01, 0.1, and, 1.0 mg/kg, intraperitoneally (i.p.)], fluvoxamine (0, 2, 4, and 8 mg/kg, i.p.), or milnacipran (0, 3, and 10 mg/kg, i.p.). The training time for the 3-CSRTT was significantly shorter than that for the 5-CSRTT. Nicotine increased, whereas atomoxetine decreased the number of premature responses, an index of impulsive-like action, which is consistent with earlier studies. Milnacipran, but not fluvoxamine, dose-dependently decreased premature responses. These results indicate that the 3-CSRTT could provide an appropriate and simpler rodent model of impulsive-like action and that milnacipran could have some beneficial effects on impulsivity-related disorders.

AppNL-G-F/NL-G-F mice overall do not show impaired motivation, but cored amyloid plaques in the striatum are inversely correlated with motivation.

Apathy is clinically defined as lack of motivation. Apathy is a frequent symptom in patients with Alzheimer's disease (AD). It is unclear whether amyloid ß (Aß) pathology is associated with apathy. To address this question, we employed the AppNL-G-F/NL-G-F mouse, an Aß deposition-bearing mouse without neurofibrillary tangles and neuronal cell death throughout the lifespan and used a progressive-ratio (PR) task to monitor instrumental motivation between the ages of 16 and 39 weeks. In the PR task, the number of lever presses to receive one reward increases and the number of active lever presses in the final trial a mouse completes represents a break point, which is an index of motivation. During the observation period, AppNL-G-F/NL-G-F mice overall did not show impaired motivation. However, AppNL-G-F/NL-G-F mice showed a dispersion of the break point at 39 weeks of age within the group. Therefore, we examined the association between the degree of the break point and Aß pathology; the number of cored amyloid plaques in the striatum was inversely correlated with the degree of motivation. Furthermore, we measured the dopamine transporter (DAT) levels in the subcortical tissues including the striatum using western blot analysis and showed that AppNL-G-F/NL-G-F mice have lower DAT levels than do C57BL/6J mice. Although we could not directly determine the effect of core amyloid plaques on the DAT, the results of this study suggest a pathway through which cored amyloid plaques damage the DAT and cause impaired motivation. These results will draw attention to cored amyloid plaques and will aid researchers searching for new strategies that are effective for the prevention and treatment of impaired motivation.