Frontal Norepinephrine Represents a Threat Prediction Error Under Uncertainty.

BACKGROUND: To adapt to threats in the environment, animals must predict them and engage in defensive behavior. While the representation of a prediction error signal for reward has been linked to dopamine, a neuromodulatory prediction error for aversive learning has not been identified. METHODS: We measured and manipulated norepinephrine release during threat learning using optogenetics and a novel fluorescent norepinephrine sensor. RESULTS: We found that norepinephrine response to conditioned stimuli reflects aversive memory strength. When delays between auditory stimuli and footshock are introduced, norepinephrine acts as a prediction error signal. However, temporal difference prediction errors do not fully explain norepinephrine dynamics. To explain noradrenergic signaling, we used an updated reinforcement learning model with uncertainty about time and found that it explained norepinephrine dynamics across learning and variations in temporal and auditory task structure. CONCLUSIONS: Norepinephrine thus combines cognitive and affective information into a predictive signal and links time with the anticipation of danger.

Effects of the atypical antipsychotic and D3/D2 dopamine partial agonist cariprazine on effort-based choice behavior: implications for modeling avolition.

RATIONALE: Cariprazine is an atypical antipsychotic that acts as a D3/D2 receptor partial agonist. In addition to treating positive symptoms of schizophrenia, cariprazine may have utility for treating negative symptoms. Rodent studies have focused on the effects of cariprazine on cognitive functions and behaviors thought to be related to anhedonia. Avolition, which is characterized by reduced initiation and persistence of goal-directed behavior, is another important negative symptom. OBJECTIVES: Effort-related choice tasks have been used as animal models of avolition. In these studies, cariprazine was assessed for its effects on effort-based choice in both rats and mice. Previous work has shown that D2 antagonists such as haloperidol and eticlopride produce a low-effort bias in rodents tested on effort-based choice tasks. RESULTS: Low doses of cariprazine produced a low-effort bias in rats tested on the fixed ratio 5/chow feeding choice task, decreasing lever pressing for high carbohydrate pellets but increasing chow intake. Cariprazine did not alter preference or intake of these foods in free-feeding tests. The effort-related effects of cariprazine were reversed by co-administration of the adenosine A2A antagonist istradefylline, and cariprazine failed to reverse the effort-related effects of the dopamine-depleting agent tetrabenazine. In mouse touchscreen choice tests, low doses of cariprazine also produced a low-effort bias, shifting behavior away from panel pressing. CONCLUSIONS: These results demonstrate that with these rodent models of avolition, cariprazine appears to act like a D2-family antagonist even at very low doses. Furthermore, the pharmacological regulation of avolition may differ from that of other negative symptoms.

Secondary motor cortex: Broadcasting and biasing animal's decisions through long-range circuits.

Medial secondary motor cortex (MOs or M2) constitutes the dorsal aspect of the rodent medial frontal cortex. We previously proposed that the function of MOs is to link antecedent conditions, including sensory stimuli and prior choices, to impending actions. In this review, we focus on the long-range pathways between MOs and other cortical and subcortical regions. We highlight three circuits: (1) connections with visual and auditory cortices that are essential for predictive coding of perceptual inputs; (2) connections with motor cortex and brainstem that are responsible for top-down, context-dependent modulation of movements; (3) connections with retrosplenial cortex, orbitofrontal cortex, and basal ganglia that facilitate reward-based learning. Together, these long-range circuits allow MOs to broadcast choice signals for feedback and to bias decision-making processes.

The novel atypical dopamine transport inhibitor CT-005404 has pro-motivational effects in neurochemical and inflammatory models of effort-based dysfunctions related to psychopathology.

Depressed individuals suffer from effort-related motivational symptoms such as anergia and fatigue, which are resistant to treatment with many common antidepressants. While drugs that block dopamine transport (DAT) reportedly have positive motivational effects, DAT inhibitors such as cocaine and amphetamines produce undesirable side effects. Thus, there is a need to develop and characterize novel atypical DAT inhibitors with unique and selective binding profiles. Rodent effort-based choice tasks provide useful models of motivational dysfunctions. With these tasks, animals choose between a high-effort instrumental action leading to highly valued reinforcement vs. a low effort/low reward option. The present studies focused on the initial characterization of a novel atypical DAT inhibitor, CT-005404, which binds to DAT with high selectivity relative to serotonin and norepinephrine transport, and produces long-term elevations of extracellular DA. CT-005404 was assessed for its ability to attenuate the effort-related motivational effects of the DA depleting agent tetrabenazine and the pro-inflammatory cytokine interleukin-1ß (IL-1ß) using a fixed ratio 5/chow feeding choice test. Tetrabenazine (1.0 mg/kg i.p.) shifted choice behavior, decreasing lever pressing and increasing chow intake. IL-1ß (4.0 µg/kg i.p.) also decreased lever pressing. CT-005404 was co-administered (7.5-30.0 mg/kg p.o.) with either tetrabenazine or IL-1ß, and the 15.0 and 30.0 mg/kg doses significantly reversed the effects of tetrabenazine and IL-1ß. CT-005404 administered alone produced a dose-related increase in lever pressing in rats tested on a progressive ratio/chow feeding choice task. Atypical DAT inhibitors such as CT-005404 offer potential as a new avenue for drug treatment of motivational dysfunctions in humans.

Behavioral and dopamine transporter binding properties of the modafinil analog (S, S)-CE-158: reversal of the motivational effects of tetrabenazine and enhancement of progressive ratio responding.

RATIONALE: Atypical dopamine (DA) transport blockers such as modafinil and its analogs may be useful for treating motivational symptoms of depression and other disorders. Previous research has shown that the DA depleting agent tetrabenazine can reliably induce motivational deficits in rats, as evidenced by a shift towards a low-effort bias in effort-based choice tasks. This is consistent with human studies showing that people with major depression show a bias towards low-effort activities. OBJECTIVES: Recent studies demonstrated that the atypical DA transport (DAT) inhibitor (S)-CE-123 reversed tetrabenazine-induced motivational deficits, increased progressive ratio (PROG) lever pressing, and increased extracellular DA in the nucleus accumbens. In the present studies, a recently synthesized modafinil analog, (S, S)-CE-158, was assessed in a series of neurochemical and behavioral studies in rats. RESULTS: (S, S)-CE-158 demonstrated the ability to reverse the effort-related effects of tetrabenazine and increase selection of high-effort PROG lever pressing in rats tested on PROG/chow feeding choice task. (S, S)-CE-158 showed a high selectivity for inhibiting DAT compared with other monoamine transporters, and systemic administration of (S, S)-CE-158 increased extracellular DA in the nucleus accumbens during the behaviorally active time course, which is consistent with the effects of (S)-CE-123 and other DAT inhibitors that enhance high-effort responding. CONCLUSIONS: These studies provide an initial neurochemical characterization of a novel atypical DAT inhibitor, and demonstrate that this compound is active in models of effort-related choice. This research could contribute to the development of novel compounds for the treatment of motivational dysfunctions in humans.

Effort-related decision making in humanized COMT mice: Effects of Val158Met polymorphisms and possible implications for negative symptoms in humans.

Catechol-o-methyltransferase (COMT) is an enzyme that metabolizes catecholamines, and is crucial for clearance of dopamine (DA) in prefrontal cortex. Val158Met polymorphism, which causes a valine (Val) to methionine (Met) substitution at codon 158, is reported to be associated with human psychopathologies in some studies. The Val/Val variant of the enzyme results in higher dopamine metabolism, which results in reduced dopamine transmission. Thus, it is important to investigate the relation between Val158Met polymorphisms using rodent models of psychiatric symptoms, including negative symptoms such as motivational dysfunction. In the present study, humanized COMT transgenic mice with two genotype groups (Val/Val (Val) and Met/Met (Met) homozygotes) and wild-type (WT) mice from the S129 background were tested using a touchscreen effort-based choice paradigm. Mice were trained to choose between delivery of a preferred liquid diet that reinforced panel pressing on various fixed ratio (FR) schedules (high-effort alternative), vs. intake of pellets concurrently available in the chamber (low-effort alternative). Panel pressing requirements were controlled by varying the FR levels (FR1, 2, 4, 8, 16) in ascending and descending sequences across weeks of testing. All mice were able to acquire the initial touchscreen operant training, and there was an inverse relationship between the number of reinforcers delivered by panel pressing and pellet intake across different FR levels. There was a significant group x FR level interaction in the ascending limb, with panel presses in the Val group being significantly lower than the WT group in FR1-8, and lower than Met in FR4. These findings indicate that the humanized Val allele in mice modulates FR/pellet-choice performance, as marked by lower levels of panel pressing in the Val group when the ratio requirement was moderately high. These studies may contribute to the understanding of the role of COMT polymorphisms in negative symptoms such as motivational dysfunctions in schizophrenic patients.

The dopamine depleting agent tetrabenazine alters effort-related decision making as assessed by mouse touchscreen procedures.

RATIONALE: Effort-based decision-making tasks allow animals to choose between preferred reinforcers that require high effort to obtain vs. low-effort/low reward options. Mesolimbic dopamine (DA) and related neural systems regulate effort-based choice. Tetrabenazine (TBZ) is a vesicular monoamine transport type-2 inhibitor that blocks DA storage and depletes DA. In humans, TBZ induces motivational dysfunction and depression. TBZ has been shown reliably to induce a low-effort bias in rats, but there are fewer mouse studies. OBJECTIVES: The present studies used touchscreen operant procedures (Bussey-Saksida chambers) to assess the effects of TBZ on effort-based choice in mice. METHODS: C57BL6 mice were trained to press an elevated lit panel on the touchscreen on a fixed ratio 1 schedule reinforced by strawberry milkshake, vs. approaching and consuming a concurrently available but less preferred food pellets (Bio-serv). RESULTS: TBZ (2.0-8.0 mg/kg IP) shifted choice, producing a dose-related decrease in panel pressing but an increase in pellet intake. In contrast, reinforcer devaluation by pre-feeding substantially decreased both panel pressing and pellet intake. In free-feeding choice tests, mice strongly preferred the milkshake vs. the pellets, and TBZ had no effect on milkshake intake or preference, indicating that the TBZ-induced low-effort bias was not due to changes in primary food motivation or preference. TBZ significantly decreased tissue levels of nucleus accumbens DA. CONCLUSION: The DA depleting agent TBZ induced an effort-related motivational dysfunction in mice, which may have clinical relevance for assessing novel drug targets for their potential use as therapeutic agents in patients with motivation impairments.

Lisdexamfetamine suppresses instrumental and consummatory behaviors supported by foods with varying degrees of palatability: Exploration of a binge-like eating model.

Diets high in sugar or fat are associated with multiple health conditions, including binge eating disorder (BED). BED affects approximately 2% of the US adult population, and occurs more frequently in females. It is important to develop animal models of palatable food consumption and food seeking that may have relevance for BED and other conditions associated with excessive food intake. The catecholamine uptake blocker and d-amphetamine prodrug lisdexamfetamine is used to treat BED. The present experiments studied the effect of lisdexamfetamine on food intake and food-reinforced effort-based choice in female Wistar rats. Three groups of rats received different food exposure conditions in the home cage randomly spread over several weeks: a chocolate exposure group (CE; brief access of chocolate and additional lab chow, n = 15), a lab chow exposure (LChE) group given additional access to lab chow (n = 8), and a third group given empty food dishes (n = 7). In tests of food intake under non-restricted conditions, lisdexamfetamine (0.1875-1.5 mg/kg IP) significantly reduced intake of both chocolate and chow in the CE group. In the LChE group, there was a trend towards reduced chow intake induced by lisdexamfetamine. All rats were trained on a Progressive Ratio/chow feeding choice task, in which they had a choice between working for high carbohydrate chocolate flavored pellets by lever pressing vs. approaching and consuming a concurrently available lab chow. The LChE group and the empty food dish group were combined to create one control group (n = 15). There was a significant overall dose-related suppressive effect of lisdexamfetamine on lever pressing but no group difference, and no dose x group interaction. Lisdexamfetamine significantly decreased chow intake in the CE group, but not in the control group. In conclusion, lisdexamfetamine affected both food intake and food-reinforced operant behavior, with larger effects seen in the group exposed to chocolate.

Pharmacological studies of effort-related decision making using mouse touchscreen procedures: effects of dopamine antagonism do not resemble reinforcer devaluation by removal of food restriction.

RATIONALE: Effort-based decision-making tasks offer animals choices between preferred reinforcers that require high effort to obtain vs. low effort/low reward options. The neural mechanisms of effort-based choice are widely studied in rats, and evidence indicates that mesolimbic dopamine (DA) and related neural systems play a key role. Fewer studies of effort-based choice have been performed in mice. OBJECTIVES: The present studies used touchscreen operant procedures (Bussey-Saksida boxes) to assess effort-based choice in mice. METHODS: CD1 mice were assessed on a concurrent fixed ratio 1 panel pressing/choice procedure. Mice were allowed to choose between rearing to press an elevated panel on the touchscreen for a preferred food (strawberry milkshake) vs. consuming a concurrently available less preferred alternative (high carbohydrate pellets). RESULTS: The DA D2 antagonist haloperidol (0.05-0.15 mg/kg IP) produced a dose-related decrease in panel pressing. Intake of food pellets was not reduced by haloperidol, and in fact, there was a significant quadratic trend, indicating a tendency for pellet intake to increase at low/moderate doses. In contrast, reinforcer devaluation by removing food restriction substantially decreased both panel pressing and pellet intake. In free-feeding choice tests, mice strongly preferred milkshake vs. pellets. Haloperidol did not affect food intake or preference. CONCLUSION: Haloperidol reduced the tendency to work for food, but this reduction was not due to decreases in primary food motivation or preference. Mouse touchscreen procedures demonstrate effects of haloperidol that are similar but not identical to those shown in rats. These rodent studies may be relevant for understanding motivational dysfunctions in humans.

The Novel Atypical Dopamine Uptake Inhibitor (S)-CE-123 Partially Reverses the Effort-Related Effects of the Dopamine Depleting Agent Tetrabenazine and Increases Progressive Ratio Responding.

Animal studies of effort-based choice behavior are being used to model effort-related motivational dysfunctions in humans. With these procedures, animals are offered a choice between high-effort instrumental actions leading to highly valued reinforcers vs. low effort/low reward options. Several previous studies have shown that dopamine (DA) uptake inhibitors, including GBR12909, lisdexamfetamine, methylphenidate, and PRX-14040, can reverse the effort-related effects of the vesicular monoamine transport blocker tetrabenazine, which inhibits DA storage. Because many drugs that block DA transport act as major stimulants that also release DA, and produce a number of undesirable side effects, there is a need to develop and characterize novel atypical DA transport inhibitors. (S)-CE-123 ((S)-5-((benzhydrylsulfinyl) methyl)thiazole) is a recently developed analog of modafinil with the biochemical characteristics of an atypical DA transport blocker. The present paper describes the enantioselective synthesis and initial chemical characterization of (S)-CE-123, as well as behavioral experiments involving effort-based choice and microdialysis studies of extracellular DA. Rats were assessed using the fixed ratio 5/chow feeding choice test. Tetrabenazine (1.0 mg/kg) shifted choice behavior, decreasing lever pressing and increasing chow intake. (S)-CE-123 was coadministered at doses ranging from 6.0 to 24.0 mg/kg, and the highest dose partially but significantly reversed the effects of tetrabenazine, although this dose had no effect on fixed ratio responding when administered alone. Additional experiments showed that (S)-CE-123 significantly increased lever pressing on a progressive ratio/chow feeding choice task and that the effective dose (24.0 mg/kg) increased extracellular DA in nucleus accumbens core. In summary, (S)-CE-123 has the behavioral and neurochemical profile of a compound that can block DA transport, reverse the effort-related effects of tetrabenazine, and increase selection of high-effort progressive ratio responding. This suggests that (S)-CE-123 or a similar compound could be useful as a treatment for effort-related motivational dysfunction in humans.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation.

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Effects of d-amphetamine on risk choice in rats depend on the manner in which the expected reward value is varied.

Decision making related to risks and rewards has been studied in the rodent by using several behavioral tests including the use of probability discounting tasks. However, it still remains unclear how long-term values and the different levels of risk are involved in decisions with reward uncertainty. In this study, we used a T-maze choice task to investigate the role of expected reward values in decision making under risk in the rat. The task was specifically set up to run with the expected value (EV) being equal to 1 between binary choice options (small-and-certain vs. large-but-risky rewards). The tests were carried out by providing three reward ratios that represented different levels of riskiness. Moreover, by varying the reward probabilities, the EV of the large-but-risky choice option that was set at either 0.5 or 2 was manipulated to determine whether choice could be influenced by the contrast between unequal EVs. Results demonstrated that the rats were able to distinguish different EVs because they clearly chose the option with a relatively larger EV (2 > 1 and 1 > 0.5). By way of contrast, risk-dependent choice appeared when the EVs were equal; in particular, risk-prone choice was made in a low-risk state, whereas risk-averse choice was made in a high-risk state. The systemic injection of d-amphetamine slightly increased the large-but-risky choices only under the high-risk condition in which the EVs were set to be equal; d-amphetamine did not affect the risk choices in the other two EVs set unequal. Overall, the present results implicate that the contrasts for unequal EVs and the different levels of risk are decision parameters critically involved in the rat's choice. And, relatively low doses of d-amphetamine did not have much of an effect on the present model of risk choice.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research.

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson's disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.

Dissociable contribution of nucleus accumbens and dorsolateral striatum to the acquisition of risk choice behavior in the rat.

While a growing body of research has suggested that the mesocorticolimbic dopamine systems play a key role in decision making under risk, how the nucleus accumbens (NAC) is involved in the acquisition of risk choice behavior remains unclear. This study used a T-maze task to assess risk-based decision making in which the rat was required to assess the risk by choosing to enter either a small and certain reward arm or a large but uncertain reward arm of the maze. The latter option, when chosen, resulted in provision of 2, 4, or 8 sweeten pellets with a probability (p) of 0.5, 0.25, or 0.125, respectively. Thus the latter arm provided three different conditions of reward ratio, compared to the choice of former arm, which always provided 1 pellet with p=1. This risk choice task was then run with the expected value being equality between the binary choice options. The experimental rats first received an excitoneurotoxic lesion affecting either the NAC or the dorsolateral striatum (DLS) and this was followed by post-lesion behavioral examination. The sham lesion control rats acquired a stable risk choice with regard to each reward ratio over a 10-day test. The pattern of choice behavior appeared in risk-seeking when p=0.5 to obtain 2 pellets, and was risk-averse when larger reward resulted in lower p. The NAC lesion significantly disrupted the acquisition of the aforementioned risk choice behavior and apparently shifted the choice into a risk-averse style for all three reward ratios. No such effect was observed in the rats with DLS lesions. Neither the gross motor action nor the discrimination of different reward magnitudes was impaired by the lesions affecting either the NAC or DLS as assessed by an additional experiment. These findings suggest that firstly there is heterogeneity between NAC and DLS with respect to risk-based decision making, and that secondly the NAC is involved and critical to the acquisition of behavioral choice under risk, specially when the expected value of the reward under the two choice options is equal.