Chemogenetic Modulation and Single-Photon Calcium Imaging in Anterior Cingulate Cortex Reveal a Mechanism for Effort-Based Decisions.

The ACC is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever-pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever-pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC, via either chemogenetic inhibition or excitation, reduced lever-pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we used in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared with no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options.SIGNIFICANCE STATEMENT The ACC is implicated in effort-based decision-making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever-pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high-value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared with the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.

Basolateral amygdala supports the maintenance of value and effortful choice of a preferred option.

The basolateral amygdala (BLA) is known to be involved in appetitive behavior, yet its role in cost-benefit choice of qualitatively different rewards (more/less preferred), beyond magnitude differences (larger/smaller), is poorly understood. We assessed the effects of BLA inactivations on effortful choice behavior. Rats were implanted with cannulae in BLA and trained to stable lever pressing for sucrose pellets on a progressive ratio schedule. Rats were then introduced to a choice: chow was concurrently available while they could work for the preferred sucrose pellets. Rats were infused with either vehicle control (aCSF) or baclofen/muscimol prior to test. BLA inactivations produced a significant decrease in lever presses for sucrose pellets compared to vehicle, and chow consumption was unaffected. Inactivation had no effect on sucrose pellet preference when both options were freely available. Critically, when lab chow was not concurrently available, BLA inactivations had no effect on the number of lever presses for sucrose pellets, indicating that primary motivation in the absence of choice remains intact with BLA offline. After a test under specific satiety for sucrose pellets, BLA inactivation rendered animals less sensitive to devaluation relative to control. The effects of BLA inactivations in our task are not mediated by decreased appetite, an inability to perform the task, a change in food preference, or decrements in primary motivation. Taken together, BLA supports the specific value and effortful choice of a preferred option.

Anterior cingulate cortex supports effort allocation towards a qualitatively preferred option.

The anterior cingulate cortex (ACC) is known to be involved in effortful choice, yet its role in cost-benefit evaluation of qualitatively different rewards (more/less preferred), beyond magnitude differences (larger/smaller), is poorly understood. Selecting between qualitatively different options is a decision type commonly faced by humans. Here, we assessed the role of ACC on a task that has primarily been used to probe striatal function in motivation. Rats were trained to stable performance on a progressive ratio schedule for sucrose pellets and were then given sham surgeries (control) or excitotoxic NMDA lesions of ACC. Subsequently, a choice was introduced: chow was concurrently available while animals could work for the preferred sucrose pellets. ACC lesions produced a significant decrease in lever presses for sucrose pellets compared to control, whereas chow consumption was unaffected. Lesions had no effect on sucrose pellet preference when both options were freely available. When laboratory chow was not concurrently available, ACC-lesioned rats exhibited similar lever pressing as controls. During a test under specific satiety for sucrose pellets, ACC-lesioned rats also showed intact devaluation effects. The effects of ACC lesions in our task are not mediated by decreased appetite, a change in food preference, a failure to update value or a learning deficit. Taken together, we found that ACC lesions decreased effort for a qualitatively preferred option. These results are discussed with reference to effects of striatal manipulations and our recent report of a role for basolateral amygdala in effortful choice.

Rigid patterns of effortful choice behavior after acute stress in rats.

Physical effort is a common cost of acquiring rewards, and decreased effort is a feature of many neuropsychiatric disorders. Stress affects performance on several tests of cognition and decision making in both humans and nonhumans. Only a few recent reports show impairing effects of stress in operant tasks involving effort and cognitive flexibility. Brain regions affected by stress, such as the medial prefrontal cortex and amygdala, are also implicated in mediating effortful choices. Here, we assessed effort-based decision making after an acute stress procedure known to induce persistent impairment in shuttle escape and elevated plasma corticosterone. In these animals, we also probed levels of polysialyted neural cell adhesion molecule (PSA-NCAM), a marker of structural plasticity, in medial frontal cortex and amygdala. We found that animals that consistently worked for high magnitude rewards continued to do so, even after acute shock stress. We also found that PSA-NCAM was increased in both regions after effortful choice experience but not after shock stress alone. These findings are discussed with reference to the existing broad literature on cognitive effects of stress and in the context of how acute stress may bias effortful decisions to a rigid pattern of responding.

Effort-related motivational effects of the VMAT-2 inhibitor tetrabenazine: implications for animal models of the motivational symptoms of depression.

Motivated behaviors are often characterized by a high degree of behavioral activation, and work output and organisms frequently make effort-related decisions based upon cost/benefit analyses. Moreover, people with major depression and other disorders often show effort-related motivational symptoms such as anergia, psychomotor retardation, and fatigue. It has been suggested that tasks measuring effort-related choice behavior could be used as animal models of the motivational symptoms of depression, and the present studies characterized the effort-related effects of the vesicular monoamine transport (VMAT) inhibitor tetrabenazine. Tetrabenazine produces depressive symptoms in humans and, because of its selective inhibition of VMAT-2, it preferentially depletes dopamine (DA). Rats were assessed using a concurrent fixed-ratio 5/chow feeding choice task that is known to be sensitive to dopaminergic manipulations. Tetrabenazine shifted response choice in rats, producing a dose-related decrease in lever pressing and a concomitant increase in chow intake. However, it did not alter food intake or preference in parallel free-feeding choice studies. The effects of tetrabenazine on effort-related choice were reversed by the adenosine A2A antagonist MSX-3 and the antidepressant bupropion. A behaviorally active dose of tetrabenazine decreased extracellular DA in nucleus accumbens and increased expression of DARPP-32 in accumbens medium spiny neurons in a pattern indicative of reduced transmission at both D1 and D2 DA receptors. These experiments demonstrate that tetrabenazine, which is used in animal models to produce depression-like effects, can alter effort-related choice behavior. These studies have implications for the development of animal models of the motivational symptoms of depression and related disorders.

Anterior cingulate neurons signal neutral cue pairings during sensory preconditioning.

Of all frontocortical subregions, the anterior cingulate cortex (ACC) has perhaps the most overlapping theories of function.1-3 Recording studies in rats, humans, and other primates have reported diverse neural responses that support many theories,4-12 yet nearly all these studies have in common tasks in which one event reliably predicts another. This leaves open the possibility that ACC represents associative pairing of events, independent of their overt biological significance. Sensory preconditioning13 provides an opportunity to test this. In the first phase, preconditioning, value-neutral sensory stimuli are paired (A→B). To test whether this was learned, subjects are given standard conditioning during which one of the previously neutral sensory cues is paired with a biologically meaningful outcome (B→outcome). During the final probe test, the neutral cue which was never paired with a biologically meaningful outcome is presented alone (A→) and will elicit a conditional response, suggesting that subjects had learned the associative structure during preconditioning and use that knowledge to infer presentation of the biologically relevant outcome (A→B→outcome). Inference-based responding demonstrates a fundamental property of model-based reasoning14,15 and requires learning of the associations between neutral stimuli before rewards are introduced.16-19 ACC neurons developed firing patterns that reflected the learning of sensory associations during preconditioning, even though no rewards were present. The strength of these correlates predicted rats' ability to later mobilize and use that associative information during the probe test. These results demonstrate that clear biological significance is not necessary to produce correlates of learning in ACC.

Calcium activity is a degraded estimate of spikes.

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.

Responding to preconditioned cues is devaluation sensitive and requires orbitofrontal cortex during cue-cue learning.

The orbitofrontal cortex (OFC) is necessary for inferring value in tests of model-based reasoning, including in sensory preconditioning. This involvement could be accounted for by representation of value or by representation of broader associative structure. We recently reported neural correlates of such broader associative structure in OFC during the initial phase of sensory preconditioning (Sadacca et al., 2018). Here, we used optogenetic inhibition of OFC to test whether these correlates might be necessary for value inference during later probe testing. We found that inhibition of OFC during cue-cue learning abolished value inference during the probe test, inference subsequently shown in control rats to be sensitive to devaluation of the expected reward. These results demonstrate that OFC must be online during cue-cue learning, consistent with the argument that the correlates previously observed are not simply downstream readouts of sensory processing and instead contribute to building the associative model supporting later behavior.

Quantity versus quality: Convergent findings in effort-based choice tasks.

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

Rodent Models of Adaptive Value Learning and Decision-Making.

Real-world decisions are rarely as straightforward as choosing between clearly "good" vs. "bad" options. More often, options must be evaluated carefully because they differ in relative value. For example, we typically learn about (and make decisions between) options in comparison, where one outcome may be more costly or risky than the other. Several neuropsychiatric conditions are characterized by atypical evaluation of effort and risk costs, including major depression, schizophrenia, autism, obsessive-compulsive disorder, and substance use disorders. Aberrant value learning and decision-making have long been considered a cognitive-behavioral endophenotype of these disorders and can be modeled in rodents. This chapter presents two general methodological domains that the experimenter can manipulate in animal decision-making tasks: risk and effort. Here, we present detailed methods of rodent tasks frequently employed within these domains: probabilistic reversal learning (PRL) and effort choice. These tasks recruit regions within rodent frontal cortex, the amygdala, and the striatum, and performance is heavily modulated by dopamine, making these assays highly valid measures in the study of behavioral and substance addictions, in particular.

Persistent effect of withdrawal from intravenous methamphetamine self-administration on brain activation and behavioral economic indices involving an effort cost.

Exposure to drugs of abuse produces maladaptive changes in cost-benefit decision-making, including the evaluation of time and risk. Studies probing the effects of drug exposure on such evaluations have primarily used experimenter-administered drug regimens. Similarly, while much is known about the neural bases of effort, there have been relatively fewer investigations of the effects of drug experience on effort-based choices. We recently reported that experimenter-administered methamphetamine (meth) resulted in steeper discounting of effort for food rewards in rats, when assessed in protracted withdrawal. Here, we studied rats that underwent withdrawal from weeks of meth intravenous self-administration that later could freely select between a high effort, preferred option (progressive ratio lever pressing for sucrose pellets) versus a low effort, less preferred option (freely-available lab chow). We found decreased effort for the preferred reward and changes in a behavioral economic index demonstrating an increased sensitivity to effort in meth-experienced rats. Critically, the decreased effort for the preferred option was only present in the context of a competing option, not when it was the only option. We also confirmed rats preferred sucrose pellets over chow when both were freely available. These long-lasting changes were accompanied by decreased c-Fos activation in ventral striatum and basolateral amygdala, regions known to be important in effort-based choices. Taken together with our previous observations, these results suggest a robust and enduring effect of meth on value-based decision-making, and point to the underlying neural mechanisms that support the evaluation of an effort cost.

Steep effort discounting of a preferred reward over a freely-available option in prolonged methamphetamine withdrawal in male rats.

RATIONALE: Drug addiction can be described as aberrant allocation of effort toward acquiring drug, despite associated costs. It is unclear if this behavioral pattern results from an overvaluation of reward or to an altered sensitivity to costs. OBJECTIVE: Present experiments assessed reward sensitivity and effortful choice in rats following 1 week of withdrawal from methamphetamine (mAMPH). METHODS: Rats were treated with either saline or an escalating dose mAMPH regimen, then tested after a week without the drug. In experiment 1, rats were given a free choice between water and various concentrations of sucrose solution to assess general reward sensitivity. In experiment 2, rats were presented with a choice between lever-pressing for sucrose pellets on a progressive ratio schedule or consuming freely-available chow. RESULTS: In experiment 1, we found no differences in sucrose preference between mAMPH- and saline-pretreated rats. In experiment 2, when selecting between two options, mAMPH-pretreated rats engaged in less lever-pressing for sucrose pellets (p < 0.01) and switched from this preferred reward to the chow sooner than saline-pretreated rats (p < 0.05). This effect was not consistent with general reward devaluation or loss of motivation. CONCLUSIONS: These findings demonstrate that mAMPH exposure and withdrawal lead to steeper discounting of reward value by effort, an effect that is consistent with the effect of mAMPH on discounting by delay, and which may reflect an underlying shared mechanism.

Post-training depletions of basolateral amygdala serotonin fail to disrupt discrimination, retention, or reversal learning.

In goal-directed pursuits, the basolateral amygdala (BLA) is critical in learning about changes in the value of rewards. BLA-lesioned rats show enhanced reversal learning, a task employed to measure the flexibility of response to changes in reward. Similarly, there is a trend for enhanced discrimination learning, suggesting that BLA may modulate formation of stimulus-reward associations. There is a parallel literature on the importance of serotonin (5HT) in new stimulus-reward and reversal learning. Recent postulations implicate 5HT in learning from punishment. Whereas, dopaminergic involvement is critical in behavioral activation and reinforcement, 5HT may be most critical for aversive processing and behavioral inhibition, complementary cognitive processes. Given these findings, a 5HT-mediated mechanism in BLA may mediate the facilitated learning observed previously. The present study investigated the effects of selective 5HT lesions in BLA using 5,7-dihydroxytryptamine (5,7-DHT) vs. infusions of saline (Sham) on discrimination, retention, and deterministic reversal learning. Rats were required to reach an 85% correct pairwise discrimination and single reversal criterion prior to surgery. Postoperatively, rats were then tested on the (1) retention of the pretreatment discrimination pair, (2) discrimination of a novel pair, and (3) reversal learning performance. We found statistically comparable preoperative learning rates between groups, intact postoperative retention, and unaltered novel discrimination and reversal learning in 5,7-DHT rats. These findings suggest that 5HT in BLA is not required for formation and flexible adjustment of new stimulus-reward associations when the strategy to efficiently solve the task has already been learned. Given the complementary role of orbitofrontal cortex in reward learning and its interconnectivity with BLA, these findings add to the list of dissociable mechanisms for BLA and orbitofrontal cortex in reward learning.

Effort-related motivational effects of the pro-inflammatory cytokine interleukin 1-beta: studies with the concurrent fixed ratio 5/ chow feeding choice task.

RATIONALE: Effort-related motivational symptoms such as anergia and fatigue are common in patients with depression and other disorders. Research implicates pro-inflammatory cytokines in depression, and administration of cytokines can induce effort-related motivational symptoms in humans. OBJECTIVES: The present experiments focused on the effects of the pro-inflammatory cytokine interleukin 1-beta (IL-1ß) on effort-related choice behavior. METHODS: Rats were tested on a concurrent fixed ratio 5 lever pressing/chow feeding choice procedure, which assesses the tendency of rats to work for a preferred food (high carbohydrate pellets) in the presence of a concurrently available but less preferred substitute (laboratory chow). RESULTS: IL-1ß (1.0-4.0 µg/kg IP) shifted choice behavior, significantly decreasing lever pressing and increasing intake of the freely available chow. The second experiment assessed the ability of the adenosine A2A antagonist (E)-phosphoric acid mono-[3-[8-[2-(3-methoxyphenyl)vinyl]-7-methyl-2,6-dioxo-1-prop-2-ynyl-1,2,6,7-tetrahydropurin-3-yl] propyl] ester disodium salt (MSX-3) to reverse the behavioral effects of IL-1ß. MSX-3 attenuated the effort-related impairments produced by IL-1ß, increasing lever pressing and also decreasing chow intake. In the same dose range that shifted effort-related choice behavior, IL-1ß did not alter food intake or preference in parallel free-feeding choice studies, indicating that these low doses were not generally suppressing appetite or altering preference for the high carbohydrate pellets. In addition, IL-1ß did not affect core body temperature. CONCLUSIONS: These results indicate that IL-1ß can reduce the tendency to work for food, even at low doses that do not produce a general sickness, malaise, or loss of appetite. This research has implications for the involvement of cytokines in motivational symptoms such as anergia and fatigue.