The role of prospective contingency in the control of behavior and dopamine signals during associative learning.

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. Here we examined the dopamine activity in the ventral striatum - a signal implicated in associative learning - in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a novel causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate inter-trial-interval (ITI) state representation. Recurrent neural networks trained within a TD framework develop state representations like our best 'handcrafted' model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

Worth the Work? Monkeys Discount Rewards by a Subjective Adapting Effort Cost.

All life must solve how to allocate limited energy resources to maximize benefits from scarce opportunities. Economic theory posits decision makers optimize choice by maximizing the subjective benefit (utility) of reward minus the subjective cost (disutility) of the required effort. While successful in many settings, this model does not fully account for how experience can alter reward-effort trade-offs. Here, we test how well the subtractive model of effort disutility explains the behavior of two male nonhuman primates (Macaca mulatta) in a binary choice task in which reward quantity and physical effort to obtain were varied. Applying random utility modeling to independently estimate reward utility and effort disutility, we show the subtractive effort model better explains out-of-sample choice behavior when compared with parabolic and exponential effort discounting. Furthermore, we demonstrate that effort disutility depends on previous experience of effort: in analogy to work from behavioral labor economics, we develop a model of reference-dependent effort disutility to explain the increased willingness to expend effort following previous experience of effortful options in a session. The result of this analysis suggests that monkeys discount reward by an effort cost that is measured relative to an expected effort learned from previous trials. When this subjective cost of effort, a function of context and experience, is accounted for, trial-by-trial choices can be explained by the subtractive cost model of effort. Therefore, in searching for net utility signals that may underpin effort-based decision-making in the brain, careful measurement of subjective effort costs is an essential first step.SIGNIFICANCE STATEMENT All decision-makers need to consider how much effort they need to expend when evaluating potential options. Economic theories suggest that the optimal way to choose is by cost-benefit analysis of reward against effort. To be able to do this efficiently over many decision contexts, this needs to be done flexibly, with appropriate adaptation to context and experience. Therefore, in aiming to understand how this might be achieved in the brain, it is important to first carefully measure the subjective cost of effort. Here, we show monkeys make reward-effort cost-benefit decisions, subtracting the subjective cost of effort from the subjective value of rewards. Moreover, the subjective cost of effort is dependent on the monkeys' experience of effort in previous trials.

Worth the work? Monkeys discount rewards by a subjective adapting effort cost.

All life must solve how to allocate limited energy resources to maximise benefits from scarce opportunities. Economic theory posits decision makers optimise choice by maximising the subjective benefit (utility) of reward minus the subjective cost (disutility) of the required effort. While successful in many settings, this model does not fully account for how experience can alter reward-effort trade-offs. Here we test how well the subtractive model of effort disutility explains the behavior of two non-human primates ( Macaca mulatta ) in a binary choice task in which reward quantity and physical effort to obtain were varied.Applying random utility modelling to independently estimate reward utility and effort disutility, we show the subtractive effort model better explains out-of-sample choice behavior when compared to parabolic and exponential effort discounting. Furthermore, we demonstrate that effort disutility is dependent on previous experience of effort: in analogy to work from behavioral labour economics, we develop a model of reference-dependent effort disutility to explain the increased willingness to expend effort following previous experience of effortful options in a session. The result of this analysis suggests that monkeys discount reward by an effort cost that is measured relative to an expected effort learned from previous trials. When this subjective cost of effort, a function of context and experience, is accounted for, trial-by-trial choice behavior can be explained by the subtractive cost model of effort.Therefore, in searching for net utility signals that may underpin effort-based decision-making in the brain, careful measurement of subjective effort costs is an essential first step. Significance: All decision-makers need to consider how much effort they need to expend when evaluating potential options. Economic theories suggest that the optimal way to choose is by cost-benefit analysis of reward against effort. To be able to do this efficiently over many decision contexts, this needs to be done flexibly, with appropriate adaptation to context and experience. Therefore, in aiming to understand how this might be achieved in the brain, it is important to first carefully measure the subjective cost of effort. Here we show monkeys make reward-effort cost-benefit decisions, subtracting the subjective cost of effort from the subjective value of rewards. Moreover, the subjective cost of effort is dependent on the monkeys’ experience of effort in previous trials.

Impact of low-intensity pulsed ultrasound on the growth of Schizochytrium sp. for omega-3 production.

Schizochytrium sp. is a microalga that is known for its high content of oils or lipids. It has a high percentage of polyunsaturated fatty acids in the accumulated oil, especially docosahexaenoic acid (DHA). DHA is an important additive for the human diet. Large-scale production of Schizochytrium sp. can serve as an alternative source of DHA for humans as well as for fish feed, decreasing the burden on aqua systems. Therefore, research on improving the productivity of Schizochytrium attracts a lot of attention. We studied the potential of using low-intensity pulsed ultrasound (LIPUS) in the growth cycle of Schizochytrium sp. in shake flasks. Different intensities and treatment durations were tested. A positive effect of LIPUS on biomass accumulation was observed in the Schizochytrium sp. culture. Specifically, LIPUS stimulation at the ultrasound intensity of 400 mW/cm2 with 20 min per treatment 10 times a day with equal intervals of 2.4 h between the treatments was found to enhance the growth of Schizochytrium biomass most effectively (by up to 20%). Due to the nature of cell division in Schizochytrium sp. which occurs via zoospore formation, LIPUS stimulation was inefficient if applied continuously during all 5 days of the growth cycle. Using microscopy, we studied the interval between zoospore formation in the culture and selected the optimal LIPUS application days (Days 0-1 and Days 4-5 of the 5-day growth cycle). Microscopic images have also shown that LIPUS stimulation enhances zoospore formation in Schizochytrium sp., leading to more active cell division in the culture. This study shows that LIPUS can serve as an additional tool for cost-efficiency improvement in the large-scale production of Schizochytrium as a sustainable and environmentally friendly source of omega-3 (DHA).

Role of homeostatic feedback mechanisms in modulating methylphenidate actions on phasic dopamine signaling in the striatum of awake behaving rats.

Methylphenidate is an established treatment for attention-deficit hyperactivity disorder that also has abuse potential. Both properties may relate to blocking dopamine and norepinephrine reuptake. We measured the effects of methylphenidate on dopamine dynamics in freely moving rats. Methylphenidate alone had no effect on the amplitude of phasic responses to cues or reward. However, when administered with the D2 receptor antagonist raclopride, methylphenidate increased dopamine responses, while raclopride alone had no effect. Using brain slices of substantia nigra or striatum, we confirmed that methylphenidate effects on firing rate of nigral dopamine neurons and dopamine release from terminals are constrained by negative feedback. A computational model using physiologically relevant parameters revealed that actions of methylphenidate on norepinephrine and dopamine transporters, and the effects of changes in tonic dopamine levels on D2 receptors, are necessary and sufficient to account for the experimental findings. In addition, non-linear fitting of the model to the data from freely moving animals revealed that methylphenidate significantly slowed the initial cue response dynamics. These results show that homeostatic regulation of dopamine release in the face of changing tonic levels of extracellular dopamine should be taken into account to understand the therapeutic benefits and abuse potential of methylphenidate.

Action potential and calcium dependence of tonic somatodendritic dopamine release in the Substantia Nigra pars compacta.

Despite the importance of somatodendritic dopamine (DA) release in the Substantia Nigra pars compacta (SNc), its mechanism remains poorly understood. Using a novel approach combining fast-scan controlled-adsorption voltammetry (FSCAV) and single-unit electrophysiology, we have investigated the mechanism of somatodendritic release by directly correlating basal (non-stimulated) extracellular DA concentration ([DA]out ), with pharmacologically-induced changes of firing of nigral dopaminergic neurons in rat brain slices. FSCAV measurements indicated that basal [DA]out in the SNc was 40.7 ± 2.0 nM (at 34 ± 0.5°C), which was enhanced by amphetamine, cocaine, and L-DOPA, and reduced by VMAT2 inhibitor, Ro4-1284. Complete inhibition of firing by TTX decreased basal [DA]out , but this reduction was smaller than the effect of D2 receptor agonist, quinpirole. Despite similar effects on neuronal firing, the larger decrease in [DA]out evoked by quinpirole was attributed to cell membrane hyperpolarization and greater reduction in cytosolic free Ca2+ ([Ca2+ ]in ). Decreasing extracellular Ca2+ also reduced basal [DA]out , despite increasing firing frequency. Furthermore, inhibiting L-type Ca2+ channels decreased basal [DA]out , although specific Cav 1.3 channel inhibition did not affect firing rate. Inhibition of sarcoplasmic/endoplasmic reticulum Ca2+ -ATPase (SERCA) also decreased [DA]out , demonstrating the importance of intracellular Ca2+ stores for somatodendritic release. Finally, in vivo FSCAV measurements showed that basal [DA]out in the SNc was 79.8 ± 10.9 nM in urethane-anesthetized rats, which was enhanced by amphetamine. Overall, our findings indicate that although tonic somatodendritic DA release is largely independent of action potentials, basal [DA]out is strongly regulated by voltage-dependent Ca2+ influx and release of intracellular Ca2+ . OPEN SCIENCE BADGES: This article has received a badge for *Open Materials* because it provided all relevant information to reproduce the study in the manuscript. The complete Open Science Disclosure form for this article can be found at the end of the article. More information about the Open Practices badges can be found at https://cos.io/our-services/open-science-badges/.

A novel electrochemical approach for prolonged measurement of absolute levels of extracellular dopamine in brain slices.

Tonic dopamine (DA) levels influence the activity of dopaminergic neurons and the dynamics of fast dopaminergic transmission. Although carbon fiber microelectrodes and fast-scan cyclic voltammetry (FSCV) have been extensively used to quantify stimulus-induced release and uptake of DA in vivo and in vitro, this technique relies on background subtraction and thus cannot provide information about absolute extracellular concentrations. It is also generally not suitable for prolonged (>90 s) recordings due to drift of the background current. A recently reported, modified FSCV approach called fast-scan controlled-adsorption voltammetry (FSCAV) has been used to assess tonic DA levels in solution and in the anesthetized mouse brain. Here we describe a novel extension of FSCAV to investigate pharmacologically induced, slowly occurring changes in tonic (background) extracellular DA concentration, and phasic (stimulated) DA release in brain slices. FSCAV was used to measure adsorption dynamics and changes in DA concentration (for up to 1.5 h, sampling interval 30 s, detection threshold < 10 nM) evoked by drugs affecting DA release and uptake (amphetamine, l-DOPA, pargyline, cocaine, Ro4-1284) in submerged striatal slices obtained from rats. We also show that combined FSCAV-FSCV recordings can be used for concurrent study of stimulated release and changes in tonic DA concentration. Our results demonstrate that FSCAV can be effectively used in brain slices to measure prolonged changes in extracellular level of endogenous DA expressed as absolute values, complementing studies conducted in vivo with microdialysis.