Hunger improves reinforcement-driven but not planned action.

Human decisions can be reflexive or planned, being governed respectively by model-free and model-based learning systems. These two systems might differ in their responsiveness to our needs. Hunger drives us to specifically seek food rewards, but here we ask whether it might have more general effects on these two decision systems. On one hand, the model-based system is often considered flexible and context-sensitive, and might therefore be modulated by metabolic needs. On the other hand, the model-free system's primitive reinforcement mechanisms may have closer ties to biological drives. Here, we tested participants on a well-established two-stage sequential decision-making task that dissociates the contribution of model-based and model-free control. Hunger enhanced overall performance by increasing model-free control, without affecting model-based control. These results demonstrate a generalized effect of hunger on decision-making that enhances reliance on primitive reinforcement learning, which in some situations translates into adaptive benefits.

Modeling the effects of motivation on choice and learning in the basal ganglia.

Decision making relies on adequately evaluating the consequences of actions on the basis of past experience and the current physiological state. A key role in this process is played by the basal ganglia, where neural activity and plasticity are modulated by dopaminergic input from the midbrain. Internal physiological factors, such as hunger, scale signals encoded by dopaminergic neurons and thus they alter the motivation for taking actions and learning. However, to our knowledge, no formal mathematical formulation exists for how a physiological state affects learning and action selection in the basal ganglia. We developed a framework for modelling the effect of motivation on choice and learning. The framework defines the motivation to obtain a particular resource as the difference between the desired and the current level of this resource, and proposes how the utility of reinforcements depends on the motivation. To account for dopaminergic activity previously recorded in different physiological states, the paper argues that the prediction error encoded in the dopaminergic activity needs to be redefined as the difference between utility and expected utility, which depends on both the objective reinforcement and the motivation. We also demonstrate a possible mechanism by which the evaluation and learning of utility of actions can be implemented in the basal ganglia network. The presented theory brings together models of learning in the basal ganglia with the incentive salience theory in a single simple framework, and it provides a mechanistic insight into how decision processes and learning in the basal ganglia are modulated by the motivation. Moreover, this theory is also consistent with data on neural underpinnings of overeating and obesity, and makes further experimental predictions.

Lateral Hypothalamic Area Glutamatergic Neurons and Their Projections to the Lateral Habenula Regulate Feeding and Reward.

The overconsumption of calorically dense, highly palatable foods is thought to be a major contributor to the worldwide obesity epidemic; however, the precise neural circuits that directly regulate hedonic feeding remain elusive. Here, we show that lateral hypothalamic area (LHA) glutamatergic neurons, and their projections to the lateral habenula (LHb), negatively regulate the consumption of palatable food. Genetic ablation of LHA glutamatergic neurons increased daily caloric intake and produced weight gain in mice that had access to a high-fat diet, while not altering general locomotor activity. Anterior LHA glutamatergic neurons send a functional glutamatergic projection to the LHb, a brain region involved in processing aversive stimuli and negative reward prediction outcomes. Pathway-specific, optogenetic stimulation of glutamatergic LHA-LHb circuit resulted in detectable glutamate-mediated EPSCs as well as GABA-mediated IPSCs, although the net effect of neurotransmitter release was to increase the firing of most LHb neurons. In vivo optogenetic inhibition of LHA-LHb glutamatergic fibers produced a real-time place preference, whereas optogenetic stimulation of LHA-LHb glutamatergic fibers had the opposite effect. Furthermore, optogenetic inhibition of LHA-LHb glutamatergic fibers acutely increased the consumption of a palatable liquid caloric reward. Collectively, these results demonstrate that LHA glutamatergic neurons are well situated to bidirectionally regulate feeding and potentially other behavioral states via their functional circuit connectivity with the LHb and potentially other brain regions. SIGNIFICANCE STATEMENT: In this study, we show that the genetic ablation of LHA glutamatergic neurons enhances caloric intake. Some of these LHA glutamatergic neurons project to the lateral habenula, a brain area important for generating behavioral avoidance. Optogenetic stimulation of this circuit has net excitatory effects on postsynaptic LHb neurons. This is the first study to characterize the functional connectivity and behavioral relevance of this circuit within the context of feeding and reward-related behavior.

Dissociating the role of endocannabinoids in the pleasurable and motivational properties of social play behaviour in rats.

Social play behaviour is a vigorous form of social interaction, abundant during the juvenile and adolescent phases of life in many mammalian species, including humans. Social play is highly rewarding and it is important for social and cognitive development. Being a rewarding activity, social play can be dissociated in its pleasurable and motivational components. We have previously shown that endocannabinoids modulate the expression of social play behaviour in rats. In the present study, we investigated whether endocannabinoids modulate the motivational and pleasurable properties of social play behaviour, using operant and place conditioning paradigms, respectively. Treatment with the anandamide hydrolysis inhibitor URB597 did not affect operant responding or social play-induced conditioned place preference (CPP) when administered at a dose (0.1mg/kg) known to increase the expression of social play behaviour, while it modestly reduced operant responding at a higher dose (0.2mg/kg). The cannabinoid-1 (CB1) receptor antagonist rimonabant reduced operant responding when administered at a dose (1mg/kg) known to decrease the expression of social play behaviour, although this effect may be secondary to concurrent drug-induced stereotypic behaviours (i.e., grooming and scratching). These data demonstrate that enhancing endocannabinoid levels does not differentially affect the motivational and pleasurable aspects of social play behaviour, whereas CB1 receptor blockade reduces the motivational aspects of social play behaviour, possibly due to response competition. Thus, endocannabinoids likely drive the expression of social play behaviour as a whole, without differentially affecting its motivational or pleasurable properties.

Incident Use of Hydroxychloroquine for the Treatment of Rheumatoid Arthritis and Systemic Lupus Erythematosus During the COVID-19 Pandemic.

OBJECTIVE: We studied whether the use of hydroxychloroquine (HCQ) for COVID-19 resulted in supply shortages for patients with rheumatoid arthritis (RA) and systemic lupus erythematosus (SLE). METHODS: We used US claims data (IQVIA PHARMETRICS® Plus for Academics [PHARMETRICS]) and hospital electronic records from Spain (Institut Municipal d'Assistència Sanitària Information System [IMASIS]) to estimate monthly rates of HCQ use between January 2019 and March 2022, in the general population and in patients with RA and SLE. Methotrexate (MTX) use was estimated as a control. RESULTS: More than 13.5 million individuals (13,311,811 PHARMETRICS, 207,646 IMASIS) were included in the general population cohort. RA and SLE cohorts enrolled 135,259 and 39,295 patients, respectively, in PHARMETRICS. Incidence of MTX and HCQ were stable before March 2020. On March 2020, the incidence of HCQ increased by 9- and 67-fold in PHARMETRICS and IMASIS, respectively, and decreased in May 2020. Usage rates of HCQ went back to prepandemic trends in Spain but remained high in the United States, mimicking waves of COVID-19. No significant changes in HCQ use were noted among patients with RA and SLE. MTX use rates decreased during HCQ approval period for COVID-19 treatment. CONCLUSION: Use of HCQ increased dramatically in the general population in both Spain and the United States during March and April 2020. Whereas Spain returned to prepandemic rates after the first wave, use of HCQ remained high and followed waves of COVID-19 in the United States. However, we found no evidence of general shortages in the use of HCQ for both RA and SLE in the United States.

Gambling on an empty stomach: Hunger modulates preferences for learned but not described risks.

INTRODUCTION: We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, decision-making from descriptions is often considered flexible and context sensitive, and might therefore be modulated by metabolic needs. On the other hand, preferences learned through reinforcement might be more strongly coupled to biological drives. METHOD: Thirty-two healthy participants (females: 20, mean age: 25.6 ± 6.5 years) with a normal weight (Body Mass Index: 22.9 ± 3.2 kg/m2 ) were tested in a within-subjects counterbalanced, randomized crossover design for the effects of hunger on two separate risk-taking tasks. We asked participants to choose between two options with different risks to obtain monetary outcomes. In one task, the outcome probabilities were described numerically, whereas in a second task, they were learned. RESULT: In agreement with previous studies, we found that rewarding contexts induced risk-aversion when risks were explicitly described (F1,31  = 55.01, p < .0001, ηp 2  = .64), but risk-seeking when they were learned through experience (F1,31  = 10.28, p < .003, ηp 2  = .25). Crucially, hunger attenuated these contextual biases, but only for learned risks (F1,31  = 8.38, p < .007, ηp 2  = .21). CONCLUSION: The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.

Model-based learning retrospectively updates model-free values.

Reinforcement learning (RL) is widely regarded as divisible into two distinct computational strategies. Model-free learning is a simple RL process in which a value is associated with actions, whereas model-based learning relies on the formation of internal models of the environment to maximise reward. Recently, theoretical and animal work has suggested that such models might be used to train model-free behaviour, reducing the burden of costly forward planning. Here we devised a way to probe this possibility in human behaviour. We adapted a two-stage decision task and found evidence that model-based processes at the time of learning can alter model-free valuation in healthy individuals. We asked people to rate subjective value of an irrelevant feature that was seen at the time a model-based decision would have been made. These irrelevant feature value ratings were updated by rewards, but in a way that accounted for whether the selected action retrospectively ought to have been taken. This model-based influence on model-free value ratings was best accounted for by a reward prediction error that was calculated relative to the decision path that would most likely have led to the reward. This effect occurred independently of attention and was not present when participants were not explicitly told about the structure of the environment. These findings suggest that current conceptions of model-based and model-free learning require updating in favour of a more integrated approach. Our task provides an empirical handle for further study of the dialogue between these two learning systems in the future.

Contrasting Roles of Dopamine and Noradrenaline in the Motivational Properties of Social Play Behavior in Rats.

Social play behavior, abundant in the young of most mammalian species, is thought to be important for social and cognitive development. Social play is highly rewarding, and as such, the expression of social play depends on its pleasurable and motivational properties. Since the motivational properties of social play have only sporadically been investigated, we developed a setup in which rats responded for social play under a progressive ratio schedule of reinforcement. Dopaminergic neurotransmission plays a key role in incentive motivational processes, and both dopamine and noradrenaline have been implicated in the modulation of social play behavior. Therefore, we investigated the role of dopamine and noradrenaline in the motivation for social play. Treatment with the psychostimulant drugs methylphenidate and cocaine increased responding for social play, but suppressed its expression during reinforced play periods. The dopamine reuptake inhibitor GBR-12909 increased responding for social play, but did not affect its expression, whereas the noradrenaline reuptake inhibitor atomoxetine decreased responding for social play as well as its expression. The effects of methylphenidate and cocaine on responding for social play, but not their play-suppressant effects, were blocked by pretreatment with the dopamine receptor antagonist α-flupenthixol. In contrast, pretreatment with the α2-adrenoceptor antagonist RX821002 prevented the play-suppressant effect of methylphenidate, but left its effect on responding for social play unaltered. In sum, the present study introduces a novel method to study the incentive motivational properties of social play behavior in rats. Using this paradigm, we demonstrate dissociable roles for dopamine and noradrenaline in social play behavior: dopamine stimulates the motivation for social play, whereas noradrenaline negatively modulates the motivation for social play behavior and its expression.