Cyclic estrogen and progesterone during instrumental acquisition contributes to habit formation in female rats.

Habit formation is thought to involve two parallel processes that are mediated by distinct neural substates: one that suppresses goal-directed behavior, and one that facilitates stimulus-response (S-R) learning, which underscores habitual behavior. In previous studies we showed that habitual responding emerges early during instrumental training in gonadally-intact female, compared to male, rats. The present study aimed to determine the role of ovarian hormones during instrumental acquisition in the transition from goal-directed to habitual behavior in female rats. Ovariectomized (OVX) female rats were given subcutaneous silastic capsules that released low levels of 17-ß estradiol (E2) to maintain estrogen receptor availability. Rats were assigned to one of three hormone treatment conditions: no additional hormone replacement (Control group), replacement with high E2 (High E2 group), or replacement with high E2 followed by progesterone (High E2 + P4 group). Hormone replacement occurred twice during acquisition to mimic natural hormone fluctuations. At test, the Control and High E2 groups demonstrated responding that was sensitive to devaluation by lithium chloride-induced illness, indicating goal-directed behavior. In contrast, the High E2 + P4 group exhibited a pattern of devaluation-insensitive, habitual responding, that suggested the suppression of goal-directed processes. In a follow-up experiment, similar procedures were conducted, however during acquisition, OVX rats were given cyclic high E2 plus medroxy-progesterone (MPA), a form of progesterone that does not metabolize to neuroactive metabolites. In this group, goal-directed behavior was observed. These data indicate that habit formation is not facilitated in low estrogen states, nor in the presence of cyclic high E2. However, cyclic high E2, together with progesterone during acquisition, appears to facilitate the early emergence of habitual responding. Furthermore, these data suggest that a neuroactive progesterone metabolite, like allopregnanolone, in combination with high cyclic E2, supports this phenomenon.

Competing influences on healthy food choices: Mindsetting versus contextual food cues.

Food choices are influenced by one's current mindset, suggesting that supporting health (vs. a palatability) mindsets could improve daily food choices. The question rises, however, to what extent internal mindsets still guide choices when people are exposed to external food-context stimuli in an obesogenic environment. To examine these two competing effects we induced health vs. palatability mindsets, and investigated the robustness of the mindset effect by presenting food-context stimuli during a Pavlovian-to-Instrumental-Transfer (PIT) task in two separate cohorts of 102 (76 females) Dutch and 120 (60 females) German participants. For the mindset induction, participants rated food items on visual analogue scales (VAS), based on healthiness and palatability, respectively. In each cohort, half of the participants received a health, the other half a palatability mindset induction. Additionally, we explored whether 'mindset triggers' could be used to further shape behavior. Triggers were established by placing unfamiliar logos at the extreme ends of the VASs used for the mindset inductions. Independent of the mindset, food-associated stimuli influenced food choices in accordance with the previously learned association in each test phase. Health mindset induction biased food choices towards healthier, palatability mindset towards unhealthier choices in the first cohort, but not in the second. The mindset triggers had a more robust effect. These induced healthier (triggers for healthy and not-palatable) and unhealthier (triggers for unhealthy and palatable) food choices in both cohorts alike. Interestingly, these effects did not tamper with the overall effect of Pavlovian cues and were thus true in the presence and absence of food-context stimuli. Therefore, we show that, in our experimental setting, food-associated mindset triggers can be used to bias food choices towards a healthy snack even in an obesogenic environment.

Some factors that restore goal-direction to a habitual behavior.

Recent findings from our laboratory suggest that an extensively-practiced instrumental behavior can appear to be a goal-directed action (rather than a habit) when a second behavior is added and reinforced during intermixed final sessions (Shipman et al., 2018). The present experiments were designed to explore and understand this finding. All used the taste aversion method of devaluing the reinforcer to distinguish between goal-directed actions and habits. Experiment 1 confirmed that reinforcing a second response in a separate context (but not mere exposure to that context) can return an extensively-trained habit to the status of goal-directed action. Experiment 2 showed that training of the second response needs to be intermixed with training of the first response to produce this effect; training the second response after the first-response training was complete preserved the first response as a habit. Experiment 3 demonstrated that reinforcing the second response with a different reinforcer breaks the habit status of the first response. Experiment 4 found that free reinforcers (that were not response-contingent) were sufficient to restore goal-directed performance. Together, the results suggest that unexpected reinforcer delivery can render a habitual response goal-directed again.

Action-outcome relationships are represented differently by medial prefrontal and orbitofrontal cortex neurons during action execution.

Internal representations of action-outcome relationships are necessary for flexible adaptation of motivated behavior in dynamic environments. Prefrontal cortex (PFC) is implicated in flexible planning and execution of goal-directed actions, but little is known about how information about action-outcome relationships is represented across functionally distinct regions of PFC. Here, we observe distinct patterns of action-evoked single unit activity in the medial prefrontal cortex (mPFC) and orbitofrontal cortex (OFC) during a task in which the relationship between outcomes and actions was independently manipulated. The mPFC encoded changes in the number of actions required to earn a reward, but not fluctuations in outcome magnitude. In contrast, OFC neurons decreased firing rates as outcome magnitude was increased, but were insensitive to changes in action requirement. A subset of OFC neurons also tracked outcome availability. Pre-outcome anticipatory activity in both mPFC and OFC was altered when reward expectation was reduced, but did not differ with outcome magnitude. These data provide novel evidence that PFC regions encode distinct information about the relationship between actions and impending outcomes during action execution.

Lipopolysaccharide does not alter behavioral response to successive negative contrast in mice.

BACKGROUND: Reduced motivation is one of the main symptomatic features of inflammation-induced depression. However, the exact nature of inflammation-induced alterations in motivation remains to be fully defined. As inflammation has been shown to increase sensitivity to negative stimuli, the present series of experiments was initiated to determine whether systemic inflammation induced by infra-septic doses of lipopolysaccharide (LPS) in mice influences consummatory and instrumental responding to successive negative contrast. METHODS: Successive negative contrast was operationally defined by a shift to a lower value reward than the one mice were trained with. Mice were trained to drink a high sucrose concentration solution and exposed to an acute shift to a lower concentration of sucrose. In another series of experiments, mice were trained to nose poke for chocolate pellets according to a fixed reinforcement schedule 10 (10 nose pokes for the food reinforcement) and exposed to a shift to a lower reward value (decreased number of chocolate pellets or replacement of chocolate pellets by less preferred grain pellets). Lipopolysaccharide (LPS) was administered at the dose of 0.33 1 mg/kg 24 h before the shift. RESULTS: Mice trained to drink a high sucrose concentration responded to the shift in reward value by a reduction in the volume of sucrose consumed and a decrease in lick numbers and bout durations. Mice trained to nose poke for chocolate pellets responded to the shift by alterations in their total number of nose pokes. In both conditions, LPS had no consistent effect on the response to the shift in reward value. CONCLUSIONS: These findings indicate a high variability in the effects of LPS on successive negative contrast and fail to provide evidence in favor of the hypothesis that LPS increases sensitivity to decreases in expected rewards.

Activation of Basolateral Amygdala to Nucleus Accumbens Projection Neurons Attenuates Chronic Corticosterone-Induced Behavioral Deficits in Male Mice.

The basolateral amygdala (BLA) is critical for reward behaviors via a projection to the nucleus accumbens (NAc). Specifically, BLA-NAc projections are involved in reinforcement learning, reward-seeking, sustained instrumental responding, and risk behaviors. However, it remains unclear whether chronic stress interacts with BLA-NAc projection neurons to result in maladaptive behaviors. Here we take a chemogenetic, projection-specific approach to clarify how NAc-projecting BLA neurons affect avoidance, reward, and feeding behaviors in male mice. Then, we examine whether chemogenetic activation of NAc-projecting BLA neurons attenuates the maladaptive effects of chronic corticosterone (CORT) administration on these behaviors. CORT mimics the behavioral and neural effects of chronic stress exposure. We found a nuanced role of BLA-NAc neurons in mediating reward behaviors. Surprisingly, activation of BLA-NAc projections rescues CORT-induced deficits in the novelty suppressed feeding, a behavior typically associated with avoidance. Activation of BLA-NAc neurons also increases instrumental reward-seeking without affecting free-feeding in chronic CORT mice. Taken together, these data suggest that NAc-projecting BLA neurons are involved in chronic CORT-induced maladaptive reward and motivation behaviors.

Avoidance and its bi-directional relationship with conditioned fear: Mechanisms, moderators, and clinical implications.

Fear motivates different types of defensive behaviors. These behaviors are, however, not mere byproducts of fear. In this review, we highlight a bi-directional relationship between conditioned fear and instrumental defensive behavior in humans. We discuss mechanisms involved in the link from fear to goal-directed avoidance (e.g., relief, generalization), that may become habitual. These defensive behaviors may in turn reduce, preserve, or amplify fear responding (e.g., protection-from-extinction, behavior-as-information). Multiple factors moderate the bi-directional relationship. Evidence for amplifying and dampening effects of inter-individual differences (e.g., trait anxiety, distress tolerance), intra-individual states (e.g., stress), and external factors (e.g., incentives for competing behavior) on goal-directed and/or habitual defensive behavior is reviewed. However, the exact mechanisms by which these factors moderate the bi-directional relationship are still largely unknown (e.g., modulating avoidance directly vs. indirectly via conditioned fear). Finally, we discuss major implications: First, understanding factors moderating the bi-directional relationship provides insights into risk and resilience factors for anxious psychopathology. Second, specific experimental models and clinical interventions can be mapped onto distinct defensive behaviors (e.g., goal-directed vs. habitual avoidance). More precise matching will help to develop nuanced models and interventions to reduce pathological behaviors and individualize treatments.

The reoccurrence of voluntary behavior in humans is reduced by retrieval cues from extinction.

Changes in the temporal as well as the physical context produces the reappearance of extinguished behaviors. Furthermore, combining both kinds of contextual stimuli often causes greater levels of recovery. The current experiment explored the impact of extinction reminders on spontaneous recovery, renewal, and a combination of both effects using an instrumental learning task with humans. All participants learned to shoot at enemies in a videogame. Then, throughout extinction, the instrumental response was eliminated. We found a return of the extinguished behavior by introducing a retention interval of 48 h, by changing the physical background and by testing participants in a spatiotemporal context different from the extinction context. However, we also found that the presentation of a stimulus directly associated with extinction attenuates all three forms of operant reoccurrence. These results are consistent with the perspective that emphasizes that context plays a key role in response-recovery phenomena. Moreover, our findings may be promissory for therapeutic strategies involving relapse treatment.

A Reward-Based Framework of Perceived Control.

Perceived control can be broadly defined as the belief in one's ability to exert control over situations or events. It has long been known that perceived control is a major contributor toward mental and physical health as well as a strong predictor of achievements in life. However, one issue that limits a mechanistic understanding of perceived control is the heterogeneity of how the term is defined in models in psychology and neuroscience, and used in experimental settings across a wide spectrum of studies. Here, we propose a framework for studying perceived control by integrating the ideas from traditionally separate work on perceived control. Specifically, we discuss key properties of perceived control from a reward-based framework, including choice opportunity, instrumental contingency, and success/reward rate. We argue that these separate reward-related processes are integral to fostering an enhanced perception of control and influencing an individual's behavior and well-being. We draw on select studies to elucidate how these reward-related elements are implicated separately and collectively in the investigation of perceived control. We highlight the role of dopamine within corticostriatal pathways shared by reward-related processes and perceived control. Finally, through the lens of this reward-based framework of perceived control, we consider the implications of perceived control in clinical deficits and how these insights could help us better understand psychopathology and treatment options.

Pharmacological Blockade of Adenosine A2A but Not A1 Receptors Enhances Goal-Directed Valuation in Satiety-Based Instrumental Behavior.

The balance and smooth shift between flexible, goal-directed behaviors and repetitive, habitual actions are critical to optimal performance of behavioral tasks. The striatum plays an essential role in control of goal-directed versus habitual behaviors through a rich interplay of the numerous neurotransmitters and neuromodulators to modify the input, processing and output functions of the striatum. The adenosine receptors (namely A2AR and A1R), with their high expression pattern in the striatum and abilities to interact and integrate dopamine, glutamate and cannabinoid signals in the striatum, may represent novel therapeutic targets for modulating instrumental behavior. In this study, we examined the effects of pharmacological blockade of the A2ARs and A1Rs on goal-directed versus habitual behaviors in different information processing phases of instrumental learning using a satiety-based instrumental behavior procedure. We found that A2AR antagonist acts at the coding, consolidation and expression phases of instrumental learning to modulate animals' sensitivity to goal-directed valuation without modifying action-outcome contingency. However, pharmacological blockade and genetic knockout of A1Rs did not affect acquisition or sensitivity to goal-valuation of instrumental behavior. These findings provide pharmacological evidence for a potential therapeutic strategy to control abnormal instrumental behaviors associated with drug addiction and obsessive-compulsive disorder by targeting the A2AR.

Methylphenidate has nonlinear dose effects on cued response inhibition in adults but not adolescents.

Ongoing development of the dopamine system during adolescence may provide a partial mechanism for behavioral and psychiatric vulnerabilities. Despite early evidence for a hyperactive adolescent dopaminergic system, recent data suggest that adolescent dopamine may be functionally hypoactive compared to in adults. While this distinction has been established in response to dopaminergic drugs and natural rewards, little is known about age-related differences in cognitive efficacy of dopaminergic drugs. Using a recently established Cued Response Inhibition Task, we tested the effects of acute systemic methylphenidate, commonly known as Ritalin, on response inhibition and response initiation in adolescent and adults rats. First, we replicated previous data that adolescents are able to inhibit a response to a cue on par with adults, but are slower to produce a rewarded response after a stop cue. Next, we observed that methylphenidate modulated response inhibition in adult rats, with low dose (0.3mg/kg) improving inhibition, and high dose (3mg/kg) impairing performance. This dose-response pattern is commonly observed with psychostimulant cognitive modulation. In adolescents, however, methylphenidate had no effect on response inhibition at any dose. Latency of response initiation after the stop cue was not affected by methylphenidate in either adult or adolescent rats. These data establish that dose-response of a commonly prescribed psychostimulant medication is different in adolescents and adults. They further demonstrate that healthy adolescent response inhibition is not as sensitive to psychostimulants as in adults, supporting the idea that the dopamine system is hypoactive in adolescence. This article is part of a Special Issue entitled SI: Adolescent plasticity.

Decoupling Actions from Consequences: Dorsal Hippocampal Lesions Facilitate Instrumental Performance, but Impair Behavioral Flexibility in Rats.

The present study is part of a series of experiments, where we analyze why and how damage of the rat's dorsal hippocampus (dHC) can enhance performance in a sequential reaction time task (SRTT). In this task, sequences of distinct visual stimulus presentations are food-rewarded in a fixed-ratio-13-schedule. Our previous study (Busse and Schwarting, 2016) had shown that rats with lesions of the dHC show substantially shorter session times and post-reinforcement pauses (PRPs) than controls, which allows for more practice when daily training is kept constant. Since sequential behavior is based on instrumental performance, a sequential benefit might be secondary to that. In order to test this hypothesis in the present study, we performed two experiments, where pseudorandom rather than sequential stimulus presentation was used in rats with excitotoxic dorsal hippocampal lesions. Again, we found enhanced performance in the lesion-group in terms of shorter session times and PRPs. During the sessions we found that the lesion-group spent less time with non-instrumental behavior (i.e., grooming, sniffing, and rearing) after prolonged instrumental training. Also, such rats showed moderate evidence for an extinction impairment under devalued food reward conditions and significant deficits in a response-outcome (R-O)-discrimination task in comparison to a control-group. These findings suggest that facilitatory effects on instrumental performance after dorsal hippocampal lesions may be primarily a result of complex behavioral changes, i.e., reductions of behavioral flexibility and/or alterations in motivation, which then result in enhanced instrumental learning.

Psychosocial problems and recruitment of incentive neurocircuitry: exploring individual differences in healthy adolescents.

Maturational differences in brain responsiveness to rewards have been implicated in the increased rates of injury and death in adolescents from behavior-related causes. However, much of this morbidity is related to drug intoxication or other externalizing behaviors, and may be concentrated in a subset of adolescents who are at psychosocial or neurobiological risk. To examine whether individual differences in psychosocial and behavioral symptomatology relate to activation of motivational neurocircuitry, we scanned 26 psychiatrically healthy adolescents using fMRI as they performed a monetary incentive delay task. Overall Problem Density on the Drug Use Screening Inventory (DUSI-OPD) correlated positively with activation of ventral mesofrontal cortex (mFC) during anticipation of responding for rewards (vs responding for no incentive). In addition, DUSI-OPD correlated positively with right ventral striatum recruitment during anticipation of responding to win rewards (vs responding for no incentive or to avoid losses of identical magnitudes). Finally, a psychophysiological interaction (PPI) analysis indicated that increased connectivity between nucleus accumbens and portions of anterior cingulate and mFC as a function of reward prospects also correlated with DUSI-OPD. These findings extend previous reports demonstrating that in adolescents, individual differences in reactivity of motivational neurocircuitry relate to different facets of impulsivity or externalizing behaviors.