An analysis of reinstatement after extinction of a conditioned taste aversion.

Taste aversion learning has sometimes been considered a specialized form of learning. In several other conditioning preparations, after a conditioned stimulus (CS) is conditioned and extinguished, reexposure to the unconditioned stimulus (US) by itself can reinstate the extinguished conditioned response. Reinstatement has been widely studied in fear and appetitive Pavlovian conditioning, as well as operant conditioning, but its status in taste aversion learning is more controversial. Six taste-aversion experiments with rats therefore sought to discover conditions that might encourage it there. The results often yielded little to no evidence of reinstatement, and we also found no evidence of concurrent recovery, a related phenomenon in which responding to a CS that has been conditioned and extinguished is restored if a second CS is separately conditioned. However, a key result was that reinstatement occurred when the conditioning procedure involved multiple closely spaced conditioning trials that could have allowed the animal to learn that a US presentation signaled or set the occasion for another trial with a US. Such a mechanism is precluded in many taste aversion experiments because they often use very few conditioning trials. Overall, the results suggest that taste aversion learning is experimentally unique, though not necessarily biologically or evolutionarily unique. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior.

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

Chemogenetic inhibition of the ventral hippocampus but not its direct projection to the prelimbic cortex attenuates context-specific operant responding.

Previous work has demonstrated the importance of the prelimbic cortex (PL) in contextual control of operant behavior. However, the associated neural circuitry responsible for providing contextual information to the PL is not well understood. In Pavlovian fear conditioning the ventral hippocampus (vH) and its projection to the PL have been shown to be important in supporting the effects of context on learning. The present experiments used chemogenetic inhibition of the direct vH-PL projection or the vH to determine involvement in expression of context-specific operant behavior. Rats were injected with an inhibitory DREADD (hM4Di) or mCherry-only into the vH, and subsequently trained to perform a lever press response for a food pellet in a distinct context. The DREADD ligand clozapine-n-oxide (CNO) was then delivered directly into the PL (experiment 1) and then systemically (experiment 2) prior to tests of the response in the training context as well as an equally familiar but untrained context. vH (systemic CNO) but not vH-PL (intra-PL CNO) inhibition was found to attenuate operant responding in its acquisition context. A third experiment, using the same rats, showed that chemogenetic inhibition of vH also reduced Pavlovian contextual fear. The present results suggest that multisynapatic connections between the vH and PL may be responsible for integration of contextual information with operant behavior.

Habit and persistence.

Voluntary behaviors (operants) can come in two varieties: Goal-directed actions, which are emitted based on the remembered value of the reinforcer, and habits, which are evoked by antecedent cues and performed without the reinforcer's value in active memory. The two are perhaps most clearly distinguished with the reinforcer-devaluation test: Goal-directed actions are suppressed when the reinforcer is separately devalued and responding is tested in extinction, and habitual behaviors are not. But what is the function of habit learning? Habits are often thought to be strong and unusually persistent. The present selective review examines this idea by asking whether habits identified by the reinforcer-devaluation test are more resistant to extinction, resistant to the effects of other contingency change, vulnerable to relapse, resistant to the weakening effects of context change, or permanently in place once they are learned. Surprisingly little evidence supports the idea that habits are permanent or more persistent. Habits are more context-specific than goal-directed actions are. Methods that make behavior persistent do not necessarily work by encouraging habit. The function of habit learning may not be to make a behavior strong or more persistent but to make it automatic and efficient in a particular context.

Prelimbic cortex inactivation prevents ABA renewal based on stress state.

Our recent research suggests that the interoceptive state associated with stress can function as a contextual stimulus for operant behavior. In the present experiment, we investigated the role of the rodent prelimbic cortex (PL), a brain region that is critical in contextual control of operant behavior, in the ability of a stressed state to produce ABA renewal of an extinguished operant response. Rats were trained to perform a lever press response for a food pellet reward during daily sessions that followed exposure to a stressor that changed each day. The response was then extinguished in the absence of stress. ABA renewal of extinguished responding occurred following exposure to another stressor (different from any used during acquisition) in control rats but not in rats that received a PL-inactivating infusion (baclofen/muscimol). Results confirm that the interoceptive state of stress can play the role of a contextual stimulus and initiate renewal (relapse) of an inhibited behavior when stress has previously been associated with the behavior. In conjunction with our previous work, the present results support the hypothesis that the PL is important for contexts, both exteroceptive and interoceptive, to exert such control over operant behavior. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Response-specific effects of punishment of a discriminated operant response.

To determine whether the punishment of a discriminated operant behavior has effects that are specific to the punished response, rats were reinforced for performing two different instrumental responses (lever pressing and chain pulling) in the presence of a single discriminative stimulus (S). They were then either punished with mild footshock for performing one of the responses (R1) in S, or they received the same shocks in a noncontingent manner while performing R1 in S (i.e., a yoked control). In final tests of both R1 and R2 in S, the punished rats were more suppressed to R1 than R2, but the yoked rats were not. The results extend previous results with extinction rather than punishment learning (Bouton, Trask, & Carranza-Jasso, 2016) and support a larger parallel between extinction and punishment of both free-operant and discriminated-operant responding. Punishment is like extinction in creating a response-specific inhibition of either free or discriminated operant behavior.

Early-life seizures alter habit behavior formation and fronto-striatal circuit dynamics.

Obsessive compulsive disorder (OCD) can occur comorbidly with epilepsy; both are complex, disruptive disorders that lower quality of life. Both OCD and epilepsy are disorders of hyperexcitable circuits, but it is unclear whether common circuit pathology may underlie the co-occurrence of these two neuropsychiatric disorders. Here, we induced early-life seizures (ELS) in rats to examine habit formation as a model for compulsive behaviors. Compulsive, repetitive behaviors in OCD utilize the same circuitry as habit formation. We hypothesized that rats with ELS could be more susceptible to habit formation than littermate controls, and that altered behavior would correspond to altered signaling in fronto-striatal circuits that underlie decision-making and action initiation. Here, we show instead that rats with ELS were significantly less likely to form habit behaviors compared with control rats. This behavioral difference corresponded with significant alterations to temporal coordination within and between brain regions that underpin the action to habit transition: 1) phase coherence between the lateral orbitofrontal cortex and dorsomedial striatum (DMS) and 2) theta-gamma coupling within DMS. Finally, we used cortical electrical stimulation as a model of transcranial magnetic stimulation (TMS) to show that temporal coordination of fronto-striatal circuits in control and ELS rats are differentially susceptible to potentiating and suppressive stimulation, suggesting that altered underlying circuit physiology may lead to altered response to therapeutic interventions such as TMS.

Prelimbic cortex inactivation prevents ABA renewal based on satiety state.

We have previously shown that the rat prelimbic cortex (PL) is necessary for contexts to promote the performance of instrumental behaviors that have been learned in them, whether the context is physical (operant chamber) or behavioral (recent performance of a behavior that has historically preceded the target in a behavior chain). In the present experiment, we investigated the role of the PL in satiety level as an interoceptive acquisition context. Rats were trained to lever-press for sweet/fat pellets while sated (22 hrs continuous food access) followed by the extinction of the response while hungry (22 hrs food deprived). Pharmacological inactivation of the PL (with baclofen/muscimol infusion) attenuated renewal of the response that occurred upon a return to the sated context. In contrast, animals that received a vehicle (saline) infusion showed renewal of the previously extinguished response. These results support the hypothesis that the PL monitors the relevant contextual elements (physical, behavioral, or satiety state) associated with reinforcement of a response and promotes the subsequent performance of that response in their presence.

Learning to stop responding.

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

A comparison of renewal, spontaneous recovery, and reacquisition after punishment and extinction.

Punishment and extinction are both effective methods of reducing instrumental responding and may involve similar learning mechanisms. To characterize the similarities and differences between them, we examined three well-established recovery or "relapse" effects -renewal, spontaneous recovery, and reacquisition - following either punishment or extinction of an instrumental response. In Experiment 1a, both punished and extinguished responses renewed to similar degrees following a context change at test (ABA renewal). In Experiment 1b, responding spontaneously recovered to similar degrees following punishment or extinction. In Experiment 2, responding was rapidly reacquired when the response was reinforced again following extinction but not following punishment, as predicted by the idea that the reinforcer delivered in reacquisition is part of the context of punishment, but not extinction. The results collectively suggest that both punishment and extinction produce similar context-dependent retroactive interference effects. More broadly, they also suggest that punished and extinguished responses may be equally likely to return following a change of context despite the intuition that punishment might provide a more extreme and effective means of suppressing behavior. To our knowledge, this is the first direct behavioral comparison of response recovery after punishment and extinction within individual experiments.

Inhibition in discriminated operant learning: Tests of response-specificity after feature-negative and extinction learning.

Six experiments with rats examined the nature of inhibition learned in an operant feature-negative (FN) discrimination. The results of prior experiments that examined instrumental extinction rather than FN learning suggest that inhibition can be very specific to the inhibited response. In Experiment 1, we trained lever-press and chain-pull responses in separate but parallel FN discriminations (AR1+, ABR1-, CR2+, and CDR2-) and then tested both inhibitors (B and D) with both responses. Of primary interest was the extent to which the inhibitors suppressed the response they were trained with (same-response inhibition) versus the other response (cross-response inhibition). We found that cross-response inhibition was robust and essentially equal to same-response inhibition. Experiment 2 replicated this result and confirmed stronger inhibition after FN learning than after a differential inhibition procedure (AR1+, BR1-, CR2+, and DR2-). There was also little evidence that cross-response inhibition was due to a demonstrable competing response. Experiment 3 found that cross-response inhibition did not depend on having the two responses reinforced by a common outcome. Experiments 4 and 5 then found that cross-response inhibition depended substantially (though not completely) on the transfer target response having been trained in its own FN discrimination. However, Experiment 6 found that inhibition after instrumental extinction (as opposed to FN learning) was still highly response-specific when the transferred-to response had been trained in an FN discrimination. The overall results suggest that the characteristics of inhibition in instrumental extinction and FN learning differ and that transfer of FN inhibition across responses depends at least partly on previous "inhibitability" of the target response. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Partial reinforcement effects on acquisition and extinction of a conditioned taste aversion.

Four experiments with rat subjects asked whether a partial reinforcement extinction effect (PREE) occurs in taste aversion learning. The question has received little attention in the literature, and to our knowledge no taste aversion experiment has previously demonstrated a PREE. In each of the present experiments, experimental groups received a taste mixed in drinking water for 20 min; such taste exposures were sometimes paired with a lithium chloride (LiCl) injection and sometimes not. Control groups received only taste-LiCl pairings. There was evidence that each reinforced and non-reinforced trial produced increments and decrements in aversion strength (respectively), and trials mattered more than accumulated time during the conditioned stimulus and during the background (as emphasized in time-accumulation models like those of Gallistel and Gibbon, Psychological Review, 107, 289-344, 2000, and Gibbon and Balsam, Autoshaping and conditioning theory, Academic Press, New York, pp. 219-235, 1981). In addition, a partial reinforcement extinction effect was observed when there was a relatively large number of conditioning trials. The results extend our understanding of extinction in taste aversion learning and provide more evidence that aversion learning might follow rules that are qualitatively similar to those of other forms of learning.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior.

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

Context, attention, and the switch between habit and goal-direction in behavior.

This article reviews recent findings from the author's laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) "in mind." Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Effect of context on the instrumental reinforcer devaluation effect produced by taste-aversion learning.

Four experiments manipulated the context in which taste-aversion conditioning occurred when the reinforcer was devalued after instrumental learning. In all experiments, rats learned to lever press in an operant conditioning chamber and then had an aversion to the food-pellet reinforcer conditioned by pairing it with lithium chloride (LiCl) in either that context or a different context. Lever pressing was then tested in extinction to assess its status as a goal-directed action. In Experiment 1, aversion conditioning in the operant conditioning chamber suppressed lever-pressing during the test, but aversion conditioning in the home cage did not. Exposure to the averted pellet in the operant conditioning chamber after conditioning in the home cage did not change this effect (Experiment 2). The same pattern was observed when the different context was a second operant-style chamber (counterbalanced), exposure to the contexts was controlled, and pellets were presented in them in the same manner (Experiment 3). The greater effect of aversion conditioning in the instrumental context was not merely due to potentiated contextual conditioning (Experiment 4). Importantly, consumption tests revealed that the aversion conditioned in the different context had transferred to the test context. Thus, when reinforcer devaluation occurred in a different context, the rats lever pressed in extinction for a reinforcer they would otherwise reject. The results suggest that animals encode contextual information about the reinforcer during instrumental learning and suggest caution in making inferences about action versus habit learning when the reinforcer is devalued in a different context. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Reinforcer predictability and stimulus salience promote discriminated habit learning.

Instrumental (operant) behavior can be goal directed, but after extended practice it can become a habit triggered by environmental stimuli. There is little information, however, about the variables that encourage habit learning, or about the development of discriminated habits that are actually triggered by specific stimuli. (Most studies of habit in animal learning have used free-operant methods.) In the present experiments, rats received training in which a lever press was reinforced only in the presence of a discrete stimulus (S) and the status of the behavior as goal-directed or habitual was determined by reinforcer devaluation tests. Experiment 1 compared lever insertion and an auditory cue (tone) in their ability to support habit learning. Despite prior speculation in the literature, the "salient" lever insertion S was not better than the tone at supporting habit, although the rats learned more rapidly to respond in its presence. Experiment 2 then examined the role of reinforcer predictability with the brief (6-s) tone S. Lever pressing during the tone was reinforced on either every trial or on 50% of trials; habit was observed only with the highly predictable (100%) relationship between S and the reinforcer. Experiment 3 replicated this effect with the tone in a modified procedure and found that lever insertion contrastingly encouraged habit regardless of reinforcer predictability. The results support an interactive role for reinforcer predictability and stimulus salience in discriminated habit learning. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING.

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Renewal of goal direction with a context change after habit learning.

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Correction of response error versus stimulus error in the extinction of discriminated operant learning.

Two experiments with rat subjects separated learning about the discriminative stimulus versus the operant response in the extinction of discriminated operant learning. Each was designed to separate 2 forms of error that could generate extinction learning from an error-correction perspective: Stimulus error, where the discriminative stimulus overpredicts the reinforcer in extinction, and response error, where the response is higher than what the current reinforcer supports. Stimulus error would cause correction of the Pavlovian stimulus-reinforcer association, whereas response error could cause correction of the instrumental response through adjustment of the response-reinforcer association or direct inhibition of the response. Previous research has supported a role for prediction error in instrumental extinction but has confounded these 2 potential sources of error. Using new variations of the concurrent excitor paradigm (Experiment 1) and the overexpectation paradigm (Experiment 2), the present experiments manipulated response error while controlling stimulus error. Both demonstrated that response error plays a role in instrumental extinction. When a discriminated operant response is not reinforced, response error correction may cause the animal to learn to suppress that specific response. (PsycInfo Database Record (c) 2020 APA, all rights reserved).