Resurgence of goal-directed actions and habits.

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

Kamin blocking is disrupted by low-dose ketamine in mice: Further implications for aberrant stimulus processing in schizophrenia.

Previous studies have shown that low doses of ketamine, an N-methyl-D-aspartate receptor antagonist, produce aberrantly strong internal representations of associatively activated but absent stimuli in humans and nonhuman animals, suggesting the validity of ketamine treatment as a preclinical model of the positive symptoms of schizophrenia, including hallucinations and delusions. However, whether acute ketamine treatment also impairs the ability to ignore present but informationally redundant stimuli, which is another hallmark of schizophrenia, remains unclear. Accordingly, the present study investigated whether injections of low-dose ketamine attenuate Kamin blocking in an appetitive conditioning preparation in mice. Mice in the blocking group were initially trained with A+ conditioning (i.e., conditioned stimulus A paired with a sucrose solution), followed by compound AX+ training, before the conditioned responses to the cue X were tested in extinction. The animals in the control group received B+ training before the AX+ training. Half of the mice in each group received an injection of 16 mg/kg ketamine before each compound conditioning session and the extinction test, whereas the other half received saline. The results showed a reliable blocking effect in the saline-treated mice, whereas the blocking effect was absent in the ketamine-treated mice. Specifically, the absence of blocking was due to the ketamine-treated mice learning about the blocked cues. This finding further validates the use of low-dose ketamine as a preclinical model of schizophrenia. It also suggests a possible link between hallucination-like aberrant processing of absent events and a reduced ability to suppress attentional processing of task-irrelevant stimuli. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Transition between habits and goal-directed actions in the renewal effect.

Three experiments with rats explored whether previously extinguished goal-directed and habitual responding recover with the same status using an ABA renewal preparation. In Experiments 1a and 1b, a lever-press response was minimally (four sessions) or extensively (16 sessions) trained in one context (Context A) and extinguished in another context (Context B). Then, outcome devaluation took place in either Context A or Context B in which a food pellet reinforcing the response was paired with lithium chloride (LiCl) for devalued groups and with saline for a control group. Finally, renewal of the extinguished response was tested in both Contexts A and B. We confirmed that both minimally and extensively trained responses renewed as goal-directed action regardless of the context in which devaluation took place. This finding was replicated in Experiment 2 even after more extended acquisition training (32 sessions). However, another group that received outcome devaluation before but not after extinction training showed habitual performance during extinction training as well as in a subsequent renewal test. Experiment 3 replicated these results and confirmed that renewal of goal direction for rats that received extinction training immediately prior to outcome devaluation was not an artifact of consecutive LiCl exposures over a short period of time in Experiments 1 and 2, using a more reliable devaluation protocol. Overall, the present results extend previous findings suggesting that actions and habits renew with the same status by returning to the original context after extinction. The most critical finding is the differential effects of pre- and postextinction devaluation on the expression of habitual behavior; extinction prior to devaluation may convert a habitual performance into a goal-directed action. This novel finding is discussed in relation to recent studies that identified several factors contributing to a transition from habitual to goal-directed control of instrumental behavior. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Ketamine facilitates appetitive trace conditioning in mice: Further evidence for abnormal stimulus representation in schizophrenia model animals.

Recent studies indicated that positive symptoms of schizophrenia, such as hallucination and delusion, can be modeled using Pavlovian conditioning procedures. Various schizophrenia model animals exhibit abnormally strong associative activations of absent stimuli (i.e., conditioned hallucination) and readily form further associations involving the absent cues (i.e., enhanced mediated conditioning). In the present study using mice, we examined whether the acquisition of appetitive trace conditioning, another Pavlovian task in which animals must form associations between two stimuli that never occur together, is facilitated by injections of ketamine, an N-methyl-D-aspartate-receptor antagonist and a known hallucinogen at low doses in humans and nonhuman animals. Ketamine administration before each conditioning session significantly enhanced the acquisition of 4-s trace conditioning but not delay conditioning. The trace conditioning-specific facilitatory effect of ketamine was replicated in subsequent experiments in which slightly modified procedures were used to enhance the overall levels of conditioned responses. Taken together, the current results demonstrated that low-dose ketamine promotes associative learning between stimuli over a temporal gap, which adds to existing literature illustrating aberrant learning involving absent stimuli in schizophrenia model animals. We discuss potential associative mechanisms through which ketamine promoted trace conditioning with reference to Wagner's (1981) Standard Operating Procedures model. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Measurement of the exploration-exploitation response of dogs through a concurrent visual discrimination task.

In many species, the allocation of exploration and exploitation responses to environmental stimuli is important for survival. In this exploratory study, we determined whether dogs (Canis familiaris) explored novel stimuli in a visual discrimination task using food reinforcers. Initially, the dogs were trained with two pairs of simultaneous visual discrimination tasks. Having achieved the learning criterion, the dogs were presented with a pair of stimuli including a novel stimulus and a previously reinforced stimulus in the probe trials (familiar stimulus). Dogs were reinforced by 50% for novel stimuli and 100% for familiar stimuli. The proportions of responses to novel and familiar stimuli in the probe trials were considered to reflect the propensity for exploration and exploitation, respectively. The five dogs tested selected the novel stimulus more frequently (in 22 of the 30 probe trials; binomial test, P = 0.016). Therefore, dogs prefer novel stimuli over familiar ones, suggesting that this species, which is less neophobic than wolves (Canis lupus), would likely allocate more responses to exploration. Comparisons among breeds or with wolves are warranted in future studies.

Higher-Order Conditioning in the Spatial Domain.

Spatial learning and memory, the processes through which a wide range of living organisms encode, compute, and retrieve information from their environment to perform goal-directed navigation, has been systematically investigated since the early twentieth century to unravel behavioral and neural mechanisms of learning and memory. Early theories about learning to navigate space considered that animals learn through trial and error and develop responses to stimuli that guide them to a goal place. According to a trial-and error learning view, organisms can learn a sequence of motor actions that lead to a goal place, a strategy referred to as response learning, which contrasts with place learning where animals learn locations with respect to an allocentric framework. Place learning has been proposed to produce a mental representation of the environment and the cartesian relations between stimuli within it-which Tolman coined the cognitive map. We propose to revisit some of the best empirical evidence of spatial inference in animals, and then discuss recent attempts to account for spatial inferences within an associative framework as opposed to the traditional cognitive map framework. We will first show how higher-order conditioning can successfully account for inferential goal-directed navigation in a variety of situations and then how vectors derived from path integration can be integrated via higher-order conditioning, resulting in the generation of higher-order vectors that explain novel route taking. Finally, implications to cognitive map theories will be discussed.

Spatial inference without a cognitive map: the role of higher-order path integration.

The cognitive map has been taken as the standard model for how agents infer the most efficient route to a goal location. Alternatively, path integration - maintaining a homing vector during navigation - constitutes a primitive and presumably less-flexible strategy than cognitive mapping because path integration relies primarily on vestibular stimuli and pace counting. The historical debate as to whether complex spatial navigation is ruled by associative learning or cognitive map mechanisms has been challenged by experimental difficulties in successfully neutralizing path integration. To our knowledge, there are only three studies that have succeeded in resolving this issue, all showing clear evidence of novel route taking, a behaviour outside the scope of traditional associative learning accounts. Nevertheless, there is no mechanistic explanation as to how animals perform novel route taking. We propose here a new model of spatial learning that combines path integration with higher-order associative learning, and demonstrate how it can account for novel route taking without a cognitive map, thus resolving this long-standing debate. We show how our higher-order path integration (HOPI) model can explain spatial inferences, such as novel detours and shortcuts. Our analysis suggests that a phylogenetically ancient, vector-based navigational strategy utilizing associative processes is powerful enough to support complex spatial inferences.

Spontaneous object-location memory based on environmental geometry is impaired by both hippocampal and dorsolateral striatal lesions.

We examined the role of the hippocampus and the dorsolateral striatum in the representation of environmental geometry using a spontaneous object recognition procedure. Rats were placed in a kite-shaped arena and allowed to explore two distinctive objects in each of the right-angled corners. In a different room, rats were then placed into a rectangular arena with two identical copies of one of the two objects from the exploration phase, one in each of the two adjacent right-angled corners that were separated by a long wall. Time spent exploring these two objects was recorded as a measure of recognition memory. Since both objects were in different locations with respect to the room (different between exploration and test phases) and the global geometry (also different between exploration and test phases), differential exploration of the objects must be a result of initial habituation to the object relative to its local geometric context. The results indicated an impairment in processing the local geometric features of the environment for both hippocampus and dorsolateral striatum lesioned rats compared with sham-operated controls, though a control experiment showed these rats were unimpaired in a standard object recognition task. The dorsolateral striatum has previously been implicated in egocentric route-learning, but the results indicate an unexpected role for the dorsolateral striatum in processing the spatial layout of the environment. The results provide the first evidence that lesions to the hippocampus and dorsolateral striatum impair spontaneous encoding of local environmental geometric features.

En route to delineating hippocampal roles in spatial learning.

The precise role played by the hippocampus in spatial learning tasks, such as the Morris Water Maze (MWM), is not fully understood. One theory is that the hippocampus is not required for 'knowing where' but rather is crucial in 'getting there'. To explore this idea in the MWM, we manipulated 'getting there' variables, such as passive transport or active swimming towards the hidden platform, in rats with and without hippocampal lesions. Our results suggested that for intact rats, self-motion cues enroute to the hidden goal were a necessary component for 'place learning' to progress. Specifically, intact rats could not learn the hidden goal location, when passively transported to it, despite extensive training. However, when rats were either given hippocampal lesions, or placed in a light-tight box during transportation to the hidden goal, passive-placement spatial learning was facilitated. In a subsequent experiment, the 'getting there' component of place navigation was simplified, via the placement of two overhead landmarks, one of which served as a beacon. When 'getting there' was made easier in this way, hippocampal lesions did not induce deficits in 'knowing where' the goal was. In fact, similar to the facilitation observed in passive-placement spatial learning, hippocampal lesions improved landmark learning relative to controls. Finally, demonstrating that our lesions were sufficiently deleterious, hippocampal-lesioned rats were impaired, as predicted, in an environmental-boundary based learning task. We interpret these results in terms of competition between multiple memory systems, and the importance of self-generated motion cues in hippocampal spatial mapping.

The response strategy and the place strategy in a plus-maze have different sensitivities to devaluation of expected outcome.

Previous studies have suggested that spatial navigation can be achieved with at least two distinct learning processes, involving either cognitive map-like representations of the local environment, referred to as the "place strategy", or simple stimulus-response (S-R) associations, the "response strategy". A similar distinction between cognitive/behavioral processes has been made in the context of non-spatial, instrumental conditioning, with the definition of two processes concerning the sensitivity of a given behavior to the expected value of its outcome as well as to the response-outcome contingency ("goal-directed action" and "S-R habit"). Here we investigated whether these two versions of dichotomist definitions of learned behavior, one spatial and the other non-spatial, correspond to each other in a formal way. Specifically, we assessed the goal-directed nature of two navigational strategies, using a combination of an outcome devaluation procedure and a spatial probe trial frequently used to dissociate the two navigational strategies. In Experiment 1, rats trained in a dual-solution T-maze task were subjected to an extinction probe trial from the opposite start arm, with or without prefeeding-induced devaluation of the expected outcome. We found that a non-significant preference for the place strategy in the non-devalued condition was completely reversed after devaluation, such that significantly more animals displayed the use of the response strategy. The result suggests that the place strategy is sensitive to the expected value of the outcome, while the response strategy is not. In Experiment 2, rats with hippocampal lesions showed significant reliance on the response strategy, regardless of whether the expected outcome was devalued or not. The result thus offers further evidence that the response strategy conforms to the definition of an outcome-insensitive, habitual form of instrumental behavior. These results together attest a formal correspondence between two types of dual-process accounts of animal learning and behavior.

Striatonigral direct pathway activation is sufficient to induce repetitive behaviors.

Pharmacological intervention in the substantia nigra is known to induce repetitive behaviors in rodents, but a direct causal relationship between a specific neural circuit and repetitive behavior has not yet been established. Here we demonstrate that optogenetic activation of dopamine D1 receptor-expressing MSNs terminals in the substantia nigra pars reticulata resulted in sustained and chronic repetitive behaviors. These data show for the first time that activation of the striatonigral direct pathway is sufficient to generate motor stereotypies.

Mice lacking hippocampal left-right asymmetry show non-spatial learning deficits.

Left-right asymmetry is known to exist at several anatomical levels in the brain and recent studies have provided further evidence to show that it also exists at a molecular level in the hippocampal CA3-CA1 circuit. The distribution of N-methyl-d-aspartate (NMDA) receptor NR2B subunits in the apical and basal synapses of CA1 pyramidal neurons is asymmetrical if the input arrives from the left or right CA3 pyramidal neurons. In the present study, we examined the role of hippocampal asymmetry in cognitive function using ß2-microglobulin knock-out (ß2m KO) mice, which lack hippocampal asymmetry. We tested ß2m KO mice in a series of spatial and non-spatial learning tasks and compared the performances of ß2m KO and C57BL6/J wild-type (WT) mice. The ß2m KO mice appeared normal in both spatial reference memory and spatial working memory tasks but they took more time than WT mice in learning the two non-spatial learning tasks (i.e., a differential reinforcement of lower rates of behavior (DRL) task and a straight runway task). The ß2m KO mice also showed less precision in their response timing in the DRL task and showed weaker spontaneous recovery during extinction in the straight runway task. These results indicate that hippocampal asymmetry is important for certain characteristics of non-spatial learning.

Impaired Pavlovian predictive learning between temporally phasic but not static events in autism-model strain mice.

Autism-spectrum disorder (ASD) is a multi-aspect developmental disorder characterised by various social and non-social behavioural abnormalities. Using BTBR T+ tf mouse strain (BTBR), a promising animal model displaying a number of behavioural and neural characteristics associated with ASD, we tested the hypothesis that at the core of various symptoms of ASD lies a fundamental deficit in predictive learning between events. In five experiments, we conducted a variety of Pavlovian conditioning tasks, some requiring the establishment of associations between temporally phasic events and others involving static events. BTBR mice were impaired in the acquisition of conditioned magazine approach responses with an appetitive unconditioned stimulus (US) (Experiment 1) and conditioned freezing with an electric shock US (Experiment 2). Both of these tasks had temporally phasic conditioned stimuli (CSs). Conversely, these mice showed normal acquisition of conditioned place preference (CPP), whether the US was a systemic injection of methamphetamine (Experiment 3A) or the presence of food (Experiment 3B). Experiment 4 showed normal acquisition of conditioned taste aversion (CTA) to a flavour-taste compound CS, although BTBR mice still exhibited an abnormal stimulus selection when learning for each element of the compound CS was assessed separately. Experiment 5 revealed a weaker latent inhibition of CTA in BTBR mice. The BTBR mouse's impaired predictive learning between phasic events and intact associations between static events are discussed in terms of dysfunctional contingency-based, but not contiguity-based learning, which may accompany abnormal selective attention to relevant cues. We propose that such dysfunctional contingency learning mechanisms may underlie the development of various social and non-social symptoms of ASD.

Conditioned social preference, but not place preference, produced by intranasal oxytocin in female mice.

Oxytocin (OT) has been implicated in a variety of mammalian reproductive and social behaviors, and the use of intranasal OT for clinical purposes is on the rise. However, basic actions of OT, including the rewarding or reinforcing properties of the drug, are currently not fully understood. In this study, the authors investigated whether intranasally administered OT has different reinforcing properties for social and nonsocial stimuli and whether such effects are variable between male and female subjects. Conditioned social preference (CSP) and conditioned place preference (CPP) paradigms were used to examine social and nonsocial reinforcing properties of OT. In CSP, the presence of a same-sex unfamiliar conspecific was repeatedly paired with intranasal OT, while a different conspecific was associated with saline. The reinforcing effect of OT was assessed in a postconditioning choice test under a drug-free condition. In CPP, the 2 conspecifics were replaced with nonsocial black and white compartments. The authors found that intranasal OT (12 µg) in females supported the formation of CSP (Experiment 1) but not CPP (Experiment 3). Neither CSP (Experiment 2) nor CPP (Experiment 4) was formed in males. Extended conditioning with higher dose OT (36 µg), however, abolished the initial CSP in females and produced an aversion to the OT-paired stimulus mouse. Experiment 5 indicated that it was the repeated administrations rather than the higher dose that produced the abolition of the original preference. Overall, the current results demonstrate for the first time a sex- and stimulus-dependent reinforcing property of intranasal OT in mice. (PsycINFO Database Record

Asymmetrical generalization of length in the rat.

Two groups of rats in Experiment 1 were required to escape from a square pool by swimming to 1 of 2 submerged platforms that were situated beside the centers of 2 opposite walls. To help rats find a platform, black panels of equal width were pasted to the middle of the walls that were adjacent to the platforms. The width of the 2 panels was 50 cm for Group 50, and 100 cm for Group 100. Test trials were then conducted in the same pool, but with the platforms removed and with a 50-cm panel on 1 wall and a 100-cm panel on the opposite wall. Group 50 expressed a stronger preference for the 100-cm than the 50-cm panel during the test, whereas Group 100 expressed a similar preference for both panels. Thus the degree of generalization from the short to the long panel was greater than from the long to the short panel. Experiments 2 and 3 pointed to the same conclusion. They were of a similar design to Experiment 1, except that the lengths of the panels for the 2 groups were 25 and 50 cm in Experiment 2, and 25 and 100 cm in Experiment 3. The results are explained by assuming the original training results in the walls without black panels entering into inhibitory associations. This inhibition is then assumed to generalize more to the short than the long test panels and thereby result in an asymmetry in the gradients of generalization between the different lengths. (PsycINFO Database Record

Evidence for concrete but not abstract representation of length during spatial learning in rats.

In 4 experiments, rats had to discriminate between the lengths of 2 objects of the same color, black or white, before a test trial with the same objects but of opposite color. The experiments took place in a pool from which rats had to escape by swimming to 1 of 2 submerged platforms. For Experiments 1 and 2, the platforms were situated near the centers of panels of 1 length, but not another, that were pasted onto the gray walls of a square arena. The acquired preference for the correct length was eliminated by changing the color of the panels. In Experiment 3, the platforms were situated near the middle of the long walls of a rectangular pool, and in Experiment 4 they were situated in 1 pair of diagonally opposite corners of the same pool. Changing the color of the walls markedly disrupted the effects of the original training in both experiments. The results indicate that rats represent the length of objects not by their abstract, geometric attributes but in a more concrete fashion such as by a mental snapshot or by the amount of color stimulation they provide.

Dorsolateral striatal lesions impair navigation based on landmark-goal vectors but facilitate spatial learning based on a "cognitive map".

In three experiments, the nature of the interaction between multiple memory systems in rats solving a variation of a spatial task in the water maze was investigated. Throughout training rats were able to find a submerged platform at a fixed distance and direction from an intramaze landmark by learning a landmark-goal vector. Extramaze cues were also available for standard place learning, or "cognitive mapping," but these cues were valid only within each session, as the position of the platform moved around the pool between sessions together with the intramaze landmark. Animals could therefore learn the position of the platform by taking the consistent vector from the landmark across sessions or by rapidly encoding the new platform position on each session with reference to the extramaze cues. Excitotoxic lesions of the dorsolateral striatum impaired vector-based learning but facilitated cognitive map-based rapid place learning when the extramaze cues were relatively poor (Experiment 1) but not when they were more salient (Experiments 2 and 3). The way the lesion effects interacted with cue availability is consistent with the idea that the memory systems involved in the current navigation task are functionally cooperative yet associatively competitive in nature.

The role of the hippocampus in passive and active spatial learning.

Rats with lesions of the hippocampus or sham lesions were required in four experiments to escape from a square swimming pool by finding a submerged platform. Experiments 1 and 2 commenced with passive training in which rats were repeatedly placed on the platform in one corner-the correct corner-of a pool with distinctive walls. A test trial then revealed a strong preference for the correct corner in the sham but not the hippocampal group. Subsequent active training of being required to swim to the platform resulted in both groups acquiring a preference for the correct corner in the two experiments. In Experiments 3 and 4, rats were required to solve a discrimination between different panels pasted to the walls of the pool, by swimming to the middle of a correct panel. Hippocampal lesions prevented a discrimination being formed between panels of different lengths (Experiment 3), but not between panels showing lines of different orientations (Experiment 4); rats with sham lesions mastered both problems. It is suggested that an intact hippocampus is necessary for the formation of stimulus-goal associations that permit successful passive spatial leaning. It is further suggested that an intact hippocampus is not necessary for the formation of stimulus-response associations, except when they involve information about length or distance.

Within-compound associations explain potentiation and failure to overshadow learning based on geometry by discrete landmarks.

In three experiments, rats were trained to locate a submerged platform in one of the base corners of a triangular arena above each of which was suspended one of two distinctive landmarks. In Experiment 1, it was established that these landmarks differed in their salience by the differential control they gained over behavior after training in compound with geometric cues. In Experiment 2, it was shown that locating the platform beneath the less salient landmark potentiated learning based on geometry compared with control rats for which landmarks provided ambiguous information about the location of the platform. The presence of the more salient landmark above the platform for another group of animals appeared to have no effect on learning based on geometry. Experiment 3 established that these landmark and geometry cues entered into within-compound associations during compound training. We argue that these within-compound associations can account for the potentiation seen in Experiment 2, as well as previous failures to demonstrate overshadowing of geometric cues. We also suggest that these within-compound associations need not be of different magnitudes, despite the different effects of each of the landmarks on learning based on geometry seen in Experiment 2. Instead, within-compound associations appear to mitigate the overshadowing effects that traditional theories of associative learning would predict.

Asymmetry in the discrimination of length during spatial learning.

The ability of rats to solve a discrimination between two objects that differ in length was investigated in five experiments. Using a rectangular swimming pool, Experiment 1 revealed it is easier to locate a submerged platform when it is near the center of a long rather than a short wall. For Experiments 2-4, the objects were black or white panels pasted onto the gray walls of a square pool, with two long panels pasted to two opposing walls and two short panels pasted to the remaining walls. The platform was easier to locate when it was placed near the middle of a long rather than a short panel. This effect was found when the long panels were twice (Experiments 2-4) or four times the length of the short panels (Experiment 4). Experiment 5 demonstrated that rats can solve a discrimination between panels of length 15 and 45 cm more readily than when they are 70 and 100 cm. The results are consistent with the claim that generalization gradients based on stimulus magnitude are steeper for stimuli that are weaker rather than stronger than the stimulus used for the original training.