Contribution of dorsal versus ventral hippocampus to the hierarchical modulation of goal-directed actions in rats.

Adaptive behaviour often necessitates that animals learn about events in a manner that is specific to a particular context or environment. These hierarchical organisations allow the animal to decide which action is the most appropriate when faced with ambiguous or conflicting possibilities. This study examined the role of hippocampus in enabling animals to use the context to guide action selection. We used a hierarchical instrumental outcome devaluation task in which male rats learn that the context provides information about the unique action-outcome relations that are in effect. We first confirmed that rats encode and use hierarchical context-(action-outcome) relations. We then show that chemogenetic inhibition of ventral hippocampus impairs both the encoding and retrieval of these associations, while inhibition of dorsal hippocampus impairs only the retrieval. Importantly, neither dorsal nor ventral hippocampus was required for goal-directed behaviour per se as these impairments only emerged when rats were forced to use the context to identify the current action-outcome relationships. These findings are discussed with respect to the role of the hippocampus and its broader circuitry in the contextual modulation of goal-directed behaviour and the importance of hierarchical associations in flexible behaviour.

Insular and Ventrolateral Orbitofrontal Cortices Differentially Contribute to Goal-Directed Behavior in Rodents.

The medial prefrontal cortex (mPFC) has long been considered a critical site in action control. However, recent evidence indicates that the contribution of cortical areas to goal-directed behavior likely extends beyond mPFC. Here, we examine the function of both insular (IC) and ventrolateral orbitofrontal (vlOFC) cortices in action-dependent learning. We used chemogenetics to study the consequences of IC or vlOFC inhibition on acquisition and performance of instrumental actions using the outcome devaluation task. Rats first learned to associate actions with desirable outcomes. Then, one of these outcomes was devalued and we assessed the rats' choice between the 2 actions. Typically, rats will bias their selection towards the action that delivers the still valued outcome. We show that chemogenetic-induced inhibition of IC during choice abolishes goal-directed control whereas inhibition during instrumental acquisition is without effect. IC is therefore necessary for action selection based on current outcome value. By contrast, vlOFC inhibition during acquisition or the choice test impaired goal-directed behavior but only following a shift in the instrumental contingencies. Our results provide clear evidence that vlOFC plays a critical role in action-dependent learning, which challenges the popular idea that this region of OFC is exclusively involved in stimulus-dependent behaviors.

Intermittent feeding alters sensitivity to changes in reward value.

The influence of binge-like feeding schedules on subsequent food-related behavior is not well understood. We investigated the effect of repeated cycles of restriction and refeeding on two food-related behaviors; goal-directed responding for a palatable food reward and sensory-specific satiety. Hungry rats were trained to perform two instrumental actions for two distinct food outcomes and were then subjected to repeated cycles of restricted and unrestricted access to their maintenance chow for 30-days or were maintained on food restriction. Goal-directed control was then assessed using specific satiety-induced outcome devaluation. Rats were given 1 h access to one of theoutcomes and were then immediately given a choice between the two actions. Rats maintained on restriction responded more for the valued than the devalued reward but rats with a history of restriction and refeeding failed to show this effect. Importantly, all rats showed sensory-specific satiety when offered a choice between the two foods, indicating that pre-feeding selectively reduced the value of the pre-fed food. By contrast, sensory-specific satiety was not observed in rats with a history of intermittent feeding when the foods were offered sequentially. These results indicate that, similar to calorically dense diets, intermittent feeding patterns can impair the performance of goal-directed actions as well as the ability to reject a pre-fed food when it is offered alone.

The effect of high-fat diet consumption on appetitive instrumental behavior in rats.

Evidence now indicates that the chronic consumption of high-calorie foods, such as a high-fat diet (HFD), is associated with impaired control over food-seeking, yet the extent of this alteration is not fully understood. Using different reinforcement schedules, we evaluated whether HFD intake from weaning to adulthood modifies instrumental responding and induces a shift from goal-directed actions to habitual responding. We first observed reduced instrumental performance and motivation for a food reward in HFD-fed rats trained under schedules of reinforcement that facilitate habitual responding [Random Interval (RI)]. However, this deficit was alleviated if rats trained under RI were subsequently trained with reinforcement schedules that promote goal-directed strategies [Random Ratio (RR)]. Using an outcome devaluation procedure, we then demonstrated that consumption of a HFD promoted habitual behavior in rats trained under RI but not RR schedules. Finally, extended HFD exposure did not interfere with the ability of RR training to overcome impaired RI instrumental performance and to favor goal-directed behavior. These results indicate that chronic consumption of a HFD changes the co-ordination of goal-directed actions and habits and that alteration of food-seeking may be reversed under particular behavioral conditions.

Interaction of insular cortex and ventral striatum mediates the effect of incentive memory on choice between goal-directed actions.

The anterior insular cortex (IC) and the nucleus accumbens (NAc) core have been separately implicated in the selection and performance of actions based on the incentive value of the instrumental outcome. Here, we examined the role of connections between the IC and the NAc core in the performance of goal-directed actions. Rats were trained on two actions for distinct outcomes, after which one of the two outcomes was devalued by specific satiety immediately before a choice extinction test. We first confirmed the projection from the IC to the NAc core and then disconnected these structures via asymmetrical excitotoxic lesions before training. Contralateral, but not ipsilateral, disconnection of the IC and NAc core disrupted outcome devaluation. We hypothesized that communication between the IC and NAc core is necessary for the retrieval of incentive value at test. To test this, we infused the GABAA agonist muscimol into the IC and the µ-opioid receptor antagonist CTAP into the contralateral NAc before the choice extinction test. As expected, inactivation of the IC in one hemisphere and blocking µ-opioid receptors in the contralateral NAc core abolished outcome-selective devaluation. These results suggest that the IC and NAc core form part of a circuit mediating the retrieval of outcome values and the subsequent choice between goal-directed actions based on those values.

Microglial activation enhances associative taste memory through purinergic modulation of glutamatergic neurotransmission.

The cerebral innate immune system is able to modulate brain functioning and cognitive processes. During activation of the cerebral innate immune system, inflammatory factors produced by microglia, such as cytokines and adenosine triphosphate (ATP), have been directly linked to modulation of glutamatergic system on one hand and learning and memory functions on the other hand. However, the cellular mechanisms by which microglial activation modulates cognitive processes are still unclear. Here, we used taste memory tasks, highly dependent on glutamatergic transmission in the insular cortex, to investigate the behavioral and cellular impacts of an inflammation restricted to this cortical area in rats. We first show that intrainsular infusion of the endotoxin lipopolysaccharide induces a local inflammation and increases glutamatergic AMPA, but not NMDA, receptor expression at the synaptic level. This cortical inflammation also enhances associative, but not incidental, taste memory through increase of glutamatergic AMPA receptor trafficking. Moreover, we demonstrate that ATP, but not proinflammatory cytokines, is responsible for inflammation-induced enhancement of both associative taste memory and AMPA receptor expression in insular cortex. In conclusion, we propose that inflammation restricted to the insular cortex enhances associative taste memory through a purinergic-dependent increase of glutamatergic AMPA receptor expression at the synapse.

Acquisition of specific response-outcome associations requires NMDA receptor activation in the basolateral amygdala but not in the insular cortex.

The basolateral amygdala (BLA) and the gustatory region of the insular cortex (IC) are required for the encoding and retrieval of outcome value. Here, we examined if these regions are also necessary to learn associations between actions and their outcomes. Hungry rats were first trained to press two levers for a common outcome. Next, specific response-outcome (R-O) associations were introduced such that each response now earned a distinct food outcome. Prior to each specific R-O training session, rats received a bilateral infusion of the N-methyl-D-aspartate (NMDA) receptor antagonist, DL-APV, into either the BLA or the IC. One of the two outcomes was then devalued immediately prior to a choice test. Inhibition of NMDA receptor activity in the BLA, but not the IC, during the acquisition of specific R-O associations abolished selective devaluation. These results indicate that the BLA is critical for learning the association between actions and their specific consequences.

A time course analysis of satiety-induced instrumental outcome devaluation.

Sensory-specific satiety is commonly used in studies of decision making to selectively devalue a food reward. Devaluation is reflected in an immediate reduction in the subsequent intake of the food and in the performance of actions that gain access to that food. Despite its frequent use, the lasting effects of satiety-induced devaluation on instrumental actions are unknown. Here, we examined the time course and contextual dependency of sensory-specific satiety-induced devaluation on instrumental responding and consumption. Rats were trained to perform two instrumental actions for two distinct food rewards. Then, one of the instrumental outcomes was provided ad libitum for 1 hour in separate feeding cages and the effect of this devaluation was assessed 0, 2, or 5 hours after satiation. At a delay of 0 or 2 hours, both intake and instrumental responding were sensitive to the satiety treatment. That is, rats consumed less of the devalued outcome and responded less for the devalued outcome than for the valued outcome. By contrast, after 5 hours, rats showed sensitivity to devaluation in consumption but not in instrumental responding. Strikingly, sensitivity to devaluation was restored for the instrumental response after a 5 hour delay when devaluation was performed in the instrumental context. These results indicate that, in rats, specific satiety-induced devaluation endures and is context-independent for up to 2 hours post-satiation. At longer delays, the impact of sensory-specific satiety on instrumental responding is context-dependent, suggesting that contextual cues may be required for the value of specific outcomes to control instrumental responding.

Incentive memory: evidence the basolateral amygdala encodes and the insular cortex retrieves outcome values to guide choice between goal-directed actions.

Choice between goal-directed actions is determined by the relative value of their consequences. Such values are encoded during incentive learning and later retrieved to guide performance. Although the basolateral amygdala (BLA) and the gustatory region of insular cortex (IC) have been implicated in these processes, their relative contribution is still a matter of debate. Here we assessed whether these structures interact during incentive learning and retrieval to guide choice. In these experiments, rats were trained on two actions for distinct outcomes after which one of the two outcomes was devalued by specific satiety immediately before a choice extinction test. We first confirmed that, relative to appropriate controls, outcome devaluation recruited both the BLA and IC based on activation of the immediate early gene Arc; however, we found that infusion of the NMDAr antagonist ifenprodil into the BLA only abolished outcome devaluation when given before devaluation. In contrast, ifenprodil infusion into the IC was effective whether made before devaluation or test. We hypothesized that the BLA encodes and the IC retrieves incentive value for choice and, to test this, developed a novel sequential disconnection procedure. Blocking NMDAr activation unilaterally in the BLA before devaluation and then contralaterally in the IC before test abolished selective devaluation. In contrast, reversing the order of these infusions left devaluation intact. These results confirm that the BLA and IC form a circuit mediating the encoding and retrieval of outcome values, with the BLA encoding and the IC retrieving such values to guide choice.

The basolateral amygdala is critical for learning about neutral stimuli in the presence of danger, and the perirhinal cortex is critical in the absence of danger.

The perirhinal cortex (PRh) and basolateral amygdala (BLA) appear to mediate distinct aspects of learning and memory. Here, we used rats to investigate the involvement of the PRh and BLA in acquisition and extinction of associations between two different environmental stimuli (e.g., a tone and a light) in higher-order conditioning. When both stimuli were neutral, infusion of the GABAA, muscimol, or the NMDA receptor (NMDAR) antagonist ifenprodil into the PRh impaired associative formation. However, when one stimulus was neutral and the other was a learned danger signal, acquisition and extinction of the association between them was unaffected by manipulations targeting the PRh. Temporary inactivation of the BLA had the opposite effect: formation and extinction of an association between two stimuli was spared when both stimuli were neutral, but impaired when one stimulus was a learned danger signal. Subsequent experiments showed that the experience of fear per se shifts processing of an association between neutral stimuli from the PRh to the BLA. When training was conducted in a dangerous environment, formation and extinction of an association between neutral stimuli was impaired by BLA inactivation or NMDAR blockade in this region, but was unaffected by PRh inactivation. These double dissociations in the roles of the PRh and BLA in learning under different stimulus and environmental conditions imply that fear-induced activation of the amygdala changes how the brain processes sensory stimuli. Harmless stimuli are treated as potentially harmful, resulting in a shift from cortical to subcortical processing in the BLA.

Differential role of insular cortex muscarinic and NMDA receptors in one-trial appetitive taste learning.

Our current understanding of the neurobiology of taste learning and memory has been greatly facilitated by the use of a reliable behavioural model, conditioned taste aversion (CTA). This model has revealed that the insular cortex (IC), specifically muscarinic and N-methyl-d-aspartate (NMDA) receptor activation in the IC, is critical for the formation of aversive taste memories. In contrast, current models of appetitive taste learning are less adequate, relying on the use of neophobic tastes (attenuation of neophobia) or on the integration of appetitive and aversive taste memories (latent inhibition of CTA). While these models have implicated IC muscarinic receptors, the involvement of NMDA receptors in the IC remains unclear. Here, we examined the role of both muscarinic and NMDA receptors in appetitive taste learning using a simple paradigm that is independent of neophobic and aversive components. First, we demonstrated that a single exposure to a novel taste, saccharin 0.1%, is sufficient to promote an appetitive taste memory as revealed by an increase in saccharin consumption during the second presentation. This increase was blocked by bilateral infusion in the IC of the muscarinic receptor antagonist, scopolamine. In contrast, infusion of the NMDA receptor antagonist, AP5, did not block appetitive taste learning but did abolish CTA. Therefore, common and distinct molecular substrates within the IC mediate appetitive versus aversive learning about the same taste.

A hippocampo-cortical pathway detects changes in the validity of an action as a predictor of reward.

Much research has been dedicated to understanding the psychological and neural bases of goal-directed action, yet the relationship between context and goal-directed action is not well understood. Here, we used excitotoxic lesions, chemogenetics, and circuit-specific manipulations to demonstrate the role of the ventral hippocampus (vHPC) in contextual learning that supports sensitivity to action-outcome contingencies, a hallmark of goal-directed action. We found that chemogenetic inhibition of the ventral, but not dorsal, hippocampus attenuated sensitivity to instrumental contingency degradation. We then tested the hypothesis that this deficit was due to an inability to discern the relative validity of the action compared with the context as a predictor of reward. Using latent inhibition and Pavlovian context conditioning, we confirm that degradation of action-outcome contingencies relies on intact context-outcome learning and show that this learning is dependent on vHPC. Finally, we show that chemogenetic inhibition of vHPC terminals in the medial prefrontal cortex also impairs both instrumental contingency degradation and context-outcome learning. These results implicate a hippocampo-cortical pathway in adapting to changes in instrumental contingencies and indicate that the psychological basis of this deficit is an inability to learn the predictive value of the context. Our findings contribute to a broader understanding of the neural bases of goal-directed action and its contextual regulation.

Pathway specific interventions reveal the multiple roles of ventral hippocampus projections in cognitive functions.

Since the 1950s study of Scoville and Milner on the case H.M., the hippocampus has attracted neuroscientists' attention. The hippocampus has been traditionally divided into dorsal and ventral parts, each of which projects to different brain structures and mediates various functions. Despite a predominant interest in its dorsal part in animal models, especially regarding episodic-like and spatial cognition, recent data highlight the role of the ventral hippocampus (vHPC), as the main hippocampal output, in cognitive processes. Here, we review recent studies conducted in rodents that have used advanced in vivo functional techniques to specifically monitor and manipulate vHPC efferent pathways and delineate the roles of these specific projections in learning and memory processes. Results highlight that vHPC projections to basal amygdala are implicated in emotional memory, to nucleus accumbens in social memory and instrumental actions and to prefrontal cortex in all the above as well as in object-based memory. Some of these hippocampal projections also modulate feeding and anxiety-like behaviours providing further evidence that the "one pathway-one function" view is outdated and future directions are proposed to better understand the role of hippocampal pathways and shed further light on its connectivity and function.

Inhibition of noradrenergic signalling in rodent orbitofrontal cortex impairs the updating of goal-directed actions.

In a constantly changing environment, organisms must track the current relationship between actions and their specific consequences and use this information to guide decision-making. Such goal-directed behaviour relies on circuits involving cortical and subcortical structures. Notably, a functional heterogeneity exists within the medial prefrontal, insular, and orbitofrontal cortices (OFC) in rodents. The role of the latter in goal-directed behaviour has been debated, but recent data indicate that the ventral and lateral subregions of the OFC are needed to integrate changes in the relationships between actions and their outcomes. Neuromodulatory agents are also crucial components of prefrontal functions and behavioural flexibility might depend upon the noradrenergic modulation of the prefrontal cortex. Therefore, we assessed whether noradrenergic innervation of the OFC plays a role in updating action-outcome relationships in male rats. We used an identity-based reversal task and found that depletion or chemogenetic silencing of noradrenergic inputs within the OFC rendered rats unable to associate new outcomes with previously acquired actions. Silencing of noradrenergic inputs in the prelimbic cortex or depletion of dopaminergic inputs in the OFC did not reproduce this deficit. Together, our results suggest that noradrenergic projections to the OFC are required to update goal-directed actions.

The basolateral amygdala is critical for the acquisition and extinction of associations between a neutral stimulus and a learned danger signal but not between two neutral stimuli.

We studied the neural substrates of higher-order conditioned fear in rats. We first studied acquisition and extinction of second-order fear. One stimulus (S1) was paired with a footshock unconditioned stimulus (US), a second stimulus (S2) was paired with S1, and, finally, fear of S2 was extinguished by S2 alone exposures. Reversible inactivation of the basolateral amygdala (BLA) via muscimol, systemic injection of the NMDA receptor (NMDAr) antagonist MK-801 [(+)-5-methyl-10,11-dihydro-5H-dibenzo [a,d] cyclohepten-5,10-imine maleate], or intra-amygdala infusion of the NMDAr NR2B subunit selective antagonist ifenprodil impaired both acquisition and extinction of fear to S2. We then studied acquisition and extinction of sensory preconditioned fear. S2 was first paired with S1, S1 was then paired with the US, and, finally, fear of S2 was extinguished by S2 alone exposures. Extinction of sensory preconditioned fear was impaired by a BLA infusion of muscimol or ifenprodil and by MK-801. Acquisition of the S2-S1 association and extinction of this association by S2 alone exposures before conditioning of S1 was impaired by MK-801 but not by a BLA infusion of muscimol. These results show that NMDAr activation in the BLA is critical for acquisition of second-order conditioned fear and for the extinction of both second-order and sensory preconditioned fear just as it is for acquisition and extinction of first-order conditioned fear. Acquisition and extinction of an association between two neutral stimuli also require NMDAr activation. However, the present results show that the acquisition and extinction of these associations do not require the BLA.

Role of the basolateral amygdala and NMDA receptors in higher-order conditioned fear.

Laboratory rats learn to fear relatively innocuous stimuli which signal the imminent arrival of an innate source of danger, typically brief but aversive foot shock. Much is now known about the neural substrates underlying the acquisition, consolidation and subsequent expression of this fear. Rats also learn to fear stimuli which signal learned sources of danger but relatively little is known about the neural substrates underlying this form of fear. Two Pavlovian conditioning paradigms used to study this form of fear are second-order conditioning and sensory preconditioning. In second-order conditioning, rats are first exposed to a signaling relationship between one stimulus, such as a tone, and aversive foot shock, and then to a signaling relationship between a second stimulus, such as a light, and the now dangerous tone. In sensory preconditioning, these phases are reversed: rats are first exposed to a signaling relationship between the light and the tone and then to a signaling relationship between the tone and the foot shock. In both paradigms, rats exhibit fear when tested with the light. In this review paper, we describe the evidence for higher-order forms of conditioning, the conditions which promote this learning and its contents. We compare and contrast the substrates of the learning underlying second-order and sensory preconditioning fear with those known to underlie the better studied first-order conditioned fear. We conclude with some comments as to the role of higher-order processes in anxiety disorders.

Dopamine and noradrenaline modulation of goal-directed behavior in orbital and medial prefrontal cortex: Toward a division of labor?

The prefrontal cortex is considered to be at the core of goal-directed behaviors. Notably, the medial prefrontal cortex (mPFC) is known to play an important role in learning action-outcome (A-O) associations, as well as in detecting changes in this contingency. Previous studies have also highlighted a specific engagement of the dopaminergic pathway innervating the mPFC in adapting to changes in action causality. While previous research on goal-directed actions has primarily focused on the mPFC region, recent findings have revealed a distinct and specific role of the ventral and lateral orbitofrontal cortex (vlOFC). Indeed, vlOFC is not necessary to learn about A-O associations but appears specifically involved when outcome identity is unexpectedly changed. Unlike the mPFC, the vlOFC does not receive a strong dopaminergic innervation. However, it receives a dense noradrenergic innervation which might indicate a crucial role for this neuromodulator. In addition, several lines of evidence highlight a role for noradrenaline in adapting to changes in the environment. We, therefore, propose that the vlOFC's function in action control might be under the strong influence of the noradrenergic system. In the present article, we review anatomical and functional evidence consistent with this proposal and suggest a direction for future studies that aim to shed light on the orbitofrontal mechanisms for flexible action control. Specifically, we suggest that dopaminergic modulation in the mPFC and noradrenergic modulation in the vlOFC may underlie distinct processes related to updating one's actions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents.

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Targeting Reciprocally Connected Brain Regions Through CAV-2 Mediated Interventions.

An important issue in contemporary neuroscience is to identify functional principles at play within neural circuits. The reciprocity of the connections between two distinct brain areas appears as an intriguing feature of some of these circuits. This organization has been viewed as "re-entry," a process whereby two or more brain regions concurrently stimulate and are stimulated by each other, thus supporting the synchronization of neural firing required for rapid neural integration. However, until relatively recently, it was not possible to provide a comprehensive functional assessment of such reciprocal pathways. In this Brief Research Report, we highlight the use of a chemogenetic strategy to target projection-defined neurons in reciprocally connected areas through CAV-2 mediated interventions in the rat. Specifically, we targeted the bidirectional pathways between the dorsomedial prefrontal cortex (dmPFC) and the mediodorsal thalamus, as well as those connecting the insular cortex (IC) and the basolateral complex of the amygdala (BLA). These data showcase the usefulness of CAV-2-related strategies to address circuit-level issues. Moreover, we illustrate the inherent limitation of Cre-dependent adeno-associated virues (AAVs) with "leaked" expression of the gene of interest in the absence of Cre and highlight the need for appropriate control conditions.

A thalamocortical circuit for updating action-outcome associations.

The ability to flexibly use knowledge is one cardinal feature of goal-directed behaviors. We recently showed that thalamocortical and corticothalamic pathways connecting the medial prefrontal cortex and the mediodorsal thalamus (MD) contribute to adaptive decision-making (Alcaraz et al., 2018). In this study, we examined the impact of disconnecting the MD from its other main cortical target, the orbitofrontal cortex (OFC) in a task assessing outcome devaluation after initial instrumental training and after reversal of action-outcome contingencies. Crossed MD and OFC lesions did not impair instrumental performance. Using the same approach, we found however that disconnecting the OFC from its other main thalamic afferent, the submedius nucleus, produced a specific impairment in adaptive responding following action-outcome reversal. Altogether, this suggests that multiple thalamocortical circuits may act synergistically to achieve behaviorally relevant functions.