Striatal Acetylcholine and Dopamine Interactions Produce Situation-appropriate Action Selection.

Individuals often learn how to perform new actions for particular outcomes against a complex background of existing action-outcome associations. As such, this new knowledge can interfere or even compete with existing knowledge, such that individuals must use internal and external cues to determine which action is appropriate to the current situation. The question thus remains as to how this problem is solved at a neural level. Research over the last decade or so has begun to determine how the brain achieves situation-appropriate action selection. Several converging lines of evidence suggest that it is achieved through the complex interactions of acetylcholine and dopamine within the striatum in a manner that relies on glutamatergic inputs from the cortex and thalamus. Here we briefly review this evidence, then relate it to several very recent findings to provide new, speculative insights regarding the precise nature of striatal acetylcholine/dopamine interaction dynamics and their relation to situation- appropriate action selection.

Diet and obesity effects on cue-driven food-seeking: insights from studies of Pavlovian-instrumental transfer in rodents and humans.

Our modern environment is said to be obesogenic, promoting the consumption of calorically dense foods and reducing energy expenditure. One factor thought to drive excess energy intake is the abundance of cues signaling the availability of highly palatable foods. Indeed, these cues exert powerful influences over food-related decision-making. Although obesity is associated with changes to several cognitive domains, the specific role of cues in producing this shift and on decision-making more generally, remains poorly understood. Here we review the literature examining how obesity and palatable diets affect the ability of Pavlovian cues to influence instrumental food-seeking behaviors by examining rodent and human studies incorporating Pavlovian-instrumental transfer (PIT) protocols. There are two types of PIT: (a) general PIT that tests whether cues can energize actions elicited in the pursuit of food generally, and (b) specific PIT which tests whether cues can elicit an action that earns a specific food outcome when faced with a choice. Both types of PIT have been shown to be vulnerable to alterations as a result of changes to diet and obesity. However, effects appear to be driven less by increases in body fat and more by palatable diet exposure per se. We discuss the limitations and implications of the current findings. The challenges for future research are to uncover the mechanisms underlying these alterations to PIT, which appear unrelated to excess weight itself, and to better model the complex determinants of food choice in humans.

L-Proline Prevents Endoplasmic Reticulum Stress in Microglial Cells Exposed to L-azetidine-2-carboxylic Acid.

L-Azetidine-2-carboxylic acid (AZE) is a non-protein amino acid that shares structural similarities with its proteogenic L-proline amino acid counterpart. For this reason, AZE can be misincorporated in place of L-proline, contributing to AZE toxicity. In previous work, we have shown that AZE induces both polarization and apoptosis in BV2 microglial cells. However, it is still unknown if these detrimental effects involve endoplasmic reticulum (ER) stress and whether L-proline co-administration prevents AZE-induced damage to microglia. Here, we investigated the gene expression of ER stress markers in BV2 microglial cells treated with AZE alone (1000 µM), or co-treated with L-proline (50 µM), for 6 or 24 h. AZE reduced cell viability, nitric oxide (NO) secretion and caused a robust activation of the unfolded protein response (UPR) genes (ATF4, ATF6, ERN1, PERK, XBP1, DDIT3, GADD34). These results were confirmed by immunofluorescence in BV2 and primary microglial cultures. AZE also altered the expression of microglial M1 phenotypic markers (increased IL-6, decreased CD206 and TREM2 expression). These effects were almost completely prevented upon L-proline co-administration. Finally, triple/quadrupole mass spectrometry demonstrated a robust increase in AZE-bound proteins after AZE treatment, which was reduced by 84% upon L-proline co-supplementation. This study identified ER stress as a pathogenic mechanism for AZE-induced microglial activation and death, which is reversed by co-administration of L-proline.

Cognitive effects of thalamostriatal degeneration are ameliorated by normalizing striatal cholinergic activity.

The loss of neurons in parafascicular thalamus (Pf) and their inputs to dorsomedial striatum (DMS) in Lewy body disease (LBD) and Parkinson's disease dementia (PDD) have been linked to the effects of neuroinflammation. We found that, in rats, these inputs were necessary for both the function of striatal cholinergic interneurons (CINs) and the flexible encoding of the action-outcome (AO) associations necessary for goal-directed action, producing a burst-pause pattern of CIN firing but only during the remapping elicited by a shift in AO contingency. Neuroinflammation in the Pf abolished these changes in CIN activity and goal-directed control after the shift in contingency. However, both effects were rescued by either the peripheral or the intra-DMS administration of selegiline, a monoamine oxidase B inhibitor that we found also enhances adenosine triphosphatase activity in CINs. These findings suggest a potential treatment for the cognitive deficits associated with neuroinflammation affecting the function of the Pf and related structures.

Goal-Directed Action Is Initially Impaired in a hAPP-J20 Mouse Model of Alzheimer's Disease.

Cognitive-behavioral testing in preclinical models of Alzheimer's disease has failed to capture deficits in goal-directed action control. Here, we provide the first comprehensive investigation of goal-directed action in a transgenic mouse model of Alzheimer's disease. Specifically, we tested outcome devaluation performance in male and female human amyloid precursor protein (hAPP)-J20 mice. Mice were first trained to press left and right levers for pellet and sucrose outcomes, respectively (counterbalanced), over 4 d. On test, mice were prefed one of the outcomes to satiety and given a choice between levers. Devaluation performance was intact for 36-week-old wild-types of both sexes, who responded more on the valued relative to the devalued lever (Valued > Devalued). By contrast, devaluation was impaired (Valued = Devalued) for J20 mice of both sexes, and for 52-week-old male mice regardless of genotype. After additional lever press training (i.e., 8-d lever pressing in total), devaluation was intact for all mice, demonstrating that the initial deficits were not a result of a nonspecific impairment in reward processing, depression, or locomotor activity in J20 or aging mice. Follow-up analyses revealed that microglial expression in the dorsal CA1 region of the hippocampus was associated with poorer outcome devaluation performance on initial, but not later tests. Together, these data demonstrate that goal-directed action is initially impaired in J20 mice of both sexes and in aging male mice regardless of genotype, and that this impairment is related to neuroinflammation in the dorsal CA1 hippocampal region.

Association learning: Dopamine and the formation of backward associations.

The activity of dopamine neurons is critical for the ability to learn and update cue-reward associations. New work in rats shows that dopamine transients are also critical for the formation of backward associations in which the reward precedes the neutral stimulus.

Outcome-selective reinstatement is predominantly context-independent, and associated with c-Fos activation in the posterior dorsomedial striatum.

Research from human and animal studies has found that after responding has been successfully reduced following treatment it can return upon exposure to certain contexts. An individual in recovery from alcohol use disorder, for example, might relapse to drinking upon visiting their favourite bar. However, most of these data have been derived from experiments involving a single (active) response, and the context-dependence of returned responding in situations involving choice between multiple actions and outcomes is less well-understood. We thus investigated how outcome-selective reinstatement - a procedure involving choice between two actions and outcomes - was affected by altering the physical context in rats. In Experiment 1, rats were trained over 6 days to press a left lever for one food outcome (pellets or sucrose) and a right lever for the other outcome. Then, rats received an extinction session in either the same context (A) as lever press training, or in a different context (B). Rats were tested immediately (5 min) after extinction in Context A or B such that there were four groups in total: AAA, ABB, ABA, and AAB. Reinstatement testing consisted of one food outcome being delivered 'freely' (i.e. unearned by lever pressing and unsignalled by cues) to the food magazine every 4 min in the following order: Sucrose, Pellet, Pellet, Sucrose. Selective reinstatement was considered intact if pellet delivery increased pressing selectively on the pellet lever, and sucrose delivery selectively increased pressing on the sucrose lever. This result (Reinstated > Nonreinstated) was observed for rats in group AAA and ABB, but not rats in groups ABA and AAB. Experiment 2 was conducted identically, except that rats received two extinction sessions over two days and tested one day later. This time, all groups demonstrated intact outcome-selective reinstatement regardless of context. Analysis of c-Fos expression in several brain regions revealed that only c-Fos expression in the posterior dorsomedial striatum (pDMS) was related to intact reinstatement performance. Overall, these results suggest that outcome-selective reinstatement is predominantly context-independent, and that intact reinstatement is related to neuronal activity in the pDMS.

Does disrupting the orbitofrontal cortex alter sensitivity to punishment? A potential mechanism of compulsivity.

Abnormal orbitofrontal cortex (OFC) activity is one of the most common findings from neuroimaging studies of individuals with compulsive disorders such as substance use disorder and obsessive-compulsive disorder. The nature of this abnormality is complex, however, with some studies reporting the OFC to be over-active in compulsive individuals relative to controls, whereas other studies report it being under-active, and a further set of studies reporting OFC abnormality in both directions within the same individuals. The OFC has been implicated in a broad range of cognitive processes such as decision-making and goal-directed action. OFC dysfunction could thus impair decision-making and goal-directed action, leading to the kinds of cognitive/behavioral deficits observed in individuals with compulsive disorders. One such deficit that could arise as a result of OFC dysfunction is an altered sensitivity to punishment, which is one of the core characteristics displayed by individuals across multiple types of compulsive disorders. It is, therefore, the aim of the current review to assess the evidence implicating the OFC in adaptation to punishment and to attempt to identify the critical factors that determine this relationship. We distill from this analysis some guidelines for future studies attempting to determine the precise role of the OFC in punishment. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Threat perception: Fear and the retrorubal field.

A new study has found that neurons within a structure of the rat midbrain known as the retrorubral field show diverse responses to stimuli that signal different levels of threat, as well as a separate pattern of diverse responses to differentially predicted aversive outcomes.

Parafascicular Thalamic and Orbitofrontal Cortical Inputs to Striatum Represent States for Goal-Directed Action Selection.

Several lines of evidence accrued over the last 5-10 years have converged to suggest that the parafascicular nucleus of the thalamus and the lateral orbitofrontal cortex each represent or contribute to internal state/context representations that guide action selection in partially observable task situations. In rodents, inactivations of each structure have been found to selectively impair performance in paradigms testing goal-directed action selection, but only when that action selection relies on state representations. Electrophysiological evidence has suggested that each structure achieves this function via inputs onto cholinergic interneurons (CINs) in the dorsomedial striatum. Here, we briefly review these studies, then point to anatomical evidence regarding the afferents of each structure and what they suggest about the specific features that each contribute to internal state representations. Finally, we speculate as to whether this role might be achieved interdependently through direct PF→OFC projections, or through the convergence of independent direct orbitofrontal cortex (OFC) and parafascicular nucleus of the thalamus (PF) inputs onto striatal targets.

Adaptive behaviour under conflict: Deconstructing extinction, reversal, and active avoidance learning.

In complex environments, organisms must respond adaptively to situations despite conflicting information. Under natural (i.e. non-laboratory) circumstances, it is rare that cues or responses are consistently paired with a single outcome. Inconsistent pairings are more common, as are situations where cues and responses are associated with multiple outcomes. Such inconsistency creates conflict, and a response that is adaptive in one scenario may not be adaptive in another. Learning to adjust responses accordingly is important for species to survive and prosper. Here we review the behavioural and brain mechanisms of responding under conflict by focusing on three popular behavioural procedures: extinction, reversal learning, and active avoidance. Extinction involves adapting from reinforcement to non-reinforcement, reversal learning involves swapping the reinforcement of cues or responses, and active avoidance involves performing a response to avoid an aversive outcome, which may conflict with other defensive strategies. We note that each of these phenomena relies on somewhat overlapping neural circuits, suggesting that such circuits may be critical for the general ability to respond appropriately under conflict.

Goal-directed actions transiently depend on dorsal hippocampus.

The role of the hippocampus in goal-directed action is currently unclear; studies investigating this issue have produced contradictory results. Here we reconcile these contradictions by demonstrating that, in rats, goal-directed action relies on the dorsal hippocampus, but only transiently, immediately after initial acquisition. Furthermore, we found that goal-directed action also depends transiently on physical context, suggesting a psychological basis for the hippocampal regulation of goal-directed action control.

Medial Orbitofrontal Cortex Regulates Instrumental Conditioned Punishment, but not Pavlovian Conditioned Fear.

Bidirectionally aberrant medial orbitofrontal cortical (mOFC) activity has been consistently linked with compulsive disorders and related behaviors. Although rodent studies have established a causal link between mOFC excitation and compulsive-like actions, no such link has been made with mOFC inhibition. Here, we use excitotoxic lesions of mOFC to investigate its role in sensitivity to punishment; a core characteristic of many compulsive disorders. In our first experiment, we demonstrated that mOFC lesions prevented rats from learning to avoid a lever that was punished with a stimulus that coterminated with footshock. Our second experiment demonstrated that retrieval of punishment learning is also somewhat mOFC-dependent, as lesions prevented the extended retrieval of punishment contingencies relative to shams. In contrast, mOFC lesions did not prevent rats from reacquiring the ability to avoid a punished lever when it was learned prior to lesions being administered. In both experiments, Pavlovian fear conditioning to the stimulus was intact for all animals. Together, these results reveal that the mOFC regulates punishment learning and retrieval in a manner that is separate from any role in Pavlovian fear conditioning. These results imply that aberrant mOFC activity may contribute to the punishment insensitivity that is observed across multiple compulsive disorders.

Rodent medial and lateral orbitofrontal cortices represent unique components of cognitive maps of task space.

The orbitofrontal cortex (OFC) has been proposed to function as a cognitive map of task space: a mental model of the steps involved in a task. This idea has proven popular because it provides a cohesive explanation for a number of disparate findings regarding the OFC's role in a broad array of tasks. Concurrently, evidence has begun to reveal the functional heterogeneity of OFC subregions, particularly the medial and lateral OFC. How these subregions uniquely contribute to the OFC's role as a cognitive map of task space, however, has not been explored. Here we propose that, in rodents, the lateral OFC represents the agent's initial position within that task map (i.e. initial state), determining which actions are available as a consequence of that position, whereas the medial OFC represents the agent's future position within the task map (i.e. terminal state), influencing which actions are selected to achieve that position. We argue that these processes are achieved somewhat independently and somewhat interdependently, and are achieved through similar but non-identical circuitry.

Punishment insensitivity emerges from impaired contingency detection, not aversion insensitivity or reward dominance.

Our behaviour is shaped by its consequences - we seek rewards and avoid harm. It has been reported that individuals vary markedly in their avoidance of detrimental consequences, that is in their sensitivity to punishment. The underpinnings of this variability are poorly understood; they may be driven by differences in aversion sensitivity, motivation for reward, and/or instrumental control. We examined these hypotheses by applying several analysis strategies to the behaviour of rats (n = 48; 18 female) trained in a conditioned punishment task that permitted concurrent assessment of punishment, reward-seeking, and Pavlovian fear. We show that punishment insensitivity is a unique phenotype, unrelated to differences in reward-seeking and Pavlovian fear, and due to a failure of instrumental control. Subjects insensitive to punishment are afraid of aversive events, they are simply unable to change their behaviour to avoid them.

The Thalamostriatal Pathway and the Hierarchical Control of Action.

Sequential ordering of motor commands is required for the simplest of our daily activities. In this issue of Neuron, Díaz-Hernández et al. (2018) show that distinct thalamic inputs to different regions of the dorsal striatum critically modulate the initiation and execution of action sequences.

Inferring action-dependent outcome representations depends on anterior but not posterior medial orbitofrontal cortex.

Although studies examining orbitofrontal cortex (OFC) often treat it as though it were functionally homogeneous, recent evidence has questioned this assumption. Not only are the various subregions of OFC (lateral, ventral, and medial) hetereogeneous, but there is further evidence of heterogeneity within those subregions. For example, several studies in both humans and monkeys have revealed a functional subdivision along the anterior-posterior gradient of the medial OFC (mOFC). Given our previous findings suggesting that, in rats, the mOFC is responsible for inferring the likelihood of unobservable action outcomes (Bradfield, Dezfouli, van Holstein, Chieng, & Balleine, 2015), and given the anterior nature of the placements of our prior manipulations, we decided to assess whether the rat mOFC also differs in connection and function along its anteroposterior axis. We first used retrograde tracing to compare the density of efferents from mOFC to several structures known to contribute to goal-directed action: the mediodorsal thalamus, basolateral amygdala, posterior dorsomedial striatum, nucleus accumbens core and ventral tegmental area. We then compared the functional effects of anterior versus posterior mOFC excitotoxic lesions on tests of Pavlovian-instrumental transfer, instrumental outcome devaluation and outcome-specific reinstatement. We found evidence that the anterior mOFC had greater connectivity with the accumbens core and greater functional involvement in goal-directed action than the posterior mOFC. Consistent with previous findings across species, therefore, these results suggest that the anterior and posterior mOFC of the rat are indeed functionally distinct, and that it is the anterior mOFC that is particularly critical for inferring unobservable action outcomes.

The Bilateral Prefronto-striatal Pathway Is Necessary for Learning New Goal-Directed Actions.

The acquisition of new goal-directed actions requires the encoding of action-outcome associations. At a neural level, this encoding has been hypothesized to involve a prefronto-striatal circuit extending between the prelimbic cortex (PL) and the posterior dorsomedial striatum (pDMS); however, no research identifying this pathway with any precision has been reported. We started by mapping the prelimbic input to the dorsal and ventral striatum using a combination of retrograde and anterograde tracing with CLARITY and established that PL-pDMS projections share some overlap with projections to the nucleus accumbens core (NAc) in rats. We then tested whether each of these pathways were functionally required for goal-directed learning; we used a pathway-specific dual-virus chemogenetic approach to selectively silence pDMS-projecting or NAc-projecting PL neurons during instrumental training and tested rats for goal-directed action. We found that silencing PL-pDMS projections abolished goal-directed learning, whereas silencing PL-NAc projections left goal-directed learning intact. Finally, we used a three-virus approach to silence bilateral and contralateral pDMS-projecting PL neurons and again blocked goal-directed learning. These results establish that the acquisition of new goal-directed actions depends on the bilateral PL-pDMS pathway driven by intratelencephalic cortical neurons.

Prefrontal Corticostriatal Disconnection Blocks the Acquisition of Goal-Directed Action.

The acquisition of goal-directed action requires encoding of the association between an action and its specific consequences or outcome. At a neural level, this encoding has been hypothesized to involve a prefrontal corticostriatal circuit involving the projection from the prelimbic cortex (PL) to the posterior dorsomedial striatum (pDMS); however, no direct evidence for this claim has been reported. In a series of experiments, we performed functional disconnection of this pathway using targeted lesions of the anterior corpus callosum to disrupt contralateral corticostriatal projections with asymmetrical lesions of the PL and/or pDMS to block plasticity in this circuit in rats. We first demonstrated that unilaterally blocking the PL input to the pDMS prevented the phosphorylation of extracellular signal-related kinase/mitogen activated protein kinase (pERK/pMAPK) induced by instrumental training. Next, we used a full bilateral disconnection of the PL from the pDMS and assessed goal-directed action using an outcome-devaluation test. Importantly, we found evidence that rats maintaining an ipsilateral and/or contralateral connection between the PL and the pDMS were able to acquire goal-directed actions. In contrast, bilateral PL-pDMS disconnection abolished the acquisition of goal-directed actions. Finally, we used a temporary pharmacological disconnection to disrupt PL inputs to the pDMS by infusing the NMDA antagonist dl-2-amino-5-phosphonopentanoic acid into the pDMS during instrumental training and found that this manipulation also disrupted goal-directed learning. These results establish that, in rats, the acquisition of new goal-directed actions depends on a prefrontal-corticostriatal circuit involving a connection between the PL and the pDMS.SIGNIFICANCE STATEMENT It has been hypothesized that the prelimbic cortex (PL) and posterior dorsomedial striatum (pDMS) in rodents interact in a corticostriatal circuit to mediate goal-directed learning. However, no direct evidence supporting this claim has been reported. Using targeted lesions, we performed functional disconnection of the PL-pDMS pathway to assess its role in goal-directed learning. In the first experiment, we demonstrated that PL input to the pDMS is necessary for instrumental training-induced neuronal activity. Next, we disrupted ipsilateral, contralateral, or bilateral PL-pDMS connections and found that only bilateral PL-pDMS disconnection disrupted the acquisition of goal-directed actions, a finding we replicated in our final study using a pharmacological disconnection procedure.