Cerebellar Neurodynamics Predict Decision Timing and Outcome on the Single-Trial Level.

Goal-directed behavior requires the interaction of multiple brain regions. How these regions and their interactions with brain-wide activity drive action selection is less understood. We have investigated this question by combining whole-brain volumetric calcium imaging using light-field microscopy and an operant-conditioning task in larval zebrafish. We find global, recurring dynamics of brain states to exhibit pre-motor bifurcations toward mutually exclusive decision outcomes. These dynamics arise from a distributed network displaying trial-by-trial functional connectivity changes, especially between cerebellum and habenula, which correlate with decision outcome. Within this network the cerebellum shows particularly strong and predictive pre-motor activity (>10 s before movement initiation), mainly within the granule cells. Turn directions are determined by the difference neuroactivity between the ipsilateral and contralateral hemispheres, while the rate of bi-hemispheric population ramping quantitatively predicts decision time on the trial-by-trial level. Our results highlight a cognitive role of the cerebellum and its importance in motor planning.

Cognitive control persistently enhances hippocampal information processing.

Could learning that uses cognitive control to judiciously use relevant information while ignoring distractions generally improve brain function, beyond forming explicit memories? According to a neuroplasticity hypothesis for how some cognitive behavioural therapies are effective, cognitive control training (CCT) changes neural circuit information processing1-3. Here we investigated whether CCT persistently alters hippocampal neural circuit function. We show that mice learned and remembered a conditioned place avoidance during CCT that required ignoring irrelevant locations of shock. CCT facilitated learning new tasks in novel environments for several weeks, relative to unconditioned controls and control mice that avoided the same place during reduced distraction. CCT rapidly changes entorhinal cortex-to-dentate gyrus synaptic circuit function, resulting in an excitatory-inhibitory subcircuit change that persists for months. CCT increases inhibition that attenuates the dentate response to medial entorhinal cortical input, and through disinhibition, potentiates the response to strong inputs, pointing to overall signal-to-noise enhancement. These neurobiological findings support the neuroplasticity hypothesis that, as well as storing item-event associations, CCT persistently optimizes neural circuit information processing.

Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex.

Deciding on a course of action requires both an accurate estimation of option values and the right amount of effort invested in deliberation to reach sufficient confidence in the final choice. In a previous study, we have provided evidence, across a series of judgment and choice tasks, for a dissociation between the ventromedial prefrontal cortex (vmPFC), which would represent option values, and the dorsomedial prefrontal cortex (dmPFC), which would represent the duration of deliberation. Here, we first replicate this dissociation and extend it to the case of an instrumental learning task, in which 24 human volunteers (13 women) choose between options associated with probabilistic gains and losses. According to fMRI data recorded during decision-making, vmPFC activity reflects the sum of option values generated by a reinforcement learning model and dmPFC activity the deliberation time. To further generalize the role of the dmPFC in mobilizing effort, we then analyze fMRI data recorded in the same participants while they prepare to perform motor and cognitive tasks (squeezing a handgrip or making numerical comparisons) to maximize gains or minimize losses. In both cases, dmPFC activity is associated with the output of an effort regulation model, and not with response time. Taken together, these results strengthen a general theory of behavioral control that implicates the vmPFC in the estimation of option values and the dmPFC in the energization of relevant motor and cognitive processes.

Interacting neural ensembles in orbitofrontal cortex for social and feeding behaviour.

Categorically distinct basic drives (for example, for social versus feeding behaviour1-3) can exert potent influences on each other; such interactions are likely to have important adaptive consequences (such as appropriate regulation of feeding in the context of social hierarchies) and can become maladaptive (such as in clinical settings involving anorexia). It is known that neural systems regulating natural and adaptive caloric intake, and those regulating social behaviours, involve related circuitry4-7, but the causal circuit mechanisms of these drive adjudications are not clear. Here we investigate the causal role in behaviour of cellular-resolution experience-specific neuronal populations in the orbitofrontal cortex, a major reward-processing hub that contains diverse activity-specific neuronal populations that respond differentially to various aspects of caloric intake8-13 and social stimuli14,15. We coupled genetically encoded activity imaging with the development and application of methods for optogenetic control of multiple individually defined cells, to both optically monitor and manipulate the activity of many orbitofrontal cortex neurons at the single-cell level in real time during rewarding experiences (caloric consumption and social interaction). We identified distinct populations within the orbitofrontal cortex that selectively responded to either caloric rewards or social stimuli, and found that activity of individually specified naturally feeding-responsive neurons was causally linked to increased feeding behaviour; this effect was selective as, by contrast, single-cell resolution activation of naturally social-responsive neurons inhibited feeding, and activation of neurons responsive to neither feeding nor social stimuli did not alter feeding behaviour. These results reveal the presence of potent cellular-level subnetworks within the orbitofrontal cortex that can be precisely engaged to bidirectionally control feeding behaviours subject to, for example, social influences.

Adolescents flexibly adapt action selection based on controllability inferences.

From early in life, we encounter both controllable environments, in which our actions can causally influence the reward outcomes we experience, and uncontrollable environments, in which they cannot. Environmental controllability is theoretically proposed to organize our behavior. In controllable contexts, we can learn to proactively select instrumental actions that bring about desired outcomes. In uncontrollable environments, Pavlovian learning enables hard-wired, reflexive reactions to anticipated, motivationally salient events, providing "default" behavioral responses. Previous studies characterizing the balance between Pavlovian and instrumental learning systems across development have yielded divergent findings, with some studies observing heightened expression of Pavlovian learning during adolescence and others observing a reduced influence of Pavlovian learning during this developmental stage. In this study, we aimed to investigate whether a theoretical model of controllability-dependent arbitration between learning systems might explain these seemingly divergent findings in the developmental literature, with the specific hypothesis that adolescents' action selection might be particularly sensitive to environmental controllability. To test this hypothesis, 90 participants, aged 8-27, performed a probabilistic-learning task that enables estimation of Pavlovian influence on instrumental learning, across both controllable and uncontrollable conditions. We fit participants' data with a reinforcement-learning model in which controllability inferences adaptively modulate the dominance of Pavlovian versus instrumental control. Relative to children and adults, adolescents exhibited greater flexibility in calibrating the expression of Pavlovian bias to the degree of environmental controllability. These findings suggest that sensitivity to environmental reward statistics that organize motivated behavior may be heightened during adolescence.

Anticipation of Appetitive Operant Action Induces Sustained Dopamine Release in the Nucleus Accumbens.

The mesolimbic dopamine system is implicated in signaling reward-related information as well as in actions that generate rewarding outcomes. These implications are commonly investigated in either pavlovian or operant reinforcement paradigms, where only the latter requires instrumental action. To parse contributions of reward- and action-related information to dopamine signals, we directly compared the two paradigms: male rats underwent either pavlovian or operant conditioning while dopamine release was measured in the nucleus accumbens, a brain region central for processing this information. Task conditions were identical with the exception of the operant-lever response requirement. Rats in both groups released the same quantity of dopamine at the onset of the reward-predictive cue. However, only the operant-conditioning group showed a subsequent, sustained plateau in dopamine concentration throughout the entire 5 s cue presentation (preceding the required action). This dopamine ramp was unaffected by probabilistic reward delivery, occurred exclusively before operant actions, and was not related to task performance or task acquisition as it persisted throughout the 2 week daily behavioral training. Instead, the ramp flexibly increased in duration with longer cue presentation, seemingly modulating the initial cue-onset-triggered dopamine release, that is, the reward prediction error (RPE) signal, as both signal amplitude and sustainment diminished when reward timing was made more predictable. Thus, our findings suggest that RPE and action components of dopamine release can be differentiated temporally into phasic and ramping/sustained signals, respectively, where the latter depends on the former and presumably reflects the anticipation or incentivization of appetitive action, conceptually akin to motivation.SIGNIFICANCE STATEMENT It is unclear whether the components of dopamine signals that are related to reward-associated information and reward-driven approach behavior can be separated. Most studies investigating the dopamine system use either pavlovian or operant conditioning, which both involve the delivery of reward and necessitate appetitive approach behavior. Thus, used exclusively, neither paradigm can disentangle the contributions of these components to dopamine release. However, by combining both paradigms in the same study, we find that anticipation of a reward-driven operant action induces a modulation of reward-prediction-associated dopamine release, producing so-called dopamine ramps. Therefore, our findings provide new insight into dopamine ramps and suggest that dopamine signals integrate reward and appetitive action in a temporally distinguishable, yet dependent, manner.

Motor learning changes the axon initial segment of the spinal motoneuron.

We are studying the mechanisms of H-reflex operant conditioning, a simple form of learning. Modelling studies in the literature and our previous data suggested that changes in the axon initial segment (AIS) might contribute. To explore this, we used blinded quantitative histological and immunohistochemical methods to study in adult rats the impact of H-reflex conditioning on the AIS of the spinal motoneuron that produces the reflex. Successful, but not unsuccessful, H-reflex up-conditioning was associated with greater AIS length and distance from soma; greater length correlated with greater H-reflex increase. Modelling studies in the literature suggest that these increases may increase motoneuron excitability, supporting the hypothesis that they may contribute to H-reflex increase. Up-conditioning did not affect AIS ankyrin G (AnkG) immunoreactivity (IR), p-p38 protein kinase IR, or GABAergic terminals. Successful, but not unsuccessful, H-reflex down-conditioning was associated with more GABAergic terminals on the AIS, weaker AnkG-IR, and stronger p-p38-IR. More GABAergic terminals and weaker AnkG-IR correlated with greater H-reflex decrease. These changes might potentially contribute to the positive shift in motoneuron firing threshold underlying H-reflex decrease; they are consistent with modelling suggesting that sodium channel change may be responsible. H-reflex down-conditioning did not affect AIS dimensions. This evidence that AIS plasticity is associated with and might contribute to H-reflex conditioning adds to evidence that motor learning involves both spinal and brain plasticity, and both neuronal and synaptic plasticity. AIS properties of spinal motoneurons are likely to reflect the combined influence of all the motor skills that share these motoneurons. KEY POINTS: Neuronal action potentials normally begin in the axon initial segment (AIS). AIS plasticity affects neuronal excitability in development and disease. Whether it does so in learning is unknown. Operant conditioning of a spinal reflex, a simple learning model, changes the rat spinal motoneuron AIS. Successful, but not unsuccessful, H-reflex up-conditioning is associated with greater AIS length and distance from soma. Successful, but not unsuccessful, down-conditioning is associated with more AIS GABAergic terminals, less ankyrin G, and more p-p38 protein kinase. The associations between AIS plasticity and successful H-reflex conditioning are consistent with those between AIS plasticity and functional changes in development and disease, and with those predicted by modelling studies in the literature. Motor learning changes neurons and synapses in spinal cord and brain. Because spinal motoneurons are the final common pathway for behaviour, their AIS properties probably reflect the combined impact of all the behaviours that use these motoneurons.

AMPA and NMDA receptors in dentate gyrus mediate memory for sucrose in two port discrimination task.

Although the phenomenon of memory formation and recall associated with the use of psychotropic drugs has been extensively studied, mechanisms underlying memories for natural reward have not been clarified. Herein, we test the hypothesis that glutamatergic receptors in the dentate gyrus play a role in memories associated with sucrose. We used pellet self-administration protocol to generate memories in two-port nose-poke discrimination task using male Wistar rats. During non-rewarded probe trial, the conditioned animals readily discriminated the active port versus inactive port and showed massive increase in mRNA expression of AMPA receptor subunit genes (gria2, gria3) as well as c-Fos protein in the DG. Access to sweet pellet further enhanced c-Fos expression in the DG. However, animals pre-treated with AMPA receptor antagonist CNQX (intra-DG), on exposure to operant chamber (no pellet), showed decreased discrimination as well as c-Fos expression. We suggest that AMPA receptors in DG mediate recall and consolidation of memories associated with sucrose consumption. CNQX pre-treated animals, if presented with sweet pellet on nose poke, exhibited high discrimination index coupled with increased c-Fos expression. In these CNQX treated rats, the DI was again decreased following administration of NMDA receptor antagonist AP5. We suggest that, although AMPA receptors are blocked, the access to sweet pellet may induce surge of glutamate in the DG, which in turn may reinstate memories via activation of erstwhile silent synapses in NMDA dependant manner.

Augmenting glutamatergic, but not dopaminergic, activity in the nucleus accumbens shell disrupts responding to a discrete alcohol cue in an alcohol context.

Discrete alcohol cues and contexts are relapse triggers for people with alcohol use disorder exerting particularly powerful control over behaviour when they co-occur. Here, we investigated the neural substrates subserving the capacity for alcohol-associated contexts to elevate responding to an alcohol-predictive conditioned stimulus (CS). Specifically, rats were trained in a distinct 'alcohol context' to respond by entering a fluid port during a discrete auditory CS that predicted the delivery of alcohol and were familiarized with a 'neutral context' wherein alcohol was never available. When conditioned CS responding was tested by presenting the CS without alcohol, we found that augmenting glutamatergic activity in the nucleus accumbens (NAc) shell by microinfusing α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) reduced responding to an alcohol CS in an alcohol, but not neutral, context. Further, AMPA microinfusion robustly affected behaviour, attenuating the number, duration and latency of CS responses selectively in the alcohol context. Although dopaminergic inputs to the NAc shell were previously shown to be necessary for CS responding in an alcohol context, here, chemogenetic excitation of ventral tegmental area (VTA) dopamine neurons and their inputs to the NAc shell did not affect CS responding. Critically, chemogenetic excitation of VTA dopamine neurons affected feeding behaviour and elevated c-fos immunoreactivity in the VTA and NAc shell, validating the chemogenetic approach. These findings enrich our understanding of the substrates underlying Pavlovian responding for alcohol and reveal that the capacity for contexts to modulate responding to discrete alcohol cues is delicately underpinned by the NAc shell.

Opening new vistas on obsessive-compulsive disorder with the observing response task.

Obsessive-compulsive disorder (OCD), a highly prevalent and debilitating disorder, is incompletely understood in terms of underpinning behavioural, psychological, and neural mechanisms. This is attributable to high symptomatic heterogeneity; cardinal features comprise obsessions and compulsions, including clinical subcategories. While obsessive and intrusive thoughts are arguably unique to humans, dysfunctional behaviours analogous to those seen in clinical OCD have been examined in nonhuman animals. Genetic, ethological, pharmacological, and neurobehavioural approaches all contribute to understanding the emergence and persistence of compulsive behaviour. One behaviour of particular interest is maladaptive checking, whereby human patients excessively perform checking rituals despite these serving no purpose. Dysfunctional and excessive checking is the most common symptom associated with OCD and can be readily operationalised in rodents. This review considers animal models of OCD, the neural circuitries associated with impairments in habit-based and goal-directed behaviour, and how these may link to the compulsions observed in OCD. We further review the Observing Response Task (ORT), an appetitive instrumental learning procedure that distinguishes between functional and dysfunctional checking, with translational application in humans and rodents. By shedding light on the psychological and neural bases of compulsive-like checking, the ORT has potential to offer translational insights into the underlying mechanisms of OCD, in addition to being a platform for testing psychological and neurochemical treatment approaches.

Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors.

Rat autoshaping procedures generate two readily measurable conditioned responses: During lever presentations that have previously signaled food, rats approach the food well (called goal-tracking) and interact with the lever itself (called sign-tracking). We investigated how reinforced and nonreinforced trials affect the overall and temporal distributions of these two responses across 10-second lever presentations. In two experiments, reinforced trials generated more goal-tracking than sign-tracking, and nonreinforced trials resulted in a larger reduction in goal-tracking than sign-tracking. The effect of reinforced trials was evident as an increase in goal-tracking and reduction in sign-tracking across the duration of the lever presentations, and nonreinforced trials resulted in this pattern transiently reversing and then becoming less evident with further training. These dissociations are consistent with a recent elaboration of the Rescorla-Wagner model, HeiDI (Honey, R.C., Dwyer, D.M., & Iliescu, A.F. (2020a). HeiDI: A model for Pavlovian learning and performance with reciprocal associations. Psychological Review, 127, 829-852.), a model in which responses related to the nature of the unconditioned stimulus (e.g., goal-tracking) have a different origin than those related to the nature of the conditioned stimulus (e.g., sign-tracking).

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior.

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

Investigating the interactions between multiple memory stores in the pond snail Lymnaea stagnalis.

The pond snail Lymnaea stagnalis exhibits various forms of associative learning including (1) operant conditioning of aerial respiration where snails are trained not to open their pneumostome in a hypoxic pond water environment using a weak tactile stimulus to their pneumostome as they attempt to open it; and (2) a 24 h-lasting taste-specific learned avoidance known as the Garcia effect utilizing a lipopolysaccharide (LPS) injection just after snails eat a novel food substance (carrot). Typically, lab-inbred snails require two 0.5 h training sessions to form long-term memory (LTM) for operant conditioning of aerial respiration. However, some stressors (e.g., heat shock or predator scent) act as memory enhancers and thus a single 0.5 h training session is sufficient to enhance LTM formation lasting at least 24 h. Here, we found that snails forming a food-aversion LTM following Garcia-effect training exhibited enhanced LTM following operant condition of aerial respiration if trained in the presence of the food substance (carrot) they became averse to. Control experiments led us to conclude that carrot becomes a 'sickness' risk signal and acts as a stressor, sufficient to enhance LTM formation for another conditioning procedure.

Levonorgestrel maintains goal-directed behavior in habit-trained intact female rats.

Hormonal contraceptives are utilized by millions of women worldwide. However, it remains unclear if these powerful endocrine modulators may alter cognitive function. Habit formation involves the progression of instrumental learning as it goes from being a conscious goal-directed process to a cue-driven automatic habitual motor response. Dysregulated goal and/or habit is implicated in numerous psychopathologies, underscoring the relevance of examining the effect of hormonal contraceptives on goal-directed and habitual behavior. This study examined the effect of levonorgestrel (LNG), a widely used progestin-type contraceptive, on the development of habit in intact female rats. Rats were implanted with subcutaneous capsules that slowly released LNG over the course of the experiment or cholesterol-filled capsules. All female rats underwent operant training followed by reward devaluation to test for habit. One group of females was trained at a level that is sub-threshold to habit, while another group of females was trained to a level well over the habit threshold observed in intact females. The results reveal that all sub-threshold trained rats remained goal-directed irrespective of LGN treatment, suggesting LNG is not advancing habit formation in female rats at this level of reinforcement. However, in rats that were overtrained well above the threshold, cholesterol females showed habitual behavior, thus replicating a portion of our original studies. In contrast, LNG-treated habit-trained rats remained goal-directed, indicating that LNG impedes the development and/or expression of habit following this level of supra-threshold to habit training. Thus, LNG may offset habit formation by sustaining attentional or motivational processes during learning in intact female rats. These results may be clinically relevant to women using this type of hormonal contraceptive as well as in other progestin-based hormone therapies.

How do animals weigh conflicting information about reward sources over time? Comparing dynamic averaging models.

Optimal foraging theory suggests that animals make decisions which maximize their food intake per unit time when foraging, but the mechanisms animals use to track the value of behavioral alternatives and choose between them remain unclear. Several models for how animals integrate past experience have been suggested. However, these models make differential predictions for the occurrence of spontaneous recovery of choice: a behavioral phenomenon in which a hiatus from the experimental environment results in animals reverting to a behavioral allocation consistent with a reward distribution from the more distant past, rather than one consistent with their most recently experienced distribution. To explore this phenomenon and compare these models, three free-operant experiments with rats were conducted using a serial reversal design. In Phase 1, two responses (A and B) were baited with pellets on concurrent variable interval schedules, favoring option A. In Phase 2, lever baiting was reversed to favor option B. Rats then entered a delay period, where they were maintained at weight in their home cages and no experimental sessions took place. Following this delay, preference was assessed using initial responding in test sessions where levers were presented, but not baited. Models were compared in performance, including an exponentially weighted moving average, the Temporal Weighting Rule, and variants of these models. While the data provided strong evidence of spontaneous recovery of choice, the form and extent of recovery was inconsistent with the models under investigation. Potential interpretations are discussed in relation to both the decision rule and valuation functions employed.

The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning.

Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people's behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.

A role of anterior cingulate cortex in the emergence of worker-parasite relationship.

We studied the brain mechanisms underlying action selection in a social dilemma setting in which individuals' effortful gains are unfairly distributed among group members. A stable "worker-parasite" relationship developed when three individually operant-conditioned rats were placed together in a Skinner box equipped with response lever and food dispenser on opposite sides. Specifically, one rat, the "worker," engaged in lever-pressing while the other two "parasitic" rats profited from the worker's effort by crowding the feeder in anticipation of food. Anatomically, c-Fos expression in the anterior cingulate cortex (ACC) was significantly higher in worker rats than in parasite rats. Functionally, ACC inactivation suppressed the worker's lever-press behavior drastically under social, but only mildly under individual, settings. Transcriptionally, GABAA receptor- and potassium channel-related messenger RNA expressions were reliably lower in the worker's, relative to parasite's, ACC. These findings indicate the requirement of ACC activation for the expression of exploitable, effortful behavior, which could be mediated by molecular pathways involving GABAA receptor/potassium channel proteins.

Specific Plasticity Loci and Their Synergism Mediate Operant Conditioning.

Despite numerous studies examining the mechanisms of operant conditioning (OC), the diversity of OC plasticity loci and their synergism have not been examined sufficiently. In the well-characterized feeding neural circuit of Aplysia, in vivo and in vitro appetitive OC increases neuronal excitability and electrical coupling among several neurons leading to an increase in expression of ingestive behavior. Here, we used the in vitro analog of OC to investigate whether OC reduces the excitability of a neuron, B4, whose inhibitory connections decrease expression of ingestive behavior. We found OC decreased the excitability of B4. This change appeared intrinsic to B4 because it could be replicated with an analog of OC in isolated cultures of B4 neurons. In addition to changes in B4 excitability, OC decreased the strength of B4's inhibitory connection to a key decision-making neuron, B51. The OC-induced changes were specific without affecting the excitability of another neuron critical for feeding behavior, B8, or the B4-to-B8 inhibitory connection. A conductance-based circuit model indicated that reducing the B4-to-B51 synapse, or increasing B51 excitability, mediated the OC phenotype more effectively than did decreasing B4 excitability. We combined these modifications to examine whether they could act synergistically. Combinations including B51 synergistically enhanced feeding. Taken together, these results suggest modifications of diverse loci work synergistically to mediate OC and that some neurons are well suited to work synergistically with plasticity in other loci.SIGNIFICANCE STATEMENT The ways in which synergism of diverse plasticity loci mediate the change in motor patterns in operant conditioning (OC) are poorly understood. Here, we found that OC was in part mediated by decreasing the intrinsic excitability of a critical neuron of Aplysia feeding behavior, and specifically reducing the strength of one of its inhibitory connections that targets a key decision-making neuron. A conductance-based computational model indicated that the known plasticity loci showed a surprising level of synergism to mediate the behavioral changes associated with OC. These results highlight the importance of understanding the diversity, specificity and synergy among different types of plasticity that encode memory. Also, because OC in Aplysia is mediated by dopamine (DA), the present study provides insights into specific and synergistic mechanisms of DA-mediated reinforcement of behaviors.

Cortico-Striatal Control over Adaptive Goal-Directed Responding Elicited by Cues Signaling Sucrose Reward or Punishment.

The medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) have been associated with the expression of adaptive and maladaptive behavior elicited by fear-related and drug-associated cues. However, reported effects of mPFC manipulations on cue-elicited natural reward-seeking and inhibition thereof have been varied, with few studies examining cortico-striatal contributions in tasks that require adaptive responding to cues signaling reward and punishment within the same session. The current study aimed to better elucidate the role of mPFC and NAc subdivisions, and their functional connectivity in cue-elicited adaptive responding using a novel discriminative cue responding task. Male Long-Evans rats learned to lever-press on a VR5 schedule for a discriminative cue signaling reward, and to avoid pressing the same lever in the presence of another cue signaling punishment. Postacquisition, prelimbic (PL) and infralimbic (IL) areas of the mPFC, NAc core, shell, PL-core, or IL-shell circuits were pharmacologically or chemogenetically inhibited while animals performed under (1) nonreinforced (extinction) conditions, where the appetitive and aversive cues were presented in alternating trials alone or as a compound stimulus; and (2) reinforced conditions, whereby cued responding was accompanied by associated outcomes. PL and IL inactivation attenuated nonreinforced and reinforced goal-directed cue responding, whereas NAc core and shell inactivation impaired nonreinforced responding for the appetitive, but not aversive cue. Furthermore, PL-core and IL-shell inhibition disinhibited nonreinforced but not reinforced cue responding. Our findings implicate the mPFC as a site of confluence of motivationally significant cues and outcomes, and in the regulation of nonreinforced cue responding via downstream NAc targets.SIGNIFICANCE STATEMENT The ability to discriminate and respond appropriately to environmental cues that signal availability of reward or punishment is essential for survival. The medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) have been implicated in adaptive and maladaptive behavior elicited by fear-related and drug-associated cues. However, less is known about the role they play in orchestrating adaptive responses to natural reward and punishment cues within the same behavioral task. Here, using a novel discriminative cue responding task combined with pharmacological or chemogenetic inhibition of mPFC, NAc and mPFC-NAc circuits, we report that mPFC is critically involved in responding to changing cued response-outcomes, both when the responses are reinforced, and nonreinforced. Furthermore, the mPFC coordinates nonreinforced discriminative cue responding by suppressing inappropriate responding via downstream NAc targets.

Neuronal Encoding of Emotional Valence and Intensity in the Monkey Amygdala.

Neuropsychological and neuroimaging studies have suggested that the primate amygdala plays an essential role in processing the emotional valence and intensity of visual stimuli, which is necessary for determining whether to approach or avoid a stimulus. However, the neuronal mechanisms underlying the evaluation of emotional value remain unknown. In the present study, we trained male macaque monkeys to perform an operant conditioning task in which fractal visual patterns were associated with three different amounts of air puff delivered to the cheek (negative) or liquid reward (positive). After confirming that the monkeys successfully differentiated the emotional valence and intensity of the visual stimuli, we analyzed neuronal responses to the stimuli in the amygdala. Most amygdala neurons conveyed information concerning the emotional valence and/or intensity of the visual stimuli, and the majority of those conveying information about emotional valence responded optimally to negative stimuli. Further, some amygdala neurons conveyed information related to both emotional valence and intensity, whereas a small portion conveyed information related to emotional intensity alone. These results indicate that the primate amygdala encodes both emotional valence and intensity, highlighting its important role in the avoidance of dangerous stimuli and animal survival.SIGNIFICANCE STATEMENT Evaluating the emotional value of visual stimuli is essential for animal survival, especially in primates. Emotional value is estimated from the emotional valence and intensity of stimuli, and evidence indicates that the amygdala is likely to play a major role in processing these types of information. To our knowledge, the current study is the first to examine the responses of neurons in the monkey amygdala to visual stimuli that differ in emotional valence and intensity simultaneously. Our data suggest that the amygdala plays an important role in the evaluation of emotional stimuli and in the decision to escape negative and harmful stimuli.