Temporal Dynamics Underlying Prelimbic Prefrontal Cortical Regulation of Action Selection and Outcome Evaluation during Risk/Reward Decision-Making.

Risk/reward decision-making is a dynamic process that includes periods of deliberation before action selection and evaluation of the action outcomes that bias subsequent choices. Inactivation of the prelimbic (PL) cortex has revealed its integral role in updating decision biases in the face of changes in probabilistic reward contingencies, yet how phasic PL signals during different phases of the decision process influence choice remains unclear. We used temporally specific optogenetic inhibition to selectively disrupt PL activity coinciding with action selection and outcome phases to examine how these signals influence choice. Male rats expressing the inhibitory opsin eArchT within PL excitatory neurons were well trained on a probabilistic discounting task, entailing choice between small/certain versus large/risky rewards, the probability of which varied over a session (50-12.5%). During testing, brief light pulses suppressed PL activity before choice or after different outcomes. Prechoice suppression reduced bias toward more preferred/higher utility options and disrupted how recent outcomes influenced subsequent choice. Inhibition during risky losses induced a similar profile, but here, the impact of reward omissions were either amplified or diminished, relative to the context of the estimated profitability of the risky option. Inhibition during large or small reward receipt reduced risky choice when this option was more profitable, suggesting these signals can both reinforce rewarded risky choices and also act as a relative value comparator signal that augments incentive for larger rewards. These findings reveal multifaceted contributions by the PL in implementing decisions and integrating action-outcome feedback to assign context to the decision space.SIGNIFICANCE STATEMENT The PL prefrontal cortex plays an integral role in guiding risk/reward decisions, but how activity in this region during different phases of the decision process influences choice is unclear. By using temporally specific optogenetic manipulations of this activity, the present study unveiled previously uncharacterized and differential contributions by PL in implementing decision policies and how evaluation of decision outcomes shape subsequent choice. These findings provide novel insight into the dynamic processes engaged by the PL that underlie action selection in situations involving reward uncertainty that may aid in understanding the mechanism underlying normal and aberrant decision-making processes.

Early life adversity promotes resilience to opioid addiction-related phenotypes in male rats and sex-specific transcriptional changes.

Experiencing some early life adversity can have an "inoculating" effect that promotes resilience in adulthood. However, the mechanisms underlying stress inoculation are unknown, and animal models are lacking. Here we used the limited bedding and nesting (LBN) model of adversity to evaluate stress inoculation of addiction-related phenotypes. In LBN, pups from postnatal days 2 to 9 and their dams were exposed to a low-resource environment. In adulthood, they were tested for addiction-like phenotypes and compared to rats raised in standard housing conditions. High levels of impulsivity are associated with substance abuse, but in males, LBN reduced impulsive choice compared to controls. LBN males also self-administered less morphine and had a lower breakpoint on a progressive ratio reinforcement schedule than controls. These effects of LBN on addiction-related behaviors were not found in females. Because the nucleus accumbens (NAc) mediates these behaviors, we tested whether LBN altered NAc physiology in drug-naïve and morphine-exposed rats. LBN reduced the frequency of spontaneous excitatory postsynaptic currents in males, but a similar effect was not observed in females. Only in males did LBN prevent a morphine-induced increase in the AMPA/NMDA ratio. RNA sequencing was performed to delineate the molecular signature in the NAc associated with LBN-derived phenotypes. LBN produced sex-specific changes in transcription, including in genes related to glutamate transmission. Collectively, these studies reveal that LBN causes a male-specific stress inoculation effect against addiction-related phenotypes. Identifying factors that promote resilience to addiction may reveal novel treatment options for patients.

Distinct Medial Orbitofrontal-Striatal Circuits Support Dissociable Component Processes of Risk/Reward Decision-Making.

The medial orbitofrontal cortex (mOFC) regulates a variety of cognitive functions, including refining action selection involving reward uncertainty. This region sends projections to numerous subcortical targets, including the ventral and dorsal striatum, yet how these corticostriatal circuits differentially regulate risk/reward decision-making is unknown. The present study examined the contribution of mOFC circuits linking the nucleus accumbens (NAc) and dorsomedial striatum (DMS) to risk/reward decision-making using pharmacological disconnections. Male rats were well trained on a probabilistic discounting task involving choice between small/certain or large/risky rewards, with the probability of obtaining the larger reward decreasing or increasing over a session. Disconnection of mOFC-striatal pathways was achieved using infusions of GABA agonists inactivating the mOFC in one hemisphere, combined with NAc or DMS inactivation in the contralateral or ipsilateral hemisphere. Perturbing mOFC → NAc circuits induced suboptimal, near-random patterns of choice that manifested as a flattening of the discounting curve. Animals were equally likely to stay or shift following rewarded/nonrewarded choices, suggesting this pathway mediates use of information about reward history to stabilize decision biases. In contrast, mOFC → DMS disconnection impaired adjustments in decision biases, causing opposing changes in risky choice depending on how probabilities varied over time. This was driven by alterations in lose-shift behavior, suggesting mOFC → DMS circuits track volatility in nonrewarded actions to adjust choice in accordance with changes in profitability. Thus, separate mOFC-striatal projection pathways regulate dissociable processes underlying decision-making, with mOFC → NAc circuits aiding in establishing and stabilizing tasks states and mOFC → DMS circuits facilitating transitions across states to promote flexible reward seeking.SIGNIFICANCE STATEMENT The medial orbitofrontal cortex regulates a variety of goal-directed behaviors, yet the functional circuits through which it mediates higher order decision-making functions are unclear. The present study revealed that different mOFC projection pathways facilitate diverse aspects of decision-making involving risks and rewards by engaging separate networks of neurons that interface with distinct ventral and dorsal striatal targets. These findings clarify some of the normal functions of these corticostriatal pathways and may have implications for understanding how dysfunction in these circuits relate to certain psychiatric disorders.

Ventral Pallidum GABA Neurons Mediate Motivation Underlying Risky Choice.

Pursuing rewards while avoiding danger is an essential function of any nervous system. Here, we examine a new mechanism helping rats negotiate the balance between risk and reward when making high-stakes decisions. Specifically, we focus on GABA neurons within an emerging mesolimbic circuit nexus: the ventral pallidum (VP). These neurons play a distinct role from other VP neurons in simple motivated behaviors in mice, but their role in more complex motivated behaviors is unknown. Here, we interrogate the behavioral functions of VPGABA neurons in male and female transgenic GAD1:Cre rats (and WT littermates), using a reversible chemogenetic inhibition approach. Using a behavioral assay of risky decision-making, and of the food-seeking and shock-avoidance components of this task, we show that engaging inhibitory Gi/o signaling specifically in VPGABA neurons suppresses motivation to pursue highly salient palatable foods, and possibly also motivation to avoid being shocked. In contrast, inhibiting these neurons did not affect seeking of low-value food, free consumption of palatable food, or unconditioned affective responses to shock. Accordingly, when rats considered whether to pursue food despite potential for shock in a risky decision-making task, inhibiting VPGABA neurons caused them to more readily select a small but safe reward over a large but dangerous one, an effect not seen in the absence of shock threat. Together, results indicate that VPGABA neurons are critical for high-stakes adaptive responding that is necessary for survival, but which may also malfunction in psychiatric disorders.SIGNIFICANCE STATEMENT In a dynamic world, it is essential to implement appropriate behaviors under circumstances involving rewards, threats, or both. Here, we demonstrate a crucial role for VPGABA neurons in high-stakes motivated behavior of several types. We show that this VPGABA role in motivation impacts decision-making, as inhibiting these neurons yields a conservative, risk-averse strategy not seen when the task is performed without threat of shock. These new roles for VPGABA neurons in behavior may inform future strategies for treating addiction, and other disorders of maladaptive decision-making.

Hippocampal neurogenesis promotes effortful responding but does not regulate effort-based choice.

A fundamental trait of depression is low motivation. Hippocampal neurogenesis has been associated with motivational deficits but detailed evidence on how it regulates human-relevant behavioral traits is still missing. We used the hGFAP-TK rat model to deplete actively dividing neural stem cells in the rat hippocampus. Use of the effort-discounting operant task allowed us to identify specific and detailed deficits in motivation behavior. In this task, rats are given a choice between small and large food rewards, where 2-20 lever presses are required to obtain the large reward (four sugar pellets) versus one press to receive the smaller reward (two sugar pellets). We found that depleting adult neurogenesis did not affect effort-based choice or general motivation to complete the task. However, lack of adult neurogenesis reduced the pressing rate and thus increased time to complete the required presses to obtain a reward. In summary, the present study finds that adult hippocampal neurogenesis specifically reduces response vigor to obtain rewards and thus deepens our understanding in how neurogenesis shapes depression.

Hippocampal neurogenesis promotes preference for future rewards.

Adult hippocampal neurogenesis has been implicated in a number of disorders where reward processing is disrupted but whether new neurons regulate specific aspects of reward-related decision making remains unclear. Given the role of the hippocampus in future-oriented cognition, here we tested whether adult neurogenesis regulates preference for future, advantageous rewards in a delay discounting paradigm for rats. Indeed, blocking neurogenesis caused a profound aversion for delayed rewards, and biased choice behavior toward immediately available, but smaller, rewards. Consistent with a role for the ventral hippocampus in impulsive decision making and future-thinking, neurogenesis-deficient animals displayed reduced activity in the ventral hippocampus. In intact animals, delay-based decision making restructured dendrites and spines in adult-born neurons and specifically activated adult-born neurons in the ventral dentate gyrus, relative to dorsal activation in rats that chose between immediately-available rewards. Putative developmentally-born cells, located in the superficial granule cell layer, did not display task-specific activity. These findings identify a novel and specific role for neurogenesis in decisions about future rewards, thereby implicating newborn neurons in disorders where short-sighted gains are preferred at the expense of long-term health.

Prelimbic and Infralimbic Prefrontal Regulation of Active and Inhibitory Avoidance and Reward-Seeking.

Flexible initiation or suppression of actions to avoid aversive events is crucial for survival. The prelimbic (PL) and infralimbic (IL) regions of the medial prefrontal cortex (mPFC) have been implicated in different aspects of avoidance and reward-seeking, but their respective contribution in instigating versus suppressing actions in aversive contexts remains to be clarified. We examined mPFC involvement in different forms of avoidance in rats well trained on different cued lever-press avoidance tasks. Active/inhibitory avoidance required flexible discrimination between auditory cues signaling foot-shock could be avoided by making or withholding instrumental responses. On a simpler active avoidance task, a single cue signaled when a lever press would avoid shock. PL inactivation disrupted active but not inhibitory avoidance on the discriminative task while having no effect on single-cued avoidance. In comparison, IL inactivation broadly impaired active and inhibitory avoidance. Conversely, on a cued appetitive go/no-go task, both IL and PL inactivation impaired inhibitory but not active reward-seeking, the latter effect being diametrically opposite to that observed on the avoidance task. These findings highlight the complex manner in which different mPFC regions aid in initiating or inhibiting actions in the service of avoiding aversive outcomes or obtaining rewarding ones. IL facilitates active avoidance but suppress inappropriate actions in appetitive and aversive contexts. In contrast, contextual valence plays a critical role in how the PL is recruited in initiating or suppressing actions, which may relate to the degree of cognitive control required to flexibly negotiate response or motivational conflicts and override prepotent behaviors.SIGNIFICANCE STATEMENT Choosing to make or withhold actions in a context-appropriate manner to avoid aversive events or obtain other goals is a critical survival skill. Different medial prefrontal cortex (mPFC) regions have been implicated in certain aspects of avoidance, but their contributions to instigating or suppressing actions remains to be clarified. Here, we show that the dorsal, prelimbic (PL) region of the medial PFC aids active avoidance in situations requiring flexible mitigation of response conflicts, but also aids in withholding responses to obtain rewards. In comparison the ventral infralimbic (IL) cortex plays a broader role in active and inhibitory avoidance as well as suppressing actions to obtain rewards. These findings provide insight into mechanisms underlying normal and maladaptive avoidance behaviors and response inhibition.

Dorsomedial striatal contributions to different forms of risk/reward decision making.

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

Prefrontal cortical and nucleus accumbens contributions to discriminative conditioned suppression of reward-seeking.

Fear can potently inhibit ongoing behavior, including reward-seeking, yet the neural circuits that underlie such suppression remain to be clarified. Prior studies have demonstrated that distinct subregions of the rodent medial prefrontal cortex (mPFC) differentially affect fear behavior, whereby fear expression is promoted by the more dorsal prelimbic cortex (PL) and inhibited by the more ventral infralimbic cortex (IL). These mPFC regions project to subregions of the nucleus accumbens, the core (NAcC) and shell (NAcS), that differentially contribute to reward-seeking as well as affective processes that may be relevant to fear expression. Here, we investigated how these mPFC and NAc subregions contribute to discriminative fear conditioning, assessed by conditioned suppression of reward-seeking. Bilateral inactivation of the NAcS or PL reduced the expression of conditioned suppression to a shock-associated CS+, whereas NAcC inactivation reduced reward-seeking without affecting suppression. IL inactivation caused a general reduction in conditioned suppression following discriminative conditioning, but not when using a single-stimulus design. Pharmacological disconnection of the PL → NAcS pathway revealed that this projection mediates conditioned suppression. These data add to a growing literature implicating discrete cortico-striatal pathways in the suppression of reward-seeking in response to aversive stimuli. Dysfunction within related structures may contribute to aberrant patterns of behavior in psychiatric illnesses including substance use disorders.

The Abused Inhalant Toluene Impairs Medial Prefrontal Cortex Activity and Risk/Reward Decision-Making during a Probabilistic Discounting Task.

Inhalant (e.g., toluene) misuse is linked to behavioral and cognitive deficits in humans, yet preclinical studies of the effect of inhalants on higher-order cognition are limited. We addressed this gap in the literature by examining the effect of toluene vapor exposure on risk/reward decision-making in male and female Sprague-Dawley rats using a probabilistic discounting task. In this task, rodents chose a risky/large reward or a safe/small reward, with the odds of risky reinforcement descending or ascending throughout the test session. We observed a dose-dependent, sex-independent deficit in behavioral flexibility during probabilistic discounting caused by acute toluene exposure. Rats exposed to toluene vapor during adolescence and tested as adults performed comparably to air-treated controls and were susceptible to the effects of an acute toluene challenge. These behavioral flexibility deficits observed suggests dysfunctional medial prefrontal cortex (mPFC) activity. To address this hypothesis, we virally expressed the genetically encoded calcium sensor GCaMP6f in glutamatergic mPFC neurons and monitored calcium transients in real-time using in vivo fiber photometry. mPFC activity peaked before either lever press during free-choice trials in toluene- and air-treated animals. During forced-choice trials, GCaMP6f transients shifted from pre-risky to pre-safe choice, an effect mitigated by acute toluene exposure. mPFC activity decreased during rewarded trials, with larger decreases following risky/large wins compared with safe/small wins. Toluene-treated animals also had decreased mPFC activity during rewarded trials, but there was no distinction between risky/large wins and safe/small wins. These results provide physiological evidence for mPFC-dependent behavioral deficits caused by toluene.SIGNIFICANCE STATEMENT Inhalants (e.g., toluene) are an understudied class of drugs of abuse that cause devastating behavioral and cognitive deficits in humans. Understanding the neurobiological interactions of toluene vapor using animal models is important for developing effective treatment strategies for inhalant addicts. Here we find that toluene vapor reduces behavioral flexibility in rodents making risk/reward-based decisions. The medial prefrontal cortex (mPFC) drives behavioral flexibility during this type of decision-making and we show that toluene reduces the ability of mPFC neurons to track optimal choices as reward probabilities change. Toluene also reduces these neurons' ability to distinguish between small and large rewards. A combination of these factors likely leads to the impaired performance in probabilistic discounting following acute toluene exposure.

Differential effects of corticotropin-releasing factor and acute stress on different forms of risk/reward decision-making.

Acute stress and corticotropin-releasing factor (CRF) have been show to perturb cost/benefit decision making involving effort costs. However, previous studies on how stress manipulations affect decisions involving reward uncertainty have yielded variable results. To provide additional insight into this issue, the current study investigated how central CRF infusion and acute restraint stress alter different forms of risk/reward decision-making guided by internal representations of risk/reward contingencies or external informative cues. Male rats were well-trained on one of two tasks that required choice between a small/certain or a large/risky reward. On a probabilistic discounting task, the probability of obtaining the larger reward increased or decreased systematically over blocks of trials (100-6.25%). On a cue-guided Blackjack task, reward probabilities (50% or 12.5%) were signaled by discriminative auditory cues. CRF (1 or 3 µg) was infused intracerebroventricularly (ICV) or one-hour of restraint stress was administered prior to behavioral testing. Neither CRF nor acute stress altered risky choice on probabilistic discounting, but did increase trial omissions in the latter part of the session. Conversely on the Blackjack task, CRF reduced risky choice on good-odds trials (50%), whereas acute stress increased reward sensitivity. CRF but not acute stress also slowed decision latencies across tasks. These data reveal complex and differential manners in which increased CRF activity and acute stress alter distinct forms of risk/reward decision-making, particularly those guided by external cues.

Risk-based decision making in rats: Modulation by sex and amphetamine.

Decision-making is a complex process essential to daily adaptation in many species. Risk is an inherent aspect of decision-making and it is influenced by gonadal hormones. Testosterone and 17ß-estradiol may modulate decision making and impact the mesocorticolimbic dopamine pathway. Here, we explored sex differences, the effect of gonadal hormones and the dopamine agonist amphetamine on risk-based decision making. Intact or gonadectomised (GDX) male and female rats underwent to a probabilistic discounting task. High and low doses of testosterone propionate (1.0 or 0.2 mg) and 17ß-estradiol benzoate (0.3 µg) were administered to assess acute effects on risk-based decision making. After 3-days of washout period, intact and GDX rats received high or low (0.5 or 0.125 mg/kg) doses of amphetamine and re-tested in the probabilistic discounting task. Under baseline conditions, males made more risky choices during probability discounting compared to female rats, particularly in the lower probability blocks, but GDX did not influence risky choice. The high, but not the low dose, of testosterone modestly reduced risky decision making in GDX male rats. Conversely, 17ß-estradiol had no significant effect on risky choice regardless of GDX status in either sex. Lastly, a higher dose of amphetamine increased risky decision making in both intact males and females, but had no effect in GDX rats. These findings demonstrated sex differences in risk-based decision making, with males showing a stronger bias toward larger, uncertain rewards. GDX status influenced the effects of amphetamine, suggesting different dopaminergic regulation in risk-based choices among males and females.

Effects of aging on testosterone and androgen receptors in the mesocorticolimbic system of male rats.

As males age, systemic testosterone (T) levels decline. T regulates executive function, a collection of cognitive processes that are mediated by the mesocorticolimbic system. Here, we examined young adult (5 months) and aged (22 months) male Fischer 344 × Brown Norway rats, and measured systemic T levels in serum and local T levels in microdissected nodes of the mesocorticolimbic system (ventral tegmental area (VTA), nucleus accumbens (NAc), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC)). We also measured androgen receptor (AR) immunoreactivity (-ir) in the mesocorticolimbic system. As expected, systemic T levels decreased with age. Local T levels in mesocorticolimbic regions - except the VTA - also decreased with age. Mesocorticolimbic T levels were higher than serum T levels at both ages. AR-ir was present in the VTA, NAc, mPFC, and OFC and decreased with age in the mPFC. Taken together with previous results, the data suggest that changes in androgen signaling may contribute to changes in executive function during aging.

Differential Contributions of Nucleus Accumbens Subregions to Cue-Guided Risk/Reward Decision Making and Implementation of Conditional Rules.

The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions.SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been associated with neuropsychiatric disorders, such as attention deficit hyperactivity disorder and schizophrenia, which in turn has been linked to aberrant processing in the nucleus accumbens. However, many preclinical studies have often assessed risk/reward decision making in the absence of explicit cues. The current study fills that gap by using a novel task that allows for the assessment of cue-guided risk/reward decision making in rodents. Our findings identified distinct yet complementary roles for the medial versus lateral portions of this nucleus that provide a broader understanding of the differential contributions it makes to decision making and reward seeking guided by discriminative stimuli.

DREADD-mediated modulation of locus coeruleus inputs to mPFC improves strategy set-shifting.

Appropriate modification of behavior in response to our dynamic environment is essential for adaptation and survival. This adaptability allows organisms to maximize the utility of behavior-related energy expenditure. Modern theories of locus coeruleus (LC) function implicate a pivotal role for the noradrenergic nucleus in mediating switches between focused behavior during periods of high utility (exploit) versus disengagement of behavior and exploration of other, more rewarding opportunities. Two modes of activity in LC neurons have been characterized as elements in an Adaptive Gain Theory (AGT) of LC function. In this theory, during periods of accurate and focused behavior, LC neurons exhibit task-related phasic bursts. However, as behavioral utility wanes, phasic activity is suppressed and baseline (tonic) impulse activity increases to facilitate exploration. Our experiments sought to exogenously induce an elevated pattern of activity in LC neurons and their medial prefrontal cortical (mPFC) targets to test the tenets of the AGT. This theory posits that tonic activation immediately following a rule change should increase exploration and thereby improve performance on a set-shifting task. Indeed, DREADD mediated stimulation of LC terminals within mPFC decreased trials to reach criterion. However, this effect resulted from improved application of the new rule once the original rule is jettisoned rather than earlier disengagement from the old, ineffective strategy. Such improvements were not seen with global manipulation of LC, consistent with the view that LC-mediated exploration involves specific sub-circuits targeting mPFC. These findings extend our understanding of the role of LC in PFC and flexible behavior.

Prefrontal Dopamine D1 and D2 Receptors Regulate Dissociable Aspects of Decision Making via Distinct Ventral Striatal and Amygdalar Circuits.

Mesocortical dopamine (DA) regulates a variety of cognitive functions via actions on D1 and/or D2 receptors. For example, risk/reward decision making is modulated differentially by these two receptors within the prefrontal cortex (PFC), with D2 receptors enabling flexible decision making and D1 receptors promoting persistence in choice biases. However, it is unclear how DA mediates opposing patterns of behavior by acting on different receptors within the same terminal region. We explored the possibility that DA may act on separate networks of PFC neurons that are modulated by D1 or D2 receptors and in turn interface with divergent downstream structures such as the basolateral amygdala (BLA) or nucleus accumbens (NAc). Decision making was assessed using a probabilistic discounting task in which well trained male rats chose between small/certain or large/risky rewards, with the odds of obtaining the larger reward changing systematically within a session. Selective disruption of D1 or D2 modulation of separate PFC output pathways was achieved using unilateral intra-PFC infusions of DA antagonists combined with contralateral inactivation of the BLA or NAc. Disrupting D2 (but not D1) modulation of PFC→BLA circuitry impaired adjustments in decision biases in response to changes in reward probabilities. In contrast, disrupting D1 modulation of PFC→NAc networks reduced risky choice, attenuating reward sensitivity and increasing sensitivity to reward omissions. These findings reveal that mesocortical DA can facilitate dissociable components of reward seeking and action selection by acting on different functional networks of PFC neurons that can be distinguished by the subcortical projection targets with which they interface.SIGNIFICANCE STATEMENT Prefrontal cortical dopamine regulates a variety of executive functions governed by the frontal lobes via actions on D1 and D2 receptors. These receptors can in some instances mediate different patterns of behavior, but the mechanisms underlying these dissociable actions are unclear. Using a selective disconnection approach, we reveal that D1 and D2 receptors can facilitate diverse aspects of decision making by acting on separate networks of prefrontal neurons that interface with distinct striatal or amygdalar targets. These findings reveal an additional level of complexity in how mesocortical DA regulates different forms of cognition via actions on different receptors, highlighting how it may act upon distinct cortical microcircuits to drive different patterns of behavior.

Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes.

Maladaptive decision-making is increasingly recognized to play a significant role in numerous psychiatric disorders, such that therapeutics capable of ameliorating core impairments in judgment may be beneficial in a range of patient populations. The field of "decision neuroscience" is therefore in its ascendancy, with researchers from diverse fields bringing their expertise to bear on this complex and fascinating problem. In addition to the advances in neuroimaging and computational neuroscience that contribute enormously to this area, an increase in the complexity and sophistication of behavioral paradigms designed for nonhuman laboratory animals has also had a significant impact on researchers' ability to test the causal nature of hypotheses pertaining to the neural circuitry underlying the choice process. Multiple such decision-making assays have been developed to investigate the neural and neurochemical bases of different types of cost/benefit decisions. However, what may seem like relatively trivial variation in behavioral methodologies can actually result in recruitment of distinct cognitive mechanisms, and alter the neurobiological processes that regulate choice. Here we focus on two areas of particular interest, namely, decisions that involve an assessment of uncertainty or effort, and compare some of the most prominent behavioral paradigms that have been used to investigate these processes in laboratory rodents. We illustrate how an appreciation of the diversity in the nature of these tasks can lead to important insights into the circumstances under which different neural regions make critical contributions to decision making.

Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning.

Different subregions of the prefrontal cortex (PFC) contribute to the ability to respond flexibly to changes in reward contingencies, with the medial versus orbitofrontal cortex (OFC) subregions contributing differentially to processes such as set-shifting and reversal learning. To date, the manner in which these regions may facilitate reversal learning in situations involving reward uncertainty remains relatively unexplored. We investigated the involvement of five distinct regions of the rat OFC (lateral and medial) and medial PFC (prelimbic, infralimbic, and anterior cingulate) on probabilistic reversal learning wherein "correct" versus "incorrect" responses were rewarded on 80% and 20% of trials, respectively. Contingencies were reversed repeatedly within a session. In well trained rats, inactivation of the medial or lateral OFC induced dissociable impairments in performance (indexed by fewer reversals completed) when outcomes were probabilistic, but not when they were assured. Medial OFC inactivation impaired probabilistic learning during the first discrimination, increased perseverative responding and reduced sensitivity to positive and negative feedback, suggestive of a deficit in incorporating information about previous action outcomes to guide subsequent behavior. Lateral OFC inactivation preferentially impaired performance during reversal phases. In contrast, prelimbic inactivation caused an apparent improvement in performance by increasing the number of reversals completed. This was associated with enhanced sensitivity to recently rewarded actions and reduced sensitivity to negative feedback. Infralimbic inactivation had no effect, whereas the anterior cingulate appeared to play a permissive role in this form of reversal learning. These results clarify the dissociable contributions of different regions of the frontal lobes to probabilistic learning. SIGNIFICANCE STATEMENT: The ability to adjust behavior in response to changes involving uncertain or probabilistic reward contingencies is an essential survival skill that is impaired in a variety of psychiatric disorders. It is well established that different forms of cognitive flexibility are mediated by anatomically distinct regions of the frontal lobes when reinforcement contingencies are assured, however, less is known about the contribution of these regions to probabilistic reinforcement learning. Here we show that different regions of the orbitofrontal and medial prefrontal cortex make distinct contributions to probabilistic reversal learning. These findings provide novel information about the complex interplay between frontal lobe regions in mediating these processes and accordingly provide insight into possible pathophysiology that underlies impairments in cognitive flexibility observed in mental illnesses.

Contributions of basolateral amygdala and nucleus accumbens subregions to mediating motivational conflict during punished reward-seeking.

The involvement of different nodes within meso-cortico-limbic-striatal circuitry in mediating reward-seeking has been well described, yet comparatively less is known about how such circuitry may regulate appetitively-motivated behaviors that may be punished. The basolateral amygdala (BLA) is one nucleus that has been implicated in suppressing punished reward-seeking, and this structure can modulate goal-directed behavior via projections to subregions of the nucleus accumbens (NAc). Here, we examined the effects of reversible inactivations of the BLA, NAc Shell (NAcS), and core (NAcC) on performance of a "Conflict" task where rats pressed a lever for sucrose reinforcement during three distinct 5min phases. During the first and last phases of a session, rats lever-pressed for food reward delivered on a VI-15/FR5 schedule. In between these phases was a signaled "Conflict" period, where each lever-press yielded food, but 50% of presses were also punished with foot-shock. Under control conditions, well-trained rats responded vigorously during the two "safe" VI-15/FR5 periods, but reduced responding during the punished Conflict period. Inactivation of either the BLA or the NAcS via infusions of baclofen/muscimol disinhibited punished seeking, increasing lever-pressing during the conflict period, while attenuating pressing during VI-15/FR5 phases. In contrast, NAcC inactivation markedly decreased responding across all three phases. Similar inactivation of the BLA or NAcS did not alter responding in a separate control experiment where rats pressed for food on schedules identical to the Conflict task in the absence of any punishment, while NAcC inactivation again suppressed responding. These results imply that BLA and NAcS are part of a circuit that suppresses reward-seeking in the face of danger, which in turn may have implications for disorders characterized by punishment resistance, including substance abuse and obsessive-compulsive disorder.

The nucleus accumbens: an interface between cognition, emotion, and action.

Nearly 40 years of research on the function of the nucleus accumbens (NAc) has provided a wealth of information on its contributions to behavior but has also yielded controversies and misconceptions regarding these functions. A primary tenet of this review is that, rather than serving as a "reward" center, the NAc plays a key role in action selection, integrating cognitive and affective information processed by frontal and temporal lobe regions to augment the efficiency and vigor of appetitively or aversively motivated behaviors. Its involvement in these functions is most prominent when the appropriate course of action is ambiguous, uncertain, laden with distractors, or in a state of flux. To this end, different subregions of the NAc play dissociable roles in refining action selection, promoting approach toward motivationally relevant stimuli, suppressing inappropriate actions so that goals may be obtained more efficiently, and encoding action outcomes that guide the direction of subsequent ones.