Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related.

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

The effects of fatty acid amide hydrolase inhibition and monoacylglycerol lipase inhibition on habit formation in mice.

Emerging data indicate that endocannabinoid signaling is critical to the formation of habitual behavior. Previous work demonstrated that antagonism of cannabinoid receptor type 1 (CB1R) with AM251 during operant training impairs habit formation, but it is not known if this behavioral effect is specific to disrupted signaling of the endocannabinoid ligands anandamide or 2-arachidonoyl glycerol (2-AG). Here, we used selective pharmacological compounds during operant training to determine the impact of fatty acid amide hydrolase (FAAH) inhibition to increase anandamide (and other n-acylethanolamines) or monoacylglycerol lipase (MAGL) inhibition to increase 2-AG levels on the formation of habitual behaviors in mice using a food-reinforced contingency degradation procedure. We found, contrary to our hypothesis, that inhibition of FAAH and of MAGL disrupted the formation of habits. Next, AM251 was administered during training to verify that impaired habit formation could be assessed using contingency degradation. AM251-exposed mice responded at lower rates during training and at higher rates in the test. To understand the inconsistency with published data, we performed a proof-of-principle dose-response experiment to compare AM251 in our vehicle-solution to the published vehicle-suspension on response rates. We found consistent reductions in response rate with increasing doses of AM251 in solution and an inconsistent dose-response relationship with AM251 in suspension. Together, our data suggest that further characterization of the role of CB1R signaling in the formation of habitual responding is warranted and that augmenting endocannabinoids may have clinical utility for prophylactically preventing aberrant habit formation such as that hypothesized to occur in substance use disorders.

Reinforcement Learning during Adolescence in Rats.

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.

Medial Nucleus Accumbens Projections to the Ventral Tegmental Area Control Food Consumption.

Decades of research have shown that the NAc is a critical region influencing addiction, mood, and food consumption through its effects on reinforcement learning, motivation, and hedonic experience. Pharmacological studies have demonstrated that inhibition of the NAc shell induces voracious feeding, leading to the hypothesis that the inhibitory projections that emerge from the NAc normally act to restrict feeding. While much of this work has focused on projections to the lateral hypothalamus, the role of NAc projections to the VTA in the control food intake has been largely unexplored. Using a retrograde viral labeling technique and real-time monitoring of neural activity with fiber photometry, we find that medial NAc shell projections to the VTA (mNAc→VTA) are inhibited during food-seeking and food consumption in male mice. We also demonstrate that this circuit bidirectionally controls feeding: optogenetic activation of NAc projections to the VTA inhibits food-seeking and food intake (in both sexes), while optogenetic inhibition of this circuit potentiates food-seeking behavior. Additionally, we show that activity of the NAc to VTA pathway is necessary for adaptive inhibition of food intake in response to external cues. These data provide new insight into NAc control over feeding in mice, and contribute to an emerging literature elucidating the role of inhibitory midbrain feedback within the mesolimbic circuit.SIGNIFICANCE STATEMENT The medial NAc has long been known to control consummatory behavior, with particular focus on accumbens projections to the lateral hypothalamus. Conversely, NAc projections to the VTA have mainly been studied in the context of drug reward. We show that NAc projections to the VTA bidirectionally control food intake, consistent with a permissive role in feeding. Additionally, we show that this circuit is normally inactivated during consumption and food-seeking. Together, these findings elucidate how mesolimbic circuits control food consumption.

Neurochemical and Behavioral Dissections of Decision-Making in a Rodent Multistage Task.

Flexible decision-making in dynamic environments requires both retrospective appraisal of reinforced actions and prospective reasoning about the consequences of actions. These complementary reinforcement-learning systems can be characterized computationally with model-free and model-based algorithms, but how these processes interact at a neurobehavioral level in normal and pathological states is unknown. Here, we developed a translationally analogous multistage decision-making (MSDM) task to independently quantify model-free and model-based behavioral mechanisms in rats. We provide the first direct evidence that male rats, similar to humans, use both model-free and model-based learning when making value-based choices in the MSDM task and provide novel analytic approaches for independently quantifying these reinforcement-learning strategies. Furthermore, we report that ex vivo dopamine tone in the ventral striatum and orbitofrontal cortex correlate with model-based, but not model-free, strategies, indicating that the biological mechanisms mediating decision-making in the multistage task are conserved in rats and humans. This new multistage task provides a unique behavioral platform for conducting systems-level analyses of decision-making in normal and pathological states.SIGNIFICANCE STATEMENT Decision-making is influenced by both a retrospective "model-free" system and a prospective "model-based" system in humans, but the biobehavioral mechanisms mediating these learning systems in normal and disease states are unknown. Here, we describe a translationally analogous multistage decision-making task to provide a behavioral platform for conducting neuroscience studies of decision-making in rats. We provide the first evidence that choice behavior in rats is influenced by model-free and model-based systems and demonstrate that model-based, but not model-free, learning is associated with corticostriatal dopamine tone. This novel behavioral paradigm has the potential to yield critical insights into the mechanisms mediating decision-making alterations in mental disorders.

Endocannabinoid contributions to alcohol habits and motivation: Relevance to treatment.

Individuals with alcohol use disorder exhibit compulsive habitual behaviors that are thought to be, in part, a consequence of chronic and persistent use of alcohol. The endocannabinoid system plays a critical role in habit learning and in ethanol self-administration, but the role of this neuromodulatory system in the expression of habitual alcohol seeking is unknown. Here, we investigated the role of the endocannabinoid system in established alcohol habits using contingency degradation in male C57BL/6 mice. We found that administration of the novel diacyl glycerol lipase inhibitor DO34, which decreases the biosynthesis of the endocannabinoid 2-arachidonoyl glycerol (2-AG), reduced habitual responding for ethanol and ethanol approach behaviors. Moreover, administration of the endocannabinoid transport inhibitor AM404 or the cannabinoid receptor type 1 antagonist AM251 produced similar reductions in habitual responding for ethanol and ethanol approach behaviors. Notably, AM404 was also able to reduce ethanol seeking and consumption in mice that were insensitive to lithium chloride-induced devaluation of ethanol. Conversely, administration of JZL184, a monoacyl glycerol lipase inhibitor that increases levels of 2-AG, increased motivation to respond for ethanol on a progressive ratio schedule of reinforcement. These results demonstrate an important role for endocannabinoid signaling in the motivation to seek ethanol, in ethanol-motivated habits, and suggest that pharmacological manipulations of endocannabinoid signaling could be effective therapeutics for treating alcohol use disorder.

Bi-directional modulation of food habit expression by the endocannabinoid system.

The compulsive, habitual behaviors that have been observed in individuals diagnosed with substance use disorders may be due to disruptions in the neural circuits that mediate goal-directed actions. The endocannabinoid system has been shown to play a critical role in habit learning, but the role of this neuromodulatory system in habit expression is unclear. Here, we investigated the role of the endocannabinoid system in established habitual actions using contingency degradation in male C57BL/6 mice. We found that administration of the endocannabinoid transport inhibitor AM404 reduced habitual responding for food and that antagonism of cannabinoid receptor type 1 (CB1), but not transient receptor potential cation subfamily V (TRPV1), receptors produced a similar reduction in habitual responding. Moreover, pharmacological stimulation of CB1 receptors increased habitual responding for food. Co-administration of an enzyme inhibitor that selectively increases the endocannabinoid 2-arachidonoyl glycerol (2-AG) with AM404 partially restored habitual responding for food. Together, these findings demonstrate an important role for the endocannabinoid system in the expression of habits and provide novel insights into potential pharmacological strategies for reducing habitual behaviors in mental disorders.

Dopamine D3 Receptor Availability Is Associated with Inflexible Decision Making.

UNLABELLED: Dopamine D2/3 receptor signaling is critical for flexible adaptive behavior; however, it is unclear whether D2, D3, or both receptor subtypes modulate precise signals of feedback and reward history that underlie optimal decision making. Here, PET with the radioligand [(11)C]-(+)-PHNO was used to quantify individual differences in putative D3 receptor availability in rodents trained on a novel three-choice spatial acquisition and reversal-learning task with probabilistic reinforcement. Binding of [(11)C]-(+)-PHNO in the midbrain was negatively related to the ability of rats to adapt to changes in rewarded locations, but not to the initial learning. Computational modeling of choice behavior in the reversal phase indicated that [(11)C]-(+)-PHNO binding in the midbrain was related to the learning rate and sensitivity to positive, but not negative, feedback. Administration of a D3-preferring agonist likewise impaired reversal performance by reducing the learning rate and sensitivity to positive feedback. These results demonstrate a previously unrecognized role for D3 receptors in select aspects of reinforcement learning and suggest that individual variation in midbrain D3 receptors influences flexible behavior. Our combined neuroimaging, behavioral, pharmacological, and computational approach implicates the dopamine D3 receptor in decision-making processes that are altered in psychiatric disorders. SIGNIFICANCE STATEMENT: Flexible decision-making behavior is dependent upon dopamine D2/3 signaling in corticostriatal brain regions. However, the role of D3 receptors in adaptive, goal-directed behavior has not been thoroughly investigated. By combining PET imaging with the D3-preferring radioligand [(11)C]-(+)-PHNO, pharmacology, a novel three-choice probabilistic discrimination and reversal task and computational modeling of behavior in rats, we report that naturally occurring variation in [(11)C]-(+)-PHNO receptor availability relates to specific aspects of flexible decision making. We confirm these relationships using a D3-preferring agonist, thus identifying a unique role of midbrain D3 receptors in decision-making processes.

In the blink of an eye: relating positive-feedback sensitivity to striatal dopamine D2-like receptors through blink rate.

For >30 years, positron emission tomography (PET) has proven to be a powerful approach for measuring aspects of dopaminergic transmission in the living human brain; this technique has revealed important relationships between dopamine D2-like receptors and dimensions of normal behavior, such as human impulsivity, and psychopathology, particularly behavioral addictions. Nevertheless, PET is an indirect estimate that lacks cellular and functional resolution and, in some cases, is not entirely pharmacologically specific. To identify the relationships between PET estimates of D2-like receptor availability and direct in vitro measures of receptor number, affinity, and function, we conducted neuroimaging and behavioral and molecular pharmacological assessments in a group of adult male vervet monkeys. Data gathered from these studies indicate that variation in D2-like receptor PET measurements is related to reversal-learning performance and sensitivity to positive feedback and is associated with in vitro estimates of the density of functional dopamine D2-like receptors. Furthermore, we report that a simple behavioral measure, eyeblink rate, reveals novel and crucial links between neuroimaging assessments and in vitro measures of dopamine D2 receptors.

Primate phencyclidine model of schizophrenia: sex-specific effects on cognition, brain derived neurotrophic factor, spine synapses, and dopamine turnover in prefrontal cortex.

BACKGROUND: Cognitive deficits are a core symptom of schizophrenia, yet they remain particularly resistant to treatment. The model provided by repeatedly exposing adult nonhuman primates to phencyclidine has generated important insights into the neurobiology of these deficits, but it remains possible that administration of this psychotomimetic agent during the pre-adult period, when the dorsolateral prefrontal cortex in human and nonhuman primates is still undergoing significant maturation, may provide a greater understanding of schizophrenia-related cognitive deficits. METHODS: The effects of repeated phencyclidine treatment on spine synapse number, dopamine turnover and BDNF expression in dorsolateral prefrontal cortex, and working memory accuracy were examined in pre-adult monkeys. RESULTS: One week following phencyclidine treatment, juvenile and adolescent male monkeys demonstrated a greater loss of spine synapses in dorsolateral prefrontal cortex than adult male monkeys. Further studies indicated that in juvenile males, a cognitive deficit existed at 4 weeks following phencyclidine treatment, and this impairment was associated with decreased dopamine turnover, decreased brain derived neurotrophic factor messenger RNA, and a loss of dendritic spine synapses in dorsolateral prefrontal cortex. In contrast, female juvenile monkeys displayed no cognitive deficit at 4 weeks after phencyclidine treatment and no alteration in dopamine turnover or brain derived neurotrophic factor messenger RNA or spine synapse number in dorsolateral prefrontal cortex. In the combined group of male and female juvenile monkeys, significant linear correlations were detected between dopamine turnover, spine synapse number, and cognitive performance. CONCLUSIONS: As the incidence of schizophrenia is greater in males than females, these findings support the validity of the juvenile primate phencyclidine model and highlight its potential usefulness in understanding the deficits in dorsolateral prefrontal cortex in schizophrenia and developing novel treatments for the cognitive deficits associated with schizophrenia.

Sex differences in oxycodone-taking behaviors are linked to disruptions in reward-guided, decision-making functions.

Problematic opioid use that emerges in a subset of individuals may be due to pre-existing disruptions in the biobehavioral mechanisms that regulate drug use. The identity of these mechanisms is not known, but emerging evidence suggests that suboptimal decision-making that is observable prior to drug use may contribute to the pathology of addiction and, notably, serve as a powerful phenotype for interrogating biologically based differences in opiate-taking behaviors. The current study investigated the relationship between decision-making phenotypes and opioid-taking behaviors in male and female Long Evans rats. Adaptive decision-making processes were assessed using a probabilistic reversal-learning task and oxycodone- (or vehicle, as a control) taking behaviors assessed for 32 days using a saccharin fading procedure that promoted dynamic intake of oxycodone. Tests of motivation, extinction, and reinstatement were also performed. Computational analyses of decision-making and opioid-taking behaviors revealed that attenuated reward-guided decision-making was associated with greater self-administration of oxycodone and addiction-relevant behaviors. Moreover, pre-existing impairments in reward-guided decision-making observed in female rats was associated with greater oxycodone use and addiction-relevant behaviors when compared to males. These results provide new insights into the biobehavioral mechanisms that regulate opiate-taking behaviors and offer a novel phenotypic approach for interrogating sex differences in addiction susceptibility and opioid use disorders.

Lesions to the mediodorsal thalamus, but not orbitofrontal cortex, enhance volatility beliefs linked to paranoia.

Beliefs-attitudes toward some state of the environment-guide action selection and should be robust to variability but sensitive to meaningful change. Beliefs about volatility (expectation of change) are associated with paranoia in humans, but the brain regions responsible for volatility beliefs remain unknown. The orbitofrontal cortex (OFC) is central to adaptive behavior, whereas the magnocellular mediodorsal thalamus (MDmc) is essential for arbitrating between perceptions and action policies. We assessed belief updating in a three-choice probabilistic reversal learning task following excitotoxic lesions of the MDmc (n = 3) or OFC (n = 3) and compared performance with that of unoperated monkeys (n = 14). Computational analyses indicated a double dissociation: MDmc, but not OFC, lesions were associated with erratic switching behavior and heightened volatility belief (as in paranoia in humans), whereas OFC, but not MDmc, lesions were associated with increased lose-stay behavior and reward learning rates. Given the consilience across species and models, these results have implications for understanding paranoia.

Dysregulation of D2-mediated dopamine transmission in monkeys after chronic escalating methamphetamine exposure.

Compulsive drug-seeking and drug-taking are important substance-abuse behaviors that have been linked to alterations in dopaminergic neurotransmission and to impaired inhibitory control. Evidence supports the notions that abnormal D2 receptor-mediated dopamine transmission and inhibitory control may be heritable risk factors for addictions, and that they also reflect drug-induced neuroadaptations. To provide a mechanistic explanation for the drug-induced emergence of inhibitory-control deficits, this study examined how a chronic, escalating-dose regimen of methamphetamine administration affected dopaminergic neurochemistry and cognition in monkeys. Dopamine D2-like receptor and dopamine transporter (DAT) availability and reversal-learning performance were measured before and after exposure to methamphetamine (or saline), and brain dopamine levels were assayed at the conclusion of the study. Exposure to methamphetamine reduced dopamine D2-like receptor and DAT availability and produced transient, selective impairments in the reversal of a stimulus-outcome association. Furthermore, individual differences in the change in D2-like receptor availability in the striatum were related to the change in response to positive feedback. These data provide evidence that chronic, escalating-dose methamphetamine administration alters the dopamine system in a manner similar to that observed in methamphetamine-dependent humans. They also implicate alterations in positive-feedback sensitivity associated with D2-like receptor dysfunction as the mechanism by which inhibitory control deficits emerge in stimulant-dependent individuals. Finally, a significant degree of neurochemical and behavioral variation in response to methamphetamine was detected, indicating that individual differences affect the degree to which drugs of abuse alter these processes. Identification of these factors ultimately may assist in the development of individualized treatments for substance dependence.

Examining sex differences in responses to footshock stress and the role of the metabotropic glutamate receptor 5: an [18F]FPEB and positron emission tomography study in rats.

Clinical investigations suggest involvement of the metabotropic glutamate receptor 5 (mGluR5) in the pathophysiology of fear learning that underlies trauma-related disorders. Here, we utilized a 4-day fear learning paradigm combined with positron emission tomography (PET) to examine the relationship between mGluR5 availability and differences in the response of rats to repeated footshock exposure (FE). Specifically, on day 1, male (n = 16) and female (n = 12) rats received 15 footshocks and were compared with control rats who did not receive footshocks (n = 7 male; n = 4 female). FE rats were classified as low responders (LR) or high responders (HR) based on freezing to the context the following day (day 2). PET with [18F]FPEB was used to calculate regional mGluR5 binding potential (BPND) at two timepoints: prior to FE (i.e., baseline), and post-behavioral testing. Additionally, we used an unbiased proteomics approach to assess group and sex differences in prefrontal cortex (PFC) protein expression. Post-behavioral testing we observed decreased BPND in LR females, but increased BPND in HR males relative to baseline. Further, individuals displaying the greatest freezing during the FE context memory test had the largest increases in PFC BPND. Males and females displayed unique post-test molecular profiles: in males, the greatest differences were between FE and CON, including upregulation of mGluR5 and related molecular networks in FE, whereas the greatest differences among females were between the LR and HR groups. These findings suggest greater mGluR5 availability increases following footshock exposure may be related to greater contextual fear memory. Results additionally reveal sex differences in the molecular response to footshock, including differential involvement of mGluR5-related molecular networks.

Dorsal striatal D2-like receptor availability covaries with sensitivity to positive reinforcement during discrimination learning.

Deviations in reward sensitivity and behavioral flexibility, particularly in the ability to change or stop behaviors in response to changing environmental contingencies, are important phenotypic dimensions of several neuropsychiatric disorders. Neuroimaging evidence suggests that variation in dopamine signaling through dopamine D(2)-like receptors may influence these phenotypes, as well as associated psychiatric conditions, but the specific neurocognitive mechanisms through which this influence is exerted are unknown. To address this question, we examined the relationship between behavioral sensitivity to reinforcement during discrimination learning and D(2)-like receptor availability in vervet monkeys. Monkeys were assessed for their ability to acquire, retain, and reverse three-choice, visual-discrimination problems, and once behavioral performance had stabilized, they received positron emission tomography (PET) scans. D(2)-like receptor availability in dorsal aspects of the striatum was not related to individual differences in the ability to acquire or retain visual discriminations but did relate to the number of trials required to reach criterion in the reversal phase of the task. D(2)-like receptor availability was also strongly correlated with behavioral sensitivity to positive, but not negative, feedback during learning. These results go beyond electrophysiological findings by demonstrating the involvement of a striatal dopaminergic marker in individual differences in feedback sensitivity and behavioral flexibility, providing insight into the neural mechanisms that are affected in neuropsychiatric disorders that feature these deficits.

Cognitive control and the dopamine D2-like receptor: a dimensional understanding of addiction.

The phenotypic complexity of psychiatric conditions is revealed by the dimensional nature of these disorders, which consist of multiple behavioral, affective, and cognitive dysfunctions that can result in substantial psychosocial impairment. The high degree of heterogeneity in symptomatology and comorbidity suggests that simple categorical diagnoses of "affected" or "unaffected" may fail to capture the true characteristics of the disorder in a manner relevant to individualized treatment. A particular dimension of interest is cognitive control ability because impairments in the capacity to control thoughts, feelings, and actions are key to several psychiatric disorders. Here, we describe evidence suggesting that cognitive control over behavior is a crucial dimension of function relevant to addictions. Moreover, dopamine (DA) D(2)-receptor transmission is increasingly being identified as a point of convergence for these behavioral and cognitive processes. Consequently, we argue that measures of cognitive control and D(2) DA receptor function may be particularly informative markers of individual function and treatment response in addictions.

Adolescent reinforcement-learning trajectories predict cocaine-taking behaviors in adult male and female rats.

The anatomical, structural, and functional adaptations that occur in the brain during adolescence are thought to facilitate improvements in decision-making functions that are known to occur during this stage of development. The mechanisms that underlie these neural adaptations are not known, but deviations in developmental trajectories have been proposed to contribute to the emergence of mental illness, including addiction. Direct evidence supporting this hypothesis, however, has been limited. Here, we used a recently developed reversal-learning protocol to investigate the predictive relationship between adolescent decision-making trajectories and cocaine-taking behaviors in adulthood. Decision-making functions in the reversal-learning task were assessed throughout adolescence and into adulthood in male and female Long-Evans rats. Trial-by-trial choice data was fitted with a reinforcement-learning model to quantify the degree to which choice behavior of individual rats was influenced by rewarded (e.g., ∆+ parameter) and unrewarded (e.g., ∆0 parameter) outcomes. We report that reversal-learning performance improved during adolescence and that this was due to an increase in value updating for rewarded outcomes (e.g., ∆+ parameter). Furthermore, the rate of change in the ∆+ parameter predicted individual differences in the ∆+ parameter and, notably, cocaine-taking behaviors in adulthood: Rats that had a shallower adolescent trajectory were found to have a lower ∆+ parameter and greater cocaine self-administration in adulthood. These data indicate that adolescent development plays a critical role in drug use susceptibility. Future studies aimed at understanding the neurobiological mechanisms that underlie these age-related changes in decision-making could provide new insights into the biobehavioral mechanisms mediating addiction susceptibility.

Reinforcement learning detuned in addiction: integrative and translational approaches.

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

MicroPET evidence for a hypersensitive neuroinflammatory profile of gp120 mouse model of HIV.

Despite increased survivability for people living with HIV (PLWH), HIV-related cognitive deficits persist. Determining biological mechanism(s) underlying abnormalities is critical to minimize the long-term impact of HIV. Positron emission tomography (PET) studies reveal that PLWH exhibit elevated neuroinflammation, potentially contributing to these problems. PLWH are hypersensitive to environmental insults that drive elevated inflammatory profiles. Gp120 is an envelope glycoprotein exposed on the surface of the HIV envelope which enables HIV entry into a cell contributing to HIV-related neurotoxicity. In vivo evidence for mice overexpressing gp120 (transgenic) mice exhibiting neuroinflammation remains unclear. Here, we conducted microPET imaging in gp120 transgenic and wildtype mice, using the radiotracer [(18)F]FEPPA (binds to the translocator protein expressed by activated microglial serving as a neuroinflammatory marker). Imaging was performed at baseline and 24 h after lipopolysaccharide (LPS; 5 mg/kg) treatment (endotoxin that triggers an immune response). Gp120 transgenic mice exhibited elevated [(18F)]FEPPA in response to LPS vs. wildtype mice throughout the brain including dorsal and ventral striata, hypothalamus, and hippocampus. Gp120 transgenic mice are hypersensitive to environmental inflammatory insults, consistent with PLWH, measurable in vivo. It remains to-be-determined whether this heightened sensitivity is connected to the behavioral abnormalities of these mice or sensitive to any treatments.

Unlocking the reinforcement-learning circuits of the orbitofrontal cortex.

Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders. (PsycInfo Database Record (c) 2021 APA, all rights reserved).