Distinct action signals by subregions in nucleus accumbens during STOP-change performance.

Nucleus accumbens (NAc) is thought to contribute to motivated behavior by signaling the value of reward-predicting cues and the delivery of anticipated reward. NAc is subdivided into core and shell, with each region containing different populations of neurons that increase or decrease firing to rewarding events. While there are various and numerous theories of functions pertaining to these subregions and cell types, most are in the context of reward processing, with fewer considering that NAc might serve functions related to action selection more generally. We recorded from single neurons in NAc as rats of both sexes performed a STOP-change task that is commonly used to study motor control and impulsivity. In this task, rats respond quickly to a spatial cue on 80% of trials (GO) and must stop and redirect planned movement on 20% of trials (STOP). We found that the activity of reward-excited neurons signaled accurate response direction on GO, but not STOP trials, and that these neurons exhibited higher pre-cue firing after correct trials resulting in stronger firing during correct GO trials and errant STOP trials, while reward-inhibited neurons significantly represented response direction on STOP trials at the time of the instrumental response. Finally, the proportion of reward-excited to reward-inhibited neurons and the strength of pre-cue firing decreased as the electrode traversed NAc. We conclude that reward-excited cells (more common in core) promote proactive action selection, while reward-inhibited cells (more common in shell) contribute to accurate responding on STOP trials that require reactive suppression and redirection of behavior.Significant Statement The ability to appropriately adapt behavior is an important part of human cognition, and one that is disrupted by many neuropsychiatric disorders. Here we recorded from neurons in the nucleus accumbens (NAc) as rats performed a cognitive control task and found cell type- and subregion-specific firing patterns. Core and reward-excited cells track trial outcome history, proactively driving behavior to the first cue-a strategy that is appropriate for most trials. Conversely, shell and reward-inhibited neurons signal accurate response direction on trials requiring redirection of behavior. Together, these data suggest that NAc neuronal populations differentially contribute to action selection.

Optogenetic Inhibition of Rat Anterior Cingulate Cortex Impairs the Ability to Initiate and Stay on Task.

Our prior research has identified neural correlates of cognitive control in the anterior cingulate cortex (ACC), leading us to hypothesize that the ACC is necessary for increasing attention as rats flexibly learn new contingencies during a complex reward-guided decision-making task. Here, we tested this hypothesis by using optogenetics to transiently inhibit the ACC, while rats of either sex performed the same two-choice task. ACC inhibition had a profound impact on behavior that extended beyond deficits in attention during learning when expected outcomes were uncertain. We found that ACC inactivation slowed and reduced the number of trials rats initiated and impaired both their accuracy and their ability to complete sessions. Furthermore, drift-diffusion model analysis suggested that free-choice performance and evidence accumulation (i.e., reduced drift rates) were degraded during initial learning-leading to weaker associations that were more easily overridden in later trial blocks (i.e., stronger bias). Together, these results suggest that in addition to attention-related functions, the ACC contributes to the ability to initiate trials and generally stay on task.

Prior cocaine self-administration does not impair the ability to delay gratification in rats during diminishing returns.

Previous exposure to drugs of abuse produces impairments in studies of reversal learning, delay discounting and response inhibition tasks. While these studies contribute to the understanding of normal decision-making and how it is impaired by drugs of abuse, they do not fully capture how decision-making impacts the ability to delay gratification for greater long-term benefit. To address this issue, we used a diminishing returns task to study decision-making in rats that had previously self-administered cocaine. This task was designed to test the ability of the rat to choose to delay gratification in the short-term to obtain more reward over the course of the entire behavioral session. Rats were presented with two choices. One choice had a fixed amount of time delay needed to obtain reward [i.e. fixed delay (FD)], while the other choice had a progressive delay (PD) that started at 0 s and progressively increased by 1 s each time the PD option was selected. During the 'reset' variation of the task, rats could choose the FD option to reset the time delay associated with the PD option. Consistent with previous results, we found that prior cocaine exposure reduced rats' overall preference for the PD option in post-task reversal testing during 'no-reset' sessions, suggesting that cocaine exposure made rats more sensitive to the increasing delay of the PD option. Surprisingly, however, we found that rats that had self-administered cocaine 1-month prior, adapted behavior during 'reset' sessions by delaying gratification to obtain more reward in the long run similar to control rats.

Minimal cross-trial generalization in learning the representation of an odor-guided choice task.

There is no single way to represent a task. Indeed, despite experiencing the same task events and contingencies, different subjects may form distinct task representations. As experimenters, we often assume that subjects represent the task as we envision it. However, such a representation cannot be taken for granted, especially in animal experiments where we cannot deliver explicit instruction regarding the structure of the task. Here, we tested how rats represent an odor-guided choice task in which two odor cues indicated which of two responses would lead to reward, whereas a third odor indicated free choice among the two responses. A parsimonious task representation would allow animals to learn from the forced trials what is the better option to choose in the free-choice trials. However, animals may not necessarily generalize across odors in this way. We fit reinforcement-learning models that use different task representations to trial-by-trial choice behavior of individual rats performing this task, and quantified the degree to which each animal used the more parsimonious representation, generalizing across trial types. Model comparison revealed that most rats did not acquire this representation despite extensive experience. Our results demonstrate the importance of formally testing possible task representations that can afford the observed behavior, rather than assuming that animals' task representations abide by the generative task structure that governs the experimental design.

Anterior cingulate cortex is necessary for adaptation of action plans.

Previous research has focused on the anterior cingulate cortex (ACC) as a key brain region in the mitigation of the competition that arises from two simultaneously active signals. However, to date, no study has demonstrated that ACC is necessary for this form of behavioral flexibility, nor have any studies shown that ACC acts by modulating downstream brain regions such as the dorsal medial striatum (DMS) that encode action plans necessary for task completion. Here, we performed unilateral excitotoxic lesions of ACC while recording downstream from the ipsilateral hemisphere of DMS in rats, performing a variant of the STOP-signal task. We show that on STOP trials lesioned rats perform worse, in part due to the failure of timely directional action plans to emerge in the DMS, as well as the overrepresentation of the to-be-inhibited behavior. Collectively, our findings suggest that ACC is necessary for the mitigation of competing inputs and validates many of the existing theoretical predictions for the role of ACC in cognitive control.

Prior Cocaine Exposure Increases Firing to Immediate Reward While Attenuating Cue and Context Signals Related to Reward Value in the Insula.

The insula contributes to behavioral control and is disrupted by substance abuse, yet we know little about the neural signals underlying these functions or how they are disrupted after chronic drug self-administration. Here, male and female rats self-administered either cocaine (experimental group) or sucrose (control) for 12 consecutive days. After a 1 month withdrawal period, we recorded from insula while rats performed a previously learned reward-guided decision-making task. Cocaine-exposed rats were more sensitive to value manipulations and were faster to respond. These behavioral changes were accompanied by elevated counts of neurons in the insula that increased firing to reward. These neurons also fired more strongly at the start of long-delay trials, when a more immediate reward would be expected, and fired less strongly in anticipation of the actual delivery of delayed rewards. Although reward-related firing to immediate reward was enhanced after cocaine self-administration, reward-predicting cue and context signals were attenuated. In addition to revealing novel firing patterns unique to insula, our data suggest changes in such neural activity likely contribute to impaired decision making observed after drug use.SIGNIFICANCE STATEMENT The insula plays a clear role in drug addiction and drug-induced impairments of decision making, yet there is little understanding of its underlying neural signals. We found that chronic cocaine self-administration reduces cue and context encoding in insula while enhancing signals related to immediate reward. These changes in neural activity likely contribute to impaired decision making and impulsivity observed after drug use.

Overexpressing Histone Deacetylase 5 in Rat Dorsal Striatum Alters Reward-Guided Decision-Making and Associated Neural Encoding.

Accumulating evidence in the past decade implicates histone-modifying enzymes, such as class I histone deacetylases (HDACs), in learning and memory and, recently, habit formation. However, it is unclear whether HDACs play roles in complex cognitive function. To address this issue, we examined the role of dorsal striatal HDAC5, a class II HDAC, in reward-guided decision-making and associated neural encoding in rats. We first injected adeno-associated virus to overexpress a nuclear-localized HDAC5 in dorsal striatum (DS). We then recorded neural correlates from dorsolateral striatum (DLS) as rats performed two reward-guided choice tasks, in which we manipulated either the size of or delay to reward. During these tasks, rats first learned which of two options led to the better reward and then reversed those contingencies in a second block of trials. We found that rats with HDAC5 overexpression in DS responded faster and chose higher value reward more often during the first block of trials but were less able to reverse those contingencies in the second block of trials. At the neural level, HDAC5 overexpression in DS elevated and reduced the number of cells in DLS that increased firing to stimuli and reward, respectively, and shifted encoding toward cues that predicted more immediate reward. These results suggest that the HDAC5 overexpression in DS contributes to inflexible decision-making, demonstrating a role of histone-modifying enzymes in complex cognitive function.SIGNIFICANCE STATEMENT HDACs are important for learning and habit formation. Here, we expanded on these functions and found that overexpression of HDAC5 produced faster and more automatic behavior, and related changes in dorsolateral striatal neural firing in rats performing a value-based decision-making task. These results implicate HDAC5 as a potential therapeutic target for psychiatric conditions that impair decision-making and executive function.

Neural Signals in Red Nucleus during Reactive and Proactive Adjustments in Behavior.

The ability to adjust behavior is an essential component of cognitive control. Much is known about frontal and striatal processes that support cognitive control, but few studies have investigated how motor signals change during reactive and proactive adjustments in motor output. To address this, we characterized neural signals in red nucleus (RN), a brain region linked to motor control, as male and female rats performed a novel variant of the stop-signal task. We found that activity in RN represented the direction of movement and was strongly correlated with movement speed. Additionally, we found that directional movement signals were amplified on STOP trials before completion of the response and that the strength of RN signals was modulated when rats exhibited cognitive control. These results provide the first evidence that neural signals in RN integrate cognitive control signals to reshape motor outcomes reactively within trials and proactivity across them.SIGNIFICANCE STATEMENT Healthy human behavior requires the suppression or inhibition of errant or maladaptive motor responses, often called cognitive control. While much is known about how frontal brain regions facilitate cognitive control, less is known about how motor regions respond to rapid and unexpected changes in action selection. To address this, we recorded from neurons in the red nucleus, a motor region thought to be important for initiating movement in rats performing a cognitive control task. We show that red nucleus tracks motor plans and that selectivity was modulated on trials that required shifting from one motor response to another. Collectively, these findings suggest that red nucleus contributes to modulating motor behavior during cognitive control.

In Vitro and In Vivo Sequestration of Phencyclidine by Me4 Cucurbit[8]uril*.

We report investigations of the use of cucurbit[8]uril (CB[8]) macrocycles as an antidote to counteract the in vivo biological effects of phencyclidine. We investigate the binding of CB[8] and its derivative Me4 CB[8] toward ten drugs of abuse (3-9, 12-14) by a combination of 1 H NMR spectroscopy and isothermal titration calorimetry in phosphate buffered water. We find that the cavity of CB[8] and Me4 CB[8] are able to encapsulate the 1-amino-1-aryl-cyclohexane ring system of phencyclidine (PCP) and ketamine as well as the morphinan skeleton of morphine and hydromorphone with Kd values ≤50 nm. In vitro cytotoxicity (MTS metabolic and adenylate kinase cell death assays in HEK293 and HEPG2 cells) and in vivo maximum tolerated dose studies (Swiss Webster mice) which were performed for Me4 CB[8] indicated good tolerability. The tightest host⋅guest pair (Me4 CB[8]⋅PCP; Kd =2 nm) was advanced to in vivo efficacy studies. The results of open field tests demonstrate that pretreatment of mice with Me4 CB[8] prevents subsequent hyperlocomotion induction by PCP and also that treatment of animals previously dosed with PCP with Me4 CB[8] significantly reduces the locomotion levels.

In Vitro and In Vivo Sequestration of Methamphetamine by a Sulfated Acyclic CB[n]-Type Receptor.

We report the synthesis of two new acyclic sulfated acyclic CB[n]-type receptors (TriM0 and Me4 TetM0) and investigations of their binding properties toward a panel of drugs of abuse (1-13) by a combination of 1 H NMR spectroscopy and isothermal titration calorimetry. TetM0 is the most potent receptor with Ka ≥106  M-1 toward methamphetamine, fentanyl, MDMA and mephedrone. TetM0 is not cytotoxic toward HepG2 and HEK 293 cells below 100 µM according to MTS metabolic and adenylate kinase release assays and is well tolerated in vivo when dosed at 46 mg kg-1 . TetM0 does not inhibit the hERG ion channel and is not mutagenic based on the Ames fluctuation test. Finally, in vivo efficacy studies show that the hyperlocomotion of mice treated with methamphetamine can be greatly reduced by treatment with TetM0 up to 5 minutes later. TetM0 has potential as a broad spectrum in vivo sequestrant for drugs of abuse.

Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release.

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Enduring consequences of perinatal fentanyl exposure in mice.

Opioid use by pregnant women is an understudied consequence associated with the opioid epidemic, resulting in a rise in the incidence of neonatal opioid withdrawal syndrome (NOWS) and lifelong neurobehavioral deficits that result from perinatal opioid exposure. There are few preclinical models that accurately recapitulate human perinatal drug exposure and few focus on fentanyl, a potent synthetic opioid that is a leading driver of the opioid epidemic. To investigate the consequences of perinatal opioid exposure, we administered fentanyl to mouse dams in their drinking water throughout gestation and until litters were weaned at postnatal day (PD) 21. Fentanyl-exposed dams delivered smaller litters and had higher litter mortality rates compared with controls. Metrics of maternal care behavior were not affected by the treatment, nor were there differences in dams' weight or liquid consumption throughout gestation and 21 days postpartum. Twenty-four hours after weaning and drug cessation, perinatal fentanyl-exposed mice exhibited signs of spontaneous somatic withdrawal behavior and sex-specific weight fluctuations that normalized in adulthood. At adolescence (PD 35), they displayed elevated anxiety-like behaviors and decreased grooming, assayed in the elevated plus maze and sucrose splash tests. Finally, by adulthood (PD 55), they displayed impaired performance in a two-tone auditory discrimination task. Collectively, our findings suggest that perinatal fentanyl-exposed mice exhibit somatic withdrawal behavior and change into early adulthood reminiscent of humans born with NOWS.

Single Neurons in Anterior Cingulate Cortex Signal the Need to Change Action During Performance of a Stop-change Task that Induces Response Competition.

Several human imaging studies have suggested that anterior cingulate cortex (ACC) is highly active when participants receive competing inputs, and that these signals may be important for influencing the downstream planning of actions. Despite increasing evidence from several neuroimaging studies, no study has examined ACC activity at the level of the single neuron in rodents performing similar tasks. To fill this gap, we recorded from single neurons in ACC while rats performed a stop-change task. We found higher firing on trials with competing inputs (STOP trials), and that firing rates were positively correlated with accuracy and movement speed, suggesting that when ACC was engaged, rats tended to slow down and perform better. Finally, firing was the strongest when STOP trials were preceded by GO trials and was reduced when rats adapted their behavior on trials subsequent to a STOP trial. These data provide the first evidence that activity of single neurons in ACC is elevated when 2 responses are in competition with each other when there is a need to change the course of action to obtain reward.

Neural Activity in Ventral Medial Prefrontal Cortex Is Modulated More Before Approach Than Avoidance During Reinforced and Extinction Trial Blocks.

Ventromedial prefrontal cortex (vmPFC) is thought to provide regulatory control over Pavlovian fear responses and has recently been implicated in appetitive approach behavior, but much less is known about its role in contexts in which appetitive and aversive outcomes can be obtained and avoided, respectively. To address this issue, we recorded from single neurons in vmPFC while male rats performed our combined approach and avoidance task under reinforced and non-reinforced (extinction) conditions. Surprisingly, we found that cues predicting reward modulated cell firing in vmPFC more often and more robustly than cues preceding avoidable shock; in addition, firing of vmPFC neurons was both response (press or no-press) and outcome (reinforced or extinction) selective. These results suggest a complex role for vmPFC in regulating behavior and support its role in appetitive contexts during both reinforced and non-reinforced conditions.SIGNIFICANCE STATEMENT Selecting context-appropriate behaviors to gain reward or avoid punishment is critical for survival. Although the role of ventromedial prefrontal cortex (vmPFC) in mediating fear responses is well established, vmPFC has also been implicated in the regulation of reward-guided approach and extinction. Many studies have used indirect methods and simple behavioral procedures to study vmPFC, which leaves the literature incomplete. We recorded vmFPC neural activity during a complex cue-driven combined approach and avoidance task and during extinction. Surprisingly, we found very little vmPFC modulation to cues predicting avoidable shock, whereas cues predicting reward approach robustly modulated vmPFC firing in a response- and outcome-selective manner. This suggests a more complex role for vmPFC than current theories suggest, specifically regarding context-specific behavioral optimization.

The impact of drugs of abuse on executive function: characterizing long-term changes in neural correlates following chronic drug exposure and withdrawal in rats.

Addiction has long been characterized by diminished executive function, control, and impulsivity management. In particular, these deficits often manifest themselves as impairments in reversal learning, delay discounting, and response inhibition. Understanding the neurobiological substrates of these behavioral deficits is of paramount importance to our understanding of addiction. Within the cycle of addiction, periods during and after withdrawal represent a particularly difficult point of intervention in that the negative physical symptoms associated with drug removal and drug craving increase the likelihood that the patient will relapse and return to drug use in order to abate these symptoms. Moreover, it is often during this time that drug induced deficits in executive function hinder the ability of the patient to refrain from drug use. Thus, it is necessary to understand the physiological and behavioral changes associated with withdrawal and drug craving-largely manifesting as deficits in executive control-to develop more effective treatment strategies. In this review, we address the long-term impact that drugs of abuse have on the behavioral and neural correlates that give rise to executive control as measured by reversal learning, delay discounting, and stop-signal tasks, focusing particularly on our work using rats as a model system.

Prior Cocaine Self-Administration Increases Response-Outcome Encoding That Is Divorced from Actions Selected in Dorsal Lateral Striatum.

Dorsal lateral striatum (DLS) is a highly associative structure that encodes relationships among environmental stimuli, behavioral responses, and predicted outcomes. DLS is known to be disrupted after chronic drug abuse; however, it remains unclear what neural signals in DLS are altered. Current theory suggests that drug use enhances stimulus-response processing at the expense of response-outcome encoding, but this has mostly been tested in simple behavioral tasks. Here, we investigated what neural correlates in DLS are affected by previous cocaine exposure as rats performed a complex reward-guided decision-making task in which predicted reward value was independently manipulated by changing the delay to or size of reward associated with a response direction across a series of trial blocks. After cocaine self-administration, rats exhibited stronger biases toward higher-value reward and firing in DLS more strongly represented action-outcome contingencies independent from actions subsequently taken rather than outcomes predicted by selected actions (chosen-outcome contingencies) and associations between stimuli and actions (stimulus-response contingencies). These results suggest that cocaine self-administration strengthens action-outcome encoding in rats (as opposed to chosen-outcome or stimulus-response encoding), which abnormally biases behavior toward valued reward when there is a choice between two options during reward-guided decision-making.SIGNIFICANCE STATEMENT Current theories suggest that the impaired decision-making observed in individuals who chronically abuse drugs reflects a decrease in goal-directed behaviors and an increase in habitual behaviors governed by neural representations of response-outcome (R-O) and stimulus-response associations, respectively. We examined the impact that prior cocaine self-administration had on firing in dorsal lateral striatum (DLS), a brain area known to be involved in habit formation and affected by drugs of abuse, during performance of a complex reward-guided decision-making task. Surprisingly, we found that previous cocaine exposure enhanced R-O associations in DLS. This suggests that there may be more complex consequences of drug abuse than current theories have explored, especially when examining brain and behavior in the context of a complex two-choice decision-making task.

Executive control signals in orbitofrontal cortex during response inhibition.

Orbitofrontal cortex (OFC) lesions produce deficits in response inhibition and imaging studies suggest that activity in OFC is stronger on trials that require suppression of behavior, yet few studies have examined neural correlates at the single-unit level in a behavioral task that probes response inhibition without varying other factors, such as anticipated outcomes. Here we recorded from single neurons in lateral OFC in a task that required animals in the minority of trials to STOP or inhibit an ongoing movement and respond in the opposite direction. We found that population and single-unit firing was modulated primarily by response direction and movement speed, and that very few OFC neurons exhibited a response independent inhibition signal. Remarkably, the strength of the directional signal was not diminished on STOP trials and was actually stronger on STOP trials during conflict adaptation. Finally, directional signals were stronger during sessions in which rats had the most difficulty inhibiting behavior. These results suggest that "inhibition" deficits observed with OFC interference studies reflect deficits unrelated to signaling the need to inhibit behavior, but instead support a role for OFC in executive functions related to dissociating between two perceptually similar actions during response conflict.

Altered basolateral amygdala encoding in an animal model of schizophrenia.

It has been proposed that schizophrenia results, in part, from the inappropriate or spurious attribution of salience to cues in the environment. We have recently reported neural correlates of salience in the basolateral amygdala (ABL) of rats during learning in an odor-guided discrimination task. Here we tested whether this dopamine-dependent salience signal is altered in rats with neonatal ventral hippocampal lesions (NVHLs), a rodent model of schizophrenia. We found that ABL signals related to violations in reward prediction were only mildly affected by NVHL; however, neurons in rats with NVHLs showed significantly stronger selectivity during odor sampling, particularly for the more salient large-reward cue. The elevated cue-evoked activity in NVHL rats was correlated with heightened orienting behavior and also with changes in firing to the shifts in reward, suggesting that it reflected abnormal signaling of the large reward-predicting cue's salience. These results are broadly consistent with the proposal that schizophrenics suffer from enhanced signaling of salience.

Rule encoding in dorsal striatum impacts action selection.

Cognitive flexibility is a hallmark of prefrontal cortical (PFC) function yet little is known about downstream area involvement. The medial dorsal striatum (mDS) receives major projections from the PFC and is uniquely situated to perform the integration of responses with rule information. In this study, we use a novel rule shifting task in rats that mirrors non-human primate and human studies in its temporal precision and counterbalanced responses. We record activity from single neurons in the mDS while rats switch between different rules for reward. Additionally, we pharmacologically inactivate mDS by infusion of a baclofen/muscimol cocktail. Inactivation of mDS impaired the ability to shift to a new rule and increased the number of regressive errors. While recording in mDS, we identified neurons modulated by direction whose activity reflected the conflict between competing rule information. We show that a subset of these neurons was also rule selective, and that the conflict between competing rule cues was resolved as behavioural performance improved. Other neurons were modulated by rule, but not direction. These neurons became selective before behavioural performance accurately reflected the current rule. These data provide an additional locus for investigating the mechanisms underlying behavioural flexibility. Converging lines of evidence from multiple human psychiatric disorders have implicated dorsal striatum as an important and understudied neural substrate of flexible cognition. Our data confirm the importance of mDS, and suggest a mechanism by which mDS mediates abstract cognition functions.

From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making.

The striatum is critical for reward-guided and habitual behavior. Anatomical and interference studies suggest a functional heterogeneity within striatum. Medial regions, such as nucleus accumbens core and dorsal medial striatum play roles in goal-directed behavior, while dorsal lateral striatum is critical for control of habitual action. Subdivisions of striatum are topographically connected with different cortical and subcortical structures forming channels that carry information related to limbic, associative, and sensorimotor functions. Here, we describe data showing that as one progresses from ventral-medial to dorsal-lateral striatum, there is a shift from more prominent value encoding to activity more closely related to associative and motor aspects of decision-making. In addition, we will describe data suggesting that striatal circuits work in parallel to control behavior and that regions within striatum can compensate for each other when functions are disrupted.