Minimal cross-trial generalization in learning the representation of an odor-guided choice task.

There is no single way to represent a task. Indeed, despite experiencing the same task events and contingencies, different subjects may form distinct task representations. As experimenters, we often assume that subjects represent the task as we envision it. However, such a representation cannot be taken for granted, especially in animal experiments where we cannot deliver explicit instruction regarding the structure of the task. Here, we tested how rats represent an odor-guided choice task in which two odor cues indicated which of two responses would lead to reward, whereas a third odor indicated free choice among the two responses. A parsimonious task representation would allow animals to learn from the forced trials what is the better option to choose in the free-choice trials. However, animals may not necessarily generalize across odors in this way. We fit reinforcement-learning models that use different task representations to trial-by-trial choice behavior of individual rats performing this task, and quantified the degree to which each animal used the more parsimonious representation, generalizing across trial types. Model comparison revealed that most rats did not acquire this representation despite extensive experience. Our results demonstrate the importance of formally testing possible task representations that can afford the observed behavior, rather than assuming that animals' task representations abide by the generative task structure that governs the experimental design.

Prior Cocaine Self-Administration Increases Response-Outcome Encoding That Is Divorced from Actions Selected in Dorsal Lateral Striatum.

Dorsal lateral striatum (DLS) is a highly associative structure that encodes relationships among environmental stimuli, behavioral responses, and predicted outcomes. DLS is known to be disrupted after chronic drug abuse; however, it remains unclear what neural signals in DLS are altered. Current theory suggests that drug use enhances stimulus-response processing at the expense of response-outcome encoding, but this has mostly been tested in simple behavioral tasks. Here, we investigated what neural correlates in DLS are affected by previous cocaine exposure as rats performed a complex reward-guided decision-making task in which predicted reward value was independently manipulated by changing the delay to or size of reward associated with a response direction across a series of trial blocks. After cocaine self-administration, rats exhibited stronger biases toward higher-value reward and firing in DLS more strongly represented action-outcome contingencies independent from actions subsequently taken rather than outcomes predicted by selected actions (chosen-outcome contingencies) and associations between stimuli and actions (stimulus-response contingencies). These results suggest that cocaine self-administration strengthens action-outcome encoding in rats (as opposed to chosen-outcome or stimulus-response encoding), which abnormally biases behavior toward valued reward when there is a choice between two options during reward-guided decision-making.SIGNIFICANCE STATEMENT Current theories suggest that the impaired decision-making observed in individuals who chronically abuse drugs reflects a decrease in goal-directed behaviors and an increase in habitual behaviors governed by neural representations of response-outcome (R-O) and stimulus-response associations, respectively. We examined the impact that prior cocaine self-administration had on firing in dorsal lateral striatum (DLS), a brain area known to be involved in habit formation and affected by drugs of abuse, during performance of a complex reward-guided decision-making task. Surprisingly, we found that previous cocaine exposure enhanced R-O associations in DLS. This suggests that there may be more complex consequences of drug abuse than current theories have explored, especially when examining brain and behavior in the context of a complex two-choice decision-making task.

Altered basolateral amygdala encoding in an animal model of schizophrenia.

It has been proposed that schizophrenia results, in part, from the inappropriate or spurious attribution of salience to cues in the environment. We have recently reported neural correlates of salience in the basolateral amygdala (ABL) of rats during learning in an odor-guided discrimination task. Here we tested whether this dopamine-dependent salience signal is altered in rats with neonatal ventral hippocampal lesions (NVHLs), a rodent model of schizophrenia. We found that ABL signals related to violations in reward prediction were only mildly affected by NVHL; however, neurons in rats with NVHLs showed significantly stronger selectivity during odor sampling, particularly for the more salient large-reward cue. The elevated cue-evoked activity in NVHL rats was correlated with heightened orienting behavior and also with changes in firing to the shifts in reward, suggesting that it reflected abnormal signaling of the large reward-predicting cue's salience. These results are broadly consistent with the proposal that schizophrenics suffer from enhanced signaling of salience.

From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making.

The striatum is critical for reward-guided and habitual behavior. Anatomical and interference studies suggest a functional heterogeneity within striatum. Medial regions, such as nucleus accumbens core and dorsal medial striatum play roles in goal-directed behavior, while dorsal lateral striatum is critical for control of habitual action. Subdivisions of striatum are topographically connected with different cortical and subcortical structures forming channels that carry information related to limbic, associative, and sensorimotor functions. Here, we describe data showing that as one progresses from ventral-medial to dorsal-lateral striatum, there is a shift from more prominent value encoding to activity more closely related to associative and motor aspects of decision-making. In addition, we will describe data suggesting that striatal circuits work in parallel to control behavior and that regions within striatum can compensate for each other when functions are disrupted.

Increased firing to cues that predict low-value reward in the medial orbitofrontal cortex.

Anatomical, imaging, and lesion work have suggested that medial and lateral aspects of orbitofrontal cortex (OFC) play different roles in reward-guided decision-making, yet few single-neuron recording studies have examined activity in more medial parts of the OFC (mOFC) making it difficult to fully assess its involvement in motivated behavior. Previously, we have shown that neurons in lateral parts of the OFC (lOFC) selectively fire for rewards of different values. In that study, we trained rats to respond to different fluid wells for rewards of different sizes or delivered at different delays. Rats preferred large over small reward, and rewards delivered after short compared with long delays. Here, we recorded from single neurons in rat rostral mOFC as they performed the same task. Similar to the lOFC, activity was attenuated for rewards that were delivered after long delays and was enhanced for delivery of larger rewards. However, unlike lOFC, odor-responsive neurons in the mOFC were more active when cues predicted low-value outcomes. These data suggest that odor-responsive mOFC neurons signal the association between environmental cues and unfavorable outcomes during decision making.

Ventral striatum encodes past and predicted value independent of motor contingencies.

The ventral striatum (VS) is thought to signal the predicted value of expected outcomes. However, it is still unclear whether VS can encode value independently from variables often yoked to value such as response direction and latency. Expectations of high value reward are often associated with a particular action and faster latencies. To address this issue we trained rats to perform a task in which the size of the predicted reward was signaled before the instrumental response was instructed. Instrumental directional cues were presented briefly at a variable onset to reduce accuracy and increase reaction time. Rats were more accurate and slower when a large versus small reward was at stake. We found that activity in VS was high during odors that predicted large reward even though reaction times were slower under these conditions. In addition to these effects, we found that activity before the reward predicting cue reflected past and predicted reward. These results demonstrate that VS can encode value independent of motor contingencies and that the role of VS in goal-directed behavior is not just to increase vigor of specific actions when more is at stake.

Basolateral amygdala encodes upcoming errors but not response conflict.

Adaptive behavior depends on the detection of potential errors so that ongoing behavior might be corrected. Here, we ask whether basolateral amygdala (ABL) might serve this function by examining activity in rats performing a task in which errors were induced by pitting two behavioral responses against each other. This response competition or conflict was created by forcing rats to respond away from the direction in which they were freely choosing on the majority of trials. Rats were slower and less accurate on these incongruent trial types. We found that activity in ABL fired more strongly prior to errant responses, but did not signal the potential for errors on correctly performed incongruent trials. These data support a role for ABL in processing errors prior to their occurrence and suggest that ABL is not involved in monitoring conflict so that ongoing behavior might be corrected.

Prior cocaine self-administration impairs attention signals in anterior cingulate cortex.

Although maladaptive decision-making is a defining feature of drug abuse and addiction, we have yet to ascertain how cocaine self-administration disrupts neural signals in anterior cingulate cortex (ACC), a brain region thought to contribute to attentional control. To address this issue, rats were trained on a reward-guided decision-making task; reward value was manipulated by independently varying the size of or the delay to reward over several trial blocks. Subsequently, rats self-administered either a cocaine (experimental group) or sucrose (control) during 12 consecutive days, after which they underwent a 1-month withdrawal period. Upon completion of this period, rats performed the previously learned reward-guided decision-making task while we recorded from single neurons in ACC. We demonstrate that prior cocaine self-administration attenuates attention and attention-related ACC signals in an intake-dependent manner, and that changes in attention are decoupled from ACC firing. These effects likely contribute to the impaired decision-making-typified by chronic substance abuse and relapse-observed after drug use.

Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum.

The nucleus accumbens core (NAc) is important for integrating and providing information to downstream areas about the timing and value of anticipated reward. Although NAc is one of the first brain regions to be affected by drugs of abuse, we still do not know how neural correlates related to reward expectancy are affected by previous cocaine self-administration. To address this issue, we recorded from single neurons in the NAc of rats that had previously self-administered cocaine or sucrose (control). Neural recordings were then taken while rats performed an odor-guided decision-making task in which we independently manipulated value of expected reward by changing the delay to or size of reward across a series of trial blocks. We found that previous cocaine self-administration made rats more impulsive, biasing choice behavior toward more immediate reward. Further, compared to controls, cocaine-exposed rats showed significantly fewer neurons in the NAc that were responsive during odor cues and reward delivery, and in the reward-responsive neurons that remained, diminished directional and value encoding was observed. Lastly, we found that after cocaine exposure, reward-related firing during longer delays was reduced compared to controls. These results demonstrate that prior cocaine self-administration alters reward-expectancy encoding in NAc, which could contribute to poor decision making observed after chronic cocaine use.

Prenatal Nicotine Exposure Impairs Executive Control Signals in Medial Prefrontal Cortex.

Prenatal nicotine exposure (PNE) is linked to numerous psychiatric disorders including attention deficit hyperactivity disorder (ADHD). Current literature suggests that core deficits observed in ADHD reflect abnormal inhibitory control governed by the prefrontal cortex. Yet, it is unclear how neural activity in the medial prefrontal cortex (mPFC) is modulated during tasks that assess response inhibition or if these neural correlates, along with behavior, are affected by PNE. To address this issue, we recorded from single mPFC neurons in control and PNE rats as they performed a stop-signal task. We found that PNE rats were faster for all trial-types, made more premature responses, and were less likely to inhibit behavior on 'STOP' trials during which rats had to inhibit an already initiated response. Activity in mPFC was modulated by response direction and was positively correlated with accuracy and movement time in control but not PNE rats. Although the number of single neurons correlated with response direction was significantly reduced by PNE, neural activity observed on general STOP trials was largely unaffected. However, dramatic behavioral deficits on STOP trials immediately following non-conflicting (GO) trials in the PNE group appear to be mediated by the loss of conflict monitoring signals in mPFC. We conclude that prenatal nicotine exposure makes rats impulsive and disrupts firing of mPFC neurons that carry signals related to response direction and conflict monitoring.

Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum.

BACKGROUND: The development of addiction is thought to reflect a transition from goal-directed to stimulus-response driven behavior, functions attributed to ventral (VS) and dorsal striatum (DS), respectively. In line with this theory, neuroadaptations that occur during prolonged drug use progress from VS to DS. Here we ask if VS dysfunction alone, independent of drug use, can affect neural selectivity in DS. METHODS: To address this issue, we recorded from single neurons in DS while rats performed an odor-guided choice task for differently valued rewards in rats with and without unilateral VS lesions. In a separate group of animals, we used bilateral VS lesions to determine if VS was critical for performance on this task. RESULTS: We describe data showing that unilateral lesions of VS enhance neural representations in DS during performance of a task that is dependent on VS. Furthermore, we show that VS is critical for reward-guided decision-making initially, but that rats regain function after several days. CONCLUSIONS: These results suggest that loss of VS function, independent of chronic drug use, can trigger stronger encoding in DS in a reward-guided decision-making task and that the transition from VS to DS governed behavior observed in addiction might be due, in part, to initial loss of VS function.

Separate populations of neurons in ventral striatum encode value and motivation.

Neurons in the ventral striatum (VS) fire to cues that predict differently valued rewards. It is unclear whether this activity represents the value associated with the expected reward or the level of motivation induced by reward anticipation. To distinguish between the two, we trained rats on a task in which we varied value independently from motivation by manipulating the size of the reward expected on correct trials and the threat of punishment expected upon errors. We found that separate populations of neurons in VS encode expected value and motivation.

Response inhibition signals and miscoding of direction in dorsomedial striatum.

The ability to inhibit action is critical for everyday behavior and is affected by a variety of disorders. Behavioral control and response inhibition is thought to depend on a neural circuit that includes the dorsal striatum, yet the neural signals that lead to response inhibition and its failure are unclear. To address this issue, we recorded from neurons in rat dorsomedial striatum (mDS) in a novel task in which rats responded to a spatial cue that signaled that reward would be delivered either to the left or to the right. On 80% of trials rats were instructed to respond in the direction cued by the light (GO). On 20% of trials a second light illuminated instructing the rat to refrain from making the cued movement and move in the opposite direction (STOP). Many neurons in mDS encoded direction, firing more or less strongly for GO movements made ipsilateral or contralateral to the recording electrode. Neurons that fired more strongly for contralateral GO responses were more active when rats were faster, showed reduced activity on STOP trials, and miscoded direction on errors, suggesting that when these neurons were overly active, response inhibition failed. Neurons that decreased firing for contralateral movement were excited during trials in which the rat was required to stop the ipsilateral movement. For these neurons activity was reduced when errors were made and was negatively correlated with movement time suggesting that when these neurons were less active on STOP trials, response inhibition failed. Finally, the activity of a significant number of neurons represented a global inhibitory signal, firing more strongly during response inhibition regardless of response direction. Breakdown by cell type suggests that putative medium spiny neurons (MSNs) tended to fire more strongly under STOP trials, whereas putative interneurons exhibited both activity patterns.