Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release.

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum.

The nucleus accumbens core (NAc) is important for integrating and providing information to downstream areas about the timing and value of anticipated reward. Although NAc is one of the first brain regions to be affected by drugs of abuse, we still do not know how neural correlates related to reward expectancy are affected by previous cocaine self-administration. To address this issue, we recorded from single neurons in the NAc of rats that had previously self-administered cocaine or sucrose (control). Neural recordings were then taken while rats performed an odor-guided decision-making task in which we independently manipulated value of expected reward by changing the delay to or size of reward across a series of trial blocks. We found that previous cocaine self-administration made rats more impulsive, biasing choice behavior toward more immediate reward. Further, compared to controls, cocaine-exposed rats showed significantly fewer neurons in the NAc that were responsive during odor cues and reward delivery, and in the reward-responsive neurons that remained, diminished directional and value encoding was observed. Lastly, we found that after cocaine exposure, reward-related firing during longer delays was reduced compared to controls. These results demonstrate that prior cocaine self-administration alters reward-expectancy encoding in NAc, which could contribute to poor decision making observed after chronic cocaine use.

Prior Cocaine Self-Administration Increases Response-Outcome Encoding That Is Divorced from Actions Selected in Dorsal Lateral Striatum.

Dorsal lateral striatum (DLS) is a highly associative structure that encodes relationships among environmental stimuli, behavioral responses, and predicted outcomes. DLS is known to be disrupted after chronic drug abuse; however, it remains unclear what neural signals in DLS are altered. Current theory suggests that drug use enhances stimulus-response processing at the expense of response-outcome encoding, but this has mostly been tested in simple behavioral tasks. Here, we investigated what neural correlates in DLS are affected by previous cocaine exposure as rats performed a complex reward-guided decision-making task in which predicted reward value was independently manipulated by changing the delay to or size of reward associated with a response direction across a series of trial blocks. After cocaine self-administration, rats exhibited stronger biases toward higher-value reward and firing in DLS more strongly represented action-outcome contingencies independent from actions subsequently taken rather than outcomes predicted by selected actions (chosen-outcome contingencies) and associations between stimuli and actions (stimulus-response contingencies). These results suggest that cocaine self-administration strengthens action-outcome encoding in rats (as opposed to chosen-outcome or stimulus-response encoding), which abnormally biases behavior toward valued reward when there is a choice between two options during reward-guided decision-making.SIGNIFICANCE STATEMENT Current theories suggest that the impaired decision-making observed in individuals who chronically abuse drugs reflects a decrease in goal-directed behaviors and an increase in habitual behaviors governed by neural representations of response-outcome (R-O) and stimulus-response associations, respectively. We examined the impact that prior cocaine self-administration had on firing in dorsal lateral striatum (DLS), a brain area known to be involved in habit formation and affected by drugs of abuse, during performance of a complex reward-guided decision-making task. Surprisingly, we found that previous cocaine exposure enhanced R-O associations in DLS. This suggests that there may be more complex consequences of drug abuse than current theories have explored, especially when examining brain and behavior in the context of a complex two-choice decision-making task.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action.

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Neural correlates of rules and conflict in medial prefrontal cortex during decision and feedback epochs.

The ability to properly adjust behavioral responses to cues in a changing environment is crucial for survival. Activity in the medial Prefrontal Cortex (mPFC) is thought to both represent rules to guide behavior as well as detect and resolve conflicts between rules in changing contingencies. However, while lesion and pharmacological studies have supported a crucial role for mPFC in this type of set-shifting, an understanding of how mPFC represents current rules or detects and resolves conflict between different rules is unclear. Here, we directly address the role of rat mPFC in shifting rule based behavioral strategies using a novel behavioral task designed to tease apart neural signatures of rules, conflict and direction. We demonstrate that activity of single neurons in rat mPFC represent distinct rules. Further, we show increased firing on high conflict trials in a separate population of mPFC neurons. Reduced firing in both populations of neurons was associated with poor performance. Moreover, activity in both populations increased and decreased firing during the outcome epoch when reward was and was not delivered on correct and incorrect trials, respectively. In addition, outcome firing was modulated by the current rule and the degree of conflict associated with the previous decision. These results promote a greater understanding of the role that mPFC plays in switching between rules, signaling both rule and conflict to promote improved behavioral performance.

Rule encoding in dorsal striatum impacts action selection.

Cognitive flexibility is a hallmark of prefrontal cortical (PFC) function yet little is known about downstream area involvement. The medial dorsal striatum (mDS) receives major projections from the PFC and is uniquely situated to perform the integration of responses with rule information. In this study, we use a novel rule shifting task in rats that mirrors non-human primate and human studies in its temporal precision and counterbalanced responses. We record activity from single neurons in the mDS while rats switch between different rules for reward. Additionally, we pharmacologically inactivate mDS by infusion of a baclofen/muscimol cocktail. Inactivation of mDS impaired the ability to shift to a new rule and increased the number of regressive errors. While recording in mDS, we identified neurons modulated by direction whose activity reflected the conflict between competing rule information. We show that a subset of these neurons was also rule selective, and that the conflict between competing rule cues was resolved as behavioural performance improved. Other neurons were modulated by rule, but not direction. These neurons became selective before behavioural performance accurately reflected the current rule. These data provide an additional locus for investigating the mechanisms underlying behavioural flexibility. Converging lines of evidence from multiple human psychiatric disorders have implicated dorsal striatum as an important and understudied neural substrate of flexible cognition. Our data confirm the importance of mDS, and suggest a mechanism by which mDS mediates abstract cognition functions.

Interneurons are necessary for coordinated activity during reversal learning in orbitofrontal cortex.

BACKGROUND: Cerebral cortical gamma-aminobutyric acidergic interneuron dysfunction is hypothesized to lead to cognitive deficits comorbid with human neuropsychiatric disorders, including schizophrenia, autism, and epilepsy. We have previously shown that mice that harbor mutations in the Plaur gene, which is associated with schizophrenia, have deficits in frontal cortical parvalbumin-expressing interneurons. Plaur mice have impaired reversal learning, similar to deficits observed in patients with schizophrenia. METHODS: We examined the role of parvalbumin interneurons in orbitofrontal cortex during reversal learning by recording single unit activity from 180 control and 224 Plaur mouse neurons during a serial reversal task. Neural activity was analyzed during correct and incorrect decision choices and reward receipt. RESULTS: Neurons in control mice exhibited strong phasic responses both during discrimination and reversal learning to decisions and rewards, and the strength of the response was correlated with behavioral performance. Although baseline firing was significantly enhanced in Plaur mice, neural selectivity for correct or erroneous decisions was diminished and not correlated with behavior, and reward encoding was downscaled. In addition, Plaur mice showed a significant reduction in the number of neurons that encoded expected outcomes across task phases during the decision period. CONCLUSIONS: These data indicate that parvalbumin interneurons are necessary for the representation of outcomes in orbitofrontal cortex. Deficits in inhibition blunt selective neural firing during key decisions, contributing to behavioral inflexibility. These data provide a potential explanation for disorders of cognitive control that accompany the loss of these gamma-aminobutyric acidergic interneurons in human neuropsychiatric disorders, such as autism, epilepsy, and schizophrenia.

Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures.

Decision-making is motivated by the possibility of obtaining reward and/or avoiding punishment. Though many have investigated behavior associated with appetitive or aversive outcomes, few have examined behaviors that rely on both. Fewer still have addressed questions related to how anticipated appetitive and aversive outcomes interact to alter neural signals related to expected value, motivation, and salience. Here we review recent rodent, monkey, and human research that address these issues. Further development of this area will be fundamental to understanding the etiology behind human psychiatric diseases and cultivating more effective treatments.

Prefrontal cognitive deficits in mice with altered cerebral cortical GABAergic interneurons.

Alterations of inhibitory GABAergic neurons are implicated in multiple psychiatric and neurological disorders, including schizophrenia, autism and epilepsy. In particular, interneuron deficits in prefrontal areas, along with presumed decreased inhibition, have been reported in several human patients. The majority of forebrain GABAergic interneurons arise from a single subcortical source before migrating to their final regional destination. Factors that govern the interneuron populations have been identified, demonstrating that a single gene mutation may globally affect forebrain structures or a single area. In particular, mice lacking the urokinase plasminogen activator receptor (Plaur) gene have decreased GABAergic interneurons in frontal and parietal, but not caudal, cortical regions. Plaur assists in the activation of hepatocyte growth factor/scatter factor (HGF/SF), and several of the interneuron deficits are correlated with decreased levels of HGF/SF. In some cortical regions, the interneuron deficit can be remediated by endogenous overexpression of HGF/SF. In this study, we demonstrate decreased parvalbumin-expressing interneurons in the medial frontal cortex, but not in the hippocampus or basal lateral amygdala in the Plaur null mouse. The Plaur null mouse demonstrates impaired medial frontal cortical function in extinction of cued fear conditioning and the inability to form attentional sets. Endogenous HGF/SF overexpression increased the number of PV-expressing cells in medial frontal cortical areas to levels greater than found in wildtype mice, but did not remediate the behavioral deficits. These data suggest that proper medial frontal cortical function is dependent upon optimum levels of inhibition and that a deficit or excess of interneuron numbers impairs normal cognition.

Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum.

BACKGROUND: The development of addiction is thought to reflect a transition from goal-directed to stimulus-response driven behavior, functions attributed to ventral (VS) and dorsal striatum (DS), respectively. In line with this theory, neuroadaptations that occur during prolonged drug use progress from VS to DS. Here we ask if VS dysfunction alone, independent of drug use, can affect neural selectivity in DS. METHODS: To address this issue, we recorded from single neurons in DS while rats performed an odor-guided choice task for differently valued rewards in rats with and without unilateral VS lesions. In a separate group of animals, we used bilateral VS lesions to determine if VS was critical for performance on this task. RESULTS: We describe data showing that unilateral lesions of VS enhance neural representations in DS during performance of a task that is dependent on VS. Furthermore, we show that VS is critical for reward-guided decision-making initially, but that rats regain function after several days. CONCLUSIONS: These results suggest that loss of VS function, independent of chronic drug use, can trigger stronger encoding in DS in a reward-guided decision-making task and that the transition from VS to DS governed behavior observed in addiction might be due, in part, to initial loss of VS function.

Separate populations of neurons in ventral striatum encode value and motivation.

Neurons in the ventral striatum (VS) fire to cues that predict differently valued rewards. It is unclear whether this activity represents the value associated with the expected reward or the level of motivation induced by reward anticipation. To distinguish between the two, we trained rats on a task in which we varied value independently from motivation by manipulating the size of the reward expected on correct trials and the threat of punishment expected upon errors. We found that separate populations of neurons in VS encode expected value and motivation.

Neural structures underlying set-shifting: roles of medial prefrontal cortex and anterior cingulate cortex.

Impaired attentional set-shifting and inflexible decision-making are problems frequently observed during normal aging and in several psychiatric disorders. To understand the neuropathophysiology of underlying inflexible behavior, animal models of attentional set-shifting have been developed to mimic tasks such as the Wisconsin Card Sorting Task (WCST), which tap into a number of cognitive functions including stimulus-response encoding, working memory, attention, error detection, and conflict resolution. Here, we review many of these tasks in several different species and speculate on how prefrontal cortex and anterior cingulate cortex might contribute to normal performance during set-shifting.

Attention-related Pearce-Kaye-Hall signals in basolateral amygdala require the midbrain dopaminergic system.

BACKGROUND: Neural activity in basolateral amygdala has recently been shown to reflect surprise or attention as predicted by the Pearce-Kaye-Hall model (PKH)--an influential model of associative learning. Theoretically, a PKH attentional signal originates in prediction errors of the kind associated with phasic firing of dopamine neurons. This requirement for prediction errors, coupled with projections from the midbrain dopamine system into basolateral amygdala, suggests that the PKH signal in amygdala may depend on dopaminergic input. METHODS: To test this, we recorded single unit activity in basolateral amygdala in rats with 6-hydroxydopamine or sham lesions of the ipsilateral midbrain region. Neurons were recorded as the rats performed a task previously used to demonstrate both dopaminergic reward prediction errors and attentional signals in basolateral amygdala neurons. RESULTS: We found that neurons recorded in sham lesioned rats exhibited the same attention-related PKH signal observed in previous studies. By contrast, neurons recorded in rats with ipsilateral 6-hydroxydopamine lesions failed to show attentional signaling. CONCLUSIONS: These results indicate a linkage between the neural instantiations of the basolateral complex of the amygdala attentional signal and dopaminergic prediction errors. Such a linkage would have important implications for understanding both normal and aberrant learning and behavior, particularly in diseases thought to have a primary effect on dopamine systems, such as addiction and schizophrenia.

Versatility of the mouse reversal/set-shifting test: effects of topiramate and sex.

The ability to learn a rule to guide behavior is crucial for cognition and executive function. However, in a constantly changing environment, flexibility in terms of learning and changing rules is paramount. Research suggests there may be common underlying causes for the similar rule learning impairments observed in many psychiatric disorders. One of these common anatomical manifestations involves deficits to the GABAergic system, particularly in the frontal cerebral cortical regions. Many common anti-epileptic drugs and mood stabilizers activate the GABA system with the reported adverse side effects of cognitive dysfunction. The mouse reversal/set-shifting test was used to evaluate effects in mice given topiramate, which is reported to impair attention in humans. Here we report that in mice topiramate prevents formation of the attentional set, but does not alter reversal learning. Differences in the GABA system are also found in many neuropsychiatric disorders that are more common in males, including schizophrenia and autism. Initial findings with the reversal/set-shifting task excluded female subjects. In this study, female mice tested on the standard reversal/set-shifting task showed similar reversal learning, but were not able to form the attentional set. The behavioral paradigm was modified and when presented with sufficient discrimination tasks, female mice performed the same as male mice, requiring the same number of trials to reach criterion and form the attentional set. The notable difference was that female mice had an extended latency to complete the trials for all discriminations. In summary, the reversal/set-shifting test can be used to screen for cognitive effects of potential therapeutic compounds in both male and female mice.

Reversal learning and attentional set-shifting in mice.

Schizophrenia is a complex developmental disorder that presents challenges to modern neuroscience in terms of discovering etiology and aiding in effective treatment of afflicted humans. One approach is to divide the constellation of symptoms of human neuropsychiatric disorders into discrete units for study. Multiple animal models are used to study brain ontogeny, response to psychoactive compounds, substrates of defined behaviors. Frontal cortical areas have been found to have abnormal anatomy and neurotransmitter levels in postmortem brains from schizophrenic patients. The mouse model has the advantage of rather straightforward genetic manipulation and offers numerous genetic variations within the same species. However, until recently, the behavioral analyses in the mice lagged behind the primate and rat, especially with respect to testing of frontal cortical regions. Current reports of mouse prefrontal anatomy and function advocate the mouse as a feasible animal model to study prefrontal cortical function. This review highlights the most recent developments from behavioral paradigms for testing orbital and medial prefrontal cortical function in pharmacological and genetic models of human schizophrenia.

Astrocyte-mediated hepatocyte growth factor/scatter factor supplementation restores GABAergic interneurons and corrects reversal learning deficits in mice.

Many psychiatric and neurological disorders present persistent neuroanatomical abnormalities in multiple brain regions that may reflect a common origin for a developmental disturbance. In mammals, many of the local GABAergic inhibitory interneurons arise from a single subcortical source. Perturbations in the ontogeny of the GABAergic interneurons may be reflected in the adult by interneuron deficits in both frontal cerebral cortical and striatal regions. Disrupted GABAergic circuitry has been reported in patients with schizophrenia and frontal lobe epilepsy and may contribute to their associated impairments in behavioral flexibility. The present study demonstrates that one type of behavioral flexibility, reversal learning, is dependent upon proper numbers of GABAergic interneurons. Mice with abnormal interneuron ontogeny have reduced numbers of parvalbumin-expressing GABAergic local interneurons in the orbitofrontal cortical and striatal regions and impaired reversal leaning. Using a genetic approach, both the anatomical and functional deficiencies are restored with exogenous postnatal growth factor supplementation. These results show that GABAergic local circuitry is critical for modulating behavioral flexibility and that birth defects can be corrected by replenishing crucial growth factors.

Hepatocyte growth factor (HGF) modulates GABAergic inhibition and seizure susceptibility.

Disrupted ontogeny of forebrain inhibitory interneurons leads to neurological disorders, including epilepsy. Adult mice lacking the urokinase plasminogen activator receptor (Plaur) have decreased numbers of neocortical GABAergic interneurons and spontaneous seizures, attributed to a reduction of hepatocyte growth factor/scatter factor (HGF/SF). We report that by increasing endogenous HGF/SF concentration in the postnatal Plaur null mouse brain maintains the interneuron populations in the adult, reverses the seizure behavior and stabilizes the spontaneous electroencephalogram activity. The perinatal intervention provides a pathway to reverse potential birth defects and ameliorate seizures in the adult.

Double dissociation of the effects of medial and orbital prefrontal cortical lesions on attentional and affective shifts in mice.

Many neuropsychiatric diseases are associated with cognitive rigidity linked to prefrontal dysfunction. For example, schizophrenia and Parkinson's disease are associated with performance deficits on the Wisconsin Card Sorting Test, which evaluates attentional set shifting. Although the genetic underpinnings of these disorders can be reproduced in mice, there are few models for testing the functional consequences. Here, we demonstrate that an analog of the Wisconsin Card Sorting Test, developed in marmosets and recently adapted to rats, is a behavioral model of prefrontal function in mice. Systematic analysis demonstrated that formation of the attentional set in mice is dependent on the number of problem sets. We found that mice, like rats and primates, exhibit both affective and attentional sets, and these functions are disrupted by neurotoxic damage to orbitofrontal and medial prefrontal cortical areas, respectively. These data are identical to studies in rats and similar to the deficits reported after prefrontal damage in a comparable task in marmosets. These results provide a behavioral model to assess prefrontal function in mice.