Time-scale invariant contingency yields one-shot reinforcement learning despite extremely long delays to reinforcement.

Reinforcement learning inspires much theorizing in neuroscience, cognitive science, machine learning, and AI. A central question concerns the conditions that produce the perception of a contingency between an action and reinforcement-the assignment-of-credit problem. Contemporary models of associative and reinforcement learning do not leverage the temporal metrics (measured intervals). Our information-theoretic approach formalizes contingency by time-scale invariant temporal mutual information. It predicts that learning may proceed rapidly even with extremely long action-reinforcer delays. We show that rats can learn an action after a single reinforcement, even with a 16-min delay between the action and reinforcement (15-fold longer than any delay previously shown to support such learning). By leveraging metric temporal information, our solution obviates the need for windows of associability, exponentially decaying eligibility traces, microstimuli, or distributions over Bayesian belief states. Its three equations have no free parameters; they predict one-shot learning without iterative simulation.

Pavlovian-to-instrumental transfer in individuals with chronic pain.

Avoidance of pain has been argued to be key factor leading pain events to chronic disability. In this respect, research has focused on investigating the working mechanisms of avoidance's acquisition. Avoidance of painful stimuli has been traditionally studied using a combination of Pavlovian and Instrumental procedures. However, such approach seems to go against real-life scenarios where avoidance is commonly acquired more readily. Using a novel pain avoidance paradigm, we tested whether pain avoidance can be installed in absence of associations between a cue and pain omission, and whether such avoidance differs between pain patients and healthy controls. Participants first learned to avoid painful stimuli by pressing a grip bar. Then, they passively encountered pairings of one geometrical shape with pain and of another geometrical shape without pain. Lastly, participants encountered the geometrical shapes while being able to use the grip bar. Results showed that participants pressed the bar more vigorously when encountering the previously pain-related shape compared to the pain-unrelated shape. This effect did not seem to differ between pain patients and healthy control. Our study could inspire a new way in measuring avoidance in pain, possibly paving the way to better understanding how avoidance is installed in chronic pain.

Characterizing the dynamic learning process: Implications of a quantitative analysis.

Understanding the neural mechanisms involved in learning processes is crucial for unraveling the complexities of behavior and cognition. Sudden change from the untrained level to the fully-learned level is a pivotal feature of instrumental learning. However, the concept of change point and suitable methods to conveniently analyze the characteristics of sudden change in groups remain elusive, which might hinder a fuller understanding of the neural mechanism underlying dynamic leaning process. In the current study, we investigated the learning processes of mice that were trained in an aversive instrumental learning task, and introduced a novel strategy to analyze behavioral variations in instrumental learning, leading to improved clarity on the concept of sudden change and enabling comprehensive group analysis. By applying this novel strategy, we examined the effects of cocaine and a cannabinoid receptor agonist on instrumental learning. Intriguingly, our analysis revealed significant differences in timing and occurrence of sudden changes that were previously overlooked using traditional analysis. Overall, our research advances understanding of behavioral variation during instrumental learning and the interplay between learning behaviors and neurotransmitter systems, contributing to a deeper comprehension of learning processes and informing future investigations and therapeutic interventions.

Measuring context-response associations that drive habits.

People achieve important life outcomes of health, financial security, and productivity by repeating operant behavior. To identify whether such operants reflect goal pursuit or habit, the present research introduces a new paradigm that yields objective measures of learning and controls for the motivations of goal pursuit. In two experiments, participants practiced a sequential task of making sushi and then completed a test of the strength of cue-response (habit) associations in memory. Finally, they repeated the sushi task without instructions while under cognitive load (designed to impede deliberation about goals). As predicted, greater task practice yielded stronger cue-response associations, which in turn promoted task success. Practice did not improve performance by enhancing goal intentions or other task motivations. We conclude that repetition facilitates performance by creating mental associations that automatically activate practiced, habitual responses upon perception of recurring context cues.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance.

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Learning about me and you: Only deterministic stimulus associations elicit self-prioritization.

Self-relevant material has been shown to be prioritized over stimuli relating to others (e.g., friend, stranger), generating benefits in attention, memory, and decision-making. What is not yet understood, however, is whether the conditions under which self-related knowledge is acquired impacts the emergence of self-bias. To address this matter, here we used an associative-learning paradigm in combination with a stimulus-classification task to explore the effects of different learning experiences (i.e., deterministic vs. probabilistic) on self-prioritization. The results revealed an effect of prior learning on task performance, with self-prioritization only emerging when participants acquired target-related associations (i.e., self vs. friend) under conditions of certainty (vs. uncertainty). A further computational (i.e., drift diffusion model) analysis indicated that differences in the efficiency of stimulus processing (i.e., rate of information uptake) underpinned this self-prioritization effect. The implications of these findings for accounts of self-function are considered.

Effects of a single dose of amisulpride on functional brain changes during reward- and motivation-related processing using task-based fMRI in healthy subjects and patients with major depressive disorder - study protocol for a randomized clinical trial.

BACKGROUND: Anhedonia and other deficits in reward- and motivation-related processing in psychiatric patients, including patients with major depressive disorder (MDD), represent a high unmet medical need. Neurobiologically, these deficits in MDD patients are mainly associated with low dopamine function in a frontostriatal network. In this study, alterations in brain activation changes during reward processing and at rest in MDD patients compared with healthy subjects are explored and the effects of a single low dose of the dopamine D2 receptor antagonist amisulpride are investigated. METHODS: This is a randomized, controlled, double-blind, single-dose, single-center parallel-group clinical trial to assess the effects of a single dose of amisulpride (100 mg) on blood-oxygenation-level-dependent (BOLD) responses during reward- and motivation-related processing in healthy subjects (n = 60) and MDD patients (n = 60). Using functional magnetic resonance imaging (fMRI), BOLD responses are assessed during the monetary incentive delay (MID) task (primary outcome). Exploratory outcomes include BOLD responses and behavioral measures during the MID task, instrumental learning task, effort-based decision-making task, social incentive delay task, and probabilistic reward task as well as changes in resting state functional connectivity and cerebral blood flow. DISCUSSION: This study broadly covers all aspects of reward- and motivation-related processing as categorized by the National Institute of Mental Health Research Domain Criteria and is thereby an important step towards precision psychiatry. Results regarding the immediate effects of a dopaminergic drug on deficits in reward- and motivation-related processing not only have the potential to significantly broaden our understanding of underlying neurobiological processes but might eventually also pave the way for new treatment options. TRIAL REGISTRATION: ClinicalTrials.gov NCT05347199. April 12, 2022.

Immp2l knockdown in male mice increases stimulus-driven instrumental behaviour but does not alter goal-directed learning or neuron density in cortico-striatal circuits in a model of Tourette syndrome and autism spectrum disorder.

Cortico-striatal neurocircuits mediate goal-directed and habitual actions which are necessary for adaptive behaviour. It has recently been proposed that some of the core symptoms of autism spectrum disorder (ASD) and Gilles de la Tourette syndrome (GTS), such as tics and other repetitive behaviours, may emerge because of imbalances in these neurocircuits. We have recently developed a model of ASD and GTS by knocking down Immp2l, a mitochondrial gene frequently associated with these disorders. The current study sought to determine whether Immp2l knockdown (KD) in male mice alters flexible, goal- or cue- driven behaviour using procedures specifically designed to examine response-outcome and stimulus-response associations, which underlie goal-directed and habitual behaviour, respectively. Whether Immp2l KD alters neuron density in cortico-striatal neurocircuits known to regulate these behaviours was also examined. Immp2l KD mice and wild type-like mice (WT) were trained on Pavlovian and instrumental learning procedures where auditory cues predicted food delivery and lever-press responses earned a food outcome. It was demonstrated that goal-directed learning was not changed for Immp2l KD mice compared to WT mice, as lever-press responses were sensitive to changes in the value of the food outcome, and to contingency reversal and degradation. There was also no difference in the capacity of KD mice to form habitual behaviours compared to WT mice following extending training of the instrumental action. However, Immp2l KD mice were more responsive to auditory stimuli paired with food as indicated by a non-specific increase in lever response rates during Pavlovian-to-instrumental transfer. Finally, there were no alterations to neuron density in striatum or any prefrontal cortex or limbic brain structures examined. Thus, the current study suggests that Immp2l is not necessary for learned maladaptive goal or stimulus driven behaviours in ASD or GTS, but that it may contribute to increased capacity for external stimuli to drive behaviour. Alterations to stimulus-driven behaviour could potentially influence the expression of tics and repetitive behaviours, suggesting that genetic alterations to Immp2l may contribute to these core symptoms in ASD and GTS. Given that this is the first application of this battery of instrumental learning procedures to a mouse model of ASD or GTS, it is an important initial step in determining the contribution of known risk-genes to goal-directed versus habitual behaviours, which should be more broadly applied to other rodent models of ASD and GTS in the future.

Response-specific effects of punishment of a discriminated operant response.

To determine whether the punishment of a discriminated operant behavior has effects that are specific to the punished response, rats were reinforced for performing two different instrumental responses (lever pressing and chain pulling) in the presence of a single discriminative stimulus (S). They were then either punished with mild footshock for performing one of the responses (R1) in S, or they received the same shocks in a noncontingent manner while performing R1 in S (i.e., a yoked control). In final tests of both R1 and R2 in S, the punished rats were more suppressed to R1 than R2, but the yoked rats were not. The results extend previous results with extinction rather than punishment learning (Bouton, Trask, & Carranza-Jasso, 2016) and support a larger parallel between extinction and punishment of both free-operant and discriminated-operant responding. Punishment is like extinction in creating a response-specific inhibition of either free or discriminated operant behavior.

One-trial appetitive learning tasks for drug targeting.

One-trial appetitive learning developed from one-trial passive avoidance learning as a standard test of retrograde amnesia. It consists of one learning trial followed by a retention test, in which physiological manipulations are presented. As in passive avoidance learning, food- or water-deprived rats or mice finding food or water inside an enclosure are vulnerable to the retrograde amnesia produced by electroconvulsive shock treatment or the injection of various drugs. In one-trial taste or odor learning conducted in rats, birds, snails, bees, and fruit flies, there is an association between a food item or odorant and contextual stimuli or the unconditioned stimulus of Pavlovian conditioning. The odor-related task in bees was sensitive to protein synthesis inhibition as well as cholinergic receptor blockade, both analogous to results found on the passive avoidance response in rodents, while the task in fruit flies was sensitive to genetic modifications and aging, as seen in the passive avoidance response of genetically modified and aged rodents. These results provide converging evidence of interspecies similarities underlying the neurochemical basis of learning.

Habits, Goals, and Behavioral Signs of Cognitive Perseveration on Wisconsin Card-Sorting Tasks.

Wisconsin card-sorting tasks provide unique opportunities to study cognitive flexibility and its limitations, which express themselves behaviorally as perseverative errors (PE). PE refer to those behavioral errors on Wisconsin card-sorting tasks that are committed when cognitive rules are maintained even though recently received outcomes demand to switch to other rules (i.e., cognitive perseveration). We explored error-suppression effects (ESE) across three Wisconsin card-sorting studies. ESE refer to the phenomenon that PE are reduced on repetitive trials compared to non-repetitive trials. We replicated ESE in all three Wisconsin card-sorting studies. Study 1 revealed that non-associative accounts of ESE, in particular the idea that cognitive inhibition may account for them, are not tenable. Study 2 suggested that models of instrumental learning are among the most promising associative accounts of ESE. Instrumental learning comprises goal-directed control and the formation of corresponding associative memories over and above the formation of habitual memories according to dual-process models of instrumental learning. Study 3 showed that cognitive, rather than motor, representations of responses should be conceptualized as elements entering goal-directed instrumental memories. Collectively, the results imply that ESE on Wisconsin card-sorting tasks are not only a highly replicable phenomenon, but they also indicate that ESE provide an opportunity to study cognitive mechanisms of goal-directed instrumental control. Based on the reported data, we present a novel theory of cognitive perseveration (i.e., the 'goal-directed instrumental control' GIC model), which is outlined in the Concluding Discussion.

Restoring the youthful state of striatal plasticity in aged mice re-enables cognitive control of action.

Multidisciplinary evidence suggests that the control of voluntary action arbitrates between two major forms of behavioral processing: cognitively guided (or goal directed) and autonomously guided (or habitual). Brain-state irregularities affecting the striatum-such as aging-commonly shift control toward the latter, although the responsible neural mechanisms remain unknown. Combining instrumental conditioning with cell-specific mapping and chemogenetics in striatal neurons, we explored strategies that invigorate goal-directed capacity in aged mice. We found that, under conditions favoring goal-directed control, aged animals resiliently expressed autonomously guided behavior, a response that was underpinned by a characteristic one-to-one functional engagement of the two main neuronal populations in the striatum-D1- and D2-dopamine receptor-expressing spiny projection neurons (SPNs). Chemogenetically induced desensitization of D2-SPN signaling in aged transgenic mice recapitulated the striatal plasticity state observed in young mice, an effect that shifted behavior toward vigorous, goal-directed action. Our findings contribute to the understanding of the neural bases of behavioral control and propose neural system interventions that enhance cognitive functioning in habit-prone brains.

Instrumental learning and behavioral persistence in children with attention-deficit/hyperactivity-disorder: does reinforcement frequency matter?

BACKGROUND: Prominent theoretical accounts of attention-deficit/hyperactivity-disorder (ADHD) hypothesize that reinforcement learning deficits underlie symptoms of ADHD. The Dynamic Developmental Theory and the Dopamine Transfer Deficit hypothesis assume impairments in both the acquisition and extinction of behavior, especially when learning occurs under partial (non-continuous) reinforcement, and subsequently the Partial Reinforcement Extinction Effect (PREE). Few studies have evaluated instrumental learning in ADHD and the results are inconsistent. The current study investigates instrumental learning under partial and continuous reinforcement schedules and subsequent behavioral persistence when reinforcement is withheld (extinction) in children with and without ADHD. METHODS: Large well-defined samples of children with ADHD (n = 93) and typically developing (TD) children (n = 73) completed a simple instrumental learning task. The children completed acquisition under continuous (100%) or partial (20%) reinforcement, followed by a 4-min extinction phase. Two-way (diagnosis by condition) ANOVAs evaluated responses needed to reach the learning criterion during acquisition, and target and total responses during extinction. RESULTS: Children with ADHD required more trials to reach criterion compared to TD children under both continuous and partial reinforcement. After partial reinforcement, children with ADHD executed fewer target responses during extinction than TD children. Children with ADHD executed more responses than TD children during extinction, irrespective of learning condition. CONCLUSIONS: The findings demonstrate general difficulties in instrumental learning in ADHD, that is, slower learning irrespective of reinforcement schedule. They also show faster extinction following learning under partial reinforcement in those with ADHD, that is, a diminished PREE. Children with ADHD executed more responses during extinction. Results are theoretically important, with clinical implications for understanding and managing learning difficulties in those with ADHD, as they suggest poorer reinforcement learning and lower behavioral persistence.

Does stress consistently favor habits over goal-directed behaviors? Data from two preregistered exact replication studies.

Instrumental learning is controlled by two distinct parallel systems: goal-directed (action-outcome) and habitual (stimulus-response) processes. Seminal research by Schwabe and Wolf (2009, 2010) has demonstrated that stress renders behavior more habitual by decreasing goal-directed control. More recent studies yielded equivocal evidence for a stress-induced shift towards habitual responding, yet these studies used different paradigms to evaluate instrumental learning or used different stressors. Here, we performed exact replications of the original studies by exposing participants to an acute stressor either before (cf. Schwabe and Wolf, 2009) or directly after (cf. Schwabe and Wolf, 2010) an instrumental learning phase in which they had learned that distinct actions led to distinct, rewarding food outcomes (i.e., instrumental learning). Then, following an outcome devaluation phase in which one of the food outcomes was consumed until participants were satiated, action-outcome associations were tested in extinction. Despite successful instrumental learning and outcome devaluation and increased subjective and physiological stress levels following stress exposure, the stress and no-stress groups in both replication studies responded indifferently to valued and devalued outcomes. That is, non-stressed participants failed to demonstrate goal-directed behavioral control, thereby rendering the critical test of a shift from goal-directed to habitual control in the stress group inapt. Several reasons for these replication failures are discussed, including the rather indiscriminate devaluation of outcomes that may have contributed to indifferent responding during extinction, which emphasize the need to further our understanding of the boundary conditions in research aimed at demonstrating a stress-induced shift towards habitual control.

The impact of ADHD symptom severity on reinforcement and punishment learning among adults.

Introduction: Aberrations in feedback learning are hypothesised to contribute to the behavioural disruptions and impairment of attention-deficit/hyperactivity disorder (ADHD). However, few studies have evaluated the relation of reward/punishment feedback and ADHD symptom severity on learning. The current study evaluates the differential effects of reward and punishment feedback on learning among adults with elevated ADHD. Methods: One hundred five participants self-reported their level of current ADHD symptoms and completed an innovative instrumental learning task. Results: Consistent with predictions, participants with low self-reported ADHD symptom severity benefitted equally from reward and punishment feedback during the learning task, whereas participants with high self-reported symptom severity performed better (indexed by accuracy on learning task) from reward than punishment feedback trials. Conclusions: Overall, adults with high self-reported symptom severity of ADHD learned more from reward-based feedback, which provides critical implications for motivational theories about ADHD, as well as for treatment protocols. Future work should examine the translatability of results within a treatment setting.

Adolescent social isolation shifts the balance of decision-making strategy from goal-directed action to habitual response in adulthood via suppressing the excitatory neurotransmission onto the direct pathway of the dorsomedial striatum.

Adverse experience, such as social isolation, during adolescence is one of the major causes of neuropsychiatric disorders that extend from adolescence into adulthood, such as substance addiction, obsessive-compulsive disorder, and eating disorders leading to obesity. A common behavioral feature of these neuropsychiatric disorders is a shift in the balance of decision-making strategy from goal-directed action to habitual response. This study has verified that adolescent social isolation directly shifts the balance of decision-making strategy from goal-directed action to habitual response, and that it cannot be reversed by simple regrouping. This study has further revealed that adolescent social isolation induces a suppression in the excitatory neurotransmission onto the direct-pathway medium spiny neurons of the dorsomedial striatum (DMS), and that chemogenetically compensating this suppression effect shifts the balance of decision-making strategy from habitual response back to goal-directed action. These findings suggest that the plasticity in the DMS causes the shift in the balance of decision-making strategy, which would potentially help to develop a general therapy to treat the various neuropsychiatric disorders caused by adolescent social isolation. Such a study is especially necessary under the circumstances that social distancing and lockdown have caused during times of world-wide, society-wide pandemic.

Instant habits versus flexible tenacity: Do implementation intentions accelerate habit formation?

Implementation intentions (strategic "if-then" plans) have been shown to support behaviour change. This may be achieved by mentally forming stimulus-response associations, thereby promoting habit formation. Does this deliberate attempt to instal "strategic automaticity" only offer advantages, or does it also come at the cost of reduced flexibility that characterises learnt habits? To investigate this, we tested healthy, young participants on a computerised instrumental learning task. Critically, we introduced implementation intentions ("if I see stimulus X, then I will respond") versus goal intentions ("for outcome Z, I will respond)" during instrumental acquisition, and subsequently assessed behavioural flexibility in an outcome-revaluation test. In Experiment 1, we conducted a between-subjects manipulation of strategic planning, and in Experiment 2, a within-subject manipulation. We hypothesised that implementation intentions would lead to strong stimulus-response associations and consequently impair performance when the signalled outcome value changed and therefore required a different response, while benefitting performance when the outcome value (and required response) remained the same. We found that implementation intentions supported instrumental learning, but impaired test performance overall (most robustly in Experiment 2), irrespective of whether the signalled outcome value had changed. We argue that this general detrimental effect of implementation intentions on test performance is likely a consequence of their negative effect on stimulus-outcome learning. Our findings warrant caution when applying if-then plans to situations where the agent does not already possess perfect knowledge of behavioural contingencies.While implementation intentions may support efficient and fast behavioural execution, this may come at the expense of behavioural flexibility.

Evaluating instrumental learning and striatal-cortical functional connectivity in adolescent alcohol and cannabis use.

Adolescence is a vulnerable time for the acquisition of substance use disorders, potentially relating to ongoing development of neural circuits supporting instrumental learning. Striatal-cortical circuits undergo dynamic changes during instrumental learning and are implicated in contemporary addiction theory. Human studies have not yet investigated these dynamic changes in relation to adolescent substance use. Here, functional magnetic resonance imaging was used while 135 adolescents without (AUD-CUDLow ) and with significant alcohol (AUDHigh ) or cannabis use disorder symptoms (CUDHigh ) performed an instrumental learning task. We assessed how cumulative experience with instrumental cues altered cue selection preferences and functional connectivity strength between reward-sensitive striatal and cortical regions. Adolescents in AUDHigh and CUDHigh groups were slower in learning to select optimal instrumental cues relative to AUD-CUDLow adolescents. The relatively fast learning observed for AUD-CUDLow adolescents coincided with stronger functional connectivity between striatal and frontoparietal regions during early relative to later periods of task experience, whereas the slower learning for the CUDHigh group coincided with the opposite pattern. The AUDHigh group not only exhibited slower learning but also produced more instrumental choice errors relative to AUD-CUDLow adolescents. For the AUDHigh group, Bayesian analyses evidenced moderate support for no experience-related changes in striatal-frontoparietal connectivity strength during the task. Findings suggest that adolescent cannabis use is related to slowed instrumental learning and delays in peak functional connectivity strength between the striatal-frontoparietal regions that support this learning, whereas adolescent alcohol use may be more closely linked to broader impairments in instrumental learning and a general depression of the neural circuits supporting it.

Unconscious knowledge of rewards guides instrumental behaviors via conscious judgments.

The demonstration that unconscious learning supports instrumental behaviors (i.e., choosing the stimuli that lead to rewards) is central for the tenet that unconscious cognition sustains human adaptation. Recent studies, using reliable subliminal conditioning paradigms and improved awareness measurements have found evidence against unconscious knowledge sustaining accurate instrumental responses. The present preregistered study proposes a paradigm, in which unconscious processing is stimulated not by subliminally exposing the predictive (conditioned) stimuli, but by employing predictive regularities that are complex and difficult to detect consciously. Participants (N = 211) were exposed to letter strings that, unknown to them, were built from two complex artificial grammars: a "rewarded" or a "non-rewarded" grammar. On each trial, participants memorized a string, and subsequently had to discriminate the memorized string from a distractor. Correct discriminations were rewarded only when the identified string followed the rewarded grammar, but not when it followed the non-rewarded grammar. In a subsequent test phase, participants were presented with new strings from the rewarded and from the unrewarded grammar. Their task was now to directly choose the strings from the rewarded grammar, in order to collect more rewards. A trial-by-trial awareness measure revealed that participants accurately choose novel strings from the rewarded grammar when they had no conscious knowledge of the grammar. The awareness measure also showed that participants were accurate only when the unconsciously learned grammar led to conscious judgments. The present study shows that unconscious knowledge can guide instrumental responses, but only to the extent it supports conscious judgments.

Male DAT Val559 Mice Exhibit Compulsive Behavior under Devalued Reward Conditions Accompanied by Cellular and Pharmacological Changes.

Identified across multiple psychiatric disorders, the dopamine (DA) transporter (DAT) Ala559Val substitution triggers non-vesicular, anomalous DA efflux (ADE), perturbing DA neurotransmission and behavior. We have shown that DAT Val559 mice display a waiting impulsivity and changes in cognitive performance associated with enhanced reward motivation. Here, utilizing a within-subject, lever-pressing paradigm designed to bias the formation of goal-directed or habitual behavior, we demonstrate that DAT Val559 mice modulate their nose poke behavior appropriately to match context, but demonstrate a perseverative checking behavior. Although DAT Val559 mice display no issues with the cognitive flexibility required to acquire and re-learn a visual pairwise discrimination task, devaluation of reward evoked habitual reward seeking in DAT Val559 mutants in operant tasks regardless of reinforcement schedule. The direct DA agonist apomorphine also elicits locomotor stereotypies in DAT Val559, but not WT mice. Our observation that dendritic spine density is increased in the dorsal medial striatum (DMS) of DAT Val559 mice speaks to an imbalance in striatal circuitry that might underlie the propensity of DAT Val559 mutants to exhibit compulsive behaviors when reward is devalued. Thus, DAT Val559 mice represent a model for dissection of how altered DA signaling perturbs circuits that normally balance habitual and goal-directed behaviors.