Dopamine Modulation of Intertemporal Decision-making: Evidence from Parkinson Disease.

Choosing between smaller prompt rewards and larger later rewards is a common choice problem, and studies widely agree that frontostriatal circuits heavily innervated by dopamine are centrally involved. Understanding how dopamine modulates intertemporal choice has important implications for neurobiological models and for understanding the mechanisms underlying maladaptive decision-making. However, the specific role of dopamine in intertemporal decisions is not well understood. Dopamine may play a role in multiple aspects of intertemporal choices--the valuation of choice outcomes and sensitivity to reward delays. To assess the role of dopamine in intertemporal decisions, we tested Parkinson disease patients who suffer from dopamine depletion in the striatum, in either high (on medication, PDON) or low (off medication, PDOFF) dopaminergic states. Compared with both PDOFF and healthy controls, PDON made more farsighted choices and reduced their valuations less as a function of increasing time to reward. Furthermore, reduced discounting in the high dopaminergic state was robust across multiple measures, providing new evidence for dopamine's role in making decisions about the future.

Episodic memory encoding interferes with reward learning and decreases striatal prediction errors.

Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making.

A trade-off between feedback-based learning and episodic memory for feedback events: evidence from Parkinson's disease.

Parkinson's disease (PD) is characterized by a loss of dopaminergic projections to the striatum, leading to both motor and cognitive impairments. The cognitive impairments are relatively selective and include deficits in incremental learning from trial-by-trial feedback, while other forms of learning, such as hippocampal-dependent episodic memory, remain intact. Interestingly, it has been suggested that the striatum and the hippocampus compete during learning, leading to the intriguing prediction that the striatal disruption in PD could lead to enhanced performance on tasks that depend on the hippocampus. We tested this prediction by simultaneously assessing incremental learning and episodic memory for trial-unique feedback events, within a single task, in patients with PD. Further, in order to modulate the engagement of the striatum versus the hippocampus, we manipulated the timing of feedback during learning, building on prior results showing that delaying feedback by a few seconds shifts learning to depend on the hippocampus instead of the striatum. We found that Parkinson's patients were impaired at learning from immediate feedback, but had enhanced episodic memory for those immediate feedback events. Thus, our results provide evidence for concurrent impaired and enhanced learning and memory functions within the same group of patients from a single task.

Retroactive and graded prioritization of memory by reward.

Many decisions are based on an internal model of the world. Yet, how such a model is constructed from experience and represented in memory remains unknown. We test the hypothesis that reward shapes memory for sequences of events by retroactively prioritizing memory for objects as a function of their distance from reward. Human participants encountered neutral objects while exploring a series of mazes for reward. Across six data sets, we find that reward systematically modulates memory for neutral objects, retroactively prioritizing memory for objects closest to the reward. This effect of reward on memory emerges only after a 24-hour delay and is stronger for mazes followed by a longer rest interval, suggesting a role for post-reward replay and overnight consolidation, as predicted by neurobiological data in animals. These findings demonstrate that reward retroactively prioritizes memory along a sequential gradient, consistent with the role of memory in supporting adaptive decision-making.

Motivational modes and learning in Parkinson's disease.

Learning and motivation are intrinsically related, and both have been linked to dopamine. Parkinson's disease results from a progressive loss of dopaminergic inputs to the striatum and leads to impairments in motivation and learning from feedback. However, the link between motivation and learning in Parkinson's disease is not well understood. To address this gap, we leverage a well-established psychological theory of motivation, regulatory mode theory, which distinguishes between two functionally independent motivational concerns in regulating behavior: a concern with having an effect by initiating and maintaining movement (Locomotion) and a concern with establishing what is correct by critically evaluating goal pursuit means and outcomes (Assessment). We examined Locomotion and Assessment in patients with Parkinson's disease and age-matched controls. Parkinson's disease patients demonstrated a selective decrease in Assessment motivation but no change in Locomotion motivation, suggesting that Parkinson's disease leads to a reduced tendency to evaluate and monitor outcomes. Moreover, weaker Assessment motivation was correlated with poorer performance on a feedback-based learning task previously shown to depend on the striatum. Together, these findings link a questionnaire-based personality inventory with performance on a well-characterized experimental task, advancing our understanding of how Parkinson's disease affects motivation with implications for well-being and treatment outcomes.

Mind matters: placebo enhances reward learning in Parkinson's disease.

Expectations have a powerful influence on how we experience the world. Neurobiological and computational models of learning suggest that dopamine is crucial for shaping expectations of reward and that expectations alone may influence dopamine levels. However, because expectations and reinforcers are typically manipulated together, the role of expectations per se has remained unclear. We separated these two factors using a placebo dopaminergic manipulation in individuals with Parkinson's disease. We combined a reward learning task with functional magnetic resonance imaging to test how expectations of dopamine release modulate learning-related activity in the brain. We found that the mere expectation of dopamine release enhanced reward learning and modulated learning-related signals in the striatum and the ventromedial prefrontal cortex. These effects were selective to learning from reward: neither medication nor placebo had an effect on learning to avoid monetary loss. These findings suggest a neurobiological mechanism by which expectations shape learning and affect.