From movement to motivation: a proposed framework to understand the antidepressant effect of exercise.

Depression is the leading cause of disability worldwide, exerting a profound negative impact on quality of life in those who experience it. Depression is associated with disruptions to several closely related neural and cognitive processes, including dopamine transmission, fronto-striatal brain activity and connectivity, reward processing and motivation. Physical activity, especially aerobic exercise, reduces depressive symptoms, but the mechanisms driving its antidepressant effects are poorly understood. Here we propose a novel hypothesis for understanding the antidepressant effects of exercise, centred on motivation, across different levels of explanation. There is robust evidence that aerobic exercise decreases systemic inflammation. Inflammation is known to reduce dopamine transmission, which in turn is strongly implicated in effort-based decision making for reward. Drawing on a broad range of research in humans and animals, we propose that by reducing inflammation and boosting dopamine transmission, with consequent effects on effort-based decision making for reward, exercise initially specifically improves 'interest-activity' symptoms of depression-namely anhedonia, fatigue and subjective cognitive impairment - by increasing propensity to exert effort. Extending this framework to the topic of cognitive control, we explain how cognitive impairment in depression may also be conceptualised through an effort-based decision-making framework, which may help to explain the impact of exercise on cognitive impairment. Understanding the mechanisms underlying the antidepressant effects of exercise could inform the development of novel intervention strategies, in particular personalised interventions and boost social prescribing.

Roles of feedback and feed-forward networks of dopamine subsystems: insights from Drosophila studies.

Across animal species, dopamine-operated memory systems comprise anatomically segregated, functionally diverse subsystems. Although individual subsystems could operate independently to support distinct types of memory, the logical interplay between subsystems is expected to enable more complex memory processing by allowing existing memory to influence future learning. Recent comprehensive ultrastructural analysis of the Drosophila mushroom body revealed intricate networks interconnecting the dopamine subsystems-the mushroom body compartments. Here, we review the functions of some of these connections that are beginning to be understood. Memory consolidation is mediated by two different forms of network: A recurrent feedback loop within a compartment maintains sustained dopamine activity required for consolidation, whereas feed-forward connections across compartments allow short-term memory formation in one compartment to open the gate for long-term memory formation in another compartment. Extinction and reversal of aversive memory rely on a similar feed-forward circuit motif that signals omission of punishment as a reward, which triggers plasticity that counteracts the original aversive memory trace. Finally, indirect feed-forward connections from a long-term memory compartment to short-term memory compartments mediate higher-order conditioning. Collectively, these emerging studies indicate that feedback control and hierarchical connectivity allow the dopamine subsystems to work cooperatively to support diverse and complex forms of learning.

Just do it: A neuropsychological theory of agency, cognition, mood, and dopamine.

Agency is the sense that one has control over one's own actions and the consequences of those actions. Despite the critical role that agency plays in the human condition, little is known about its neural basis. A novel theory proposes that increases in agency disinhibit the dopamine system and thereby increase the number of tonically active dopamine neurons in the ventral tegmental area. The theory, called ADDS (Agency Disinhibits the Dopamine System), proposes a specific neural network that mediates these effects. ADDS accurately predicts a variety of relevant neuroscience results, and makes many novel predictions, including that increases in an agency will (a) increase motivation, (b) improve executive function, (c) facilitate procedural learning, but only in the presence of immediate trial-by-trial feedback, (d) have little or no effect on learning-related effects of stimulus repetition or on standard eyeblink conditioning, (e) facilitate the development of automatic behaviors, but have little or no effect on the production of behaviors that are already automatized, (f) amplify the cognitive benefits of positive mood, and (g) reduce pain. The implications of this new theory are considered for several purely psychological theories that assign prominent roles to agency, including self-efficacy theory, hope theory, and goal-focused positive psychotherapy. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A mathematical model for the role of dopamine-D2 self-regulation in the production of ultradian rhythms.

Many self-motivated and goal-directed behaviours display highly flexible, approximately 4 hour ultradian (shorter than a day) oscillations. Despite lacking direct correspondence to physical cycles in the environment, these ultradian rhythms may be involved in optimizing functional interactions with the environment and reflect intrinsic neural dynamics. Current evidence supports a role of mesostriatal dopamine (DA) in the expression and propagation of ultradian rhythmicity, however, the biochemical processes underpinning these oscillations remain to be identified. Here, we use a mathematical model to investigate D2 autoreceptor-dependent DA self-regulation as the source of ultradian behavioural rhythms. DA concentration at the midbrain-striatal synapses is governed through a dual-negative feedback-loop structure, which naturally gives rise to rhythmicity. This model shows the propensity of striatal DA to produce an ultradian oscillation characterized by a flexible period that is highly sensitive to parameter variations. Circadian (approximately 24 hour) regulation consolidates the ultradian oscillations and alters their response to the phase-dependent, rapid-resetting effect of a transient excitatory stimulus. Within a circadian framework, the ultradian rhythm orchestrates behavioural activity and enhances responsiveness to an external stimulus. This suggests a role for the circadian-ultradian timekeeping hierarchy in governing organized behaviour and shaping daily experience through coordinating the motivation to engage in recurring, albeit not highly predictable events, such as social interactions.

Effect of positive social comparative feedback on the resting state connectivity of dopaminergic neural pathways: A preliminary investigation.

Positive social comparative feedback is hypothesized to generate a dopamine response in the brain, similar to reward, by enhancing expectancies to support motor skill learning. However, no studies have utilized neuroimaging to examine this hypothesized dopaminergic mechanism. Therefore, the aim of this preliminary study was to investigate the effect of positive social comparative feedback on dopaminergic neural pathways measured by resting state connectivity. Thirty individuals practiced an implicit, motor sequence learning task and were assigned to groups that differed in feedback type. One group received feedback about their actual response time to complete the task (RT ONLY), while the other group received feedback about their response time with positive social comparison (RT + POS). Magnetic resonance imaging was acquired at the beginning and end of repetitive motor practice with feedback to measure practice-dependent changes in resting state brain connectivity. While both groups showed improvements in task performance and increases in performance expectancies, ventral tegmental area and the left nucleus accumbens (mesolimbic dopamine pathway) resting state connectivity increased in the RT + POS group but not in the RT ONLY group. Instead, the RT ONLY group showed increased connectivity between ventral tegmental area and primary motor cortex. Positive social comparative feedback during practice of a motor sequence task may induce a dopaminergic response in the brain along the mesolimbic pathway. However, given that absence of effects on expectancies and motor learning, more robust and individualized approaches may be needed to provide beneficial psychological and behavioral effects.

Pharmacological manipulations of the dorsomedial and dorsolateral striatum during fear extinction reveal opposing roles in fear renewal.

Systemic manipulations that enhance dopamine (DA) transmission around the time of fear extinction can strengthen fear extinction and reduce conditioned fear relapse. Prior studies investigating the brain regions where DA augments fear extinction focus on targets of mesolimbic and mesocortical DA systems originating in the ventral tegmental area, given the role of these DA neurons in prediction error. The dorsal striatum (DS), a primary target of the nigrostriatal DA system originating in the substantia nigra (SN), is implicated in behaviors beyond its canonical role in movement, such as reward and punishment, goal-directed action, and stimulus-response associations, but whether DS DA contributes to fear extinction is unknown. We have observed that chemogenetic stimulation of SN DA neurons during fear extinction prevents the return of fear in contexts different from the extinction context, a form of relapse called renewal. This effect of SN DA stimulation is mimicked by a DA D1 receptor (D1R) agonist injected into the DS, thus implicating DS DA in fear extinction. Different DS subregions subserve unique functions of the DS, but it is unclear where in the DS D1R agonist acts during fear extinction to reduce renewal. Furthermore, although fear extinction increases neural activity in DS subregions, whether neural activity in DS subregions is causally involved in fear extinction is unknown. To explore the role of DS subregions in fear extinction, adult, male Long-Evans rats received microinjections of either the D1R agonist SKF38393 or a cocktail consisting of GABAA/GABAB receptor agonists muscimol/baclofen selectively into either dorsomedial (DMS) or dorsolateral (DLS) DS subregions immediately prior to fear extinction, and extinction retention and renewal were subsequently assessed drug-free. While increasing D1R signaling in the DMS during fear extinction did not impact fear extinction retention or renewal, DMS inactivation reduced later renewal. In contrast, DLS inactivation had no effect on fear extinction retention or renewal but increasing D1R signaling in the DLS during extinction reduced fear renewal. These data suggest that DMS and DLS activity during fear extinction can have opposing effects on later fear renewal, with the DMS promoting renewal and the DLS opposing renewal. Mechanisms through which the DS could influence the contextual gating of fear extinction are discussed.

Dopamine encoding of novelty facilitates efficient uncertainty-driven exploration.

When facing an unfamiliar environment, animals need to explore to gain new knowledge about which actions provide reward, but also put the newly acquired knowledge to use as quickly as possible. Optimal reinforcement learning strategies should therefore assess the uncertainties of these action-reward associations and utilise them to inform decision making. We propose a novel model whereby direct and indirect striatal pathways act together to estimate both the mean and variance of reward distributions, and mesolimbic dopaminergic neurons provide transient novelty signals, facilitating effective uncertainty-driven exploration. We utilised electrophysiological recording data to verify our model of the basal ganglia, and we fitted exploration strategies derived from the neural model to data from behavioural experiments. We also compared the performance of directed exploration strategies inspired by our basal ganglia model with other exploration algorithms including classic variants of upper confidence bound (UCB) strategy in simulation. The exploration strategies inspired by the basal ganglia model can achieve overall superior performance in simulation, and we found qualitatively similar results in fitting model to behavioural data compared with the fitting of more idealised normative models with less implementation level detail. Overall, our results suggest that transient dopamine levels in the basal ganglia that encode novelty could contribute to an uncertainty representation which efficiently drives exploration in reinforcement learning.

Neuroscience: Therapy modulates decision-making in Parkinson's disease.

There is mounting evidence that decision-making can be affected by treatment in Parkinson's disease. A new study shows that dopamine and deep brain stimulation, two mainstay treatments of Parkinson's, differently affect how patients make decisions weighing rewards against effort costs.

Local regulation of striatal dopamine: A diversity of circuit mechanisms for a diversity of behavioral functions?

Striatal dopamine governs a wide range of behavioral functions, yet local dopamine concentrations can be dissociated from somatic activity. Here, we discuss how dopamine's diverse roles in behavior may be driven by local circuit mechanisms shaping dopamine release. We first look at historical and recent work demonstrating that striatal circuits interact with dopaminergic terminals to either initiate the release of dopamine or modulate the release of dopamine initiated by spiking in midbrain dopamine neurons, with particular attention to GABAergic and cholinergic local circuit mechanisms. Then we discuss some of the first in vivo studies of acetylcholine-dopamine interactions in striatum and broadly discuss necessary future work in understanding the roles of midbrain versus striatal dopamine regulation.

Striatal and Behavioral Responses to Reward Vary by Socioeconomic Status in Adolescents.

Disparities in socioeconomic status (SES) lead to unequal access to financial and social support. These disparities are believed to influence reward sensitivity, which in turn are hypothesized to shape how individuals respond to and pursue rewarding experiences. However, surprisingly little is known about how SES shapes reward sensitivity in adolescence. Here, we investigated how SES influenced adolescent responses to reward, both in behavior and the striatum-a brain region that is highly sensitive to reward. We examined responses to both immediate reward (tracked by phasic dopamine) and average reward rate fluctuations (tracked by tonic dopamine) as these distinct signals independently shape learning and motivation. Adolescents (n = 114; 12-14 years; 58 female) performed a gambling task during functional magnetic resonance imaging. We manipulated trial-by-trial reward and loss outcomes, leading to fluctuations between periods of reward scarcity and abundance. We found that a higher reward rate hastened behavioral responses, and increased guess switching, consistent with the idea that reward abundance increases response vigor and exploration. Moreover, immediate reward reinforced previously rewarding decisions (win-stay, lose-switch) and slowed responses (postreward pausing), particularly when rewards were scarce. Notably, lower-SES adolescents slowed down less after rare rewards than higher-SES adolescents. In the brain, striatal activations covaried with the average reward rate across time and showed greater activations during rewarding blocks. However, these striatal effects were diminished in lower-SES adolescents. These findings show that the striatum tracks reward rate fluctuations, which shape decisions and motivation. Moreover, lower SES appears to attenuate reward-driven behavioral and brain responses.

Glutamate inputs send prediction error of reward, but not negative value of aversive stimuli, to dopamine neurons.

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation. Notably, whereas glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli into more positive responses, whereas excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

NMDA- and 6-OHDA-induced Lesions in the Nucleus Accumbens Differently Affect Maternal and Infanticidal Behavior in Pup-naïve Female and Male Mice.

Virgin and pups-naïve female and male adult mice display two opposite responses when they are exposed to pups for the first time. While females generally take care of the pups, males attack them. Since the nucleus accumbens (NA), and its dopaminergic modulation, is critical in integrating information and processing reward and aversion, we investigated if NMDA- and 6-OHDA-induced lesions, damaging mostly NA output and dopaminergic inputs respectively, affected female maternal behavior (MB) or male infanticidal behavior (IB) in mice. Our results revealed minor or no effects of both smaller and larger NMDA-induced lesions in MB and IB. On the other hand, while 6-OHDA-induced lesions in females reduced the incidence of full MB (12.5% 6-OHDA vs. 85.7% SHAM) increasing the latency to retrieve the pups, those lesions did not affect IB in males. There were no differences in locomotor and exploratory activity between the lesioned- and SHAM- females. Despite those lesions did not induce any major effect on IB, NMDA-lesioned males spent less time in the central area of an open field, while dopaminergic-lesioned males showed reduced number of rearing and peripheral crosses. The current study shows that an intact NA is not necessary for the expression of MB and IB. However, dopaminergic inputs to NA play different role in MB and IB. While damaging dopaminergic terminals into the NA did not affect IB, it clearly delayed the more flexible and rewarding expression of parental behavior.

Dopamine regulates attitude toward risk.

Specific brain pathways can lower or raise the willingness of monkeys to take risks.

Dopamine's reach: Unlocked by sleep loss.

In this issue of Neuron, Wu et al.1 employ cutting-edge techniques to provide a mechanistic understanding of how sleep deprivation induces an altered affective state. They reveal a key function for dopaminergic signaling, and the formation of cortical spines, in this process.

Dopamine-independent effect of rewards on choices through hidden-state inference.

Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine activity, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model where cortex learned to track hidden task states by predicting observations, while basal ganglia learned values and actions via RPEs. This shows that the influence of rewards on choices can stem from dopamine-independent information they convey about the world's state, not the dopaminergic RPEs they produce.

Prediction error in dopamine neurons during associative learning.

Dopamine neurons have long been thought to facilitate learning by broadcasting reward prediction error (RPE), a teaching signal used in machine learning, but more recent work has advanced alternative models of dopamine's computational role. Here, I revisit this critical issue and review new experimental evidences that tighten the link between dopamine activity and RPE. First, I introduce the recent observation of a gradual backward shift of dopamine activity that had eluded researchers for over a decade. I also discuss several other findings, such as dopamine ramping, that were initially interpreted to conflict but later found to be consistent with RPE. These findings improve our understanding of neural computation in dopamine neurons.

An excitatory projection from the basal forebrain to the ventral tegmental area that underlies anorexia-like phenotypes.

Maladaptation in balancing internal energy needs and external threat cues may result in eating disorders. However, brain mechanisms underlying such maladaptations remain elusive. Here, we identified that the basal forebrain (BF) sends glutamatergic projections to glutamatergic neurons in the ventral tegmental area (VTA) in mice. Glutamatergic neurons in both regions displayed correlated responses to various stressors. Notably, in vivo manipulation of BF terminals in the VTA revealed that the glutamatergic BF → VTA circuit reduces appetite, increases locomotion, and elicits avoidance. Consistently, activation of VTA glutamatergic neurons reduced body weight, blunted food motivation, and caused hyperactivity with behavioral signs of anxiety, all hallmarks of typical anorexia symptoms. Importantly, activation of BF glutamatergic terminals in the VTA reduced dopamine release in the nucleus accumbens. Collectively, our results point to overactivation of the glutamatergic BF → VTA circuit as a potential cause of anorexia-like phenotypes involving reduced dopamine release.

65 years of research on dopamine's role in classical fear conditioning and extinction: A systematic review.

Dopamine, a catecholamine neurotransmitter, has historically been associated with the encoding of reward, whereas its role in aversion has received less attention. Here, we systematically gathered the vast evidence of the role of dopamine in the simplest forms of aversive learning: classical fear conditioning and extinction. In the past, crude methods were used to augment or inhibit dopamine to study its relationship with fear conditioning and extinction. More advanced techniques such as conditional genetic, chemogenic and optogenetic approaches now provide causal evidence for dopamine's role in these learning processes. Dopamine neurons encode conditioned stimuli during fear conditioning and extinction and convey the signal via activation of D1-4 receptor sites particularly in the amygdala, prefrontal cortex and striatum. The coordinated activation of dopamine receptors allows for the continuous formation, consolidation, retrieval and updating of fear and extinction memory in a dynamic and reciprocal manner. Based on the reviewed literature, we conclude that dopamine is crucial for the encoding of classical fear conditioning and extinction and contributes in a way that is comparable to its role in encoding reward.

Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.

Striatal dopamine integrates cost, benefit, and motivation.

Striatal dopamine (DA) release has long been linked to reward processing, but it remains controversial whether DA release reflects costs or benefits and how these signals vary with motivation. Here, we measure DA release in the nucleus accumbens (NAc) and dorsolateral striatum (DLS) while independently varying costs and benefits and apply behavioral economic principles to determine a mouse's level of motivation. We reveal that DA release in both structures incorporates both reward magnitude and sunk cost. Surprisingly, motivation was inversely correlated with reward-evoked DA release. Furthermore, optogenetically evoked DA release was also heavily dependent on sunk cost. Our results reconcile previous disparate findings by demonstrating that striatal DA release simultaneously encodes cost, benefit, and motivation but in distinct manners over different timescales. Future work will be necessary to determine whether the reduction in phasic DA release in highly motivated animals is due to changes in tonic DA levels.