Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases.

"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.

Striatal BOLD and Midfrontal Theta Power Express Motivation for Action.

Action selection is biased by the valence of anticipated outcomes. To assess mechanisms by which these motivational biases are expressed and controlled, we measured simultaneous EEG-fMRI during a motivational Go/NoGo learning task (N = 36), leveraging the temporal resolution of EEG and subcortical access of fMRI. VmPFC BOLD encoded cue valence, importantly predicting trial-by-trial valence-driven response speed differences and EEG theta power around cue onset. In contrast, striatal BOLD encoded selection of active Go responses and correlated with theta power around response time. Within trials, theta power ramped in the fashion of an evidence accumulation signal for the value of making a "Go" response, capturing the faster responding to reward cues. Our findings reveal a dual nature of midfrontal theta power, with early components reflecting the vmPFC contribution to motivational biases, and late components reflecting their striatal translation into behavior, in line with influential recent "value of work" theories of striatal processing.

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans.

BACKGROUND: The non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain incompletely understood. Here we examined how LSD affects probabilistic reversal learning (PRL) in healthy humans. METHODS: Healthy volunteers received intravenous LSD (75 µg in 10 mL saline) or placebo (10 mL saline) in a within-subjects design and completed a PRL task. Participants had to learn through trial and error which of three stimuli was rewarded most of the time, and these contingencies switched in a reversal phase. Computational models of reinforcement learning (RL) were fitted to the behavioural data to assess how LSD affected the updating ('learning rates') and deployment of value representations ('reinforcement sensitivity') during choice, as well as 'stimulus stickiness' (choice repetition irrespective of reinforcement history). RESULTS: Raw data measures assessing sensitivity to immediate feedback ('win-stay' and 'lose-shift' probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was the enhancement of the reward learning rate. The punishment learning rate was also elevated. Stimulus stickiness was decreased by LSD, reflecting heightened exploration. Reinforcement sensitivity differed by phase. CONCLUSIONS: Increased RL rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur in the clinical application of LSD.

Challenging the negative learning bias hypothesis of depression: reversal learning in a naturalistic psychiatric sample.

BACKGROUND: Classic theories posit that depression is driven by a negative learning bias. Most studies supporting this proposition used small and selected samples, excluding patients with comorbidities. However, comorbidity between psychiatric disorders occurs in up to 70% of the population. Therefore, the generalizability of the negative bias hypothesis to a naturalistic psychiatric sample as well as the specificity of the bias to depression, remain unclear. In the present study, we tested the negative learning bias hypothesis in a large naturalistic sample of psychiatric patients, including depression, anxiety, addiction, attention-deficit/hyperactivity disorder, and/or autism. First, we assessed whether the negative bias hypothesis of depression generalized to a heterogeneous (and hence more naturalistic) depression sample compared with controls. Second, we assessed whether negative bias extends to other psychiatric disorders. Third, we adopted a dimensional approach, by using symptom severity as a way to assess associations across the sample. METHODS: We administered a probabilistic reversal learning task to 217 patients and 81 healthy controls. According to the negative bias hypothesis, participants with depression should exhibit enhanced learning and flexibility based on punishment v. reward. We combined analyses of traditional measures with more sensitive computational modeling. RESULTS: In contrast to previous findings, this sample of depressed patients with psychiatric comorbidities did not show a negative learning bias. CONCLUSIONS: These results speak against the generalizability of the negative learning bias hypothesis to depressed patients with comorbidities. This study highlights the importance of investigating unselected samples of psychiatric patients, which represent the vast majority of the psychiatric population.

Frontal network dynamics reflect neurocomputational mechanisms for reducing maladaptive biases in motivated action.

Motivation exerts control over behavior by eliciting Pavlovian responses, which can either match or conflict with instrumental action. We can overcome maladaptive motivational influences putatively through frontal cognitive control. However, the neurocomputational mechanisms subserving this control are unclear; does control entail up-regulating instrumental systems, down-regulating Pavlovian systems, or both? We combined electroencephalography (EEG) recordings with a motivational Go/NoGo learning task (N = 34), in which multiple Go options enabled us to disentangle selective action learning from nonselective Pavlovian responses. Midfrontal theta-band (4 Hz-8 Hz) activity covaried with the level of Pavlovian conflict and was associated with reduced Pavlovian biases rather than reduced instrumental learning biases. Motor and lateral prefrontal regions synchronized to the midfrontal cortex, and these network dynamics predicted the reduction of Pavlovian biases over and above local, midfrontal theta activity. This work links midfrontal processing to detecting Pavlovian conflict and highlights the importance of network processing in reducing the impact of maladaptive, Pavlovian biases.

Effects of dopamine on reinforcement learning in Parkinson's disease depend on motor phenotype.

Parkinson's disease is clinically defined by bradykinesia, along with rigidity and tremor. However, the severity of these motor signs is greatly variable between individuals, particularly the presence or absence of tremor. This variability in tremor relates to variation in cognitive/motivational impairment, as well as the spatial distribution of neurodegeneration in the midbrain and dopamine depletion in the striatum. Here we ask whether interindividual heterogeneity in tremor symptoms could account for the puzzlingly large variability in the effects of dopaminergic medication on reinforcement learning, a fundamental cognitive function known to rely on dopamine. Given that tremor-dominant and non-tremor Parkinson's disease patients have different dopaminergic phenotypes, we hypothesized that effects of dopaminergic medication on reinforcement learning differ between tremor-dominant and non-tremor patients. Forty-three tremor-dominant and 20 non-tremor patients with Parkinson's disease were recruited to be tested both OFF and ON dopaminergic medication (200/50 mg levodopa-benserazide), while 22 age-matched control subjects were recruited to be tested twice OFF medication. Participants performed a reinforcement learning task designed to dissociate effects on learning rate from effects on motivational choice (i.e. the tendency to 'Go/NoGo' in the face of reward/threat of punishment). In non-tremor patients, dopaminergic medication improved reward-based choice, replicating previous studies. In contrast, in tremor-dominant patients, dopaminergic medication improved learning from punishment. Formal modelling showed divergent computational effects of dopaminergic medication as a function of Parkinson's disease motor phenotype, with a modulation of motivational choice bias and learning rate in non-tremor and tremor patients, respectively. This finding establishes a novel cognitive/motivational difference between tremor and non-tremor Parkinson's disease patients, and highlights the importance of considering motor phenotype in future work.

GABAergic changes in the thalamocortical circuit in Parkinson's disease.

Parkinson's disease is characterized by bradykinesia, rigidity, and tremor. These symptoms have been related to an increased gamma-aminobutyric acid (GABA)ergic inhibitory drive from globus pallidus onto the thalamus. However, in vivo empirical evidence for the role of GABA in Parkinson's disease is limited. Some discrepancies in the literature may be explained by the presence or absence of tremor. Specifically, recent functional magnetic resonance imaging (fMRI) findings suggest that Parkinson's tremor is associated with reduced, dopamine-dependent thalamic inhibition. Here, we tested the hypothesis that GABA in the thalamocortical motor circuit is increased in Parkinson's disease, and we explored differences between clinical phenotypes. We included 60 Parkinson patients with dopamine-resistant tremor (n = 17), dopamine-responsive tremor (n = 23), or no tremor (n = 20), and healthy controls (n = 22). Using magnetic resonance spectroscopy, we measured GABA-to-total-creatine ratio in motor cortex, thalamus, and a control region (visual cortex) on two separate days (ON and OFF dopaminergic medication). GABA levels were unaltered by Parkinson's disease, clinical phenotype, or medication. However, motor cortex GABA levels were inversely correlated with disease severity, particularly rigidity and tremor, both ON and OFF medication. We conclude that cortical GABA plays a beneficial rather than a detrimental role in Parkinson's disease, and that GABA depletion may contribute to increased motor symptom expression.

Pavlovian Control of Escape and Avoidance.

To survive in complex environments, animals need to have mechanisms to select effective actions quickly, with minimal computational costs. As perhaps the computationally most parsimonious of these systems, Pavlovian control accomplishes this by hardwiring specific stereotyped responses to certain classes of stimuli. It is well documented that appetitive cues initiate a Pavlovian bias toward vigorous approach; however, Pavlovian responses to aversive stimuli are less well understood. Gaining a deeper understanding of aversive Pavlovian responses, such as active avoidance, is important given the critical role these behaviors play in several psychiatric conditions. The goal of the current study was to establish a behavioral and computational framework to examine aversive Pavlovian responses (activation vs. inhibition) depending on the proximity of an aversive state (escape vs. avoidance). We introduce a novel task in which participants are exposed to primary aversive (noise) stimuli and characterized behavior using a novel generative computational model. This model combines reinforcement learning and drift-diffusion models so as to capture effects of invigoration/inhibition in both explicit choice behavior as well as changes in RT. Choice and RT results both suggest that escape is associated with a bias for vigorous action, whereas avoidance is associated with behavioral inhibition. These results lay a foundation for future work seeking insights into typical and atypical aversive Pavlovian responses involved in psychiatric disorders, allowing us to quantify both implicit and explicit indices of vigorous choice behavior in the context of aversion.

Dopamine controls Parkinson's tremor by inhibiting the cerebellar thalamus.

Parkinson's resting tremor is related to altered cerebral activity in the basal ganglia and the cerebello-thalamo-cortical circuit. Although Parkinson's disease is characterized by dopamine depletion in the basal ganglia, the dopaminergic basis of resting tremor remains unclear: dopaminergic medication reduces tremor in some patients, but many patients have a dopamine-resistant tremor. Using pharmacological functional magnetic resonance imaging, we test how a dopaminergic intervention influences the cerebral circuit involved in Parkinson's tremor. From a sample of 40 patients with Parkinson's disease, we selected 15 patients with a clearly tremor-dominant phenotype. We compared tremor-related activity and effective connectivity (using combined electromyography-functional magnetic resonance imaging) on two occasions: ON and OFF dopaminergic medication. Building on a recently developed cerebral model of Parkinson's tremor, we tested the effect of dopamine on cerebral activity associated with the onset of tremor episodes (in the basal ganglia) and with tremor amplitude (in the cerebello-thalamo-cortical circuit). Dopaminergic medication reduced clinical resting tremor scores (mean 28%, range -12 to 68%). Furthermore, dopaminergic medication reduced tremor onset-related activity in the globus pallidus and tremor amplitude-related activity in the thalamic ventral intermediate nucleus. Network analyses using dynamic causal modelling showed that dopamine directly increased self-inhibition of the ventral intermediate nucleus, rather than indirectly influencing the cerebello-thalamo-cortical circuit through the basal ganglia. Crucially, the magnitude of thalamic self-inhibition predicted the clinical dopamine response of tremor. Dopamine reduces resting tremor by potentiating inhibitory mechanisms in a cerebellar nucleus of the thalamus (ventral intermediate nucleus). This suggests that altered dopaminergic projections to the cerebello-thalamo-cortical circuit have a role in Parkinson's tremor.aww331media15307619934001.

Dopaminergic Modulation of the Functional Ventrodorsal Architecture of the Human Striatum.

Interactions between motivational, cognitive, and motor regions of the striatum are crucial for implementing behavioral control. Work with experimental animals indicates that such interactions are sensitive to modulation by dopamine. Using systematic pharmacological manipulation of dopamine D2-receptors and resting-state functional imaging, we defined the functional architecture of the human striatum and quantified the effects of dopaminergic drugs on intrinsic effective connectivity between striatal subregions. We found that dopamine modulates interactions between motivational and cognitive regions, as well cognitive and motor regions of the striatum. Stimulation and blockade of the dopamine D2-receptor had opposite (increasing and decreasing) effects on the efficacy of those interactions. Furthermore, trait impulsivity was specifically associated with dopaminergic modulation of ventral-to-dorsal striatal connectivity. Individuals with high trait impulsivity exhibited greater drug-induced increases (after stimulation) and decreases (after blockade) of ventral-to-dorsal striatal connectivity than those with low trait impulsivity. These observations establish a key link between dopamine, intrinsic effective connectivity between striatal subregions, and trait impulsivity.

A hemodynamic model for layered BOLD signals.

High-resolution blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) at the sub-millimeter scale has become feasible with recent advances in MR technology. In principle, this would enable the study of layered cortical circuits, one of the fundaments of cortical computation. However, the spatial layout of cortical blood supply may become an important confound at such high resolution. In particular, venous blood draining back to the cortical surface perpendicularly to the layered structure is expected to influence the measured responses in different layers. Here, we present an extension of a hemodynamic model commonly used for analyzing fMRI data (in dynamic causal models or biophysical network models) that accounts for such blood draining effects by coupling local hemodynamics across layers. We illustrate the properties of the model and its inversion by a series of simulations and show that it successfully captures layered fMRI data obtained during a simple visual experiment. We conclude that for future studies of the dynamics of layered neuronal circuits with high-resolution fMRI, it will be pivotal to include effects of blood draining, particularly when trying to infer on the layer-specific connections in cortex--a theme of key relevance for brain disorders like schizophrenia and for theories of brain function such as predictive coding.

Selective attentional enhancement and inhibition of fronto-posterior connectivity by the basal ganglia during attention switching.

The prefrontal cortex and the basal ganglia interact to selectively gate a desired action. Recent studies have shown that this selective gating mechanism of the basal ganglia extends to the domain of attention. Here, we investigate the nature of this action-like gating mechanism for attention using a spatial attention-switching paradigm in combination with functional neuroimaging and dynamic causal modeling. We show that the basal ganglia guide attention by focally releasing inhibition of task-relevant representations, while simultaneously inhibiting task-irrelevant representations by selectively modulating prefrontal top-down connections. These results strengthen and specify the role of the basal ganglia in attention. Moreover, our findings have implications for psychological theorizing by suggesting that inhibition of unattended sensory regions is not only a consequence of mutual suppression, but is an active process, subserved by the basal ganglia.

Serotonin and aversive Pavlovian control of instrumental behavior in humans.

Adaptive decision-making involves interaction between systems regulating Pavlovian and instrumental control of behavior. Here we investigate in humans the role of serotonin in such Pavlovian-instrumental transfer in both the aversive and the appetitive domain using acute tryptophan depletion, known to lower central serotonin levels. Acute tryptophan depletion attenuated the inhibiting effect of aversive Pavlovian cues on instrumental behavior, while leaving unaltered the activating effect of appetitive Pavlovian cues. These data suggest that serotonin is selectively involved in Pavlovian inhibition due to aversive expectations and have implications for our understanding of the mechanisms underlying a range of affective, impulsive, and aggressive neuropsychiatric disorders.

Reduced serotonin transporter availability decreases prefrontal control of the amygdala.

After a threatening event, the risk of developing social psychopathologies is increased in short-allele (s) carriers of the serotonin transporter gene. The amygdala becomes overresponsive to emotional stimuli, an effect that could be driven by local hypersensitivity or by reduced prefrontal regulation. This study distinguishes between these two hypotheses by using dynamic causal modeling of fMRI data acquired in a preselected cohort of human s-carriers and homozygous long-allele carriers. Increased amygdala activity in s-carriers originates from reduced prefrontal inhibitory regulation when social emotional behavior needs to be controlled, suggesting a mechanism for increased vulnerability to psychopathologies.

Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases.

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

Aversive Pavlovian control of instrumental behavior in humans.

Adaptive behavior involves interactions between systems regulating Pavlovian and instrumental control of actions. Here, we present the first investigation of the neural mechanisms underlying aversive Pavlovian-instrumental transfer using fMRI in humans. Recent evidence indicates that these Pavlovian influences on instrumental actions are action-specific: Instrumental approach is invigorated by appetitive Pavlovian cues but inhibited by aversive Pavlovian cues. Conversely, instrumental withdrawal is inhibited by appetitive Pavlovian cues but invigorated by aversive Pavlovian cues. We show that BOLD responses in the amygdala and the nucleus accumbens were associated with behavioral inhibition by aversive Pavlovian cues, irrespective of action context. Furthermore, BOLD responses in the ventromedial prefrontal cortex differed between approach and withdrawal actions. Aversive Pavlovian conditioned stimuli modulated connectivity between the ventromedial prefrontal cortex and the caudate nucleus. These results show that action-specific aversive control of instrumental behavior involves the modulation of fronto-striatal interactions by Pavlovian conditioned stimuli.

Goal-directed recruitment of Pavlovian biases through selective visual attention.

Prospective outcomes bias behavior in a "Pavlovian" manner: Reward prospect invigorates action, while punishment prospect suppresses it. Theories have posited Pavlovian biases as global action "priors" in unfamiliar or uncontrollable environments. However, this account fails to explain the strength of these biases-causing frequent action slips-even in well-known environments. We propose that Pavlovian control is additionally useful if flexibly recruited by instrumental control. Specifically, instrumental action plans might shape selective attention to reward/punishment information and thus the input to Pavlovian control. In two eye-tracking samples (N = 35/64), we observed that Go/NoGo action plans influenced when and for how long participants attended to reward/punishment information, which in turn biased their responses in a Pavlovian manner. Participants with stronger attentional effects showed higher performance. Thus, humans appear to align Pavlovian control with their instrumental action plans, extending its role beyond action defaults to a powerful tool ensuring robust action execution. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Effective connectivity determines the nature of subjective experience in grapheme-color synesthesia.

Synesthesia provides an elegant model to investigate neural mechanisms underlying individual differences in subjective experience in humans. In grapheme-color synesthesia, written letters induce color sensations, accompanied by activation of color area V4. Competing hypotheses suggest that enhanced V4 activity during synesthesia is either induced by direct bottom-up cross-activation from grapheme processing areas within the fusiform gyrus, or indirectly via higher-order parietal areas. Synesthetes differ in the way synesthetic color is perceived: "projector" synesthetes experience color externally colocalized with a presented grapheme, whereas "associators" report an internally evoked association. Using dynamic causal modeling for fMRI, we show that V4 cross-activation during synesthesia was induced via a bottom-up pathway (within fusiform gyrus) in projector synesthetes, but via a top-down pathway (via parietal lobe) in associators. These findings show how altered coupling within the same network of active regions leads to differences in subjective experience. Our findings reconcile the two most influential cross-activation accounts of synesthesia.

Aversive Pavlovian inhibition in adult attention-deficit/hyperactivity disorder and its restoration by mindfulness-based cognitive therapy.

Background: Control over the tendency to make or withhold responses guided by contextual Pavlovian information plays a key role in understanding impulsivity and hyperactivity. Here we set out to assess (1) the understudied relation between contextual Pavlovian inhibitory control and hyperactivity/impulsivity in adults with ADHD and (2) whether this inhibition can be enhanced by mindfulness based cognitive therapy (MBCT). Methods: Within the framework of a randomized controlled trial 50 Adult ADHD patients were assessed before and after 8 weeks of treatment as usual (TAU) with (n = 24) or without (n = 26) MBCT. We employed a well-established behavioral Pavlovian-to-instrumental transfer task that quantifies Pavlovian inhibitory control over instrumental behavior. Results: Task results revealed (1) less aversive Pavlovian inhibition in ADHD patients with clinically relevant hyperactivity/impulsivity than in those without; and (2) enhanced Pavlovian inhibition across all ADHD patients after TAU+MBCT compared with TAU. Conclusion: These findings offer new insights in the neurocognitive mechanisms of hyperactivity/impulsivity in ADHD and its treatment: We reveal a role for Pavlovian inhibitory mechanisms in understanding hyperactive/impulsive behaviors in ADHD and point toward MBCT as an intervention that might influence these mechanisms.

Cortical dopamine reduces the impact of motivational biases governing automated behaviour.

Motivations shape our behaviour: the promise of reward invigorates, while in the face of punishment, we hold back. Abnormalities of motivational processing are implicated in clinical disorders characterised by excessive habits and loss of top-down control, notably substance and behavioural addictions. Striatal and frontal dopamine have been hypothesised to play complementary roles in the respective generation and control of these motivational biases. However, while dopaminergic interventions have indeed been found to modulate motivational biases, these previous pharmacological studies used regionally non-selective pharmacological agents. Here, we tested the hypothesis that frontal dopamine controls the balance between Pavlovian, bias-driven automated responding and instrumentally learned action values. Specifically, we examined whether selective enhancement of cortical dopamine either (i) enables adaptive suppression of Pavlovian control when biases are maladaptive; or (ii) non-specifically modulates the degree of bias-driven automated responding. Healthy individuals (n = 35) received the catechol-o-methyltransferase (COMT) inhibitor tolcapone in a randomised, double-blind, placebo-controlled cross-over design, and completed a motivational Go NoGo task known to elicit motivational biases. In support of hypothesis (ii), tolcapone globally decreased motivational bias. Specifically, tolcapone improved performance on trials where the bias was unhelpful, but impaired performance in bias-congruent conditions. These results indicate a non-selective role for cortical dopamine in the regulation of motivational processes underpinning top-down control over automated behaviour. The findings have direct relevance to understanding neurobiological mechanisms underpinning addiction and obsessive-compulsive disorders, as well as highlighting a potential trans-diagnostic novel mechanism to address such symptoms.