Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning.

Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [11C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases.

Motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function.

Dopaminergic neurotransmission plays a pivotal role in appetitively motivated behavior in mammals, including humans. Notably, action and valence are not independent in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward. We have previously observed that the carriers of the DRD2/ANKK1 TaqIA A1 allele, that has been associated with reduced striatal dopamine D2 receptor expression, showed a diminished learning performance when required to learn response inhibition to obtain rewards, a finding that was replicated in two independent cohorts. With our present study, we followed two aims: first, we aimed to replicate our finding on the DRD2/ANKK1 TaqIA polymorphism in a third independent cohort (N = 99) and to investigate the nature of the genetic effects more closely using trial-by-trial behavioral analysis and computational modeling in the combined dataset (N = 281). Second, we aimed to assess a potentially modulatory role of prefrontal dopamine availability, using the widely studied COMT Val108/158Met polymorphism as a proxy. We first report a replication of the above mentioned finding. Interestingly, after combining all three cohorts, exploratory analyses regarding the COMT Val108/158Met polymorphism suggest that homozygotes for the Met allele, which has been linked to higher prefrontal dopaminergic tone, show a lower learning bias. Our results corroborate the importance of genetic variability of the dopaminergic system in individual learning differences of action-valence interaction and, furthermore, suggest that motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function.

Corticostriatal White Matter Integrity and Dopamine D1 Receptor Availability Predict Age Differences in Prefrontal Value Signaling during Reward Learning.

Probabilistic reward learning reflects the ability to adapt choices based on probabilistic feedback. The dopaminergically innervated corticostriatal circuit in the brain plays an important role in supporting successful probabilistic reward learning. Several components of the corticostriatal circuit deteriorate with age, as it does probabilistic reward learning. We showed previously that D1 receptor availability in NAcc predicts the strength of anticipatory value signaling in vmPFC, a neural correlate of probabilistic learning that is attenuated in older participants and predicts probabilistic reward learning performance. We investigated how white matter integrity in the pathway between nucleus accumbens (NAcc) and ventromedial prefrontal cortex (vmPFC) relates to the strength of anticipatory value signaling in vmPFC in younger and older participants. We found that in a sample of 22 old and 23 young participants, fractional anisotropy in the pathway between NAcc and vmPFC predicted the strength of value signaling in vmPFC independently from D1 receptor availability in NAcc. These findings provide tentative evidence that integrity in the dopaminergic and white matter pathways of corticostriatal circuitry supports the expression of value signaling in vmPFC which supports reward learning, however, the limited sample size calls for independent replication. These and future findings could add to the improved understanding of how corticostriatal integrity contributes to reward learning ability.

The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7 T MRI Aging Study.

Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7 Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age.

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models.

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.

Dopamine and reward-related vigor in younger and older adults.

Vigor reflects how motivated people are to respond to stimuli. We previously showed that, on average, humans are more vigorous when a higher rate of reward is available, and that this relationship is modulated by the dopamine precursor levodopa. Dopamine signaling and probabilistic reward learning deteriorate across the adult life span, and thus, the relationship between vigor and reward may also change in aging. We tested this assertion and assessed whether it correlates with D1 dopamine receptor availability, measured using Positron Emission Tomography. We registered response times of 30 older and 30 younger participants during an oddball discrimination task where rewards varied systematically between trials. The average reward rate had a similar impact on vigor in both age groups. There was a weak positive association between ventral striatal dopamine receptor availability and the effect of average reward rate on response time. Overall, the effect of reward on response vigor was similar in younger and older adults, and weakly correlated with dopamine D1 receptor availability.

Model-based representational similarity analysis of blood-oxygen-level-dependent fMRI captures threat learning in social interactions.

Past research has shown that attributions of intentions to other's actions determine how we experience these actions and their consequences. Yet, it is unknown how such attributions affect our learning and memory. Addressing this question, we combined neuroimaging with an interactive threat learning paradigm in which two interaction partners (confederates) made choices that had either threatening (shock) or safe (no shock) consequences for the participants. Importantly, participants were led to believe that one partner intentionally caused the delivery of shock, whereas the other did not (i.e. unintentional partner). Following intentional versus unintentional shocks, participants reported an inflated number of shocks and a greater increase in anger and vengeance. We applied a model-based representational similarity analysis to blood-oxygen-level-dependent (BOLD)-MRI patterns during learning. Surprisingly, we did not find any effects of intentionality. The threat value of actions, however, was represented as a trial-by-trial increase in representational similarity in the insula and the inferior frontal gyrus. Our findings illustrate how neural pattern formation can be used to study a complex interaction.

Role of dopamine and gray matter density in aging effects and individual differences of functional connectomes.

With increasing age, functional connectomes become dissimilar across normal individuals, reflecting heterogenous aging effects on functional connectivity (FC). We investigated the distribution of these effects across the connectome and their relationship with age-related differences in dopamine (DA) D1 receptor availability and gray matter density (GMD). With this aim, we determined aging effects on mean and interindividual variance of FC using fMRI in 30 younger and 30 older healthy subjects and mapped the contribution of each connection to the patterns of age-related similarity loss. Aging effects on mean FC accounted mainly for the dissimilarity between connectomes of younger and older adults, and were related, across brain regions, to aging effects on DA D1 receptor availability. Aging effects on the variance of FC indicated a global increase in variance with advancing age, explained connectome dissimilarity among older subjects and were related to aging effects on variance of GMD. The relationship between aging and the similarity of connectomes can thus be partly explained by age differences in DA modulation and gray matter structure.

Learning in anticipation of reward and punishment: perspectives across the human lifespan.

Learning to act to receive reward and to withhold to avoid punishment has been found to be easier than learning the opposite contingencies in young adults. To what extent this type of behavioral adaptation might develop during childhood and adolescence and differ during aging remains unclear. We therefore tested 247 healthy individuals across the human life span (7-80 years) with an orthogonalized valenced go/no-go learning task. Computational modeling revealed that peak performance in young adults was attributable to greater sensitivity to both reward and punishment. However, in children and adolescents, we observed an increased bias toward action but not reward sensitivity. By contrast, reduced learning in midlife and older adults was accompanied by decreased reward sensitivity and especially punishment sensitivity along with an age-related increase in the Pavlovian bias. These findings reveal distinct motivation-dependent learning capabilities across the human life span, which cannot be probed using conventional go/reward no-go/punishment style paradigms that have important implications in lifelong education.

Attenuation of dopamine-modulated prefrontal value signals underlies probabilistic reward learning deficits in old age.

Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability.

Acting without being in control: Exploring volition in Parkinson's disease with impulsive compulsive behaviours.

BACKGROUND: Several aspects of volitional control of action may be relevant in the pathophysiology of impulsive-compulsive behaviours (ICB) in Parkinson's disease (PD). We aimed to explore multiple aspects of action control, assessing reward-related behaviour, inhibition (externally and internally triggered) and sense of agency in PD patients, with and without ICB compared to healthy subjects. METHODS: Nineteen PD patients with ICB (PD-ICB), 19 PD without ICB (PD-no-ICB) and 19 healthy controls (HC) underwent a battery of tests including: Intentional Binding task which measures sense of agency; Stop Signal Reaction Time (SSRT) measuring capacity for reactive inhibition; the Marble task, assessing intentional inhibition; Balloon Analog Risk Task for reward sensitivity. RESULTS: One-way ANOVA showed significant main effect of group for action binding (p = 0.004, F = 6.27). Post hoc analysis revealed that PD-ICB had significantly stronger action binding than HC (p = 0.004), and PD-no-ICB (p = 0.04). There was no difference between PD-no-ICB and HC. SSRT did not differ between PD groups, whereas a significant difference between PD-no-ICB and HC was detected (p = 0.01). No other differences were found among groups in the other tasks. CONCLUSIONS: PD patients with ICB have abnormal performance on a psychophysical task assessing sense of agency, which might be related to a deficit in action representation at cognitive/experiential level. Yet, they have no deficit on tasks evaluating externally and internally triggered inhibitory control, or in reward-based decision-making. We conclude that impaired sense of agency may be a factor contributing to ICB in PD patients.

Salience-driven overestimation of total somatosensory stimulation.

Psychological characterisation of sensory systems often focusses on minimal units of perception, such as thresholds, acuity, selectivity and precision. Research on how these units are aggregated to create integrated, synthetic experiences is rarer. We investigated mechanisms of somatosensory integration by asking volunteers to judge the total intensity of stimuli delivered to two fingers simultaneously. Across four experiments, covering physiological pathways for tactile, cold and warm stimuli, we found that judgements of total intensity were particularly poor when the two simultaneous stimuli had different intensities. Total intensity of discrepant stimuli was systematically overestimated. This bias was absent when the two stimulated digits were on different hands. Taken together, our results showed that the weaker stimulus of a discrepant pair was not extinguished, but contributed less to the perception of the total than the stronger stimulus. Thus, perception of somatosensory totals is biased towards the most salient element. 'Peak' biases in human judgements are well-known, particularly in affective experience. We show that a similar mechanism also influences sensory experience.

Sharing a Context with Other Rewarding Events Increases the Probability that Neutral Events will be Recollected.

Although reward is known to enhance memory for reward-predicting events, the extent to which such memory effects spread to associated (neutral) events is unclear. Using a between-subject design, we examined how sharing a background context with rewarding events influenced memory for motivationally neutral events (tested after a 5 days delay). We found that sharing a visually rich context with rewarding objects during encoding increased the probability that neutral objects would be successfully recollected during memory test, as opposed to merely being recognized without any recall of associative detail. In contrast, such an effect was not seen when the context was not explicitly demarcated and objects were presented against a blank black background. These qualitative changes in memory were observed in the absence of any effects on overall recognition (as measured by d'). Additionally, a follow-up study failed to find any evidence to suggest that the mere presence of a context picture in the background during encoding (i.e., without the reward manipulation) produced any such qualitative changes in memory. These results suggest that reward enhances recollection for rewarding objects as well as other non-rewarding events that are representationally linked to the same context.

Oral somatosensory awareness.

Oral somatosensory awareness refers to the somatic sensations arising within the mouth, and to the information these sensations provide about the state and structure of the mouth itself, and objects in the mouth. Because the oral tissues have a strong somatosensory innervation, they are the locus of some of our most intense and vivid bodily experiences. The salient pain of toothache, or the habit of running one's tongue over one's teeth when someone mentions "dentist", provide two very different indications of the power of oral somatosensory awareness in human experience and behaviour. This paper aims to review the origins and structure of oral somatosensory awareness, focussing on quantitative, mechanistic studies in humans. We first extend a model of levels of bodily awareness to the specific case of the mouth. We then briefly summarise the sensory innervation of oral tissues, and their projections in the brain. We next describe how these peripheral inputs give rise to perceptions of objects in the mouth, such as foods, liquids and oral devices, and also of the mouth tissues themselves. Finally, we consider the concept of a conscious mouth image, and the somatosensory basis of "mouth feel". The theoretical framework outlined in this paper is intended to facilitate scientific studies of this important site of human experience.