Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex.

Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.

Working memory gating in obesity: Insights from a case-control fMRI study.

Computational models and neurophysiological data propose that a 'gating mechanism' coordinates distractor-resistant maintenance and flexible updating of working memory contents: While maintenance of information is mainly implemented in the prefrontal cortex, updating of information is signaled by phasic increases in dopamine in the striatum. Previous literature demonstrates structural and functional alterations in these brain areas, as well as differential dopamine transmission among individuals with obesity, suggesting potential impairments in these processes. To test this hypothesis, we conducted an observational case-control fMRI study, dividing participants into groups with and without obesity based on their BMI. We probed maintenance and updating of working memory contents using a modified delayed match to sample task and investigated the effects of SNPs related to the dopaminergic system. While the task elicited the anticipated brain responses, our findings revealed no evidence for group differences in these two processes, neither at the neural level nor behaviorally. However, depending on Taq1A genotype, which affects dopamine receptor density in the striatum, participants with obesity performed worse on the task. In conclusion, this study does not support the existence of overall obesity-related differences in working memory gating. Instead, we propose that potentially subtle alterations may manifest specifically in individuals with a 'vulnerable' genotype.

Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole.

Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.

Cortical Grey Matter Mediates Increases in Model-Based Control and Learning from Positive Feedback from Adolescence to Adulthood.

Cognition and brain structure undergo significant maturation from adolescence into adulthood. Model-based (MB) control is known to increase across development, which is mediated by cognitive abilities. Here, we asked two questions unaddressed in previous developmental studies. First, what are the brain structural correlates of age-related increases in MB control? Second, how are age-related increases in MB control from adolescence to adulthood influenced by motivational context? A human developmental sample (n = 103; age, 12-50, male/female, 55:48) completed structural MRI and an established task to capture MB control. The task was modified with respect to outcome valence by including (1) reward and punishment blocks to manipulate the motivational context and (2) an additional choice test to assess learning from positive versus negative feedback. After replicating that an age-dependent increase in MB control is mediated by cognitive abilities, we demonstrate first-time evidence that gray matter density (GMD) in the parietal cortex mediates the increase of MB control with age. Although motivational context did not relate to age-related changes in MB control, learning from positive feedback improved with age. Meanwhile, negative feedback learning showed no age effects. We present a first report that an age-related increase in positive feedback learning was mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. Our findings indicate that brain maturation, putatively reflected in lower GMD, in distinct and partially overlapping brain regions could lead to a more efficient brain organization and might thus be a key developmental step toward age-related increases in planning and value-based choice.SIGNIFICANCE STATEMENT Changes in model-based decision-making are paralleled by extensive maturation in cognition and brain structure across development. Still, to date the neuroanatomical underpinnings of these changes remain unclear. Here, we demonstrate for the first time that parietal GMD mediates age-dependent increases in model-based control. Age-related increases in positive feedback learning were mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. A manipulation of motivational context did not have an impact on age-related changes in model-based control. These findings highlight that brain maturation in distinct and overlapping cortical regions constitutes a key developmental step toward improved value-based choices.

Measuring self-regulation in everyday life: Reliability and validity of smartphone-based experiments in alcohol use disorder.

Self-regulation, the ability to guide behavior according to one's goals, plays an integral role in understanding loss of control over unwanted behaviors, for example in alcohol use disorder (AUD). Yet, experimental tasks that measure processes underlying self-regulation are not easy to deploy in contexts where such behaviors usually occur, namely outside the laboratory, and in clinical populations such as people with AUD. Moreover, lab-based tasks have been criticized for poor test-retest reliability and lack of construct validity. Smartphones can be used to deploy tasks in the field, but often require shorter versions of tasks, which may further decrease reliability. Here, we show that combining smartphone-based tasks with joint hierarchical modeling of longitudinal data can overcome at least some of these shortcomings. We test four short smartphone-based tasks outside the laboratory in a large sample (N = 488) of participants with AUD. Although task measures indeed have low reliability when data are analyzed traditionally by modeling each session separately, joint modeling of longitudinal data increases reliability to good and oftentimes excellent levels. We next test the measures' construct validity and show that extracted latent factors are indeed in line with theoretical accounts of cognitive control and decision-making. Finally, we demonstrate that a resulting cognitive control factor relates to a real-life measure of drinking behavior and yields stronger correlations than single measures based on traditional analyses. Our findings demonstrate how short, smartphone-based task measures, when analyzed with joint hierarchical modeling and latent factor analysis, can overcome frequently reported shortcomings of experimental tasks.

Sufficient reliability of the behavioral and computational readouts of a probabilistic reversal learning task.

Task-based measures that capture neurocognitive processes can help bridge the gap between brain and behavior. To transfer tasks to clinical application, reliability is a crucial benchmark because it imposes an upper bound to potential correlations with other variables (e.g., symptom or brain data). However, the reliability of many task readouts is low. In this study, we scrutinized the retest reliability of a probabilistic reversal learning task (PRLT) that is frequently used to characterize cognitive flexibility in psychiatric populations. We analyzed data from N = 40 healthy subjects, who completed the PRLT twice. We focused on how individual metrics are derived, i.e., whether data were partially pooled across participants and whether priors were used to inform estimates. We compared the reliability of the resulting indices across sessions, as well as the internal consistency of a selection of indices. We found good to excellent reliability for behavioral indices as derived from mixed-effects models that included data from both sessions. The internal consistency was good to excellent. For indices derived from computational modeling, we found excellent reliability when using hierarchical estimation with empirical priors and including data from both sessions. Our results indicate that the PRLT is well equipped to measure individual differences in cognitive flexibility in reinforcement learning. However, this depends heavily on hierarchical modeling of the longitudinal data (whether sessions are modeled separately or jointly), on estimation methods, and on the combination of parameters included in computational models. We discuss implications for the applicability of PRLT indices in psychiatric research and as diagnostic tools.

Loss of control over eating: A systematic review of task based research into impulsive and compulsive processes in binge eating.

Recurring episodes of excessive food intake in binge eating disorder can be understood through the lens of behavioral control systems: patients repeat maladaptive behaviors against their explicit intent. Self-report measures show enhanced impulsivity and compulsivity in binge eating (BE) but are agnostic as to the processes that might lead to impulsive and compulsive behavior in the moment. Task-based neurocognitive investigations can tap into those processes. In this systematic review, we synthesize neurocognitive research on behavioral impulsivity and compulsivity in BE in humans and animals, published between 2010-2020. Findings on impulsivity are heterogeneous. Findings on compulsivity are sparse but comparatively consistent, indicating an imbalance of goal-directed and habitual control as well as deficits in reversal learning. We urge researchers to address heterogeneity related to mood states and the temporal dynamics of symptoms, to systematically differentiate contributions of body weight and BE, and to ascertain the validity and reliability of tasks. Moreover, we propose to further scrutinize the compulsivity findings to unravel the computational mechanisms of a potential reinforcement learning deficit.

Multi-echo fMRI, resting-state connectivity, and high psychometric schizotypy.

Disrupted striatal functional connectivity is proposed to play a critical role in the development of psychotic symptoms. Previous resting-state functional magnetic resonance imaging (rs-fMRI) studies typically reported disrupted striatal connectivity in patients with psychosis and in individuals at clinical and genetic high risk of the disorder relative to healthy controls. This has not been widely studied in healthy individuals with subclinical psychotic-like experiences (schizotypy). Here we applied the emerging technology of multi-echo rs-fMRI to examine corticostriatal connectivity in this group, which is thought to drastically maximize physiological noise removal and increase BOLD contrast-to-noise ratio. Multi-echo rs-fMRI data (echo times, 12, 28, 44, 60 ms) were acquired from healthy individuals with low (LS, n = 20) and high (HS, n = 19) positive schizotypy as determined with the Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE). After preprocessing to ensure optimal contrast and removal of non-BOLD signal components, whole-brain functional connectivity from six striatal seeds was compared between the HS and LS groups. Effects were considered significant at cluster-level p < .05 family-wise error correction. Compared to LS, HS subjects showed lower rs-fMRI connectivity between ventromedial prefrontal regions and ventral striatal regions. Lower connectivity was also observed between the dorsal putamen and the hippocampus, occipital regions, as well as the cerebellum. These results demonstrate that subclinical positive psychotic-like experiences in healthy individuals are associated with striatal hypoconnectivity as detected using multi-echo rs-fMRI. Further application of this approach may aid in characterizing functional connectivity abnormalities across the extended psychosis phenotype.