Observational reinforcement learning in children and young adults.

Observational learning is essential for the acquisition of new behavior in educational practices and daily life and serves as an important mechanism for human cognitive and social-emotional development. However, we know little about its underlying neurocomputational mechanisms from a developmental perspective. In this study we used model-based fMRI to investigate differences in observational learning and individual learning between children and younger adults. Prediction errors (PE), the difference between experienced and predicted outcomes, related positively to striatal and ventral medial prefrontal cortex activation during individual learning and showed no age-related differences. PE-related activation during observational learning was more pronounced when outcomes were worse than predicted. Particularly, negative PE-coding in the dorsal medial prefrontal cortex was stronger in adults compared to children and was associated with improved observational learning in children and adults. The current findings pave the way to better understand observational learning challenges across development and educational settings.

Diminished State Space Theory of Human Aging.

Many new technologies, such as smartphones, computers, or public-access systems (like ticket-vending machines), are a challenge for older adults. One feature that these technologies have in common is that they involve underlying, partially observable, structures (state spaces) that determine the actions that are necessary to reach a certain goal (e.g., to move from one menu to another, to change a function, or to activate a new service). In this work we provide a theoretical, neurocomputational account to explain these behavioral difficulties in older adults. Based on recent findings from age-comparative computational- and cognitive-neuroscience studies, we propose that age-related impairments in complex goal-directed behavior result from an underlying deficit in the representation of state spaces of cognitive tasks. Furthermore, we suggest that these age-related deficits in adaptive decision-making are due to impoverished neural representations in the orbitofrontal cortex and hippocampus.

Neural evidence for age-related deficits in the representation of state spaces.

Under high cognitive demands, older adults tend to resort to simpler, habitual, or model-free decision strategies. This age-related shift in decision behavior has been attributed to deficits in the representation of the cognitive maps, or state spaces, necessary for more complex model-based decision-making. Yet, the neural mechanisms behind this shift remain unclear. In this study, we used a modified 2-stage Markov task in combination with computational modeling and single-trial EEG analyses to establish neural markers of age-related changes in goal-directed decision-making under different demands on the representation of state spaces. Our results reveal that the shift to simpler decision strategies in older adults is due to (i) impairments in the representation of the transition structure of the task and (ii) a diminished signaling of the reward value associated with decision options. In line with the diminished state space hypothesis of human aging, our findings suggest that deficits in goal-directed, model-based behavior in older adults result from impairments in the representation of state spaces of cognitive tasks.

Human ageing is associated with more rigid concept spaces.

Prevalence-induced concept change describes a cognitive mechanism by which someone's definition of a concept shifts as the prevalence of instances of that concept changes. While this phenomenon has been established in young adults, it is unclear how it affects older adults. In this study, we explore how prevalence-induced concept change affects older adults' lower-level, perceptual, and higher-order, ethical judgements. We find that older adults are less sensitive to prevalence-induced concept change than younger adults across both domains. Using computational modeling, we demonstrate that these age-related changes in judgements reflect more cautious and deliberate responding in older adults. Based on these findings, we argue that while overly cautious responding by older adults may be maladaptive in some cognitive domains, in the case of prevalence-induced concept change, it might be protective against biased judgements.

Changes in the Prevalence of Thin Bodies Bias Young Women's Judgments About Body Size.

Body dissatisfaction is pervasive among young women in Western countries. Among the many forces that contribute to body dissatisfaction, the overrepresentation of thin bodies in visual media has received notable attention. In this study, we proposed that prevalence-induced concept change may be one of the cognitive mechanisms that explain how beauty standards shift. We conducted a preregistered online experiment with young women (N = 419) and found that when the prevalence of thin bodies in the environment increased, the concept of being overweight expanded to include bodies that would otherwise be judged as "normal." Exploratory analyses revealed significant individual differences in sensitivity to this effect, in terms of women's judgments about other bodies as well as their own. These results suggest that women's judgments about other women's bodies are biased by an overrepresentation of thinness and lend initial support to policies designed to increase size-inclusive representation in the media.

Need for cognition does not account for individual differences in metacontrol of decision making.

Humans show metacontrol of decision making, that is they adapt their reliance on decision-making strategies toward situational differences such as differences in reward magnitude. Specifically, when higher rewards are at stake, individuals increase reliance on a more accurate but cognitively effortful strategy. We investigated whether the personality trait Need for Cognition (NFC) explains individual differences in metacontrol. Based on findings of cognitive effort expenditure in executive functions, we expected more metacontrol in individuals low in NFC. In two independent studies, metacontrol was assessed by means of a decision-making task that dissociates different reinforcement-learning strategies and in which reward magnitude was manipulated across trials. In contrast to our expectations, NFC did not account for individual differences in metacontrol of decision making. In fact, a Bayesian analysis provided moderate to strong evidence against a relationship between NFC and metacontrol. Beyond this, there was no consistent evidence for relationship between NFC and overall model-based decision making. These findings show that the effect of rewards on the engagement of effortful decision-making strategies is largely independent of the intrinsic motivation for engaging in cognitively effortful tasks and suggest a differential role of NFC for the regulation of cognitive effort in decision making and executive functions.

Valence bias in metacontrol of decision making in adolescents and young adults.

The development of metacontrol of decision making and its susceptibility to framing effects were investigated in a sample of 201 adolescents and adults in Germany (12-25 years, 111 female, ethnicity not recorded). In a task that dissociates model-free and model-based decision making, outcome magnitude and outcome valence were manipulated. Both adolescents and adults showed metacontrol and metacontrol tended to increase across adolescence. Furthermore, model-based decision making was more pronounced for loss compared to gain frames but there was no evidence that this framing effect differed with age. Thus, the strategic adaptation of decision making continues to develop into young adulthood and for both adolescents and adults, losses increase the motivation to invest cognitive resources into an effortful decision-making strategy.

Seizing the opportunity: Lifespan differences in the effects of the opportunity cost of time on cognitive control.

Previous work suggests that lifespan developmental differences in cognitive control reflect maturational and aging-related changes in prefrontal cortex functioning. However, complementary explanations exist: It could be that children and older adults differ from younger adults in how they balance the effort of engaging in control against its potential benefits. Here we test whether the degree of cognitive effort expenditure depends on the opportunity cost of time (average reward rate per unit time): if the average reward rate is high, participants should withhold cognitive effort whereas if it is low, they should invest more. In Experiment 1, we examine this hypothesis in children, adolescents, younger, and older adults, by applying a reward rate manipulation in two cognitive control tasks: a modified Erikson Flanker and a task-switching paradigm. We found that young adults and adolescents reflexively withheld effort when the opportunity cost of time was high, whereas older adults and, to a lesser degree children, invested more resources to accumulate reward as quickly as possible. We tentatively interpret these results in terms of age- and task-specific differences in the processing of the opportunity cost of time. We qualify our findings in a second experiment in younger adults in which we address an alternative explanation of our results and show that the observed age differences in effort expenditure may not result from differences in task difficulty. To conclude, we think that our results present an interesting first step at relating opportunity costs to motivational processes across the lifespan. We frame the implications of further work in this area within a recent developmental model of resource-rationality, which points to developmental sweet spots in cognitive control.

Meta-control: From psychology to computational neuroscience.

Research in the past decades shed light on the different mechanisms that underlie our capacity for cognitive control. However, the meta-level processes that regulate cognitive control itself remain poorly understood. Following the terminology from artificial intelligence, meta-control can be defined as a collection of mechanisms that (a) monitor the progress of controlled processing and (b) regulate the underlying control parameters in the service of current task goals and in response to internal or external constraints. From a psychological perspective, meta-control is an important concept because it may help explain and predict how and when human agents select different types of behavioral strategies. From a cognitive neuroscience viewpoint, meta-control is a useful concept for understanding the complex networks in the prefrontal cortex that guide higher-level behavior as well as their interactions with neuromodulatory systems (such as the dopamine or norepinephrine system). The purpose of the special issue is to integrate hitherto segregated strands of research across three different perspectives: 1) a psychological perspective that specifies meta-control processes on a functional level and aims to operationalize them in experimental tasks; 2) a computational perspective that builds on ideas from artificial intelligence to formalize normative solutions to meta-control problems; and 3) a cognitive neuroscience perspective that identifies neural correlates of and mechanisms underlying meta-control.

Human aging alters social inference about others' changing intentions.

Decoding others' intentions accurately in order to adapt one's own behavior is pivotal throughout life. In this study, we asked how younger and older adults deal with uncertainty in dynamic social environments. We used an advice-taking paradigm together with Bayesian modeling to characterize effects of aging on learning about others' time-varying intentions. We observed age differences when comparing learning on two levels of social uncertainty: the fidelity of the adviser and the volatility of intentions. Older adults expected the adviser to change his/her intentions more frequently (i.e., a higher volatility of the adviser). They also showed higher confidence (i.e., precision) in their volatility beliefs and were less willing to change their beliefs about volatility over the course of the experiment. This led them to update their predictions about the fidelity of the adviser more quickly. Potentially indicative of stereotype effects, we observed that older advisers were perceived as more volatile, but also more faithful than younger advisers. This offers new insights into adult age differences in response to social uncertainty.

Resource-rational approach to meta-control problems across the lifespan.

Over the last decade, research on cognitive control and decision-making has revealed that individuals weigh the costs and benefits of engaging in or refraining from control and that whether and how they engage in these cost-benefit analyses may change across development and during healthy aging. In the present article, we examine how lifespan age differences in cognitive abilities affect the meta-control of behavioral strategies across the lifespan and how motivation affects these trade-offs. Based on accumulated evidence, we highlight two hypotheses that may explain the existing results better than current models. In contrast to previous theoretical accounts, we assume that age differences in the engagement in cost-benefit trade-offs reflect a resource-rational adaptation to internal and external constraints that arise across the lifespan. This article is categorized under: Psychology > Development and Aging Psychology > Reasoning and Decision Making.

Metacontrol of decision-making strategies in human aging.

Humans employ different strategies when making decisions. Previous research has reported reduced reliance on model-based strategies with aging, but it remains unclear whether this is due to cognitive or motivational factors. Moreover, it is not clear how aging affects the metacontrol of decision making, that is the dynamic adaptation of decision-making strategies to varying situational demands. In this cross-sectional study, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based strategies. In contrast to previous research, model-based strategies led to higher payoffs. Moreover, we manipulated the costs and benefits of model-based strategies by varying reward magnitude and the stability of the task structure. Compared to younger adults, older adults showed reduced model-based decision making and less adaptation of decision-making strategies. Our findings suggest that aging affects the metacontrol of decision-making strategies and that reduced model-based strategies in older adults are due to limited cognitive abilities.

Risk contagion by peers affects learning and decision-making in adolescents.

Adolescence is a period of life in which social influences-particularly if they come from peers-play a critical role in shaping learning and decision preferences. Recent studies in adults show evidence of a risk contagion effect; that is, individual risk preferences are modulated by observing and learning from others' risk-related decisions. In this study, using choice data and computational modeling, we demonstrate stronger risk contagion in male adolescents when observing peers compared to nonpeers. This effect was only present when the observed peer showed risk-seeking preferences. Moreover, adolescents represented the peers' decisions better than those of adults. Intriguingly, the degree of peer-biased risk contagion in adolescents was positively associated with real-life social integration. Contrary to previous accounts, our data suggest that peer conformity during risky decision-making in adolescence is a socially motivated, deliberative process. Susceptibility to peer influence in adolescence might be adaptive, associated with higher degrees of social functioning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

L-DOPA reduces model-free control of behavior by attenuating the transfer of value to action.

Dopamine is a key neurotransmitter in action control. However, influential theories of dopamine function make conflicting predictions about the effect of boosting dopamine neurotransmission. Here, we tested if increases in dopamine tone by administration of L-DOPA upregulate reward learning as predicted by reinforcement learning theories, and if increases are specific for deliberative "model-based" control or reflexive "model-free" control. Alternatively, L-DOPA may impair learning as suggested by "value" or "thrift" theories of dopamine. To this end, we employed a two-stage Markov decision-task to investigate the effect of L-DOPA (randomized cross-over) on behavioral control while brain activation was measured using fMRI. L-DOPA led to attenuated model-free control of behavior as indicated by the reduced impact of reward on choice. Increased model-based control was only observed in participants with high working memory capacity. Furthermore, L-DOPA facilitated exploratory behavior, particularly after a stream of wins in the task. Correspondingly, in the brain, L-DOPA decreased the effect of reward at the outcome stage and when the next decision had to be made. Critically, reward-learning rates and prediction error signals were unaffected by L-DOPA, indicating that differences in behavior and brain response to reward were not driven by differences in learning. Taken together, our results suggest that L-DOPA reduces model-free control of behavior by attenuating the transfer of value to action. These findings provide support for the value and thrift accounts of dopamine and call for a refined integration of valuation and action signals in reinforcement learning models.

Developmental differences in the neural dynamics of observational learning.

Learning from vicarious experience is central for educational practice, but not well understood with respect to its ontogenetic development and underlying neural dynamics. In this age-comparative study we compared behavioral and electrophysiological markers of learning from vicarious and one's own experience in children (age 8-10) and young adults. Behaviorally both groups benefitted from integrating vicarious experience into their own choices however, adults learned much faster from social information than children. The electrophysiological results show learning-related changes in the P300 to experienced and observed rewards in adults, but not in children, indicating that adults were more efficient in integrating observed and experienced information during learning. In comparison to adults, children showed an enhanced FRN for observed and experienced feedback, indicating that they focus more on valence information than adults. Taken together, children compared to adults seem to be less able to rapidly assess the informational value of observed and experienced feedback during learning and consequently up-regulate their response to both, observed and experienced (particularly negative) feedback. When transferring the current findings to an applied context, educational practice should strengthen children's ability to use feedback information for learning and be particularly cautious with negative social feedback during supervised learning.

Repetitive transcranial magnetic stimulation over dorsolateral prefrontal cortex modulates value-based learning during sequential decision-making.

Adaptive behavior in daily life often requires the ability to acquire and represent sequential contingencies between actions and the associated outcomes. Although accumulating evidence implicates the role of dorsolateral prefrontal cortex (dlPFC) in complex value-based learning and decision-making, direct evidence for involvements of this region in integrating information across sequential decision states is still scarce. Using a 3-stage deterministic Markov decision task, here we applied offline, inhibitory low-frequency 1-Hz repetitive transcranial magnetic stimulation (rTMS) over the left dlPFC in young male adults (n = 31, mean age = 23.8 years, SD = 2.5 years) in a within-subject cross-over design to study the roles of this region in influencing value-based sequential decision-making. In two separate sessions, each participant received 1-Hz rTMS stimulation either over the left dlPFC or over the vertex. The results showed that transiently inhibiting the left dlPFC impaired choice accuracy, particularly in situations in which the acquisition of sequential transitions between decision states and temporally lagged action-outcome contingencies played a greater role. Estimating parameters of a diffusion model from behavioral choices, we found that the diffusion drift rate, which reflects the efficiency of information integration, was attenuated by the stimulation. Moreover, the effects of rTMS interacted with session: individuals who could not efficiently integrate information across sequential states in the first session due to disrupted dlPFC function also could not catch up in performance during the second session with those individuals who could learn sequential transitions with intact dlPFC function in the first session. Taken together, our findings suggest that the left dlPFC is crucially involved in the acquisition of complex sequential relations and in the potential of such learning.

Computational neuroscience across the lifespan: Promises and pitfalls.

In recent years, the application of computational modeling in studies on age-related changes in decision making and learning has gained in popularity. One advantage of computational models is that they provide access to latent variables that cannot be directly observed from behavior. In combination with experimental manipulations, these latent variables can help to test hypotheses about age-related changes in behavioral and neurobiological measures at a level of specificity that is not achievable with descriptive analysis approaches alone. This level of specificity can in turn be beneficial to establish the identity of the corresponding behavioral and neurobiological mechanisms. In this paper, we will illustrate applications of computational methods using examples of lifespan research on risk taking, strategy selection and reinforcement learning. We will elaborate on problems that can occur when computational neuroscience methods are applied to data of different age groups. Finally, we will discuss potential targets for future applications and outline general shortcomings of computational neuroscience methods for research on human lifespan development.

Age Differences in the Neural Mechanisms of Intertemporal Choice Under Subjective Decision Conflict.

Older decision-makers may capitalize on their greater experiences in financial decisions and by this offset decline in cognitive abilities. However, this pattern of results should reverse in situations that place high demands on cognitive control functions. In this study, we investigated how decision conflict affects the neural mechanisms of intertemporal decision-making in younger and older adults. To individually adjust the level of decision conflict we determined the indifference point (IDP) in intertemporal decision-making for each participant. During functional magnetic resonance imaging, participants performed choice options close to their IDP (high conflict) or far away from the IDP (low conflict). In younger adults, decision conflict leads to reduced delay discounting and lower discount rates are associated with higher working memory (WM) capacity. In older adults, high decision conflict is associated with enhanced discounting, hypoactivation in the ventral striatum as well diminished ventral striatal representations of differences in subjective values. Taken together, our results show that under enhanced decision conflict, younger adults engage in a more reflective decision mode that reflects individual differences in WM capacity. In contrast, older adults get more present-oriented under high demands on cognitive control and this decision bias is associated with changes in striatal value signaling.

Developmental Changes in Learning: Computational Mechanisms and Social Influences.

Our ability to learn from the outcomes of our actions and to adapt our decisions accordingly changes over the course of the human lifespan. In recent years, there has been an increasing interest in using computational models to understand developmental changes in learning and decision-making. Moreover, extensions of these models are currently applied to study socio-emotional influences on learning in different age groups, a topic that is of great relevance for applications in education and health psychology. In this article, we aim to provide an introduction to basic ideas underlying computational models of reinforcement learning and focus on parameters and model variants that might be of interest to developmental scientists. We then highlight recent attempts to use reinforcement learning models to study the influence of social information on learning across development. The aim of this review is to illustrate how computational models can be applied in developmental science, what they can add to our understanding of developmental mechanisms and how they can be used to bridge the gap between psychological and neurobiological theories of development.

The Aging of the Social Mind - Differential Effects on Components of Social Understanding.

Research in younger adults dissociates cognitive from affective facets of social information processing, rather than promoting a monolithic view of social intelligence. An influential theory on adult development suggests differential effects of aging on cognitive and affective functions. However, this dissociation has not been directly tested in the social domain. Employing a newly developed naturalistic paradigm that disentangles facets of the social mind within an individual, we show multi-directionality of age-related differences. Specifically, components of the socio-cognitive route - Theory of Mind and metacognition - are impaired in older relative to younger adults. Nevertheless, these social capacities are still less affected by aging than factual reasoning and metacognition regarding non-social content. Importantly, the socio-affective route is well-functioning, with no decline in empathy and elevated compassion in the elderly. These findings contribute to an integrated theory of age-related change in social functioning and inform interventions tailored to specifically reinstate socio-cognitive skills in old age.