Expecting the unexpected: a review of learning under uncertainty across development.

Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.

Age differences in intertemporal choice among children, adolescents, and adults.

When choosing between sooner-smaller and later-larger rewards (i.e., intertemporal choices), adults typically prefer later-larger rewards more often than children. Intertemporal choice preferences have been implicated in various impulsivity-related psychopathologies, making it important to understand the underlying mechanisms not only in terms of how reward magnitude and delay affect choice but also in terms of how these mechanisms develop across age. We administered an intertemporal choice paradigm to 60 children (8-11 years), 79 adolescents (14-16 years), and 60 young adults (18-23 years). The paradigm systematically varied amounts and delays of the available rewards, allowing us to identify mechanisms underlying age-related differences in patience. Compared with young adults, both children and adolescents made fewer later-larger choices. In terms of underlying mechanisms, variation in delays, absolute reward magnitudes, and relative amount differences affected choice in each age group, indicating that children showed sensitivity to the same choice-relevant factors as young adults. Sensitivity to both absolute reward magnitude and relative amount differences showed a further monotonic age-related increase, whereas no change in delay sensitivity occurred. Lastly, adolescents and young adults weakly displayed a present bias (i.e., overvaluing immediate vs. future rewards; nonsignificant and trend, respectively), whereas children showed a nonsignificant but opposite pattern, possibly indicating that specifically dealing with future rewards changed with age. These findings shed light on the underlying mechanisms that contribute to the development of patience. By decomposing overt choices, our results suggest that the age-related increase in patience may be driven specifically by stronger sensitivity to amount differences with age.

Longitudinal changes in DLPFC activation during childhood are related to decreased aggression following social rejection.

Regulating aggression after social feedback is an important prerequisite for developing and maintaining social relations, especially in the current times with larger emphasis on online social evaluation. Studies in adults highlighted the role of the dorsolateral prefrontal cortex (DLPFC) in regulating aggression. Little is known about the development of aggression regulation following social feedback during childhood, while this is an important period for both brain maturation and social relations. The current study used a longitudinal design, with 456 twins undergoing two functional MRI sessions across the transition from middle (7 to 9 y) to late (9 to 11 y) childhood. Aggression regulation was studied using the Social Network Aggression Task. Behavioral aggression after social evaluation decreased over time, whereas activation in the insula, dorsomedial PFC and DLPFC increased over time. Brain-behavior analyses showed that increased DLPFC activation after negative feedback was associated with decreased aggression. Change analyses further revealed that children with larger increases in DLPFC activity from middle to late childhood showed stronger decreases in aggression over time. These findings provide insights into the development of social evaluation sensitivity and aggression control in childhood.

Parental Support and Positive Mood Buffer Adolescents' Academic Motivation During the COVID-19 Pandemic.

School closures during the first COVID-19 lockdown in 2020 severely disrupted adolescents' lives. We used a daily diary method for 20 days, including online and physical school days, assessing daily mood, social support and conflict, and academic motivation in 102 adolescents aged 12-16 years. We found that adolescents' academic motivation was lower on online compared with physical school days. In general, positive mood was positively associated with academic motivation, and friend conflict related negatively to academic motivation. Moreover, lower levels of parental support were related to lower academic motivation on online versus physical school days. Overall, these findings identified some critical changes in adolescents' daily experiences during the COVID-19 school closure and social-emotional factors that may buffer decreases in adolescents' academic motivation.

Behavioral and Neural Pathways Supporting the Development of Prosocial and Risk-Taking Behavior Across Adolescence.

This study tested the pathways supporting adolescent development of prosocial and rebellious behavior. Self-report and structural brain development data were obtained in a three-wave, longitudinal neuroimaging study (8-29 years, N = 210 at Wave 3). First, prosocial and rebellious behavior assessed at Wave 3 were positively correlated. Perspective taking and intention to comfort uniquely predicted prosocial behavior, whereas fun seeking (current levels and longitudinal changes) predicted both prosocial and rebellious behaviors. These changes were accompanied by developmental declines in nucleus accumbens and medial prefrontal cortex (MPFC) volumes, but only faster decline of MPFC (faster maturity) related to less rebellious behavior. These findings point toward a possible differential susceptibility marker, fun seeking, as a predictor of both prosocial and rebellious developmental outcomes.

Developmental Maturation of the Precuneus as a Functional Core of the Default Mode Network.

Efforts to map the functional architecture of the developing human brain have shown that connectivity between and within functional neural networks changes from childhood to adulthood. Although prior work has established that the adult precuneus distinctively modifies its connectivity during task versus rest states [Utevsky, A. V., Smith, D. V., & Huettel, S. A. Precuneus is a functional core of the default-mode network. Journal of Neuroscience, 34, 932-940, 2014], it remains unknown how these connectivity patterns emerge over development. Here, we use fMRI data collected at two longitudinal time points from over 250 participants between the ages of 8 and 26 years engaging in two cognitive tasks and a resting-state scan. By applying independent component analysis to both task and rest data, we identified three canonical networks of interest-the rest-based default mode network and the task-based left and right frontoparietal networks (LFPN and RFPN, respectively)-which we explored for developmental changes using dual regression analyses. We found systematic state-dependent functional connectivity in the precuneus, such that engaging in a task (compared with rest) resulted in greater precuneus-LFPN and precuneus-RFPN connectivity, whereas being at rest (compared with task) resulted in greater precuneus-default mode network connectivity. These cross-sectional results replicated across both tasks and at both developmental time points. Finally, we used longitudinal mixed models to show that the degree to which precuneus distinguishes between task and rest states increases with age, due to age-related increasing segregation between precuneus and LFPN at rest. Our results highlight the distinct role of the precuneus in tracking processing state, in a manner that is both present throughout and strengthened across development.

Are individual differences quantitative or qualitative? An integrated behavioral and fMRI MIMIC approach.

In cognitive neuroscience there is a growing interest in individual differences. We propose the Multiple Indicators Multiple Causes (MIMIC) model of combined behavioral and fMRI data to determine whether such differences are quantitative or qualitative in nature. A simulation study revealed the MIMIC model to have adequate power for this goal, and parameter recovery to be satisfactory. The MIMIC model was illustrated with a re-analysis of Van Duijvenvoorde et al. (2016) and Blankenstein et al. (2018) decision making data. This showed individual differences in Van Duijvenvoorde et al. (2016) to originate in qualitative differences in decision strategies. Parameters indicated some individuals to use an expected value decision strategy, while others used a loss minimizing strategy, distinguished by individual differences in vmPFC activity. Individual differences in Blankenstein et al. (2018) were explained by quantitative differences in risk aversion. Parameters showed that more risk averse individuals preferred safe over risky choices, as predicted by heightened vmPFC activity. We advocate using the MIMIC model to empirically determine, rather than assume, the nature of individual differences in combined behavioral and fMRI datasets.

A three-wave longitudinal study of subcortical-cortical resting-state connectivity in adolescence: Testing age- and puberty-related changes.

Adolescence is the transitional period between childhood and adulthood, characterized by substantial changes in reward-driven behavior. Although reward-driven behavior is supported by subcortical-medial prefrontal cortex (PFC) connectivity, the development of these circuits is not well understood. Particularly, while puberty has been hypothesized to accelerate organization and activation of functional neural circuits, the relationship between age, sex, pubertal change, and functional connectivity has hardly been studied. Here, we present an analysis of resting-state functional connectivity between subcortical structures and the medial PFC, in 661 scans of 273 participants between 8 and 29 years, using a three-wave longitudinal design. Generalized additive mixed model procedures were used to assess the effects of age, sex, and self-reported pubertal status on connectivity between subcortical structures (nucleus accumbens, caudate, putamen, hippocampus, and amygdala) and cortical medial structures (dorsal anterior cingulate, ventral anterior cingulate, subcallosal cortex, frontal medial cortex). We observed an age-related strengthening of subcortico-subcortical and cortico-cortical connectivity. Subcortical-cortical connectivity, such as, between the nucleus accumbens-frontal medial cortex, and the caudate-dorsal anterior cingulate cortex, however, weakened across age. Model-based comparisons revealed that for specific connections pubertal development described developmental change better than chronological age. This was particularly the case for changes in subcortical-cortical connectivity and distinctively for boys and girls. Together, these findings indicate changes in functional network strengthening with pubertal development. These changes in functional connectivity may maximize the neural efficiency of interregional communication and set the stage for further inquiry of biological factors driving adolescent functional connectivity changes.

Neural tracking of subjective value under riskand ambiguity in adolescence.

Although many neuroimaging studies on adolescent risk taking have focused on brain activation during outcome valuation, less attention has been paid to the neural correlates of choice valuation. Subjective choice valuation may be particularly influenced by whether a choice presents risk (known probabilities) or ambiguity (unknown probabilities), which has rarely been studied in developmental samples. Therefore, we examined the neural tracking of subjective value during choice under risk and ambiguity in a large sample of adolescents (N = 188, 12-22 years). Specifically, we investigated which brain regions tracked subjective value coding under risk and ambiguity. A model-based approach to estimate individuals' risk and ambiguity attitudes showed prominent variation in individuals' aversions to risk and ambiguity. Furthermore, participants subjectively experienced the ambiguous options as being riskier than the risky options. Subjective value tracking under risk was coded by activation in ventral striatum and superior parietal cortex. Subjective value tracking under ambiguity was coded by dorsolateral prefrontal cortex (PFC) and superior temporal gyrus activation. Finally, overlapping activation in the dorsomedial PFC was observed for subjective value under both conditions. Overall, this is the first study to chart brain activation patterns for subjective choice valuation under risk and ambiguity in an adolescent sample, which shows that the building blocks for risk and ambiguity processing are already present in early adolescence. Finally, we highlight the potential of combining behavioral modeling with fMRI for investigating choice valuation in adolescence, which may ultimately aid in understanding who takes risks and why.

Heritability of aggression following social evaluation in middle childhood: An fMRI study.

Middle childhood marks an important phase for developing and maintaining social relations. At the same time, this phase is marked by a gap in our knowledge of the genetic and environmental influences on brain responses to social feedback and their relation to behavioral aggression. In a large developmental twin sample (509 7- to 9-year-olds), the heritability and neural underpinnings of behavioral aggression following social evaluation were investigated, using the Social Network Aggression Task (SNAT). Participants viewed pictures of peers that gave positive, neutral, or negative feedback to the participant's profile. Next, participants could blast a loud noise toward the peer as an index of aggression. Genetic modeling revealed that aggression following negative feedback was influenced by both genetics and environmental (shared as well as unique environment). On a neural level (n = 385), the anterior insula and anterior cingulate cortex gyrus (ACCg) responded to both positive and negative feedback, suggesting they signal for social salience cues. The medial prefrontal cortex (mPFC) and inferior frontal gyrus (IFG) were specifically activated during negative feedback, whereas positive feedback resulted in increased activation in caudate, supplementary motor cortex (SMA), and dorsolateral prefrontal cortex (DLPFC). Decreased SMA and DLPFC activation during negative feedback was associated with more aggressive behavior after negative feedback. Moreover, genetic modeling showed that 13%-14% of the variance in dorsolateral PFC activity was explained by genetics. Our results suggest that the processing of social feedback is partly explained by genetic factors, whereas shared environmental influences play a role in behavioral aggression following feedback.

Frontostriatal White Matter Integrity Predicts Development of Delay of Gratification: A Longitudinal Study.

The ability to delay gratification increases considerably across development. Here, we test the hypothesis that this impulse control capacity is driven by increased maturation of frontostriatal circuitry using a fiber-tracking approach combined with longitudinal imaging. In total, 192 healthy volunteers between 8 and 26 years underwent diffusion tensor imaging scanning and completed a delay-discounting task twice, separated by a 2-year interval. We investigated dynamic associations between frontostriatal white matter (WM) integrity and delay of gratification skills. Moreover, we examined the predictive value of frontostriatal WM integrity for future delay of gratification skills. Results showed that delay discounting increases with age in a quadratic fashion, with greatest patience during late adolescence. Data also indicated nonlinear development of frontostriatal WM, with relative fast development during childhood and early adulthood and--on average--little change during mid-adolescence. Furthermore, the positive association between age and delay discounting was further increased in individuals with higher WM integrity of the frontostriatal tracts. Predictive analysis showed that frontostriatal WM development explained unique variance in current and future delay of gratification skills. This study adds to a descriptive relation between WM integrity and delay of gratification by showing that maturation of frontostriatal connectivity predicts changes in delay of gratification skills. These findings have implications for studies examining deviances in impulse control by showing that the developmental path between striatum and prefrontal cortex may be an important predictor for when development goes astray. SIGNIFICANCE STATEMENT: During the transition from childhood to adulthood, individuals generally show increased patience and become better in delaying gratification. The exact neural correlates of delay of gratification, however, remain poorly understood. By measuring both frontostriatal white matter (WM) integrity and delay of gratification skills at two time points, we were able to provide links for our understanding of the neural mechanisms underlying this type of impulse regulation capacity. We demonstrate that the ability to delay gratification improves between childhood and young adulthood and this improvement is predicted by the integrity of frontostriatal WM connections. This study adds to a descriptive relation between WM quality and delay of gratification by showing that maturation of frontostriatal connectivity predicts improvements in delay of gratification skills.

Neural Mechanisms Underlying Risk and Ambiguity Attitudes.

Individual differences in attitudes to risk (a taste for risk, known probabilities) and ambiguity (a tolerance for uncertainty, unknown probabilities) differentially influence risky decision-making. However, it is not well understood whether risk and ambiguity are coded differently within individuals. Here, we tested whether individual differences in risk and ambiguity attitudes were reflected in distinct neural correlates during choice and outcome processing of risky and ambiguous gambles. To these ends, we developed a neuroimaging task in which participants ( n = 50) chose between a sure gain and a gamble, which was either risky or ambiguous, and presented decision outcomes (gains, no gains). From a separate task in which the amount, probability, and ambiguity level were varied, we estimated individuals' risk and ambiguity attitudes. Although there was pronounced neural overlap between risky and ambiguous gambling in a network typically related to decision-making under uncertainty, relatively more risk-seeking attitudes were associated with increased activation in valuation regions of the brain (medial and lateral OFC), whereas relatively more ambiguity-seeking attitudes were related to temporal cortex activation. In addition, although striatum activation was observed during reward processing irrespective of a prior risky or ambiguous gamble, reward processing after an ambiguous gamble resulted in enhanced dorsomedial PFC activation, possibly functioning as a general signal of uncertainty coding. These findings suggest that different neural mechanisms reflect individual differences in risk and ambiguity attitudes and that risk and ambiguity may impact overt risk-taking behavior in different ways.

Amygdala-orbitofrontal connectivity predicts alcohol use two years later: a longitudinal neuroimaging study on alcohol use in adolescence.

This study tested the relation between cortical-subcortical functional connectivity and alcohol consumption in adolescents using an accelerated longitudinal design, as well as normative developmental patterns for these measures. Participants between ages 8 and 27 completed resting-state neuroimaging scans at two time points separated by two years (N = 274 at T1, N = 231 at T2). In addition, participants between ages 12 and 27 reported on recent and lifetime alcohol use (N = 193 at T1, N = 244 at T2). Resting-state connectivity analyses focused on amygdala-orbitofrontal connectivity given prior research linking reduced coupling between these regions to alcohol use. Mixed model analyses revealed that age had a cubic relationship with alcohol use, with little to no use in childhood, steep increases in adolescence and leveling off in adulthood. No age effects were found for amygdala-OFC connectivity. Prediction analyses showed that left amygdala-orbitofrontal connectivity at the first time point predicted recent and lifetime alcohol use two years later. There was no evidence for the reversed relation, suggesting that brain connectivity measures precede explorative risk-taking behavior in adolescence, possibly because decreased subcortical-frontal connectivity biases towards more explorative or risky behavior.

Longitudinal changes in adolescent risk-taking: a comprehensive study of neural responses to rewards, pubertal development, and risk-taking behavior.

Prior studies have highlighted adolescence as a period of increased risk-taking, which is postulated to result from an overactive reward system in the brain. Longitudinal studies are pivotal for testing these brain-behavior relations because individual slopes are more sensitive for detecting change. The aim of the current study was twofold: (1) to test patterns of age-related change (i.e., linear, quadratic, and cubic) in activity in the nucleus accumbens, a key reward region in the brain, in relation to change in puberty (self-report and testosterone levels), laboratory risk-taking and self-reported risk-taking tendency; and (2) to test whether individual differences in pubertal development and risk-taking behavior were contributors to longitudinal change in nucleus accumbens activity. We included 299 human participants at the first time point and 254 participants at the second time point, ranging between ages 8-27 years, time points were separated by a 2 year interval. Neural responses to rewards, pubertal development (self-report and testosterone levels), laboratory risk-taking (balloon analog risk task; BART), and self-reported risk-taking tendency (Behavior Inhibition System/Behavior Activation System questionnaire) were collected at both time points. The longitudinal analyses confirmed the quadratic age pattern for nucleus accumbens activity to rewards (peaking in adolescence), and the same quadratic pattern was found for laboratory risk-taking (BART). Nucleus accumbens activity change was further related to change in testosterone and self-reported reward-sensitivity (BAS Drive). Thus, this longitudinal analysis provides new insight in risk-taking and reward sensitivity in adolescence: (1) confirming an adolescent peak in nucleus accumbens activity, and (2) underlining a critical role for pubertal hormones and individual differences in risk-taking tendency.

Neural correlates of expected risks and returns in risky choice across development.

Adolescence is often described as a period of increased risk taking relative to both childhood and adulthood. This inflection in risky choice behavior has been attributed to a neurobiological imbalance between earlier developing motivational systems and later developing top-down control regions. Yet few studies have decomposed risky choice to investigate the underlying mechanisms or tracked their differential developmental trajectory. The current study uses a risk-return decomposition to more precisely assess the development of processes underlying risky choice and to link them more directly to specific neural mechanisms. This decomposition specifies the influence of changing risks (outcome variability) and changing returns (expected value) on the choices of children, adolescents, and adults in a dynamic risky choice task, the Columbia Card Task. Behaviorally, risk aversion increased across age groups, with adults uniformly risk averse and adolescents showing substantial individual differences in risk sensitivity, ranging from risk seeking to risk averse. Neurally, we observed an adolescent peak in risk-related activation in the anterior insula and dorsal medial PFC. Return sensitivity, on the other hand, increased monotonically across age groups and was associated with increased activation in the ventral medial PFC and posterior cingulate cortex with age. Our results implicate adolescence as a developmental phase of increased neural risk sensitivity. Importantly, this work shows that using a behaviorally validated decision-making framework allows a precise operationalization of key constructs underlying risky choice that inform the interpretation of results.

Neural Mechanisms Underlying Compensatory and Noncompensatory Strategies in Risky Choice.

Individuals may differ systematically in their applied decision strategies, which has critical implications for decision neuroscience but is yet scarcely studied. Our study's main focus was therefore to investigate the neural mechanisms underlying compensatory versus noncompensatory strategies in risky choice. Here, we compared people using a compensatory expected value maximization with people using a simplified noncompensatory loss-minimizing choice strategy. To this end, we used a two-choice paradigm including a set of "simple" items (e.g., simple condition), in which one option was superior on all attributes, and a set of "conflict" items, in which one option was superior on one attribute but inferior on other attributes. A binomial mixture analysis of the decisions elicited by these items differentiated between decision-makers using either a compensatory or a noncompensatory strategy. Behavioral differences were particularly pronounced in the conflict condition, and these were paralleled by neural results. That is, we expected compensatory decision-makers to use an integrated value comparison during choice in the conflict condition. Accordingly, the compensatory group tracked the difference in expected value between choice options reflected in neural activation in the parietal cortex. Furthermore, we expected noncompensatory, compared with compensatory, decision-makers to experience increased conflict when attributes provided conflicting information. Accordingly, the noncompensatory group showed greater dorsomedial PFC activation only in the conflict condition. These pronounced behavioral and neural differences indicate the need for decision neuroscience to account for individual differences in risky choice strategies and to broaden its scope to noncompensatory risky choice strategies.

Annual Research Review: Neural contributions to risk-taking in adolescence--developmental changes and individual differences.

BACKGROUND: Risk-taking, which involves voluntary choices for behaviors where outcomes remain uncertain, undergoes considerable developmental changes during childhood, adolescence, and early adulthood. In addition, risk-taking is thought to be a key element of many externalizing disorders, such as ADHD, delinquency, conduct disorder, and substance abuse. In this review, we will discuss the potential adaptive and nonadaptive properties of risk-taking in childhood and adolescence. FINDINGS: We propose that the changes in brain architecture and function are a crucial element underlying these developmental trajectories. We first identify how subcortical and cortical interactions are important for understanding risk-taking behavior in adults. Next, we show how developmental changes in this network underlie changes in risk-taking behavior. Finally, we explore how these differences can be important for understanding externalizing behavioral disorders in childhood and adolescence. CONCLUSIONS: We conclude that longitudinal studies are of crucial importance for understanding these developmental trajectories, and many of these studies are currently underway.

A cross-sectional and longitudinal analysis of reward-related brain activation: effects of age, pubertal stage, and reward sensitivity.

Neurobiological models suggest that adolescents are driven by an overactive ventral striatum (VS) response to rewards that may lead to an adolescent increase in risk-taking behavior. However, empirical studies showed mixed findings of adolescents' brain response to rewards. In this study, we aimed to elucidate the relationship between reward-related brain activation and risky decision-making. In addition, we examined effects of age, puberty, and individuals' reward sensitivity. We collected two datasets: Experiment 1 reports cross-sectional brain data from 75 participants (ages 10-25) who played a risky decision task. Experiment 2 presents a longitudinal extension in which a subset of these adolescents (n=33) was measured again 2years later. Results showed that (1) a reward-related network including VS and medial PFC was consistently activated over time, (2) the propensity to choose the risky option was related to increased reward-related activation in VS and medial PFC, and (3) longitudinal comparisons indicated that self-reported reward sensitivity was specifically related to VS activation over time. Together, these results advance our insights in the brain circuitry underlying reward processing across adolescence.

What is and what could have been: experiencing regret and relief across childhood.

Counterfactual emotions, such as regret and relief, are considered important in daily-life choice behaviour, learning and emotion regulation. A prominent question is from which age counterfactual emotions develop. In this study, we compared a more "traditional" analysis with a latent-class analysis (LCA) that allows the study of individual differences and a more detailed assessment of counterfactual emotions. Four groups of children (5-6 years, 7-8 years, 9-10 years and 11-13 years) and a group of young adults performed a choice task in which they encountered a Regret situation (chosen option was worse than alternative), a Relief situation (chosen option was better than alternative) and a Baseline situation (chosen option was equal to alternative). Traditional analyses indicated regret and relief to be present from ages 7 to 8. In contrast, the LCA indicated that subgroups experiencing regret and relief were present in all age groups, although regret and relief subgroups increased with age. Moreover, analyses indicated that higher reasoning scores increased the probability to belong to regret and relief subgroups and that the experience of regret dependent on trial order, being more prominent in later trials. We conclude that an individual-difference approach can advance insight into emotional development.

Is the unconscious, if it exists, a superior decision maker?

Newell & Shanks (N&S) show that there is no convincing evidence that processes assumed to be unconscious and superior are indeed unconscious. We take their argument one step further by showing that there is also no convincing evidence that these processes are superior. We review alternative paradigms that may provide more convincing tests of the superiority of (presumed) unconscious processes.