Goal-Dependent Hippocampal Representations Facilitate Self-Control.

Hippocampal activity linking past experiences and simulations of the future with current goals can play an important role in decision-making. The representation of information within the hippocampus may be especially critical in situations where one needs to overcome past rewarding experiences and exert self-control. Self-control success or failure may depend on how information is represented in the hippocampus and how effectively the representation process can be modified to achieve a specific goal. We test this hypothesis using representational similarity analyses of human (female/male) neuroimaging data during a dietary self-control task in which individuals must overcome taste temptations to choose healthy foods. We find that self-control is indeed associated with the way individuals represent taste information (valance) in the hippocampus and how taste representations there adapt to align with different goals/contexts. Importantly, individuals who were able to shift their hippocampal representations to a larger degree to align with the current motivation were better able to exert self-control when facing a dietary challenge. These results suggest an alternative or complementary neurobiological pathway leading to self-control success and indicate the need to update the classical view of self-control to continue to advance our understanding of its behavioral and neural underpinnings.SIGNIFICANCE STATEMENT The paper provides a new perspective on what leads to successful self-control at the behavioral and neurobiological levels. Our data suggest that self-control is enhanced when individuals adjust hippocampal processing to align with current goals.

Value certainty and choice confidence are multidimensional constructs that guide decision-making.

The degree of certainty that decision-makers have about their evaluations of available choice alternatives and their confidence about selecting the subjectively best alternative are important factors that affect current and future value-based choices. Assessments of the alternatives in a given choice set are rarely unidimensional; their values are usually derived from a combination of multiple distinct attributes. For example, the taste, texture, quantity, and nutritional content of a snack food may all be considered when determining whether to consume it. We examined how certainty about the levels of individual attributes of an option relates to certainty about the overall value of that option as a whole and/or to confidence in having chosen the subjectively best available option. We found that certainty and confidence are derived from unequally weighted combinations of attribute certainties rather than simple, equal combinations of all sources of uncertainty. Attributes that matter more in determining choice outcomes also are weighted more in metacognitive evaluations of certainty or confidence. Moreover, we found that the process of deciding between two alternatives leads to refinements in both attribute estimations and the degree of certainty in those estimates. Attributes that are more important in determining choice outcomes are refined more during the decision process in terms of both estimates and certainty. Although certainty and confidence are typically treated as unidimensional, our results indicate that they, like value estimates, are subjective, multidimensional constructs.

Intra-individual variability in task performance after cognitive training is associated with long-term outcomes in children.

The potential benefits and mechanistic effects of working memory training (WMT) in children are the subject of much research and debate. We show that after five weeks of school-based, adaptive WMT 6-9 year-old primary school children had greater activity in prefrontal and striatal brain regions, higher task accuracy, and reduced intra-individual variability in response times compared to controls. Using a sequential sampling decision model, we demonstrate that this reduction in intra-individual variability can be explained by changes to the evidence accumulation rates and thresholds. Critically, intra-individual variability is useful in quantifying the immediate impact of cognitive training interventions, being a better predictor of academic skills and well-being 6-12 months after the end of training than task accuracy. Taken together, our results suggest that attention control is the initial mechanism that leads to the long-run benefits from adaptive WMT. Selective and sustained attention abilities may serve as a scaffold for subsequent changes in higher cognitive processes, academic skills, and general well-being. Furthermore, these results highlight that the selection of outcome measures and the timing of the assessments play a crucial role in detecting training efficacy. Thus, evaluating intra-individual variability, during or directly after training could allow for the early tailoring of training interventions in terms of duration or content to maximise their impact.

Distinct and similar patterns of emotional development in adolescents and young adults.

Adolescence is a developmental period of increased sensitivity to social emotional cues, but it is less known whether young adults demonstrate similar social emotional sensitivity. The current study tested variation in reaction times to emotional face cues during different phases of emotional development. Ex-Gaussian parameters mu, sigma, and tau were computed, in addition to mean, median and standard deviation (SD) in reaction times (RT) during an emotional go/nogo-paradigm with fearful, happy, and calm facial expressions in 377 participants, 6-30 years of age. Across development, mean RT showed slowing to fearful facial expressions relative to both calm and happy facial cues, but mu revealed that this pattern was specific to adolescence. In young adulthood, increased variability to fearful expressions relative to both happy and calm ones was captured by SD and tau. The findings that adolescents had longer response latencies to fearful faces, whereas young adults demonstrated greater response variability to fearful faces, together reflect how social emotional processing continues to evolve from adolescence into early adulthood. The findings suggest that young adulthood is also a vulnerable period for processing social emotional cues that ultimately may be important to better understand why different psychopathologies emerge in early adulthood.

Higher Heart-Rate Variability Is Associated with Ventromedial Prefrontal Cortex Activity and Increased Resistance to Temptation in Dietary Self-Control Challenges.

Higher levels of self-control in decision making have been linked to better psychosocial and physical health. A similar link to health outcomes has been reported for heart-rate variability (HRV), a marker of physiological flexibility. Here, we sought to link these two, largely separate, research domains by testing the hypothesis that greater HRV would be associated with better dietary self-control in humans. Specifically, we examined whether total HRV at sedentary rest (measured as the SD of normal-to-normal intervals) can serve as a biomarker for the neurophysiological adaptability that putatively underlies self-controlled behavior. We found that HRV explained a significant portion of the individual variability in dietary self-control, with individuals having higher HRV being better able to downregulate their cravings in the face of taste temptations. Furthermore, HRV was associated with activity patterns in the ventromedial prefrontal cortex (vmPFC), a key node in the brain's valuation and decision circuitry. Specifically, individuals with higher HRV showed both higher overall vmPFC blood-oxygen-level-dependent activity and attenuated taste representations when presented with a dietary self-control challenge. Last, the behavioral and neural associations with HRV were consistent across both our stress induction and control experimental conditions. The stability of this association across experimental conditions suggests that HRV may serve as both a readily obtainable and robust biomarker for self-control ability across environmental contexts. SIGNIFICANCE STATEMENT: Self-control is associated with better health, but behavioral and psychometric self-control measures allow only indirect associations with health outcomes and may be distorted by reporting bias. We tested whether resting heart-rate variability (HRV), a physiological indicator of psychological and physical health, can predict individual differences in dietary self-control in humans. We found that higher HRV was associated with better self-control and improved predictions of choice behavior. Specifically, higher HRV was associated with more effective downregulation of taste temptations, and with a diminished neural representation of taste temptations during self-control challenges. Our results suggest that HRV may serve as an easily acquired, noninvasive, and low-cost biomarker for self-control ability.

Identifying the mechanisms through which behavioral weight-loss treatment improves food decision-making in obesity.

OBJECTIVES: Behavioral weight loss (BWL) programs are the recommended treatment for obesity, yet it is unknown whether these programs change one's ability to use self-control in food choices and what specific mechanisms support such change. Using experimental economics methods, we investigated whether changes in dietary behavior in individuals with obesity following BWL are driven by one or more of the following potential mechanisms: changes in the perception of the 1) health or 2) taste of food items, and/or 3) shifting decision weights for health versus taste attributes. Therefore, we compared these mechanisms between obese participants and lifetime normal weight controls (NW) both before and after BWL. METHODS: Females with obesity (N = 37, mean BMI = 33.2) completed a food choice task involving health ratings, taste ratings, and decision-making pre- and post-standard BWL intervention. NW controls (N = 30, BMI = 22.4) completed the same task. RESULTS: Individuals with obesity exhibited increased self-control (selecting healthier, less tasty food choices) post-treatment. However, their rates of self-control remained significantly lower than NW. We found no differences in initial health perceptions across groups, and no changes with treatment. In contrast, taste ratings and the relative value of taste versus health decreased following treatment. Although, post-treatment participants continued to perceive unhealthy foods as tastier and used less self-control than NW controls, they showed significant improvements in these domains following a BWL intervention. CONCLUSIONS: To help individuals improve dietary decisions, additional research is needed to determine how to make greater changes in taste preferences and/or the assignment of value to taste versus health attributes in food choices.

Enhanced Neural Responses to Imagined Primary Rewards Predict Reduced Monetary Temporal Discounting.

The pervasive tendency to discount the value of future rewards varies considerably across individuals and has important implications for health and well-being. Here, we used fMRI with human participants to examine whether an individual's neural representation of an imagined primary reward predicts the degree to which the value of delayed monetary payments is discounted. Because future rewards can never be experienced at the time of choice, imagining or simulating the benefits of a future reward may play a critical role in decisions between alternatives with either immediate or delayed benefits. We found that enhanced ventromedial prefrontal cortex response during imagined primary reward receipt was correlated with reduced discounting in a separate monetary intertemporal choice task. Furthermore, activity in enhanced ventromedial prefrontal cortex during reward imagination predicted temporal discounting behavior both between- and within-individual decision makers with 62% and 73% mean balanced accuracy, respectively. These results suggest that the quality of reward imagination may impact the degree to which future outcomes are discounted. Significance statement: We report a novel test of the hypothesis that an important factor influencing the discount rate for future rewards is the quality with which they are imagined or estimated in the present. Previous work has shown that temporal discounting is linked to individual characteristics ranging from general intelligence to the propensity for addiction. We demonstrate that individual differences in a neurobiological measure of primary reward imagination are significantly correlated with discounting rates for future monetary payments. Moreover, our neurobiological measure of imagination can be used to accurately predict choice behavior both between and within individuals. These results suggest that improving reward imagination may be a useful therapeutic target for individuals whose high discount rates promote detrimental behaviors.

Transcranial Stimulation over Frontopolar Cortex Elucidates the Choice Attributes and Neural Mechanisms Used to Resolve Exploration-Exploitation Trade-Offs.

Optimal behavior requires striking a balance between exploiting tried-and-true options or exploring new possibilities. Neuroimaging studies have identified different brain regions in humans where neural activity is correlated with exploratory or exploitative behavior, but it is unclear whether this activity directly implements these choices or simply reflects a byproduct of the behavior. Moreover, it remains unknown whether arbitrating between exploration and exploitation can be influenced with exogenous methods, such as brain stimulation. In our study, we addressed these questions by selectively upregulating and downregulating neuronal excitability with anodal or cathodal transcranial direct current stimulation over right frontopolar cortex during a reward-learning task. This caused participants to make slower, more exploratory or faster, more exploitative decisions, respectively. Bayesian computational modeling revealed that stimulation affected how much participants took both expected and obtained rewards into account when choosing to exploit or explore: Cathodal stimulation resulted in an increased focus on the option expected to yield the highest payout, whereas anodal stimulation led to choices that were less influenced by anticipated payoff magnitudes and were more driven by recent negative reward prediction errors. These findings suggest that exploration is triggered by a neural mechanism that is sensitive to prior less-than-expected choice outcomes and thus pushes people to seek out alternative courses of action. Together, our findings establish a parsimonious neurobiological mechanism that causes exploration and exploitation, and they provide new insights into the choice features used by this mechanism to direct decision-making.

Early developmental emergence of human amygdala-prefrontal connectivity after maternal deprivation.

Under typical conditions, medial prefrontal cortex (mPFC) connections with the amygdala are immature during childhood and become adult-like during adolescence. Rodent models show that maternal deprivation accelerates this development, prompting examination of human amygdala-mPFC phenotypes following maternal deprivation. Previously institutionalized youths, who experienced early maternal deprivation, exhibited atypical amygdala-mPFC connectivity. Specifically, unlike the immature connectivity (positive amygdala-mPFC coupling) of comparison children, children with a history of early adversity evidenced mature connectivity (negative amygdala-mPFC coupling) and thus, resembled the adolescent phenotype. This connectivity pattern was mediated by the hormone cortisol, suggesting that stress-induced modifications of the hypothalamic-pituitary-adrenal axis shape amygdala-mPFC circuitry. Despite being age-atypical, negative amygdala-mPFC coupling conferred some degree of reduced anxiety, although anxiety was still significantly higher in the previously institutionalized group. These findings suggest that accelerated amygdala-mPFC development is an ontogenetic adaptation in response to early adversity.

Interactions between dorsolateral and ventromedial prefrontal cortex underlie context-dependent stimulus valuation in goal-directed choice.

External circumstances and internal bodily states often change and require organisms to flexibly adapt valuation processes to select the optimal action in a given context. Here, we investigate the neurobiology of context-dependent valuation in 22 human subjects using functional magnetic resonance imaging. Subjects made binary choices between visual stimuli with three attributes (shape, color, and pattern) that were associated with monetary values. Context changes required subjects to deviate from the default shape valuation and to integrate a second attribute to comply with the goal to maximize rewards. Critically, this binary choice task did not involve any conflict between opposing monetary, temporal, or social preferences. We tested the hypothesis that interactions between regions of dorsolateral prefrontal cortex (dlPFC) and ventromedial prefrontal cortex (vmPFC) implicated in self-control choices would also underlie the more general function of context-dependent valuation. Consistent with this idea, we found that the degree to which stimulus attributes were reflected in vmPFC activity varied as a function of context. In addition, activity in dlPFC increased when context changes required a reweighting of stimulus attribute values. Moreover, the strength of the functional connectivity between dlPFC and vmPFC was associated with the degree of context-specific attribute valuation in vmPFC at the time of choice. Our findings suggest that functional interactions between dlPFC and vmPFC are a key aspect of context-dependent valuation and that the role of this network during choices that require self-control to adjudicate between competing outcome preferences is a specific application of this more general neural mechanism.

A developmental shift from positive to negative connectivity in human amygdala-prefrontal circuitry.

Recent human imaging and animal studies highlight the importance of frontoamygdala circuitry in the regulation of emotional behavior and its disruption in anxiety-related disorders. Although tracing studies have suggested changes in amygdala-cortical connectivity through the adolescent period in rodents, less is known about the reciprocal connections within this circuitry across human development, when these circuits are being fine-tuned and substantial changes in emotional control are observed. The present study examined developmental changes in amygdala-prefrontal circuitry across the ages of 4-22 years using task-based functional magnetic resonance imaging. Results suggest positive amygdala-prefrontal connectivity in early childhood that switches to negative functional connectivity during the transition to adolescence. Amygdala-medial prefrontal cortex functional connectivity was significantly positive (greater than zero) among participants younger than 10 years, whereas functional connectivity was significantly negative (less than zero) among participants 10 years and older, over and above the effect of amygdala reactivity. The developmental switch in functional connectivity was paralleled by a steady decline in amygdala reactivity. Moreover, the valence switch might explain age-related improvement in task performance and a developmentally normative decline in anxiety. Initial positive connectivity followed by a valence shift to negative connectivity provides a neurobiological basis for regulatory development and may present novel insight into a more general process of developing regulatory connections.

Teens impulsively react rather than retreat from threat.

There is a significant inflection in risk taking and criminal behavior during adolescence, but the basis for this increase remains largely unknown. An increased sensitivity to rewards has been suggested to explain these behaviors, yet juvenile offences often occur in emotionally charged situations of negative valence. How behavior is altered by changes in negative emotional processes during adolescence has received less attention than changes in positive emotional processes. The current study uses a measure of impulsivity in combination with cues that signal threat or safety to assess developmental changes in emotional responses to threat cues. We show that adolescents, especially males, impulsively react to threat cues relative to neutral ones more than adults or children, even when instructed not to respond. This adolescent-specific behavioral pattern is paralleled by enhanced activity in limbic cortical regions implicated in the detection and assignment of emotional value to inputs and in the subsequent regulation of responses to them when successfully suppressing impulsive responses to threat cues. In contrast, prefrontal control regions implicated in detecting and resolving competing responses show an adolescent-emergent pattern (i.e. greater activity in adolescents and adults relative to children) during successful suppression of a response regardless of emotion. Our findings suggest that adolescence is a period of heightened sensitivity to social and emotional cues that results in diminished regulation of behavior in their presence.

Transformation of stimulus value signals into motor commands during simple choice.

Decision-making can be broken down into several component processes: assigning values to stimuli under consideration, selecting an option by comparing those values, and initiating motor responses to obtain the reward. Although much is known about the neural encoding of stimulus values and motor commands, little is known about the mechanisms through which stimulus values are compared, and the resulting decision is transmitted to motor systems. We investigated this process using human fMRI in a task where choices were indicated using the left or right hand. We found evidence consistent with the hypothesis that value signals are computed in the ventral medial prefrontal cortex, they are passed to regions of dorsomedial prefrontal cortex and intraparietal sulcus, implementing a comparison process, and the output of the comparator regions modulates activity in motor cortex to implement the choice. These results describe the network through which stimulus values are transformed into actions during a simple choice task.

Rationality, preferences, and emotions with biological constraints: it all starts from our senses.

Is the role of our sensory systems to represent the physical world as accurately as possible? If so, are our preferences and emotions, often deemed irrational, decoupled from these 'ground-truth' sensory experiences? We show why the answer to both questions is 'no'. Brain function is metabolically costly, and the brain loses some fraction of the information that it encodes and transmits. Therefore, if brains maximize objective functions that increase the fitness of their species, they should adapt to the objective-maximizing rules of the environment at the earliest stages of sensory processing. Consequently, observed 'irrationalities', preferences, and emotions stem from the necessity for our early sensory systems to adapt and process information while considering the metabolic costs and internal states of the organism.

Rethinking model-based and model-free influences on mental effort and striatal prediction errors.

A standard assumption in neuroscience is that low-effort model-free learning is automatic and continuously used, whereas more complex model-based strategies are only used when the rewards they generate are worth the additional effort. We present evidence refuting this assumption. First, we demonstrate flaws in previous reports of combined model-free and model-based reward prediction errors in the ventral striatum that probably led to spurious results. More appropriate analyses yield no evidence of model-free prediction errors in this region. Second, we find that task instructions generating more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration between model-based and model-free strategies. Together, our data indicate that model-free learning may not be automatic. Instead, humans can reduce mental effort by using a model-based strategy alone rather than arbitrating between multiple strategies. Our results call for re-evaluation of the assumptions in influential theories of learning and decision-making.

Sensory perception relies on fitness-maximizing codes.

Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.

Focusing attention on the health aspects of foods changes value signals in vmPFC and improves dietary choice.

Attention is thought to play a key role in the computation of stimulus values at the time of choice, which suggests that attention manipulations could be used to improve decision-making in domains where self-control lapses are pervasive. We used an fMRI food choice task with non-dieting human subjects to investigate whether exogenous cues that direct attention to the healthiness of foods could improve dietary choices. Behaviorally, we found that subjects made healthier choices in the presence of health cues. In parallel, stimulus value signals in ventromedial prefrontal cortex were more responsive to the healthiness of foods in the presence of health cues, and this effect was modulated by activity in regions of dorsolateral prefrontal cortex. These findings suggest that the neural mechanisms used in successful self-control can be activated by exogenous attention cues, and provide insights into the processes through which behavioral therapies and public policies could facilitate self-control.

Older adults process the probability of winning sooner but weigh it less during lottery decisions.

Empirical evidence has shown that visually enhancing the saliency of reward probabilities can ease the cognitive demands of value comparisons and improve value-based decisions in old age. In the present study, we used a time-varying drift diffusion model that includes starting time parameters to better understand (1) how increasing the saliency of reward probabilities may affect the dynamics of value-based decision-making and (2) how these effects may interact with age. We examined choices made by younger and older adults in a mixed lottery choice task. On a subset of trials, we used a color-coding scheme to highlight the saliency of reward probabilities, which served as a decision-aid. The results showed that, in control trials, older adults started to consider probability relative to magnitude information sooner than younger adults, but that their evidence accumulation processes were less sensitive to reward probabilities than that of younger adults. This may indicate a noisier and more stochastic information accumulation process during value-based decisions in old age. The decision-aid increased the influence of probability information on evidence accumulation rates in both age groups, but did not alter the relative timing of accumulation for probability versus magnitude in either group.

Reconciling psychological and neuroscientific accounts of reduced motivation in aging.

Motivation is a hallmark of healthy aging, but the motivation to engage in effortful behavior diminishes with increasing age. Most neurobiological accounts of altered motivation in older adults assume that these deficits are caused by a gradual decline in brain tissue, while some psychological theories posit a switch from gain orientation to loss avoidance in motivational goals. Here, we contribute to reconcile the psychological and neural perspectives by providing evidence that the frontopolar cortex (FPC), a brain region involved in cost-benefit weighting, increasingly underpins effort avoidance rather than engagement with age. Using anodal transcranial direct current stimulation together with effort-reward trade-offs, we find that the FPC's function in effort-based decisions remains focused on cost-benefit calculations but appears to switch from reward-seeking to cost avoidance with increasing age. This is further evidenced by the exploratory, independent analysis of structural brain changes, showing that the relationship between the density of the frontopolar neural tissue and the willingness to exert effort differs in young vs older adults. Our results inform aging-related models of decision-making by providing preliminary evidence that, in addition to cortical thinning, changes in goal orientation need to be considered in order to understand alterations in decision-making over the life span.

Value computations in ventral medial prefrontal cortex during charitable decision making incorporate input from regions involved in social cognition.

Little is known about the neural networks supporting value computation during complex social decisions. We investigated this question using functional magnetic resonance imaging while subjects made donations to different charities. We found that the blood oxygenation level-dependent signal in ventral medial prefrontal cortex (VMPFC) correlated with the subjective value of voluntary donations. Furthermore, the region of the VMPFC identified showed considerable overlap with regions that have been shown to encode for the value of basic rewards at the time of choice, suggesting that it might serve as a common valuation system during decision making. In addition, functional connectivity analyses indicated that the value signal in VMPFC might integrate inputs from networks, including the anterior insula and posterior superior temporal cortex, that are thought to be involved in social cognition.