Rationality, preferences, and emotions with biological constraints: it all starts from our senses.

Is the role of our sensory systems to represent the physical world as accurately as possible? If so, are our preferences and emotions, often deemed irrational, decoupled from these 'ground-truth' sensory experiences? We show why the answer to both questions is 'no'. Brain function is metabolically costly, and the brain loses some fraction of the information that it encodes and transmits. Therefore, if brains maximize objective functions that increase the fitness of their species, they should adapt to the objective-maximizing rules of the environment at the earliest stages of sensory processing. Consequently, observed 'irrationalities', preferences, and emotions stem from the necessity for our early sensory systems to adapt and process information while considering the metabolic costs and internal states of the organism.

Goal-Dependent Hippocampal Representations Facilitate Self-Control.

Hippocampal activity linking past experiences and simulations of the future with current goals can play an important role in decision-making. The representation of information within the hippocampus may be especially critical in situations where one needs to overcome past rewarding experiences and exert self-control. Self-control success or failure may depend on how information is represented in the hippocampus and how effectively the representation process can be modified to achieve a specific goal. We test this hypothesis using representational similarity analyses of human (female/male) neuroimaging data during a dietary self-control task in which individuals must overcome taste temptations to choose healthy foods. We find that self-control is indeed associated with the way individuals represent taste information (valance) in the hippocampus and how taste representations there adapt to align with different goals/contexts. Importantly, individuals who were able to shift their hippocampal representations to a larger degree to align with the current motivation were better able to exert self-control when facing a dietary challenge. These results suggest an alternative or complementary neurobiological pathway leading to self-control success and indicate the need to update the classical view of self-control to continue to advance our understanding of its behavioral and neural underpinnings.SIGNIFICANCE STATEMENT The paper provides a new perspective on what leads to successful self-control at the behavioral and neurobiological levels. Our data suggest that self-control is enhanced when individuals adjust hippocampal processing to align with current goals.

Sensory perception relies on fitness-maximizing codes.

Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.

Rethinking model-based and model-free influences on mental effort and striatal prediction errors.

A standard assumption in neuroscience is that low-effort model-free learning is automatic and continuously used, whereas more complex model-based strategies are only used when the rewards they generate are worth the additional effort. We present evidence refuting this assumption. First, we demonstrate flaws in previous reports of combined model-free and model-based reward prediction errors in the ventral striatum that probably led to spurious results. More appropriate analyses yield no evidence of model-free prediction errors in this region. Second, we find that task instructions generating more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration between model-based and model-free strategies. Together, our data indicate that model-free learning may not be automatic. Instead, humans can reduce mental effort by using a model-based strategy alone rather than arbitrating between multiple strategies. Our results call for re-evaluation of the assumptions in influential theories of learning and decision-making.

Value certainty and choice confidence are multidimensional constructs that guide decision-making.

The degree of certainty that decision-makers have about their evaluations of available choice alternatives and their confidence about selecting the subjectively best alternative are important factors that affect current and future value-based choices. Assessments of the alternatives in a given choice set are rarely unidimensional; their values are usually derived from a combination of multiple distinct attributes. For example, the taste, texture, quantity, and nutritional content of a snack food may all be considered when determining whether to consume it. We examined how certainty about the levels of individual attributes of an option relates to certainty about the overall value of that option as a whole and/or to confidence in having chosen the subjectively best available option. We found that certainty and confidence are derived from unequally weighted combinations of attribute certainties rather than simple, equal combinations of all sources of uncertainty. Attributes that matter more in determining choice outcomes also are weighted more in metacognitive evaluations of certainty or confidence. Moreover, we found that the process of deciding between two alternatives leads to refinements in both attribute estimations and the degree of certainty in those estimates. Attributes that are more important in determining choice outcomes are refined more during the decision process in terms of both estimates and certainty. Although certainty and confidence are typically treated as unidimensional, our results indicate that they, like value estimates, are subjective, multidimensional constructs.

Intra-individual variability in task performance after cognitive training is associated with long-term outcomes in children.

The potential benefits and mechanistic effects of working memory training (WMT) in children are the subject of much research and debate. We show that after five weeks of school-based, adaptive WMT 6-9 year-old primary school children had greater activity in prefrontal and striatal brain regions, higher task accuracy, and reduced intra-individual variability in response times compared to controls. Using a sequential sampling decision model, we demonstrate that this reduction in intra-individual variability can be explained by changes to the evidence accumulation rates and thresholds. Critically, intra-individual variability is useful in quantifying the immediate impact of cognitive training interventions, being a better predictor of academic skills and well-being 6-12 months after the end of training than task accuracy. Taken together, our results suggest that attention control is the initial mechanism that leads to the long-run benefits from adaptive WMT. Selective and sustained attention abilities may serve as a scaffold for subsequent changes in higher cognitive processes, academic skills, and general well-being. Furthermore, these results highlight that the selection of outcome measures and the timing of the assessments play a crucial role in detecting training efficacy. Thus, evaluating intra-individual variability, during or directly after training could allow for the early tailoring of training interventions in terms of duration or content to maximise their impact.

Older adults process the probability of winning sooner but weigh it less during lottery decisions.

Empirical evidence has shown that visually enhancing the saliency of reward probabilities can ease the cognitive demands of value comparisons and improve value-based decisions in old age. In the present study, we used a time-varying drift diffusion model that includes starting time parameters to better understand (1) how increasing the saliency of reward probabilities may affect the dynamics of value-based decision-making and (2) how these effects may interact with age. We examined choices made by younger and older adults in a mixed lottery choice task. On a subset of trials, we used a color-coding scheme to highlight the saliency of reward probabilities, which served as a decision-aid. The results showed that, in control trials, older adults started to consider probability relative to magnitude information sooner than younger adults, but that their evidence accumulation processes were less sensitive to reward probabilities than that of younger adults. This may indicate a noisier and more stochastic information accumulation process during value-based decisions in old age. The decision-aid increased the influence of probability information on evidence accumulation rates in both age groups, but did not alter the relative timing of accumulation for probability versus magnitude in either group.

Reconciling psychological and neuroscientific accounts of reduced motivation in aging.

Motivation is a hallmark of healthy aging, but the motivation to engage in effortful behavior diminishes with increasing age. Most neurobiological accounts of altered motivation in older adults assume that these deficits are caused by a gradual decline in brain tissue, while some psychological theories posit a switch from gain orientation to loss avoidance in motivational goals. Here, we contribute to reconcile the psychological and neural perspectives by providing evidence that the frontopolar cortex (FPC), a brain region involved in cost-benefit weighting, increasingly underpins effort avoidance rather than engagement with age. Using anodal transcranial direct current stimulation together with effort-reward trade-offs, we find that the FPC's function in effort-based decisions remains focused on cost-benefit calculations but appears to switch from reward-seeking to cost avoidance with increasing age. This is further evidenced by the exploratory, independent analysis of structural brain changes, showing that the relationship between the density of the frontopolar neural tissue and the willingness to exert effort differs in young vs older adults. Our results inform aging-related models of decision-making by providing preliminary evidence that, in addition to cortical thinning, changes in goal orientation need to be considered in order to understand alterations in decision-making over the life span.

Healthy decisions in the cued-attribute food choice paradigm have high test-retest reliability.

Food choice paradigms are commonly used to study decision mechanisms, individual differences, and intervention efficacy. Here, we measured behavior from twenty-three healthy young adults who completed five repetitions of a cued-attribute food choice paradigm over two weeks. This task includes cues prompting participants to explicitly consider the healthiness of the food items before making a selection, or to choose naturally based on whatever freely comes to mind. We found that the average patterns of food choices following both cue types and ratings about the palatability (i.e. taste) and healthiness of the food items were similar across all five repetitions. At the individual level, the test-retest reliability for choices in both conditions and healthiness ratings was excellent. However, test-retest reliability for taste ratings was only fair, suggesting that estimates about palatability may vary more from day to day for the same individual.

Genetic underpinnings of risky behaviour relate to altered neuroanatomy.

Previous research points to the heritability of risk-taking behaviour. However, evidence on how genetic dispositions are translated into risky behaviour is scarce. Here, we report a genetically informed neuroimaging study of real-world risky behaviour across the domains of drinking, smoking, driving and sexual behaviour in a European sample from the UK Biobank (N = 12,675). We find negative associations between risky behaviour and grey-matter volume in distinct brain regions, including amygdala, ventral striatum, hypothalamus and dorsolateral prefrontal cortex (dlPFC). These effects are replicated in an independent sample recruited from the same population (N = 13,004). Polygenic risk scores for risky behaviour, derived from a genome-wide association study in an independent sample (N = 297,025), are inversely associated with grey-matter volume in dlPFC, putamen and hypothalamus. This relation mediates roughly 2.2% of the association between genes and behaviour. Our results highlight distinct heritable neuroanatomical features as manifestations of the genetic propensity for risk taking.

Humans primarily use model-based inference in the two-stage task.

Distinct model-free and model-based learning processes are thought to drive both typical and dysfunctional behaviours. Data from two-stage decision tasks have seemingly shown that human behaviour is driven by both processes operating in parallel. However, in this study, we show that more detailed task instructions lead participants to make primarily model-based choices that have little, if any, simple model-free influence. We also demonstrate that behaviour in the two-stage task may falsely appear to be driven by a combination of simple model-free and model-based learning if purely model-based agents form inaccurate models of the task because of misconceptions. Furthermore, we report evidence that many participants do misconceive the task in important ways. Overall, we argue that humans formulate a wide variety of learning models. Consequently, the simple dichotomy of model-free versus model-based learning is inadequate to explain behaviour in the two-stage task and connections between reward learning, habit formation and compulsivity.

BOLD activity during emotion reappraisal positively correlates with dietary self-control success.

We combined established emotion regulation and dietary choice tasks with fMRI to investigate behavioral and neural associations in self-regulation across the two domains in human participants. We found that increased BOLD activity during the successful reappraisal of positive and negative emotional stimuli was associated with dietary self-control success. This cross-task correlation was present in medial and lateral prefrontal cortex as well as the striatum. In contrast, BOLD activity during the food choice task was not associated with self-reported emotion regulation efficacy. These results suggest that neural processes utilized during the reappraisal of emotional stimuli may also facilitate dietary choices that override palatability in favor of healthfulness. In summary, our findings indicate that the neural systems supporting emotion reappraisal can generalize to other behavioral contexts that require reevaluation of rewarding stimuli and outcomes to promote choices that conform with the current goal.

Dissociable mechanisms govern when and how strongly reward attributes affect decisions.

Theories and computational models of decision-making usually focus on how strongly different attributes are weighted in choice, for example, as a function of their importance or salience to the decision-maker. However, when different attributes affect the decision process is a question that has received far less attention. Here, we investigated whether the timing of attribute consideration has a unique influence on decision-making by using a time-varying drift diffusion model and data from four separate experiments. Experimental manipulations of attention and neural activity demonstrated that we can dissociate the processes that determine the relative weighting strength and timing of attribute consideration. Thus, the processes determining either the weighting strengths or the timing of attributes in decision-making can independently adapt to changes in the environment or goals. Quantifying these separate influences of timing and weighting on choice improves our understanding and predictions of individual differences in decision behaviour.

Distinct and similar patterns of emotional development in adolescents and young adults.

Adolescence is a developmental period of increased sensitivity to social emotional cues, but it is less known whether young adults demonstrate similar social emotional sensitivity. The current study tested variation in reaction times to emotional face cues during different phases of emotional development. Ex-Gaussian parameters mu, sigma, and tau were computed, in addition to mean, median and standard deviation (SD) in reaction times (RT) during an emotional go/nogo-paradigm with fearful, happy, and calm facial expressions in 377 participants, 6-30 years of age. Across development, mean RT showed slowing to fearful facial expressions relative to both calm and happy facial cues, but mu revealed that this pattern was specific to adolescence. In young adulthood, increased variability to fearful expressions relative to both happy and calm ones was captured by SD and tau. The findings that adolescents had longer response latencies to fearful faces, whereas young adults demonstrated greater response variability to fearful faces, together reflect how social emotional processing continues to evolve from adolescence into early adulthood. The findings suggest that young adulthood is also a vulnerable period for processing social emotional cues that ultimately may be important to better understand why different psychopathologies emerge in early adulthood.

Good practice in food-related neuroimaging.

The use of neuroimaging tools, especially functional magnetic resonance imaging, in nutritional research has increased substantially over the past 2 decades. Neuroimaging is a research tool with great potential impact on the field of nutrition, but to achieve that potential, appropriate use of techniques and interpretation of neuroimaging results is necessary. In this article, we present guidelines for good methodological practice in functional magnetic resonance imaging studies and flag specific limitations in the hope of helping researchers to make the most of neuroimaging tools and avoid potential pitfalls. We highlight specific considerations for food-related studies, such as how to adjust statistically for common confounders, like, for example, hunger state, menstrual phase, and BMI, as well as how to optimally match different types of food stimuli. Finally, we summarize current research needs and future directions, such as the use of prospective designs and more realistic paradigms for studying eating behavior.

Differential modulation of cognitive control networks by monetary reward and punishment.

Incentives are primary determinants of if and how well an organism will perform a given behavior. Here, we examined how incentive valence and magnitude influence task switching, a critical cognitive control process, and test the predictions that the anterior cingulate cortex (ACC) and the ventral striatum (vStr) function as key nodes linking motivation and control systems in the brain. Our results indicate that reward and punishment incentives have both common and distinct effects on cognitive control at the behavioral and neurobiological levels. For example, reward incentives led to greater activity in the ACC during the engagement of control relative to punishments. Furthermore, the neural responses to reward and punishment differed as a function of individual sensitivity to each incentive valence. Functional connectivity analyses suggest a role for vStr in signaling motivational value during cognitive control and as a potential link between motivation and control networks. Overall, our findings suggest that similar changes in observed behavior (e.g. response accuracy) under reward and punishment incentives are mediated by, at least partially, distinct neurobiological substrates.

Computational and neurobiological foundations of leadership decisions.

Leaders must take responsibility for others and thus affect the well-being of individuals, organizations, and nations. We identify the effects of responsibility on leaders' choices at the behavioral and neurobiological levels and document the widespread existence of responsibility aversion, that is, a reduced willingness to make decisions if the welfare of others is at stake. In mechanistic terms, basic preferences toward risk, loss, and ambiguity do not explain responsibility aversion, which, instead, is driven by a second-order cognitive process reflecting an increased demand for certainty about the best choice when others' welfare is affected. Finally, models estimating levels of information flow between brain regions that process separate choice components provide the first step in understanding the neurobiological basis of individual variability in responsibility aversion and leadership scores.

A note on the analysis of two-stage task results: How changes in task structure affect what model-free and model-based strategies predict about the effects of reward and transition on the stay probability.

Many studies that aim to detect model-free and model-based influences on behavior employ two-stage behavioral tasks of the type pioneered by Daw and colleagues in 2011. Such studies commonly modify existing two-stage decision paradigms in order to better address a given hypothesis, which is an important means of scientific progress. It is, however, critical to fully appreciate the impact of any modified or novel experimental design features on the expected results. Here, we use two concrete examples to demonstrate that relatively small changes in the two-stage task design can substantially change the pattern of actions taken by model-free and model-based agents as a function of the reward outcomes and transitions on previous trials. In the first, we show that, under specific conditions, purely model-free agents will produce the reward by transition interactions typically thought to characterize model-based behavior on a two-stage task. The second example shows that model-based agents' behavior is driven by a main effect of transition-type in addition to the canonical reward by transition interaction whenever the reward probabilities of the final states do not sum to one. Together, these examples emphasize the task-dependence of model-free and model-based behavior and highlight the benefits of using computer simulations to determine what pattern of results to expect from both model-free and model-based agents performing a given two-stage decision task in order to design choice paradigms and analysis strategies best suited to the current question.

Brain Stimulation Over the Frontopolar Cortex Enhances Motivation to Exert Effort for Reward.

BACKGROUND: Loss of motivation is a characteristic feature of several psychiatric and neurological disorders. However, the neural mechanisms underlying human motivation are far from being understood. Here, we investigate the role that the frontopolar cortex (FPC) plays in motivating cognitive and physical effort exertion by computing subjective effort equivalents. METHODS: We manipulated neural processing with transcranial direct current stimulation targeting the FPC while 141 healthy participants decided whether or not to engage in cognitive or physical effort to obtain rewards. RESULTS: We found that brain stimulation targeting the FPC increased the amount of both types of effort participants were willing to exert for rewards. CONCLUSIONS: Our findings provide important insights into the neural mechanisms involved in motivating effortful behavior. Moreover, they suggest that considering the motivation-related activity of the FPC could facilitate the development of treatments for the loss of motivation commonly seen in psychiatric and other neurological disorders.

Food-Predicting Stimuli Differentially Influence Eye Movements and Goal-Directed Behavior in Normal-Weight, Overweight, and Obese Individuals.

Obese individuals have been shown to exhibit abnormal sensitivity to rewards and reward-predicting cues as for example food-associated cues frequently used in advertisements. It has also been shown that food-associated cues can increase goal-directed behavior but it is currently unknown, whether this effect differs between normal-weight, overweight, and obese individuals. Here, we investigate this question by using a Pavlovian-to-instrumental transfer (PIT) task in normal-weight (N = 20), overweight (N = 17), and obese (N = 17) individuals. Furthermore, we applied eye tracking during Pavlovian conditioning to measure the participants' conditioned response as a proxy of the incentive salience of the predicted reward. Our results show that the goal-directed behavior of overweight individuals was more strongly influenced by food-predicting cues (i.e., stronger PIT effect) than that of normal-weight and obese individuals (p < 0.001). The weight groups were matched for age, gender, education, and parental education. Eye movements during Pavlovian conditioning also differed between weight categories (p < 0.05) and were used to categorize individuals based on their fixation style into "high eye index" versus "low eye index" as well. Our main finding was that the fixation style exhibited a complex interaction with the weight category. Furthermore, we found that normal-weight individuals of the group "high eye index" had higher body mass index within the healthy range than individuals of the group "low eye index" (p < 0.001), but this relationship was not found within in the overweight or obese groups (p > 0.646). Our findings are largely consistent with the incentive sensitization theory predicting that overweight individuals are more susceptible to food-related cues than normal-weight controls. However, this hypersensitivity might be reduced in obese individuals, possibly due to habitual/compulsive overeating or differences in reward valuation.