How conformity can lead to polarised social behaviour.

Learning social behaviour of others strongly influences one's own social attitudes. We compare several distinct explanations of this phenomenon, testing their predictions using computational modelling across four experimental conditions. In the experiment, participants chose repeatedly whether to pay for increasing (prosocial) or decreasing (antisocial) the earnings of an unknown other. Halfway through the task, participants predicted the choices of an extremely prosocial or antisocial agent (either a computer, a single participant, or a group of participants). Our analyses indicate that participants polarise their social attitude mainly due to normative expectations. Specifically, most participants conform to presumed demands by the authority (vertical influence), or because they learn that the observed human agents follow the norm very closely (horizontal influence).

The Effect of Counterfactual Information on Outcome Value Coding in Medial Prefrontal and Cingulate Cortex: From an Absolute to a Relative Neural Code.

Adaptive coding of stimuli is well documented in perception, where it supports efficient encoding over a broad range of possible percepts. Recently, a similar neural mechanism has been reported also in value-based decision, where it allows optimal encoding of vast ranges of values in PFC: neuronal response to value depends on the choice context (relative coding), rather than being invariant across contexts (absolute coding). Additionally, value learning is sensitive to the amount of feedback information: providing complete feedback (both obtained and forgone outcomes) instead of partial feedback (only obtained outcome) improves learning. However, it is unclear whether relative coding occurs in all PFC regions and how it is affected by feedback information. We systematically investigated univariate and multivariate feedback encoding in various mPFC regions and compared three modes of neural coding: absolute, partially-adaptive and fully-adaptive.Twenty-eight human participants (both sexes) performed a learning task while undergoing fMRI scanning. On each trial, they chose between two symbols associated with a certain outcome. Then, the decision outcome was revealed. Notably, in one-half of the trials participants received partial feedback, whereas in the other half they got complete feedback. We used univariate and multivariate analysis to explore value encoding in different feedback conditions.We found that both obtained and forgone outcomes were encoded in mPFC, but with opposite sign in its ventral and dorsal subdivisions. Moreover, we showed that increasing feedback information induced a switch from absolute to relative coding. Our results suggest that complete feedback information enhances context-dependent outcome encoding.SIGNIFICANCE STATEMENT This study offers a systematic investigation of the effect of the amount of feedback information (partial vs complete) on univariate and multivariate outcome value encoding, within multiple regions in mPFC and cingulate cortex that are critical for value-based decisions and behavioral adaptation. Moreover, we provide the first comparison of three possible models of neural coding (i.e., absolute, partially-adaptive, and fully-adaptive coding) of value signal in these regions, by using commensurable measures of prediction accuracy. Taken together, our results help build a more comprehensive picture of how the human brain encodes and processes outcome value. In particular, our results suggest that simultaneous presentation of obtained and foregone outcomes promotes relative value representation.

Dissociation between Private and Social Counterfactual Value Signals Following Ventromedial Prefrontal Cortex Damage.

Individuals learn by comparing the outcome of chosen and unchosen actions. A negative counterfactual value signal is generated when this comparison is unfavorable. This can happen in private as well as in social settings-where the foregone outcome results from the choice of another person. We hypothesized that, despite sharing similar features such as supporting learning, these two counterfactual signals might implicate distinct brain networks. We conducted a neuropsychological study on the role of private and social counterfactual value signals in risky decision-making. Patients with lesions in the ventromedial prefrontal cortex (vmPFC), lesion controls, and healthy controls repeatedly chose between lotteries. In private trials, participants could observe the outcomes of their choices and the outcomes of the unselected lotteries. In social trials, participants could also see the other player's choices and outcome. At the time of outcome, vmPFC patients were insensitive to private counterfactual value signals, whereas their responses to social comparison were similar to those of control participants. At the time of choice, intact vmPFC was necessary to integrate counterfactual signals in decisions, although amelioration was observed during the course of the task, possibly driven by social trials. We conclude that if the vmPFC is critical in processing private counterfactual signals and in integrating those signals in decision-making, then distinct brain areas might support the processing of social counterfactual signals.

The Computational Development of Reinforcement Learning during Adolescence.

Adolescence is a period of life characterised by changes in learning and decision-making. Learning and decision-making do not rely on a unitary system, but instead require the coordination of different cognitive processes that can be mathematically formalised as dissociable computational modules. Here, we aimed to trace the developmental time-course of the computational modules responsible for learning from reward or punishment, and learning from counterfactual feedback. Adolescents and adults carried out a novel reinforcement learning paradigm in which participants learned the association between cues and probabilistic outcomes, where the outcomes differed in valence (reward versus punishment) and feedback was either partial or complete (either the outcome of the chosen option only, or the outcomes of both the chosen and unchosen option, were displayed). Computational strategies changed during development: whereas adolescents' behaviour was better explained by a basic reinforcement learning algorithm, adults' behaviour integrated increasingly complex computational features, namely a counterfactual learning module (enabling enhanced performance in the presence of complete feedback) and a value contextualisation module (enabling symmetrical reward and punishment learning). Unlike adults, adolescent performance did not benefit from counterfactual (complete) feedback. In addition, while adults learned symmetrically from both reward and punishment, adolescents learned from reward but were less likely to learn from punishment. This tendency to rely on rewards and not to consider alternative consequences of actions might contribute to our understanding of decision-making in adolescence.

Reputational priors magnify striatal responses to violations of trust.

Humans learn to trust each other by evaluating the outcomes of repeated interpersonal interactions. However, available prior information on the reputation of traders may alter the way outcomes affect learning. Our functional magnetic resonance imaging study is the first to allow the direct comparison of interaction-based and prior-based learning. Twenty participants played repeated trust games with anonymous counterparts. We manipulated two experimental conditions: whether or not reputational priors were provided, and whether counterparts were generally trustworthy or untrustworthy. When no prior information is available our results are consistent with previous studies in showing that striatal activation patterns correlate with behaviorally estimated reinforcement learning measures. However, our study additionally shows that this correlation is disrupted when reputational priors on counterparts are provided. Indeed participants continue to rely on priors even when experience sheds doubt on their accuracy. Notably, violations of trust from a cooperative counterpart elicited stronger caudate deactivations when priors were available than when they were not. However, tolerance to such violations appeared to be mediated by prior-enhanced connectivity between the caudate nucleus and ventrolateral prefrontal cortex, which anticorrelated with retaliation rates. Moreover, on top of affecting learning mechanisms, priors also clearly oriented initial decisions to trust, reflected in medial prefrontal cortex activity.

Emotional valence and the free-energy principle.

The free-energy principle has recently been proposed as a unified Bayesian account of perception, learning and action. Despite the inextricable link between emotion and cognition, emotion has not yet been formulated under this framework. A core concept that permeates many perspectives on emotion is valence, which broadly refers to the positive and negative character of emotion or some of its aspects. In the present paper, we propose a definition of emotional valence in terms of the negative rate of change of free-energy over time. If the second time-derivative of free-energy is taken into account, the dynamics of basic forms of emotion such as happiness, unhappiness, hope, fear, disappointment and relief can be explained. In this formulation, an important function of emotional valence turns out to regulate the learning rate of the causes of sensory inputs. When sensations increasingly violate the agent's expectations, valence is negative and increases the learning rate. Conversely, when sensations increasingly fulfil the agent's expectations, valence is positive and decreases the learning rate. This dynamic interaction between emotional valence and learning rate highlights the crucial role played by emotions in biological agents' adaptation to unexpected changes in their world.

Medial prefrontal cortex and striatum mediate the influence of social comparison on the decision process.

We compared private and social decision making to investigate the neural underpinnings of the effect of social comparison on risky choices. We measured brain activity using functional MRI while participants chose between two lotteries: in the private condition, they observed the outcome of the unchosen lottery, and in the social condition, the outcome of the lottery chosen by another person. The striatum, a reward-related brain structure, showed higher activity when participants won more than their counterpart (social gains) compared with winning in isolation and lower activity when they won less than their counterpart (social loss) compared with private loss. The medial prefrontal cortex, implicated in social reasoning, was more activated by social gains than all other events. Sensitivity to social gains influenced both brain activity and behavior during subsequent choices. Specifically, striatal activity associated with social gains predicted medial prefrontal cortex activity during social choices, and experienced social gains induced more risky and competitive behavior in later trials. These results show that interplay between reward and social reasoning networks mediates the influence of social comparison on the decision process.

Neural correlates of depth of strategic reasoning in medial prefrontal cortex.

We used functional MRI (fMRI) to investigate human mental processes in a competitive interactive setting--the "beauty contest" game. This game is well-suited for investigating whether and how a player's mental processing incorporates the thinking process of others in strategic reasoning. We apply a cognitive hierarchy model to classify subject's choices in the experimental game according to the degree of strategic reasoning so that we can identify the neural substrates of different levels of strategizing. According to this model, high-level reasoners expect the others to behave strategically, whereas low-level reasoners choose based on the expectation that others will choose randomly. The data show that high-level reasoning and a measure of strategic IQ (related to winning in the game) correlate with the neural activity in the medial prefrontal cortex, demonstrating its crucial role in successful mentalizing. This supports a cognitive hierarchy model of human brain and behavior.

Timing of social feedback shapes observational learning in strategic interaction.

Many types of social interaction require the ability to anticipate others' behavior, which is commonly referred to as strategic sophistication. In this context, observational learning can represent a decisive tool for behavioral adaptation. However, little is known on whether and when individuals learn from observation in interactive settings. In the current study, 321 participants played one-shot interactive games and, at a given time along the experiment, they could observe the choices of an overtly efficient player. This social feedback could be provided before or after the participant's choice in each game. Results reveal that players with a sufficient level of strategic skills increased their level of sophistication only when the social feedback was provided after their choices, whereas they relied on blind imitation when they received feedback before their decision. Conversely, less sophisticated players did not increase their level of sophistication, regardless of the type of social feedback. Our findings disclose the interplay between endogenous and exogenous factors modulating observational learning in strategic interaction.

Adolescents' heightened risk-seeking in a probabilistic gambling task.

This study investigated adolescent males' decision-making under risk, and the emotional response to decision outcomes, using a probabilistic gambling task designed to evoke counterfactually mediated emotions (relief and regret). Participants were 20 adolescents (aged 9-11), 26 young adolescents (aged 12-15), 20 mid-adolescents (aged 15-18) and 17 adults (aged 25-35). All were male. The ability to maximize expected value improved with age. However, there was an inverted U-shaped developmental pattern for risk-seeking. The age at which risk-taking was highest was 14.38 years. Although emotion ratings overall did not differ across age, there was an increase between childhood and young adolescence in the strength of counterfactually mediated emotions (relief and regret) reported after receiving feedback about the gamble outcome. We suggest that continuing development of the emotional response to outcomes may be a factor contributing to adolescents' risky behaviour.

An Empirical Test of the Role of Value Certainty in Decision Making.

Most contemporary models of value-based decisions are built on value estimates that are typically self-reported by the decision maker. Such models have been successful in accounting for choice accuracy and response time, and more recently choice confidence. The fundamental driver of such models is choice difficulty, which is almost always defined as the absolute value difference between the subjective value ratings of the options in a choice set. Yet a decision maker is not necessarily able to provide a value estimate with the same degree of certainty for each option that he encounters. We propose that choice difficulty is determined not only by absolute value distance of choice options, but also by their value certainty. In this study, we first demonstrate the reliability of the concept of an option-specific value certainty using three different experimental measures. We then demonstrate the influence that value certainty has on choice, including accuracy (consistency), choice confidence, response time, and choice-induced preference change (i.e., the degree to which value estimates change from pre- to post-choice evaluation). We conclude with a suggestion of how popular contemporary models of choice (e.g., race model, drift-diffusion model) could be improved by including option-specific value certainty as one of their inputs.

Gaze data reveal individual differences in relational representation processes.

In our everyday life, we often need to anticipate the potential occurrence of events and their consequences. In this context, the way we represent contingencies can determine our ability to adapt to the environment. However, it is not clear how agents encode and organize available knowledge about the future to react to possible states of the world. In the present study, we investigated the process of contingency representation with three eye-tracking experiments. In Experiment 1, we introduced a novel relational-inference task in which participants had to learn and represent conditional rules regulating the occurrence of interdependent future events. A cluster analysis on early gaze data revealed the existence of 2 distinct types of encoders. A group of (sophisticated) participants built exhaustive contingency models that explicitly linked states with each of their potential consequences. Another group of (unsophisticated) participants simply learned binary conditional rules without exploring the underlying relational complexity. Analyses of individual cognitive measures revealed that cognitive reflection is associated with the emergence of either sophisticated or unsophisticated representation behavior. In Experiment 2, we observed that unsophisticated participants switched toward the sophisticated strategy after having received information about its existence, suggesting that representation behavior was modulated by strategy generation mechanisms. In Experiment 3, we showed that the heterogeneity in representation strategy emerges also in conditional reasoning with verbal sequences, indicating the existence of a general disposition in building either sophisticated or unsophisticated models of contingencies. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

The differential impact of friendship on cooperative and competitive coordination.

Friendship is commonly assumed to reduce strategic uncertainty and enhance tacit coordination. However, this assumption has never been tested across two opposite poles of coordination involving either strategic complementarity or substitutability. We had participants interact with friends or strangers in two classic coordination games: the stag-hunt game, which exhibits strategic complementarity and may foster "cooperation", and the entry game, which exhibits strategic substitutability and may foster "competition". Both games capture a frequent trade-off between a potentially high paying but uncertain option and a low paying but safe alternative. We find that, relative to strangers, friends are more likely to choose options involving uncertainty in stag-hunt games, but the opposite is true in entry games. Furthermore, in stag-hunt games, friends "tremble" less between options, coordinate better and earn more, but these advantages are largely decreased or lost in entry games. We further investigate how these effects are modulated by risk attitudes, friendship qualities, and interpersonal similarities.

Regret and its avoidance: a neuroimaging study of choice behavior.

Human decisions can be shaped by predictions of emotions that ensue after choosing advantageously or disadvantageously. Indeed, anticipating regret is a powerful predictor of future choices. We measured brain activity using functional magnetic resonance imaging (fMRI) while subjects selected between two gambles wherein regret was induced by providing information about the outcome of the unchosen gamble. Increasing regret enhanced activity in the medial orbitofrontal region, the anterior cingulate cortex and the hippocampus. Notably, across the experiment, subjects became increasingly regret-aversive, a cumulative effect reflected in enhanced activity within medial orbitofrontal cortex and amygdala. This pattern of activity reoccurred just before making a choice, suggesting that the same neural circuitry mediates direct experience of regret and its anticipation. These results demonstrate that medial orbitofrontal cortex modulates the gain of adaptive emotions in a manner that may provide a substrate for the influence of high-level emotions on decision making.

Brain, emotion and decision making: the paradigmatic example of regret.

Human decisions cannot be explained solely by rational imperatives but are strongly influenced by emotion. Theoretical and behavioral studies provide a sound empirical basis to the impact of the emotion of regret in guiding choice behavior. Recent neuropsychological and neuroimaging data have stressed the fundamental role of the orbitofrontal cortex in mediating the experience of regret. Functional magnetic resonance imaging data indicate that reactivation of activity within the orbitofrontal cortex and amygdala occurring during the phase of choice, when the brain is anticipating possible future consequences of decisions, characterizes the anticipation of regret. In turn, these patterns reflect learning based on cumulative emotional experience. Moreover, affective consequences can induce specific mechanisms of cognitive control of the choice processes, involving reinforcement or avoidance of the experienced behavior.

"Betting on nature" or "betting on others": anti-coordination induces uniquely high levels of entropy.

Uncertainty in the form of risk or ambiguity can arise from the interaction with nature and other players, while strategic uncertainty arises only in interactions with others. Here, we systematically compare binary decisions between a safe option and a potentially higher paying but uncertain option in four experimental conditions with the same potential monetary outcomes: coordination vs. anti coordination games, as well as risky and ambiguous lotteries. In each condition, we progressively increase the value of the safe option and measure subjects' certainty equivalents (i.e., the specific safe payoff-threshold that makes a subject indifferent between the two options). We find that anti-coordination games and ambiguous lotteries elicit equally high aversion to uncertainty, relative to the other domains. In spite of this similarity, we find that subjects alternate between the safe and uncertain options much more frequently, thus displaying higher entropy, under anti-coordination relative to any of the other environments. These differences are predicted by theories of recursive reasoning in strategic games (e.g., thinking what others think we think etc.). Indeed, this can occur when interacting with intentional counterparts, but not with nature.

The Role of Intelligence in Social Learning.

Studies in cultural evolution have uncovered many types of social learning strategies that are adaptive in certain environments. The efficiency of these strategies also depends on the individual characteristics of both the observer and the demonstrator. We investigate the relationship between intelligence and the ways social and individual information is utilised to make decisions in an uncertain environment. We measure fluid intelligence and study experimentally how individuals learn from observing the choices of a demonstrator in a 2-armed bandit problem with changing probabilities of a reward. Participants observe a demonstrator with high or low fluid intelligence. In some treatments they are aware of the intelligence score of the demonstrator and in others they are not. Low fluid intelligence individuals imitate the demonstrator more when her fluid intelligence is known than when it is not. Conversely, individuals with high fluid intelligence adjust their use of social information, as the observed behaviour changes, independently of the knowledge of the intelligence of the demonstrator. We provide evidence that intelligence determines how social and individual information is integrated in order to make choices in a changing uncertain environment.

Neural mechanisms mediating degrees of strategic uncertainty.

In social interactions, strategic uncertainty arises when the outcome of one's choice depends on the choices of others. An important question is whether strategic uncertainty can be resolved by assessing subjective probabilities to the counterparts' behavior, as if playing against nature, and thus transforming the strategic interaction into a risky (individual) situation. By means of functional magnetic resonance imaging with human participants we tested the hypothesis that choices under strategic uncertainty are supported by the neural circuits mediating choices under individual risk and deliberation in social settings (i.e. strategic thinking). Participants were confronted with risky lotteries and two types of coordination games requiring different degrees of strategic thinking of the kind 'I think that you think that I think etc.' We found that the brain network mediating risk during lotteries (anterior insula, dorsomedial prefrontal cortex and parietal cortex) is also engaged in the processing of strategic uncertainty in games. In social settings, activity in this network is modulated by the level of strategic thinking that is reflected in the activity of the dorsomedial and dorsolateral prefrontal cortex. These results suggest that strategic uncertainty is resolved by the interplay between the neural circuits mediating risk and higher order beliefs (i.e. beliefs about others' beliefs).

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences.

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

Goal Directed and Self-Control Systems in Bulimia Nervosa: An fMRI Study.

BACKGROUND: Binge eating is apparently the opposite of the strict control over food intake typically set by "maladaptive dieters". Using functional magnetic resonance imaging (fMRI), we investigated the role of goal-directed behaviors, and the related use of self-control, in binge-related food choices in patients with Bulimia Nervosa (BN). METHOD: While undergoing fMRI, women aged 18-35 with BN (N = 35) and healthy control women (N = 26) rated foods for healthiness and tastiness and then made food choices on a 5 points Likert scale between two conflicting options: one food with lower healthiness and higher tastiness (defined as uncontrolled choice) than the other food (defined as controlled choice). RESULTS: BN and healthy participants made more uncontrolled than controlled choices (63% vs 24% and 65% vs 18% respectively). While healthy participants used only food tastiness (chose tastier foods more often) to make food choices (p < .001), BN patients used both food healthiness (chose unhealthy food more often, p < .001) and food tastiness (p < .001) to make binge-related food choices. Activity in the ventromedial prefrontal cortex (vmPFC), which correlated with food choices (pFWE = 0.02), reflected this difference in the integration of food healthiness and food tastiness into a decision value. Functional connectivity analysis showed that the activity in the dorsolateral prefrontal cortex was coupled with vmPFC activity in uncontrolled food choices (pFWE = 0.03). INTERPRETATION: Contrary to what might be expected, not only food tastiness but also unhealthiness (a more abstract cognitive-based attribute than food tastiness) plays a role in uncontrolled choices in BN. These choices are likely goal-directed behaviors and recruit self-control.