Behavioral contagion during learning about another agent's risk-preferences acts on the neural representation of decision-risk.

Our attitude toward risk plays a crucial role in influencing our everyday decision-making. Despite its importance, little is known about how human risk-preference can be modulated by observing risky behavior in other agents at either the behavioral or the neural level. Using fMRI combined with computational modeling of behavioral data, we show that human risk-preference can be systematically altered by the act of observing and learning from others' risk-related decisions. The contagion is driven specifically by brain regions involved in the assessment of risk: the behavioral shift is implemented via a neural representation of risk in the caudate nucleus, whereas the representations of other decision-related variables such as expected value are not affected. Furthermore, we uncover neural computations underlying learning about others' risk-preferences and describe how these signals interact with the neural representation of risk in the caudate. Updating of the belief about others' preferences is associated with neural activity in the dorsolateral prefrontal cortex (dlPFC). Functional coupling between the dlPFC and the caudate correlates with the degree of susceptibility to the contagion effect, suggesting that a frontal-subcortical loop, the so-called dorsolateral prefrontal-striatal circuit, underlies the modulation of risk-preference. Taken together, these findings provide a mechanistic account for how observation of others' risky behavior can modulate an individual's own risk-preference.

Neural Mechanisms Behind Identification of Leptokurtic Noise and Adaptive Behavioral Response.

Large-scale human interaction through, for example, financial markets causes ceaseless random changes in outcome variability, producing frequent and salient outliers that render the outcome distribution more peaked than the Gaussian distribution, and with longer tails. Here, we study how humans cope with this evolutionary novel leptokurtic noise, focusing on the neurobiological mechanisms that allow the brain, 1) to recognize the outliers as noise and 2) to regulate the control necessary for adaptive response. We used functional magnetic resonance imaging, while participants tracked a target whose movements were affected by leptokurtic noise. After initial overreaction and insufficient subsequent correction, participants improved performance significantly. Yet, persistently long reaction times pointed to continued need for vigilance and control. We ran a contrasting treatment where outliers reflected permanent moves of the target, as in traditional mean-shift paradigms. Importantly, outliers were equally frequent and salient. There, control was superior and reaction time was faster. We present a novel reinforcement learning model that fits observed choices better than the Bayes-optimal model. Only anterior insula discriminated between the 2 types of outliers. In both treatments, outliers initially activated an extensive bottom-up attention and belief network, followed by sustained engagement of the fronto-parietal control network.

Economic choices reveal probability distortion in macaque monkeys.

Economic choices are largely determined by two principal elements, reward value (utility) and probability. Although nonlinear utility functions have been acknowledged for centuries, nonlinear probability weighting (probability distortion) was only recently recognized as a ubiquitous aspect of real-world choice behavior. Even when outcome probabilities are known and acknowledged, human decision makers often overweight low probability outcomes and underweight high probability outcomes. Whereas recent studies measured utility functions and their corresponding neural correlates in monkeys, it is not known whether monkeys distort probability in a manner similar to humans. Therefore, we investigated economic choices in macaque monkeys for evidence of probability distortion. We trained two monkeys to predict reward from probabilistic gambles with constant outcome values (0.5 ml or nothing). The probability of winning was conveyed using explicit visual cues (sector stimuli). Choices between the gambles revealed that the monkeys used the explicit probability information to make meaningful decisions. Using these cues, we measured probability distortion from choices between the gambles and safe rewards. Parametric modeling of the choices revealed classic probability weighting functions with inverted-S shape. Therefore, the animals overweighted low probability rewards and underweighted high probability rewards. Empirical investigation of the behavior verified that the choices were best explained by a combination of nonlinear value and nonlinear probability distortion. Together, these results suggest that probability distortion may reflect evolutionarily preserved neuronal processing.

Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism.

For making decisions in everyday life we often have first to infer the set of environmental features that are relevant for the current task. Here we investigated the computational mechanisms underlying the evolution of beliefs about the relevance of environmental features in a dynamical and noisy environment. For this purpose we designed a probabilistic Wisconsin card sorting task (WCST) with belief solicitation, in which subjects were presented with stimuli composed of multiple visual features. At each moment in time a particular feature was relevant for obtaining reward, and participants had to infer which feature was relevant and report their beliefs accordingly. To test the hypothesis that attentional focus modulates the belief update process, we derived and fitted several probabilistic and non-probabilistic behavioral models, which either incorporate a dynamical model of attentional focus, in the form of a hierarchical winner-take-all neuronal network, or a diffusive model, without attention-like features. We used Bayesian model selection to identify the most likely generative model of subjects' behavior and found that attention-like features in the behavioral model are essential for explaining subjects' responses. Furthermore, we demonstrate a method for integrating both connectionist and Bayesian models of decision making within a single framework that allowed us to infer hidden belief processes of human subjects.

The human brain encodes event frequencies while forming subjective beliefs.

To make adaptive choices, humans need to estimate the probability of future events. Based on a Bayesian approach, it is assumed that probabilities are inferred by combining a priori, potentially subjective, knowledge with factual observations, but the precise neurobiological mechanism remains unknown. Here, we study whether neural encoding centers on subjective posterior probabilities, and data merely lead to updates of posteriors, or whether objective data are encoded separately alongside subjective knowledge. During fMRI, young adults acquired prior knowledge regarding uncertain events, repeatedly observed evidence in the form of stimuli, and estimated event probabilities. Participants combined prior knowledge with factual evidence using Bayesian principles. Expected reward inferred from prior knowledge was encoded in striatum. BOLD response in specific nodes of the default mode network (angular gyri, posterior cingulate, and medial prefrontal cortex) encoded the actual frequency of stimuli, unaffected by prior knowledge. In this network, activity increased with frequencies and thus reflected the accumulation of evidence. In contrast, Bayesian posterior probabilities, computed from prior knowledge and stimulus frequencies, were encoded in bilateral inferior frontal gyrus. Here activity increased for improbable events and thus signaled the violation of Bayesian predictions. Thus, subjective beliefs and stimulus frequencies were encoded in separate cortical regions. The advantage of such a separation is that objective evidence can be recombined with newly acquired knowledge when a reinterpretation of the evidence is called for. Overall this study reveals the coexistence in the brain of an experience-based system of inference and a knowledge-based system of inference.

Activity in inferior parietal and medial prefrontal cortex signals the accumulation of evidence in a probability learning task.

In an uncertain environment, probabilities are key to predicting future events and making adaptive choices. However, little is known about how humans learn such probabilities and where and how they are encoded in the brain, especially when they concern more than two outcomes. During functional magnetic resonance imaging (fMRI), young adults learned the probabilities of uncertain stimuli through repetitive sampling. Stimuli represented payoffs and participants had to predict their occurrence to maximize their earnings. Choices indicated loss and risk aversion but unbiased estimation of probabilities. BOLD response in medial prefrontal cortex and angular gyri increased linearly with the probability of the currently observed stimulus, untainted by its value. Connectivity analyses during rest and task revealed that these regions belonged to the default mode network. The activation of past outcomes in memory is evoked as a possible mechanism to explain the engagement of the default mode network in probability learning. A BOLD response relating to value was detected only at decision time, mainly in striatum. It is concluded that activity in inferior parietal and medial prefrontal cortex reflects the amount of evidence accumulated in favor of competing and uncertain outcomes.

Evidence for model-based computations in the human amygdala during Pavlovian conditioning.

Contemporary computational accounts of instrumental conditioning have emphasized a role for a model-based system in which values are computed with reference to a rich model of the structure of the world, and a model-free system in which values are updated without encoding such structure. Much less studied is the possibility of a similar distinction operating at the level of Pavlovian conditioning. In the present study, we scanned human participants while they participated in a Pavlovian conditioning task with a simple structure while measuring activity in the human amygdala using a high-resolution fMRI protocol. After fitting a model-based algorithm and a variety of model-free algorithms to the fMRI data, we found evidence for the superiority of a model-based algorithm in accounting for activity in the amygdala compared to the model-free counterparts. These findings support an important role for model-based algorithms in describing the processes underpinning Pavlovian conditioning, as well as providing evidence of a role for the human amygdala in model-based inference.

Not so smart? "Smart" drugs increase the level but decrease the quality of cognitive effort.

The efficacy of pharmaceutical cognitive enhancers in everyday complex tasks remains to be established. Using the knapsack optimization problem as a stylized representation of difficulty in tasks encountered in daily life, we discover that methylphenidate, dextroamphetamine, and modafinil cause knapsack value attained in the task to diminish significantly compared to placebo, even if the chance of finding the optimal solution (~50%) is not reduced significantly. Effort (decision time and number of steps taken to find a solution) increases significantly, but productivity (quality of effort) decreases significantly. At the same time, productivity differences across participants decrease, even reverse, to the extent that above-average performers end up below average and vice versa. The latter can be attributed to increased randomness of solution strategies. Our findings suggest that "smart drugs" increase motivation, but a reduction in quality of effort, crucial to solve complex problems, annuls this effect.

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings.

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.

Task-independent metrics of computational hardness predict human cognitive performance.

The survival of human organisms depends on our ability to solve complex tasks in the face of limited cognitive resources. However, little is known about the factors that drive the complexity of those tasks. Here, building on insights from computational complexity theory, we quantify the computational hardness of cognitive tasks using a set of task-independent metrics related to the computational resource requirements of individual instances of a task. We then examine the relation between those metrics and human behavior and find that they predict both time spent on a task as well as accuracy in three canonical cognitive tasks. Our findings demonstrate that performance in cognitive tasks can be predicted based on generic metrics of their inherent computational hardness.

A behavioral and neural evaluation of prospective decision-making under risk.

Making the best choice when faced with a chain of decisions requires a person to judge both anticipated outcomes and future actions. Although economic decision-making models account for both risk and reward in single-choice contexts, there is a dearth of similar knowledge about sequential choice. Classical utility-based models assume that decision-makers select and follow an optimal predetermined strategy, regardless of the particular order in which options are presented. An alternative model involves continuously reevaluating decision utilities, without prescribing a specific future set of choices. Here, using behavioral and functional magnetic resonance imaging (fMRI) data, we studied human subjects in a sequential choice task and use these data to compare alternative decision models of valuation and strategy selection. We provide evidence that subjects adopt a model of reevaluating decision utilities, in which available strategies are continuously updated and combined in assessing action values. We validate this model by using simultaneously acquired fMRI data to show that sequential choice evokes a pattern of neural response consistent with a tracking of anticipated distribution of future reward, as expected in such a model. Thus, brain activity evoked at each decision point reflects the expected mean, variance, and skewness of possible payoffs, consistent with the idea that sequential choice evokes a prospective evaluation of both available strategies and possible outcomes.

Separate encoding of model-based and model-free valuations in the human brain.

Behavioral studies have long shown that humans solve problems in two ways, one intuitive and fast (System 1, model-free), and the other reflective and slow (System 2, model-based). The neurobiological basis of dual process problem solving remains unknown due to challenges of separating activation in concurrent systems. We present a novel neuroeconomic task that predicts distinct subjective valuation and updating signals corresponding to these two systems. We found two concurrent value signals in human prefrontal cortex: a System 1 model-free reinforcement signal and a System 2 model-based Bayesian signal. We also found a System 1 updating signal in striatal areas and a System 2 updating signal in lateral prefrontal cortex. Further, signals in prefrontal cortex preceded choices that are optimal according to either updating principle, while signals in anterior cingulate cortex and globus pallidus preceded deviations from optimal choice for reinforcement learning. These deviations tended to occur when uncertainty regarding optimal values was highest, suggesting that disagreement between dual systems is mediated by uncertainty rather than conflict, confirming recent theoretical proposals.

The human prefrontal cortex mediates integration of potential causes behind observed outcomes.

Prefrontal cortex has long been implicated in tasks involving higher order inference in which decisions must be rendered, not only about which stimulus is currently rewarded, but also which stimulus dimensions are currently relevant. However, the precise computational mechanisms used to solve such tasks have remained unclear. We scanned human participants with functional MRI, while they performed a hierarchical intradimensional/extradimensional shift task to investigate what strategy subjects use while solving higher order decision problems. By using a computational model-based analysis, we found behavioral and neural evidence that humans solve such problems not by occasionally shifting focus from one to the other dimension, but by considering multiple explanations simultaneously. Activity in human prefrontal cortex was better accounted for by a model that integrates over all available evidences than by a model in which attention is selectively gated. Importantly, our model provides an explanation for how the brain determines integration weights, according to which it could distribute its attention. Our results demonstrate that, at the point of choice, the human brain and the prefrontal cortex in particular are capable of a weighted integration of information across multiple evidences.

Differentiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning.

To understand how the human amygdala contributes to associative learning, it is necessary to differentiate the contributions of its subregions. However, major limitations in the techniques used for the acquisition and analysis of functional magnetic resonance imaging (fMRI) data have hitherto precluded segregation of function with the amygdala in humans. Here, we used high-resolution fMRI in combination with a region-of-interest-based normalization method to differentiate functionally the contributions of distinct subregions within the human amygdala during two different types of instrumental conditioning: reward and avoidance learning. Through the application of a computational-model-based analysis, we found evidence for a dissociation between the contributions of the basolateral and centromedial complexes in the representation of specific computational signals during learning, with the basolateral complex contributing more to reward learning, and the centromedial complex more to avoidance learning. These results provide unique insights into the computations being implemented within fine-grained amygdala circuits in the human brain.

MAOA-L carriers are better at making optimal financial decisions under risk.

Genes can affect behaviour towards risks through at least two distinct neurocomputational mechanisms: they may affect the value assigned to different risky options, or they may affect the way in which the brain adjudicates between options based on their value. We combined methods from neuroeconomics and behavioural genetics to investigate the impact that the genes encoding for monoamine oxidase-A (MAOA), the serotonin transporter (5-HTT) and the dopamine D4 receptor (DRD4) have on these two computations. Consistent with previous literature, we found that carriers of the MAOA-L polymorphism were more likely to take financial risks. Our computational choice model, rooted in established decision theory, showed that MAOA-L carriers exhibited such behaviour because they are able to make better financial decisions under risk, and not because they are more impulsive. In contrast, we found no behavioural or computational differences among the 5-HTT and DRD4 polymorphisms.

Neural correlates of mentalizing-related computations during strategic interactions in humans.

Competing successfully against an intelligent adversary requires the ability to mentalize an opponent's state of mind to anticipate his/her future behavior. Although much is known about what brain regions are activated during mentalizing, the question of how this function is implemented has received little attention to date. Here we formulated a computational model describing the capacity to mentalize in games. We scanned human subjects with functional MRI while they participated in a simple two-player strategy game and correlated our model against the functional MRI data. Different model components captured activity in distinct parts of the mentalizing network. While medial prefrontal cortex tracked an individual's expectations given the degree of model-predicted influence, posterior superior temporal sulcus was found to correspond to an influence update signal, capturing the difference between expected and actual influence exerted. These results suggest dissociable contributions of different parts of the mentalizing network to the computations underlying higher-order strategizing in humans.

How Neurobiology Elucidates the Role of Emotions in Financial Decision-Making.

Over the last 15 years, a revolution has been taking place in neuroscience, whereby models and methods of economics have led to deeper insights into the neurobiological foundations of human decision-making. These have revealed a number of widespread mis-conceptions, among others, about the role of emotions. Furthermore, the findings suggest that a purely behavior-based approach to studying decisions may miss crucial features of human choice long appreciated in biology, such as Pavlovian approach. The findings could help economists formalize elusive concepts such as intuition, as I show here for financial "trading intuition."

Epilepsy and Ecstatic Experiences: The Role of the Insula.

Ecstatic epilepsy is a rare form of focal epilepsy in which the aura (beginning of the seizures) consists of a blissful state of mental clarity/feeling of certainty. Such a state has also been described as a "religious" or mystical experience. While this form of epilepsy has long been recognized as a temporal lobe epilepsy, we have accumulated evidence converging toward the location of the symptomatogenic zone in the dorsal anterior insula during the 10 last years. The neurocognitive hypothesis for the genesis of a mental clarity is the suppression of the interoceptive prediction errors and of the unexpected surprise associated with any incoming internal or external signal, usually processed by the dorsal anterior insula. This mimics a perfect prediction of the world and induces a feeling of certainty. The ecstatic epilepsy is thus an amazing model for the role of anterior insula in uncertainty and surprise.

Transferring cognitive talent across domains to reduce the disposition effect in investment.

We consider Theory of Mind (ToM), the ability to correctly predict the intentions of others. To an important degree, good ToM function requires abstraction from one's own particular circumstances. Here, we posit that such abstraction can be transferred successfully to other, non-social contexts. We consider the disposition effect, which is a pervasive cognitive bias whereby investors, including professionals, improperly take their personal trading history into account when deciding on investments. We design an intervention policy whereby we attempt to transfer good ToM function, subconsciously, to personal investment decisions. In a within-subject repeated-intervention laboratory experiment, we record how the disposition effect is reduced by a very significant 85%, but only for those with high scores on the social-cognitive dimension of ToM function. No such transfer is observed in subjects who score well only on the social-perceptual dimension of ToM function. Our findings open up a promising way to exploit cognitive talent in one domain in order to alleviate cognitive deficiencies elsewhere.

Neural differentiation of expected reward and risk in human subcortical structures.

In decision-making under uncertainty, economic studies emphasize the importance of risk in addition to expected reward. Studies in neuroscience focus on expected reward and learning rather than risk. We combined functional imaging with a simple gambling task to vary expected reward and risk simultaneously and in an uncorrelated manner. Drawing on financial decision theory, we modeled expected reward as mathematical expectation of reward, and risk as reward variance. Activations in dopaminoceptive structures correlated with both mathematical parameters. These activations differentiated spatially and temporally. Temporally, the activation related to expected reward was immediate, while the activation related to risk was delayed. Analyses confirmed that our paradigm minimized confounds from learning, motivation, and salience. These results suggest that the primary task of the dopaminergic system is to convey signals of upcoming stochastic rewards, such as expected reward and risk, beyond its role in learning, motivation, and salience.