Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability.

How much we like something, whether it be a bottle of wine or a new film, is affected by the opinions of others. However, the social information that we receive can be contradictory and vary in its reliability. Here, we tested whether the brain incorporates these statistics when judging value and confidence. Participants provided value judgments about consumer goods in the presence of online reviews. We found that participants updated their initial value and confidence judgments in a Bayesian fashion, taking into account both the uncertainty of their initial beliefs and the reliability of the social information. Activity in dorsomedial prefrontal cortex tracked the degree of belief update. Analogous to how lower-level perceptual information is integrated, we found that the human brain integrates social information according to its reliability when judging value and confidence.SIGNIFICANCE STATEMENT The field of perceptual decision making has shown that the sensory system integrates different sources of information according to their respective reliability, as predicted by a Bayesian inference scheme. In this work, we hypothesized that a similar coding scheme is implemented by the human brain to process social signals and guide complex, value-based decisions. We provide experimental evidence that the human prefrontal cortex's activity is consistent with a Bayesian computation that integrates social information that differs in reliability and that this integration affects the neural representation of value and confidence.

A range-normalization model of context-dependent choice: a new model and evidence.

Most utility theories of choice assume that the introduction of an irrelevant option (called the decoy) to a choice set does not change the preference between existing options. On the contrary, a wealth of behavioral data demonstrates the dependence of preference on the decoy and on the context in which the options are presented. Nevertheless, neural mechanisms underlying context-dependent preference are poorly understood. In order to shed light on these mechanisms, we design and perform a novel experiment to measure within-subject decoy effects. We find within-subject decoy effects similar to what have been shown previously with between-subject designs. More importantly, we find that not only are the decoy effects correlated, pointing to similar underlying mechanisms, but also these effects increase with the distance of the decoy from the original options. To explain these observations, we construct a plausible neuronal model that can account for decoy effects based on the trial-by-trial adjustment of neural representations to the set of available options. This adjustment mechanism, which we call range normalization, occurs when the nervous system is required to represent different stimuli distinguishably, while being limited to using bounded neural activity. The proposed model captures our experimental observations and makes new predictions about the influence of the choice set size on the decoy effects, which are in contrast to previous models of context-dependent choice preference. Critically, unlike previous psychological models, the computational resource required by our range-normalization model does not increase exponentially as the set size increases. Our results show that context-dependent choice behavior, which is commonly perceived as an irrational response to the presence of irrelevant options, could be a natural consequence of the biophysical limits of neural representation in the brain.

Amygdala damage eliminates monetary loss aversion.

Losses are a possibility in many risky decisions, and organisms have evolved mechanisms to evaluate and avoid them. Laboratory and field evidence suggests that people often avoid risks with losses even when they might earn a substantially larger gain, a behavioral preference termed "loss aversion." The cautionary brake on behavior known to rely on the amygdala is a plausible candidate mechanism for loss aversion, yet evidence for this idea has so far not been found. We studied two rare individuals with focal bilateral amygdala lesions using a series of experimental economics tasks. To measure individual sensitivity to financial losses we asked participants to play a variety of monetary gambles with possible gains and losses. Although both participants retained a normal ability to respond to changes in the gambles' expected value and risk, they showed a dramatic reduction in loss aversion compared to matched controls. The findings suggest that the amygdala plays a key role in generating loss aversion by inhibiting actions with potentially deleterious outcomes.

Goals, usefulness and abstraction in value-based choice.

Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.

Humans actively sample evidence to support prior beliefs.

No one likes to be wrong. Previous research has shown that participants may underweight information incompatible with previous choices, a phenomenon called confirmation bias. In this paper, we argue that a similar bias exists in the way information is actively sought. We investigate how choice influences information gathering using a perceptual choice task and find that participants sample more information from a previously chosen alternative. Furthermore, the higher the confidence in the initial choice, the more biased information sampling becomes. As a consequence, when faced with the possibility of revising an earlier decision, participants are more likely to stick with their original choice, even when incorrect. Critically, we show that agency controls this phenomenon. The effect disappears in a fixed sampling condition where presentation of evidence is controlled by the experimenter, suggesting that the way in which confirmatory evidence is acquired critically impacts the decision process. These results suggest active information acquisition plays a critical role in the propagation of strongly held beliefs over time.

Value signals guide abstraction during learning.

The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules based on simple visual features. Reinforcement-learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participant behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals - the ventromedial prefrontal cortex - prioritised and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such a coding scheme predicts a causal role for valuation. Hence, in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex, as a mechanism of abstraction. Tagging the neural representation of a task feature with rewards evoked abstraction-based decisions. Together, these findings provide a novel interpretation of value as a goal-dependent, key factor in forging abstract representations.

How choice reveals and shapes expected hedonic outcome.

Humans tend to modify their attitudes to align with past action. For example, after choosing between similarly valued alternatives, people rate the selected option as better than they originally did, and the rejected option as worse. However, it is unknown whether these modifications in evaluation reflect an underlying change in the physiological representation of a stimulus' expected hedonic value and our emotional response to it. Here, we addressed this question by combining participants' estimations of the pleasure they will derive from future events, with brain imaging data recorded while they imagined those events, both before, and after, choosing between them. Participants rated the selected alternatives as better after the decision stage relative to before, whereas discarded alternatives were valued less. Our functional magnetic resonance imaging findings reveal that postchoice changes in preference are tracked in caudate nucleus activity. Specifically, the difference in blood oxygenation level-dependent (BOLD) signal associated with the selected and rejected stimuli was enhanced after a decision was taken, reflecting the choice that had just been made. This finding suggests that the physiological representation of a stimulus' expected hedonic value is altered by a commitment to it. Furthermore, before any revaluation induced by the decision process, our data show that BOLD signal in this same region reflects the choices we are likely to make at a later time.

The neurobiology of reference-dependent value computation.

A key focus of current research in neuroeconomics concerns how the human brain computes value. Although, value has generally been viewed as an absolute measure (e.g., expected value, reward magnitude), much evidence suggests that value is more often computed with respect to a changing reference point, rather than in isolation. Here, we present the results of a study aimed to dissociate brain regions involved in reference-independent (i.e., "absolute") value computations, from those involved in value computations relative to a reference point. During functional magnetic resonance imaging, subjects acted as buyers and sellers during a market exchange of lottery tickets. At a behavioral level, we demonstrate that subjects systematically accorded a higher value to objects they owned relative to those they did not, an effect that results from a shift in reference point (i.e., status quo bias or endowment effect). Our results show that activity in orbitofrontal cortex and dorsal striatum track parameters such as the expected value of lottery tickets indicating the computation of reference-independent value. In contrast, activity in ventral striatum indexed the degree to which stated prices, at a within-subjects and between-subjects level, were distorted with respect to a reference point. The findings speak to the neurobiological underpinnings of reference dependency during real market value computations.

A genetically mediated bias in decision making driven by failure of amygdala control.

Genetic variation at the serotonin transporter-linked polymorphic region (5-HTTLPR) is associated with altered amygdala reactivity and lack of prefrontal regulatory control. Similar regions mediate decision-making biases driven by contextual cues and ambiguity, for example the "framing effect." We hypothesized that individuals hemozygous for the short (s) allele at the 5-HTTLPR would be more susceptible to framing. Participants, selected as homozygous for either the long (la) or s allele, performed a decision-making task where they made choices between receiving an amount of money for certain and taking a gamble. A strong bias was evident toward choosing the certain option when the option was phrased in terms of gains and toward gambling when the decision was phrased in terms of losses (the frame effect). Critically, this bias was significantly greater in the ss group compared with the lala group. In simultaneously acquired functional magnetic resonance imaging data, the ss group showed greater amygdala during choices made in accord, compared with those made counter to the frame, an effect not seen in the lala group. These differences were also mirrored by differences in anterior cingulate-amygdala coupling between the genotype groups during decision making. Specifically, lala participants showed increased coupling during choices made counter to, relative to those made in accord with, the frame, with no such effect evident in ss participants. These data suggest that genetically mediated differences in prefrontal-amygdala interactions underpin interindividual differences in economic decision making.

Enhanced processing of threat stimuli under limited attentional resources.

The ability to process stimuli that convey potential threat, under conditions of limited attentional resources, confers adaptive advantages. This study examined the neurobiology underpinnings of this capacity. Employing an attentional blink paradigm, in conjunction with functional magnetic resonance imaging, we manipulated the salience of the second of 2 face target stimuli (T2), by varying emotionality. Behaviorally, fearful T2 faces were identified significantly more than neutral faces. Activity in fusiform face area increased with correct identification of T2 faces. Enhanced activity in rostral anterior cingulate cortex (rACC) accounted for the benefit in detection of fearful stimuli reflected in a significant interaction between target valence and correct identification. Thus, under conditions of limited attention resources activation in rACC correlated with enhanced processing of emotional stimuli. We suggest that these data support a model in which a prefrontal "gate" mechanism controls conscious access of emotional information under conditions of limited attentional resources.

The NMDA agonist D-cycloserine facilitates fear memory consolidation in humans.

Animal research suggests that the consolidation of fear and extinction memories depends on N-methyl D-aspartate (NMDA)-type glutamate receptors. Using a fear conditioning and extinction paradigm in healthy normal volunteers, we show that postlearning administration of the NMDA partial agonist D-cycloserine (DCS) facilitates fear memory consolidation, evidenced behaviorally by enhanced skin conductance responses, relative to placebo, for presentations of a conditioned stimulus (CS) at a memory test performed 72 h later. DCS also enhanced CS-evoked neural responses in a posterior hippocampus/collateral sulcus region and in the medial prefrontal cortex at test. Our data suggest a role for NMDA receptors in regulating fear memory consolidation in humans.

Visual attention modulates the integration of goal-relevant evidence and not value.

When choosing between options, such as food items presented in plain view, people tend to choose the option they spend longer looking at. The prevailing interpretation is that visual attention increases value. However, in previous studies, 'value' was coupled to a behavioural goal, since subjects had to choose the item they preferred. This makes it impossible to discern if visual attention has an effect on value, or, instead, if attention modulates the information most relevant for the goal of the decision-maker. Here, we present the results of two independent studies-a perceptual and a value-based task-that allow us to decouple value from goal-relevant information using specific task-framing. Combining psychophysics with computational modelling, we show that, contrary to the current interpretation, attention does not boost value, but instead it modulates goal-relevant information. This work provides a novel and more general mechanism by which attention interacts with choice.

Explaining enhanced logical consistency during decision making in autism.

The emotional responses elicited by the way options are framed often results in lack of logical consistency in human decision making. In this study, we investigated subjects with autism spectrum disorder (ASD) using a financial task in which the monetary prospects were presented as either loss or gain. We report both behavioral evidence that ASD subjects show a reduced susceptibility to the framing effect and psycho-physiological evidence that they fail to incorporate emotional context into the decision-making process. On this basis, we suggest that this insensitivity to contextual frame, although enhancing choice consistency in ASD, may also underpin core deficits in this disorder. These data highlight both benefits and costs arising from multiple decision processes in human cognition.

Blocking central opiate function modulates hedonic impact and anterior cingulate response to rewards and losses.

Reward processing is linked to specific neuromodulatory systems with a dopaminergic contribution to reward learning and motivational drive being well established. Neuromodulatory influences on hedonic responses to actual receipt of reward, or punishment, referred to as experienced utility are less well characterized, although a link to the endogenous opioid system is suggested. Here, in a combined functional magnetic resonance imaging-psychopharmacological investigation, we used naloxone to block central opioid function while subjects performed a gambling task associated with rewards and losses of different magnitudes, in which the mean expected value was always zero. A graded influence of naloxone on reward outcome was evident in an attenuation of pleasure ratings for larger reward outcomes, an effect mirrored in attenuation of brain activity to increasing reward magnitude in rostral anterior cingulate cortex. A more striking effect was seen for losses such that under naloxone all levels of negative outcome were rated as more unpleasant. This hedonic effect was associated with enhanced activity in anterior insula and caudal anterior cingulate cortex, areas implicated in aversive processing. Our data indicate that a central opioid system contributes to both reward and loss processing in humans and directly modulates the hedonic experience of outcomes.

Confidence modulates exploration and exploitation in value-based learning.

Uncertainty is ubiquitous in cognitive processing. In this study, we aim to investigate the ability agents possess to track and report the noise inherent in their mental operations, often in the form of confidence judgments. Here, we argue that humans can use uncertainty inherent in their representations of value beliefs to arbitrate between exploration and exploitation. Such uncertainty is reflected in explicit confidence judgments. Using a novel variant of a multi-armed bandit paradigm, we studied how beliefs were formed and how uncertainty in the encoding of these value beliefs (belief confidence) evolved over time. We found that people used uncertainty to arbitrate between exploration and exploitation, reflected in a higher tendency toward exploration when their confidence in their value representations was low. We furthermore found that value uncertainty can be linked to frameworks of metacognition in decision making in two ways. First, belief confidence drives decision confidence, i.e. people's evaluation of their own choices. Second, individuals with higher metacognitive insight into their choices were also better at tracing the uncertainty in their environment. Together, these findings argue that such uncertainty representations play a key role in the context of cognitive control.

The neural and cognitive architecture for learning from a small sample.

Artificial intelligence algorithms are capable of fantastic exploits, yet they are still grossly inefficient compared with the brain's ability to learn from few exemplars or solve problems that have not been explicitly defined. What is the secret that the evolution of human intelligence has unlocked? Generalization is one answer, but there is more to it. The brain does not directly solve difficult problems, it is able to recast them into new and more tractable problems. Here, we propose a model whereby higher cognitive functions profoundly interact with reinforcement learning to drastically reduce the degrees of freedom of the search space, simplifying complex problems, and fostering more efficient learning.

Noradrenergic neuromodulation of human attention for emotional and neutral stimuli.

INTRODUCTION: Norepinephrine (NE) has a regulatory role in human attention. OBJECTIVE: To examine its role in emotional modulation of attention, we used an attentional blink (AB) paradigm, in the context of psychopharmacological manipulation, where targets were either emotional or neutral items. RESULTS AND DISCUSSION: We report behavioural evidence that beta-adrenergic blockade with propranolol impairs attention independent of target valence. Furthermore, this effect is centrally mediated as administration of the peripheral beta-adrenergic antagonist nadolol did not impair attention. By contrast, increasing NE tone, using the selective NE reuptake inhibitor reboxetine, improves detection of emotional stimuli. CONCLUSION: In line with theoretical and animal models, these findings provide human behavioural evidence that the adrenergic system has a modulatory influence on selective attention that in some instances depends on item valence.

Prior preferences beneficially influence social and non-social learning.

Our personal preferences affect a broad array of social behaviors. This includes the way we learn the preferences of others, an ability that often relies on limited or ambiguous information. Here we report an egocentric influence on this type of social learning that is reflected in both performance and response times. Using computational models that combine inter-trial learning and intra-trial choice, we find transient effects of participants' preferences on the learning process, through the influence of priors, and persistent effects on the choice process. A second experiment shows that these effects generalize to non-social learning, though participants in the social learning experiment appeared to additionally benefit by using their knowledge about the popularity of certain preferences. We further find that the domain-general egocentric influences we identify can yield performance advantages in uncertain environments.People often assume that other people share their preferences, but how exactly this bias manifests itself in learning and decision-making is unclear. Here, authors show that a person's own preferences influence learning in both social and non-social situations, and that this bias improves performance.

Compulsivity Reveals a Novel Dissociation between Action and Confidence.

Confidence and actions are normally tightly interwoven-if I am sure that it is going to rain, I will take an umbrella-therefore, it is difficult to understand their interplay. Stimulated by the ego-dystonic nature of obsessive-compulsive disorder (OCD), where compulsive actions are recognized as disproportionate, we hypothesized that action and confidence might be independently updated during learning. Participants completed a predictive-inference task designed to identify how action and confidence evolve in response to surprising changes in the environment. While OCD patients (like controls) correctly updated their confidence according to changes in the environment, their actions (unlike those of controls) mostly disregarded this knowledge. Therefore, OCD patients develop an accurate, internal model of the environment but fail to use it to guide behavior. Results demonstrated a novel dissociation between confidence and action, suggesting a cognitive architecture whereby confidence estimates can accurately track the statistic of the environment independently from performance.