The Attraction Effect Modulates Reward Prediction Errors and Intertemporal Choices.

Classical economic theory contends that the utility of a choice option should be independent of other options. This view is challenged by the attraction effect, in which the relative preference between two options is altered by the addition of a third, asymmetrically dominated option. Here, we leveraged the attraction effect in the context of intertemporal choices to test whether both decisions and reward prediction errors (RPE) in the absence of choice violate the independence of irrelevant alternatives principle. We first demonstrate that intertemporal decision making is prone to the attraction effect in humans. In an independent group of participants, we then investigated how this affects the neural and behavioral valuation of outcomes using a novel intertemporal lottery task and fMRI. Participants' behavioral responses (i.e., satisfaction ratings) were modulated systematically by the attraction effect and this modulation was correlated across participants with the respective change of the RPE signal in the nucleus accumbens. Furthermore, we show that, because exponential and hyperbolic discounting models are unable to account for the attraction effect, recently proposed sequential sampling models might be more appropriate to describe intertemporal choices. Our findings demonstrate for the first time that the attraction effect modulates subjective valuation even in the absence of choice. The findings also challenge the prospect of using neuroscientific methods to measure utility in a context-free manner and have important implications for theories of reinforcement learning and delay discounting. SIGNIFICANCE STATEMENT: Many theories of value-based decision making assume that people first assess the attractiveness of each option independently of each other and then pick the option with the highest subjective value. The attraction effect, however, shows that adding a new option to a choice set can change the relative value of the existing options, which is a violation of the independence principle. Using an intertemporal choice framework, we tested whether such violations also occur when the brain encodes the difference between expected and received rewards (i.e., the reward prediction error). Our results suggest that neither intertemporal choice nor valuation without choice adhere to the independence principle.

Correction: Deciding Not to Decide: Computational and Neural Evidence for Hidden Behavior in Sequential Choice.

[This corrects the article DOI: 10.1371/journal.pcbi.1003309.].

Neural evidence for adaptive strategy selection in value-based decision-making.

In everyday life, humans often encounter complex environments in which multiple sources of information can influence their decisions. We propose that in such situations, people select and apply different strategies representing different cognitive models of the decision problem. Learning advances by evaluating the success of using a strategy and eventually by switching between strategies. To test our strategy selection model, we investigated how humans solve a dynamic learning task with complex auditory and visual information, and assessed the underlying neural mechanisms with functional magnetic resonance imaging. Using the model, we were able to capture participants' choices and to successfully attribute expected values and reward prediction errors to activations in the dopaminoceptive system (e.g., ventral striatum [VS]) as well as decision conflict to signals in the anterior cingulate cortex. The model outperformed an alternative approach that did not update decision strategies, but the relevance of information itself. Activation of sensory areas depended on whether the selected strategy made use of the respective source of information. Selection of a strategy also determined how value-related information influenced effective connectivity between sensory systems and the VS. Our results suggest that humans can structure their search for and use of relevant information by adaptively selecting between decision strategies.

Adaptive coding of reward prediction errors is gated by striatal coupling.

To efficiently represent all of the possible rewards in the world, dopaminergic midbrain neurons dynamically adapt their coding range to the momentarily available rewards. Specifically, these neurons increase their activity for an outcome that is better than expected and decrease it for an outcome worse than expected, independent of the absolute reward magnitude. Although this adaptive coding is well documented, it remains unknown how this rescaling is implemented. To investigate the adaptive coding of prediction errors and its underlying rescaling process, we used human functional magnetic resonance imaging (fMRI) in combination with a reward prediction task that involved different reward magnitudes. We demonstrate that reward prediction errors in the human striatum are expressed according to an adaptive coding scheme. Strikingly, we show that adaptive coding is gated by changes in effective connectivity between the striatum and other reward-sensitive regions, namely the midbrain and the medial prefrontal cortex. Our results provide evidence that striatal prediction errors are normalized by a magnitude-dependent alteration in the interregional connectivity within the brain's reward system.

The neural basis of following advice.

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people's learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this "outcome-bonus" accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model's outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice.

Deciding not to decide: computational and neural evidence for hidden behavior in sequential choice.

Understanding the cognitive and neural processes that underlie human decision making requires the successful prediction of how, but also of when, people choose. Sequential sampling models (SSMs) have greatly advanced the decision sciences by assuming decisions to emerge from a bounded evidence accumulation process so that response times (RTs) become predictable. Here, we demonstrate a difficulty of SSMs that occurs when people are not forced to respond at once but are allowed to sample information sequentially: The decision maker might decide to delay the choice and terminate the accumulation process temporarily, a scenario not accounted for by the standard SSM approach. We developed several SSMs for predicting RTs from two independent samples of an electroencephalography (EEG) and a functional magnetic resonance imaging (fMRI) study. In these studies, participants bought or rejected fictitious stocks based on sequentially presented cues and were free to respond at any time. Standard SSM implementations did not describe RT distributions adequately. However, by adding a mechanism for postponing decisions to the model we obtained an accurate fit to the data. Time-frequency analysis of EEG data revealed alternating states of de- and increasing oscillatory power in beta-band frequencies (14-30 Hz), indicating that responses were repeatedly prepared and inhibited and thus lending further support for the existence of a decision not to decide. Finally, the extended model accounted for the results of an adapted version of our paradigm in which participants had to press a button for sampling more information. Our results show how computational modeling of decisions and RTs support a deeper understanding of the hidden dynamics in cognition.

Absolute and relative stability of loss aversion across contexts.

Individuals' decisions under risk tend to be in line with the notion that "losses loom larger than gains." This loss aversion in decision making is commonly understood as a stable individual preference that is manifested across different contexts. The presumed stability and generality, which underlies the prominence of loss aversion in the literature at large, has been recently questioned by studies reporting how loss aversion can disappear, and even reverse, as a function of the choice context. The present study investigated whether loss aversion reflects a trait-like attitude of avoiding losses or rather individuals' adaptability to different contexts. We report three experiments investigating the within-subject context sensitivity of loss aversion in a two-alternative forced-choice task. Our results show that the choice context can shift people's loss aversion, though somewhat inconsistently. Moreover, individual estimates of loss aversion are shown to have a considerable degree of stability. Altogether, these results indicate that even though the absolute value of loss aversion can be affected by external factors such as the choice context, estimates of people's loss aversion still capture the relative dispositions toward gains and losses across individuals. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Deciding when to decide: time-variant sequential sampling models explain the emergence of value-based decisions in the human brain.

The cognitive and neuronal mechanisms of perceptual decision making have been successfully linked to sequential sampling models. These models describe the decision process as a gradual accumulation of sensory evidence over time. The temporal evolution of economic choices, however, remains largely unexplored. We tested whether sequential sampling models help to understand the formation of value-based decisions in terms of behavior and brain responses. We used functional magnetic resonance imaging (fMRI) to measure brain activity while human participants performed a buying task in which they freely decided upon how and when to choose. Behavior was accurately predicted by a time-variant sequential sampling model that uses a decreasing rather than fixed decision threshold to estimate the time point of the decision. Presupplementary motor area, caudate nucleus, and anterior insula activation was associated with the accumulation of evidence over time. Furthermore, at the beginning of the decision process the fMRI signal in these regions accounted for trial-by-trial deviations from behavioral model predictions: relatively high activation preceded relatively early responses. The updating of value information was correlated with signals in the ventromedial prefrontal cortex, left and right orbitofrontal cortex, and ventral striatum but also in the primary motor cortex well before the response itself. Our results support a view of value-based decisions as emerging from sequential sampling of evidence and suggest a close link between the accumulation process and activity in the motor system when people are free to respond at any time.

Classic EEG motor potentials track the emergence of value-based decisions.

Making a value-based decision is a cognitively complex phenomenon and divisible into several sub-processes, such as the perception, evaluation, and final selection of choice options. Although previous research has attempted to dissociate these processes in the brain, there is emerging evidence that late action selection mechanisms are influenced continuously throughout the entire decision act. We used electroencephalography (EEG) and an established sequential decision making paradigm to investigate the extent to which the readiness potential (RP) and the lateralized readiness potential (LRP), two classic preparatory EEG motor components, reflect the ongoing evaluation process in value-based choices. During the task, human participants sequentially sampled probabilistic information to buy or reject offers of unknown value (using both hands) and were allowed to respond at any time. The pressure to respond was manipulated by charging low or high costs for collecting information. We modeled how and when decisions were made and found that participants adaptively lowered their threshold for required evidence with information costs and elapsed time. These shifts were accompanied by an increased RP-like signal during the decision process. The RP was further influenced by the amount of accumulated evidence. In addition, an LRP could be measured from the start of the decision process, well in advance and independent of the final decision. Our results are consistent with a continuous involvement of the brain's motor system in emerging value-based decisions and advocate using classic EEG motor potentials for studying neurocognitive theories of decision making.

Deliberation's blindsight: how cognitive load can improve judgments.

Multitasking poses a major challenge in modern work environments by putting the worker under cognitive load. Performance decrements often occur when people are under high cognitive load because they switch to less demanding--and often less accurate--cognitive strategies. Although cognitive load disturbs performance over a wide range of tasks, it may also carry benefits. In the experiments reported here, we showed that judgment performance can increase under cognitive load. Participants solved a multiple-cue judgment task in which high performance could be achieved by using a similarity-based judgment strategy but not by using a more demanding rule-based judgment strategy. Accordingly, cognitive load induced a shift to a similarity-based judgment strategy, which consequently led to more accurate judgments. By contrast, shifting to a similarity-based strategy harmed judgments in a task best solved by using a rule-based strategy. These results show how important it is to consider the cognitive strategies people rely on to understand how people perform in demanding work environments.

A robust Bayesian test for identifying context effects in multiattribute decision-making.

Research on multiattribute decision-making has repeatedly shown that people's preferences for options depend on the set of other options they are presented with, that is, the choice context. As a result, recent years have seen the development of a number of psychological theories explaining context effects. However, much less attention has been given to the statistical analyses of context effects. Traditionally, context effects are measured as a change in preference for a target option across two different choice sets (the so-called relative choice share of the target, or RST). We first show that the frequently used definition of the RST measure has some weaknesses and should be replaced by a more appropriate definition that we provide. We then show through a large-scale simulation that the RST measure as previously defined can lead to biased inferences. As an alternative, we suggest a Bayesian approach to estimating an accurate RST measure that is robust to various circumstances. We applied the two approaches to the data of five published studies (total participants, N = 738), some of which used the biased approach. Additionally, we introduce the absolute choice share of the target (or AST) as the appropriate measure for the attraction effect. Our approach is an example of evaluating and proposing proper statistical tests for axiomatic principles of decision-making. After applying the AST and the robust RST to published studies, we found qualitatively different results in at least one-fourth of the cases. These results highlight the importance of utilizing robust statistical tests as a foundation for the development of new psychological theories.

The link between cognitive abilities and risk preference depends on measurement.

Risk preference is an important construct for understanding individual differences in risk taking throughout the behavioral sciences. An active stream of research has focused on better understanding risk preference through its connection to other psychological constructs, in particular, cognitive abilities. Here, we examine two large-scale multimethod data sets and demonstrate that the method used to measure risk preference is an important moderator. In self-report measures, we found small but consistent positive correlations between working memory capacity/numeracy, facets of cognitive abilities, and risk tolerance. In behavioral measures, we found, on average, no correlation and large intermethod heterogeneity. This heterogeneity can be explained by the choice architecture that is created in behavioral methods-in particular, the relation between risk and reward and the impact of decision error in a task. Consequently, investigating how risk preference relates to psychological constructs such as cognitive abilities require a profound understanding of the choice architecture in measurements of risk preference and in the real world.

Testing three coping strategies for time pressure in categorizations and similarity judgments.

This article compares three psychological mechanisms to make multi-attribute inferences under time pressure in the domains of categorization and similarity judgments. Specifically, we test if people under time pressure attend to fewer object features (attention focus), if they respond less precisely (lower choice sensitivity), or if they simplify a psychological similarity function (simplified similarity). The simpler psychological similarity considers the number of matching features but ignores the actual feature value differences. We conducted three experiments (two of them preregistered) in which we manipulated time pressure: one was a categorization task, which was designed based on optimal experimental design principles, and the other two involved a similarity judgment task. Computational cognitive modeling following an exemplar-similarity framework showed that the behavior of most participants under time pressure is in line with a lower choice sensitivity, this means less precise response selection, especially when people make similarity judgments. We find that the variability of participants' behavior increases with time pressure, to a point where participants are unlikely to make inferences anymore but instead start choosing readily available response options repeatedly. These findings are consistent with related research in other cognitive domains, such as risky choices, and add to growing evidence that time pressure and other forms of cognitive load do not necessarily alter core cognitive processes themselves but rather affect the precision of response selection.

Neurobiology of value integration: when value impacts valuation.

Everyday choice options have advantages (positive values) and disadvantages (negative values) that need to be integrated into an overall subjective value. For decades, economic models have assumed that when a person evaluates a choice option, different values contribute independently to the overall subjective value of the option. However, human choice behavior often violates this assumption, suggesting interactions between values. To investigate how qualitatively different advantages and disadvantages are integrated into an overall subjective value, we measured the brain activity of human subjects using fMRI while they were accepting or rejecting choice options that were combinations of monetary reward and physical pain. We compared different subjective value models on behavioral and neural data. These models all made similar predictions of choice behavior, suggesting that behavioral data alone are not sufficient to uncover the underlying integration mechanism. Strikingly, a direct model comparison on brain data decisively demonstrated that interactive value integration (where values interact and affect overall valuation) predicts neural activity in value-sensitive brain regions significantly better than the independent mechanism. Furthermore, effective connectivity analyses revealed that value-dependent changes in valuation are associated with modulations in subgenual anterior cingulate cortex-amygdala coupling. These results provide novel insights into the neurobiological underpinnings of human decision making involving the integration of different values.

Social sampling explains apparent biases in judgments of social environments.

How people assess their social environments plays a central role in how they evaluate their life circumstances. Using a large probabilistic national sample, we investigated how accurately people estimate characteristics of the general population. For most characteristics, people seemed to underestimate the quality of others' lives and showed apparent self-enhancement, but for some characteristics, they seemed to overestimate the quality of others' lives and showed apparent self-depreciation. In addition, people who were worse off appeared to enhance their social position more than those who were better off. We demonstrated that these effects can be explained by a simple social-sampling model. According to the model, people infer how others are doing by sampling from their own immediate social environments. Interplay of these sampling processes and the specific structure of social environments leads to the apparent biases. The model predicts the empirical results better than alternative accounts and highlights the importance of considering environmental structure when studying human cognition.

Mine or ours? Neural basis of the exploitation of common-pool resources.

Why do people often exhaust unregulated common (shared) natural resources but manage to preserve similar private resources? To answer this question, in this study we combine a neurobiological, economic and cognitive modeling approach. Using functional magnetic resonance imaging on 50 participants, we show that a sharp decrease of common and private resources is associated with deactivation of the ventral striatum, a brain region involved in the valuation of outcomes. Across individuals, when facing a common resource, ventral striatal activity is anticorrelated with resource preservation (less harvesting), whereas with private resources the opposite pattern is observed. This indicates that neural value signals distinctly modulate behavior in response to the depletion of common vs private resources. Computational modeling suggested that overharvesting of common resources was facilitated by the modulatory effect of social comparison on value signals. These results provide an explanation of people's tendency to over-exploit unregulated common natural resources.

Cognitive abilities affect decision errors but not risk preferences: A meta-analysis.

When making risky decisions, people should evaluate the consequences and the chances of the outcome occurring. We examine the risk-preference hypothesis, which states that people's cognitive abilities affect their evaluation of choice options and consequently their risk-taking behavior. We compared the risk-preference hypothesis against a parsimonious error hypothesis, which states that lower cognitive abilities increase decision errors. Increased decision errors can be misinterpreted as more risk-seeking behavior because in most risk-taking tasks, random choice behavior is often misclassified as risk-seeking behavior. We tested these two competing hypotheses against each other with a systematic literature review and a Bayesian meta-analysis summarizing the empirical correlations. Results based on 30 studies and 62 effect sizes revealed no credible association between cognitive abilities and risk aversion. Apparent correlations between cognitive abilities and risk aversion can be explained by biased risk-preference-elicitation tasks, where more errors are misinterpreted as specific risk preferences. In sum, the reported associations between cognitive abilities and risk preferences are spurious and mediated by a misinterpretation of erroneous choice behavior. This result also has general implications for any research area in which treatment effects, such as decreased cognitive attention or motivation, could increase decision errors and be misinterpreted as specific preference changes.

Empirical underidentification in estimating random utility models: The role of choice sets and standardizations.

A standard approach to distinguishing people's risk preferences is to estimate a random utility model using a power utility function to characterize the preferences and a logit function to capture choice consistency. We demonstrate that with often-used choice situations, this model suffers from empirical underidentification, meaning that parameters cannot be estimated precisely. With simulations of estimation accuracy and Kullback-Leibler divergence measures we examined factors that potentially mitigate this problem. First, using a choice set that guarantees a switch in the utility order between two risky gambles in the range of plausible values leads to higher estimation accuracy than randomly created choice sets or the purpose-built choice sets common in the literature. Second, parameter estimates are regularly correlated, which contributes to empirical underidentification. Examining standardizations of the utility scale, we show that they mitigate this correlation and additionally improve the estimation accuracy for choice consistency. Yet, they can have detrimental effects on the estimation accuracy of risk preference. Finally, we also show how repeated versus distinct choice sets and an increase in observations affect estimation accuracy. Together, these results should help researchers make informed design choices to estimate parameters in the random utility model more precisely.

The influence of sample size on preferences from experience.

People often learn from experience about the distribution of outcomes of risky options. Typically, people draw small samples, when they can actively sample information from risky gambles to make decisions. We examine how the size of the sample that people experience in decision from experience affects their preferences between risky options. In two studies (N = 40 each), we manipulated the size of samples that people could experience from risky gambles and measured subjective selling prices and the confidence in selling price judgements after sampling. The results show that, on average, sample size influenced neither the selling prices nor confidence. However, cognitive modelling of individual-level learning showed that around half of the participants could be classified as Bayesian learners, whereas the other half adhered to a frequentist learning strategy and that if learning was cognitively simpler more participants adhered to the latter. The observed selling prices of Bayesian learners changed with sample size as predicted by Bayesian principles, whereas sample size affected the judgements of frequentist learners much less. These results illustrate the variability in how people learn from sampled information and provide an explanation for why sample size often does not affect judgements.

When easy comes hard: the development of adaptive strategy selection.

Can children learn to select the right strategy for a given problem? In one experiment, 9- to 10-year-olds (N = 50), 11- to 12-year-olds (N = 50), and adults (N = 50) made probabilistic inferences. Participants encountered environments favoring either an information-intensive strategy that integrates all available information or an information-frugal strategy that relies only on the most valid pieces of information. Nine- to 10-year-olds but not older children or adults had more difficulties learning to select an information-frugal strategy than an information-intensive strategy. This counterintuitive finding is explained by children's less developed ability to selectively attend to relevant information, an ability that seems to develop during late childhood. The results suggest that whether a strategy can be considered "easy" depends on the development of specific cognitive abilities.