The Future of Decisions From Experience: Connecting Real-World Decision Problems to Cognitive Processes.

In many important real-world decision domains, such as finance, the environment, and health, behavior is strongly influenced by experience. Renewed interest in studying this influence led to important advancements in the understanding of these decisions from experience (DfE) in the last 20 years. Building on this literature, we suggest ways the standard experimental design should be extended to better approach important real-world DfE. These extensions include, for example, introducing more complex choice situations, delaying feedback, and including social interactions. When acting upon experiences in these richer and more complicated environments, extensive cognitive processes go into making a decision. Therefore, we argue for integrating cognitive processes more explicitly into experimental research in DfE. These cognitive processes include attention to and perception of numeric and nonnumeric experiences, the influence of episodic and semantic memory, and the mental models involved in learning processes. Understanding these basic cognitive processes can advance the modeling, understanding and prediction of DfE in the laboratory and in the real world. We highlight the potential of experimental research in DfE for theory integration across the behavioral, decision, and cognitive sciences. Furthermore, this research could lead to new methodology that better informs decision-making and policy interventions.

Rare and extreme outcomes in risky choice.

Many real-world decisions involving rare events also involve extreme outcomes. Despite this confluence, decisions-from-experience research has only examined the impact of rarity and extremity in isolation. With rare events, people typically choose as if they underestimate the probability of a rare outcome happening. Separately, people typically overestimate the probability of an extreme outcome happening. Here, for the first time, we examine the confluence of these two biases in decisions-from-experience. In a between-groups behavioural experiment, we examine people's risk preferences for rare extreme outcomes and for rare non-extreme outcomes. When outcomes are both rare and extreme, people's risk preferences shift away from traditional risk patterns for rare events: they show reduced underweighting for events that are both rare and extreme. We simulate these results using a small-sample model of decision-making that accounts for both the underweighting of rare events and the overweighting of extreme events. These separable influences on risk preferences suggest that to understand real-world risk for rare events we must also consider the extremity of the outcomes.

From eye-blinks to state construction: Diagnostic benchmarks for online representation learning.

We present three new diagnostic prediction problems inspired by classical-conditioning experiments to facilitate research in online prediction learning. Experiments in classical conditioning show that animals such as rabbits, pigeons, and dogs can make long temporal associations that enable multi-step prediction. To replicate this remarkable ability, an agent must construct an internal state representation that summarizes its interaction history. Recurrent neural networks can automatically construct state and learn temporal associations. However, the current training methods are prohibitively expensive for online prediction-continual learning on every time step-which is the focus of this paper. Our proposed problems test the learning capabilities that animals readily exhibit and highlight the limitations of the current recurrent learning methods. While the proposed problems are nontrivial, they are still amenable to extensive testing and analysis in the small-compute regime, thereby enabling researchers to study issues in isolation, ultimately accelerating progress towards scalable online representation learning methods.

Biased confabulation in risky choice.

When people make risky decisions based on past experience, they must rely on memory. The nature of the memory representations that support these decisions is not yet well understood. A key question concerns the extent to which people recall specific past episodes or whether they have learned a more abstract rule from their past experience. To address this question, we examined the precision of the memories used in risky decisions-from-experience. In three pre-registered experiments, we presented people with risky options, where the outcomes were drawn from continuous ranges (e.g., 100-190 or 500-590), and then assessed their memories for the outcomes experienced. In two preferential tasks, people were more risk seeking for high-value than low-value options, choosing as though they overweighted the outcomes from more extreme ranges. Moreover, in two preferential tasks and a parallel evaluation task, people were very poor at recalling the exact outcomes encountered, but rather confabulated outcomes that were consistent with the outcomes they had seen and were biased towards the more extreme ranges encountered. This common pattern suggests that the observed decision bias in the preferential task reflects a basic cognitive process to overweight extreme outcomes in memory. These results highlight the importance of the edges of the distribution in providing the encoding context for memory recall. They also suggest that episodic memory influences decision-making through gist memory and not through direct recall of specific instances.

Impact of the "when the fun stops, stop" gambling message on online gambling behaviour: a randomised, online experimental study.

BACKGROUND: Safer gambling messages are a common freedom-preserving method of protecting individuals from gambling-related harm. Yet, there is little independent and rigorous evidence assessing the effectiveness of safer gambling messages. In our study, we aimed to test the effect of the historically most commonly-used UK safer gambling message on concurrent gambling behaviour of people who gamble in the UK. METHODS: In this study, three preregistered, incentivised, and randomised online experiments, testing the UK's "when the fun stops, stop" message, were carried out via the crowdsourcing platform Prolific. Adults based in the UK who had previously participated in the gambling activities relevant to each experiment were eligible to participate. Experiments 1 and 3 involved bets on real soccer events, and experiment 2 used a commercially available online roulette game. Safer gambling message presence was varied between participants in each experiment. In experiment 2, exposed participants could be shown either a yellow or a black-and-white version of the safer gambling message. Participants were provided with a monetary endowment with which they were allowed to bet. Any of this money not bet was afterwards paid to participants as a bonus, in addition to the payouts from any winning bets. In experiment 2 participants had the opportunity to re-wager any winnings from the roulette game. The primary outcome in experiment 1 was participants' decisions to accept (or reject) a series of football bets, which varied in their specificity (and payoffs), and the primary outcomes of experiments 2 and 3 were the proportion of available funds bet, which were defined as the total amount of money bet by a participant out of the total that could have been bet. FINDINGS: Participants for all three experiments were recruited between May 17, 2019, and Oct 17, 2020. Of the 506 participants in experiment 1, 41·3% of available bets were made by the 254 participants in the gambling message condition, which was not significantly different (p=0·15, odds ratio 1·22 [95% CI 0·93 to 1·61]) to the 37·8% of available bets made by the 252 participants in the control condition. In experiment 2, the only credible difference between conditions was that the 501 participants in the condition with the yellow version of the gambling message bet 3·64% (95% Bayesian credibility interval 0·00% to 7·27%) more of available funds left over than the 499 participants in the control condition. There were no credible differences between the bets made by the 500 participants in the black-and-white gambling message condition and the other conditions. In experiment 3, there were no credible differences between the 502 participants in the gambling message condition and the 501 participants in the control condition, with the largest effect being a 5·87% (95% Bayesian credibility interval -1·44% to 13·20%) increase in the probability of betting everything in the gambling message condition. INTERPRETATION: In our study, no evidence was found for a protective effect of the most common UK safer gambling message. Alternative interventions should be considered as part of an evidence-based public health approach to reducing gambling-related harm. FUNDING: University of Warwick, British Academy and Leverhume, Swiss National Science Foundation.

Risk communication improvements for gambling: House-edge information and volatility statements.

OBJECTIVE: Some gambling product messages are designed to inform gamblers about the long-run cost of gambling, for example, "this game has an average percentage payout of 90%." This message is in the "return-to-player" format and is meant to convey that for every £100 bet about £90 will be paid out in prizes. Some previous research has found that restating this information in the "house-edge" format, for example, "this game keeps 10% of all money bet on average," is better understood by gamblers and reduces gamblers' perceived chances of winning. Here we additionally test another potential risk communication improvement: A "volatility statement" highlighting that return-to-player and house-edge percentages are long-run statistical averages, which may not be experienced in any short period of gambling. METHOD: Gambling information format and volatility statement presence were manipulated in an online experiment involving 2,025 U.K. gamblers. RESULTS: The house-edge format and the presence of volatility statements both additively reduced gamblers' perceived chances of winning. In terms of gamblers' understanding, house-edge messages were understood the best, but no consistent effect of volatility statements was observed. CONCLUSIONS: The return-to-player gambling messages in current widespread use can be improved by switching to the house-edge format and via the addition of a volatility statement. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Of tinfoil hats and thinking caps: Reasoning is more strongly related to implausible than plausible conspiracy beliefs.

People who strongly endorse conspiracy theories typically exhibit biases in domain-general reasoning. We describe an overfitting hypothesis, according to which (a) such theories overfit conspiracy-related data at the expense of wider generalisability, and (b) reasoning biases reflect, at least in part, the need to reduce the resulting dissonance between the conspiracy theory and wider data. This hypothesis implies that reasoning biases should be more closely associated with belief in implausible conspiracy theories (e.g., the moon landing was faked) than with more plausible ones (e.g., the Russian Federation orchestrated the attack on Sergei Skripal). In two pre-registered studies, we found that endorsement of implausible conspiracy theories, but not plausible ones, was associated with reduced information sampling in an information-foraging task and with less reflective reasoning. Thus, the relationship between belief in conspiracy theories and reasoning is not homogeneous, and reasoning is not linked specifically to the "conspiracy" aspect of conspiracy theories. Instead, it may reflect an adaptive response to the tension between implausible theories and other beliefs and data.

Encoding Context Determines Risky Choice.

Both memory and choice are influenced by context: Memory is enhanced when encoding and retrieval contexts match, and choice is swayed by available options. Here, we assessed how context influences risky choice in an experience-based task in two main experiments (119 and 98 participants retained, respectively) and two additional experiments reported in the Supplemental Material available online (152 and 106 participants retained, respectively). Within a single session, we created two separate contexts by presenting blocks of trials in distinct backgrounds. Risky choices were context dependent; given the same choice, people chose differently depending on other outcomes experienced in that context. Choices reflected an overweighting of the most extreme outcomes within each local context rather than the global context of all outcomes. When tested in the nontrained context, people chose according to the context at encoding and not retrieval. In subsequent memory tests, people displayed biases specific to distinct contexts: Extreme outcomes from each context were more accessible and judged as more frequent. These results pose a challenge for theories of choice that rely on retrieval as guiding choice.

Reduced risk-seeking in chimpanzees in a zero-outcome game.

A key component of economic decisions is the integration of information about reward outcomes and probabilities in selecting between competing options. In many species, risky choice is influenced by the magnitude of available outcomes, probability of success and the possibility of extreme outcomes. Chimpanzees are generally regarded to be risk-seeking. In this study, we examined two aspects of chimpanzees' risk preferences: first, whether setting the value of the non-preferred outcome of a risky option to zero changes chimpanzees' risk preferences, and second, whether individual risk preferences are stable across two different measures. Across two experiments, we found chimpanzees (Pan troglodytes, n = 23) as a group to be risk-neutral to risk-avoidant with highly stable individual risk preferences. We discuss how the possibility of going empty-handed might reduce chimpanzees' risk-seeking relative to previous studies. This malleability in risk preferences as a function of experimental parameters and individual differences raises interesting questions about whether it is appropriate or helpful to categorize a species as a whole as risk-seeking or risk-avoidant. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Request-a-bet sports betting products indicate patterns of bettor preference and bookmaker profits.

BACKGROUND AND AIMS: Request-a-bet services are a modern gambling product delivered via the social network Twitter, which allow sports bettors to design custom bets. The public nature of Twitter data provided a unique opportunity to investigate patterns of bettor preference and the bookmaker profit margin in soccer, the UK's favorite sport. METHODS: Two multi-method studies. Twitter users' engagement with request-a-bet services was monitored unobtrusively (n = 1,406), meaning that potential patterns across users' requests could be observed, and the bookmaker profit margin could be estimated. Twitter users were also surveyed directly (n = 55), providing self-report measures of request-a-bet usage. RESULTS: Twitter users requested bets with an average potential payoff of £56.5 per £1 risked (median = £9). Overall, 9.7% of requested bets paid-off, but these were mostly bets at short odds. This meant that requests yielded a high bookmaker profit margin of 43.7% (roughly eight times higher than current margins in conventional soccer bets), which increased to 74.6% for bets at longer odds. Requested bets also tended to involve star players from the best teams. Finally, 92.7% of surveyed Twitter users reported placing at least one bet via request-a-bet services (mean = 44.4 bets). DISCUSSION AND CONCLUSIONS: Researchers can use request-a-bet products to increase their understanding of sports betting behavior. Sports bettors should be given information about how much higher the bookmaker profit margin can be in modern sports bets compared to the conventional sports bets that they may be more familiar with.

Reward Devaluation in Autistic Children and Adolescents with Complex Needs: A Feasibility Study.

Rewards act as a motivator for positive behavior and learning. Although compounding evidence indicates that reward processing operates differently in autistic individuals who do not have co-occurring learning disabilities, little is known about individuals who have such difficulties or other complex needs. This study aimed first to assess the feasibility of using an adapted reward devaluation paradigm to examine basic reward processes in this underrepresented population, and second to investigate whether autistic children and adolescents with complex needs would show dynamic behavioral changes in response to changes in the motivational value of a reward. Twenty-seven autistic children and adolescents with complex needs and 20 typically developing 5-year-old children took part in the study. Participants were presented with two visual cues on a touchscreen laptop, which triggered the delivery of a video, music, or physical reward. One of the rewards was then presented in abundance to decrease its motivational value. Participants showed decreased interest in the video and music rewards after devaluation. The experimental setup was found to be suitable to test individuals with complex needs, although recommendations are made for the use of physical rewards. The results suggest that autistic participants with complex needs demonstrate goal-directed behavior and that it is feasible to develop experimental paradigms that can shed important light on learning processes that are fundamental to many education and intervention strategies for this population. Autism Res 2020, 13: 1915-1928. © 2020 The Authors. Autism Research published by International Society for Autism Research and Wiley Periodicals LLC. LAY SUMMARY: We adapted an experimental task to conduct research with autistic children and adolescents with complex needs, who remain grossly underrepresented in autism research. We found that once a reward was presented in great quantity, participants were less motivated to obtain it, showing that they adapted their behavior to changes in the value of that reward. This is an important finding to help promote learning and design better interventions for this population.

Effects of winning cues and relative payout on choice between simulated slot machines.

BACKGROUND AND AIMS: Cues associated with winning may encourage gambling. We assessed the effects on risky choice of slot machine of: (1) neutral sounds paired with winning, (2) casino-related cues (such as the sound of coins dropping and pictures of dollar signs) and (3) relative payouts. DESIGN: Experimental studies in which participants repeatedly chose between safer and riskier simulated slot machines. Safer slot machines paid the same amount regardless of which symbols lined up. Risky machines paid different amounts depending on which symbols lined up. Effects of initially neutral sounds paired with the best payout were assessed between-groups (experiment 1a) and within-participants (experiment 1b). In experiment 2, pairing of casino-related audiovisual cues with payout was assessed within participants, and cue timing was assessed between groups. SETTING: A university research laboratory in Edmonton, Canada. PARTICIPANTS: Undergraduate students (n = 630 across three experiments). MEASUREMENTS: Preference for riskier over safer machines, preference between machines that differed in cues, payout recall and frequency estimates for payouts. Risky choice was calculated as the proportion of choices of the risky machine when presented with a fixed machine of the same expected value. FINDINGS: In experiment 1a, risky choice was slightly increased by pairing a sound with the best payout compared with pairing the sound with a lower payout (P = 0.04, d = 0.28) but not compared with no sound [P = 0.36, d = 0.13, Bayes factors (BF)10  = 0.22]. In experiment 1b, people did not prefer a machine with a best-payout sound over one with a lower-payout sound (P = 0.67, d = 0.03, BF10  = 0.11). Relative payout affected choice: risky choices were higher for high- than low-payout decisions (P < 0.001, d = 0.53). In experiment 2, people preferred machines with casino-related cues paired with winning (P < 0.001, r2  = 0.11) and cue timing (at choice or concurrently with the win) had no effect (P = 0.95, r2  = 0.0, BF10  = 0.05). Casino-related cues also enhanced payout memory (P = 0.013 and 0.006). Cue effects were not specific to risk: people also preferred fixed-payout machines with casino-related cues (P < 0.001, r2  = 0.16). CONCLUSIONS: In a gambling simulation, student participants chose more risky slot machines when payouts were relatively higher and when casino-related cues were associated with payouts. Pairing a neutral sound with the best payout did not consistently affect slot machine choice, and the effect of casino cues did not depend on their timing. Casino-related cues enhanced payout memory.

Equivalent gambling warning labels are perceived differently.

BACKGROUND AND AIMS: The same information may be perceived differently, depending on how it is described. The risk information given on many gambling warning labels tends to accentuate what a gambler might expect to win, e.g. 'This game has an average percentage payout of 90%' (return-to-player), rather than what a gambler might expect to lose, e.g. 'This game keeps 10% of all money bet on average' (house-edge). We compared gamblers' perceived chances of winning and levels of warning label understanding under factually equivalent return-to-player and house-edge formats. DESIGN: Online surveys: experiment 1 was designed to test how gamblers' perceived chances of winning would vary under equivalent warning labels, and experiment 2 explored how often equivalent warning labels were correctly understood by gamblers. SETTING: United Kingdom. PARTICIPANTS: UK nationals, aged 18 years and over and with experience of virtual on-line gambling games, such as on-line roulette, were recruited from an on-line crowd-sourcing panel (experiment 1, n = 399; experiment 2, n = 407). MEASUREMENTS: The main dependent variables were a gambler's perceived chances of winning on a seven-point Likert scale (experiment 1) and a multiple-choice measure of warning label understanding (experiment 2). FINDINGS: The house-edge label led to lower perceived chances of winning in experiment 1, F(1, 388)  = 19.03, P < 0.001. In experiment 2, the house-edge warning label was understood by more gamblers [66.5, 95% confidence interval (CI) = 60.0%, 73.0%] than the return-to-player warning label (45.6%, 95% CI = 38.8%, 52.4%, z = 4.22, P < 0.001). CONCLUSIONS: House-edge warning labels on electronic gambling machines and on-line casino games, which explain what a gambler might expect to lose, could help gamblers to pay greater attention to product risk and would be better understood by gamblers than equivalent return-to-player labels.

The power of nothing: Risk preference in pigeons, but not people, is driven primarily by avoidance of zero outcomes.

Both human and nonhuman animals regularly need to make choices where the outcomes of their actions are unpredictable or probabilistic in some way. These are often termed "risky" choices. Faced with uncertain rewards, people (Homo sapiens) and pigeons (Columba livia) often show similar choice patterns. When the reward probabilities of risky choices are learned through experience, preferences in both species seem to be disproportionately influenced by the extreme (highest and lowest) outcomes in the decision context. Overweighting of these extremes increases preference for risky alternatives that lead to the highest outcome and decreases preference for risky alternatives that lead to the lowest outcome. In a series of studies, we systematically examine how this overweighting of extreme outcomes in risky choice generalizes across 2 evolutionary distant species: pigeons and humans. Both species showed risky choices consistent with an overweighting of extreme outcomes when the low-value risky option could yield an outcome of 0. When all outcome values were increased such that none of the options could lead to 0, people but not pigeons still overweighted the extremes. Unlike people, pigeons no longer avoided a low-value risky option when it yielded a nonzero food outcome. These results suggest that, despite some similarities, different mechanisms underlie risky choice in pigeons and people. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Habits without values.

Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Costly curiosity: People pay a price to resolve an uncertain gamble early.

Humans are inherently curious creatures, continuously seeking out information about future outcomes. Such advance information is often valuable, potentially allowing people to select better courses of action. In non-human animals, this drive for information can be so strong that they forego food or water to find out a few seconds earlier whether an uncertain option will provide a reward. Here, we assess whether people will exhibit a similar sub-optimal preference for advance information. Participants played a card-flipping task where they were probabilistically rewarded based on the pattern of 3 cards that were revealed after a 5-s delay. During this delay, participants could instead pay a cost to find out the next card's identity immediately. This choice to find out early did not influence the eventual outcome. Participants preferred to find out early about 80% of the time when the information was free; they were even willing to incur an expense to get advance information about the eventual outcome. The expected magnitude of the outcome, however, had little impact on the likelihood of finding out early. These results suggest that humans, like animals, value non-instrumental information and will pay a price for such information, independent of its utility.

Comparative inspiration: From puzzles with pigeons to novel discoveries with humans in risky choice.

Both humans and non-human animals regularly encounter decisions involving risk and uncertainty. This paper provides an overview of our research program examining risky decisions in which the odds and outcomes are learned through experience in people and pigeons. We summarize the results of 15 experiments across 8 publications, with a total of over 1300 participants. We highlight 4 key findings from this research: (1) people choose differently when the odds and outcomes are learned through experience compared to when they are described; (2) when making decisions from experience, people overweight values at or near the ends of the distribution of experienced values (i.e., the best and the worst, termed the "extreme-outcome rule"), which leads to more risk seeking for relative gains than for relative losses; (3) people show biases in self-reported memory whereby they are more likely to report an extreme outcome than an equally-often experienced non-extreme outcome, and they judge these extreme outcomes as having occurred more often; and (4) under certain circumstances pigeons show similar patterns of risky choice as humans, but the underlying processes may not be identical. This line of research has stimulated other research in the field of judgement and decision making, illustrating how investigations from a comparative perspective can lead in surprising directions.

Living near the edge: How extreme outcomes and their neighbors drive risky choice.

Extreme stimuli are often more salient in perception and memory than moderate stimuli. In risky choice, when people learn the odds and outcomes from experience, the extreme outcomes (best and worst) also stand out. This additional salience leads to more risk-seeking for relative gains than for relative losses-the opposite of what people do when queried in terms of explicit probabilities. Previous research has suggested that this pattern arises because the most extreme experienced outcomes are more prominent in memory. An important open question, however, is what makes these extreme outcomes more prominent? Here we assess whether extreme outcomes stand out because they fall at the edges of the experienced outcome distributions or because they are distinct from other outcomes. Across four experiments, proximity to the edge determined what was treated as extreme: Outcomes at or near the edge of the outcome distribution were both better remembered and more heavily weighted in choice. This prominence did not depend on two metrics of distinctiveness: lower frequency or distance from other outcomes. This finding adds to evidence from other domains that the values at the edges of a distribution have a special role. (PsycINFO Database Record (c) 2018 APA, all rights reserved).