Reference-Point Theory: An Account of Individual Differences in Risk Preferences.

We propose an account of individual differences in risk preferences called "reference-point theory" for choices between sure things and gambles. Like most descriptive theories of risky choice, preferences depend on two drivers-hedonic sensitivities to change and beliefs about risk. But unlike most theories, these drivers are estimated from judged feelings about choice options and gamble outcomes. Furthermore, the reference point is assumed to be the less risky option (i.e., sure thing). Loss aversion (greater impact of negative change than positive change) and pessimism (belief the worst outcome is likelier) predict risk aversion. Gain seeking (greater impact of positive change than negative change and optimism (belief the best outcome is likelier) predict risk seeking. But other combinations of hedonic sensitivities and beliefs are possible, and they also predict risk preferences. Finally, feelings about the reference point predict hedonic sensitivities. When decision makers feel good about the reference point, they are frequently loss averse. When they feel bad about it, they are often gain seeking. Three studies show that feelings about reference points, feelings about options and feelings about outcomes predict risky choice and help explain why individuals differ in their risk preferences.

Human and Algorithmic Predictions in Geopolitical Forecasting: Quantifying Uncertainty in Hard-to-Quantify Domains.

Research on clinical versus statistical prediction has demonstrated that algorithms make more accurate predictions than humans in many domains. Geopolitical forecasting is an algorithm-unfriendly domain, with hard-to-quantify data and elusive reference classes that make predictive model-building difficult. Furthermore, the stakes can be high, with missed forecasts leading to mass-casualty consequences. For these reasons, geopolitical forecasting is typically done by humans, even though algorithms play important roles. They are essential as aggregators of crowd wisdom, as frameworks to partition human forecasting variance, and as inputs to hybrid forecasting models. Algorithms are extremely important in this domain. We doubt that humans will relinquish control to algorithms anytime soon-nor do we think they should. However, the accuracy of forecasts will greatly improve if humans are aided by algorithms.

Income and emotional well-being: A conflict resolved.

Do larger incomes make people happier? Two authors of the present paper have published contradictory answers. Using dichotomous questions about the preceding day, [Kahneman and Deaton, Proc. Natl. Acad. Sci. U.S.A. 107, 16489-16493 (2010)] reported a flattening pattern: happiness increased steadily with log(income) up to a threshold and then plateaued. Using experience sampling with a continuous scale, [Killingsworth, Proc. Natl. Acad. Sci. U.S.A. 118, e2016976118 (2021)] reported a linear-log pattern in which average happiness rose consistently with log(income). We engaged in an adversarial collaboration to search for a coherent interpretation of both studies. A reanalysis of Killingsworth's experienced sampling data confirmed the flattening pattern only for the least happy people. Happiness increases steadily with log(income) among happier people, and even accelerates in the happiest group. Complementary nonlinearities contribute to the overall linear-log relationship. We then explain why Kahneman and Deaton overstated the flattening pattern and why Killingsworth failed to find it. We suggest that Kahneman and Deaton might have reached the correct conclusion if they had described their results in terms of unhappiness rather than happiness; their measures could not discriminate among degrees of happiness because of a ceiling effect. The authors of both studies failed to anticipate that increased income is associated with systematic changes in the shape of the happiness distribution. The mislabeling of the dependent variable and the incorrect assumption of homogeneity were consequences of practices that are standard in social science but should be questioned more often. We flag the benefits of adversarial collaboration.

Megastudies improve the impact of applied behavioural science.

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens' decisions and outcomes1. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals2. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy-a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research3-6. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

Conflicting Goals Influence Physicians' Expressed Beliefs to Patients and Colleagues.

BACKGROUND: Physicians who communicate their prognostic beliefs to patients must balance candor against other competing goals, such as preserving hope, acknowledging the uncertainty of medicine, or motivating patients to follow their treatment regimes. OBJECTIVE: To explore possible differences between the beliefs physicians report as their own and those they express to patients and colleagues. DESIGN: An online panel of 398 specialists in internal medicine who completed their medical degrees and practiced in the United States provided their estimated diagnostic accuracy and prognostic assessments for a randomly assigned case. In addition, they reported the diagnostic and prognostic assessments they would report to patients and colleagues more generally. Physicians answered questions about how and why their own beliefs differed from their expressed beliefs to patients and colleagues in the specific case and more generally in their practice. RESULTS: When discussing beliefs about prognoses to patients and colleagues, most physicians expressed beliefs that differed from their own beliefs. Physicians were more likely to express greater optimism when talking to patients about poor prognoses than good prognoses. Physicians were also more likely to express greater uncertainty to patients when prognoses were poor than when they were good. The most common reasons for the differences between physicians' own beliefs and their expressed beliefs were preserving hope and acknowledging the inherent uncertainty of medicine. CONCLUSION: To balance candor against other communicative goals, physicians tended to express beliefs that were more optimistic and contained greater uncertainty than the beliefs they said were their own, especially in discussions with patients whose prognoses were poor.

How perceptions of autonomy relate to beliefs about inequality and fairness.

Although inequality in the US has increased since the 1960s, several studies show that Americans underestimate it. Reasons include overreliance on one's local perspective and ideologically-motivated cognition. We propose a novel mechanism to account for the misperceptions of income inequality. We hypothesize that compared to those who feel less autonomy, the people who believe they are autonomous and have control over their lives also believe that (1) income inequality is lower and (2) income inequality is more acceptable. Using a representative sample of 3,427 Americans, we find evidence to support these hypotheses.

From discipline-centered rivalries to solution-centered science: Producing better probability estimates for policy makers.

From 2011 to 2015, the U.S. intelligence community sponsored a series of forecasting tournaments that challenged university-based researchers to invent measurably better methods of forecasting political events. Our group, the Good Judgment Project, won these tournaments by balancing the collaboration and competition of members across disciplines. At the outset, psychologists were ahead of economists in identifying individual differences in forecasting skill and developing methods of debiasing forecasts, whereas economists were ahead of psychologists in designing simple market mechanisms that distilled predictive signals from noisy individual-level data. Working closely with statisticians, psychologists eventually beat the markets by producing better probability estimates that funneled top forecasters into elite teams and aggregated their judgments using a log-odds formula tuned to the diversity of the forecasters. Our research group performed best when team members strove to get as much as possible from their home disciplines, but acknowledged their limitations and welcomed help from outsiders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Affective responses to uncertain real-world outcomes: Sentiment change on Twitter.

We use data from Twitter.com to study the interplay between affect and expectations about uncertain outcomes. In two studies, we obtained tweets about candidates in the 2014 US Senate elections and tweets about National Football League (NFL) teams in the 2014/2015 NFL season. We chose these events because a) their outcomes are highly uncertain and b) they attract a lot of attention and feature heavily in the communication on social media. We also obtained a priori expectations for the events from political forecasting and sport betting websites. Using this quasi-experimental design, we found that unexpected events are associated with more intense affect than expected events. Moreover, the effect of expectations is larger for outcomes that fall below expectations than outcomes that exceed expectations. Our results are consistent with fundamental principles in psychological science, such as reference-dependence in experienced affect. We discuss how naturally occurring online data can be used to test psychological predictions and develop novel psychological insights.

Forecasting tournaments, epistemic humility and attitude depolarization.

People often express political opinions in starkly dichotomous terms, such as "Trump will either trigger a ruinous trade war or save U.S. factory workers from disaster." This mode of communication promotes polarization into ideological in-groups and out-groups. We explore the power of an emerging methodology, forecasting tournaments, to encourage clashing factions to do something odd: to translate their beliefs into nuanced probability judgments and track accuracy over time and questions. In theory, tournaments advance the goals of "deliberative democracy" by incentivizing people to be flexible belief updaters whose views converge in response to facts, thus depolarizing unnecessarily polarized debates. We examine the hypothesis that, in the process of thinking critically about their beliefs, tournament participants become more moderate in their own political attitudes and those they attribute to the other side. We view tournaments as belonging to a broader class of psychological inductions that increase epistemic humility and that include asking people to explore alternative perspectives, probing the depth of their cause-effect understanding and holding them accountable to audiences with difficult-to-guess views.

Bringing probability judgments into policy debates via forecasting tournaments.

Political debates often suffer from vague-verbiage predictions that make it difficult to assess accuracy and improve policy. A tournament sponsored by the U.S. intelligence community revealed ways in which forecasters can better use probability estimates to make predictions-even for seemingly "unique" events-and showed that tournaments are a useful tool for generating knowledge. Drawing on the literature about the effects of accountability, the authors suggest that tournaments may hold even greater potential as tools for depolarizing political debates and resolving policy disputes.

Anticipated regret in shared decision-making: a randomized experimental study.

BACKGROUND: Explicit consideration of anticipated regret is not part of the standard shared decision-making protocols. This pilot study aimed to compare decisions about a hypothetical surgery for breast cancer and examined whether regret is a consideration in treatment decisions. METHODS: In this randomized experimental study, 184 healthy female volunteers were randomized to receive a standard decision aid (control) or one with information on post-surgical regret (experimental). The main outcome measures were the proportion of subjects choosing lumpectomy vs. mastectomy and the proportion reporting that regret played a role in the decision made. We hypothesized that a greater proportion of the experimental group (regret-incorporated decision aid) would make a surgical treatment preference that favored the less regret-inducing option and that they would be more likely to consider regret in their decision-making process as compared to the control group. RESULTS: A significantly greater proportion of the experimental group subjects reported regret played a role in their decision-making process compared to the control counterparts (78 vs. 65 %; p = 0.039). Recipients of the regret-incorporated experimental decision aid had a threefold increased odds of choosing the less regret-inducing surgery (OR = 2.97; 95 % CI = 1.25, 7.09; p value = 0.014). CONCLUSIONS: In this hypothetical context, the incorporation of regret in a decision aid for preference-sensitive surgery impacted decision-making. This finding suggests that keying in on anticipated regret may be an important element of shared decision-making strategies. Our results make a strong argument for applying this design and pursuing further research in a surgical patient population. TRIAL REGISTRATION: Clinicaltrials.gov, NCT02563808.

Balancing Fairness and Efficiency: The Impact of Identity-Blind and Identity-Conscious Accountability on Applicant Screening.

This study compared two forms of accountability that can be used to promote diversity and fairness in personnel selections: identity-conscious accountability (holding decision makers accountable for which groups are selected) versus identity-blind accountability (holding decision makers accountable for making fair selections). In a simulated application screening process, undergraduate participants (majority female) sorted applicants under conditions of identity-conscious accountability, identity-blind accountability, or no accountability for an applicant pool in which white males either did or did not have a human capital advantage. Under identity-conscious accountability, participants exhibited pro-female and pro-minority bias, particularly in the white-male-advantage applicant pool. Under identity-blind accountability, participants exhibited no biases and candidate qualifications dominated interview recommendations. Participants exhibited greater resentment toward management under identity-conscious accountability.

Improving Intelligence Analysis With Decision Science.

Intelligence analysis plays a vital role in policy decision making. Key functions of intelligence analysis include accurately forecasting significant events, appropriately characterizing the uncertainties inherent in such forecasts, and effectively communicating those probabilistic forecasts to stakeholders. We review decision research on probabilistic forecasting and uncertainty communication, drawing attention to findings that could be used to reform intelligence processes and contribute to more effective intelligence oversight. We recommend that the intelligence community (IC) regularly and quantitatively monitor its forecasting accuracy to better understand how well it is achieving its functions. We also recommend that the IC use decision science to improve these functions (namely, forecasting and communication of intelligence estimates made under conditions of uncertainty). In the case of forecasting, decision research offers suggestions for improvement that involve interventions on data (e.g., transforming forecasts to debias them) and behavior (e.g., via selection, training, and effective team structuring). In the case of uncertainty communication, the literature suggests that current intelligence procedures, which emphasize the use of verbal probabilities, are ineffective. The IC should, therefore, leverage research that points to ways in which verbal probability use may be improved as well as exploring the use of numerical probabilities wherever feasible.

Identifying and cultivating superforecasters as a method of improving probabilistic predictions.

Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters. Defying expectations of regression toward the mean 2 years in a row, superforecasters maintained high accuracy across hundreds of questions and a wide array of topics. We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created-and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance.

The psychology of intelligence analysis: drivers of prediction accuracy in world politics.

This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs.

Psychological strategies for winning a geopolitical forecasting tournament.

Five university-based research groups competed to recruit forecasters, elicit their predictions, and aggregate those predictions to assign the most accurate probabilities to events in a 2-year geopolitical forecasting tournament. Our group tested and found support for three psychological drivers of accuracy: training, teaming, and tracking. Probability training corrected cognitive biases, encouraged forecasters to use reference classes, and provided forecasters with heuristics, such as averaging when multiple estimates were available. Teaming allowed forecasters to share information and discuss the rationales behind their beliefs. Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together. Results showed that probability training, team collaboration, and tracking improved both calibration and resolution. Forecasting is often viewed as a statistical problem, but forecasts can be improved with behavioral interventions. Training, teaming, and tracking are psychological interventions that dramatically increased the accuracy of forecasts. Statistical algorithms (reported elsewhere) improved the accuracy of the aggregation. Putting both statistics and psychology to work produced the best forecasts 2 years in a row.

What might have been? The role of the ventromedial prefrontal cortex and lateral orbitofrontal cortex in counterfactual emotions and choice.

Counterfactual feelings of regret occur when people make comparisons between an actual outcome and a better outcome that would have occurred under a different choice. We investigated the choices of individuals with damage to the ventral medial prefrontal cortex (VMPFC) and the lateral orbital frontal cortex (LOFC) to see whether their emotional responses were sensitive to regret. Participants made choices between gambles, each with monetary outcomes. After every choice, subjects learned the consequences of both gambles and rated their emotional response to the outcome. Normal subjects and lesion control subjects tended to make better choices and reported post-decision emotions that were sensitive to regret comparisons. VMPFC patients tended to make worse choices, and, contrary to our predictions, they reported emotions that were sensitive to regret comparisons. In contrast, LOFC patients made better choices, but reported emotional reactions that were insensitive to regret comparisons. We suggest the VMPFC is involved in the association between choices and anticipated emotions that guide future choices, while the LOFC is involved in experienced emotions that follow choices, emotions that may signal the need for behavioral change.

Surprise: a belief or an emotion?

Surprise is a fundamental link between cognition and emotion. It is shaped by cognitive assessments of likelihood, intuition, and superstition, and it in turn shapes hedonic experiences. We examine this connection between cognition and emotion and offer an explanation called decision affect theory. Our theory predicts the affective consequences of mistaken beliefs, such as overconfidence and hindsight. It provides insight about why the pleasure of a gain can loom larger than the pain of a comparable loss. Finally, it explains cross-cultural differences in emotional reactions to surprising events. By changing the nature of the unexpected (from chance to good luck), one can alter the emotional reaction to surprising events.

Intelligent management of intelligence agencies: beyond accountability ping-pong.

The intelligence community (IC) is asked to predict outcomes that may often be inherently unpredictable-and is blamed for the inevitable forecasting failures, be they false positives or false negatives. To move beyond blame games of accountability ping-pong that incentivize bureaucratic symbolism over substantive reform, it is necessary to reach bipartisan agreements on performance indicators that are transparent enough to reassure clashing elites (to whom the IC must answer) that estimates have not been politicized. Establishing such transideological credibility requires (a) developing accuracy metrics for decoupling probability and value judgments; (b) using the resulting metrics as criterion variables in validity tests of the IC's selection, training, and incentive systems; and (c) institutionalizing adversarial collaborations that conduct level-playing-field tests of clashing perspectives.