A synthesis of evidence for policy from behavioural science during COVID-19.

Scientific evidence regularly guides policy decisions1, with behavioural science increasingly part of this process2. In April 2020, an influential paper3 proposed 19 policy recommendations ('claims') detailing how evidence from behavioural science could contribute to efforts to reduce impacts and end the COVID-19 pandemic. Here we assess 747 pandemic-related research articles that empirically investigated those claims. We report the scale of evidence and whether evidence supports them to indicate applicability for policymaking. Two independent teams, involving 72 reviewers, found evidence for 18 of 19 claims, with both teams finding evidence supporting 16 (89%) of those 18 claims. The strongest evidence supported claims that anticipated culture, polarization and misinformation would be associated with policy effectiveness. Claims suggesting trusted leaders and positive social norms increased adherence to behavioural interventions also had strong empirical support, as did appealing to social consensus or bipartisan agreement. Targeted language in messaging yielded mixed effects and there were no effects for highlighting individual benefits or protecting others. No available evidence existed to assess any distinct differences in effects between using the terms 'physical distancing' and 'social distancing'. Analysis of 463 papers containing data showed generally large samples; 418 involved human participants with a mean of 16,848 (median of 1,699). That statistical power underscored improved suitability of behavioural science research for informing policy decisions. Furthermore, by implementing a standardized approach to evidence selection and synthesis, we amplify broader implications for advancing scientific evidence in policy formulation and prioritization.

Shifting attention to accuracy can reduce misinformation online.

In recent years, there has been a great deal of concern about the proliferation of false and misleading news on social media1-4. Academics and practitioners alike have asked why people share such misinformation, and sought solutions to reduce the sharing of misinformation5-7. Here, we attempt to address both of these questions. First, we find that the veracity of headlines has little effect on sharing intentions, despite having a large effect on judgments of accuracy. This dissociation suggests that sharing does not necessarily indicate belief. Nonetheless, most participants say it is important to share only accurate news. To shed light on this apparent contradiction, we carried out four survey experiments and a field experiment on Twitter; the results show that subtly shifting attention to accuracy increases the quality of news that people subsequently share. Together with additional computational analyses, these findings indicate that people often share misinformation because their attention is focused on factors other than accuracy-and therefore they fail to implement a strongly held preference for accurate sharing. Our results challenge the popular claim that people value partisanship over accuracy8,9, and provide evidence for scalable attention-based interventions that social media platforms could easily implement to counter misinformation online.

On the Efficacy of Accuracy Prompts Across Partisan Lines: An Adversarial Collaboration.

The spread of misinformation is a pressing societal challenge. Prior work shows that shifting attention to accuracy increases the quality of people's news-sharing decisions. However, researchers disagree on whether accuracy-prompt interventions work for U.S. Republicans/conservatives and whether partisanship moderates the effect. In this preregistered adversarial collaboration, we tested this question using a multiverse meta-analysis (k = 21; N = 27,828). In all 70 models, accuracy prompts improved sharing discernment among Republicans/conservatives. We observed significant partisan moderation for single-headline "evaluation" treatments (a critical test for one research team) such that the effect was stronger among Democrats than Republicans. However, this moderation was not consistently robust across different operationalizations of ideology/partisanship, exclusion criteria, or treatment type. Overall, we observed significant partisan moderation in 50% of specifications (all of which were considered critical for the other team). We discuss the conditions under which moderation is observed and offer interpretations.

Timing matters when correcting fake news.

Countering misinformation can reduce belief in the moment, but corrective messages quickly fade from memory. We tested whether the longer-term impact of fact-checks depends on when people receive them. In two experiments (total N = 2,683), participants read true and false headlines taken from social media. In the treatment conditions, "true" and "false" tags appeared before, during, or after participants read each headline. Participants in a control condition received no information about veracity. One week later, participants in all conditions rated the same headlines' accuracy. Providing fact-checks after headlines (debunking) improved subsequent truth discernment more than providing the same information during (labeling) or before (prebunking) exposure. This finding informs the cognitive science of belief revision and has practical implications for social media platform designers.

Deliberation is (probably) triggered and sustained by multiple mechanisms.

De Neys proposes that deliberation is triggered and sustained by uncertainty. I argue that there are cases where deliberation occurs with low uncertainty - such as when problems are excessively complicated and the reasoner decides against engaging in deliberation - and that there are likely multiple factors that lead to (or undermine) deliberation. Nonetheless, De Neys is correct to surface these issues.

Fighting misinformation on social media using crowdsourced judgments of news source quality.

Reducing the spread of misinformation, especially on social media, is a major challenge. We investigate one potential approach: having social media platform algorithms preferentially display content from news sources that users rate as trustworthy. To do so, we ask whether crowdsourced trust ratings can effectively differentiate more versus less reliable sources. We ran two preregistered experiments (n = 1,010 from Mechanical Turk and n = 970 from Lucid) where individuals rated familiarity with, and trust in, 60 news sources from three categories: (i) mainstream media outlets, (ii) hyperpartisan websites, and (iii) websites that produce blatantly false content ("fake news"). Despite substantial partisan differences, we find that laypeople across the political spectrum rated mainstream sources as far more trustworthy than either hyperpartisan or fake news sources. Although this difference was larger for Democrats than Republicans-mostly due to distrust of mainstream sources by Republicans-every mainstream source (with one exception) was rated as more trustworthy than every hyperpartisan or fake news source across both studies when equally weighting ratings of Democrats and Republicans. Furthermore, politically balanced layperson ratings were strongly correlated (r = 0.90) with ratings provided by professional fact-checkers. We also found that, particularly among liberals, individuals higher in cognitive reflection were better able to discern between low- and high-quality sources. Finally, we found that excluding ratings from participants who were not familiar with a given news source dramatically reduced the effectiveness of the crowd. Our findings indicate that having algorithms up-rank content from trusted media outlets may be a promising approach for fighting the spread of misinformation on social media.

Nudging Social Media toward Accuracy.

A meaningful portion of online misinformation sharing is likely attributable to Internet users failing to consider accuracy when deciding what to share. As a result, simply redirecting attention to the concept of accuracy can increase sharing discernment. Here we discuss the importance of accuracy and describe a limited-attention utility model that is based on a theory about inattention to accuracy on social media. We review research that shows how a simple nudge or prompt that shifts attention to accuracy increases the quality of news that people share (typically by decreasing the sharing of false content), and then discuss outstanding questions relating to accuracy nudges, including the need for more work relating to persistence and habituation as well as the dearth of cross-cultural research on these topics. We also make several recommendations for policy-makers and social media companies for how to implement accuracy nudges.

Character deprecation in fake news: Is it in supply or demand?

A major focus of current research is understanding why people fall for and share fake news on social media. While much research focuses on understanding the role of personality-level traits for those who share the news, such as partisanship and analytic thinking, characteristics of the articles themselves have not been studied. Across two pre-registered studies, we examined whether character-deprecation headlines - headlines designed to deprecate someone's character, but which have no impact on policy or legislation - increased the likelihood of self-reported sharing on social media. In Study 1 we harvested fake news items from online sources and compared sharing intentions between Republicans and Democrats. Results showed that, compared to Democrats, Republicans had greater intention to share character-deprecation headlines compared to news with policy implications. We then applied these findings experimentally. In Study 2 we developed a set of fake news items that was matched for content across pro-Democratic and pro-Republican headlines and across news focusing on a specific person (e.g., Trump) versus a generic person (e.g., a Republican). We found that, contrary to Study 1, Republicans were no more inclined toward character deprecation than Democrats. However, these findings suggest that while character assassination may be a feature of pro-Republican news, it is not more attractive to Republicans versus Democrats. News with policy implications, whether fake or real, seems consistently more attractive to members of both parties regardless of whether it attempts to deprecate an opponent's character. Thus, character deprecation in fake news may in be in supply, but not in demand.

Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention.

Across two studies with more than 1,700 U.S. adults recruited online, we present evidence that people share false claims about COVID-19 partly because they simply fail to think sufficiently about whether or not the content is accurate when deciding what to share. In Study 1, participants were far worse at discerning between true and false content when deciding what they would share on social media relative to when they were asked directly about accuracy. Furthermore, greater cognitive reflection and science knowledge were associated with stronger discernment. In Study 2, we found that a simple accuracy reminder at the beginning of the study (i.e., judging the accuracy of a non-COVID-19-related headline) nearly tripled the level of truth discernment in participants' subsequent sharing intentions. Our results, which mirror those found previously for political fake news, suggest that nudging people to think about accuracy is a simple way to improve choices about what to share on social media.

Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking.

OBJECTIVE: Fake news represents a particularly egregious and direct avenue by which inaccurate beliefs have been propagated via social media. We investigate the psychological profile of individuals who fall prey to fake news. METHOD: We recruited 1,606 participants from Amazon's Mechanical Turk for three online surveys. RESULTS: The tendency to ascribe profundity to randomly generated sentences-pseudo-profound bullshit receptivity-correlates positively with perceptions of fake news accuracy, and negatively with the ability to differentiate between fake and real news (media truth discernment). Relatedly, individuals who overclaim their level of knowledge also judge fake news to be more accurate. We also extend previous research indicating that analytic thinking correlates negatively with perceived accuracy by showing that this relationship is not moderated by the presence/absence of the headline's source (which has no effect on accuracy), or by familiarity with the headlines (which correlates positively with perceived accuracy of fake and real news). CONCLUSION: Our results suggest that belief in fake news may be driven, to some extent, by a general tendency to be overly accepting of weak claims. This tendency, which we refer to as reflexive open-mindedness, may be partly responsible for the prevalence of epistemically suspect beliefs writ large.

You are not your data.

Scientists should, above all else, value the truth. To do this effectively, scientists should separate their identities from the data they produce. It will be easier to make replications mainstream if scientists are rewarded based on their stance toward the truth - such as when a scientist reacts positively to a failure to replicate - as opposed to a particular finding.

The cognitive reflection test is robust to multiple exposures.

The cognitive reflection test (CRT) is a widely used measure of the propensity to engage in analytic or deliberative reasoning in lieu of gut feelings or intuitions. CRT problems are unique because they reliably cue intuitive but incorrect responses and, therefore, appear simple among those who do poorly. By virtue of being composed of so-called "trick problems" that, in theory, could be discovered as such, it is commonly held that the predictive validity of the CRT is undermined by prior experience with the task. Indeed, recent studies have shown that people who have had previous experience with the CRT score higher on the test. Naturally, however, it is not obvious that this actually undermines the predictive validity of the test. Across six studies with ~ 2,500 participants and 17 variables of interest (e.g., religious belief, bullshit receptivity, smartphone usage, susceptibility to heuristics and biases, and numeracy), we did not find a single case in which the predictive power of the CRT was significantly undermined by repeated exposure. This occurred despite the fact that we replicated the previously reported increase in accuracy among individuals who reported previous experience with the CRT. We speculate that the CRT remains robust after multiple exposures because less reflective (more intuitive) individuals fail to realize that being presented with apparently easy problems more than once confers information about the task's actual difficulty.

The evolution of analytic thought?

We argue that the truly unique aspect of human intelligence is not the variety of cognitive skills that are ontogenetically constructed, but rather the capacity to decide when to develop and apply said skills. Even if there is good evidence for g in nonhuman animals, we are left with major questions about how the disposition to think analytically can evolve.

Analytic cognitive style, not delusional ideation, predicts data gathering in a large beads task study.

INTRODUCTION: It has been proposed that deluded and delusion-prone individuals gather less evidence before forming beliefs than those who are not deluded or delusion-prone. The primary source of evidence for this "jumping to conclusions" (JTC) bias is provided by research that utilises the "beads task" data-gathering paradigm. However, the cognitive mechanisms subserving data gathering in this task are poorly understood. METHODS: In the largest published beads task study to date (n = 558), we examined data gathering in the context of influential dual-process theories of reasoning. RESULTS: Analytic cognitive style (the willingness or disposition to critically evaluate outputs from intuitive processing and engage in effortful analytic processing) predicted data gathering in a non-clinical sample, but delusional ideation did not. CONCLUSION: The relationship between data gathering and analytic cognitive style suggests that dual-process theories of reasoning can contribute to our understanding of the beads task. It is not clear why delusional ideation was not found to be associated with data gathering or analytic cognitive style.

Is the cognitive reflection test a measure of both reflection and intuition?

The Cognitive Reflection Test (CRT) is one of the most widely used tools to assess individual differences in intuitive-analytic cognitive styles. The CRT is of broad interest because each of its items reliably cues a highly available and superficially appropriate but incorrect response, conventionally deemed the "intuitive" response. To do well on the CRT, participants must reflect on and question the intuitive responses. The CRT score typically employed is the sum of correct responses, assumed to indicate greater "reflectiveness" (i.e., CRT-Reflective scoring). Some recent researchers have, however, inverted the rationale of the CRT by summing the number of intuitive incorrect responses, creating a putative measure of intuitiveness (i.e., CRT-Intuitive). We address the feasibility and validity of this strategy by considering the problem of the structural dependency of these measures derived from the CRT and by assessing their respective associations with self-report measures of intuitive-analytic cognitive styles: the Faith in Intuition and Need for Cognition scales. Our results indicated that, to the extent that the dependency problem can be addressed, the CRT-Reflective but not the CRT-Intuitive measure predicts intuitive-analytic cognitive styles. These results provide evidence that the CRT is a valid measure of reflective but not of intuitive thinking.

What makes us think? A three-stage dual-process model of analytic engagement.

The distinction between intuitive and analytic thinking is common in psychology. However, while often being quite clear on the characteristics of the two processes ('Type 1' processes are fast, autonomous, intuitive, etc. and 'Type 2' processes are slow, deliberative, analytic, etc.), dual-process theorists have been heavily criticized for being unclear on the factors that determine when an individual will think analytically or rely on their intuition. We address this issue by introducing a three-stage model that elucidates the bottom-up factors that cause individuals to engage Type 2 processing. According to the model, multiple Type 1 processes may be cued by a stimulus (Stage 1), leading to the potential for conflict detection (Stage 2). If successful, conflict detection leads to Type 2 processing (Stage 3), which may take the form of rationalization (i.e., the Type 1 output is verified post hoc) or decoupling (i.e., the Type 1 output is falsified). We tested key aspects of the model using a novel base-rate task where stereotypes and base-rate probabilities cued the same (non-conflict problems) or different (conflict problems) responses about group membership. Our results support two key predictions derived from the model: (1) conflict detection and decoupling are dissociable sources of Type 2 processing and (2) conflict detection sometimes fails. We argue that considering the potential stages of reasoning allows us to distinguish early (conflict detection) and late (decoupling) sources of analytic thought. Errors may occur at both stages and, as a consequence, bias arises from both conflict monitoring and decoupling failures.

Cognitive style and religiosity: the role of conflict detection.

Recent research has indicated a negative relation between the propensity for analytic reasoning and religious beliefs and practices. Here, we propose conflict detection as a mechanism underlying this relation, on the basis of the hypothesis that more-analytic people are less religious, in part, because they are more sensitive to conflicts between immaterial religious beliefs and beliefs about the material world. To examine cognitive conflict sensitivity, we presented problems containing stereotypes that conflicted with base-rate probabilities in a task with no religious content. In three studies, we found evidence that religiosity is negatively related to conflict detection during reasoning. Independent measures of analytic cognitive style also positively predicted conflict detection. The present findings provide evidence for a mechanism potentially contributing to the negative association between analytic thinking and religiosity, and more generally, they illustrate the insights to be gained from integrating individual-difference factors and contextual factors to investigate analytic reasoning.

Crowds Can Effectively Identify Misinformation at Scale.

Identifying successful approaches for reducing the belief and spread of online misinformation is of great importance. Social media companies currently rely largely on professional fact-checking as their primary mechanism for identifying falsehoods. However, professional fact-checking has notable limitations regarding coverage and speed. In this article, we summarize research suggesting that the "wisdom of crowds" can be harnessed successfully to help identify misinformation at scale. Despite potential concerns about the abilities of laypeople to assess information quality, recent evidence demonstrates that aggregating judgments of groups of laypeople, or crowds, can effectively identify low-quality news sources and inaccurate news posts: Crowd ratings are strongly correlated with fact-checker ratings across a variety of studies using different designs, stimulus sets, and subject pools. We connect these experimental findings with recent attempts to deploy crowdsourced fact-checking in the field, and we close with recommendations and future directions for translating crowdsourced ratings into effective interventions.