Drawing attention to previous studies can reduce confidence in a new research finding, even when confidence should increase.

People often learn of new scientific findings from brief news reports, and may discount or ignore prior research, potentially contributing to misunderstanding of findings. In this preregistered study, we investigated how people interpret a brief news report on a new drug for weight loss. Participants read an article that either highlighted the importance of prior research when judging the drug's effectiveness, or made no mention of this issue. For articles describing no prior research, mean confidence in the drug was 62%. For articles that noted prior research was conducted, confidence increased as the proportion of studies with positive findings increased. When prior research was highlighted, confidence decreased by a small amount, even when it should have increased (i.e., even when most of the evidence supported the drug's effectiveness). Thus, people's judgements were more sceptical, but not necessarily more accurate. Judgements were not affected by education level, statistics experience, or personal relevance of the research topic.

Does context recollection depend on the base-rate of contextual features?

Episodic recollection is defined by the re-experiencing of contextual and target details of a past event. The base-rate dependency hypothesis assumes that the retrieval of one contextual feature from an integrated episodic trace cues the retrieval of another associated feature, and that the more often a particular configuration of features occurs, the more effective this mutual cueing will be. Alternatively, the conditional probability of one feature given another feature may be neglected in memory for contextual features since they are not directly bound to one another. Three conjoint recognition experiments investigated whether memory for context is sensitive to the base-rates of features. Participants studied frequent versus infrequent configurations of features and, during the test, they were asked to recognise one of these features with (vs. without) another feature reinstated. The results showed that the context recollection parameter, representing the re-experience of contextual features in the dual-recollection model, was higher for frequent than infrequent feature configurations only when the binding of feature information was made easier and the differences in the base-rates were extreme, otherwise no difference was found. Similarly, base-rates of features influenced response guessing only in the condition with salient differences in base-rates. The Bayes factor analyses showed that the evidence from two of our experiments favoured the base-rate neglect hypothesis over the base-rate dependency hypothesis; the opposite result was obtained in the third experiment, but only when high base-rate disproportion and facilitated feature binding conditions were used.

Conflict detection and base-rate extremity.

People tend to ignore the probabilistic rules cued by the base-rate information and rely on the heuristic intuition cued by the descriptive information to make "stereotypical" responses in base-rate problems. Conflict detection studies have shown that reasoners can detect conflicts between heuristic intuition and probabilistic considerations despite ultimately stereotypical responses. However, these studies primarily used extreme base-rate tasks. A critical open question is the extent to which successful conflict detection relies on an extreme base rate. The present study explores this issue by manipulating the base-rate extremity of problems in which the descriptive information and the base-rate information conflict or not. As a result, when reasoners made stereotypical responses in the conflict version of the moderate base-rate task, they took longer to respond, had lower confidence in their responses, and were slower to evaluate their confidence than in the no-conflict version of the task. All three measures indicate that stereotypical reasoners can stably detect conflict in moderate base-rate tasks, which expands the scope of successful conflict detection. Moreover, our response confidence data found a larger detection effect size in the extreme base-rate condition than in the moderate base-rate condition. This suggests that conflict detection is more efficient as the base-rate extremity increases. Implications for the boundary conditions of conflict detection are discussed.

Toward a more nuanced understanding of probability estimation biases.

In real life, we often have to make judgements under uncertainty. One such judgement task is estimating the probability of a given event based on uncertain evidence for the event, such as estimating the chances of actual fire when the fire alarm goes off. On the one hand, previous studies have shown that human subjects often significantly misestimate the probability in such cases. On the other hand, these studies have offered divergent explanations as to the exact causes of these judgment errors (or, synonymously, biases). For instance, different studies have attributed the errors to the neglect (or underweighting) of the prevalence (or base rate) of the given event, or the overweighting of the evidence for the individual event ('individuating information'), etc. However, whether or to what extent any such explanation can fully account for the observed errors remains unclear. To help fill this gap, we studied the probability estimation performance of non-professional subjects under four different real-world problem scenarios: (i) Estimating the probability of cancer in a mammogram given the relevant evidence from a computer-aided cancer detection system, (ii) estimating the probability of drunkenness based on breathalyzer evidence, and (iii & iv) estimating the probability of an enemy sniper based on two different sets of evidence from a drone reconnaissance system. In each case, we quantitatively characterized the contributions of the various potential explanatory variables to the subjects' probability judgements. We found that while the various explanatory variables together accounted for about 30 to 45% of the overall variance of the subjects' responses depending on the problem scenario, no single factor was sufficient to account for more than 53% of the explainable variance (or about 16 to 24% of the overall variance), let alone all of it. Further analyses of the explained variance revealed the surprising fact that no single factor accounted for significantly more than its 'fair share' of the variance. Taken together, our results demonstrate quantitatively that it is statistically untenable to attribute the errors of probabilistic judgement to any single cause, including base rate neglect. A more nuanced and unifying explanation would be that the actual biases reflect a weighted combination of multiple contributing factors, the exact mix of which depends on the particular problem scenario.

The paradoxical effects of time pressure on base rate neglect.

Base rate neglect refers to the well-documented tendency for people to primarily rely on diagnostic information to identify event probabilities while discounting information about relative probabilities (base rates). It is often postulated that using base rate information requires some form of working memory intensive processes. However, recent studies have put this interpretation into doubt, showing that rapid judgments can also lead to base rate use. Here we examine the idea that base rate neglect can be explained by the degree of attention paid to diagnostic information, which predicts that having more time should lead to greater rates of base rate neglect. Participants were presented with base rate problems either with a limited time to respond or with no time restrictions. Results show that having more time results in a decrease in base rate use.

Implicit and explicit processing on base rate neglect problems.

Base rate reasoning as assessed on Base rate neglect (BRN) Tasks has been studied extensively, with a sizable body of findings indicating that both logical (base rate) and belief-based (case description) processing contribute to responses on the task. Various task conditions have been found to influence which type of processing controls responding. The present study compares two instructional sets, one which requires responding in terms of the base rate information and one which requires responding in accordance with the case description. This manipulation allows for a distinction between explicit processing (set-consistent) and implicit processing (set-inconsistent and potentially interfering). We also manipulated the extremity of the base rates employed in the task and the extremity of the stereotypes contained in the case description. We argue that extremity effects should be present in implicit, but not explicit, processing, suggesting that these effects are the result of limitations in the control of set-inconsistent processing. The results generally supported the predictions. In addition, a proclivity for analytical thinking, as measured by actively open-minded thinking (AOT), was associated with less interference of belief-based processing on logical responding, but greater interference of logical processing on belief-based responding.

The role of base-rate neglect in cyberchondria and health anxiety.

Cyberchondria is characterized by excessive health-related online search behavior associated with an unfounded escalation of concerns about common symptomatology. It often co-occurs with health anxiety. We investigated whether base-rate neglect-the cognitive bias to ignore a priori probabilities (e.g., of serious diseases)-plays a significant role in cyberchondria and health anxiety. 368 participants were randomly assigned to eight experimental conditions, manipulating the base-rate (30 % vs. 70 %), the judgment domain (health-neutral versus health-related), and the salience of base-rate information (low vs. high) in a 2×2×2 between-subjects design when asking them for probability judgments with versus without disease relevance. We found that high salience decreased base-rate neglect in participants with low, but not in those with elevated levels of either cyberchondria or health anxiety. Under low salience conditions, however, both cyberchondria and health anxiety severity were uncorrelated with base-rate neglect. These effects were independent of whether health-related or health-neutral problems were evaluated. Our findings suggest a domain-general probabilistic reasoning style that may play a causal role in the pathogenesis of cyberchondria and health anxiety.

How well do practicing radiologists interpret the results of CAD technology? A quantitative characterization.

Many studies have shown that using a computer-aided detection (CAD) system does not significantly improve diagnostic accuracy in radiology, possibly because radiologists fail to interpret the CAD results properly. We tested this possibility using screening mammography as an illustrative example. We carried out two experiments, one using 28 practicing radiologists, and a second one using 25 non-professional subjects. During each trial, subjects were shown the following four pieces of information necessary for evaluating the actual probability of cancer in a given unseen mammogram: the binary decision of the CAD system as to whether the mammogram was positive for cancer, the true-positive and false-positive rates of the system, and the prevalence of breast cancer in the relevant patient population. Based only on this information, the subjects had to estimate the probability that the unseen mammogram in question was positive for cancer. Additionally, the non-professional subjects also had to decide, based on the same information, whether to recall the patients for additional testing. Both groups of subjects similarly (and significantly) overestimated the cancer probability regardless of the categorical CAD decision, suggesting that this effect is not peculiar to either group. The misestimations were not fully attributable to causes well-known in other contexts, such as base rate neglect or inverse fallacy. Non-professional subjects tended to recall the patients at high rates, even when the actual probably of cancer was at or near zero. Moreover, the recall rates closely reflected the subjects' estimations of cancer probability. Together, our results show that subjects interpret CAD system output poorly when only the probabilistic information about the underlying decision parameters is available to them. Our results also highlight the need for making the output of CAD systems more readily interpretable, and for providing training and assistance to radiologists in evaluating the output.

On the generality and cognitive basis of base-rate neglect.

Base rate neglect refers to people's apparent tendency to underweight or even ignore base rate information when estimating posterior probabilities for events, such as the probability that a person with a positive cancer-test outcome actually does have cancer. While often replicated, almost all evidence for the phenomenon comes from studies that used problems with extremely low base rates, high hit rates, and low false alarm rates. It is currently unclear whether the effect generalizes to reasoning problems outside this "corner" of the entire problem space. Another limitation of previous studies is that they have focused on describing empirical patterns of the effect at the group level and not so much on the underlying strategies and individual differences. Here, we address these two limitations by testing participants on a broader problem space and modeling their responses at a single-participant level. We find that the empirical patterns that have served as evidence for base-rate neglect generalize to a larger problem space, albeit with large individual differences in the extent with which participants "neglect" base rates. In particular, we find a bi-modal distribution consisting of one group of participants who almost entirely ignore the base rate and another group who almost entirely account for it. This heterogeneity is reflected in the cognitive modeling results: participants in the former group were best captured by a linear-additive model, while participants in the latter group were best captured by a Bayesian model. We find little evidence for heuristic models. Altogether, these results suggest that the effect known as "base-rate neglect" generalizes to a large set of reasoning problems, but varies largely across participants and may need a reinterpretation in terms of the underlying cognitive mechanisms.

Can a fast thinker be a good thinker? The neural correlates of base-rate neglect measured using a two-response paradigm.

Traditionally, it has been assumed that logical thinking requires deliberation. However, people can also make logical responses quickly, exhibiting logical intuitions. We examined the neural correlates of logical intuitions by administering base rate problems during fMRI scanning using a two-response paradigm where participants first responded quickly and then reflectively to problems that did or did not pit a normative response against an intuitively-cued stereotypical response (i.e., conflict vs. non-conflict problems). As predicted, participants were less likely to make judgments in accordance with base rates on conflict problems. Critically, in only 4% of cases did longer deliberation change an initially biased response to a normatively correct response. The fMRI data revealed that intuitively-made initial biased judgments nevertheless activate regions typically involved in cognitive control, executive functions and attention, including anterior, inferior, middle and superior frontal cortex, suggesting that even when errors are made, there might be very early awareness of conflict.

Base rate neglect and neural computations for subjective weight in decision under uncertainty.

Base rate neglect, an important bias in estimating probability of uncertain events, describes humans' tendency to underweight base rate (prior) relative to individuating information (likelihood). However, the neural mechanisms that give rise to this bias remain elusive. In this study, subjects chose between uncertain prospects where estimating reward probability was essential. We found that when the variability of prior and likelihood information about reward probability were systematically manipulated, prior variability significantly affected the degree to which subjects underweight the base rate of reward probability. Activity in the orbitofrontal cortex, medial prefrontal cortex, and putamen represented the relative subjective weight that reflected such bias. Further, sensitivity to likelihood relative to prior variability in the putamen correlated with individuals' overall tendency to underweight base rate. These findings suggest that in combining prior and likelihood, relative sensitivity to information variability and subjective-weight computations critically contribute to the individual heterogeneity in base rate neglect.

The environmental malleability of base-rate neglect.

Across two experiments (N=799) we demonstrate that people's use of quantitative information (e.g., base-rates) when making a judgment varies as the causal link of qualitative information (e.g., stereotypes) changes. That is, when a clear causal link for stereotypes is provided, people make judgments that are far more in line with them. When the causal link is heavily diminished, people readily incorporate non-causal base-rates into their judgments instead. We suggest that people use and integrate all of the information that is provided to them to make judgements, but heavily prioritize information that is causal in nature. Further, people are sensitive to the underlying causal structures in their environment and adapt their decision making as such.

Bias in Clinical Practice.

Role of bias in errors of decision making is receiving increasing attention. It is turning out to be one of the main sources of mistakes. Hence, it is important to be aware of biases and to design strategies toward an unbiased approach. Biases are of various types, and the potential sources of bias can be related to the consultant, patients and factors related to working conditions. Availability bias, base rate neglect, confirmation bias, conjunction rule, diagnostic momentum bias, framing effect and confirmation bias are the common types, and these have been discussed in this manuscript using a scenario-based format. Two types of human thinking, the rapid intuitive mode and the slow reflective mode, their pros and cons and their role in biases are discussed. Strategies to enhance awareness of biases, tips to improve reasoning, promote freethinking, enhance decision-making skills and resorting to checklists have been deliberated to achieve an unbiased approach.

The base rate neglect in episodic memory.

The main objective of the present research was, for the first time, to assess a potential similarity in the representational bases for the base rate neglect in memory versus conditional probability judgment. Participants learned target and filler words, each of which was presented on a separate list (List 1 or List 2) and in a distinct colour (red or blue), with a manipulation of different base rates for these list and colour categories. During recognition tests, participants made prior and posterior episodic judgments (e.g., "What colour was the word?", "Given that the word was in red, on which list was the word?") on the target words, which respectively parallel independent and conditional probability assessments that figure in Bayes' theorem. The results implied that biased prior and posterior judgments presumably cause the base rate neglect, inasmuch as the prior cue of a low (high) base rate is likely to lead to a bias toward retrieving high (low) base rate posterior evidence. There was also a finding showing that memory analogues of probability estimates reflect the base rate neglect in both low and high base rate categories, but is presumably stronger with posterior judgment of Colour|List than List|Colour relative to the high base rate category.

Physician and Nonphysician Estimates of Positive Predictive Value in Diagnostic v. Mass Screening Mammography: An Examination of Bayesian Reasoning.

BACKGROUND: The same test with the same result has different positive predictive values (PPVs) for people with different pretest probability of disease. Representative thinking theory suggests people are unlikely to realize this because they ignore or underweight prior beliefs when given new information (e.g., test results) or due to confusing test sensitivity (probability of positive test given disease) with PPV (probability of disease given positive test). This research examines whether physicians and MBAs intuitively know that PPV following positive mammography for an asymptomatic woman is less than PPV for a symptomatic woman and, if so, whether they correctly perceive the difference. DESIGN: Sixty general practitioners (GPs) and 84 MBA students were given 2 vignettes of women with abnormal (positive) mammography tests: 1 with prior symptoms (diagnostic test), the other an asymptomatic woman participating in mass screening (screening test). Respondents estimated pretest and posttest probabilities. Sensitivity and specificity were neither provided nor elicited. RESULTS: Eighty-eight percent of GPs and 46% of MBAs considered base rates and estimated PPV in diagnosis greater than PPV in screening. On average, GPs estimated a 27-point difference and MBAs an 18-point difference, compared to actual of 55 or more points. Ten percent of GPs and 46% of MBAs ignored base rates, incorrectly assessing the 2 PPVs as equal. CONCLUSIONS: Physicians and patients are better at intuitive Bayesian reasoning than is suggested by studies that make test accuracy values readily available to be confused with PPV. However, MBAs and physicians interpret a positive in screening as more similar to a positive in diagnosis than it is, with nearly half of MBAs and some physicians wrongly equating the two. This has implications for overdiagnosis and overtreatment.

The development of the representativeness heuristic in young children.

In classic examinations of the representativeness heuristic, Kahneman and Tversky (1973) presented adult participants with a description of an individual who fit their stereotype of a typical engineer. Importantly, even when participants were told that the individual was drawn from a sample of 70 lawyers and 30 engineers, they estimated that the individual was an engineer at very high levels, showing that they relied almost exclusively on the personality description. Relying on the representativeness heuristic can lead to base-rate neglect and, thus, biased judgments. Two experiments provide insight into the development of the representativeness heuristic in young children using an adaptation of the classic lawyer-engineer problem. Experiment 1 (N = 96) established that 3- to 5-year-olds can use base-rate information on its own, and 4- and 5-year-olds can use individuating information on its own, to make inferences. Experiment 2 (N = 192) varied the relevance of the individuating information across conditions to assess the pervasiveness of this bias early in development. Here 5- and 6-year-olds, much like adults, continue to attempt to rely on individuating information when making classifications even if that information is irrelevant. Together, these experiments reveal how the representativeness heuristic develops across the preschool years and suggest that the bias may strengthen between 4 and 6 years of age.

To detect or not to detect: A replication and extension of the three-stage model.

When faced with a decision, people generally show a bias toward heuristic processing, even if it leads to the incorrect decision, such as in the base-rate neglect task. The crucial question is whether people know that they are biased. Recently, the three-stage model (Pennycook, Fugelsang, & Koehler, 2015) suggested that detecting this bias (conflict detection) is imperfect and a consistent source of bias because some people do not recognize that they are making biased decisions. In Experiment 1, participants completed a base-rate neglect task as replication of Pennycook et al. (2015). In Experiment 2, a conditional reasoning task was added as an extension to test the boundary conditions of the model. Results in Experiment 1 indicated that detection failures were a significant source of bias. However, results in Experiment 2 on the conditional reasoning task indicated that the three-stage model may be incompatible with a complex task such as conditional reasoning, an issue explored in detail in the General discussion.

Not that neglected! Base rates influence related and unrelated judgments.

It is claimed that people are unable (or unwilling) to incorporate prior probabilities into posterior assessments, such as their estimation of the likelihood of a person with characteristics typical of an engineer actually being an engineer given that they are drawn from a sample including a very small number of engineers. This paper shows that base rates are incorporated in classifications (Experiment 1) and, moreover, that base rates also affect unrelated judgments, such as how well a provided description of a person fits a stereotypical engineer (Experiment 2). Finally, Experiment 3 shows that individuals who make both types of assessments - though using base rates to the same extent in the former judgments - are able to decrease the extent to which they incorporate base rates in the latter judgments.

Bayesian reasoning in residents' preliminary diagnoses.

Whether and when humans in general, and physicians in particular, use their beliefs about base rates in Bayesian reasoning tasks is a long-standing question. Unfortunately, previous research on whether doctors use their beliefs about the prevalence of diseases in diagnostic judgments has critical limitations. In this study, we assessed whether residents' beliefs about the prevalence of a disease are associated with their judgments of the likelihood of the disease in diagnosis, and whether residents' beliefs about the prevalence of diseases change across the 3 years of residency. Residents were presented with five ambiguous vignettes typical of patients presenting on the inpatient general medicine services. For each vignette, the residents judged the likelihood of five or six possible diagnoses. Afterward, they judged the prevalence within the general medicine services of all the diseases in the vignettes. Most importantly, residents who believed a disease to be more prevalent tended to rate the disease as more likely in the vignette cases, suggesting a rational tendency to incorporate their beliefs about disease prevalence into their diagnostic likelihood judgments. In addition, the residents' prevalence judgments for each disease were assessed over the 3 years of residency. The precision of the prevalence estimates increased across the 3 years of residency, though the accuracy of the prevalence estimates did not. These results imply that residents do have a rational tendency to use prevalence beliefs for diagnosis, and this finding also contributes to a larger question of whether humans intuitively use base rates for making judgments.

Delayed matching-to-sample: A tool to assess memory and other cognitive processes in pigeons.

Delayed matching-to-sample is a versatile task that has been used to assess the nature of animal memory. Although once thought to be a relatively passive process, matching research has demonstrated considerable flexibility in how animals actively represent events in memory. But delayed matching can also demonstrate how animals fail to maintain representations in memory when they are cued that they will not be tested (directed forgetting) and how the outcome expected can serve as a choice cue. When pigeons have shown divergent retention functions following training without a delay, it has been taken as evidence of the use of a single-code/default coding strategy but in many cases an alternative account may be involved. Delayed matching has also been used to investigate equivalence learning (how animals represent stimuli when they learn that the same comparison response is correct following the presentation of two different samples) and to test for metamemory (the ability of pigeons to indicate that they understand what they know) by allowing animals to decline to be tested when they are uncertain that they remember a stimulus. How animals assess the passage of time has also been studied using the matching task. And there is evidence that when memory for the sample is impaired by a delay, rather than use the probability of being correct for choice of each of the comparison stimuli, pigeons tend to choose based on the overall sample frequency (base-rate neglect). Finally, matching has been used to identify natural color categories as well as dimensional categories in pigeons. Overall, matching to sample has provided an excellent methodology for assessing an assortment of cognitive processes in animals.