The effects of communicating uncertainty on public trust in facts and numbers.

Uncertainty is inherent to our knowledge about the state of the world yet often not communicated alongside scientific facts and numbers. In the "posttruth" era where facts are increasingly contested, a common assumption is that communicating uncertainty will reduce public trust. However, a lack of systematic research makes it difficult to evaluate such claims. We conducted five experiments-including one preregistered replication with a national sample and one field experiment on the BBC News website (total n = 5,780)-to examine whether communicating epistemic uncertainty about facts across different topics (e.g., global warming, immigration), formats (verbal vs. numeric), and magnitudes (high vs. low) influences public trust. Results show that whereas people do perceive greater uncertainty when it is communicated, we observed only a small decrease in trust in numbers and trustworthiness of the source, and mostly for verbal uncertainty communication. These results could help reassure all communicators of facts and science that they can be more open and transparent about the limits of human knowledge.

Do colored cells in risk matrices affect decision-making and risk perception? Insights from randomized controlled studies.

Risk matrices communicate the likelihood and potential impact of risks and are often used to inform decision-making around risk mitigations. The merits and demerits of risk matrices in general have been discussed extensively, yet little attention has been paid to the potential influence of color in risk matrices on their users. We draw from fuzzy-trace theory and hypothesize that when color is present, individuals are likely to place greater value on reducing risks that cross color boundaries (i.e., the boundary-crossing effect), leading to sub-optimal decision making. In two randomized controlled studies, employing forced-choice and willingness-to-pay measures to investigate the boundary-crossing effect in two different color formats for risk matrices, we find preliminary evidence to support our hypotheses that color can influence decision making. The evidence also suggests that the boundary-crossing effect is only present in, or is stronger for, higher numeracy individuals. We therefore recommend that designers should consider avoiding color in risk matrices, particularly in situations where these are likely to be used by highly numerate individuals, if the communication goal is to inform in an unbiased way.

Making BRCA1 genetic test reports easier to understand through user-centered design: A randomized trial.

PURPOSE: Our objective was to apply a user-centered design process to identify phrases, graphics, and ways of communicating numerical risks that could be used to help patients understand their cancer risk and next steps on receiving BRCA1 genetic test results (positive, negative, and variants of uncertain significance). METHODS: The first phase of the study, a user-centered design process, consisted of 4 rounds of interviews (N = 42, including 13 health care professionals and 16 patients having undergone BRCA testing). The second was a randomized, between-participants experimental study of 456 United Kingdom residents that compared the resulting reports to reports used in a United Kingdom national genomic laboratory hub. Outcomes were subjective and objective comprehension, communication efficacy, actionability, and perceived risk. RESULTS: Subjective comprehension, communication efficacy, and actionability were all higher for the user-centered reports, with no difference in perceived risk. Comprehension of participants viewing user-centered reports was significantly better on 2 items, directionally (but not significantly) better on 6 items, and directionally (but not significantly) worse on 2 items. CONCLUSION: Our results imply that user-centered design is a promising approach for developing materials about complex genetic risks. We suggest wordings that are likely to lead to improved comprehension when communicating BRCA-associated risks.

How People Understand Risk Matrices, and How Matrix Design Can Improve their Use: Findings from Randomized Controlled Studies.

Risk matrices are a common way to communicate the likelihood and potential impacts of a variety of risks. Until now, there has been little empirical work on their effectiveness in supporting understanding and decision making, and on how different design choices affect these. In this pair of online experiments (total n = 2699), we show that risk matrices are not always superior to text for the presentation of risk information, and that a nonlinear/geometric labeling scheme helps matrix comprehension (when the likelihood/impact scales are nonlinear). To a lesser degree, results suggested that changing the shape of the matrix so that cells increase in size nonlinearly facilitates comprehension as compared to text alone, and that comprehension might be enhanced by integrating further details about the likelihood and impact onto the axes of the matrix rather than putting them in a separate key. These changes did not affect participants' preference for reducing impact over reducing likelihood when making decisions about risk mitigation. We recommend that designers of risk matrices consider these changes to facilitate better understanding of relationships among risks.

How Accurate Are Accuracy-Nudge Interventions? A Preregistered Direct Replication of Pennycook et al. (2020).

As part of the Systematizing Confidence in Open Research and Evidence (SCORE) program, the present study consisted of a two-stage replication test of a central finding by Pennycook et al. (2020), namely that asking people to think about the accuracy of a single headline improves "truth discernment" of intentions to share news headlines about COVID-19. The first stage of the replication test (n = 701) was unsuccessful (p = .67). After collecting a second round of data (additional n = 882, pooled N = 1,583), we found a small but significant interaction between treatment condition and truth discernment (uncorrected p = .017; treatment: d = 0.14, control: d = 0.10). As in the target study, perceived headline accuracy correlated with treatment impact, so that treatment-group participants were less willing to share headlines that were perceived as less accurate. We discuss potential explanations for these findings and an unreported change in the hypothesis (but not the analysis plan) from the preregistration in the original study.

Creating genetic reports that are understood by nonspecialists: a case study.

PURPOSE: Guidelines recommend that genetic reports should be clear to nonspecialists, including patients. We investigated the feasibility of creating reports for cystic fibrosis carrier testing through a rapid user-centered design process that built on a previously developed generic template. We evaluated the new reports' communication efficacy and effects on comprehension against comparable reports used in current clinical practice. METHODS: Thirty participants took part in three rounds of interviews. Usability problems were identified and rectified in each round. One hundred ninety-three participants took part in an evaluation of the resulting reports measuring subjective comprehension, risk probability comprehension, perceived communication efficacy, and other factors, as compared with standard reports. RESULTS: Participants viewing the user-centered reports rated them as clearer, easier to understand, and more effective at communicating key information than standard reports. Both groups ended up with equivalent knowledge of risk probabilities, although we observed differences in how those probabilities were perceived. CONCLUSION: Our findings demonstrate that by starting with a patient-friendly generic report template and modifying it for specific scenarios with a rapid user-centered design process, reports can be produced that are more effective at communicating key information. The resulting reports are now being implemented into clinical care.

Correction: Creating genetic reports that are understood by nonspecialists: a case study.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Communicating health risks in science publications: time for everyone to take responsibility.

Research that is poorly communicated or presented is as potentially damaging as research that is poorly conducted or fraudulent. Recent examples illustrate how the problem often lies with researchers, not press officers or journalists. The quest for publication and 'impact' must not outweigh the importance of accurate representation of science; herein, we suggest steps that researchers, journalists and press officers can take to help ensure this.

What Affects Perceived Trustworthiness of Online Medical Information and Subsequent Treatment Decision Making? Randomized Trials on the Role of Uncertainty and Institutional Cues.

Background. Online, algorithmically driven prognostic tools are increasingly important in medical decision making. Institutions developing such tools need to be able to communicate the precision and accuracy of the information in a trustworthy manner, and so many attempt to communicate uncertainties but also use institutional logos to underscore their trustworthiness. Bringing together theories on trust, uncertainty, and psychological distance in a novel way, we tested whether and how the communication of uncertainty and the presence of institutional logos together affected trust in medical information, the prognostic tool itself, and treatment decisions. Methods. A pilot and 2 online experiments in which UK (experiment 1) and worldwide (experiment 2) participants (Ntotal = 4,724) were randomized to 1 of 12 arms in a 3 (uncertainty cue) × 4 (institutional cue) between-subjects design. The stimulus was based on an existing medical prognostic tool. Results. Institutional trust was consistently associated with trust in the prognostic tool itself, while uncertainty information had no consistent effect. Institutional trust predicted the amount of weight participants reported placing on institutional endorsements in their decision making and the likelihood of switching from passive to active treatment in a hypothetical scenario. There was also a significant effect of psychological distance to (perceived hypotheticality of) the scenario. Conclusions/Implications. These results underline the importance of institutions demonstrating trustworthiness and building trust with their users. They also suggest that users tend to be insensitive to communications of uncertainty and that communicators may need to be highly explicit when attempting to warn of low precision or quality of evidence. The effect of the perceived hypotheticality of the scenario underscores the importance of realistic decision-making scenarios for studies and the role of familiarity with the decision dilemma generally. Highlights: In a world where information for medical decision making is increasingly going to be provided through digital, online tools, institutions providing such tools need guidance on how best to communicate about their trustworthiness and precision.We find that people are fairly insensitive to cues designed to communicate uncertainty around the outputs of such tools. Even putting "ATTENTION" in bold font or explicitly pointing out the weaknesses in the data did not appear to affect people's decision making using the tool's outputs. Institutions should take note, and further work is required to determine how best to communicate uncertainty in a way that elicits appropriate caution in lay users.People were much more sensitive to institutional logos associated with the outputs. Generalized institutional trust (rather than trust in the specific institution whose logo was shown) was associated with how trustworthy, accurate, and reliable the tool, its algorithm, and the numbers it produced were perceived to be. This underscores the role of societal trust in institutions at large.Finally, as a note to researchers, we found a significant effect of how hypothetical or believable participants felt the experimental scenario was. This is a variable that seems rarely controlled for in studies and yet played as much of a role as some of our variables of interest, so we suggest that it is measured in future experiments.

The effects of communicating uncertainty around statistics, on public trust.

Uncertainty around statistics is inevitable. However, communicators of uncertain statistics, particularly in high-stakes and potentially political circumstances, may be concerned that presenting uncertainties could undermine the perceived trustworthiness of the information or its source. In a large survey experiment (Study 1; N = 10 519), we report that communicating uncertainty around present COVID-19 statistics in the form of a numeric range (versus no uncertainty) may lead to slightly lower perceived trustworthiness of the number presented but has no impact on perceived trustworthiness of the source of the information. We also show that this minimal impact of numeric uncertainty on trustworthiness is also present when communicating future, projected COVID-19 statistics (Study 2; N = 2,309). Conversely, we find statements about the mere existence of uncertainty, without quantification, can reduce both perceived trustworthiness of the numbers and of their source. Our findings add to others suggesting that communicators can be transparent about statistical uncertainty without undermining their credibility as a source but should endeavour to provide a quantification, such as a numeric range, where possible.

The adverse effects of trastuzumab-containing regimes as a therapy in breast cancer: A piggy-back systematic review and meta-analysis.

BACKGROUND: Trastuzumab is a valuable therapy option for women with ERBB2(HER2)+ breast cancer tumours, often used in combination with chemotherapy and alongside other therapies. It is known to have adverse effects, but these have proved difficult to separate from the effects of other concurrent therapies patients are usually taking. This study aims to assess the adverse effects specifically attributable to trastuzumab, and whether they vary by patient subgroup or concurrent therapies. METHODS: As registered on PROSPERO (CRD42019146541), we used previous systematic reviews as well as the clinicaltrials.gov registry to identify randomised controlled trials in breast cancer which compared treatment regimes with and without trastuzumab. Neoadjuvant, adjuvant and metastatic settings were examined. Data was extracted from those which had, as of July 2022, reported adverse events. Risk of bias was assessed using ROB2. Primary outcomes were adverse events of any type or severity (excluding death). A standard random-effects meta-analysis was performed for each outcome independently. In order to ascertain whether adverse effects differed by individual factors such as age or tumour characteristics, or by use of trastuzumab concurrently with hormone therapy, we examined individual-level patient data for one large trial, HERA. RESULTS: 79 relevant trials were found, of which 20 contained comparable arms of trastuzumab-containing therapy and corresponding matched therapy without trastuzumab. This allowed a comparison of 8669 patients receiving trastuzumab versus 9556 receiving no trastuzumab, which gave a list of 25 statistically and clinically significant adverse effects related to trastuzumab alone: unspecified pain, asthenia, nasopharyngitis, skin disorders (mainly rash), dyspepsia, paraesthesia, infections (often respiratory), increased lacrimation, diarrhoea, myalgia, oedema (limb/peripheral), fever, nose bleeds, cardiac events, insomnia, cough, back pain, dyspnoea, chills, dizziness or vertigo, hypertension, congestive heart failure, increased levels of aspartate aminotransferase, gastrointestinal issues and dehydration. Analysis of individual patient-level data from 5102 patients suggested that nausea is slightly more likely for women taking trastuzumab who are ER+ /also taking hormone therapy than for those who are ER-/not taking hormone therapy; no other potential treatment-subgroup interactions were detected. We found no evidence for significantly increased rates of neutropenia, anaemia or lymphopenia in patients on trastuzumab-containing regimes compared to those on comparable regimes without trastuzumab. CONCLUSIONS: This meta-analysis should allow clinicians and patients to better identify and quantify the potential adverse effects of adding trastuzumab to their treatment regime for breast cancer, and hence inform their decision-making. However, limitations include serious risk of bias due to heterogeneity in reporting of the outcomes and the open-label nature of the trials.

Investigating the presentation of uncertainty in an icon array: A randomized trial.

Background: Clinicians are often advised to use pictographs to communicate risk, but whether they offer benefits when communicating risk imprecision (e.g., 65%-79%) is unknown. Purpose: To test whether any of three approaches to visualizing imprecision would more effectively communicate breast and ovarian cancer risk for BRCA1 pathogenic variant carriers. Methods: 1,300 UK residents were presented with a genetic report with information about BRCA1-related risks, with random assignment to one of four formats: no visualization (text alone), or a pictograph using shaded icons, a gradient, or arrows marking range endpoints. We also tested pictographs in two layouts. Analysis of variance (ANOVA) and regression was employed. Results: There was no effect of format. Participants shown pictographs vs. text alone had better uptake of breast cancer risk messages (p < .05, η 2 = 0.003). Pictographs facilitated memory for the specific amount of risk (p < 0.001, η 2 = 0.019), as did the tabular layout. Individuals not having completed upper secondary education may benefit most. Conclusions: We found weak evidence in favor of using simple pictographs with ranges to communicate BRCA risk (versus text alone), and of the tabular layout. Innovation: Testing different ways of communicating imprecision within pictographs is a novel and promising line of research.

COVID-19 risk perception and hoax beliefs in the US immediately before and after the announcement of President Trump's diagnosis.

A notable challenge of the SARS-CoV-2 pandemic has been public scepticism over the severity of the disease, or even its existence. Such scepticism is politically skewed in the USA, with conservatives more likely to downplay or deny the risks of the virus. However, the hospitalization of President Trump with COVID-19 in October 2020 served as a high-profile exemplar of the reality and risks of the virus, and as such may have influenced opinions, particularly for US conservatives. We investigate whether President Trump testing positive was associated with changes in public attitudes towards the virus. In two studies, we surveyed independent representative US samples before and after the announcement of Trump's illness. In Study 1, measuring risk perceptions of the virus, we find that participants surveyed before and after the announcement did not differ in their risk perception regardless of political orientation. In Study 2, measuring belief that the virus is a hoax, we find that among those on the far right of the political spectrum, hoax belief was lower for those surveyed after the announcement, suggesting that Trump's hospitalization may have affected the beliefs of those most receptive to the President's earlier suggestions that the virus might be a hoax.

Transparent communication of evidence does not undermine public trust in evidence.

Does clear and transparent communication of risks, benefits, and uncertainties increase or undermine public trust in scientific information that people use to guide their decision-making? We examined the impact of reframing messages written in traditional persuasive style to align instead with recent "evidence communication" principles, aiming to inform decision-making: communicating a balance of risks and benefits, disclosing uncertainties and evidence quality, and prebunking misperceptions. In two pre-registered experiments, UK participants read either a persuasive message or a balanced and informative message adhering to evidence communication recommendations about COVID-19 vaccines (Study 1) or nuclear power plants (Study 2). We find that balanced messages are either perceived as trustworthy as persuasive messages (Study 1), or more so (Study 2). However, we note a moderating role of prior beliefs such that balanced messages were consistently perceived as more trustworthy among those with negative or neutral prior beliefs about the message content. We furthermore note that participants who had read the persuasive message on nuclear power plants voiced significantly stronger support for nuclear power than those who had read the balanced message, despite rating the information as less trustworthy. There was no difference in vaccination intentions between groups reading the different vaccine messages.

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty.

The adverse effects of bisphosphonates in breast cancer: A systematic review and network meta-analysis.

BACKGROUND: Bisphosphonate drugs can be used to improve the outcomes of women with breast cancer. Whilst many meta-analyses have quantified their potential benefits for patients, attempts at comprehensive quantification of potential adverse effects have been limited. We undertook a meta-analysis with novel methodology to identify and quantify these adverse effects. METHODS: We systematically reviewed randomised controlled trials in breast cancer where at least one of the treatments was a bisphosphonate (zoledronic acid, ibandronate, pamidronate, alendronate or clodronate). Neoadjuvant, adjuvant and metastatic settings were examined. Primary outcomes were adverse events of any type or severity (excluding death). We carried out pairwise and network meta-analyses to estimate the size of any adverse effects potentially related to bisphosphonates. In order to ascertain whether adverse effects differed by individual factors such as age, or interacted with other common adjuvant breast cancer treatments, we examined individual-level patient data for one large trial, AZURE. FINDINGS: We identified 56 trials that reported adverse data, which included a total of 29,248 patients (18,301 receiving bisphosphonate drugs versus 10,947 not). 24 out of the 103 different adverse outcomes analysed showed a statistically and practically significant increase in patients receiving a bisphosphonate drug compared with those not (2 additional outcomes that appeared statistically significant came only from small studies with low event counts and no clinical suspicion so are likely artifacts). Most of these 24 are already clinically recognised: 'flu-like symptoms, fever, headache and chills; increased bone pain, arthralgia, myalgia, back pain; cardiac events, thromboembolic events; hypocalcaemia and osteonecrosis of the jaw; as well as possibly stiffness and nausea. Oral clodronate appeared to increase the risk of vomiting and diarrhoea (which may also be increased by other bisphosphonates), and there may be some hepatotoxicity. Four additional potential adverse effects emerged for bisphosphonate drugs in this analysis which have not classically be recognised: fatigue, neurosensory problems, hypertonia/muscle spasms and possibly dysgeusia. Several symptoms previously reported as potential side effects in the literature were not significantly increased in this analysis: constipation, insomnia, respiratory problems, oedema or thirst/dry mouth. Individual patient-level data and subgroup analysis revealed little variation in side effects between women of different ages or menopausal status, those with metastatic versus non-metastatic cancer, or between women receiving different concurrent breast cancer therapies. CONCLUSIONS: This meta-analysis has produced estimates for the absolute frequencies of a range of side effects significantly associated with bisphosphonate drugs when used by breast cancer patients. These results show good agreement with previous literature on the subject but are the first systematic quantification of side effects and their severities. However, the analysis is limited by the availability and quality of data on adverse events, and the potential for bias introduced by a lack of standards for reporting of such events. We therefore present a table of adverse effects for bisphosphonates, identified and quantified to the best of our ability from a large number of trials, which we hope can be used to improve the communication of the potential harms of these drugs to patients and their healthcare providers.

Redevelopment of the Predict: Breast Cancer website and recommendations for developing interfaces to support decision-making.

OBJECTIVES: To develop a new interface for the widely used prognostic breast cancer tool: Predict: Breast Cancer. To facilitate decision-making around post-surgery breast cancer treatments. To derive recommendations for communicating the outputs of prognostic models to patients and their clinicians. METHOD: We employed a user-centred design process comprised of background research and iterative testing of prototypes with clinicians and patients. Methods included surveys, focus groups and usability testing. RESULTS: The updated interface now caters to the needs of a wider audience through the addition of new visualisations, instantaneous updating of results, enhanced explanatory information and the addition of new predictors and outputs. A programme of future research was identified and is now underway, including the provision of quantitative data on the adverse effects of adjuvant breast cancer treatments. Based on our user-centred design process, we identify six recommendations for communicating the outputs of prognostic models including the need to contextualise statistics, identify and address gaps in knowledge, and the critical importance of engaging with prospective users when designing communications. CONCLUSIONS: For prognostic algorithms to fulfil their potential to assist with decision-making they need carefully designed interfaces. User-centred design puts patients and clinicians needs at the forefront, allowing them to derive the maximum benefit from prognostic models.

The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials.

BACKGROUND: The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public. METHODS: In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured. FINDINGS: Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was 'low', rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was 'high', and in one of the two studies, those shown 'low' quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was 'high' showed no statistically significant differences on these measures compared to those given no information about evidence quality. CONCLUSIONS: Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.

Effect of Information about COVID-19 Vaccine Effectiveness and Side Effects on Behavioural Intentions: Two Online Experiments.

The success of mass COVID-19 vaccination campaigns rests on widespread uptake. However, although vaccinations provide good protection, they do not offer full immunity and while they likely reduce transmission of the virus to others, the extent of this remains uncertain. This produces a dilemma for communicators who wish to be transparent about benefits and harms and encourage continued caution in vaccinated individuals but not undermine confidence in an important public health measure. In two large pre-registered experimental studies on quota-sampled UK public participants we investigate the effects of providing transparent communication-including uncertainty-about vaccination effectiveness on decision-making. In Study 1 (n = 2097) we report that detailed information about COVID-19 vaccines, including results of clinical trials, does not have a significant impact on beliefs about the efficacy of such vaccines, concerns over side effects, or intentions to receive a vaccine. Study 2 (n = 2217) addressed concerns that highlighting the need to maintain protective behaviours (e.g., social distancing) post-vaccination may lower perceptions of vaccine efficacy and willingness to receive a vaccine. We do not find evidence of this: transparent messages did not significantly reduce perceptions of vaccine efficacy, and in some cases increased perceptions of efficacy. We again report no main effect of messages on intentions to receive a vaccine. The results of both studies suggest that transparently informing people of the limitations of vaccinations does not reduce intentions to be vaccinated but neither does it increase intentions to engage in protective behaviours post-vaccination.