The effects of communicating uncertainty on public trust in facts and numbers.

Uncertainty is inherent to our knowledge about the state of the world yet often not communicated alongside scientific facts and numbers. In the "posttruth" era where facts are increasingly contested, a common assumption is that communicating uncertainty will reduce public trust. However, a lack of systematic research makes it difficult to evaluate such claims. We conducted five experiments-including one preregistered replication with a national sample and one field experiment on the BBC News website (total n = 5,780)-to examine whether communicating epistemic uncertainty about facts across different topics (e.g., global warming, immigration), formats (verbal vs. numeric), and magnitudes (high vs. low) influences public trust. Results show that whereas people do perceive greater uncertainty when it is communicated, we observed only a small decrease in trust in numbers and trustworthiness of the source, and mostly for verbal uncertainty communication. These results could help reassure all communicators of facts and science that they can be more open and transparent about the limits of human knowledge.

Communicating health risks in science publications: time for everyone to take responsibility.

Research that is poorly communicated or presented is as potentially damaging as research that is poorly conducted or fraudulent. Recent examples illustrate how the problem often lies with researchers, not press officers or journalists. The quest for publication and 'impact' must not outweigh the importance of accurate representation of science; herein, we suggest steps that researchers, journalists and press officers can take to help ensure this.

Implementing informative priors for heterogeneity in meta-analysis using meta-regression and pseudo data.

Many meta-analyses combine results from only a small number of studies, a situation in which the between-study variance is imprecisely estimated when standard methods are applied. Bayesian meta-analysis allows incorporation of external evidence on heterogeneity, providing the potential for more robust inference on the effect size of interest. We present a method for performing Bayesian meta-analysis using data augmentation, in which we represent an informative conjugate prior for between-study variance by pseudo data and use meta-regression for estimation. To assist in this, we derive predictive inverse-gamma distributions for the between-study variance expected in future meta-analyses. These may serve as priors for heterogeneity in new meta-analyses. In a simulation study, we compare approximate Bayesian methods using meta-regression and pseudo data against fully Bayesian approaches based on importance sampling techniques and Markov chain Monte Carlo (MCMC). We compare the frequentist properties of these Bayesian methods with those of the commonly used frequentist DerSimonian and Laird procedure. The method is implemented in standard statistical software and provides a less complex alternative to standard MCMC approaches. An importance sampling approach produces almost identical results to standard MCMC approaches, and results obtained through meta-regression and pseudo data are very similar. On average, data augmentation provides closer results to MCMC, if implemented using restricted maximum likelihood estimation rather than DerSimonian and Laird or maximum likelihood estimation. The methods are applied to real datasets, and an extension to network meta-analysis is described. The proposed method facilitates Bayesian meta-analysis in a way that is accessible to applied researchers. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

How old are you, really? Communicating chronic risk through 'effective age' of your body and organs.

In communicating chronic risks, there is increasing use of a metaphor that can be termed 'effective-age': the age of a 'healthy' person who has the same risk profile as the individual in question. Popular measures include 'real-age', 'heart-age', 'lung-age' and so on.Here we formally define this concept, and illustrate its use in a variety of areas. We explore conditions under which the years lost or gained that are associated with exposure to risk factors depends neither on current chronological age, nor the period over which the risk is defined. These conditions generally hold for all-cause adult mortality, which enables a simple and vivid translation from hazard-ratios to years lost or gained off chronological age. Finally we consider the attractiveness and impact of this concept.Under reasonable assumptions, the risks associated with specific behaviours can be expressed in terms of years gained or lost off your effective age. The idea of effective age appears a useful and attractive metaphor to vividly communicate risks to individuals.

Robust meta-analytic-predictive priors in clinical trials with historical control information.

Historical information is always relevant for clinical trial design. Additionally, if incorporated in the analysis of a new trial, historical data allow to reduce the number of subjects. This decreases costs and trial duration, facilitates recruitment, and may be more ethical. Yet, under prior-data conflict, a too optimistic use of historical data may be inappropriate. We address this challenge by deriving a Bayesian meta-analytic-predictive prior from historical data, which is then combined with the new data. This prospective approach is equivalent to a meta-analytic-combined analysis of historical and new data if parameters are exchangeable across trials. The prospective Bayesian version requires a good approximation of the meta-analytic-predictive prior, which is not available analytically. We propose two- or three-component mixtures of standard priors, which allow for good approximations and, for the one-parameter exponential family, straightforward posterior calculations. Moreover, since one of the mixture components is usually vague, mixture priors will often be heavy-tailed and therefore robust. Further robustness and a more rapid reaction to prior-data conflicts can be achieved by adding an extra weakly-informative mixture component. Use of historical prior information is particularly attractive for adaptive trials, as the randomization ratio can then be changed in case of prior-data conflict. Both frequentist operating characteristics and posterior summaries for various data scenarios show that these designs have desirable properties. We illustrate the methodology for a phase II proof-of-concept trial with historical controls from four studies. Robust meta-analytic-predictive priors alleviate prior-data conflicts ' they should encourage better and more frequent use of historical data in clinical trials.

Modelling hepatitis C infection acquired from blood transfusions in the UK between 1970 and 1991 for the Infected Blood Inquiry.

The Statistics Expert Group was convened at the request of the Infected Blood Inquiry to provide estimates of the number of infections and deaths from blood-borne infections including hepatitis B virus, human immunodeficiency virus, hepatitis C virus (HCV) and variant Creutzfeldt Jakob disease, as a direct result of contaminated blood and blood products administered in the United Kingdom of Great Britain and Northern Ireland (UK). In the absence of databases of HCV infections and related deaths for all nations of the UK, a statistical model was required to estimate the number of infections and subsequent deaths from HCV acquired from blood transfusions from January 1970 to August 1991. We present this statistical model in detail alongside the results of its application to each of the four nations in the UK. We estimated that 26 800 people (95% uncertainty interval 21 300-38 800) throughout the UK were chronically infected with HCV because of contaminated blood transfusions between January 1970 and August 1991. The number of deaths up to the end of 2019 that occurred as a result of this chronic infection is estimated to be 1820 (95% uncertainty interval 650-3320).

Impact of plain packaging of tobacco products on smoking in adults and children: an elicitation of international experts' estimates.

BACKGROUND: Governments sometimes face important decisions in the absence of direct evidence. In these cases, expert elicitation methods can be used to quantify uncertainty. We report the results of an expert elicitation study regarding the likely impact on smoking rates in adults and children of plain packaging of tobacco products. METHODS: Thirty-three tobacco control experts were recruited from the UK (n = 14), Australasia (n = 12) and North America (n = 7). Experts' estimates were individually elicited via telephone interviews, and then linearly pooled. Elicited estimates consisted of (1) the most likely, (2) the highest possible, and (3) the lowest possible value for the percentage of (a) adult smokers and (b) children trying smoking, two years after the introduction of plain packaging (all other things being constant) in a target country in the expert's region of residence. RESULTS: The median estimate for the impact on adult smoking prevalence was a 1 percentage point decline (99% range 2.25 to 0), and for the percentage of children trying smoking was a 3 percentage point decline (99% range 6.1 to 0), the latter estimated impact being larger than the former (P < 0.001, sign test). There were no differences in either estimate by region (I2: Adults: 0; Children: 0) but there was considerable variability between experts' estimates within regions (I2: Adults: 0.91; Children: 0.89). CONCLUSIONS: In the absence of direct evidence for the impact of introducing plain packaging on smoking rates in adults and children, this study shows that tobacco control experts felt the most likely outcomes would be a reduction in smoking prevalence in adults, and a greater reduction in the numbers of children trying smoking, although there was substantial variability in the estimated size of these impacts. No experts judged an increase in smoking as a likely outcome.

Use of implicit and explicit bayesian methods in health technology assessment.

OBJECTIVES: The aim of this study was to examine the use of implicit and explicit Bayesian methods in health technology assessments and to identify whether this has changed over time. METHODS: A review of all health technology assessment (HTA) reports of secondary research published by the UK National Institute of Health Research (NIHR) between 1997 and 2011. Data were extracted on the use and implementation of Bayesian methods, whether defined as such by the original authors (i.e., explicit) or not (i.e., implicit). RESULTS: A total of 155 of 375 (41 percent) NIHR HTA reports, identified as relevant to this review, contained a Bayesian analysis. Of these, 128 (83 percent) contained an implicit Bayesian analysis, 3 (2 percent) an explicit Bayesian analysis and 24 (15 percent) both implicit and explicit Bayesian analyses. Of the twenty-seven reports that explicitly used Bayes theorem, only six included prior information in the form of (informative) prior distributions. Over time, the percentage of HTA reports that used Bayesian (implicit and/or explicit) methods increased from 0 percent in 1997 to nearly 80 percent in 2011. CONCLUSIONS: This review has shown that there has been an increase in the use of Bayesian methods in HTA, which is likely to be a result of the increase in freely available resources to implement the approach. Areas where Bayesian methods have the potential to advance healthcare evaluations in the future are considered in the discussion.

Development of a transparent interactive decision interrogator to facilitate the decision-making process in health care.

BACKGROUND: Decisions about the use of new technologies in health care are often based on complex economic models. Decision makers frequently make informal judgments about evidence, uncertainty, and the assumptions that underpin these models. OBJECTIVES: Transparent interactive decision interrogator (TIDI) facilitates more formal critique of decision models by decision makers such as members of appraisal committees of the National Institute for Health and Clinical Excellence in the UK. By allowing them to run advanced statistical models under different scenarios in real time, TIDI can make the decision process more efficient and transparent, while avoiding limitations on pre-prepared analysis. METHODS: TIDI, programmed in Visual Basic for applications within Excel, provides an interface for controlling all components of a decision model developed in the appropriate software (e.g., meta-analysis in WinBUGS and the decision model in R) by linking software packages using RExcel and R2WinBUGS. TIDI's graphical controls allow the user to modify assumptions and to run the decision model, and results are returned to an Excel spreadsheet. A tool displaying tornado plots helps to evaluate the influence of individual parameters on the model outcomes, and an interactive meta-analysis module allows the user to select any combination of available studies, explore the impact of bias adjustment, and view results using forest plots. We demonstrate TIDI using an example of a decision model in antenatal care. CONCLUSION: Use of TIDI during the NICE appraisal of tumor necrosis factor-alpha inhibitors (in psoriatic arthritis) successfully demonstrated its ability to facilitate critiques of the decision models by decision makers.

How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

OBJECTIVES: To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. METHODS: Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. RESULTS: The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable-sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. CONCLUSIONS: The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.

Sound human, steer clear of jargon, and be prepared.

Kevin McConway and David Spiegelhalter offer tips to statisticians communicating through the media, especially in the time of Covid-19.

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty.

The adverse effects of bisphosphonates in breast cancer: A systematic review and network meta-analysis.

BACKGROUND: Bisphosphonate drugs can be used to improve the outcomes of women with breast cancer. Whilst many meta-analyses have quantified their potential benefits for patients, attempts at comprehensive quantification of potential adverse effects have been limited. We undertook a meta-analysis with novel methodology to identify and quantify these adverse effects. METHODS: We systematically reviewed randomised controlled trials in breast cancer where at least one of the treatments was a bisphosphonate (zoledronic acid, ibandronate, pamidronate, alendronate or clodronate). Neoadjuvant, adjuvant and metastatic settings were examined. Primary outcomes were adverse events of any type or severity (excluding death). We carried out pairwise and network meta-analyses to estimate the size of any adverse effects potentially related to bisphosphonates. In order to ascertain whether adverse effects differed by individual factors such as age, or interacted with other common adjuvant breast cancer treatments, we examined individual-level patient data for one large trial, AZURE. FINDINGS: We identified 56 trials that reported adverse data, which included a total of 29,248 patients (18,301 receiving bisphosphonate drugs versus 10,947 not). 24 out of the 103 different adverse outcomes analysed showed a statistically and practically significant increase in patients receiving a bisphosphonate drug compared with those not (2 additional outcomes that appeared statistically significant came only from small studies with low event counts and no clinical suspicion so are likely artifacts). Most of these 24 are already clinically recognised: 'flu-like symptoms, fever, headache and chills; increased bone pain, arthralgia, myalgia, back pain; cardiac events, thromboembolic events; hypocalcaemia and osteonecrosis of the jaw; as well as possibly stiffness and nausea. Oral clodronate appeared to increase the risk of vomiting and diarrhoea (which may also be increased by other bisphosphonates), and there may be some hepatotoxicity. Four additional potential adverse effects emerged for bisphosphonate drugs in this analysis which have not classically be recognised: fatigue, neurosensory problems, hypertonia/muscle spasms and possibly dysgeusia. Several symptoms previously reported as potential side effects in the literature were not significantly increased in this analysis: constipation, insomnia, respiratory problems, oedema or thirst/dry mouth. Individual patient-level data and subgroup analysis revealed little variation in side effects between women of different ages or menopausal status, those with metastatic versus non-metastatic cancer, or between women receiving different concurrent breast cancer therapies. CONCLUSIONS: This meta-analysis has produced estimates for the absolute frequencies of a range of side effects significantly associated with bisphosphonate drugs when used by breast cancer patients. These results show good agreement with previous literature on the subject but are the first systematic quantification of side effects and their severities. However, the analysis is limited by the availability and quality of data on adverse events, and the potential for bias introduced by a lack of standards for reporting of such events. We therefore present a table of adverse effects for bisphosphonates, identified and quantified to the best of our ability from a large number of trials, which we hope can be used to improve the communication of the potential harms of these drugs to patients and their healthcare providers.

The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials.

BACKGROUND: The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public. METHODS: In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured. FINDINGS: Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was 'low', rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was 'high', and in one of the two studies, those shown 'low' quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was 'high' showed no statistically significant differences on these measures compared to those given no information about evidence quality. CONCLUSIONS: Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.

Redevelopment of the Predict: Breast Cancer website and recommendations for developing interfaces to support decision-making.

OBJECTIVES: To develop a new interface for the widely used prognostic breast cancer tool: Predict: Breast Cancer. To facilitate decision-making around post-surgery breast cancer treatments. To derive recommendations for communicating the outputs of prognostic models to patients and their clinicians. METHOD: We employed a user-centred design process comprised of background research and iterative testing of prototypes with clinicians and patients. Methods included surveys, focus groups and usability testing. RESULTS: The updated interface now caters to the needs of a wider audience through the addition of new visualisations, instantaneous updating of results, enhanced explanatory information and the addition of new predictors and outputs. A programme of future research was identified and is now underway, including the provision of quantitative data on the adverse effects of adjuvant breast cancer treatments. Based on our user-centred design process, we identify six recommendations for communicating the outputs of prognostic models including the need to contextualise statistics, identify and address gaps in knowledge, and the critical importance of engaging with prospective users when designing communications. CONCLUSIONS: For prognostic algorithms to fulfil their potential to assist with decision-making they need carefully designed interfaces. User-centred design puts patients and clinicians needs at the forefront, allowing them to derive the maximum benefit from prognostic models.

Expert elicitation on the relative importance of possible SARS-CoV-2 transmission routes and the effectiveness of mitigations.

OBJECTIVES: To help people make decisions about the most effective mitigation measures against SARS-CoV-2 transmission in different scenarios, the likelihoods of transmission by different routes need to be quantified to some degree (however uncertain). These likelihoods need to be communicated in an appropriate way to illustrate the relative importance of different routes in different scenarios, the likely effectiveness of different mitigation measures along those routes, and the level of uncertainty in those estimates. In this study, a pragmatic expert elicitation was undertaken to supply the underlying quantitative values to produce such a communication tool. PARTICIPANTS: Twenty-seven individual experts from five countries and many scientific disciplines provided estimates. OUTCOME MEASURES: Estimates of transmission parameters, assessments of the quality of the evidence, references to relevant literature, rationales for their estimates and sources of uncertainty. RESULTS AND CONCLUSION: The participants' responses showed that there is still considerable disagreement among experts about the relative importance of different transmission pathways and the effectiveness of different mitigation measures due to a lack of empirical evidence. Despite these disagreements, when pooled, the majority views on each parameter formed an internally consistent set of estimates (for example, that transmission was more likely indoors than outdoors, and at closer range), which formed the basis of a visualisation to help individuals and organisations understand the factors that influence transmission and the potential benefits of different mitigation measures.