Expert elicitation on the relative importance of possible SARS-CoV-2 transmission routes and the effectiveness of mitigations.

OBJECTIVES: To help people make decisions about the most effective mitigation measures against SARS-CoV-2 transmission in different scenarios, the likelihoods of transmission by different routes need to be quantified to some degree (however uncertain). These likelihoods need to be communicated in an appropriate way to illustrate the relative importance of different routes in different scenarios, the likely effectiveness of different mitigation measures along those routes, and the level of uncertainty in those estimates. In this study, a pragmatic expert elicitation was undertaken to supply the underlying quantitative values to produce such a communication tool. PARTICIPANTS: Twenty-seven individual experts from five countries and many scientific disciplines provided estimates. OUTCOME MEASURES: Estimates of transmission parameters, assessments of the quality of the evidence, references to relevant literature, rationales for their estimates and sources of uncertainty. RESULTS AND CONCLUSION: The participants' responses showed that there is still considerable disagreement among experts about the relative importance of different transmission pathways and the effectiveness of different mitigation measures due to a lack of empirical evidence. Despite these disagreements, when pooled, the majority views on each parameter formed an internally consistent set of estimates (for example, that transmission was more likely indoors than outdoors, and at closer range), which formed the basis of a visualisation to help individuals and organisations understand the factors that influence transmission and the potential benefits of different mitigation measures.

The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials.

BACKGROUND: The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public. METHODS: In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured. FINDINGS: Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was 'low', rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was 'high', and in one of the two studies, those shown 'low' quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was 'high' showed no statistically significant differences on these measures compared to those given no information about evidence quality. CONCLUSIONS: Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.

Sound human, steer clear of jargon, and be prepared.

Kevin McConway and David Spiegelhalter offer tips to statisticians communicating through the media, especially in the time of Covid-19.

Redevelopment of the Predict: Breast Cancer website and recommendations for developing interfaces to support decision-making.

OBJECTIVES: To develop a new interface for the widely used prognostic breast cancer tool: Predict: Breast Cancer. To facilitate decision-making around post-surgery breast cancer treatments. To derive recommendations for communicating the outputs of prognostic models to patients and their clinicians. METHOD: We employed a user-centred design process comprised of background research and iterative testing of prototypes with clinicians and patients. Methods included surveys, focus groups and usability testing. RESULTS: The updated interface now caters to the needs of a wider audience through the addition of new visualisations, instantaneous updating of results, enhanced explanatory information and the addition of new predictors and outputs. A programme of future research was identified and is now underway, including the provision of quantitative data on the adverse effects of adjuvant breast cancer treatments. Based on our user-centred design process, we identify six recommendations for communicating the outputs of prognostic models including the need to contextualise statistics, identify and address gaps in knowledge, and the critical importance of engaging with prospective users when designing communications. CONCLUSIONS: For prognostic algorithms to fulfil their potential to assist with decision-making they need carefully designed interfaces. User-centred design puts patients and clinicians needs at the forefront, allowing them to derive the maximum benefit from prognostic models.

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty.

Communicating personalized risks from COVID-19: guidelines from an empirical study.

As increasing amounts of data accumulate on the effects of the novel coronavirus SARS-CoV-2 and the risk factors that lead to poor outcomes, it is possible to produce personalized estimates of the risks faced by groups of people with different characteristics. The challenge of how to communicate these then becomes apparent. Based on empirical work (total n = 5520, UK) supported by in-person interviews with the public and physicians, we make recommendations on the presentation of such information. These include: using predominantly percentages when communicating the absolute risk, but also providing, for balance, a format which conveys a contrasting (higher) perception of risk (expected frequency out of 10 000); using a visual linear scale cut at an appropriate point to illustrate the maximum risk, explained through an illustrative 'persona' who might face that highest level of risk; and providing context to the absolute risk through presenting a range of other 'personas' illustrating people who would face risks of a wide range of different levels. These 'personas' should have their major risk factors (age, existing health conditions) described. By contrast, giving people absolute likelihoods of other risks they face in an attempt to add context was considered less helpful. We note that observed effect sizes generally were small. However, even small effects are meaningful and relevant when scaled up to population levels.

Factors associated with deaths due to COVID-19 versus other causes: population-based cohort analysis of UK primary care data and linked national death registrations within the OpenSAFELY platform.

BACKGROUND: Mortality from COVID-19 shows a strong relationship with age and pre-existing medical conditions, as does mortality from other causes. We aimed to investigate how specific factors are differentially associated with COVID-19 mortality as compared to mortality from causes other than COVID-19. METHODS: Working on behalf of NHS England, we carried out a cohort study within the OpenSAFELY platform. Primary care data from England were linked to national death registrations. We included all adults (aged ≥18 years) in the database on 1st February 2020 and with >1 year of continuous prior registration; the cut-off date for deaths was 9th November 2020. Associations between individual-level characteristics and COVID-19 and non-COVID deaths, classified according to the presence of a COVID-19 code as the underlying cause of death on the death certificate, were estimated by fitting age- and sex-adjusted logistic models for these two outcomes. FINDINGS: 17,456,515 individuals were included. 17,063 died from COVID-19 and 134,316 from other causes. Most factors associated with COVID-19 death were similarly associated with non-COVID death, but the magnitudes of association differed. Older age was more strongly associated with COVID-19 death than non-COVID death (e.g. ORs 40.7 [95% CI 37.7-43.8] and 29.6 [28.9-30.3] respectively for ≥80 vs 50-59 years), as was male sex, deprivation, obesity, and some comorbidities. Smoking, history of cancer and chronic liver disease had stronger associations with non-COVID than COVID-19 death. All non-white ethnic groups had higher odds than white of COVID-19 death (OR for Black: 2.20 [1.96-2.47], South Asian: 2.33 [2.16-2.52]), but lower odds than white of non-COVID death (Black: 0.88 [0.83-0.94], South Asian: 0.78 [0.75-0.81]). INTERPRETATION: Similar associations of most individual-level factors with COVID-19 and non-COVID death suggest that COVID-19 largely multiplies existing risks faced by patients, with some notable exceptions. Identifying the unique factors contributing to the excess COVID-19 mortality risk among non-white groups is a priority to inform efforts to reduce deaths from COVID-19. FUNDING: Wellcome, Royal Society, National Institute for Health Research, National Institute for Health Research Oxford Biomedical Research Centre, UK Medical Research Council, Health Data Research UK.

The adverse effects of bisphosphonates in breast cancer: A systematic review and network meta-analysis.

BACKGROUND: Bisphosphonate drugs can be used to improve the outcomes of women with breast cancer. Whilst many meta-analyses have quantified their potential benefits for patients, attempts at comprehensive quantification of potential adverse effects have been limited. We undertook a meta-analysis with novel methodology to identify and quantify these adverse effects. METHODS: We systematically reviewed randomised controlled trials in breast cancer where at least one of the treatments was a bisphosphonate (zoledronic acid, ibandronate, pamidronate, alendronate or clodronate). Neoadjuvant, adjuvant and metastatic settings were examined. Primary outcomes were adverse events of any type or severity (excluding death). We carried out pairwise and network meta-analyses to estimate the size of any adverse effects potentially related to bisphosphonates. In order to ascertain whether adverse effects differed by individual factors such as age, or interacted with other common adjuvant breast cancer treatments, we examined individual-level patient data for one large trial, AZURE. FINDINGS: We identified 56 trials that reported adverse data, which included a total of 29,248 patients (18,301 receiving bisphosphonate drugs versus 10,947 not). 24 out of the 103 different adverse outcomes analysed showed a statistically and practically significant increase in patients receiving a bisphosphonate drug compared with those not (2 additional outcomes that appeared statistically significant came only from small studies with low event counts and no clinical suspicion so are likely artifacts). Most of these 24 are already clinically recognised: 'flu-like symptoms, fever, headache and chills; increased bone pain, arthralgia, myalgia, back pain; cardiac events, thromboembolic events; hypocalcaemia and osteonecrosis of the jaw; as well as possibly stiffness and nausea. Oral clodronate appeared to increase the risk of vomiting and diarrhoea (which may also be increased by other bisphosphonates), and there may be some hepatotoxicity. Four additional potential adverse effects emerged for bisphosphonate drugs in this analysis which have not classically be recognised: fatigue, neurosensory problems, hypertonia/muscle spasms and possibly dysgeusia. Several symptoms previously reported as potential side effects in the literature were not significantly increased in this analysis: constipation, insomnia, respiratory problems, oedema or thirst/dry mouth. Individual patient-level data and subgroup analysis revealed little variation in side effects between women of different ages or menopausal status, those with metastatic versus non-metastatic cancer, or between women receiving different concurrent breast cancer therapies. CONCLUSIONS: This meta-analysis has produced estimates for the absolute frequencies of a range of side effects significantly associated with bisphosphonate drugs when used by breast cancer patients. These results show good agreement with previous literature on the subject but are the first systematic quantification of side effects and their severities. However, the analysis is limited by the availability and quality of data on adverse events, and the potential for bias introduced by a lack of standards for reporting of such events. We therefore present a table of adverse effects for bisphosphonates, identified and quantified to the best of our ability from a large number of trials, which we hope can be used to improve the communication of the potential harms of these drugs to patients and their healthcare providers.

Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort study.

OBJECTIVE: To derive and validate a risk prediction algorithm to estimate hospital admission and mortality outcomes from coronavirus disease 2019 (covid-19) in adults. DESIGN: Population based cohort study. SETTING AND PARTICIPANTS: QResearch database, comprising 1205 general practices in England with linkage to covid-19 test results, Hospital Episode Statistics, and death registry data. 6.08 million adults aged 19-100 years were included in the derivation dataset and 2.17 million in the validation dataset. The derivation and first validation cohort period was 24 January 2020 to 30 April 2020. The second temporal validation cohort covered the period 1 May 2020 to 30 June 2020. MAIN OUTCOME MEASURES: The primary outcome was time to death from covid-19, defined as death due to confirmed or suspected covid-19 as per the death certification or death occurring in a person with confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in the period 24 January to 30 April 2020. The secondary outcome was time to hospital admission with confirmed SARS-CoV-2 infection. Models were fitted in the derivation cohort to derive risk equations using a range of predictor variables. Performance, including measures of discrimination and calibration, was evaluated in each validation time period. RESULTS: 4384 deaths from covid-19 occurred in the derivation cohort during follow-up and 1722 in the first validation cohort period and 621 in the second validation cohort period. The final risk algorithms included age, ethnicity, deprivation, body mass index, and a range of comorbidities. The algorithm had good calibration in the first validation cohort. For deaths from covid-19 in men, it explained 73.1% (95% confidence interval 71.9% to 74.3%) of the variation in time to death (R2); the D statistic was 3.37 (95% confidence interval 3.27 to 3.47), and Harrell's C was 0.928 (0.919 to 0.938). Similar results were obtained for women, for both outcomes, and in both time periods. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths within 97 days was 75.7%. People in the top 20% of predicted risk of death accounted for 94% of all deaths from covid-19. CONCLUSION: The QCOVID population based risk algorithm performed well, showing very high levels of discrimination for deaths and hospital admissions due to covid-19. The absolute risks presented, however, will change over time in line with the prevailing SARS-C0V-2 infection rate and the extent of social distancing measures in place, so they should be interpreted with caution. The model can be recalibrated for different time periods, however, and has the potential to be dynamically updated as the pandemic evolves.

The effects of communicating uncertainty on public trust in facts and numbers.

Uncertainty is inherent to our knowledge about the state of the world yet often not communicated alongside scientific facts and numbers. In the "posttruth" era where facts are increasingly contested, a common assumption is that communicating uncertainty will reduce public trust. However, a lack of systematic research makes it difficult to evaluate such claims. We conducted five experiments-including one preregistered replication with a national sample and one field experiment on the BBC News website (total n = 5,780)-to examine whether communicating epistemic uncertainty about facts across different topics (e.g., global warming, immigration), formats (verbal vs. numeric), and magnitudes (high vs. low) influences public trust. Results show that whereas people do perceive greater uncertainty when it is communicated, we observed only a small decrease in trust in numbers and trustworthiness of the source, and mostly for verbal uncertainty communication. These results could help reassure all communicators of facts and science that they can be more open and transparent about the limits of human knowledge.

Communicating uncertainty about facts, numbers and science.

Uncertainty is an inherent part of knowledge, and yet in an era of contested expertise, many shy away from openly communicating their uncertainty about what they know, fearful of their audience's reaction. But what effect does communication of such epistemic uncertainty have? Empirical research is widely scattered across many disciplines. This interdisciplinary review structures and summarizes current practice and research across domains, combining a statistical and psychological perspective. This informs a framework for uncertainty communication in which we identify three objects of uncertainty-facts, numbers and science-and two levels of uncertainty: direct and indirect. An examination of current practices provides a scale of nine expressions of direct uncertainty. We discuss attempts to codify indirect uncertainty in terms of quality of the underlying evidence. We review the limited literature about the effects of communicating epistemic uncertainty on cognition, affect, trust and decision-making. While there is some evidence that communicating epistemic uncertainty does not necessarily affect audiences negatively, impact can vary between individuals and communication formats. Case studies in economic statistics and climate change illustrate our framework in action. We conclude with advice to guide both communicators and future researchers in this important but so far rather neglected field.

Conducting fit-for-purpose food safety risk assessments.

The interplay between science, risk assessment and risk management has always been complex, and even more so in a world increasingly characterised by rapid technical innovation, new modes of communication, suspicion about authorities and experts, and demands for people to have a say in decisions that are made on their behalf. In this challenging era where scientific advice on food safety has never been in greater demand, risk managers should effectively navigate the interplay between facts and values and be able to rely on robust and fit-for-purpose risk assessments to aid them. The fact that societal resistance is often encountered when scientific advice on food safety operates at a distance from social values and fails to actively engage with citizens, has led to increasing emphasis on the need to advance forms of risk assessment that are more contextual, and socially sound and accountable. EFSA's third Scientific Conference explored how risk assessments could be constructed to most usefully meet society's needs and thus connect science with society, while remaining scientifically robust. Contributors to the conference highlighted the need to: (1) frame risk assessments by clear policy goals and decision-making criteria; (2) begin risk assessments with an explicit problem formulation to identify relevant information; (3) make use of reliable risk assessment studies; (4) be explicit about value judgements; (5) address and communicate scientific uncertainty; (6) follow trustworthy processes; (7) publish the evidence and data, and report the way in which they are used in a transparent manner; (8) ensure effective communication throughout the risk analysis process; (9) involve society, as appropriate; and (10) weigh risks and benefits on request. Implementation of these recommendations would contribute to increased credibility and trustworthiness of food safety risk assessments.

Communicating health risks in science publications: time for everyone to take responsibility.

Research that is poorly communicated or presented is as potentially damaging as research that is poorly conducted or fraudulent. Recent examples illustrate how the problem often lies with researchers, not press officers or journalists. The quest for publication and 'impact' must not outweigh the importance of accurate representation of science; herein, we suggest steps that researchers, journalists and press officers can take to help ensure this.