Statistical significance and its critics: practicing damaging science, or damaging scientific practice?

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves risk damaging science. We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim-if all thresholds are abandoned-then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variability.

Statistics, philosophy, and health: the SMAC 2021 webconference.

SMAC 2021 was a webconference organized in June 2021. The aim of this conference was to bring together data scientists, (bio)statisticians, philosophers, and any person interested in the questions of causality and Bayesian statistics, ranging from technical to philosophical aspects. This webconference consisted of keynote speakers and contributed speakers, and closed with a round-table organized in an unusual fashion. Indeed, organisers asked world renowned scientists to prepare two videos: a short video presenting a question of interest to them and a longer one presenting their point of view on the question. The first video served as a "teaser" for the conference and the second were presented during the conference as an introduction to the round-table. These videos and this round-table generated original scientific insights and discussion worthy of being shared with the community which we do by means of this paper.