ABSTRACT
Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.
Subject(s)
Research Design , Social Behavior , Humans , Prospective Studies , Reproducibility of Results , Sample SizeABSTRACT
Research in a few U.S. states has shown that candidates listed first on ballots gain extra votes as a result. This study explored name order effects for the first time in New Hampshire, where such effects might be weak or entirely absent because of high political engagement and the use of party column ballots. In general elections (in 2012 and 2016) for federal offices and the governorship and in primaries (in 2000, 2002, and 2004), evidence of primacy effects appeared in 86% of the 84 tests, including the 2016 presidential race, when Donald Trump gained 1.7 percentage points from first listing, and Hillary Clinton gained 1.5 percentage points. Consistent with theoretical predictions, primacy effects were larger in primaries and for major-party candidates in general elections than for non-major-party candidates in general elections, more pronounced in less publicized contests, and stronger in contests without an incumbent running. All of this constitutes evidence of the reliability and generalizability of evidence on candidate name order effects and their moderators.