Perspectives on scientific error.

Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.

Extraordinary claims, extraordinary evidence? A discussion.

Roberts (2020, Learning & Behavior, 48[2], 191-192) discussed research claiming honeybees can do arithmetic. Some readers of this research might regard such claims as unlikely. The present authors used this example as a basis for a debate on the criterion that ought to be used for publication of results or conclusions that could be viewed as unlikely by a significant number of readers, editors, or reviewers.

The effect of preregistration on trust in empirical research findings: results of a registered report.

The crisis of confidence has undermined the trust that researchers place in the findings of their peers. In order to increase trust in research, initiatives such as preregistration have been suggested, which aim to prevent various questionable research practices. As it stands, however, no empirical evidence exists that preregistration does increase perceptions of trust. The picture may be complicated by a researcher's familiarity with the author of the study, regardless of the preregistration status of the research. This registered report presents an empirical assessment of the extent to which preregistration increases the trust of 209 active academics in the reported outcomes, and how familiarity with another researcher influences that trust. Contrary to our expectations, we report ambiguous Bayes factors and conclude that we do not have strong evidence towards answering our research questions. Our findings are presented along with evidence that our manipulations were ineffective for many participants, leading to the exclusion of 68% of complete datasets, and an underpowered design as a consequence. We discuss other limitations and confounds which may explain why the findings of the study deviate from a previously conducted pilot study. We reflect on the benefits of using the registered report submission format in light of our results. The OSF page for this registered report and its pilot can be found here: http://dx.doi.org/10.17605/OSF.IO/B3K75.

Bayesian rank-based hypothesis testing for the rank sum test, the signed rank test, and Spearman's ρ.

Bayesian inference for rank-order problems is frustrated by the absence of an explicit likelihood function. This hurdle can be overcome by assuming a latent normal representation that is consistent with the ordinal information in the data: the observed ranks are conceptualized as an impoverished reflection of an underlying continuous scale, and inference concerns the parameters that govern the latent representation. We apply this generic data-augmentation method to obtain Bayes factors for three popular rank-based tests: the rank sum test, the signed rank test, and Spearman's ρ s .

Registered Replication Report: Strack, Martin, & Stepper (1988).

According to the facial feedback hypothesis, people's affective responses can be influenced by their own facial expression (e.g., smiling, pouting), even when their expression did not result from their emotional experiences. For example, Strack, Martin, and Stepper (1988) instructed participants to rate the funniness of cartoons using a pen that they held in their mouth. In line with the facial feedback hypothesis, when participants held the pen with their teeth (inducing a "smile"), they rated the cartoons as funnier than when they held the pen with their lips (inducing a "pout"). This seminal study of the facial feedback hypothesis has not been replicated directly. This Registered Replication Report describes the results of 17 independent direct replications of Study 1 from Strack et al. (1988), all of which followed the same vetted protocol. A meta-analysis of these studies examined the difference in funniness ratings between the "smile" and "pout" conditions. The original Strack et al. (1988) study reported a rating difference of 0.82 units on a 10-point Likert scale. Our meta-analysis revealed a rating difference of 0.03 units with a 95% confidence interval ranging from -0.11 to 0.16.

Sequential Sampling Models in Cognitive Neuroscience: Advantages, Applications, and Extensions.

Sequential sampling models assume that people make speeded decisions by gradually accumulating noisy information until a threshold of evidence is reached. In cognitive science, one such model--the diffusion decision model--is now regularly used to decompose task performance into underlying processes such as the quality of information processing, response caution, and a priori bias. In the cognitive neurosciences, the diffusion decision model has recently been adopted as a quantitative tool to study the neural basis of decision making under time pressure. We present a selective overview of several recent applications and extensions of the diffusion decision model in the cognitive neurosciences.

The falsifiability of actual decision-making models.

Jones and Dzhafarov (2014) provided a useful service in pointing out that some assumptions of modern decision-making models require additional scrutiny. Their main result, however, is not surprising: If an infinitely complex model was created by assigning its parameters arbitrarily flexible distributions, this new model would be able to fit any observed data perfectly. Such a hypothetical model would be unfalsifiable. This is exactly why such models have never been proposed in over half a century of model development in decision making. Additionally, the main conclusion drawn from this result-that the success of existing decision-making models can be attributed to assumptions about parameter distributions-is wrong. (PsycINFO Database Record (c) 2014 APA, all rights reserved).

Testing the counter model for perceptual identification: effects of repetition priming and word frequency.

The counter model for perceptual identification (Ratcliff & McKoon, 1997) differs from alternative views of word recognition in two important ways. First, it assumes that prior study of a word does not result in increased sensitivity but, rather, in bias. Second, the effects of word frequency and prior study are explained by different mechanisms. In the present experiment, study status and word frequency of target and foil were varied independently. Using a forced-choice task, we replicated the bias effect. However, we also found several interactions between frequency and prior study that are in direct conflict with the counter model. Most important, prior study of both alternatives resulted in an attenuation of the frequency effect and an increase in performance for low-frequency targets, but not for high-frequency targets. These findings suggest that the effects of frequency and prior study are not mediated by completely independent mechanisms.

A criterion-shift model for enhanced discriminability in perceptual identification: a note on the counter model.

The original version of the counter model for perceptual identification (Ratcliff & McKoon, 1997) assumed that word frequency and prior study act solely to bias the identification process (i.e., subjects have a tendency to prefer high-frequency and studied low-frequency words, irrespective of the presented word). In a recent study, using a two-alternative forced-choice paradigm, we showed an enhanced discriminability effect for high-frequency and studied low-frequency words (Wagenmakers, Zeelenberg, & Raaijmakers, 2000). These results have led to a fundamental modification of the counter model: Prior study and high frequency not only result in bias, but presumably also result in a higher rate of feature extraction (i.e., better perception). We demonstrate that a criterion-shift model, assuming limited perceptual information extracted from the flash as well as a reduced distance to an identification threshold for high-frequency and studied low-frequency words, can also account for enhanced discriminability.