Ensuring the quality and specificity of preregistrations.

Researchers face many, often seemingly arbitrary, choices in formulating hypotheses, designing protocols, collecting data, analyzing data, and reporting results. Opportunistic use of "researcher degrees of freedom" aimed at obtaining statistical significance increases the likelihood of obtaining and publishing false-positive results and overestimated effect sizes. Preregistration is a mechanism for reducing such degrees of freedom by specifying designs and analysis plans before observing the research outcomes. The effectiveness of preregistration may depend, in part, on whether the process facilitates sufficiently specific articulation of such plans. In this preregistered study, we compared 2 formats of preregistration available on the OSF: Standard Pre-Data Collection Registration and Prereg Challenge Registration (now called "OSF Preregistration," http://osf.io/prereg/). The Prereg Challenge format was a "structured" workflow with detailed instructions and an independent review to confirm completeness; the "Standard" format was "unstructured" with minimal direct guidance to give researchers flexibility for what to prespecify. Results of comparing random samples of 53 preregistrations from each format indicate that the "structured" format restricted the opportunistic use of researcher degrees of freedom better (Cliff's Delta = 0.49) than the "unstructured" format, but neither eliminated all researcher degrees of freedom. We also observed very low concordance among coders about the number of hypotheses (14%), indicating that they are often not clearly stated. We conclude that effective preregistration is challenging, and registration formats that provide effective guidance may improve the quality of research.

Why and when suffering increases the perceived likelihood of fortuitous rewards.

Cultural practices and anecdotal accounts suggest that people expect suffering to lead to fortuitous rewards. To shed light on this illusory 'suffering-reward' association, we tested why and when this effect manifests. Across three vignette studies in which we manipulated the degree of suffering experienced by the protagonist, we tested a 'just-world maintenance' explanation (suffering deserves to be compensated) and a 'virtuous suffering' explanation (suffering indicates virtues, which will be rewarded). Our findings revealed that the illusory 'suffering-reward' association (1) could serve as a way for people to cope with just-world threats posed by the suffering of innocent victims, and (2) manifested when the suffering was not caused by the victim's own behaviour and not readily attributable to bad luck. Taken together, these findings not only provide evidence for the existence of the illusory 'suffering-reward' association but also elucidate its psychological underpinnings.

Recommendations in pre-registrations and internal review board proposals promote formal power analyses but do not increase sample size.

In this preregistered study, we investigated whether the statistical power of a study is higher when researchers are asked to make a formal power analysis before collecting data. We compared the sample size descriptions from two sources: (i) a sample of pre-registrations created according to the guidelines for the Center for Open Science Preregistration Challenge (PCRs) and a sample of institutional review board (IRB) proposals from Tilburg School of Behavior and Social Sciences, which both include a recommendation to do a formal power analysis, and (ii) a sample of pre-registrations created according to the guidelines for Open Science Framework Standard Pre-Data Collection Registrations (SPRs) in which no guidance on sample size planning is given. We found that PCRs and IRBs (72%) more often included sample size decisions based on power analyses than the SPRs (45%). However, this did not result in larger planned sample sizes. The determined sample size of the PCRs and IRB proposals (Md = 90.50) was not higher than the determined sample size of the SPRs (Md = 126.00; W = 3389.5, p = 0.936). Typically, power analyses in the registrations were conducted with G*power, assuming a medium effect size, α = .05 and a power of .80. Only 20% of the power analyses contained enough information to fully reproduce the results and only 62% of these power analyses pertained to the main hypothesis test in the pre-registration. Therefore, we see ample room for improvements in the quality of the registrations and we offer several recommendations to do so.

A comprehensive meta-analysis of money priming.

Research on money priming typically investigates whether exposure to money-related stimuli can affect people's thoughts, feelings, motivations, and behaviors (for a review, see Vohs, 2015). Our study answers the call for a comprehensive meta-analysis examining the available evidence on money priming (Vadillo, Hardwicke, & Shanks, 2016). By conducting a systematic search of published and unpublished literature on money priming, we sought to achieve three key goals. First, we aimed to assess the presence of biases in the available published literature (e.g., publication bias). Second, in the case of such biases, we sought to derive a more accurate estimate of the effect size after correcting for these biases. Third, we aimed to investigate whether design factors such as prime type and study setting moderated the money priming effects. Our overall meta-analysis included 246 suitable experiments and showed a significant overall effect size estimate (Hedges' g = .31, 95% CI [0.26, 0.36]). However, publication bias and related biases are likely given the asymmetric funnel plots, Egger's test and two other tests for publication bias. Moderator analyses offered insight into the variation of the money priming effect, suggesting for various types of study designs whether the effect was present, absent, or biased. We found the largest money priming effect in lab studies investigating a behavioral dependent measure using a priming technique in which participants actively handled money. Future research should use sufficiently powerful preregistered studies to replicate these findings. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Moral judgment modulation by disgust priming via altered fronto-temporal functional connectivity.

Moral judgments are not just the product of conscious reasoning, but also involve the integration of social and emotional information. Irrelevant disgust stimuli modulate moral judgments, with individual sensitivity determining the direction and size of effects across both hypothetical and incentive-compatible experimental designs. We investigated the neural circuitry underlying this modulation using fMRI in 19 individuals performing a moral judgment task with subliminal priming of disgust facial expressions. Our results indicate that individual changes in moral acceptability due to priming covaried with individual differences in activation within the dorsomedial prefrontal cortex (dmPFC). Further, whole-brain analyses identified changes in functional connectivity between the dmPFC and the temporal-parietal junction (TPJ). High sensitivity individuals showed enhanced functional connectivity between the TPJ and dmPFC, corresponding with deactivation in the dmPFC, and rating the moral dilemmas as more acceptable. Low sensitivity individuals showed the opposite pattern of results. Post-hoc, these findings replicated in the dorsal anterior cingulate cortex (daMCC), an adjacent region implicated in converting between objective and subjective valuation. This suggests a specific computational mechanism - that disgust stimuli modulate moral judgments by altering the integration of social information to determine the subjective valuation of the considered moral actions.

Moral judgment modulation by disgust is bi-directionally moderated by individual sensitivity.

Modern theories of moral judgment predict that both conscious reasoning and unconscious emotional influences affect the way people decide about right and wrong. In a series of experiments, we tested the effect of subliminal and conscious priming of disgust facial expressions on moral dilemmas. "Trolley-car"-type scenarios were used, with subjects rating how acceptable they found the utilitarian course of action to be. On average, subliminal priming of disgust facial expressions resulted in higher rates of utilitarian judgments compared to neutral facial expressions. Further, in replication, we found that individual change in moral acceptability ratings due to disgust priming was modulated by individual sensitivity to disgust, revealing a bi-directional function. Our second replication extended this result to show that the function held for both subliminally and consciously presented stimuli. Combined across these experiments, we show a reliable bi-directional function, with presentation of disgust expression primes to individuals with higher disgust sensitivity resulting in more utilitarian judgments (i.e., number-based) and presentations to individuals with lower sensitivity resulting in more deontological judgments (i.e., rules-based). Our results may reconcile previous conflicting reports of disgust modulation of moral judgment by modeling how individual sensitivity to disgust determines the direction and degree of this effect.