Once and Again: Repeated Viewing Affects Judgments of Spontaneity and Preparation.

Reality is fleeting, and any moment can only be experienced once. Rewatching a video, however, allows people to repeatedly observe the exact same moment. We propose that people may fail to fully distinguish between merely observing behavior again (through replay) from that behavior being performed again in the exact same way. Using an assortment of stimuli that included auditions, commercials, and potential trial evidence, we demonstrated through nine experiments (N = 10,412 adults in the United States) that rewatching makes a recorded behavior appear more rehearsed and less spontaneous, as if the actors were simply precisely repeating their actions. These findings contribute to an emerging literature showing that incidental video features, like perspective or slow motion, can meaningfully change evaluations. Replay may inadvertently shape judgments in both mundane and consequential contexts. To understand how a video will influence its viewer, one will need to consider not only its content, but also how often it is viewed.

Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity.

Empirical audit and review is an approach to assessing the evidentiary value of a research area. It involves identifying a topic and selecting a cross-section of studies for replication. We apply the method to research on the psychological consequences of scarcity. Starting with the papers citing a seminal publication in the field, we conducted replications of 20 studies that evaluate the role of scarcity priming in pain sensitivity, resource allocation, materialism, and many other domains. There was considerable variability in the replicability, with some strong successes and other undeniable failures. Empirical audit and review does not attempt to assign an overall replication rate for a heterogeneous field, but rather facilitates researchers seeking to incorporate strength of evidence as they refine theories and plan new investigations in the research area. This method allows for an integration of qualitative and quantitative approaches to review and enables the growth of a cumulative science.

Psychology's Renaissance.

In 2010-2012, a few largely coincidental events led experimental psychologists to realize that their approach to collecting, analyzing, and reporting data made it too easy to publish false-positive findings. This sparked a period of methodological reflection that we review here and call Psychology's Renaissance. We begin by describing how psychologists' concerns with publication bias shifted from worrying about file-drawered studies to worrying about p-hacked analyses. We then review the methodological changes that psychologists have proposed and, in some cases, embraced. In describing how the renaissance has unfolded, we attempt to describe different points of view fairly but not neutrally, so as to identify the most promising paths forward. In so doing, we champion disclosure and preregistration, express skepticism about most statistical solutions to publication bias, take positions on the analysis and interpretation of replication failures, and contend that meta-analytical thinking increases the prevalence of false positives. Our general thesis is that the scientific practices of experimental psychologists have improved dramatically.

Pay-what-you-want, identity, and self-signaling in markets.

We investigate the role of identity and self-image consideration under "pay-what-you-want" pricing. Results from three field experiments show that often, when granted the opportunity to name the price of a product, fewer consumers choose to buy it than when the price is fixed and low. We show that this opt-out behavior is driven largely by individuals' identity and self-image concerns; individuals feel bad when they pay less than the "appropriate" price, causing them to pass on the opportunity to purchase the product altogether.

Evaluations are inherently comparative, but are compared to what?

Understanding how objective quantities are translated into subjective evaluations has long been of interest to social scientists, medical professionals, and policymakers with an interest in how people process and act on quantitative information. The theory of decision by sampling proposes a comparative procedure: Values seem larger or smaller based on how they rank in a comparison set, the decision sample. But what values are included in this decision sample? We identify and test four mechanistic accounts, each suggesting that how previously encountered attribute values are processed determines whether they linger in the sample to guide the subjective interpretation, and thus the influence, of newly encountered values. Testing our ideas through studies of loss aversion, delay discounting, and vaccine hesitancy, we find strongest support for one account: Quantities need to be subjectively evaluated-rather than merely encountered-for them to enter the decision sample, alter the subjective interpretation of other values, and then guide decision making. Discussion focuses on how the present findings inform understanding of the nature of the decision sample and identify new research directions for the longstanding question of how comparison standards influence decision making. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Prosocial spending encourages happiness: A replication of the only experiment reported in Dunn, Aknin, and Norton (2008).

Spending money on one's self, whether to solve a problem, fulfill a need, or increase enjoyment, often heightens one's sense of happiness. It is therefore both surprising and important that people can be even happier after spending money on someone else. We conducted a close replication of a key experiment from Dunn, Aknin, and Norton (2008) to verify and expand upon their findings. Participants were given money and randomly assigned to either spend it on themselves or on someone else. Although the original study (N = 46) found that the latter group was happier, when we used the same analysis in our replication (N = 133), we did not observe a significant difference. However, we report an additional analysis, focused on a more direct measure of happiness, that does show a significant effect in the direction of the original. Follow-up analyses shed new insights into people's predictions about their own and others' happiness and their actual happiness when spending money for themselves or others.

False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant.

In this article, we accomplish two things. First, we show that despite empirical psychologists' nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.

You're having fun when time flies: the hedonic consequences of subjective time progression.

Seven studies tested the hypothesis that people use subjective time progression in hedonic evaluation. When people believe that time has passed unexpectedly quickly, they rate tasks as more engaging, noises as less irritating, and songs as more enjoyable. We propose that felt time distortion operates as a metacognitive cue that people implicitly attribute to their enjoyment of an experience (i.e., time flew, so the experience must have been fun). Consistent with this attribution account, the effects of felt time distortion on enjoyment ratings were moderated by the need for attribution, the strength of the "time flies" naive theory, and the presence of an alternative attribution. These findings suggest a previously unexplored process through which subjective time progression can influence the hedonic evaluation of experiences.

Specification curve analysis.

Empirical results hinge on analytical decisions that are defensible, arbitrary and motivated. These decisions probably introduce bias (towards the narrative put forward by the authors), and they certainly involve variability not reflected by standard errors. To address this source of noise and bias, we introduce specification curve analysis, which consists of three steps: (1) identifying the set of theoretically justified, statistically valid and non-redundant specifications; (2) displaying the results graphically, allowing readers to identify consequential specifications decisions; and (3) conducting joint inference across all specifications. We illustrate the use of this technique by applying it to three findings from two different papers, one investigating discrimination based on distinctively Black names, the other investigating the effect of assigning female versus male names to hurricanes. Specification curve analysis reveals that one finding is robust, one is weak and one is not robust at all.

Publisher Correction: Specification curve analysis.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Overestimating the valuations and preferences of others.

People often make judgments about their own and others' valuations and preferences. Across 12 studies (N = 17,594), we find a robust bias in these judgments such that people overestimate the valuations and preferences of others. This overestimation arises because, when making predictions about others, people rely on their intuitive core representation of the experience (e.g., is the experience generally positive?) in lieu of a more complex representation that might also include countervailing aspects (e.g., is any of the experience negative?). We first demonstrate that the overestimation bias is pervasive for a wide range of positive (Studies 1-5) and negative experiences (Study 6). Furthermore, the bias is not merely an artifact of how preferences are measured (Study 7). Consistent with judgments based on core representations, the bias significantly reduces when the core representation is uniformly positive (Studies 8A-8B). Such judgments lead to a paradox in how people see others trade off between valuation and utility (Studies 9A-9B). Specifically, relative to themselves, people believe that an identically paying other will get more enjoyment from the same experience, but paradoxically, that an identically enjoying other will pay more for the same experience. Finally, consistent with a core representation explanation, explicitly prompting people to consider the entire distribution of others' preferences significantly reduced or eliminated the bias (Study 10). These findings suggest that social judgments of others' preferences are not only largely biased, but they also ignore how others make trade-offs between evaluative metrics. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

99% impossible: A valid, or falsifiable, internal meta-analysis.

Several researchers have relied on, or advocated for, internal meta-analysis, which involves statistically aggregating multiple studies in a paper to assess their overall evidential value. Advocates of internal meta-analysis argue that it provides an efficient approach to increasing statistical power and solving the file-drawer problem. Here we show that the validity of internal meta-analysis rests on the assumption that no studies or analyses were selectively reported. That is, the technique is only valid if (a) all conducted studies were included (i.e., an empty file drawer), and (b) for each included study, exactly one analysis was attempted (i.e., there was no p-hacking). We show that even very small doses of selective reporting invalidate internal meta-analysis. For example, the kind of minimal p-hacking that increases the false-positive rate of 1 study to just 8% increases the false-positive rate of a 10-study internal meta-analysis to 83%. If selective reporting is approximately zero, but not exactly zero, then internal meta-analysis is invalid. To be valid, (a) an internal meta-analysis would need to contain exclusively studies that were properly preregistered, (b) those preregistrations would have to be followed in all essential aspects, and (c) the decision of whether to include a given study in an internal meta-analysis would have to be made before any of those studies are run. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

P-curve won't do your laundry, but it will distinguish replicable from non-replicable findings in observational research: Comment on Bruns & Ioannidis (2016).

p-curve, the distribution of significant p-values, can be analyzed to assess if the findings have evidential value, whether p-hacking and file-drawering can be ruled out as the sole explanations for them. Bruns and Ioannidis (2016) have proposed p-curve cannot examine evidential value with observational data. Their discussion confuses false-positive findings with confounded ones, failing to distinguish correlation from causation. We demonstrate this important distinction by showing that a confounded but real, hence replicable association, gun ownership and number of sexual partners, leads to a right-skewed p-curve, while a false-positive one, respondent ID number and trust in the supreme court, leads to a flat p-curve. P-curve can distinguish between replicable and non-replicable findings. The observational nature of the data is not consequential.

False-Positive Citations.

We describe why we wrote "False-Positive Psychology," analyze how it has been cited, and explain why the integrity of experimental psychology hinges on the full disclosure of methods, the sharing of materials and data, and, especially, the preregistration of analyses.

Registered Replication Report: Dijksterhuis and van Knippenberg (1998).

Dijksterhuis and van Knippenberg (1998) reported that participants primed with a category associated with intelligence ("professor") subsequently performed 13% better on a trivia test than participants primed with a category associated with a lack of intelligence ("soccer hooligans"). In two unpublished replications of this study designed to verify the appropriate testing procedures, Dijksterhuis, van Knippenberg, and Holland observed a smaller difference between conditions (2%-3%) as well as a gender difference: Men showed the effect (9.3% and 7.6%), but women did not (0.3% and -0.3%). The procedure used in those replications served as the basis for this multilab Registered Replication Report. A total of 40 laboratories collected data for this project, and 23 of these laboratories met all inclusion criteria. Here we report the meta-analytic results for those 23 direct replications (total N = 4,493), which tested whether performance on a 30-item general-knowledge trivia task differed between these two priming conditions (results of supplementary analyses of the data from all 40 labs, N = 6,454, are also reported). We observed no overall difference in trivia performance between participants primed with the "professor" category and those primed with the "hooligan" category (0.14%) and no moderation by gender.

Rejecting a bad option feels like choosing a good one.

Across 4,151 participants, the authors demonstrate a novel framing effect, attribute matching, whereby matching a salient attribute of a decision frame with that of a decision's options facilitates decision-making. This attribute matching is shown to increase decision confidence and, ultimately, consensus estimates by increasing feelings of metacognitive ease. In Study 1, participants choosing the more attractive of two faces or rejecting the less attractive face reported greater confidence in and perceived consensus around their decision. Using positive and negative words, Study 2 showed that the attribute's extremity moderates the size of the effect. Study 3 found decision ease mediates these changes in confidence and consensus estimates. Consistent with a misattribution account, when participants were warned about this external source of ease in Study 4, the effect disappeared. Study 5 extended attribute matching beyond valence to objective judgments. The authors conclude by discussing related psychological constructs as well as downstream consequences. (PsycINFO Database Record

Intuitive confidence: choosing between intuitive and nonintuitive alternatives.

People often choose intuitive rather than equally valid nonintuitive alternatives. The authors suggest that these intuitive biases arise because intuitions often spring to mind with subjective ease, and the subjective ease leads people to hold their intuitions with high confidence. An investigation of predictions against point spreads found that people predicted intuitive options (favorites) more often than equally valid (or even more valid) nonintuitive alternatives (underdogs). Critically, though, this effect was largely determined by people's confidence in their intuitions (intuitive confidence). Across naturalistic, expert, and laboratory samples (Studies 1-3), against personally determined point spreads (Studies 4-11), and even when intuitive confidence was manipulated by altering irrelevant aspects of the decision context (e.g., font; Studies 12 and 13), the authors found that decreasing intuitive confidence reduced or eliminated intuitive biases. These findings indicate that intuitive biases are not inevitable but rather predictably determined by contextual variables that affect intuitive confidence.

Better P-curves: Making P-curve analysis more robust to errors, fraud, and ambitious P-hacking, a Reply to Ulrich and Miller (2015).

When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more low (.01 s) than high (.04 s) p values. What else can cause a right-skewed p-curve? First, we consider the possibility that researchers report only the smallest significant p value (as conjectured by Ulrich & Miller, 2015), concluding that it is a very uncommon problem. We then consider more common problems, including (a) p-curvers selecting the wrong p values, (b) fake data, (c) honest errors, and (d) ambitiously p-hacked (beyond p < .05) results. We evaluate the impact of these common problems on the validity of p-curve analysis, and provide practical solutions that substantially increase its robustness.