Theoretical false positive psychology.

A fundamental goal of scientific research is to generate true positives (i.e., authentic discoveries). Statistically, a true positive is a significant finding for which the underlying effect size (δ) is greater than 0, whereas a false positive is a significant finding for which δ equals 0. However, the null hypothesis of no difference (δ = 0) may never be strictly true because innumerable nuisance factors can introduce small effects for theoretically uninteresting reasons. If δ never equals zero, then with sufficient power, every experiment would yield a significant result. Yet running studies with higher power by increasing sample size (N) is one of the most widely agreed upon reforms to increase replicability. Moreover, and perhaps not surprisingly, the idea that psychology should attach greater value to small effect sizes is gaining currency. Increasing N without limit makes sense for purely measurement-focused research, where the magnitude of δ itself is of interest, but it makes less sense for theory-focused research, where the truth status of the theory under investigation is of interest. Increasing power to enhance replicability will increase true positives at the level of the effect size (statistical true positives) while increasing false positives at the level of theory (theoretical false positives). With too much power, the cumulative foundation of psychological science would consist largely of nuisance effects masquerading as theoretically important discoveries. Positive predictive value at the level of theory is maximized by using an optimal N, one that is neither too small nor too large.

Science is not a signal detection problem.

The perceived replication crisis and the reforms designed to address it are grounded in the notion that science is a binary signal detection problem. However, contrary to null hypothesis significance testing (NHST) logic, the magnitude of the underlying effect size for a given experiment is best conceptualized as a random draw from a continuous distribution, not as a random draw from a dichotomous distribution (null vs. alternative). Moreover, because continuously distributed effects selected using a P < 0.05 filter must be inflated, the fact that they are smaller when replicated (reflecting regression to the mean) is no reason to sound the alarm. Considered from this perspective, recent replication efforts suggest that most published P < 0.05 scientific findings are "true" (i.e., in the correct direction), with observed effect sizes that are inflated to varying degrees. We propose that original science is a screening process, one that adopts NHST logic as a useful fiction for selecting true effects that are potentially large enough to be of interest to other scientists. Unlike original science, replication science seeks to precisely measure the underlying effect size associated with an experimental protocol via large-N direct replication, without regard for statistical significance. Registered reports are well suited to (often resource-intensive) direct replications, which should focus on influential findings and be published regardless of outcome. Conceptual replications play an important but separate role in validating theories. However, because they are part of NHST-based original science, conceptual replications cannot serve as the field's self-correction mechanism. Only direct replications can do that.

The Peer Reviewers' Openness Initiative: incentivizing open research practices through peer review.

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism-reviewers withholding comprehensive review-to achieve the goal of creating the expectation of open practices as a matter of scientific principle.

Do subtle reminders of money change people's political views?

A number of researchers have reported studies showing that subtle reminders of money can alter behaviors and beliefs that are seemingly unrelated to money. In 1 set of studies published in this journal, Caruso, Vohs, Baxter, and Waytz (2013) found that incidental exposures to money led subjects to indicate greater support for inequality, socioeconomic differences, group-based discrimination, and free market economies. We conducted high-powered replication attempts of these 4 money priming effects and found no evidence of priming (weighted Cohen's d = 0.03). We later learned that Caruso et al. also found several null effects in their line of research that were not reported in the original article. In addition, the money priming effect observed in the first study of Caruso et al. was included in the Many Labs Replication Project (Klein et al., 2014), and only 1 of the 36 labs was able to find the effect.

Elastic analysis procedures: an incurable (but preventable) problem in the fertility effect literature. Comment on Gildersleeve, Haselton, and Fales (2014).

Gildersleeve, Haselton, and Fales (2014) presented a meta-analysis of the effects of fertility on mate preferences in women. Research in this area has categorized fertility using a great variety of methods, chiefly based on self-reported cycle length and time since last menses. We argue that this literature is particularly prone to hidden experimenter degrees of freedom. Studies vary greatly in the duration and timing of windows used to define fertile versus nonfertile phases, criteria for excluding subjects, and the choice of what moderator variables to include, as well as other variables. These issues raise the concern that many or perhaps all results may have been created by exploitation of unacknowledged degrees of freedom ("p-hacking"). Gildersleeve et al. sought to dismiss such concerns, but we contend that their arguments rest upon statistical and logical errors. The possibility that positive results in this literature may have been created, or at least greatly amplified, by p-hacking receives additional support from the fact that recent attempts at exact replication of fertility results have mostly failed. Our concerns are also supported by findings of another recent review of the literature (Wood, Kressel, Joshi, & Louie, 2014). We conclude on a positive note, arguing that if fertility-effect researchers take advantage of the rapidly emerging opportunities for study preregistration, the validity of this literature can be rapidly clarified.

Jealousy in dogs.

It is commonly assumed that jealousy is unique to humans, partially because of the complex cognitions often involved in this emotion. However, from a functional perspective, one might expect that an emotion that evolved to protect social bonds from interlopers might exist in other social species, particularly one as cognitively sophisticated as the dog. The current experiment adapted a paradigm from human infant studies to examine jealousy in domestic dogs. We found that dogs exhibited significantly more jealous behaviors (e.g., snapping, getting between the owner and object, pushing/touching the object/owner) when their owners displayed affectionate behaviors towards what appeared to be another dog as compared to nonsocial objects. These results lend support to the hypothesis that jealousy has some "primordial" form that exists in human infants and in at least one other social species besides humans.

Two failures to replicate high-performance-goal priming effects.

Bargh et al. (2001) reported two experiments in which people were exposed to words related to achievement (e.g., strive, attain) or to neutral words, and then performed a demanding cognitive task. Performance on the task was enhanced after exposure to the achievement related words. Bargh and colleagues concluded that better performance was due to the achievement words having activated a "high-performance goal". Because the paper has been cited well over 1100 times, an attempt to replicate its findings would seem warranted. Two direct replication attempts were performed. Results from the first experiment (n = 98) found no effect of priming, and the means were in the opposite direction from those reported by Bargh and colleagues. The second experiment followed up on the observation by Bargh et al. (2001) that high-performance-goal priming was enhanced by a 5-minute delay between priming and test. Adding such a delay, we still found no evidence for high-performance-goal priming (n = 66). These failures to replicate, along with other recent results, suggest that the literature on goal priming requires some skeptical scrutiny.

Envy, politics, and age.

In the last 5 years, the phrase "politics of envy" has appeared more than 621 times in English-language newspapers, generally in opinion essays contending that political liberalism reflects and exploits feelings of envy. Oddly, this assertion has not been tested empirically. We did so with a large adult sample (n = 357). Participants completed a Dispositional Envy Scale and questions about political ideology, socioeconomic status, and age. Envy and age were moderately correlated; younger people reported greater envy. Political ideology and envy were weakly correlated; however, this relationship was not significant when controlling for age.

Major memory for microblogs.

Online social networking is vastly popular and permits its members to post their thoughts as microblogs, an opportunity that people exploit, on Facebook alone, over 30 million times an hour. Such trivial ephemera, one might think, should vanish quickly from memory; conversely, they may comprise the sort of information that our memories are tuned to recognize, if that which we readily generate, we also readily store. In the first two experiments, participants' memory for Facebook posts was found to be strikingly stronger than their memory for human faces or sentences from books-a magnitude comparable to the difference in memory strength between amnesics and healthy controls. The second experiment suggested that this difference is not due to Facebook posts spontaneously generating social elaboration, because memory for posts is enhanced as much by adding social elaboration as is memory for book sentences. Our final experiment, using headlines, sentences, and reader comments from articles, suggested that the remarkable memory for microblogs is also not due to their completeness or simply their topic, but may be a more general phenomenon of their being the largely spontaneous and natural emanations of the human mind.

Priming of social distance? Failure to replicate effects on social and food judgments.

Williams and Bargh (2008) reported an experiment in which participants were simply asked to plot a single pair of points on a piece of graph paper, with the coordinates provided by the experimenter specifying a pair of points that lay at one of three different distances (close, intermediate, or far, relative to the range available on the graph paper). The participants who had graphed a more distant pair reported themselves as being significantly less close to members of their own family than did those who had plotted a more closely-situated pair. In another experiment, people's estimates of the caloric content of different foods were reportedly altered by the same type of spatial distance priming. Direct replications of both results were attempted, with precautions to ensure that the experimenter did not know what condition the participant was assigned to. The results showed no hint of the priming effects reported by Williams and Bargh (2008).

The immediate and delayed cardiovascular benefits of forgiving.

BACKGROUND: The putative health benefits of forgiveness may include long-term buffering against cardiovascular reactivity associated with rumination. Although studies show short-term benefits of adopting a forgiving perspective, it is uncertain whether this perspective protects against repeated future rumination on offenses, which may be necessary for long-term health benefits. Also unclear is whether forgiveness offers unique benefits beyond simple distraction. METHODS: Cardiovascular parameters (systolic blood pressure [SBP], diastolic blood pressure [DBP], and heart rate) were measured while 202 participants thought about a previous offense from an angry or forgiving perspective or were distracted. All participants were then distracted for 5 minutes, after which they freely ruminated on the offense. RESULTS: Angry rumination initially yielded the greatest increase in blood pressure from baseline (mean [M] [standard deviation {SD}]: SBP = 9.24 [11.16]; M [SD]: DBP = 4.69 [7.48]) compared with forgiveness (M [SD]: SBP = 3.30 [6.48]; M [SD]: DBP = 1.51 [4.94]) and distraction (M [SD]: SBP = 4.81 [6.28]; M [SD]: DBP = 1.75 [3.80]), which did not differ from each other (p > .30). During free rumination, however, those who had previously focused on forgiveness showed less reactivity (M [SD]: SBP = 7.33 [9.61]; M [SD]: DBP = 4.73 [7.33]) than those who had been distracted (M [SD]: SBP = 10.50 [7.77]; M [SD]: DBP = 7.71 [6.83]) and those who previously focused on angry rumination (M [SD]: SBP = 12.04 [11.74]; M [SD]: DBP = 8.64 [12.63]). There were no differences for heart rate. CONCLUSIONS: Forgiveness seems to lower reactivity both during the initial cognitive process and, more importantly, during mental recreations of an offense soon thereafter, potentially offering sustained protection, whereas effects of distraction appear transient.

Is the Replicability Crisis Overblown? Three Arguments Examined.

We discuss three arguments voiced by scientists who view the current outpouring of concern about replicability as overblown. The first idea is that the adoption of a low alpha level (e.g., 5%) puts reasonable bounds on the rate at which errors can enter the published literature, making false-positive effects rare enough to be considered a minor issue. This, we point out, rests on statistical misunderstanding: The alpha level imposes no limit on the rate at which errors may arise in the literature (Ioannidis, 2005b). Second, some argue that whereas direct replication attempts are uncommon, conceptual replication attempts are common-providing an even better test of the validity of a phenomenon. We contend that performing conceptual rather than direct replication attempts interacts insidiously with publication bias, opening the door to literatures that appear to confirm the reality of phenomena that in fact do not exist. Finally, we discuss the argument that errors will eventually be pruned out of the literature if the field would just show a bit of patience. We contend that there are no plausible concrete scenarios to back up such forecasts and that what is needed is not patience, but rather systematic reforms in scientific practice.

Menstrual Cycle and Facial Preferences Reconsidered.

Two previous articles reported that women prefer less feminized male faces during the fertile phase of their menstrual cycle, supposedly reflecting an evolved mating strategy whereby women choose mates of maximum genetic quality when conception is likely. The current article contends this theory rests on several questionable assumptions about human ancestral mating systems. A new empirical test also was conducted: 853 adults, primarily from North America, evaluated facial attractiveness of photos. The study included more complete evaluation of ovulatory status and a greater number (n = 258) of target women than past research. The results did not suggest any greater preference for masculine faces when fertilization was likely. The article concludes with general comments about evolutionary theorizing and interpersonal relationships.

Jealousy: novel methods and neural correlates.

Because of the difficulties surrounding the evocation of jealousy, past research has relied on reactions to hypothetical scenarios and recall of past experiences of jealousy. Both methodologies have limitations, however. The present research was designed to develop a method of evoking jealousy in the laboratory that would be well controlled, ethically permissible, and psychologically meaningful. Study 1 demonstrated that jealousy could be evoked in a modified version of K. D. Williams' (2007) Cyberball ostracism paradigm in which the rejecting person was computer-generated. Study 2, the first to examine neural activity during the active experience of jealousy, tested whether experienced jealousy was associated with greater relative left or right frontal cortical activation. The findings revealed that the experience of jealousy was correlated with greater relative left frontal cortical activation toward the "sexually" desired partner. This pattern of activation suggests that jealousy is associated with approach motivation. Taken together, the present studies developed a laboratory paradigm for the study of jealousy that should help foster research on one of the most social of emotions.

Does the central bottleneck encompass voluntary selection of hedonically based choices?

A large literature on multitasking bottlenecks suggests that people cannot generally make decisions or select responses in two different tasks at the same time. However, these tasks have all involved retrieving preinstructed responses, rather than spontaneously choosing actions based on anticipated hedonic consequences. To assess whether the same bottlenecks encompasses voluntary choices, a gambling decision was utilized as the second of two tasks in a psychological refractory period (PRP) design. Three decision-related factors were identified that affected latency of responding in the gambling task. All proved to be additive with stimulus-onset asynchrony (SOA) in dual-task blocks. The results indicate that making a choice to try to optimize outcomes is subject to the same processing bottleneck as the retrieval of preinstructed responses that has been the mainstay of attention and performance research.

Bodily embarrassment and judgment concern as separable factors in the measurement of medical embarrassment: psychometric development and links to treatment-seeking outcomes.

OBJECTIVES: Understanding why people do not always engage in medical examinations that might benefit them is a public health issue which is receiving increased attention. One area of promise involves the study of medical embarrassment, although current studies are weakened in that they measure medical embarrassment in a theoretically naïve and unidimensional manner and have assumed that embarrassment is exclusively a barrier to the timely seeking of treatment. DESIGN: Convenience sampling was used to recruit 116 male and 134 female students (mean age = 19.94 years, 47.2% Caucasian, 20.4% African-American, 32.4% Asian) from two large universities in different parts of the United States. METHODS: Participants completed a comprehensive measure of medical embarrassment, reported on previous treatment avoidance because of embarrassment, and recorded the frequency of psychological, general and sex-related visits across the previous 5 years. RESULTS: As expected, medical embarrassment was not unidimensional and appeared to have two distinct factors--bodily embarrassment and judgment concern. Bodily embarrassment generally predicted less frequent medical contact although not equally so across domains and it interacted with judgment concern in several cases, providing preliminary evidence that there are situations in which aspects of medical embarrassment may actually facilitate greater medical contact. CONCLUSIONS: The data highlight the importance of considering the role of emotions other than fear in health behaviour and the means by which they may facilitate or deter the timely seeking of diagnosis and treatment.

Enhanced memory for negatively emotionally charged pictures without selective rumination.

Emotionally charged materials have been found to elicit higher levels of recall in many studies. However, the use of slow presentations and/or uncontrolled retention intervals may have allowed subjects to rehearse emotional materials preferentially. The authors presented a series of 5 pictures (1 emotionally charged) at a rate of 4 pictures per second, precluding selective rehearsal. In Experiment 1, subjects recalled the pictures immediately or after performing an arithmetic task for 20 s. In Experiment 2, the pictures were described as to-be-ignored distractors, and the memory test was unexpected. Stimulus emotionality greatly enhanced recall in all conditions. The speed of the presentations and the fact that enhancement did not spread to temporally adjacent items argues against some widely discussed mechanisms for emotional enhancement.

Attention and the processing of emotional words and names: not so special after all.

Previous research has suggested that a person's own name or emotionally charged stimuli automatically "grab" attention, potentially challenging limited-capacity theories of perceptual processing. In this study, subjects were shown two digits surrounding a word and asked to make a speeded judgment about whether the parity of the two digits matched. When the subject's own name was presented on a few scattered trials, responses were markedly slowed (replicating a previous study). However, in a subsequent block of trials in which half the words were the subject's name, the slowing did not occur. The same slowing occurred (but even more fleetingly) when an emotionally charged word was presented between the digits. When the name was embedded among multiple distractor words, it ceased to slow reaction times. The results suggest that perceptual analysis of high-priority stimuli is subject to the usual capacity limitations of other stimuli, but when enough capacity is available for a high-priority stimulus to be perceived, a transient surprise reaction may interrupt ongoing processing.