Replicability, Robustness, and Reproducibility in Psychological Science.

Replication-an important, uncommon, and misunderstood practice-is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.

College Student Samples Are Not Always Equivalent: The Magnitude of Personality Differences Across Colleges and Universities.

This research examined the magnitude of personality differences across different colleges and universities to understand (a) how much students at different colleges vary from one another and (b) whether there are site-level variables that can explain observed differences. Nearly 8,600 students at 30 colleges and universities completed a Big Five personality trait measure. Site-level information was obtained from the Integrated Postsecondary Education System database (U.S. Department of Education). Multilevel models revealed that each of the Big Five traits showed significant between-site variability, even after accounting for individual-level demographic differences. Some site-level variables (e.g., enrollment size, requiring letters of recommendation) explained between-site differences in traits, but many tests were not statistically significant. Student samples at different universities differed in terms of average levels of Big Five personality domains. This raises the possibility that personality differences may explain differences in research results obtained when studying students at different colleges and universities. Furthermore, results suggest that research that compares findings for only a few sites (e.g., much cross-cultural research) runs the risk of overgeneralizing differences between specific samples to broader group differences. These results underscore the value of multisite collaborative research efforts to enhance psychological research.

A guide for social science journal editors on easing into open science.

Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org ) has collated several resources on embedding open science in journal editing ( www.dpjedi.org/resources ). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx ).

Increasing the transparency of systematic reviews: presenting a generalized registration form.

This paper presents a generalized registration form for systematic reviews that can be used when currently available forms are not adequate. The form is designed to be applicable across disciplines (i.e., psychology, economics, law, physics, or any other field) and across review types (i.e., scoping review, review of qualitative studies, meta-analysis, or any other type of review). That means that the reviewed records may include research reports as well as archive documents, case law, books, poems, etc. Items were selected and formulated to optimize broad applicability instead of specificity, forgoing some benefits afforded by a tighter focus. This PRISMA 2020 compliant form is a fallback for more specialized forms and can be used if no specialized form or registration platform is available. When accessing this form on the Open Science Framework website, users will therefore first be guided to specialized forms when they exist. In addition to this use case, the form can also serve as a starting point for creating registration forms that cater to specific fields or review types.

Conscientiousness in the classroom: a process explanation.

Although the research literature has established that Conscientiousness predicts task performance across a variety of achievement contexts (e.g., ; ), comparatively less is known about the processes that underlie these relations. To the latter end, the current research examines effortful strategies and achievement goals as mediating factors that might explain why people with higher levels of Conscientiousness are predicted to reach higher levels of academic performance. In a longitudinal study, 347 college students completed measures of personality and achievement goals at the beginning of the class, followed by measures of effortful strategies multiple times throughout the semester. Results support the hypothesis that effortful strategies mediate the association between Conscientiousness and academic performance. Moreover, the statistical effects of Conscientiousness were generally independent of achievement goals, but a small portion of the effect was mediated through approach, not avoidance, achievement goals. These results highlight the importance of examining mediating processes between personality and outcomes, and in the case of Conscientiousness, our results suggest that effortful strategies might serve as a useful target for performance-enhancing interventions.

Why Has Personality Psychology Played an Outsized Role in the Credibility Revolution?

Personality is not the most popular subfield of psychology. But, in one way or another, personality psychologists have played an outsized role in the ongoing "credibility revolution" in psychology. Not only have individual personality psychologists taken on visible roles in the movement, but our field's practices and norms have now become models for other fields to emulate (or, for those who share Baumeister's (2016, https://doi.org/10.1016/j.jesp.2016.02.003) skeptical view of the consequences of increasing rigor, a model for what to avoid). In this article we discuss some unique features of our field that may have placed us in an ideal position to be leaders in this movement. We do so from a subjective perspective, describing our impressions and opinions about possible explanations for personality psychology's disproportionate role in the credibility revolution. We also discuss some ways in which personality psychology remains less-than-optimal, and how we can address these flaws.

Evaluating the structure of psychopathic personality traits: A meta-analysis of the Psychopathic Personality Inventory.

Which core traits exemplify psychopathic personality disorder is a hotly debated question within psychology, particularly regarding the role of ostensibly adaptive traits such as stress immunity, social potency, and fearlessness. Much of the research on the interrelationships among putatively adaptive and more maladaptive traits of psychopathy has focused on the factor structure of the Psychopathic Personality Inventory (PPI) and its revision, the Psychopathic Personality Inventory-Revised (PPI-R). These instruments include content scales that have coalesced to form 2 higher order factors in some (but not all) prior studies: Fearless Dominance and Self-Centered Impulsivity. Given the inconsistencies in prior research, we performed a meta-analytic factor analysis of the 8 content scales from these instruments (total N > 18,000) and found general support for these 2 dimensions in community samples. The structure among offender samples (e.g., prisoners, forensic patients) supported a 3-factor model in which the Fearlessness content scale loaded onto Self-Centered Impulsivity (rather than Fearless Dominance). There were also indications that the Stress Immunity content scale had different relations to the other PPI scales in offender versus community samples. We discuss the theoretical and diagnostic implications of these differing factor structures for the field of psychopathy research. (PsycINFO Database Record

The Psychological Science Accelerator: Advancing Psychology through a Distributed Collaborative Network.

Concerns have been growing about the veracity of psychological research. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. The PSA's mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time-limited), efficient (in terms of re-using structures and principles for different projects), decentralized, diverse (in terms of participants and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside of the network). The PSA and other approaches to crowdsourced psychological science will advance our understanding of mental processes and behaviors by enabling rigorous research and systematically examining its generalizability.

The development of achievement goals throughout college: modeling stability and change.

Emerging adulthood, defined for many by the college years, is an active period of personality development; less is known about goal change during these years. We investigated stability and change in the 2 × 2 model of achievement goals over 4 years (N = 527). We evaluated rank-order stability and mean-level change, and tested goal coupling hypotheses--the idea that early changes in goals predict later change in other goals--using multivariate latent difference score models. Achievement goals showed moderate rank-order stability over 4 years. Three of four goals demonstrated small normative declines, excepting performance approach goals. A change in mastery approach goals was associated with levels of the other three goals; these goals jointly acted to slow the decline of mastery goals. Trajectories for the other three goals were largely independent. Results suggest that goals are relatively stable individual differences during the college years.