The Benefits, Barriers, and Risks of Big-Team Science.

Progress in psychology has been frustrated by challenges concerning replicability, generalizability, strategy selection, inferential reproducibility, and computational reproducibility. Although often discussed separately, these five challenges may share a common cause: insufficient investment of intellectual and nonintellectual resources into the typical psychology study. We suggest that the emerging emphasis on big-team science can help address these challenges by allowing researchers to pool their resources together to increase the amount available for a single study. However, the current incentives, infrastructure, and institutions in academic science have all developed under the assumption that science is conducted by solo principal investigators and their dependent trainees, an assumption that creates barriers to sustainable big-team science. We also anticipate that big-team science carries unique risks, such as the potential for big-team-science organizations to be co-opted by unaccountable leaders, become overly conservative, and make mistakes at a grand scale. Big-team-science organizations must also acquire personnel who are properly compensated and have clear roles. Not doing so raises risks related to mismanagement and a lack of financial sustainability. If researchers can manage its unique barriers and risks, big-team science has the potential to spur great progress in psychology and beyond.

Are Small Effects the Indispensable Foundation for a Cumulative Psychological Science? A Reply to Götz et al. (2022).

In the January 2022 issue of Perspectives, Götz et al. argued that small effects are "the indispensable foundation for a cumulative psychological science." They supported their argument by claiming that (a) psychology, like genetics, consists of complex phenomena explained by additive small effects; (b) psychological-research culture rewards large effects, which means small effects are being ignored; and (c) small effects become meaningful at scale and over time. We rebut these claims with three objections: First, the analogy between genetics and psychology is misleading; second, p values are the main currency for publication in psychology, meaning that any biases in the literature are (currently) caused by pressure to publish statistically significant results and not large effects; and third, claims regarding small effects as important and consequential must be supported by empirical evidence or, at least, a falsifiable line of reasoning. If accepted uncritically, we believe the arguments of Götz et al. could be used as a blanket justification for the importance of any and all "small" effects, thereby undermining best practices in effect-size interpretation. We end with guidance on evaluating effect sizes in relative, not absolute, terms.

The Psychological Science Accelerator's COVID-19 rapid-response dataset.

In response to the COVID-19 pandemic, the Psychological Science Accelerator coordinated three large-scale psychological studies to examine the effects of loss-gain framing, cognitive reappraisals, and autonomy framing manipulations on behavioral intentions and affective measures. The data collected (April to October 2020) included specific measures for each experimental study, a general questionnaire examining health prevention behaviors and COVID-19 experience, geographical and cultural context characterization, and demographic information for each participant. Each participant started the study with the same general questions and then was randomized to complete either one longer experiment or two shorter experiments. Data were provided by 73,223 participants with varying completion rates. Participants completed the survey from 111 geopolitical regions in 44 unique languages/dialects. The anonymized dataset described here is provided in both raw and processed formats to facilitate re-use and further analyses. The dataset offers secondary analytic opportunities to explore coping, framing, and self-determination across a diverse, global sample obtained at the onset of the COVID-19 pandemic, which can be merged with other time-sampled or geographic data.

A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration.

Following theories of emotional embodiment, the facial feedback hypothesis suggests that individuals' subjective experiences of emotion are influenced by their facial expressions. However, evidence for this hypothesis has been mixed. We thus formed a global adversarial collaboration and carried out a preregistered, multicentre study designed to specify and test the conditions that should most reliably produce facial feedback effects. Data from n = 3,878 participants spanning 19 countries indicated that a facial mimicry and voluntary facial action task could both amplify and initiate feelings of happiness. However, evidence of facial feedback effects was less conclusive when facial feedback was manipulated unobtrusively via a pen-in-mouth task.

A Psychological Profile of the Alt-Right.

The 2016 U.S. presidential election coincided with the rise of the "alternative right," or alt-right. Alt-right associates have wielded considerable influence on the current administration and on social discourse, but the movement's loose organizational structure has led to disparate portrayals of its members' psychology and made it difficult to decipher its aims and reach. To systematically explore the alt-right's psychology, we recruited two U.S. samples: An exploratory sample through Amazon's Mechanical Turk (N = 827, alt-right n = 447) and a larger, nationally representative sample through the National Opinion Research Center's Amerispeak panel (N = 1,283, alt-right n = 71-160, depending on the definition). We estimate that 6% of the U.S. population and 10% of Trump voters identify as alt-right. Alt-right adherents reported a psychological profile more reflective of the desire for group-based dominance than economic anxiety. Although both the alt-right and non-alt-right Trump voters differed substantially from non-alt-right, non-Trump voters, the alt-right and Trump voters were quite similar, differing mainly in the alt-right's especially high enthusiasm for Trump, suspicion of mainstream media, trust in alternative media, and desire for collective action on behalf of Whites. We argue for renewed consideration of overt forms of bias in contemporary intergroup research.

Little race or gender bias in an experiment of initial review of NIH R01 grant proposals.

Many granting agencies allow reviewers to know the identity of a proposal's principal investigator (PI), which opens the possibility that reviewers discriminate on the basis of PI race and gender. We investigated this experimentally with 48 NIH R01 grant proposals, representing a broad range of NIH-funded science. We modified PI names to create separate white male, white female, black male and black female versions of each proposal, and 412 scientists each submitted initial reviews for 3 proposals. We find little to no race or gender bias in initial R01 evaluations, and additionally find that any bias that might have been present must be negligible in size. This conclusion was robust to a wide array of statistical model specifications. Pragmatically, important bias may be present in other aspects of the granting process, but our evidence suggests that it is not present in the initial round of R01 reviews.

A meta-analysis of procedures to change implicit measures.

Using a novel technique known as network meta-analysis, we synthesized evidence from 492 studies (87,418 participants) to investigate the effectiveness of procedures in changing implicit measures, which we define as response biases on implicit tasks. We also evaluated these procedures' effects on explicit and behavioral measures. We found that implicit measures can be changed, but effects are often relatively weak (|ds| < .30). Most studies focused on producing short-term changes with brief, single-session manipulations. Procedures that associate sets of concepts, invoke goals or motivations, or tax mental resources changed implicit measures the most, whereas procedures that induced threat, affirmation, or specific moods/emotions changed implicit measures the least. Bias tests suggested that implicit effects could be inflated relative to their true population values. Procedures changed explicit measures less consistently and to a smaller degree than implicit measures and generally produced trivial changes in behavior. Finally, changes in implicit measures did not mediate changes in explicit measures or behavior. Our findings suggest that changes in implicit measures are possible, but those changes do not necessarily translate into changes in explicit measures or behavior. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Psychological Science Accelerator: Advancing Psychology through a Distributed Collaborative Network.

Concerns have been growing about the veracity of psychological research. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. The PSA's mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time-limited), efficient (in terms of re-using structures and principles for different projects), decentralized, diverse (in terms of participants and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside of the network). The PSA and other approaches to crowdsourced psychological science will advance our understanding of mental processes and behaviors by enabling rigorous research and systematically examining its generalizability.

A Gender Bias Habit-Breaking Intervention Led to Increased Hiring of Female Faculty in STEMM Departments.

Addressing the underrepresentation of women in science is a top priority for many institutions, but the majority of efforts to increase representation of women are neither evidence-based nor rigorously assessed. One exception is the gender bias habit-breaking intervention (Carnes et al., 2015), which, in a cluster-randomized trial involving all but two departmental clusters (N = 92) in the 6 STEMM focused schools/colleges at the University of Wisconsin - Madison, led to increases in gender bias awareness and self-efficacy to promote gender equity in academic science departments. Following this initial success, the present study compares, in a preregistered analysis, hiring rates of new female faculty pre- and post-manipulation. Whereas the proportion of women hired by control departments remained stable over time, the proportion of women hired by intervention departments increased by an estimated 18 percentage points (OR = 2.23, dOR = 0.34). Though the preregistered analysis did not achieve conventional levels of statistical significance (p < 0.07), our study has a hard upper limit on statistical power, as the cluster-randomized trial has a maximum sample size of 92 departmental clusters. These patterns have undeniable practical significance for the advancement of women in science, and provide promising evidence that psychological interventions can facilitate gender equity and diversity.

Breaking the prejudice habit: Mechanisms, timecourse, and longevity.

The prejudice habit-breaking intervention (Devine et al., 2012) and its offshoots (e.g., Carnes et al., 2012) have shown promise in effecting long-term change in key outcomes related to intergroup bias, including increases in awareness, concern about discrimination, and, in one study, long-term decreases in implicit bias. This intervention is based on the premise that unintentional bias is like a habit that can be broken with sufficient motivation, awareness, and effort. We conducted replication of the original habit-breaking intervention experiment in a sample more than three times the size of the original (N = 292). We also measured all outcomes every other day for 14 days and measured potential mechanisms for the intervention's effects. Consistent with previous results, the habit-breaking intervention produced a change in concern that endured two weeks post-intervention. These effects were associated with increased sensitivity to the biases of others and an increased tendency to label biases as wrong. Contrasting with the original work, both control and intervention participants decreased in implicit bias, and the effects of the habit-breaking intervention on awareness declined in the second week of the study. In a subsample recruited two years later, intervention participants were more likely than control participants to object on a public online forum to an essay endorsing racial stereotyping. Our results suggest that the habit-breaking intervention produces enduring changes in peoples' knowledge of and beliefs about race-related issues, and we argue that these changes are even more important for promoting long-term behavioral change than are changes in implicit bias.

The motivation to express prejudice.

Contemporary prejudice research focuses primarily on people who are motivated to respond without prejudice and the ways in which unintentional bias can cause these people to act in a manner inconsistent with this motivation. However, some real-world phenomena (e.g., hate speech, hate crimes) and experimental findings (e.g., Plant & Devine, 2001, 2009) suggest that some prejudice is intentional. These phenomena and findings are difficult to explain solely from the motivations to respond without prejudice. We argue that some people are motivated to express prejudice, and we develop the Motivation to Express Prejudice Scale (MP) to measure this motivation. In 7 studies involving more than 6,000 participants, we demonstrate that, across scale versions targeted at Black people and gay men, the MP has good reliability and convergent, discriminant, and predictive validity. In normative climates that prohibit prejudice, the internal and external motivations to express prejudice are functionally nonindependent, but they become more independent when normative climates permit more prejudice toward a target group. People high in the motivation to express prejudice are relatively likely to resist pressure to support programs promoting intergroup contact and to vote for political candidates who support oppressive policies. The motivation to express prejudice predicted these outcomes even when controlling for attitudes and the motivations to respond without prejudice. This work encourages contemporary prejudice researchers to give greater consideration to the intentional aspects of negative intergroup behavior and to broaden the range of phenomena, target groups, and samples that they study.

Long-term reduction in implicit race bias: A prejudice habit-breaking intervention.

We developed a multi-faceted prejudice habit-breaking intervention to produce long-term reductions in implicit race bias. The intervention is based on the premise that implicit bias is like a habit that can be reduced through a combination of awareness of implicit bias, concern about the effects of that bias, and the application of strategies to reduce bias. In a 12-week longitudinal study, people who received the intervention showed dramatic reductions in implicit race bias. People who were concerned about discrimination or who reported using the strategies showed the greatest reductions. The intervention also led to increases in concern about discrimination and personal awareness of bias over the duration of the study. People in the control group showed none of the above effects. Our results raise the hope of reducing persistent and unintentional forms of discrimination that arise from implicit bias.