Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty.

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

The breakdown of antiracist norms: A natural experiment on hate speech after terrorist attacks.

Terrorist attacks often fuel online hate and increase the expression of xenophobic and antiminority messages. Previous research has focused on the impact of terrorist attacks on prejudiced attitudes toward groups linked to the perpetrators as the cause of this increase. We argue that social norms can contain the expression of prejudice after the attacks. We report the results of a combination of a natural and a laboratory-in-the-field (lab-in-the-field) experiment in which we exploit data collected about the occurrence of two consecutive Islamist terrorist attacks in Germany, the Würzburg and Ansbach attacks, in July 2016. The experiment compares the effect of the terrorist attacks in hate speech toward refugees in contexts where a descriptive norm against the use of hate speech is evidently in place to contexts in which the norm is ambiguous because participants observe antiminority comments. Hate toward refugees, but not toward other minority groups, increased as a result of the attacks only in the absence of a strong norm. These results imply that attitudinal changes due to terrorist attacks are more likely to be voiced if norms erode.

Exposition to xenophobic content and support for right-wing populism: The asymmetric role of gender.

This paper studies whether exposure to anti-immigrant sentiment in the online context affects the willingness to support an openly anti-immigration party, and shows how gender moderates the effect. We designed an online experiment in which participants were invited to an online forum to discuss immigration issues. We manipulate the social acceptability of xenophobic views by exposing participants to an increasing proportion of comments with anti-immigrant content. As a proxy for open support for anti-immigrant policies, we ask participants to donate to a well-known German party with a strong anti-immigration discourse: Alternative für Deutschland (Alternative for Germany). We find no evidence that exposure to increasing social acceptability of xenophobic content affected the willingness to donate. In an exploratory analysis, we find that women are particularly reluctant to donate after the anti-immigrant comments raised normative concerns. The results can shed light on the heterogeneous effect of counter-normative discourses on support for anti-immigrant parties.