Toward a more credible assessment of the credibility of science by many-analyst studies.

We discuss a relatively new meta-scientific research design: many-analyst studies that attempt to assess the replicability and credibility of research based on large-scale observational data. In these studies, a large number of analysts try to answer the same research question using the same data. The key idea is the greater the variation in results, the greater the uncertainty in answering the research question and, accordingly, the lower the credibility of any individual research finding. Compared to individual replications, the large crowd of analysts allows for a more systematic investigation of uncertainty and its sources. However, many-analyst studies are also resource-intensive, and there are some doubts about their potential to provide credible assessments. We identify three issues that any many-analyst study must address: 1) identifying the source of variation in the results; 2) providing an incentive structure similar to that of standard research; and 3) conducting a proper meta-analysis of the results. We argue that some recent many-analyst studies have failed to address these issues satisfactorily and have therefore provided an overly pessimistic assessment of the credibility of science. We also provide some concrete guidance on how future many-analyst studies could provide a more constructive assessment.

Housework share between partners: Experimental evidence on gender-specific preferences.

This paper uses a novel vignette-based experimental design to investigate the reasons underlying the gendered division of housework. We are particularly interested in the role of gender-specific preferences: are there differences in the utility that men and women derive from housework, and might these be responsible for the fact that women continue to do more housework than men? It is difficult to address these questions with conventional survey data, because of inherent problems with endogeneity and ex-post rationalization; our experimental design circumvents these problems. We find remarkably little evidence of any systematic gender differences in preferences, and a general inclination towards an equal distribution of housework; this suggests that the reasons for the gendered division of housework do not derive from gender differences in preferences, and must lie elsewhere.

Same-gender citations do not indicate a substantial gender homophily bias.

Can the male citation advantage (more citations for papers written by male than female scientists) be explained by gender homophily bias, i.e., the preference of scientists to cite other scientists of the same gender category? Previous studies report much evidence that this is the case. However, the observed gender homophily bias may be overestimated by overlooking structural aspects such as the gender composition of research topics in which scientists specialize. When controlling for research topics at a high level of granularity, there is only little evidence for a gender homophily bias in citation decisions. Our study points out the importance of controlling structural aspects such as gendered specialization in research topics when investigating gender bias in science.