Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science.

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative's current status and its possible future contributions after the project is concluded in 2021.

Are most published research findings false in a continuous universe?

Diagnostic screening models for the interpretation of null hypothesis significance test (NHST) results have been influential in highlighting the effect of selective publication on the reproducibility of the published literature, leading to John Ioannidis' much-cited claim that most published research findings are false. These models, however, are typically based on the assumption that hypotheses are dichotomously true or false, without considering that effect sizes for different hypotheses are not the same. To address this limitation, we develop a simulation model that overcomes this by modeling effect sizes explicitly using different continuous distributions, while retaining other aspects of previous models such as publication bias and the pursuit of statistical significance. Our results show that the combination of selective publication, bias, low statistical power and unlikely hypotheses consistently leads to high proportions of false positives, irrespective of the effect size distribution assumed. Using continuous effect sizes also allows us to evaluate the degree of effect size overestimation and prevalence of estimates with the wrong sign in the literature, showing that the same factors that drive false-positive results also lead to errors in estimating effect size direction and magnitude. Nevertheless, the relative influence of these factors on different metrics varies depending on the distribution assumed for effect sizes. The model is made available as an R ShinyApp interface, allowing one to explore features of the literature in various scenarios.

Progenitor death drives retinal dysplasia and neuronal degeneration in a mouse model of ATRIP-Seckel syndrome.

Seckel syndrome is a type of microcephalic primordial dwarfism (MPD) that is characterized by growth retardation and neurodevelopmental defects, including reports of retinopathy. Mutations in key mediators of the replication stress response, the mutually dependent partners ATR and ATRIP, are among the known causes of Seckel syndrome. However, it remains unclear how their deficiency disrupts the development and function of the central nervous system (CNS). Here, we investigated the cellular and molecular consequences of ATRIP deficiency in different cell populations of the developing murine neural retina. We discovered that conditional inactivation of Atrip in photoreceptor neurons did not affect their survival or function. In contrast, Atrip deficiency in retinal progenitor cells (RPCs) led to severe lamination defects followed by secondary photoreceptor degeneration and loss of vision. Furthermore, we showed that RPCs lacking functional ATRIP exhibited higher levels of replicative stress and accumulated endogenous DNA damage that was accompanied by stabilization of TRP53. Notably, inactivation of Trp53 prevented apoptosis of Atrip-deficient progenitor cells and was sufficient to rescue retinal dysplasia, neurodegeneration and loss of vision. Together, these results reveal an essential role of ATRIP-mediated replication stress response in CNS development and suggest that the TRP53-mediated apoptosis of progenitor cells might contribute to retinal malformations in Seckel syndrome and other MPD disorders.This article has an associated First Person interview with the first author of the paper.

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature.

BACKGROUND: Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader's ability to independently interpret data and reproduce findings. METHODS: In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. RESULTS: Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. CONCLUSIONS: Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative's current status and its possible future contributions after the project is concluded in 2021.