Shortcut citations in the methods section: Frequency, problems, and strategies for responsible reuse.

Methods sections are often missing essential details. Methodological shortcut citations, in which authors cite previous papers instead of describing the method in detail, may contribute to this problem. This meta-research study used 3 approaches to examine shortcut citation use in neuroscience, biology, and psychiatry. First, we assessed current practices in more than 750 papers. More than 90% of papers used shortcut citations. Other common reasons for using citations in the methods included giving credit or specifying what was used (who or what citation) and providing context or a justification (why citation). Next, we reviewed 15 papers to determine what can happen when readers follow shortcut citations to find methodological details. While shortcut citations can be used effectively, they can also deprive readers of essential methodological details. Problems encountered included difficulty identifying or accessing the cited materials, missing or insufficient descriptions of the cited method, and shortcut citation chains. Third, we examined journal policies. Fewer than one quarter of journals had policies describing how authors should report previously described methods. We propose that methodological shortcut citations should meet 3 criteria; cited resources should provide (1) a detailed description of (2) the method used by the citing authors', and (3) be open access. Resources that do not meet these criteria should be cited to give credit, but not as shortcut citations. We outline actions that authors and journals can take to use shortcut citations responsibly, while fostering a culture of open and reproducible methods reporting.

Blind spots on western blots: Assessment of common problems in western blot figures and methods reporting with recommendations to improve them.

Western blotting is a standard laboratory method used to detect proteins and assess their expression levels. Unfortunately, poor western blot image display practices and a lack of detailed methods reporting can limit a reader's ability to evaluate or reproduce western blot results. While several groups have studied the prevalence of image manipulation or provided recommendations for improving western blotting, data on the prevalence of common publication practices are scarce. We systematically examined 551 articles published in the top 25% of journals in neurosciences (n = 151) and cell biology (n = 400) that contained western blot images, focusing on practices that may omit important information. Our data show that most published western blots are cropped and blot source data are not made available to readers in the supplement. Publishing blots with visible molecular weight markers is rare, and many blots additionally lack molecular weight labels. Western blot methods sections often lack information on the amount of protein loaded on the gel, blocking steps, and antibody labeling protocol. Important antibody identifiers like company or supplier, catalog number, or RRID were omitted frequently for primary antibodies and regularly for secondary antibodies. We present detailed descriptions and visual examples to help scientists, peer reviewers, and editors to publish more informative western blot figures and methods. Additional resources include a toolbox to help scientists produce more reproducible western blot data, teaching slides in English and Spanish, and an antibody reporting template.

Recommendations for empowering early career researchers to improve research culture and practice.

Early career researchers (ECRs) are important stakeholders leading efforts to catalyze systemic change in research culture and practice. Here, we summarize the outputs from a virtual unconventional conference (unconference), which brought together 54 invited experts from 20 countries with extensive experience in ECR initiatives designed to improve the culture and practice of science. Together, we drafted 2 sets of recommendations for (1) ECRs directly involved in initiatives or activities to change research culture and practice; and (2) stakeholders who wish to support ECRs in these efforts. Importantly, these points apply to ECRs working to promote change on a systemic level, not only those improving aspects of their own work. In both sets of recommendations, we underline the importance of incentivizing and providing time and resources for systems-level science improvement activities, including ECRs in organizational decision-making processes, and working to dismantle structural barriers to participation for marginalized groups. We further highlight obstacles that ECRs face when working to promote reform, as well as proposed solutions and examples of current best practices. The abstract and recommendations for stakeholders are available in Dutch, German, Greek (abstract only), Italian, Japanese, Polish, Portuguese, Spanish, and Serbian.

Learning from the past to develop data analysis curricula for the future.

Problems with statistical analyses and the shift toward big data have prompted many researchers to call for improvements in statistics education. A recent study in PLOS Biology assesses changes in the use of data analysis techniques over time to determine which skills young scientists might need.

Training early career researchers to use meta-research to improve science: A participant-guided "learn by doing" approach.

Meta-research, or the science of science, is a powerful technique that scientists can use to improve science, however most scientists are unaware that meta-research exists and courses are rare. This initiative demonstrates the feasibility of a participant-guided "learn by doing" approach, in which a multidisciplinary, global team of early career researchers learned meta-research skills by working together to design, conduct and publish a meta-research study.

Creating clear and informative image-based figures for scientific publications.

Scientists routinely use images to display data. Readers often examine figures first; therefore, it is important that figures are accessible to a broad audience. Many resources discuss fraudulent image manipulation and technical specifications for image acquisition; however, data on the legibility and interpretability of images are scarce. We systematically examined these factors in non-blot images published in the top 15 journals in 3 fields; plant sciences, cell biology, and physiology (n = 580 papers). Common problems included missing scale bars, misplaced or poorly marked insets, images or labels that were not accessible to colorblind readers, and insufficient explanations of colors, labels, annotations, or the species and tissue or object depicted in the image. Papers that met all good practice criteria examined for all image-based figures were uncommon (physiology 16%, cell biology 12%, plant sciences 2%). We present detailed descriptions and visual examples to help scientists avoid common pitfalls when publishing images. Our recommendations address image magnification, scale information, insets, annotation, and color and may encourage discussion about quality standards for bioimage publishing.

Ten simple rules for implementing open and reproducible research practices after attending a training course.

Open, reproducible, and replicable research practices are a fundamental part of science. Training is often organized on a grassroots level, offered by early career researchers, for early career researchers. Buffet style courses that cover many topics can inspire participants to try new things; however, they can also be overwhelming. Participants who want to implement new practices may not know where to start once they return to their research team. We describe ten simple rules to guide participants of relevant training courses in implementing robust research practices in their own projects, once they return to their research group. This includes (1) prioritizing and planning which practices to implement, which involves obtaining support and convincing others involved in the research project of the added value of implementing new practices; (2) managing problems that arise during implementation; and (3) making reproducible research and open science practices an integral part of a future research career. We also outline strategies that course organizers can use to prepare participants for implementation and support them during this process.

Improving the trustworthiness, usefulness, and ethics of biomedical research through an innovative and comprehensive institutional initiative.

The reproducibility crisis triggered worldwide initiatives to improve rigor, reproducibility, and transparency in biomedical research. There are many examples of scientists, journals, and funding agencies adopting responsible research practices. The QUEST (Quality-Ethics-Open Science-Translation) Center offers a unique opportunity to examine the role of institutions. The Berlin Institute of Health founded QUEST to increase the likelihood that research conducted at this large academic medical center would be trustworthy, useful for scientists and society, and ethical. QUEST researchers perform "science of science" studies to understand problems with standard practices and develop targeted solutions. The staff work with institutional leadership and local scientists to incentivize and support responsible practices in research, funding, and hiring. Some activities described in this paper focus on the institution, whereas others may benefit the national and international scientific community. Our experience, approaches, and recommendations will be informative for faculty leadership, administrators, and researchers interested in improving scientific practice.

Replacing bar graphs of continuous data with more informative graphics: are we making progress?

Recent work has raised awareness about the need to replace bar graphs of continuous data with informative graphs showing the data distribution. The impact of these efforts is not known. The present observational meta-research study examined how often scientists in different fields use various graph types, and assessed whether visualization practices have changed between 2010 and 2020. We developed and validated an automated screening tool, designed to identify bar graphs of counts or proportions, bar graphs of continuous data, bar graphs with dot plots, dot plots, box plots, violin plots, histograms, pie charts, and flow charts. Papers from 23 fields (approximately 1000 papers/field per year) were randomly selected from PubMed Central and screened (n=227998). F1 scores for different graphs ranged between 0.83 and 0.95 in the internal validation set. While the tool also performed well in external validation sets, F1 scores were lower for uncommon graphs. Bar graphs are more often used incorrectly to display continuous data than they are used correctly to display counts or proportions. The proportion of papers that use bar graphs of continuous data varies markedly across fields (range in 2020: 4-58%), with high rates in biochemistry and cell biology, complementary and alternative medicine, physiology, genetics, oncology and carcinogenesis, pharmacology, microbiology and immunology. Visualization practices have improved in some fields in recent years. Fewer than 25% of papers use flow charts, which provide information about attrition and the risk of bias. The present study highlights the need for continued interventions to improve visualization and identifies fields that would benefit most.

Mechanisms of vascular dysfunction in the interleukin-10-deficient murine model of preeclampsia indicate nitric oxide dysregulation.

Preeclampsia is a pregnancy-specific hypertensive disorder characterized by proteinuria, and vascular injury in the second half of pregnancy. We hypothesized that endothelium-dependent vascular dysfunction is present in a murine model of preeclampsia based on administration of human preeclamptic sera to interleukin-10-/- mice and studied mechanisms that underlie vascular injury. Pregnant wild type and IL-10-/- mice were injected with either normotensive or severe preeclamptic patient sera (sPE) during gestation. A preeclampsia-like phenotype was confirmed by blood pressure measurements; assessment of albuminuria; measurement of angiogenic factors; demonstration of foot process effacement and endotheliosis in kidney sections; and by accumulation of glycogen in placentas from IL-10-/- mice injected with sPE sera (IL-10-/-sPE). Vasomotor function of isolated aortas was assessed. The IL-10-/-sPE murine model demonstrated significantly augmented aortic contractions to phenylephrine and both impaired endothelium-dependent and, to a lesser extent, endothelium-independent relaxation compared to wild type normotensive mice. Treatment of isolated aortas with indomethacin, a cyclooxygenase inhibitor, improved, but failed to normalize contraction to phenylephrine to that of wild type normotensive mice, suggesting the additional contribution from nitric oxide downregulation and effects of indomethacin-resistant vasoconstricting factors. In contrast, indomethacin normalized relaxation of aortas derived from IL-10-/-sPE mice. Thus, our results identify the role of IL-10 deficiency in dysregulation of the cyclooxygenase pathway and vascular dysfunction in the IL-10-/-sPE murine model of preeclampsia and point towards a possible contribution of nitric oxide dysregulation. These compounds and related mechanisms may serve both as diagnostic markers and therapeutic targets for preventive and treatment strategies in preeclampsia.

The science of science: Clinical Science launches a new translational meta-research collection.

Clinical Science is proud to launch a new translational meta-research collection. Meta-research, or the science of science, applies the scientific method to study science itself. Meta-research is a powerful tool for identifying common problems in scientific papers, assessing their impact, and testing solutions to improve the transparency, rigor, trustworthiness, and usefulness of biomedical research. The collection welcomes science of science studies that link basic science to disease mechanisms, as well as meta-research articles highlighting opportunities to improve transparency, rigor, and reproducibility among the types of papers published in Clinical Science. Submissions might include science of science studies that explore factors linked to successful translation, or meta-research on experimental methods or study designs that are often used in translational research. We hope that this collection will encourage scientists to think critically about current practices and take advantage of opportunities to make their own research more transparent, rigorous, and reproducible.

How accurate are citations of frequently cited papers in biomedical literature?

Citations are an important, but often overlooked, part of every scientific paper. They allow the reader to trace the flow of evidence, serving as a gateway to relevant literature. Most scientists are aware of citations' errors, but few appreciate the prevalence of these problems. The purpose of the present study was to examine how often frequently cited papers in biomedical scientific literature are cited inaccurately. The study included an active participation of the first authors of included papers; to first-hand verify the citations accuracy. Findings from feasibility study, where we reviewed 1540 articles containing 2526 citations of 14 most cited articles in which the authors were affiliated with the Faculty of Medicine University of Belgrade, were further evaluated for external confirmation in an independent verification set of articles. Verification set included 4912 citations identified in 2995 articles that cited 13 most cited articles published by authors affiliated with the Mayo Clinic Division of Nephrology and Hypertension. A citation was defined as being accurate if the cited article supported or was in accordance with the statement by citing authors. At least one inaccurate citation was found in 11 and 15% of articles in the feasibility study and verification set, respectively, suggesting that inaccurate citations are common in biomedical literature. The most common problem was the citation of nonexistent findings (38.4%), followed by an incorrect interpretation of findings (15.4%). One-fifth of inaccurate citations were due to chains of inaccurate citations. Based on these findings, several actions to reduce citation inaccuracies have been proposed.

Reveal, Don't Conceal: Transforming Data Visualization to Improve Transparency.

Reports highlighting the problems with the standard practice of using bar graphs to show continuous data have prompted many journals to adopt new visualization policies. These policies encourage authors to avoid bar graphs and use graphics that show the data distribution; however, they provide little guidance on how to effectively display data. We conducted a systematic review of studies published in top peripheral vascular disease journals to determine what types of figures are used, and to assess the prevalence of suboptimal data visualization practices. Among papers with data figures, 47.7% of papers used bar graphs to present continuous data. This primer provides a detailed overview of strategies for addressing this issue by (1) outlining strategies for selecting the correct type of figure depending on the study design, sample size, and the type of variable; (2) examining techniques for making effective dot plots, box plots, and violin plots; and (3) illustrating how to avoid sending mixed messages by aligning the figure structure with the study design and statistical analysis. We also present solutions to other common problems identified in the systematic review. Resources include a list of free tools and templates that authors can use to create more informative figures and an online simulator that illustrates why summary statistics are meaningful only when there are enough data to summarize. Last, we consider steps that investigators can take to improve figures in the scientific literature.

fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community.

Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers' of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scientific literature. Alternative publication formats have recently been introduced that make it easier to publish research that is difficult to publish in traditional peer reviewed journals. These include micropublications, data repositories, data journals, preprints, publishing platforms, and journals focusing on null or neutral results. While these alternative formats have the potential to reduce publication bias, many scientists are unaware that these formats exist and don't know how to use them. Our open source file drawer data liberation effort (fiddle) tool (RRID:SCR_017327 available at: http://s-quest.bihealth.org/fiddle/) is a match-making Shiny app designed to help biomedical researchers to identify the most appropriate publication format for their data. Users can search for a publication format that meets their needs, compare and contrast different publication formats, and find links to publishing platforms. This tool will assist scientists in getting otherwise inaccessible, hidden data out of the file drawer into the scientific community and literature. We briefly highlight essential details that should be included to ensure reporting quality, which will allow others to use and benefit from research published in these new formats.

From Static to Interactive: Transforming Data Visualization to Improve Transparency.

Data presentation for scientific publications in small sample size studies has not changed substantially in decades. It relies on static figures and tables that may not provide sufficient information for critical evaluation, particularly of the results from small sample size studies. Interactive graphics have the potential to transform scientific publications from static reports of experiments into interactive datasets. We designed an interactive line graph that demonstrates how dynamic alternatives to static graphics for small sample size studies allow for additional exploration of empirical datasets. This simple, free, web-based tool (http://statistika.mfub.bg.ac.rs/interactive-graph/) demonstrates the overall concept and may promote widespread use of interactive graphics.

Correction: From Static to Interactive: Transforming Data Visualization to Improve Transparency.

[This corrects the article DOI: 10.1371/journal.pbio.1002484.].

Reinventing Biostatistics Education for Basic Scientists.

Numerous studies demonstrating that statistical errors are common in basic science publications have led to calls to improve statistical training for basic scientists. In this article, we sought to evaluate statistical requirements for PhD training and to identify opportunities for improving biostatistics education in the basic sciences. We provide recommendations for improving statistics training for basic biomedical scientists, including: 1. Encouraging departments to require statistics training, 2. Tailoring coursework to the students' fields of research, and 3. Developing tools and strategies to promote education and dissemination of statistical knowledge. We also provide a list of statistical considerations that should be addressed in statistics education for basic scientists.

Data visualization, bar naked: A free tool for creating interactive graphics.

Although bar graphs are designed for categorical data, they are routinely used to present continuous data in studies that have small sample sizes. This presentation is problematic, as many data distributions can lead to the same bar graph, and the actual data may suggest different conclusions from the summary statistics. To address this problem, many journals have implemented new policies that require authors to show the data distribution. This paper introduces a free, web-based tool for creating an interactive alternative to the bar graph (http://statistika.mfub.bg.ac.rs/interactive-dotplot/). This tool allows authors with no programming expertise to create customized interactive graphics, including univariate scatterplots, box plots, and violin plots, for comparing values of a continuous variable across different study groups. Individual data points may be overlaid on the graphs. Additional features facilitate visualization of subgroups or clusters of non-independent data. A second tool enables authors to create interactive graphics from data obtained with repeated independent experiments (http://statistika.mfub.bg.ac.rs/interactive-repeated-experiments-dotplot/). These tools are designed to encourage exploration and critical evaluation of the data behind the summary statistics and may be valuable for promoting transparency, reproducibility, and open science in basic biomedical research.

Beyond bar and line graphs: time for a new data presentation paradigm.

Figures in scientific publications are critically important because they often show the data supporting key findings. Our systematic review of research articles published in top physiology journals (n = 703) suggests that, as scientists, we urgently need to change our practices for presenting continuous data in small sample size studies. Papers rarely included scatterplots, box plots, and histograms that allow readers to critically evaluate continuous data. Most papers presented continuous data in bar and line graphs. This is problematic, as many different data distributions can lead to the same bar or line graph. The full data may suggest different conclusions from the summary statistics. We recommend training investigators in data presentation, encouraging a more complete presentation of data, and changing journal editorial policies. Investigators can quickly make univariate scatterplots for small sample size studies using our Excel templates.

Urinary Extracellular Vesicles of Podocyte Origin and Renal Injury in Preeclampsia.

Renal histologic expression of the podocyte-specific protein, nephrin, but not podocin, is reduced in preeclamptic compared with normotensive pregnancies. We hypothesized that renal expression of podocyte-specific proteins would be reflected in urinary extracellular vesicles (EVs) of podocyte origin and accompanied by increased urinary soluble nephrin levels (nephrinuria) in preeclampsia. We further postulated that podocyte injury and attendant formation of EVs are related mechanistically to cellfree fetal hemoglobin (HbF) in maternal plasma. Our study population included preeclamptic (n=49) and normotensive (n=42) pregnant women recruited at delivery. Plasma measurements included HbF concentrations and concentrations of the endogenous chelators haptoglobin, hemopexin, and α1- microglobulin. We assessed concentrations of urinary EVs containing immunologically detectable podocyte-specific proteins by digital flow cytometry and measured nephrinuria by ELISA. The mechanistic role of HbF in podocyte injury was studied in pregnant rabbits. Compared with urine from women with normotensive pregnancies, urine from women with preeclamptic pregnancies contained a high ratio of podocin-positive to nephrin-positive urinary EVs (podocin+ EVs-to-nephrin+ EVs ratio) and increased nephrinuria, both of which correlated with proteinuria. Plasma levels of hemopexin, which were decreased in women with preeclampsia, negatively correlated with proteinuria, urinary podocin+ EVs-to-nephrin+ EVs ratio, and nephrinuria. Administration of HbF to pregnant rabbits increased the number of urinary EVs of podocyte origin. These findings provide evidence that urinary EVs are reflective of preeclampsia-related altered podocyte protein expression. Furthermore, renal injury in preeclampsia associated with an elevated urinary podocin+ EVs-to-nephrin+ EVs ratio and may be mediated by prolonged exposure to cellfree HbF.