Shortcut citations in the methods section: Frequency, problems, and strategies for responsible reuse.

Methods sections are often missing essential details. Methodological shortcut citations, in which authors cite previous papers instead of describing the method in detail, may contribute to this problem. This meta-research study used 3 approaches to examine shortcut citation use in neuroscience, biology, and psychiatry. First, we assessed current practices in more than 750 papers. More than 90% of papers used shortcut citations. Other common reasons for using citations in the methods included giving credit or specifying what was used (who or what citation) and providing context or a justification (why citation). Next, we reviewed 15 papers to determine what can happen when readers follow shortcut citations to find methodological details. While shortcut citations can be used effectively, they can also deprive readers of essential methodological details. Problems encountered included difficulty identifying or accessing the cited materials, missing or insufficient descriptions of the cited method, and shortcut citation chains. Third, we examined journal policies. Fewer than one quarter of journals had policies describing how authors should report previously described methods. We propose that methodological shortcut citations should meet 3 criteria; cited resources should provide (1) a detailed description of (2) the method used by the citing authors', and (3) be open access. Resources that do not meet these criteria should be cited to give credit, but not as shortcut citations. We outline actions that authors and journals can take to use shortcut citations responsibly, while fostering a culture of open and reproducible methods reporting.

Ten simple rules for implementing open and reproducible research practices after attending a training course.

Open, reproducible, and replicable research practices are a fundamental part of science. Training is often organized on a grassroots level, offered by early career researchers, for early career researchers. Buffet style courses that cover many topics can inspire participants to try new things; however, they can also be overwhelming. Participants who want to implement new practices may not know where to start once they return to their research team. We describe ten simple rules to guide participants of relevant training courses in implementing robust research practices in their own projects, once they return to their research group. This includes (1) prioritizing and planning which practices to implement, which involves obtaining support and convincing others involved in the research project of the added value of implementing new practices; (2) managing problems that arise during implementation; and (3) making reproducible research and open science practices an integral part of a future research career. We also outline strategies that course organizers can use to prepare participants for implementation and support them during this process.

Blind spots on western blots: Assessment of common problems in western blot figures and methods reporting with recommendations to improve them.

Western blotting is a standard laboratory method used to detect proteins and assess their expression levels. Unfortunately, poor western blot image display practices and a lack of detailed methods reporting can limit a reader's ability to evaluate or reproduce western blot results. While several groups have studied the prevalence of image manipulation or provided recommendations for improving western blotting, data on the prevalence of common publication practices are scarce. We systematically examined 551 articles published in the top 25% of journals in neurosciences (n = 151) and cell biology (n = 400) that contained western blot images, focusing on practices that may omit important information. Our data show that most published western blots are cropped and blot source data are not made available to readers in the supplement. Publishing blots with visible molecular weight markers is rare, and many blots additionally lack molecular weight labels. Western blot methods sections often lack information on the amount of protein loaded on the gel, blocking steps, and antibody labeling protocol. Important antibody identifiers like company or supplier, catalog number, or RRID were omitted frequently for primary antibodies and regularly for secondary antibodies. We present detailed descriptions and visual examples to help scientists, peer reviewers, and editors to publish more informative western blot figures and methods. Additional resources include a toolbox to help scientists produce more reproducible western blot data, teaching slides in English and Spanish, and an antibody reporting template.

Reporting and transparent research practices in sports medicine and orthopaedic clinical trials: a meta-research study.

OBJECTIVES: Transparent reporting of clinical trials is essential to assess the risk of bias and translate research findings into clinical practice. While existing studies have shown that deficiencies are common, detailed empirical and field-specific data are scarce. Therefore, this study aimed to examine current clinical trial reporting and transparent research practices in sports medicine and orthopaedics. SETTING: Exploratory meta-research study on reporting quality and transparent research practices in orthopaedics and sports medicine clinical trials. PARTICIPANTS: The sample included clinical trials published in the top 25% of sports medicine and orthopaedics journals over 9 months. PRIMARY AND SECONDARY OUTCOME MEASURES: Two independent reviewers assessed pre-registration, open data and criteria related to scientific rigour, like randomisation, blinding, and sample size calculations, as well as the study sample, and data analysis. RESULTS: The sample included 163 clinical trials from 27 journals. While the majority of trials mentioned rigour criteria, essential details were often missing. Sixty per cent (95% confidence interval (CI) 53% to 68%) of trials reported sample size calculations, but only 32% (95% CI 25% to 39%) justified the expected effect size. Few trials indicated the blinding status of all main stakeholders (4%; 95% CI 1% to 7%). Only 18% (95% CI 12% to 24%) included information on randomisation type, method and concealed allocation. Most trials reported participants' sex/gender (95%; 95% CI 92% to 98%) and information on inclusion and exclusion criteria (78%; 95% CI 72% to 84%). Only 20% (95% CI 14% to 26%) of trials were pre-registered. No trials deposited data in open repositories. CONCLUSIONS: These results will aid the sports medicine and orthopaedics community in developing tailored interventions to improve reporting. While authors typically mention blinding, randomisation and other factors, essential details are often missing. Greater acceptance of open science practices, like pre-registration and open data, is needed. As these practices have been widely encouraged, we discuss systemic interventions that may improve clinical trial reporting.

Recommendations for empowering early career researchers to improve research culture and practice.

Early career researchers (ECRs) are important stakeholders leading efforts to catalyze systemic change in research culture and practice. Here, we summarize the outputs from a virtual unconventional conference (unconference), which brought together 54 invited experts from 20 countries with extensive experience in ECR initiatives designed to improve the culture and practice of science. Together, we drafted 2 sets of recommendations for (1) ECRs directly involved in initiatives or activities to change research culture and practice; and (2) stakeholders who wish to support ECRs in these efforts. Importantly, these points apply to ECRs working to promote change on a systemic level, not only those improving aspects of their own work. In both sets of recommendations, we underline the importance of incentivizing and providing time and resources for systems-level science improvement activities, including ECRs in organizational decision-making processes, and working to dismantle structural barriers to participation for marginalized groups. We further highlight obstacles that ECRs face when working to promote reform, as well as proposed solutions and examples of current best practices. The abstract and recommendations for stakeholders are available in Dutch, German, Greek (abstract only), Italian, Japanese, Polish, Portuguese, Spanish, and Serbian.

Replacing bar graphs of continuous data with more informative graphics: are we making progress?

Recent work has raised awareness about the need to replace bar graphs of continuous data with informative graphs showing the data distribution. The impact of these efforts is not known. The present observational meta-research study examined how often scientists in different fields use various graph types, and assessed whether visualization practices have changed between 2010 and 2020. We developed and validated an automated screening tool, designed to identify bar graphs of counts or proportions, bar graphs of continuous data, bar graphs with dot plots, dot plots, box plots, violin plots, histograms, pie charts, and flow charts. Papers from 23 fields (approximately 1000 papers/field per year) were randomly selected from PubMed Central and screened (n=227998). F1 scores for different graphs ranged between 0.83 and 0.95 in the internal validation set. While the tool also performed well in external validation sets, F1 scores were lower for uncommon graphs. Bar graphs are more often used incorrectly to display continuous data than they are used correctly to display counts or proportions. The proportion of papers that use bar graphs of continuous data varies markedly across fields (range in 2020: 4-58%), with high rates in biochemistry and cell biology, complementary and alternative medicine, physiology, genetics, oncology and carcinogenesis, pharmacology, microbiology and immunology. Visualization practices have improved in some fields in recent years. Fewer than 25% of papers use flow charts, which provide information about attrition and the risk of bias. The present study highlights the need for continued interventions to improve visualization and identifies fields that would benefit most.

Is the future of peer review automated?

The rising rate of preprints and publications, combined with persistent inadequate reporting practices and problems with study design and execution, have strained the traditional peer review system. Automated screening tools could potentially enhance peer review by helping authors, journal editors, and reviewers to identify beneficial practices and common problems in preprints or submitted manuscripts. Tools can screen many papers quickly, and may be particularly helpful in assessing compliance with journal policies and with straightforward items in reporting guidelines. However, existing tools cannot understand or interpret the paper in the context of the scientific literature. Tools cannot yet determine whether the methods used are suitable to answer the research question, or whether the data support the authors' conclusions. Editors and peer reviewers are essential for assessing journal fit and the overall quality of a paper, including the experimental design, the soundness of the study's conclusions, potential impact and innovation. Automated screening tools cannot replace peer review, but may aid authors, reviewers, and editors in improving scientific papers. Strategies for responsible use of automated tools in peer review may include setting performance criteria for tools, transparently reporting tool performance and use, and training users to interpret reports.

How to connect academics around the globe by organizing an asynchronous virtual unconference.

Many conferences and in-person meetings have transitioned to virtual platforms in response to the COVID-19 pandemic. Here, we share strategies and lessons learned from organizing an international virtual unconventional conference, or 'unconference'. The event focused on how early career researchers can advocate for systemic improvements in scientific publishing and research culture. The virtual unconference had three main components: (1) a virtual networking event, (2) asynchronous virtual brainstorming, and (3) a virtual open space, where participants could join or lead in-depth discussions. The unconference format was participant-driven and encouraged dialogue and collaboration between 54 attendees from 20 countries on six continents. Virtual brainstorming allowed participants to contribute to discussions at times that were convenient for them. Activity was consistently high throughout the 48 hours of virtual brainstorming and continued into the next day. The results of these discussions are collaboratively summarized in a paper entitled Empowering Early Career Researchers to Improve Science, co-authored by the unconference participants . We hope that this method report will help others to organize asynchronous virtual unconferences, while also providing new strategies for participant-driven activities that could be integrated into conventional virtual conferences.

The science of science: Clinical Science launches a new translational meta-research collection.

Clinical Science is proud to launch a new translational meta-research collection. Meta-research, or the science of science, applies the scientific method to study science itself. Meta-research is a powerful tool for identifying common problems in scientific papers, assessing their impact, and testing solutions to improve the transparency, rigor, trustworthiness, and usefulness of biomedical research. The collection welcomes science of science studies that link basic science to disease mechanisms, as well as meta-research articles highlighting opportunities to improve transparency, rigor, and reproducibility among the types of papers published in Clinical Science. Submissions might include science of science studies that explore factors linked to successful translation, or meta-research on experimental methods or study designs that are often used in translational research. We hope that this collection will encourage scientists to think critically about current practices and take advantage of opportunities to make their own research more transparent, rigorous, and reproducible.

Learning from the past to develop data analysis curricula for the future.

Problems with statistical analyses and the shift toward big data have prompted many researchers to call for improvements in statistics education. A recent study in PLOS Biology assesses changes in the use of data analysis techniques over time to determine which skills young scientists might need.

Creating clear and informative image-based figures for scientific publications.

Scientists routinely use images to display data. Readers often examine figures first; therefore, it is important that figures are accessible to a broad audience. Many resources discuss fraudulent image manipulation and technical specifications for image acquisition; however, data on the legibility and interpretability of images are scarce. We systematically examined these factors in non-blot images published in the top 15 journals in 3 fields; plant sciences, cell biology, and physiology (n = 580 papers). Common problems included missing scale bars, misplaced or poorly marked insets, images or labels that were not accessible to colorblind readers, and insufficient explanations of colors, labels, annotations, or the species and tissue or object depicted in the image. Papers that met all good practice criteria examined for all image-based figures were uncommon (physiology 16%, cell biology 12%, plant sciences 2%). We present detailed descriptions and visual examples to help scientists avoid common pitfalls when publishing images. Our recommendations address image magnification, scale information, insets, annotation, and color and may encourage discussion about quality standards for bioimage publishing.

Training early career researchers to use meta-research to improve science: A participant-guided "learn by doing" approach.

Meta-research, or the science of science, is a powerful technique that scientists can use to improve science, however most scientists are unaware that meta-research exists and courses are rare. This initiative demonstrates the feasibility of a participant-guided "learn by doing" approach, in which a multidisciplinary, global team of early career researchers learned meta-research skills by working together to design, conduct and publish a meta-research study.

How accurate are citations of frequently cited papers in biomedical literature?

Citations are an important, but often overlooked, part of every scientific paper. They allow the reader to trace the flow of evidence, serving as a gateway to relevant literature. Most scientists are aware of citations' errors, but few appreciate the prevalence of these problems. The purpose of the present study was to examine how often frequently cited papers in biomedical scientific literature are cited inaccurately. The study included an active participation of the first authors of included papers; to first-hand verify the citations accuracy. Findings from feasibility study, where we reviewed 1540 articles containing 2526 citations of 14 most cited articles in which the authors were affiliated with the Faculty of Medicine University of Belgrade, were further evaluated for external confirmation in an independent verification set of articles. Verification set included 4912 citations identified in 2995 articles that cited 13 most cited articles published by authors affiliated with the Mayo Clinic Division of Nephrology and Hypertension. A citation was defined as being accurate if the cited article supported or was in accordance with the statement by citing authors. At least one inaccurate citation was found in 11 and 15% of articles in the feasibility study and verification set, respectively, suggesting that inaccurate citations are common in biomedical literature. The most common problem was the citation of nonexistent findings (38.4%), followed by an incorrect interpretation of findings (15.4%). One-fifth of inaccurate citations were due to chains of inaccurate citations. Based on these findings, several actions to reduce citation inaccuracies have been proposed.

Mechanisms of vascular dysfunction in the interleukin-10-deficient murine model of preeclampsia indicate nitric oxide dysregulation.

Preeclampsia is a pregnancy-specific hypertensive disorder characterized by proteinuria, and vascular injury in the second half of pregnancy. We hypothesized that endothelium-dependent vascular dysfunction is present in a murine model of preeclampsia based on administration of human preeclamptic sera to interleukin-10-/- mice and studied mechanisms that underlie vascular injury. Pregnant wild type and IL-10-/- mice were injected with either normotensive or severe preeclamptic patient sera (sPE) during gestation. A preeclampsia-like phenotype was confirmed by blood pressure measurements; assessment of albuminuria; measurement of angiogenic factors; demonstration of foot process effacement and endotheliosis in kidney sections; and by accumulation of glycogen in placentas from IL-10-/- mice injected with sPE sera (IL-10-/-sPE). Vasomotor function of isolated aortas was assessed. The IL-10-/-sPE murine model demonstrated significantly augmented aortic contractions to phenylephrine and both impaired endothelium-dependent and, to a lesser extent, endothelium-independent relaxation compared to wild type normotensive mice. Treatment of isolated aortas with indomethacin, a cyclooxygenase inhibitor, improved, but failed to normalize contraction to phenylephrine to that of wild type normotensive mice, suggesting the additional contribution from nitric oxide downregulation and effects of indomethacin-resistant vasoconstricting factors. In contrast, indomethacin normalized relaxation of aortas derived from IL-10-/-sPE mice. Thus, our results identify the role of IL-10 deficiency in dysregulation of the cyclooxygenase pathway and vascular dysfunction in the IL-10-/-sPE murine model of preeclampsia and point towards a possible contribution of nitric oxide dysregulation. These compounds and related mechanisms may serve both as diagnostic markers and therapeutic targets for preventive and treatment strategies in preeclampsia.

fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community.

Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers' of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scientific literature. Alternative publication formats have recently been introduced that make it easier to publish research that is difficult to publish in traditional peer reviewed journals. These include micropublications, data repositories, data journals, preprints, publishing platforms, and journals focusing on null or neutral results. While these alternative formats have the potential to reduce publication bias, many scientists are unaware that these formats exist and don't know how to use them. Our open source file drawer data liberation effort (fiddle) tool (RRID:SCR_017327 available at: http://s-quest.bihealth.org/fiddle/) is a match-making Shiny app designed to help biomedical researchers to identify the most appropriate publication format for their data. Users can search for a publication format that meets their needs, compare and contrast different publication formats, and find links to publishing platforms. This tool will assist scientists in getting otherwise inaccessible, hidden data out of the file drawer into the scientific community and literature. We briefly highlight essential details that should be included to ensure reporting quality, which will allow others to use and benefit from research published in these new formats.

Ways to increase equity, diversity and inclusion.

The eLife Early-Career Advisory Group (ECAG), an international group of early-career researchers committed to improving research culture, calls for radical changes at eLife and other journals to address racism in the scientific community and to make science more diverse and inclusive.

Incidence and Long-Term Outcomes of Hypertensive Disorders of Pregnancy.

BACKGROUND: Hypertensive disorders of pregnancy (HDP) are associated with increased risks for cardiovascular disease later in life. The HDP incidence is commonly assessed using diagnostic codes, which are not reliable; and typically are expressed per-pregnancy, which may underestimate the number of women with an HDP history after their reproductive years. OBJECTIVES: This study sought to determine the incidence of HDP expressed as both per-pregnancy and per-woman, and to establish their associations with future chronic conditions and multimorbidity, a measure of accelerated aging, in a population-based cohort study. METHODS: Using the Rochester Epidemiology Project medical record-linkage system, the authors identified residents of Olmsted County, Minnesota, who delivered between 1976 and 1982. The authors classified pregnancies into normotensive, gestational hypertension, pre-eclampsia, eclampsia, pre-eclampsia superimposed on chronic hypertension, and chronic hypertension using a validated electronic algorithm, and calculated the incidence of HDP both per-pregnancy and per-woman. The risk of chronic conditions between women with versus those without a history of HDP (age and parity 1:2 matched) was quantified using the hazard ratio and corresponding 95% confidence interval estimated from a Cox model. RESULTS: Among 9,862 pregnancies, we identified 719 (7.3%) with HDP and 324 (3.3%) with pre-eclampsia. The incidence of HDP and pre-eclampsia doubled when assessed on a per-woman basis: 15.3% (281 of 1,839) and 7.5% (138 of 1,839), respectively. Women with a history of HDP were at increased risk for subsequent diagnoses of stroke (hazard ratio [HR]: 2.27; 95% confidence interval [CI]: 1.37 to 3.76), coronary artery disease (HR: 1.89; 95% CI: 1.26 to 2.82), cardiac arrhythmias (HR: 1.62; 95% CI: 1.28 to 2.05), chronic kidney disease (HR: 2.41; 95% CI: 1.54 to 3.78), and multimorbidity (HR: 1.25; 95% CI: 1.15 to 1.35). CONCLUSIONS: The HDP population-based incidence expressed per-pregnancy underestimates the number of women affected by this condition during their reproductive years. A history of HDP confers significant increase in risks for future chronic conditions and multimorbidity.