The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews.

BACKGROUND: Researchers performing high-quality systematic reviews search across multiple databases to identify relevant evidence. However, the same publication is often retrieved from several databases. Identifying and removing such duplicates ("deduplication") can be extremely time-consuming, but failure to remove these citations can lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews. METHODS: We evaluated ASySD's performance on 5 unseen biomedical systematic search datasets of various sizes (1845-79,880 citations). We compared the performance of ASySD with EndNote's automated deduplication option and with the Systematic Review Assistant Deduplication Module (SRA-DM). RESULTS: ASySD identified more duplicates than either SRA-DM or EndNote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of > 0.99. The tool took less than 1 h to identify and remove duplicates within each dataset. CONCLUSIONS: For duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.

Development and uptake of an online systematic review platform: the early years of the CAMARADES Systematic Review Facility (SyRF).

Preclinical research is a vital step in the drug discovery pipeline and more generally in helping to better understand human disease aetiology and its management. Systematic reviews (SRs) can be powerful in summarising and appraising this evidence concerning a specific research question, to highlight areas of improvements, areas for further research and areas where evidence may be sufficient to take forward to other research domains, for instance clinical trial. Guidance and tools for preclinical research synthesis remain limited despite their clear utility. We aimed to create an online end-to-end platform primarily for conducting SRs of preclinical studies, that was flexible enough to support a wide variety of experimental designs, was adaptable to different research questions, would allow users to adopt emerging automated tools and support them during their review process using best practice. In this article, we introduce the Systematic Review Facility (https://syrf.org.uk), which was launched in 2016 and designed to support primarily preclinical SRs from small independent projects to large, crowdsourced projects. We discuss the architecture of the app and its features, including the opportunity to collaborate easily, to efficiently manage projects, to screen and annotate studies for important features (metadata), to extract outcome data into a secure database, and tailor these steps to each project. We introduce how we are working to leverage the use of automation tools and allow the integration of these services to accelerate and automate steps in the systematic review workflow.

Quality and validity of large animal experiments in stroke: A systematic review.

An important factor for successful translational stroke research is study quality. Low-quality studies are at risk of biased results and effect overestimation, as has been intensely discussed for small animal stroke research. However, little is known about the methodological rigor and quality in large animal stroke models, which are becoming more frequently used in the field. Based on research in two databases, this systematic review surveys and analyses the methodological quality in large animal stroke research. Quality analysis was based on the Stroke Therapy Academic Industry Roundtable and the Animals in Research: Reporting In Vivo Experiments guidelines. Our analysis revealed that large animal models are utilized with similar shortcomings as small animal models. Moreover, translational benefits of large animal models may be limited due to lacking implementation of important quality criteria such as randomization, allocation concealment, and blinded assessment of outcome. On the other hand, an increase of study quality over time and a positive correlation between study quality and journal impact factor were identified. Based on the obtained findings, we derive recommendations for optimal study planning, conducting, and data analysis/reporting when using large animal stroke models to fully benefit from the translational advantages offered by these models.

Risk of bias reporting in the recent animal focal cerebral ischaemia literature.

BACKGROUND: Findings from in vivo research may be less reliable where studies do not report measures to reduce risks of bias. The experimental stroke community has been at the forefront of implementing changes to improve reporting, but it is not known whether these efforts are associated with continuous improvements. Our aims here were firstly to validate an automated tool to assess risks of bias in published works, and secondly to assess the reporting of measures taken to reduce the risk of bias within recent literature for two experimental models of stroke. METHODS: We developed and used text analytic approaches to automatically ascertain reporting of measures to reduce risk of bias from full-text articles describing animal experiments inducing middle cerebral artery occlusion (MCAO) or modelling lacunar stroke. RESULTS: Compared with previous assessments, there were improvements in the reporting of measures taken to reduce risks of bias in the MCAO literature but not in the lacunar stroke literature. Accuracy of automated annotation of risk of bias in the MCAO literature was 86% (randomization), 94% (blinding) and 100% (sample size calculation); and in the lacunar stroke literature accuracy was 67% (randomization), 91% (blinding) and 96% (sample size calculation). DISCUSSION: There remains substantial opportunity for improvement in the reporting of animal research modelling stroke, particularly in the lacunar stroke literature. Further, automated tools perform sufficiently well to identify whether studies report blinded assessment of outcome, but improvements are required in the tools to ascertain whether randomization and a sample size calculation were reported.

Systematic review and stratified meta-analysis of the efficacy of carnosine in animal models of ischemic stroke.

Carnosine is a naturally occurring pleotropic dipeptide which influences multiple deleterious mechanisms that are activated during stroke. Numerous published studies have reported that carnosine has robust efficacy in ischemic stroke models. To further evaluate these data, we have conducted a systematic review and meta-analysis of published studies. We included publications describing in vivo models of ischemic stroke where the neuroprotective efficacy of carnosine was being evaluated through the reporting of infarct volume and/or neurological score as outcomes. Overall efficacy was evaluated using weighted mean difference random effects meta-analysis. We also evaluated for study quality and publication bias. We identified eight publications that met our inclusion criteria describing a total of 29 comparisons and 454 animals. Overall methodological quality of studies was moderate (median = 4/9). Carnosine reduced infarct volume by 29.4% (95% confidence interval (CI), 24.0% to 34.9%; 29 comparisons). A clear dose-response effect was observed, and efficacy was reduced when carnosine was administered more than 6 h after ischemia. Our findings suggest that carnosine administered before or after the onset of ischemia exhibits robust efficacy in experimental ischemic stroke. However, the methodological quality of some of the studies was low and testing occurred only in healthy young male animals.

Improving our understanding of the in vivo modelling of psychotic disorders: A protocol for a systematic review and meta-analysis.

Psychosis represents a set of symptoms against which current available treatments are not universally effective and are often accompanied by adverse side effects. Clinical management could potentially be improved with a greater understanding of the underlying biology and subsequently with the introduction of novel treatments. Since many clinical drug candidates are identified through in vivo modelling, a deeper understanding of the pre-clinical field, might help us understand why translation of results from animal models to inform mental health clinical practice has so far been weak. We set out to give a shallow, but broad unbiased overview of experiments looking at the in vivo modelling of psychotic disorders using a systematic review and meta-analysis. This protocol describes the exact methodology we propose to follow in order to quantitatively review both studies characterizing a model and those experiments that investigate the effects of novel therapeutic options. We are interested in assessing the prevalence of the reporting of measures to reduce risk of bias, and the internal and external validity of the animal models and outcome measures used to validate these models. This generation of strong empirical evidence has the potential to identify areas for improvement, make suggestions for future research avenues, and ultimately inform what we think we know to improve the current attrition rate between bench and bedside in psychosis research. A review like this will also support the reduction of animal numbers used in research and the refinement of experiments to maximize their value in informing the field.

Correction: Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

[This corrects the article DOI: 10.1371/journal.pbio.1002273.].

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.