Designing, conducting, and reporting reproducible animal experiments.

In biomedicine and many other fields, there are growing concerns around the reproducibility of research findings, with many researchers being unable to replicate their own or others' results. This raises important questions as to the validity and usefulness of much published research. In this review, we aim to engage researchers in the issue of research reproducibility and equip them with the necessary tools to increase the reproducibility of their research. We first highlight the causes and potential impact of non-reproducible research and emphasise the benefits of working reproducibly for the researcher and broader research community. We address specific targets for improvement and steps that individual researchers can take to increase the reproducibility of their work. We next provide recommendations for improving the design and conduct of experiments, focusing on in vivo animal experiments. We describe common sources of poor internal validity of experiments and offer practical guidance for limiting these potential sources of bias at different experimental stages, as well as discussing other important considerations during experimental design. We provide a list of key resources available to researchers to improve experimental design, conduct, and reporting. We then discuss the importance of open research practices such as study preregistration and the use of preprints and describe recommendations around data management and sharing. Our review emphasises the importance of reproducible work and aims to empower every individual researcher to contribute to the reproducibility of research in their field.

Effectiveness of biomaterial-based combination strategies for spinal cord repair - a systematic review and meta-analysis of preclinical literature.

STUDY DESIGN: Systematic review and meta-analysis of preclinical literature. OBJECTIVES: To assess the effects of biomaterial-based combination (BMC) strategies for the treatment of Spinal Cord Injury (SCI), the effects of individual biomaterials in the context of BMC strategies, and the factors influencing their efficacy. To assess the effects of different preclinical testing paradigms in BMC strategies. METHODS: We performed a systematic literature search of Embase, Web of Science and PubMed. All controlled preclinical studies describing an in vivo or in vitro model of SCI that tested a biomaterial in combination with at least one other regenerative strategy (cells, drugs, or both) were included. Two review authors conducted the study selection independently, extracted study characteristics independently and assessed study quality using a modified CAMARADES checklist. Effect size measures were combined using random-effects models and heterogeneity was explored using meta-regression with tau2, I2 and R2 statistics. We tested for small-study effects using funnel plot-based methods. RESULTS: 134 publications were included, testing over 100 different BMC strategies. Overall, treatment with BMC therapies improved locomotor recovery by 25.3% (95% CI, 20.3-30.3; n = 102) and in vivo axonal regeneration by 1.6 SD (95% CI 1.2-2 SD; n = 117) in comparison with injury only controls. CONCLUSION: BMC strategies improve locomotor outcomes after experimental SCI. Our comprehensive study highlights gaps in current knowledge and provides a foundation for the design of future experiments.

Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis.

We report a systematic review and meta-analysis of research using animal models of chemotherapy-induced peripheral neuropathy (CIPN). We systematically searched 5 online databases in September 2012 and updated the search in November 2015 using machine learning and text mining to reduce the screening for inclusion workload and improve accuracy. For each comparison, we calculated a standardised mean difference (SMD) effect size, and then combined effects in a random-effects meta-analysis. We assessed the impact of study design factors and reporting of measures to reduce risks of bias. We present power analyses for the most frequently reported behavioural tests; 337 publications were included. Most studies (84%) used male animals only. The most frequently reported outcome measure was evoked limb withdrawal in response to mechanical monofilaments. There was modest reporting of measures to reduce risks of bias. The number of animals required to obtain 80% power with a significance level of 0.05 varied substantially across behavioural tests. In this comprehensive summary of the use of animal models of CIPN, we have identified areas in which the value of preclinical CIPN studies might be increased. Using both sexes of animals in the modelling of CIPN, ensuring that outcome measures align with those most relevant in the clinic, and the animal's pain contextualised ethology will likely improve external validity. Measures to reduce risk of bias should be employed to increase the internal validity of studies. Different outcome measures have different statistical power, and this can refine our approaches in the modelling of CIPN.

Cardiac Stem Cell Treatment in Myocardial Infarction: A Systematic Review and Meta-Analysis of Preclinical Studies.

RATIONALE: Cardiac stem cells (CSC) therapy has been clinically introduced for cardiac repair after myocardial infarction (MI). To date, there has been no systematic overview and meta-analysis of studies using CSC therapy for MI. OBJECTIVE: Here, we used meta-analysis to establish the overall effect of CSCs in preclinical studies and assessed translational differences between and within large and small animals in the CSC therapy field. In addition, we explored the effect of CSC type and other clinically relevant parameters on functional outcome to better predict and design future (pre)clinical studies using CSCs for MI. METHODS AND RESULTS: A systematic search was performed, yielding 80 studies. We determined the overall effect of CSC therapy on left ventricular ejection fraction and performed meta-regression to investigate clinically relevant parameters. We also assessed the quality of included studies and possible bias. The overall effect observed in CSC-treated animals was 10.7% (95% confidence interval 9.4-12.1; P<0.001) improvement in ejection fraction compared with placebo controls. Interestingly, CSC therapy had a greater effect in small animals compared with large animals (P<0.001). Meta-regression indicated that cell type was a significant predictor for ejection fraction improvement in small animals. Minor publication bias was observed in small animal studies. CONCLUSIONS: CSC treatment resulted in significant improvement of ejection fraction in preclinical animal models of MI compared with placebo. There was a reduction in the magnitude of effect in large compared with small animal models. Although different CSC types have overlapping culture characteristics, we observed a significant difference in their effect in post-MI animal studies.

Ensuring transparency and minimization of methodologic bias in preclinical pain research: PPRECISE considerations.

There is growing concern about lack of scientific rigor and transparent reporting across many preclinical fields of biological research. Poor experimental design and lack of transparent reporting can result in conscious or unconscious experimental bias, producing results that are not replicable. The Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership with the U.S. Food and Drug Administration sponsored a consensus meeting of the Preclinical Pain Research Consortium for Investigating Safety and Efficacy (PPRECISE) Working Group. International participants from universities, funding agencies, government agencies, industry, and a patient advocacy organization attended. Reduction of publication bias, increasing the ability of others to faithfully repeat experimental methods, and increased transparency of data reporting were specifically discussed. Parameters deemed essential to increase confidence in the published literature were clear, specific reporting of an a priori hypothesis and definition of primary outcome measure. Power calculations and whether measurement of minimal meaningful effect size to determine these should be a core component of the preclinical research effort provoked considerable discussion, with many but not all agreeing. Greater transparency of reporting should be driven by scientists, journal editors, reviewers, and grant funders. The conduct of high-quality science that is fully reported should not preclude novelty and innovation in preclinical pain research, and indeed, any efforts that curtail such innovation would be misguided. We believe that to achieve the goal of finding effective new treatments for patients with pain, the pain field needs to deal with these challenging issues.

Correction: Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

[This corrects the article DOI: 10.1371/journal.pbio.1002273.].

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.

Incidence, prevalence, and predictors of chemotherapy-induced peripheral neuropathy: A systematic review and meta-analysis.

Chemotherapy-induced peripheral neuropathy (CIPN) is a disabling pain condition resulting from chemotherapy for cancer. Severe acute CIPN may require chemotherapy dose reduction or cessation. There is no effective CIPN prevention strategy; treatment of established chronic CIPN is limited, and the prevalence of CIPN is not known. Here we used a systematic review to identify studies reporting the prevalence of CIPN. We searched Embase, Medline, CAB Abstracts, CINAHL, PubMed central, Cochrane Library, and Web of Knowledge for relevant references and used random-effects meta-regression to estimate overall prevalence. We assessed study quality using the CONSORT and STROBE guidelines, and we report findings according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance. We provide a qualitative summary of factors reported to alter the risk of CIPN. We included 31 studies with data from 4179 patients in our analysis. CIPN prevalence was 68.1% (57.7-78.4) when measured in the first month after chemotherapy, 60.0% (36.4-81.6) at 3months and 30.0% (6.4-53.5) at 6months or more. Different chemotherapy drugs were associated with differences in CIPN prevalence, and there was some evidence of publication bias. Genetic risk factors were reported in 4 studies. Clinical risk factors, identified in 4 of 31 studies, included neuropathy at baseline, smoking, abnormal creatinine clearance, and specific sensory changes during chemotherapy. Although CIPN prevalence decreases with time, at 6months 30% of patients continue to suffer from CIPN. Routine CIPN surveillance during post-chemotherapy follow-up is needed. A number of genetic and clinical risk factors were identified that require further study.

Efficacy of antidepressants in animal models of ischemic stroke: a systematic review and meta-analysis.

BACKGROUND AND PURPOSE: Poststroke depression is a prevalent complication of stroke with unclear pathogenesis. The benefits of antidepressants in this context and their effects on stroke recovery other than effects on mood are not clearly defined, with some studies suggesting efficacy in improving functional outcome in both depressed and nondepressed stroke patients. We have analyzed the preclinical animal data on antidepressant treatment in focal cerebral ischemia, modeled±depression, to help inform clinical trial design. METHODS: We performed a systematic review and meta-analysis of data from experiments testing the efficacy of antidepressants versus no treatment to reduce infarct volume or improve neurobehavioral or neurogenesis outcomes in animal models of stroke. We used random-effects metaregression to test the impact of study quality and design characteristics and used trim and fill to assess publication bias. RESULTS: We identified 44 publications describing the effects of 22 antidepressant drugs. The median quality checklist score was 5 of a possible 10 (interquartile range, 4-7). Overall, antidepressants reduced infarct volume by 27.3% (95% confidence interval, 20.7%-33.8%) and improved neurobehavioral outcomes by 53.7% (46.4%-61.1%). There was little evidence for an effect of selective serotonin reuptake inhibitors on infarct volume. For neurobehavioral outcomes there was evidence of publication bias. Selective serotonin reuptake inhibitors were the most frequently studied antidepressant subtype and improved neurobehavioral outcome by 51.8% (38.6%-64.9%) and increased neurogenesis by 2.2 SD (1.3-3.0). CONCLUSIONS: In line with current clinical data and despite some limitations, antidepressant treatments seem to improve infarct volume and neurobehavioral outcome in animal models of ischemic stroke.

Using animal models to understand cancer pain in humans.

Cancer pain is not a single entity but a complex pain state involving different pain syndromes, with inflammatory, neuropathic, compressive, and ischaemic mechanisms. Current therapeutic regimens are based largely on opioids, although opioid treatment can lead to many side effects. Studies using animal models of cancer pain are aimed at understanding cancer pain and developing novel therapies. The most frequently reported models are of bone cancer pain, predominantly modelling pain associated with tumour growth within bone marrow. Here we summarise recent findings from studies using animal models of cancer pain and discuss the methodological quality of these studies.

Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically.

The use of systematic review and meta-analysis of preclinical studies has become more common, including those of studies describing the modeling of cerebrovascular diseases. Empirical evidence suggests that too many preclinical experiments lack methodological rigor, and this leads to inflated treatment effects. The aim of this review is to describe the concepts of systematic review and meta-analysis and consider how these tools may be used to provide empirical evidence to spur the field to improve the rigor of the conduct and reporting of preclinical research akin to their use in improving the conduct and reporting of randomized controlled trials in clinical research. As with other research domains, systematic reviews are subject to bias. Therefore, we have also suggested guidance for their conduct, reporting, and critical appraisal.

Systematic review and stratified meta-analysis of the efficacy of RhoA and Rho kinase inhibitors in animal models of ischaemic stroke.

BACKGROUND: There is currently only one clinically approved drug, tissue plasminogen activator (tPA), for the treatment of acute ischaemic stroke. The RhoA pathway, including RhoA and its downstream effector Rho kinase (ROCK), has been identified as a possible therapeutic target. Our aim was to assess the impact of study design characteristics and study quality on reported measures of efficacy and to assess for the presence and impact of publication bias. METHODS: We conducted a systematic review and meta-analysis on publications describing the efficacy of RhoA and ROCK inhibitors in animal models of focal cerebral ischaemia where outcome was assessed as a change in lesion size or neurobehavioural score, or both. RESULTS: We identified 25 published papers which met our inclusion criteria. RhoA and ROCK inhibitors reduced lesion size by 37.3% in models of focal cerebral ischaemia (95% CI, 28.6% to 46.0%, 41 comparisons), and reduced neurobehavioural data by 40.5% (33.4% to 47.7%, 30 comparisons). Overall study quality was low (median=4, interquartile range 3-5) and measures to reduce bias were seldom reported. Publication bias was prevalent and associated with a substantial overstatement of efficacy for lesion size. CONCLUSIONS: RhoA and ROCK inhibitors appear to be effective in animal models of stroke. However the low quality score, publication bias and limited number of studies are areas which need attention prior to conducting clinical trials.

Animal models of bone cancer pain: systematic review and meta-analyses.

Pain can significantly decrease the quality of life of patients with advanced cancer. Current treatment strategies often provide inadequate analgesia and unacceptable side effects. Animal models of bone cancer pain are used in the development of novel pharmacological approaches. Here we conducted a systematic review and meta-analysis of publications describing in vivo modelling of bone cancer pain in which behavioural, general health, macroscopic, histological, biochemical, or electrophysiological outcomes were reported and compared to appropriate controls. In all, 150 publications met our inclusion criteria, describing 38 different models of bone cancer pain. Reported methodological quality was low; only 31% of publications reported blinded assessment of outcome, and 11% reported random allocation to group. No publication reported a sample size calculation. Studies that reported measures to reduce bias reported smaller differences in behavioural outcomes between tumour-bearing and control animals, and studies that presented a statement regarding a conflict of interest reported larger differences in behavioural outcomes. Larger differences in behavioural outcomes were reported in female animals, when cancer cells were injected into either the tibia or femur, and when MatLyLu prostate or Lewis Lung cancer cells were used. Mechanical-evoked pain behaviours were most commonly reported; however, the largest difference was observed in spontaneous pain behaviours. In the spinal cord astrocyte activation and increased levels of Substance P receptor internalisation, c-Fos, dynorphin, tumor necrosis factor-α and interleukin-1ß have been reported in bone cancer pain models, suggesting several potential therapeutic targets. However, the translational impact of animal models on clinical pain research could be enhanced by improving methodological quality.

Transparency in the reporting of in vivo pre-clinical pain research: The relevance and implications of the ARRIVE (Animal Research: Reporting In Vivo Experiments) guidelines.

Clear reporting of research is crucial to the scientific process. Poorly designed and reported studies are damaging not only to the efforts of individual researchers, but also to science as a whole. Standardised reporting methods, such as those already established for reporting randomised clinical trials, have led to improved study design and facilitated the processes of clinical systematic review and meta-analysis. Such standards were lacking in the pre-clinical field until the development of the ARRIVE (Animal Research: Reporting In Vivo Experiments) guidelines. These were prompted following a survey which highlighted a widespread lack of robust and consistent reporting of pre-clinical in vivo research, with reports frequently omitting basic information required for study replication and quality assessment. The resulting twenty item checklist in ARRIVE covers all aspects of experimental design with particular emphasis on bias reduction and methodological transparency. Influential publishers and research funders have already adopted ARRIVE. Further dissemination and acknowledgement of the importance of these guidelines is vital to their widespread implementation. Conclusions and implications Wide implementation of the ARRIVE guidelines for reporting of in vivo preclinical research, especially pain research, are essential for a much needed increased transparency and quality in publishing such research. ARRIVE will also positively influence improvements in experimental design and quality, assist the conduct of accurate replication studies of important new findings and facilitate meta-analyses of preclinical research.

Dysregulation of glucocorticoid metabolism in murine obesity: comparable effects of leptin resistance and deficiency.

In obese humans, metabolism of glucocorticoids by 11 beta-hydroxysteroid dehydrogenase type 1 (11 beta-HSD1) and A-ring reduction (by 5 alpha- and 5 beta-reductases) is dysregulated in a tissue specific manner. These changes have been recapitulated in leptin resistant obese Zucker rats but were not observed in high-fat fed Wistar rats. Recent data from mouse models suggest that such discrepancies may reflect differences in leptin signalling. We therefore compared glucocorticoid metabolism in murine models of leptin deficiency and resistance. Male ob/ob and db/db mice and their respective littermate controls (n=10-12/group) were studied at the age of 12 weeks. Enzyme activities and mRNA expression were quantified in snap-frozen tissues. The patterns of altered pathways of steroid metabolism in obesity were similar in ob/ob and db/db mice. In liver, 5 beta-reductase activity and mRNA were increased and 11 beta-HSD1 decreased in obese mice, whereas 5 alpha-reductase 1 (5 alpha R1) mRNA was not altered. In visceral adipose depots, 5 beta-reductase was not expressed, 11 beta-HSD1 activity was increased and 5 alpha R1 mRNA was not altered in obesity. By contrast, in subcutaneous adipose tissue 11 beta-HSD1 and 5 alpha R1 mRNA were decreased. Systematic differences were not found between ob/ob and db/db murine models of obesity, suggesting that variations in leptin signalling through the short splice variant of the Ob receptor do not contribute to dysregulation of glucocorticoid metabolism.