The impact of conducting preclinical systematic reviews on researchers and their research: A mixed method case study.

BACKGROUND: Systematic reviews (SRs) are cornerstones of evidence-based medicine and have contributed significantly to breakthroughs since the 1980's. However, preclinical SRs remain relatively rare despite their many advantages. Since 2011 the Dutch health funding organisation (ZonMw) has run a grant scheme dedicated to promoting the training, coaching and conduct of preclinical SRs. Our study focuses on this funding scheme to investigate the relevance, effects and benefits of conducting preclinical SRs on researchers and their research. METHODS: We recruited researchers who attended funded preclinical SR workshops and who conducted, are still conducting, or prematurely stopped a SR with funded coaching. We gathered data using online questionnaires followed by semi-structured interviews. Both aimed to explore the impact of conducting a SR on researchers' subsequent work, attitudes, and views about their research field. Data-analysis was performed using Excel and ATLAS.ti. RESULTS: Conducting preclinical SRs had two distinct types of impact. First, the researchers acquired new skills and insights, leading to a change in mindset regarding the quality of animal research. This was mainly seen in the way participants planned, conducted and reported their subsequent animal studies, which were more transparent and of a higher quality than their previous work. Second, participants were eager to share their newly acquired knowledge within their laboratories and to advocate for change within their research teams and fields of interest. In particular, they emphasised the need for preclinical SRs and improved experimental design within preclinical research, promoting these through education and published opinion papers. CONCLUSION: Being trained and coached in the conduct of preclinical SRs appears to be a contributing factor to many beneficial changes which will impact the quality of preclinical research in the long-term. Our findings suggest that this ZonMw funding scheme is helpful in improving the quality and transparency of preclinical research. Similar funding schemes should be encouraged, preferably by a broader group of funders or financers, in the future.

Introduction to the EQIPD quality system.

While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers, and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e. performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.

Is it possible to overcome issues of external validity in preclinical animal research? Why most animal models are bound to fail.

BACKGROUND: The pharmaceutical industry is in the midst of a productivity crisis and rates of translation from bench to bedside are dismal. Patients are being let down by the current system of drug discovery; of the several 1000 diseases that affect humans, only a minority have any approved treatments and many of these cause adverse reactions in humans. A predominant reason for the poor rate of translation from bench to bedside is generally held to be the failure of preclinical animal models to predict clinical efficacy and safety. Attempts to explain this failure have focused on problems of internal validity in preclinical animal studies (e.g. poor study design, lack of measures to control bias). However there has been less discussion of another key factor that influences translation, namely the external validity of preclinical animal models. REVIEW OF PROBLEMS OF EXTERNAL VALIDITY: External validity is the extent to which research findings derived in one setting, population or species can be reliably applied to other settings, populations and species. This paper argues that the reliable translation of findings from animals to humans will only occur if preclinical animal studies are both internally and externally valid. We review several key aspects that impact external validity in preclinical animal research, including unrepresentative animal samples, the inability of animal models to mimic the complexity of human conditions, the poor applicability of animal models to clinical settings and animal-human species differences. We suggest that while some problems of external validity can be overcome by improving animal models, the problem of species differences can never be overcome and will always undermine external validity and the reliable translation of preclinical findings to humans. CONCLUSION: We conclude that preclinical animal models can never be fully valid due to the uncertainties introduced by species differences. We suggest that even if the next several decades were spent improving the internal and external validity of animal models, the clinical relevance of those models would, in the end, only improve to some extent. This is because species differences would continue to make extrapolation from animals to humans unreliable. We suggest that to improve clinical translation and ultimately benefit patients, research should focus instead on human-relevant research methods and technologies.

SYRCLE's risk of bias tool for animal studies.

BACKGROUND: Systematic Reviews (SRs) of experimental animal studies are not yet common practice, but awareness of the merits of conducting such SRs is steadily increasing. As animal intervention studies differ from randomized clinical trials (RCT) in many aspects, the methodology for SRs of clinical trials needs to be adapted and optimized for animal intervention studies. The Cochrane Collaboration developed a Risk of Bias (RoB) tool to establish consistency and avoid discrepancies in assessing the methodological quality of RCTs. A similar initiative is warranted in the field of animal experimentation. METHODS: We provide an RoB tool for animal intervention studies (SYRCLE's RoB tool). This tool is based on the Cochrane RoB tool and has been adjusted for aspects of bias that play a specific role in animal intervention studies. To enhance transparency and applicability, we formulated signalling questions to facilitate judgment. RESULTS: The resulting RoB tool for animal studies contains 10 entries. These entries are related to selection bias, performance bias, detection bias, attrition bias, reporting bias and other biases. Half these items are in agreement with the items in the Cochrane RoB tool. Most of the variations between the two tools are due to differences in design between RCTs and animal studies. Shortcomings in, or unfamiliarity with, specific aspects of experimental design of animal studies compared to clinical studies also play a role. CONCLUSIONS: SYRCLE's RoB tool is an adapted version of the Cochrane RoB tool. Widespread adoption and implementation of this tool will facilitate and improve critical appraisal of evidence from animal studies. This may subsequently enhance the efficiency of translating animal research into clinical practice and increase awareness of the necessity of improving the methodological quality of animal studies.

The effects of long-term omega-3 fatty acid supplementation on cognition and Alzheimer's pathology in animal models of Alzheimer's disease: a systematic review and meta-analysis.

To date, only a few randomized clinical trials (RCTs) have investigated the effects of omega-3 fatty acids (FA) on Alzheimer's disease (AD). Some of these studies demonstrated that patients with very mild AD or mild cognitive impairment benefit from omega-3 FA treatment, but none showed significant improvements in cognitive function in patients with moderate or advanced AD. All these RCTs had a relatively short duration of supplementation, however, and we hypothesized that this might be one of the reasons why no effects of omega-3 FA supplementation could be observed in patients with "moderate" or "advanced" AD. Animal studies offer better possibilities for controlled long-term supplementation than clinical studies. Therefore, we performed a systematic review (SR) and meta-analysis of the literature that focused on effects of the relatively long-term omega-3 FA supplementation (minimum period; 10% of average total lifespan) on cognitive impairment, amyloid-ß pathology, and neuronal loss in animal models of AD. This SR shows that long-term omega-3 FA supplementation decreased the omega-6/omega-3 FA ratio and reduced the amount of amyloid-ß in experimental animal models of AD. Omega-3 FA supplementation also improved cognitive function; this effect appeared larger in rats compared to mice, and in males compared to females. Moreover, omega-3 FA supplementation diminished the amount of neuronal loss, especially in female animals. The results of this SR indicate that it might be worthwhile to perform new clinical trials with long-term omega-3 FA supplementation in AD patients.

Effects of vitamin supplementation and hyperhomocysteinemia on atherosclerosis in apoE-deficient mice.

It has been demonstrated that hyperhomocysteinemia (HHcy) accelerates atherosclerosis in apolipoprotein E-deficient (apoE(-/-)) mice. In this study, vitamin-defined chow diets were used to induce HHcy in apoE(-/-) mice in an attempt to identify possible pathogenic pathways. Six-week-old female apoE(-/-) mice were divided into seven groups: vitamin-defined purified chow diet alone (control), or same diet supplemented with either D,L-homocysteine (upward arrow Hcy) or L-homocystine (upward arrow Hcy-Hcy), or diet high in L-methionine (upward arrow Met), or diet high in B-vitamins (upward arrow vitamin), or diets deficient in folate (downward arrow folate) or vitamin B(6) ( downward arrow B(6)). Eighteen weeks later, plasma total homocysteine (tHcy), lipids and atherosclerotic plaque burden (aortic root, aortic arch, and brachiocephalic trunk) were measured. tHcy levels were similar in the upward arrow vitamin, downward arrow folate, downward arrow B(6) and control groups (9.2-10.1 micromol/l, NS), but elevated mildly in the upward arrow Hcy-Hcy group (16.1 micromol/l) and moderately in the upward arrow Met and upward arrow Hcy groups (53.6 and 51.5 micromol/l, respectively). Mice in the latter two groups had significantly more atherosclerosis in the aortic root. Although B vitamin-supplementation failed to lower tHcy levels, mice had less atherosclerosis in the aortic arch. In summary, dietary methionine and homocysteine, but not homocystine, enhanced the development of atherosclerosis. Supplementation with B vitamins appeared to confer homocysteine-independent protection against atherosclerosis. These results suggest that (1) there may be a threshold level below which homocysteine is not atherogenic; (2) the atherogenic effect of HHcy may be mediated via an intracellular pathway; and/or (3) the anti-atherogenic effect of B vitamins in normohomocysteinemic mice is independent of tHcy levels.