Variation in Model-Based Economic Evaluations of Low-Dose Computed Tomography Screening for Lung Cancer: A Methodological Review.

OBJECTIVES: There is significant heterogeneity in the results of published model-based economic evaluations of low-dose computed tomography (LDCT) screening for lung cancer. We sought to understand and demonstrate how these models differ. METHODS: An expansion and update of a previous systematic review (N = 19). Databases (including MEDLINE and Embase) were searched. Studies were included if strategies involving (single or multiple) LDCT screening were compared with no screening or other imaging modalities, in a population at risk of lung cancer. More detailed data extraction of studies from the previous review was conducted. Studies were critically appraised using the Consensus Health Economic Criteria list. RESULTS: A total of 16 new studies met the inclusion criteria, giving a total of 35 studies. There are geographic and temporal differences and differences in screening intervals and eligible populations. Studies varied in the types of models used, for example, decision tree, Markov, and microsimulation models. Most conducted a cost-effectiveness analysis (using life-years gained) or cost-utility analysis. The potential for overdiagnosis was considered in many models, unlike with other potential consequences of screening. Some studies report considering lead-time bias, but fewer mention length bias. Generally, the more recent studies, involving more complex modeling, tended to meet more of the critical appraisal criteria, with notable exceptions. CONCLUSIONS: There are many differences across the economic evaluations contributing to variation in estimates of the cost-effectiveness of LDCT screening for lung cancer. Several methodological factors and evidence needs have been highlighted that will require consideration in future economic evaluations to achieve better agreement.

Immunosuppressive agents in adult kidney transplantation in the National Health Service: a model-based economic evaluation.

BACKGROUND: Immunosuppression is required in kidney transplantation to prevent rejection and prolong graft survival. We conducted an economic evaluation to support England's National Institute for Health and Care Excellence in developing updated guidance on the use of immunosuppression, incorporating new immunosuppressive agents, and addressing changes in pricing and the evidence base. METHODS: A discrete-time state transition model was developed to simulate adult kidney transplant patients over their lifetime. A total of 16 different regimens were modelled to assess the cost-effectiveness of basiliximab and rabbit anti-thymocyte globulin (rabbit ATG) as induction agents (with no antibody induction as a comparator) and immediate-release tacrolimus, prolonged-release tacrolimus, mycophenolate mofetil, mycophenolate sodium, sirolimus, everolimus and belatacept as maintenance agents (with ciclosporin and azathioprine as comparators). Graft survival was extrapolated from acute rejection rates, graft function and post-transplant diabetes rates, all estimated at 12 months post-transplantation. National Health Service (NHS) and personal social services costs were included. Cost-effectiveness thresholds of £20 000 and £30 000 per quality-adjusted life year were used. RESULTS: Basiliximab was predicted to be more effective and less costly than rabbit ATG and induction without antibodies. Immediate-release tacrolimus and mycophenolate mofetil were cost-effective as maintenance therapies. Other therapies were either more expensive and less effective or would only be cost-effective if a threshold in excess of £100 000 per quality-adjusted life year were used. CONCLUSIONS: A regimen comprising induction with basiliximab, followed by maintenance therapy with immediate-release tacrolimus and mycophenolate mofetil, is likely to be effective for uncomplicated adult kidney transplant patients and a cost-effective use of NHS resources.

The cost-effectiveness of mandatory 20 mph zones for the prevention of injuries.

BACKGROUND: Traffic calming and speed limits are major public health strategies for further reducing road injuries, especially for vulnerable pedestrians such as children and the elderly. We conducted a cost-benefit analysis (CBA-favoured by transport economists) alongside a cost-utility analysis (CUA-favoured by health economists) of mandatory 20 mph zones, providing a unique opportunity to compare assumptions and results. METHODS: A CUA from the public sector perspective and a CBA from a broader societal perspective. One-way, threshold and probabilistic sensitivity analyses were undertaken. RESULTS: In low casualty areas the intervention was not cost-effective regardless of approach (CUA: cost per QALY = £429 800; CBA: net present value = -£25 500). In high casualty areas, the intervention was cost-effective from the CBA (a saving of £90 600), but not from the CUA [cost per quality-adjusted life year (QALY) = £86 500; assuming National Institute for Health and Clinical Excellence's benchmark for approving health technologies]. CONCLUSIONS: Mandatory 20 mph zones may be cost-effective in high casualty areas when a CBA from a societal perspective is considered. Although CBA may appear, in principle, more appropriate, the quality, age or absence of reliable data for many parameters means that there is a great deal of uncertainty and the results should be interpreted with caution.

Evolution of a cost-utility model of donepezil for Alzheimer's disease.

OBJECTIVES: The aim of this study was to describe the evolution of a cost-utility model used to inform the UK National Institute for Health and Clinical Excellence's (NICE) most recent decisions on the cost-utility of drug treatments for Alzheimer's disease (AD), and to explore the impact of structural assumptions on the cost-utility results. METHODS: Changes informed by noted limitations of the decision model used in NICE's previous decisions (in 2006) were made cumulatively to the original decision model for donepezil compared with best supportive care (for patients with mild to moderate AD). Deterministic and probabilistic analyses were undertaken for each cumulative change of the model. The expected value of perfect information (EVPI) of parameter estimates and structural assumptions was also calculated. RESULTS: Cumulative changes to the decision model highlighted how the results of the original model (incremental cost-effectiveness ratio of £81,000 per quality-adjusted life-year gained) related to those of the new model (where donepezil was estimated to be cost-saving), mainly due to uncertainty in the incremental cost of donepezil treatment over best supportive care (ranging from -£600 to £3,000 per patient). The partial EVPI analysis reflected this finding where further information on treatment discontinuations and cost parameter estimates were shown to be valuable in terms of reducing decision uncertainty. CONCLUSIONS: Assessing the evolution of the cost-utility model helped to identify and explore structural differences between cohort-based models and is likely to be useful for decision models in other disease areas. This approach makes the structural uncertainty explicit, forcing decision makers to address structural uncertainty in addition to parameter uncertainty.

Serial high-sensitivity cardiac troponin testing for the diagnosis of myocardial infarction: a scoping review.

OBJECTIVES: We aimed to assess the diversity and practices of existing studies on several assays and algorithms for serial measurements of high-sensitivity cardiac troponin (hs-cTn) for risk stratification and the diagnosis of myocardial infarction (MI) and 30-day outcomes in patients suspected of having non-ST-segment elevation MI (NSTEMI). METHODS: We searched multiple databases including MEDLINE, EMBASE, Science Citation Index, the Cochrane Database of Systematic Reviews and the CENTRAL databases for studies published between January 2006 and November 2021. Studies that assessed the diagnostic accuracy of serial hs-cTn testing in patients suspected of having NSTEMI in the emergency department (ED) were eligible. Data were analysed using the scoping review method. RESULTS: We included 86 publications, mainly from research centres in Europe, North America and Australasia. Two hs-cTn assays, manufactured by Abbott (43/86) and Roche (53/86), dominated the evaluations. The studies most commonly measured the concentrations of hs-cTn at two time points, at presentation and a few hours thereafter, to assess the two-strata or three-strata algorithm for diagnosing or ruling out MI. Although data from 83 studies (97%) were prospectively collected, 0%-90% of the eligible patients were excluded from the analysis due to missing blood samples or the lack of a final diagnosis in 53 studies (62%) that reported relevant data. Only 19 studies (22%) reported on head-to-head comparisons of alternative assays. CONCLUSION: Evidence on the accuracy of serial hs-cTn testing was largely derived from selected research institutions and relied on two specific assays. The proportions of the eligible patients excluded from the study raise concerns about directly applying the study findings to clinical practice in frontline EDs. PROSPERO REGISTRATION NUMBER: CRD42018106379.

Strategies to identify individuals with monogenic diabetes: results of an economic evaluation.

OBJECTIVES: To evaluate and compare the lifetime costs associated with strategies to identify individuals with monogenic diabetes and change their treatment to more appropriate therapy. DESIGN: A decision analytical model from the perspective of the National Health Service (NHS) in England and Wales was developed and analysed. The model was informed by the literature, routinely collected data and a clinical study conducted in parallel with the modelling. SETTING: Secondary care in the UK. PARTICIPANTS: Simulations based on characteristics of patients diagnosed with diabetes <30 years old. INTERVENTIONS: Four test-treatment strategies to identify individuals with monogenic diabetes in a prevalent cohort of diabetics diagnosed under the age of 30 years were modelled: clinician-based genetic test referral, targeted genetic testing based on clinical prediction models, targeted genetic testing based on biomarkers, and blanket genetic testing. The results of the test-treatment strategies were compared with a strategy of no genetic testing. PRIMARY AND SECONDARY OUTCOME MEASURES: Discounted lifetime costs, proportion of cases of monogenic diabetes identified. RESULTS: Based on current evidence, strategies using clinical characteristics or biomarkers were estimated to save approximately £100-£200 per person with diabetes over a lifetime compared with no testing. Sensitivity analyses indicated that the prevalence of monogenic diabetes, the uptake of testing, and the frequency of home blood glucose monitoring had the largest impact on the results (ranging from savings of £400-£50 per person), but did not change the overall findings. The model is limited by many model inputs being based on very few individuals, and some long-term data informed by clinical opinion. CONCLUSIONS: Costs to the NHS could be saved with targeted genetic testing based on clinical characteristics or biomarkers. More research should focus on the economic case for the use of such strategies closer to the time of diabetes diagnosis. TRIAL REGISTRATION NUMBER: NCT01238380.

Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study.

BACKGROUND: In meta-analysis, the presence of funnel plot asymmetry is attributed to publication or other small-study effects, which causes larger effects to be observed in the smaller studies. This issue potentially mean inappropriate conclusions are drawn from a meta-analysis. If meta-analysis is to be used to inform decision-making, a reliable way to adjust pooled estimates for potential funnel plot asymmetry is required. METHODS: A comprehensive simulation study is presented to assess the performance of different adjustment methods including the novel application of several regression-based methods (which are commonly applied to detect publication bias rather than adjust for it) and the popular Trim & Fill algorithm. Meta-analyses with binary outcomes, analysed on the log odds ratio scale, were simulated by considering scenarios with and without i) publication bias and; ii) heterogeneity. Publication bias was induced through two underlying mechanisms assuming the probability of publication depends on i) the study effect size; or ii) the p-value. RESULTS: The performance of all methods tended to worsen as unexplained heterogeneity increased and the number of studies in the meta-analysis decreased. Applying the methods conditional on an initial test for the presence of funnel plot asymmetry generally provided poorer performance than the unconditional use of the adjustment method. Several of the regression based methods consistently outperformed the Trim & Fill estimators. CONCLUSION: Regression-based adjustments for publication bias and other small study effects are easy to conduct and outperformed more established methods over a wide range of simulation scenarios.

Interspecies extrapolation in environmental exposure standard setting: A Bayesian synthesis approach.

Currently the extrapolation of evidence from studies of non-human species to the setting of environmental exposure standards for humans includes the imposition of a variety of uncertainty factors reflecting unknown aspects of the procedure, including the relevance of evidence from one species to impacts in another. This paper develops and explores more flexible modelling of aspects of this extrapolation, using models proposed by DuMouchel [DuMouchel, W.H., Harris, J.E., 1983. Bayes methods for combining the results of cancer studies in humans and other species (with comment). J. Am. Statist. Assoc. 78, 293-308.] The approaches are based on Bayesian meta-analysis methods involving explicit modelling of relevance in the prior distributions, estimated using Markov chain Monte Carlo (MCMC) methods. The methods are applied to evidence relating chlorinated by-products exposure to adverse reproductive health effects. The relative merits of various approaches are discussed, and developments and next steps are outlined.

Diagnostic accuracy of contemporary and high-sensitivity cardiac troponin assays used in serial testing, versus single-sample testing as a comparator, to triage patients suspected of acute non-ST-segment elevation myocardial infarction: a systematic review protocol.

INTRODUCTION: Although the new generation of cardiac troponin assays have revolutionised the diagnosis of myocardial infarction (MI), their application in triaging patients with suspected acute coronary syndrome requires further investigation. The objectives of the current systematic review are to evaluate the diagnostic accuracy of contemporary and high-sensitivity cardiac troponin assays used in serial testing, versus single-sample testing as a comparator, to identify patients with non-ST-segment-elevation MI in the emergency department. METHODS AND ANALYSIS: We will conduct systematic searches of MEDLINE, EMBASE, Science Citation Index, the Cochrane Database of Systematic Reviews and the CENTRAL database covering the period from 1 January 2006 to present, with no restrictions on language or publication status. Two review authors will independently screen studies for inclusion, extract data from eligible studies and assess their methodological quality using Quality Assessment of Diagnostic Accuracy Studies version 2. Studies will be included if they evaluate contemporary or high-sensitivity cardiac troponin assays used in serial testing, in patients presenting to the ED with suspicion of MI. Estimates of sensitivity and specificity from each study will be presented in forest plots and in the receiver-operating characteristics space. If appropriate, we will pool the results using Bayesian hierarchical models that allow correction for imperfect reference standard. We will obtain summary estimates of sensitivity and specificity of alternative testing protocols and compare their accuracy. We will also investigate the impact of prespecified sources of heterogeneity and methodological quality items. If pooling of results is considered inappropriate, we will present our findings in tables and diagrams and will describe them narratively. ETHICS AND DISSEMINATION: No formal ethical approval will be sought, but we will report on the ethical approval of the included studies. Dissemination of findings will be through publications in peer-reviewed journals, presentations at conferences and the websites of the universities. PROSPERO REGISTRATION NUMBER: CRD42018106379.

Comparison of two methods to detect publication bias in meta-analysis.

CONTEXT: Egger's regression test is often used to help detect publication bias in meta-analyses. However, the performance of this test and the usual funnel plot have been challenged particularly when the summary estimate is the natural log of the odds ratio (lnOR). OBJECTIVE: To compare the performance of Egger's regression test with a regression test based on sample size (a modification of Macaskill's test) with lnOR as the summary estimate. DESIGN: Simulation of meta-analyses under a number of scenarios in the presence and absence of publication bias and between-study heterogeneity. MAIN OUTCOME MEASURES: Type I error rates (the proportion of false-positive results) for each regression test and their power to detect publication bias when it is present (the proportion of true-positive results). RESULTS: Type I error rates for Egger's regression test are higher than those for the alternative regression test. The alternative regression test has the appropriate type I error rates regardless of the size of the underlying OR, the number of primary studies in the meta-analysis, and the level of between-study heterogeneity. The alternative regression test has comparable power to Egger's regression test to detect publication bias under conditions of low between-study heterogeneity. CONCLUSION: Because of appropriate type I error rates and reduction in the correlation between the lnOR and its variance, the alternative regression test can be used in place of Egger's regression test when the summary estimates are lnORs.

Evidence used in model-based economic evaluations for evaluating pharmacogenetic and pharmacogenomic tests: a systematic review protocol.

INTRODUCTION: Decision models can be used to conduct economic evaluations of new pharmacogenetic and pharmacogenomic tests to ensure they offer value for money to healthcare systems. These models require a great deal of evidence, yet research suggests the evidence used is diverse and of uncertain quality. By conducting a systematic review, we aim to investigate the test-related evidence used to inform decision models developed for the economic evaluation of genetic tests. METHODS AND ANALYSIS: We will search electronic databases including MEDLINE, EMBASE and NHS EEDs to identify model-based economic evaluations of pharmacogenetic and pharmacogenomic tests. The search will not be limited by language or date. Title and abstract screening will be conducted independently by 2 reviewers, with screening of full texts and data extraction conducted by 1 reviewer, and checked by another. Characteristics of the decision problem, the decision model and the test evidence used to inform the model will be extracted. Specifically, we will identify the reported evidence sources for the test-related evidence used, describe the study design and how the evidence was identified. A checklist developed specifically for decision analytic models will be used to critically appraise the models described in these studies. Variations in the test evidence used in the decision models will be explored across the included studies, and we will identify gaps in the evidence in terms of both quantity and quality. DISSEMINATION: The findings of this work will be disseminated via a peer-reviewed journal publication and at national and international conferences.

Can clinical features be used to differentiate type 1 from type 2 diabetes? A systematic review of the literature.

OBJECTIVE: Clinicians predominantly use clinical features to differentiate type 1 from type 2 diabetes yet there are no evidence-based clinical criteria to aid classification of patients. Misclassification of diabetes is widespread (7-15% of cases), resulting in patients receiving inappropriate treatment. We sought to identify which clinical criteria could be used to discriminate type 1 and type 2 diabetes. DESIGN: Systematic review of all diagnostic accuracy studies published since 1979 using clinical criteria to predict insulin deficiency (measured by C-peptide). DATA SOURCES: 14 databases including: MEDLINE, MEDLINE in Process and EMBASE. The search strategy took the form of: (terms for diabetes) AND (terms for C-Peptide). ELIGIBILITY CRITERIA: Diagnostic accuracy studies of any routinely available clinical predictors against a reference standard of insulin deficiency defined by cut-offs of C-peptide concentrations. No restrictions on race, age, language or country of origin. RESULTS: 10,917 abstracts were screened, and 231 full texts reviewed. 11 studies met inclusion criteria, but varied by age, race, year and proportion of participants who were C-peptide negative. Age at diagnosis was the most discriminatory feature in 7/9 studies where it was assessed, with optimal cut-offs (>70% mean sensitivity and specificity) across studies being <30 years or <40 years. Use of/time to insulin treatment and body mass index (BMI) were also discriminatory. When combining features, BMI added little over age at diagnosis and/or time to insulin (<1% improvement in classification). CONCLUSIONS: Despite finding only 11 studies, and considerable heterogeneity between studies, age at diagnosis and time to insulin were consistently the most discriminatory criteria. BMI, despite being widely used in clinical practice, adds little to these two criteria. The criteria identified are similar to the Royal College of General Practitioners National Health Service (RCGP/NHS) Diabetes classification guidelines, which use age at diagnosis <35 years and time to insulin <6 m. Until further studies are carried out, these guidelines represent a suitable classification scheme. SYSTEMATIC REVIEW REGISTRATION: PROSPERO reference CRD42012001736.

Development of an economic evaluation of diagnostic strategies: the case of monogenic diabetes.

OBJECTIVES: To describe the development process for defining an appropriate model structure for the economic evaluation of test-treatment strategies for patients with monogenic diabetes (caused by mutations in the GCK, HNF1A or HNF4A genes). DESIGN: Experts were consulted to identify and define realistic test-treatment strategies and care pathways. A systematic assessment of published diabetes models was undertaken to inform the model structure. SETTING: National Health Service in England and Wales. PARTICIPANTS: Experts in monogenic diabetes whose collective expertise spans the length of the patient care pathway. PRIMARY AND SECONDARY OUTCOMES: A defined model structure, including the test-treatment strategies, and the selection of a published diabetes model appropriate for the economic evaluation of strategies to identify patients with monogenic diabetes. RESULTS: Five monogenic diabetes test-treatment strategies were defined: no testing of any kind, referral for genetic testing based on clinical features as noted by clinicians, referral for genetic testing based on the results of a clinical prediction model, referral for genetic testing based on the results of biochemical and immunological tests, referral for genetic testing for all patients with a diagnosis of diabetes under the age of 30 years. The systematic assessment of diabetes models identified the IMS CORE Diabetes Model (IMS CDM) as a good candidate for modelling the long-term outcomes and costs of the test-treatment strategies for monogenic diabetes. The short-term test-treatment events will be modelled using a decision tree which will feed into the IMS CDM. CONCLUSIONS: Defining a model structure for any economic evaluation requires decisions to be made. Expert consultation and the explicit use of critical appraisal can inform these decisions. Although arbitrary choices have still been made, decision modelling allows investigation into such choices and the impact of assumptions that have to be made due to a lack of data.

Evaluating the impact and use of Transparent Reporting of Evaluations with Non-randomised Designs (TREND) reporting guidelines.

INTRODUCTION: Accurate and full reporting of evaluation of interventions in health research is needed for evidence synthesis and informed decision-making. Evidence suggests that biases and incomplete reporting affect the assessment of study validity and the ability to include this data in secondary research. The Transparent Reporting of Evaluations with Non-randomised Designs (TREND) reporting guideline was developed to improve the transparency and accuracy of the reporting of behavioural and public health evaluations with non-randomised designs. Evaluations of reporting guidelines have shown that they can be effective in improving reporting completeness. Although TREND occupies a niche within reporting guidelines, and despite it being 8 years since publication, no study yet has assessed its impact on reporting completeness or investigated what factors affect its use by authors and journal editors. This protocol describes two studies that aim to redress this. METHODS AND ANALYSIS: Study 1 will use an observational design to examine the uptake and use of TREND by authors, and by journals in their instructions to authors. A comparison of reporting completeness and study quality of papers that do and do not use TREND to inform reporting will be made. Study 2 will use a cross-sectional survey to investigate what factors inhibit or facilitate authors' and journal editors' use of TREND. Semistructured interviews will also be conducted with a subset of authors and editors to explore findings from study 1 and the surveys in greater depth. ETHICS AND DISSEMINATION: These studies will generate evidence of how implementation and dissemination of the TREND guideline has affected reporting completeness in studies with experimental, non-randomised designs within behavioural and public health research. The project has received ethics approval from the Research Ethics Committee of the Peninsula College of Medicine and Dentistry, Universities of Exeter and Plymouth.

Identifying clinical criteria to predict Type 1 diabetes, as defined by absolute insulin deficiency: a systematic review protocol.

INTRODUCTION: Management of a patient's diabetes is entirely dependent upon the type of diabetes they are deemed to have. Patients with Type 1 diabetes are insulin deficient so require multiple daily insulin injections, whereas patients with Type 2 diabetes still have some endogenous insulin production so insulin treatment is only required when diet and tablets do not establish good glycaemic control. Despite the importance of a correct diagnosis, classification of diabetes is based on aetiology and relies on clinical judgement. There are no clinical guidelines on how to determine whether a patient has Type 1 or Type 2 diabetes. We aim to systematically review the literature to derive evidence-based clinical criteria for the classification of the major subtypes of diabetes. METHODS AND ANALYSIS: We will perform a systematic review of diagnostic accuracy studies to establish clinical criteria that predict the subsequent development of absolute insulin deficiency seen in Type 1 diabetes. Insulin deficiency will be determined by reference standard C-peptide concentrations. Synthesis of criteria identified will be undertaken using hierarchical summary receiver operating characteristic curves. ETHICS AND DISSEMINATION: As this is a systematic review, there will be no ethical issues. We will disseminate results by writing up the final systematic review and synthesis for publication in a peer-reviewed journal and will present at national and international diabetes-related meetings.

Meta-analysis of repeated measures study designs.

RATIONALE, AIMS AND OBJECTIVES: Repeated measures studies are found in many areas of research, particularly in areas of healthcare. There is currently little information available to inform the method of meta-analysis of repeated measures studies so that the structural dependence of the data is appropriately accommodated and the findings are meaningful. METHOD: Using a published meta-analysis on the impact of diet advice on weight reduction of obese or overweight individuals, we demonstrate possible approaches for repeated measures meta-analysis. These approaches differ in terms of the type of result obtained (e.g. effect at a particular time-point, trend over time, change between time-points) and the data needed for the analysis (e.g. means, regression slope estimates). Some approaches involve violating assumptions of independence in the data structure and so to investigate the impact of this violation a simulation study is carried out. RESULTS: The different approaches described for the meta-analyses of repeated measures studies can all provide useful effect estimates depending on the question to be addressed by the meta-analysis. However, violation of the independence assumption in some approaches can lead to biased estimates. CONCLUSIONS: In practice, the methods available to carry out meta-analyses of repeated measures studies will not only depend upon the question of interest, but also on the data available from the primary studies.

Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry.

OBJECTIVES: To present the contour-enhanced funnel plot as an aid to differentiating asymmetry due to publication bias from that due to other factors. STUDY DESIGN AND SETTING: An enhancement to the usual funnel plot is proposed that allows the statistical significance of study estimates to be considered. Contour lines indicating conventional milestones in levels of statistical significance (e.g., <0.01, <0.05, <0.1) are added to funnel plots. RESULTS: This contour overlay aids the interpretation of the funnel plot. For example, if studies appear to be missing in areas of statistical nonsignificance, then this adds credence to the possibility that the asymmetry is due to publication bias. Conversely, if the supposed missing studies are in areas of higher statistical significance, this would suggest the cause of the asymmetry may be more likely to be due to factors other than publication bias, such as variable study quality. CONCLUSIONS: We believe this enhancement to funnel plots (i) is simple to implement, (ii) is widely applicable, (iii) greatly improves interpretability, and (iv) should be used routinely.

Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity.

The trim and fill method allows estimation of an adjusted meta-analysis estimate in the presence of publication bias. To date, the performance of the trim and fill method has had little assessment. In this paper, we provide a more comprehensive examination of different versions of the trim and fill method in a number of simulated meta-analysis scenarios, comparing results with those from usual unadjusted meta-analysis models and two simple alternatives, namely use of the estimate from: (i) the largest; or (ii) the most precise study in the meta-analysis. Findings suggest a great deal of variability in the performance of the different approaches. When there is large between-study heterogeneity the trim and fill method can underestimate the true positive effect when there is no publication bias. However, when publication bias is present the trim and fill method can give estimates that are less biased than the usual meta-analysis models. Although results suggest that the use of the estimate from the largest or most precise study seems a reasonable approach in the presence of publication bias, when between-study heterogeneity exists our simulations show that these estimates are quite biased. We conclude that in the presence of publication bias use of the trim and fill method can help to reduce the bias in pooled estimates, even though the performance of this method is not ideal. However, because we do not know whether funnel plot asymmetry is truly caused by publication bias, and because there is great variability in the performance of different trim and fill estimators and models in various meta-analysis scenarios, we recommend use of the trim and fill method as a form of sensitivity analysis as intended by the authors of the method.

A systematic review of systematic reviews and meta-analyses of animal experiments with guidelines for reporting.

To maximize the findings of animal experiments to inform likely health effects in humans, a thorough review and evaluation of the animal evidence is required. Systematic reviews and, where appropriate, meta-analyses have great potential in facilitating such an evaluation, making efficient use of the animal evidence while minimizing possible sources of bias. The extent to which systematic review and meta-analysis methods have been applied to evaluate animal experiments to inform human health is unknown. Using systematic review methods, we examine the extent and quality of systematic reviews and meta-analyses of in vivo animal experiments carried out to inform human health. We identified 103 articles meeting the inclusion criteria: 57 reported a systematic review, 29 a systematic review and a meta-analysis, and 17 reported a meta-analysis only. The use of these methods to evaluate animal evidence has increased over time. Although the reporting of systematic reviews is of adequate quality, the reporting of meta-analyses is poor. The inadequate reporting of meta-analyses observed here leads to questions on whether the most appropriate methods were used to maximize the use of the animal evidence to inform policy or decision-making. We recommend that guidelines proposed here be used to help improve the reporting of systematic reviews and meta-analyses of animal experiments. Further consideration of the use and methodological quality and reporting of such studies is needed.