Descriptive inference using large, unrepresentative nonprobability samples: An introduction for ecologists.

Biodiversity monitoring usually involves drawing inferences about some variable of interest across a defined landscape from observations made at a sample of locations within that landscape. If the variable of interest differs between sampled and nonsampled locations, and no mitigating action is taken, then the sample is unrepresentative and inferences drawn from it will be biased. It is possible to adjust unrepresentative samples so that they more closely resemble the wider landscape in terms of "auxiliary variables." A good auxiliary variable is a common cause of sample inclusion and the variable of interest, and if it explains an appreciable portion of the variance in both, then inferences drawn from the adjusted sample will be closer to the truth. We applied six types of survey sample adjustment-subsampling, quasirandomization, poststratification, superpopulation modeling, a "doubly robust" procedure, and multilevel regression and poststratification-to a simple two-part biodiversity monitoring problem. The first part was to estimate the mean occupancy of the plant Calluna vulgaris in Great Britain in two time periods (1987-1999 and 2010-2019); the second was to estimate the difference between the two (i.e., the trend). We estimated the means and trend using large, but (originally) unrepresentative, samples from a citizen science dataset. Compared with the unadjusted estimates, the means and trends estimated using most adjustment methods were more accurate, although standard uncertainty intervals generally did not cover the true values. Completely unbiased inference is not possible from an unrepresentative sample without knowing and having data on all relevant auxiliary variables. Adjustments can reduce the bias if auxiliary variables are available and selected carefully, but the potential for residual bias should be acknowledged and reported.

ROBITT: A tool for assessing the risk-of-bias in studies of temporal trends in ecology.

Aggregated species occurrence and abundance data from disparate sources are increasingly accessible to ecologists for the analysis of temporal trends in biodiversity. However, sampling biases relevant to any given research question are often poorly explored and infrequently reported; this can undermine statistical inference. In other disciplines, it is common for researchers to complete 'risk-of-bias' assessments to expose and document the potential for biases to undermine conclusions. The huge growth in available data, and recent controversies surrounding their use to infer temporal trends, indicate that similar assessments are urgently needed in ecology.We introduce ROBITT, a structured tool for assessing the 'Risk-Of-Bias In studies of Temporal Trends in ecology'. ROBITT has a similar format to its counterparts in other disciplines: it comprises signalling questions designed to elicit information on the potential for bias in key study domains. In answering these, users will define study inferential goal(s) and relevant statistical target populations. This information is used to assess potential sampling biases across domains relevant to the research question (e.g. geography, taxonomy, environment), and how these vary through time. If assessments indicate biases, then users must clearly describe them and/or explain what mitigating action will be taken.Everything that users need to complete a ROBITT assessment is provided: the tool, a guidance document and a worked example. Following other disciplines, the tool and guidance document were developed through a consensus-forming process across experts working in relevant areas of ecology and evidence synthesis.We propose that researchers should be strongly encouraged to include a ROBITT assessment when publishing studies of biodiversity trends, especially when using aggregated data. This will help researchers to structure their thinking, clearly acknowledge potential sampling issues, highlight where expert consultation is required and provide an opportunity to describe data checks that might go unreported. ROBITT will also enable reviewers, editors and readers to establish how well research conclusions are supported given a dataset combined with some analytical approach. In turn, it should strengthen evidence-based policy and practice, reduce differing interpretations of data and provide a clearer picture of the uncertainties associated with our understanding of reality.

PROTOCOL: The association between whole-grain dietary intake and noncommunicable diseases: A systematic review and meta-analysis.

Our primary research questions are: (1) What is the association between whole grains (WG) intake and the prevalence of NCDs (i.e., type 2 diabetes, cardiovascular disease, obesity, cancer, mortality) and their biomarkers? (2) Which biomarker(s) has/have the greatest association with WG intake when combining multiple biomarkers together in the same analysis? Our secondary research question is: (3) Are there dose-response relationships between WG intake and biomarkers and prevalence of NCDs which could help inform a universal recommendation for WG intake?

Author Correction: Eight problems with literature reviews and how to fix them.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Eight problems with literature reviews and how to fix them.

Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to synthesizing evidence across studies; they rely on a suite of evidence-based methods aimed at maximizing rigour and minimizing susceptibility to bias. Despite the increasing popularity of systematic reviews in the environmental field, evidence synthesis methods continue to be poorly applied in practice, resulting in the publication of syntheses that are highly susceptible to bias. Recognizing the constraints that researchers can sometimes feel when attempting to plan, conduct and publish rigorous and comprehensive evidence syntheses, we aim here to identify major pitfalls in the conduct and reporting of systematic reviews, making use of recent examples from across the field. Adopting a 'critical friend' role in supporting would-be systematic reviews and avoiding individual responses to police use of the 'systematic review' label, we go on to identify methodological solutions to mitigate these pitfalls. We then highlight existing support available to avoid these issues and call on the entire community, including systematic review specialists, to work towards better evidence syntheses for better evidence and better decisions.

Consumer attitudes towards production diseases in intensive production systems.

Many members of the public and important stakeholders operating at the upper end of the food chain, may be unfamiliar with how food is produced, including within modern animal production systems. The intensification of production is becoming increasingly common in modern farming. However, intensive systems are particularly susceptible to production diseases, with potentially negative consequences for farm animal welfare (FAW). Previous research has demonstrated that the public are concerned about FAW, yet there has been little research into attitudes towards production diseases, and their approval of interventions to reduce these. This research explores the public's attitudes towards, and preferences for, FAW interventions in five European countries (Finland, Germany, Poland, Spain and the UK). An online survey was conducted for broilers (n = 789), layers (n = 790) and pigs (n = 751). Data were analysed by means of Kruskal-Wallis ANOVA, exploratory factor analysis and structural equation modelling. The results suggest that the public have concerns regarding intensive production systems, in relation to FAW, naturalness and the use of antibiotics. The most preferred interventions were the most "proactive" interventions, namely improved housing and hygiene measures. The least preferred interventions were medicine-based, which raised humane animal care and food safety concerns amongst respondents. The results highlighted the influence of the identified concerns, perceived risks and benefits on attitudes and subsequent behavioural intention, and the importance of supply chain stakeholders addressing these concerns in the subsequent communications with the public.

Using the Value of Information to improve conservation decision making.

Conservation decisions are challenging, not only because they often involve difficult conflicts among outcomes that people value, but because our understanding of the natural world and our effects on it is fraught with uncertainty. Value of Information (VoI) methods provide an approach for understanding and managing uncertainty from the standpoint of the decision maker. These methods are commonly used in other fields (e.g. economics, public health) and are increasingly used in biodiversity conservation. This decision-analytical approach can identify the best management alternative to select where the effectiveness of interventions is uncertain, and can help to decide when to act and when to delay action until after further research. We review the use of VoI in the environmental domain, reflect on the need for greater uptake of VoI, particularly for strategic conservation planning, and suggest promising areas for new research. We also suggest common reporting standards as a means of increasing the leverage of this powerful tool. The environmental science, ecology and biodiversity categories of the Web of Knowledge were searched using the terms 'Value of Information,' 'Expected Value of Perfect Information,' and the abbreviation 'EVPI.' Google Scholar was searched with the same terms, and additionally the terms decision and biology, biodiversity conservation, fish, or ecology. We identified 1225 papers from these searches. Included studies were limited to those that showed an application of VoI in biodiversity conservation rather than simply describing the method. All examples of use of VOI were summarised regarding the application of VoI, the management objectives, the uncertainties, the models used, how the objectives were measured, and the type of VoI. While the use of VoI appears to be on the increase in biodiversity conservation, the reporting of results is highly variable, which can make it difficult to understand the decision context and which uncertainties were considered. Moreover, it was unclear if, and how, the papers informed management and policy interventions, which is why we suggest a range of reporting standards that would aid the use of VoI. The use of VoI in conservation settings is at an early stage. There are opportunities for broader applications, not only for species-focussed management problems, but also for setting local or global research priorities for biodiversity conservation, making funding decisions, or designing or improving protected area networks and management. The long-term benefits of applying VoI methods to biodiversity conservation include a more structured and decision-focused allocation of resources to research.

The Need for Formal Evidence Synthesis in Food Policy: A Case Study of Willingness-to-Pay.

Meta-analysis is increasingly utilised in the understanding of consumer behaviour, including in relation to farm animal welfare. However, the issue of publication bias has received little attention. As willingness-to-pay (WTP) is widely used in policy, it is important to explore publication bias. This research aimed to evaluate publication bias in WTP, specifically public WTP for farm animal welfare. A systematic review of four databases yielded 54 studies for random effects meta-analysis. Publication bias was assessed by the Egger test, rank test, contour-enhanced funnel plots, and the Vevea and Hedges weight-function model. Results consistently indicated the presence of publication bias, highlighting an overestimation of WTP for farm animal welfare. Stakeholders should be wary of WTP estimates that have not been critically evaluated for publication bias.

The use of Bayesian networks to assess the quality of evidence from research synthesis: 1.

BACKGROUND: The grades of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in systematic reviews, health technology assessment and guideline development organisations throughout the world. A key advantage to this approach is that it aids transparency regarding judgments on the quality of evidence. However, the intricacies of making judgments about research methodology and evidence make the GRADE system complex and challenging to apply without training. METHODS: We have developed a semi-automated quality assessment tool (SAQAT) l based on GRADE. This is informed by responses by reviewers to checklist questions regarding characteristics that may lead to unreliability. These responses are then entered into the Bayesian network to ascertain the probabilities of risk of bias, inconsistency, indirectness, imprecision and publication bias conditional on review characteristics. The model then combines these probabilities to provide a probability for each of the GRADE overall quality categories. We tested the model using a range of plausible scenarios that guideline developers or review authors could encounter. RESULTS: Overall, the model reproduced GRADE judgements for a range of scenarios. Potential advantages over standard assessment are use of explicit and consistent weightings for different review characteristics, forcing consideration of important but sometimes neglected characteristics and principled downgrading where small but important probabilities of downgrading are accrued across domains. CONCLUSIONS: Bayesian networks have considerable potential for use as tools to assess the validity of research evidence. The key strength of such networks lies in the provision of a statistically coherent method for combining probabilities across a complex framework based on both belief and evidence. In addition to providing tools for less experienced users to implement reliability assessment, the potential for sensitivity analyses and automation may be beneficial for application and the methodological development of reliability tools.

Higher antioxidant and lower cadmium concentrations and lower incidence of pesticide residues in organically grown crops: a systematic literature review and meta-analyses.

Demand for organic foods is partially driven by consumers' perceptions that they are more nutritious. However, scientific opinion is divided on whether there are significant nutritional differences between organic and non-organic foods, and two recent reviews have concluded that there are no differences. In the present study, we carried out meta-analyses based on 343 peer-reviewed publications that indicate statistically significant and meaningful differences in composition between organic and non-organic crops/crop-based foods. Most importantly, the concentrations of a range of antioxidants such as polyphenolics were found to be substantially higher in organic crops/crop-based foods, with those of phenolic acids, flavanones, stilbenes, flavones, flavonols and anthocyanins being an estimated 19 (95 % CI 5, 33) %, 69 (95 % CI 13, 125) %, 28 (95 % CI 12, 44) %, 26 (95 % CI 3, 48) %, 50 (95 % CI 28, 72) % and 51 (95 % CI 17, 86) % higher, respectively. Many of these compounds have previously been linked to a reduced risk of chronic diseases, including CVD and neurodegenerative diseases and certain cancers, in dietary intervention and epidemiological studies. Additionally, the frequency of occurrence of pesticide residues was found to be four times higher in conventional crops, which also contained significantly higher concentrations of the toxic metal Cd. Significant differences were also detected for some other (e.g. minerals and vitamins) compounds. There is evidence that higher antioxidant concentrations and lower Cd concentrations are linked to specific agronomic practices (e.g. non-use of mineral N and P fertilisers, respectively) prescribed in organic farming systems. In conclusion, organic crops, on average, have higher concentrations of antioxidants, lower concentrations of Cd and a lower incidence of pesticide residues than the non-organic comparators across regions and production seasons.

Assessing the impact of human trampling on vegetation: a systematic review and meta-analysis of experimental evidence.

Vegetation trampling resulting from recreation can adversely impact natural habitats, leading to the loss of vegetation and the degradation of plant communities. A considerable primary literature exists on this topic, therefore it is important to assess whether this accumulated evidence can be used to reach general conclusions concerning vegetation vulnerability to inform conservation management decisions. Experimental trampling studies on a global scale were retrieved using a systematic review methodology and synthesised using random effects meta-analysis. The relationships between vegetation recovery and each of initial vegetation resistance, trampling intensity, time for recovery, Raunkiaer life-form (perennating bud position), and habitat were tested using random effects multiple meta-regressions and subgroup analyses. The systematic search yielded 304 studies; of these, nine reported relevant randomized controlled experiments, providing 188 vegetation recovery effect sizes for analysis. The synthesis indicated there was significant heterogeneity in the impact of trampling on vegetation recovery. This was related to resistance and recovery time, and the interactions of these variables with Raunkiaer life-form, but was not strongly dependent on the intensity of the trampling experienced. The available evidence suggests that vegetation dominated by hemicryptophytes and geophytes recovers from trampling to a greater extent than vegetation dominated by other life-forms. Variation in effect within the chamaephyte, hemicryptophyte and geophyte life-form sub-groups was also explained by the initial resistance of vegetation to trampling, but not by trampling intensity. Intrinsic properties of plant communities appear to be the most important factors determining the response of vegetation to trampling disturbance. Specifically, the dominant Raunkiaer life-form of a plant community accounts for more variation in the resilience of communities to trampling than the intensity of the trampling experienced, suggesting that simple assessments based on this trait could guide decisions concerning sustainable access to natural areas. Methodological and reporting limitations must be overcome before more disparate types of evidence can be synthesised; this would enable more reliable extrapolation to non-study situations, and a more comprehensive understanding of how assessments of intrinsic plant traits can be used to underpin conservation management decisions concerning access.

Statistical analysis of individual participant data meta-analyses: a comparison of methods and recommendations for practice.

BACKGROUND: Individual participant data (IPD) meta-analyses that obtain "raw" data from studies rather than summary data typically adopt a "two-stage" approach to analysis whereby IPD within trials generate summary measures, which are combined using standard meta-analytical methods. Recently, a range of "one-stage" approaches which combine all individual participant data in a single meta-analysis have been suggested as providing a more powerful and flexible approach. However, they are more complex to implement and require statistical support. This study uses a dataset to compare "two-stage" and "one-stage" models of varying complexity, to ascertain whether results obtained from the approaches differ in a clinically meaningful way. METHODS AND FINDINGS: We included data from 24 randomised controlled trials, evaluating antiplatelet agents, for the prevention of pre-eclampsia in pregnancy. We performed two-stage and one-stage IPD meta-analyses to estimate overall treatment effect and to explore potential treatment interactions whereby particular types of women and their babies might benefit differentially from receiving antiplatelets. Two-stage and one-stage approaches gave similar results, showing a benefit of using anti-platelets (Relative risk 0.90, 95% CI 0.84 to 0.97). Neither approach suggested that any particular type of women benefited more or less from antiplatelets. There were no material differences in results between different types of one-stage model. CONCLUSIONS: For these data, two-stage and one-stage approaches to analysis produce similar results. Although one-stage models offer a flexible environment for exploring model structure and are useful where across study patterns relating to types of participant, intervention and outcome mask similar relationships within trials, the additional insights provided by their usage may not outweigh the costs of statistical support for routine application in syntheses of randomised controlled trials. Researchers considering undertaking an IPD meta-analysis should not necessarily be deterred by a perceived need for sophisticated statistical methods when combining information from large randomised trials.

Effectiveness of predator removal for enhancing bird populations.

Predation pressure on vulnerable bird species has made predator control an important issue for international nature conservation. Predator removal by culling or translocation is controversial, expensive, and time-consuming, and results are often temporary. Thus, it is important to assess its effectiveness from all available evidence. We used explicit systematic review methodology to determine the impact of predator removal on four measurable responses in birds: breeding performance (hatching success and fledging success) and population size (breeding and postbreeding). We used meta-analysis to summarize results from 83 predator removal studies from six continents. We also investigated whether characteristics of the prey, predator species, location, and study methodology explained heterogeneity in effect sizes. Removing predators increased hatching success, fledging success, and breeding populations. Removing all predator species achieved a significantly larger increase in breeding population than removing only a subset. Postbreeding population size was not improved on islands, or overall, but did increase on mainlands. Heterogeneity in effect sizes for the four population parameters was not explained by whether predators were native or introduced; prey were declining, migratory, or game species; or by the study methodology. Effect sizes for fledging success were smaller for ground-nesting birds than those that nest elsewhere, but the difference was not significant. We conclude that current evidence indicates that predator removal is an effective strategy for the conservation of vulnerable bird populations. Nevertheless, the ethical and practical problems associated with predator removal may lead managers to favor alternative, nonlethal solutions. Research is needed to provide and synthesize data to determine whether these are effective management practices for future policies on bird conservation.

Effectiveness of engineered in-stream structure mitigation measures to increase salmonid abundance: a systematic review.

Engineered in-stream structures are often installed to increase salmonid abundance, either for commercial gain in fisheries or for conservation purposes in degraded habitats. Having been in widespread use for the last 80 years, at an estimated cost of hundreds of millions of U.S. dollars each year, the effectiveness of these structures is still widely debated in the literature. Many studies of varying quality have been undertaken that attempt to address this issue, but it has proved difficult for practitioners to develop a consensus regarding the utility of these structures, despite their continued use. Systematic review methodology was used to formally synthesize empirical evidence regarding the effectiveness of engineered in-stream structures as a management tool to increase salmonid abundance. Meta-analysis shows that evidence regarding the effectiveness of in-stream devices is equivocal. Heterogeneity is significant both for population size and local habitat preference. This heterogeneity is related to stream width, with in-stream devices being less effective in larger streams. Consequently, widespread use of in-stream structures for restoration, particularly in larger streams, is not supported by the current scientific evidence base.

The effectiveness of asulam for bracken (Pteridium aquilinum) control in the United Kingdom: a meta-analysis.

Bracken (Pteridium aquilinum) is a major problem for livestock-based extensive agriculture, conservation, recreation, and game management globally. It is an invasive species often achieving dominance to the detriment of other species. Control is essential to maintain plant communities such as grassland and lowland heath or if extensive grazing by domestic stock, particularly sheep, is to be viable on upland margins. Bracken is managed primarily by herbicide application or cutting but other techniques including rolling, burning, and grazing are also utilized. Here we evaluate the evidence regarding the effectiveness of asulam for the control of bracken. Thirteen studies provided data for meta-analyses which demonstrate that application of the herbicide asulam reduces bracken abundance. Subgroup analyses indicate that the number of treatments had an important impact, with multiple follow-up treatments more effective than one or two treatments. Management practices should reflect the requirement for repeated follow-up. There is insufficient available experimental evidence for quantitative analysis of the effectiveness of other management interventions, although this results from lack of reporting in papers where cutting and comparisons of cutting and asulam application are concerned. Systematic searching and meta-analytical synthesis have effectively demonstrated the limits of current knowledge, based on recorded empirical evidence, and increasing the call for more rigorous monitoring of bracken control techniques. Lack of experimental evidence on the effectiveness of management such as rolling or grazing with hardy cattle breeds contrasts with the widespread acceptance of their use through dissemination of experience.

Guidelines for systematic review in conservation and environmental management.

An increasing number of applied disciplines are utilizing evidence-based frameworks to review and disseminate the effectiveness of management and policy interventions. The rationale is that increased accessibility of the best available evidence will provide a more efficient and less biased platform for decision making. We argue that there are significant benefits for conservation in using such a framework, but the scientific community needs to undertake and disseminate more systematic reviews before the full benefit can be realized. We devised a set of guidelines for undertaking formalized systematic review, based on a health services model. The guideline stages include planning and conducting a review, including protocol formation, search strategy, data inclusion, data extraction, and analysis. Review dissemination is addressed in terms of current developments and future plans for a Web-based open-access library. By the use of case studies we highlight critical modifications to guidelines for protocol formulation, data-quality assessment, data extraction, and data synthesis for conservation and environmental management. Ecological data presented significant but soluble challenges for the systematic review process, particularly in terms of the quantity, accessibility, and diverse quality of available data. In the field of conservation and environmental management there needs to be further engagement of scientists and practitioners to develop and take ownership of an evidence-based framework.

Effectiveness of management interventions to control invasion by Rhododendron ponticum.

Rhododendron ponticum is an invasive species in many countries, including the United Kingdom, Ireland, Belgium, and France. It poses a serious threat to native flora and fauna, as it is capable of altering entire seminatural communities through its vigorous spread. Control is essential if the conservation value of some communities, such as oak woodland and lowland heath, are to be successfully maintained. Commonly used interventions are herbicide application, herbicide application postcut, and cutting (manual or mechanical) alone. Various techniques have been developed to apply these interventions, but often retreatment of the area is required, increasing the cost of control. Here, we evaluate the effectiveness of some commonly used interventions for R. ponticum control using a systematic review methodology. Eleven studies provided data for statistical analysis. Meta-analyses of captured data show that postcut application of the herbicide Glyphosate or applying the herbicides Metsulfuron-methyl or Imazapyr (no cut) can effectively reduce a R. ponticum stand. There is insufficient available experimental evidence for effectiveness of any other intervention. The systematic review process has demonstrated the lack of replicated studies with controls or long-term monitoring and increases the call for more rigorous monitoring of all conservation management interventions. The quality of experimental evidence of the effectiveness of some interventions contrasts with the acceptance of their use through dissemination of experience. The collection and objective review of experience will require active collaboration of organizations concerned with R. ponticum control.