Addressing systemic problems with exposure assessments to protect the public's health.

BACKGROUND: Understanding, characterizing, and quantifying human exposures to environmental chemicals is critical to protect public health. Exposure assessments are key to determining risks to the general population and for specific subpopulations given that exposures differ between groups. Exposure data are also important for understanding where interventions, including public policies, should be targeted and the extent to which interventions have been successful. In this review, we aim to show how inadequacies in exposure assessments conducted by polluting industries or regulatory agencies have led to downplaying or disregarding exposure concerns raised by communities; that underestimates of exposure can lead regulatory agencies to conclude that unacceptable risks are, instead, acceptable, allowing pollutants to go unregulated; and that researchers, risk assessors, and policy makers need to better understand the issues that have affected exposure assessments and how appropriate use of exposure data can contribute to health-protective decisions. METHODS: We describe current approaches used by regulatory agencies to estimate human exposures to environmental chemicals, including approaches to address limitations in exposure data. We then illustrate how some exposure assessments have been used to reach flawed conclusions about environmental chemicals and make recommendations for improvements. RESULTS: Exposure data are important for communities, public health advocates, scientists, policy makers, and other groups to understand the extent of environmental exposures in diverse populations. We identify four areas where exposure assessments need to be improved due to systemic sources of error or uncertainty in exposure assessments and illustrate these areas with examples. These include: (1) an inability of regulatory agencies to keep pace with the increasing number of chemicals registered for use or assess their exposures, as well as complications added by use of 'confidential business information' which reduce available exposure data; (2) the failure to keep assessments up-to-date; (3) how inadequate assumptions about human behaviors and co-exposures contribute to underestimates of exposure; and (4) that insufficient models of toxicokinetics similarly affect exposure estimates. CONCLUSION: We identified key issues that impact capacity to conduct scientifically robust exposure assessments. These issues must be addressed with scientific or policy approaches to improve estimates of exposure and protect public health.

Application of probabilistic methods to address variability and uncertainty in estimating risks for non-cancer health effects.

Human health risk assessment currently uses the reference dose or reference concentration (RfD, RfC) approach to describe the level of exposure to chemical hazards without appreciable risk for non-cancer health effects in people. However, this "bright line" approach assumes that there is minimal risk below the RfD/RfC with some undefined level of increased risk at exposures above the RfD/RfC and has limited utility for decision-making. Rather than this dichotomous approach, non-cancer risk assessment can benefit from incorporating probabilistic methods to estimate the amount of risk across a wide range of exposures and define a risk-specific dose. We identify and review existing approaches for conducting probabilistic non-cancer risk assessments. Using perchloroethylene (PCE), a priority chemical for the U.S. Environmental Protection Agency under the Toxic Substances Control Act, we calculate risk-specific doses for the effects on cognitive deficits using probabilistic risk assessment approaches. Our probabilistic risk assessment shows that chronic exposure to 0.004 ppm PCE is associated with approximately 1-in-1,000 risk for a 5% reduced performance on the Wechsler Memory Scale Visual Reproduction subtest with 95% confidence. This exposure level associated with a 1-in-1000 risk for non-cancer neurocognitive deficits is lower than the current RfC for PCE of 0.0059 ppm, which is based on standard point of departure and uncertainty factor approaches for the same neurotoxic effects in occupationally exposed adults. We found that the population-level risk of cognitive deficit (indicating central nervous system dysfunction) is estimated to be greater than the cancer risk level of 1-in-100,000 at a similar chronic exposure level. The extension of toxicological endpoints to more clinically relevant endpoints, along with consideration of magnitude and severity of effect, will help in the selection of acceptable risk targets for non-cancer effects. We find that probabilistic approaches can 1) provide greater context to existing RfDs and RfCs by describing the probability of effect across a range of exposure levels including the RfD/RfC in a diverse population for a given magnitude of effect and confidence level, 2) relate effects of chemical exposures to clinical disease risk so that the resulting risk assessments can better inform decision-makers and benefit-cost analysis, and 3) better reflect the underlying biology and uncertainties of population risks.

A science-based agenda for health-protective chemical assessments and decisions: overview and consensus statement.

The manufacture and production of industrial chemicals continues to increase, with hundreds of thousands of chemicals and chemical mixtures used worldwide, leading to widespread population exposures and resultant health impacts. Low-wealth communities and communities of color often bear disproportionate burdens of exposure and impact; all compounded by regulatory delays to the detriment of public health. Multiple authoritative bodies and scientific consensus groups have called for actions to prevent harmful exposures via improved policy approaches. We worked across multiple disciplines to develop consensus recommendations for health-protective, scientific approaches to reduce harmful chemical exposures, which can be applied to current US policies governing industrial chemicals and environmental pollutants. This consensus identifies five principles and scientific recommendations for improving how agencies like the US Environmental Protection Agency (EPA) approach and conduct hazard and risk assessment and risk management analyses: (1) the financial burden of data generation for any given chemical on (or to be introduced to) the market should be on the chemical producers that benefit from their production and use; (2) lack of data does not equate to lack of hazard, exposure, or risk; (3) populations at greater risk, including those that are more susceptible or more highly exposed, must be better identified and protected to account for their real-world risks; (4) hazard and risk assessments should not assume existence of a "safe" or "no-risk" level of chemical exposure in the diverse general population; and (5) hazard and risk assessments must evaluate and account for financial conflicts of interest in the body of evidence. While many of these recommendations focus specifically on the EPA, they are general principles for environmental health that could be adopted by any agency or entity engaged in exposure, hazard, and risk assessment. We also detail recommendations for four priority areas in companion papers (exposure assessment methods, human variability assessment, methods for quantifying non-cancer health outcomes, and a framework for defining chemical classes). These recommendations constitute key steps for improved evidence-based environmental health decision-making and public health protection.

Toxic Substances Control Act (TSCA) Implementation: How the Amended Law Has Failed to Protect Vulnerable Populations from Toxic Chemicals in the United States.

Exposures to industrial chemicals are widespread and can increase the risk of adverse health effects such as cancer, developmental disorders, respiratory effects, diabetes, and reproductive problems. The amended Toxic Substances Control Act (amended TSCA) requires the U.S. Environmental Protection Agency (EPA) to evaluate risks of chemicals in commerce, account for risk to potentially exposed and susceptible populations, and mitigate risks for chemicals determined to pose an unreasonable risk to human health and the environment. This analysis compares EPA's first 10 chemical risk evaluations under amended TSCA to best scientific practices for conducting risk assessments. We find EPA's risk evaluations underestimated human health risks of chemical exposures by excluding conditions of use and exposure pathways; not considering aggregate exposure and cumulative risk; not identifying all potentially exposed or susceptible subpopulations, and not quantifying differences in risk for susceptible groups; not addressing data gaps; and using flawed systematic review approaches to identify and evaluate the relevant evidence. We present specific recommendations for improving the implementation of amended TSCA using the best available science to ensure equitable, socially just safeguards to public health. Failing to remedy these shortcomings will result in continued systematic underestimation of risk for all chemicals evaluated under amended TSCA.

Conducting evaluations of evidence that are transparent, timely and can lead to health-protective actions.

BACKGROUND: In February 2021, over one hundred scientists and policy experts participated in a web-based Workshop to discuss the ways that divergent evaluations of evidence and scientific uncertainties are used to delay timely protection of human health and the environment from exposures to hazardous agents. The Workshop arose from a previous workshop organized by the European Environment Agency (EEA) in 2008 and which also drew on case studies from the EEA reports on 'Late Lessons from Early Warnings' (2001, 2013). These reports documented dozens of hazardous agents including many chemicals, for which risk reduction measures were delayed for decades after scientists and others had issued early and later warnings about the harm likely to be caused by those agents. RESULTS: Workshop participants used recent case studies including Perfluorooctanoic acid (PFOA), Extremely Low Frequency - Electrical Magnetic Fields (ELF-EMF fields), glyphosate, and Bisphenol A (BPA) to explore myriad reasons for divergent outcomes of evaluations, which has led to delayed and inadequate protection of the public's health. Strategies to overcome these barriers must, therefore, at a minimum include approaches that 1) Make better use of existing data and information, 2) Ensure timeliness, 3) Increase transparency, consistency and minimize bias in evidence evaluations, and 4) Minimize the influence of financial conflicts of interest. CONCLUSION: The recommendations should enhance the production of "actionable evidence," that is, reliable evaluations of the scientific evidence to support timely actions to protect health and environments from exposures to hazardous agents. The recommendations are applicable to policy and regulatory settings at the local, state, federal and international levels.

Reporting bias in the literature on the associations of health-related behaviors and statins with cardiovascular disease and all-cause mortality.

Reporting bias in the literature occurs when there is selective revealing or suppression of results, influenced by the direction of findings. We assessed the risk of reporting bias in the epidemiological literature on health-related behavior (tobacco, alcohol, diet, physical activity, and sedentary behavior) and cardiovascular disease mortality and all-cause mortality and provided a comparative assessment of reporting bias between health-related behavior and statin (in primary prevention) meta-analyses. We searched Medline, Embase, Cochrane Methodology Register Database, and Web of Science for systematic reviews synthesizing the associations of health-related behavior and statins with cardiovascular disease mortality and all-cause mortality published between 2010 and 2016. Risk of bias in systematic reviews was assessed using the ROBIS tool. Reporting bias in the literature was evaluated via small-study effect and excess significance tests. We included 49 systematic reviews in our study. The majority of these reviews exhibited a high overall risk of bias, with a higher extent in health-related behavior reviews, relative to statins. We reperformed 111 meta-analyses conducted across these reviews, of which 65% had statistically significant results (P < 0.05). Around 22% of health-related behavior meta-analyses showed small-study effect, as compared to none of statin meta-analyses. Physical activity and the smoking research areas had more than 40% of meta-analyses with small-study effect. We found evidence of excess significance in 26% of health-related behavior meta-analyses, as compared to none of statin meta-analyses. Half of the meta-analyses from physical activity, 26% from diet, 18% from sedentary behavior, 14% for smoking, and 12% from alcohol showed evidence of excess significance bias. These biases may be distorting the body of evidence available by providing inaccurate estimates of preventive effects on cardiovascular and all-cause mortality.

Evidence-to-decision frameworks: a review and analysis to inform decision-making for environmental health interventions.

BACKGROUND: Evidence-to-decision (EtD) frameworks provide a structured and transparent approach for groups of experts to use when formulating recommendations or making decisions. While extensively used for clinical and public health recommendations, EtD frameworks are not in widespread use in environmental health. This review sought to identify, compare and contrast key EtD frameworks for decisions on interventions used in clinical medicine, public health or environmental health. This information can be used to develop an EtD framework suitable for formulating recommendations for interventions in environmental health. METHODS: We identified a convenience sample of EtD frameworks used by a range of organizations. We searched Medline for systematic reviews of frameworks. We summarized the decision criteria in the selected frameworks and reviews in a qualitative manner. FINDINGS: Fourteen organizations provided 18 EtD frameworks; most frameworks focused on clinical medicine or public health interventions; four focused on environmental health and three on economic considerations. Harms of interventions were examined in all frameworks and benefits in all but one. Other criteria included certainty of the body of evidence (15 frameworks), resource considerations (15), feasibility (13), equity (12), values (11), acceptability (11), and human rights (2). There was variation in how specific criteria were defined. The five identified systematic reviews reported a similar spectrum of EtD criteria. INTERPRETATION: The EtD frameworks examined encompassed similar criteria, with tailoring to specific audience needs. Existing frameworks are a useful starting point for development of one tailored to decision-making in environmental health. FUNDER: JPB Foundation.

Study sponsorship and the nutrition research agenda: analysis of cohort studies examining the association between nutrition and obesity.

OBJECTIVE: To categorize the research topics covered by a sample of cohort studies exploring the association between nutrition and obesity; to describe their funding sources; and to explore the association between funding sources and research topics. DESIGN: Cross-sectional study. SUBJECTS: Cohort studies retrieved from MEDLINE and PubMed published between 2010 and 2016. RESULTS: One hundred and twenty-one studies were included. Funding source and conflicts of interest were disclosed in 95·0 and 90·1 % of the studies, respectively. Food industry sponsorship was disclosed in 8·3 % of the studies. Half of the studies analysed the consumption of a single food or food groups, 18·2 % included an analysis of dietary patterns and 17·4 % focused on specific nutrients. Highly processed foods were considered in 48·8 % of the studies and 27·3 % considered dietary behaviours (e.g. eating away from home). No statistically significant differences in research topics were observed between industry- and non-industry-funded studies. CONCLUSIONS: Cohort studies focused on more complex exposures (e.g. food or dietary patterns) rather than single nutrients. No significant differences in the research agenda by funding sources were observed. The analysis was limited by the low proportion of studies with disclosed food industry sponsorship.

Study sponsorship and the nutrition research agenda: analysis of randomized controlled trials included in systematic reviews of nutrition interventions to address obesity.

OBJECTIVE: To categorize the research topics covered by a sample of randomized controlled trials (RCT) included in systematic reviews of nutrition interventions to address obesity; to describe their funding sources; and to explore the association between funding sources and nutrition research topics. DESIGN: Cross-sectional study. SUBJECTS: RCT included in Cochrane Reviews of nutrition interventions to address obesity and/or overweight. RESULTS: Two hundred and thirteen RCT from seventeen Cochrane Reviews were included. Funding source and authors' conflicts of interest were disclosed in 82·6 and 29·6 % of the studies, respectively. RCT were more likely to test an intervention to manipulate nutrients in the context of reduced energy intake (44·2 % of studies) than food-level (11·3 %) and dietary pattern-level (0·9 %) interventions. Most of the food industry-sponsored studies focused on interventions involving manipulations of specific nutrients (66·7 %). Only 33·1 % of the industry-funded studies addressed dietary behaviours compared with 66·9 % of the non-industry-funded ones (P=0·002). The level of food processing was poorly considered across all funding sources. CONCLUSIONS: The predominance of RCT examining nutrient-specific questions could limit the public health relevance of rigorous evidence available for systematic reviews and dietary guidelines.

Authors' rebuttal to Integrated Risk Information System (IRIS) response to "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools".

This letter responds to the US Environmental Protection Agency's Integrated Risk Information System (IRIS) program letter by Radke et al. (2021) that was published in response to the application of the IRIS risk of bias tool in our recent study "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools." Their letter stated that we misrepresented the IRIS approach. Here, we respond to their three points raised and how we did not misrepresent their tool and also identified areas for improvement: (1) why it should be expected that different reviewers could reach different conclusions with the IRIS tool, as ratings are subject to reviewer judgment; (2) why our interpretation that "low" or "uninformative" studies could be excluded from a body of evidence was reasonable; and (3) why we believe the use of a rating system that generates an overall rating based on an individual domain or a combination of identified deficiencies essentially acts as a score and assumes that we know empirically how much each risk of bias domain should contribute to the overall rating for that study. We have elaborated on these points in our letter.

Identifying environmental factors that influence immune response to SARS-CoV-2: Systematic evidence map protocol.

BACKGROUND: Widespread environmental contamination can directly interact with human immune system functions. Environmental effects on the immune system may influence human susceptibility to respiratory infections as well as the severity of infectious diseases, such as the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Furthermore, the efficacy of vaccines to respiratory diseases may be impacted by environmental exposures through immune perturbations. Given the quick pace of research about COVID-19 and associated risk factors, it is critical to identify and curate the streams of evidence quickly and effectively. OBJECTIVE: We developed this systematic evidence map protocol to identify and organize existing human and animal literature on high-priority environmental chemical classes (Per- and polyfluoroalkyl substances, pesticides, phthalates, quaternary ammonium compounds, and air pollutants) and their potential to influence three key outcomes: (1) susceptibility to respiratory infection, including SARS-CoV-2 (2) severity of the resultant disease progression, and (3) impact on vaccine efficacy. The result of this project will be an online, interactive database which will show what evidence is currently available between involuntary exposures to select environmental chemicals and immune health effects, data gaps that require further research, and data rich areas that may support further analysis. SEARCH AND STUDY ELIGIBILITY: We will search PubMed for epidemiological or toxicological literature on select toxicants from each of the chemical classes and each of the three outcomes listed above. STUDY APPRAISAL AND SYNTHESIS OF METHODS: For each study, two independent reviewers will conduct title and abstract screening as well as full text review for data extraction of study characteristics. Study quality will not be evaluated in this evidence mapping. The main findings from the systematic evidence map will be visualized using a publicly available and interactive database hosted on Tableau Public.

The effect of occupational exposure to welding fumes on trachea, bronchus and lung cancer: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are the producers of the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). Welding fumes have been classified as carcinogenic to humans (Group 1) by the WHO International Agency for Research on Cancer (IARC) in IARC Monograph 118; this assessment found sufficient evidence from studies in humans that welding fumes are a cause of lung cancer. In this article, we present a systematic review and meta-analysis of parameters for estimating the number of deaths and disability-adjusted life years from trachea, bronchus, and lung cancer attributable to occupational exposure to welding fumes, to inform the development of WHO/ILO Joint Estimates on this burden of disease (if considered feasible). OBJECTIVES: We aimed to systematically review and meta-analyse estimates of the effect of any (or high) occupational exposure to welding fumes, compared with no (or low) occupational exposure to welding fumes, on trachea, bronchus, and lung cancer (three outcomes: prevalence, incidence, and mortality). DATA SOURCES: We developed and published a protocol, applying the Navigation Guide as an organizing systematic review framework where feasible. We searched electronic databases for potentially relevant records from published and unpublished studies, including Medline, EMBASE, Web of Science, CENTRAL and CISDOC. We also searched grey literature databases, Internet search engines, and organizational websites; hand-searched reference lists of previous systematic reviews; and consulted additional experts. STUDY ELIGIBILITY AND CRITERIA: We included working-age (≥15 years) workers in the formal and informal economy in any Member State of WHO and/or ILO but excluded children (<15 years) and unpaid domestic workers. We included randomized controlled trials, cohort studies, case-control studies, and other non-randomized intervention studies with an estimate of the effect of any (or high) occupational exposure to welding fumes, compared with occupational exposure to no (or low) welding fumes, on trachea, bronchus, and lung cancer (prevalence, incidence, and mortality). STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors independently screened titles and abstracts against the eligibility criteria at a first review stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. If studies reported odds ratios, these were converted to risk ratios (RRs). We combined all RRs using random-effects meta-analysis. Two or more review authors assessed the risk of bias, quality of evidence, and strength of evidence, using the Navigation Guide tools and approaches adapted to this project. Subgroup (e.g., by WHO region and sex) and sensitivity analyses (e.g., studies judged to be of "high"/"probably high" risk of bias compared with "low"/"probably low" risk of bias) were conducted. RESULTS: Forty-one records from 40 studies (29 case control studies and 11 cohort studies) met the inclusion criteria, comprising over 1,265,512 participants (≥22,761 females) in 21 countries in three WHO regions (Region of the Americas, European Region, and Western Pacific Region). The exposure and outcome were generally assessed by job title or self-report, and medical or administrative records, respectively. Across included studies, risk of bias was overall generally probably low/low, with risk judged high or probably high for several studies in the domains for misclassification bias and confounding. Our search identified no evidence on the outcome of having trachea, bronchus, and lung cancer (prevalence). Compared with no (or low) occupational exposure to welding fumes, any (or high) occupational exposure to welding fumes increased the risk of acquiring trachea, bronchus, and lung cancer (incidence) by an estimated 48 % (RR 1.48, 95 % confidence interval [CI] 1.29-1.70, 23 studies, 57,931 participants, I2 24 %; moderate quality of evidence). Compared with no (or low) occupational exposure to welding fumes, any (or high) occupational exposure to welding fumes increased the risk dying from trachea, bronchus, and lung cancer (mortality) by an estimated 27 % (RR 1.27, 95 % CI 1.04-1.56, 3 studies, 8,686 participants, I2 0 %; low quality of evidence). Our subgroup analyses found no evidence for difference by WHO region and sex. Sensitivity analyses supported the main analyses. CONCLUSIONS: Overall, for incidence and mortality of trachea, bronchus, and lung cancer, we judged the existing body of evidence for human data as "sufficient evidence of harmfulness" and "limited evidence of harmfulness", respectively. Occupational exposure to welding fumes increased the risk of acquiring and dying from trachea, bronchus, and lung cancer. Producing estimates for the burden of trachea, bronchus, and lung cancer attributable to any (or high) occupational exposure to welding fumes appears evidence-based, and the pooled effect estimates presented in this systematic review could be used as input data for the WHO/ILO Joint Estimates. PROTOCOL IDENTIFIER: https://doi.org/10.1016/j.envint.2020.106089.

Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Quality of Evidence in Studies estimating Prevalence of Exposure to Occupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further. METHODS: Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (Pi) between individual raters and rater teams for the downgrade domains. Pi ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews. RESULTS: We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range Pi: 0.36-1.00) and rater teams (0.20-1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings. CONCLUSIONS: We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area.

Reviews in environmental health: How systematic are they?

BACKGROUND: Synthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review. OBJECTIVES: To appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health. METHODS: We conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews. RESULTS: We identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement. DISCUSSION: Systematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.

A Protocol for the Use of Case Reports/Studies and Case Series in Systematic Reviews for Clinical Toxicology.

Introduction: Systematic reviews are routinely used to synthesize current science and evaluate the evidential strength and quality of resulting recommendations. For specific events, such as rare acute poisonings or preliminary reports of new drugs, we posit that case reports/studies and case series (human subjects research with no control group) may provide important evidence for systematic reviews. Our aim, therefore, is to present a protocol that uses rigorous selection criteria, to distinguish high quality case reports/studies and case series for inclusion in systematic reviews. Methods: This protocol will adapt the existing Navigation Guide methodology for specific inclusion of case studies. The usual procedure for systematic reviews will be followed. Case reports/studies and case series will be specified in the search strategy and included in separate sections. Data from these sources will be extracted and where possible, quantitatively synthesized. Criteria for integrating cases reports/studies and case series into the overall body of evidence are that these studies will need to be well-documented, scientifically rigorous, and follow ethical practices. The instructions and standards for evaluating risk of bias will be based on the Navigation Guide. The risk of bias, quality of evidence and the strength of recommendations will be assessed by two independent review teams that are blinded to each other. Conclusion: This is a protocol specified for systematic reviews that use case reports/studies and case series to evaluate the quality of evidence and strength of recommendations in disciplines like clinical toxicology, where case reports/studies are the norm.

"It's Not Smooth Sailing": Bridging the Gap Between Methods and Content Expertise in Public Health Guideline Development.

BACKGROUND: The development of reliable, high quality health-related guidelines depends on explicit and transparent processes, methods aimed at minimising risks of bias and the inclusion of all relevant expertise and perspectives. While the methodological aspects of guidelines have been a focus to improve their quality, less is known about the social processes involved, for example, how guideline group members interact and communicate with one another, and how the evidence is considered in informing recommendations. With this in in mind, we aimed to empirically examine the perspectives and experiences of the key participants involved in developing public health guidelines for the Australian National Health and Medical Research Council (NHMRC). DESIGN: This study was conducted using constructivist grounded theory as described by Charmaz, which informed our sampling, data collection, coding and analysis of interviews with key participants involved in developing public health guidelines. SETTING: Australian public health guidelines commissioned by the NHMRC. PARTICIPANTS: Twenty experts that were involved in Australian NHMRC public health guideline development, including working committee members with content topic expertise (n=16) and members of evidence review groups responsible for evaluating the evidence (n=4). RESULTS: Public health guideline development in Australia is a divided process. The division is driven by 3 related factors: the divergent disciplinary background and expertise that each group brings to the process; the methodological limitations of the framework, inherited from clinical medicine, that is used to assess the evidence; and barriers to communication between content experts and evidence reviewers around respective roles and methodological limitations. CONCLUSION: Our findings suggest several improvements for a more functional and unified guideline development process: greater education of the working committee on the methodological process employed to evaluate evidence, improved communication on the role of the evidence review groups and better facilitation of the process so that the evidence review groups feel their contribution is valued.

Association of food industry ties with findings of studies examining the effect of dairy food intake on cardiovascular disease and mortality: systematic review and meta-analysis.

OBJECTIVE: To determine if the association of dairy foods with cardiovascular disease (CVD) outcomes differs between studies with food industry ties versus those without industry ties. To determine whether studies with or without industry ties differ in their risk of bias. ELIGIBILITY CRITERIA: We included cohort and case-control studies that estimated the association of dairy foods with CVD outcomes in healthy adults. INFORMATION SOURCES: We searched eight databases on 1 February 2019 from 2000 to 2019 and hand searched reference lists. RISK OF BIAS: We used the Risk of Bias in Non-Randomised Studies-of Exposure tool. INCLUDED STUDIES: 43 studies (3 case-controls, 40 cohorts). SYNTHESIS OF RESULTS: There was no clear evidence of an association between studies with industry ties (1/14) versus no industry ties (8/29) and the reporting of favourable results, risk ratio (RR)=0.26 (95% CI 0.04 to 1.87; n=43 studies) and studies with industry ties (4/14) versus no industry ties (11/29) and favourable conclusions, RR=0.75 (95% CI 0.29 to 1.95; n=43). Studies with industry sponsorship, (HR=0.78; n=3 studies) showed a decreased magnitude of risk of CVD outcomes compared with studies with no industry sponsorship (HR=0.97; n=18) (ratio of HRs 0.80 (95% CI 0.66 to 0.97); p=0.03). STRENGTHS AND LIMITATIONS OF EVIDENCE: Every study had an overall high risk of bias rating; this was primarily due to confounding. INTERPRETATION: There was no clear evidence of an association between studies with food industry ties and the reporting of favourable results and conclusions compared with studies without industry ties. The statistically significant difference in the magnitude of effects identified in industry-sponsored studies compared with non-industry-sponsored studies, however, is important in quantifying industry influence on studies included in dietary guidelines. PROSPERO REGISTRATION NUMBER: CRD42019129659.

Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools.

BACKGROUND: Systematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program's Office of Health Assessment and Translation (OHAT), the US Environmental Protection Agency's (EPA) Integrated Risk Information System (IRIS), and by the US EPA under the Toxic Substances Control Act (TSCA), including the approach to assess risk of bias (ROB), one of the most vital steps which is used to evaluate internal validity of the studies. Our objective was to compare the performance of three tools (OHAT, IRIS, TSCA) in assessing ROB. METHODS: We selected a systematic review on polybrominated diphenyl ethers and intelligence quotient and/or attention deficit hyperactivity disorder because it had been endorsed by the National Academy of Sciences. Two reviewers followed verbatim instructions from the tools and independently applied each tool to assess ROB in 15 studies previously identified. We documented the time to apply each tool and the impact the ROB ratings for each tool had on the final rating of the quality of the overall body of evidence. RESULTS: The time to complete the ROB assessments varied widely (mean = 20, 32, and 40 min per study for the OHAT, IRIS, and TSCA tools, respectively). All studies were rated overall "low" or "uninformative" using IRIS, due to "deficient" or "critically deficient" ratings in one or two domains. Similarly, all studies were rated "unacceptable" using the TSCA tool because of one "unacceptable" rating in a metric related to statistical power. Approximately half of the studies had "low" or "probably low ROB" ratings across all domains with the OHAT and Navigation Guide tools. CONCLUSIONS: Tools that use overall ROB or study quality ratings, such as IRIS and TSCA, may reduce the available evidence to assess the harms of environmental exposures by erroneously excluding studies, which leads to inaccurate conclusions about the quality of the body of evidence. We recommend using ROB tools that circumvents these issues, such as OHAT and Navigation Guide. SYSTEMATIC REVIEW REGISTRATION: This review has not been registered as it is not a systematic review.