Reviews in environmental health: How systematic are they?

BACKGROUND: Synthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review. OBJECTIVES: To appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health. METHODS: We conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews. RESULTS: We identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement. DISCUSSION: Systematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.

Exposure to formaldehyde and asthma outcomes: A systematic review, meta-analysis, and economic assessment.

BACKGROUND: Every major federal regulation in the United States requires an economic analysis estimating its benefits and costs. Benefit-cost analyses related to regulations on formaldehyde exposure have not included asthma in part due to lack of clarity in the strength of the evidence. OBJECTIVES: 1) To conduct a systematic review of evidence regarding human exposure to formaldehyde and diagnosis, signs, symptoms, exacerbations, or other measures of asthma in humans; and 2) quantify the annual economic benefit for decreases in formaldehyde exposure. METHODS: We developed and registered a protocol in PROSPERO (Record ID #38766, CRD 42016038766). We conducted a comprehensive search of articles published up to April 1, 2020. We evaluated potential risk of bias for included studies, identified a subset of studies to combine in a meta-analysis, and rated the overall quality and strength of the evidence. We quantified economics benefit to children from a decrease in formaldehyde exposure using assumptions consistent with EPA's proposed formaldehyde rule. RESULTS: We screened 4,821 total references and identified 150 human studies that met inclusion criteria; of these, we focused on 90 studies reporting asthma status of all participants with quantified measures of formaldehyde directly relevant to our study question. Ten studies were combinable in a meta-analysis for childhood asthma diagnosis and five combinable for exacerbation of childhood asthma (wheezing and shortness of breath). Studies had low to probably-low risk of bias across most domains. A 10-µg/m3 increase in formaldehyde exposure was associated with increased childhood asthma diagnosis (OR = 1.20, 95% CI: [1.02, 1.41]). We also found a positive association with exacerbation of childhood asthma (OR = 1.08, 95% CI: [0.92, 1.28]). The overall quality and strength of the evidence was rated as "moderate" quality and "sufficient" for asthma diagnosis and asthma symptom exacerbation in both children and adults. We estimated that EPA's proposed rule on pressed wood products would result in 2,805 fewer asthma cases and total economic benefit of $210 million annually. CONCLUSION: We concluded there was "sufficient evidence of toxicity" for associations between exposure to formaldehyde and asthma diagnosis and asthma symptoms in both children and adults. Our research documented that when exposures are ubiquitous, excluding health outcomes from benefit-cost analysis can underestimate the true benefits to health from environmental regulations.

Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis.

BACKGROUND: In the United States, one in six children are affected by neurodevelopmental disorders, and polybrominated diphenyl ethers (PBDEs) in flame-retardant chemicals are measured ubiquitously in children. OBJECTIVE: We conducted a systematic a systematic review regarding developmental exposure to PBDEs and intelligence or Attention Deficit/Hyperactivity Disorder (ADHD) and attention-related behavioral conditions in humans. METHODS: We searched articles published up to 26 September 2016, and included original studies that quantified exposures to PBDEs incurred any time in proximity to conception or during in utero, perinatal, or childhood time periods. We evaluated the risk of bias of individual studies and the overall quality and strength of the evidence according to the Navigation Guide systematic review methodology. We established criteria in advance to identify studies that could be combined using random effects meta-analyses (DerSimonian-Laird method). RESULTS: Fifteen studies met the inclusion criteria; 10 studies met the criteria for intelligence and nine for attention-related problems. We rated studies generally with "low" to "probably low" risk of bias and rated the overall body of evidence as "moderate" quality with "sufficient" evidence for an association between Intelligence Quotient (IQ) and PBDEs. Our meta-analysis of four studies estimated a 10-fold increase (in other words, times 10) in PBDE exposure associated with a decrement of 3.70 IQ points (95% confidence interval: 0.83, 6.56). We concluded the body of evidence was of "moderate" quality for ADHD with "limited" evidence for an association with PBDEs, based on the heterogeneity of association estimates reported by a small number of studies and the fact that chance, bias, and confounding could not be ruled out with reasonable confidence. CONCLUSION: We concluded there was sufficient evidence supporting an association between developmental PBDE exposure and reduced IQ. Preventing developmental exposure to PBDEs could help prevent loss of human intelligence. https://doi.org/10.1289/EHP1632.

A Systematic Review and Meta-Analysis of Multiple Airborne Pollutants and Autism Spectrum Disorder.

BACKGROUND: Exposure to ambient air pollution is widespread and may be detrimental to human brain development and a potential risk factor for Autism Spectrum Disorder (ASD). We conducted a systematic review of the human evidence on the relationship between ASD and exposure to all airborne pollutants, including particulate matter air pollutants and others (e.g. pesticides and metals). OBJECTIVE: To answer the question: "is developmental exposure to air pollution associated with ASD?" METHODS: We conducted a comprehensive search of the literature, identified relevant studies using inclusion/exclusion criteria pre-specified in our protocol (registered in PROSPERO, CRD # 42015017890), evaluated the potential risk of bias for each included study and identified an appropriate subset of studies to combine in a meta-analysis. We then rated the overall quality and strength of the evidence collectively across all air pollutants. RESULTS: Of 1,158 total references identified, 23 human studies met our inclusion criteria (17 case-control, 4 ecological, 2 cohort). Risk of bias was generally low across studies for most domains; study limitations were related to potential confounding and accuracy of exposure assessment methods. We rated the quality of the body of evidence across all air pollutants as "moderate." From our meta-analysis, we found statistically significant summary odds ratios (ORs) of 1.07 (95% CI: 1.06, 1.08) per 10-µg/m3 increase in PM10 exposure (n = 6 studies) and 2.32 (95% CI: 2.15, 2.51) per 10-µg/m3 increase in PM2.5 exposure (n = 3 studies). For pollutants not included in a meta-analysis, we collectively evaluated evidence from each study in rating the strength and quality of overall evidence considering factors such as inconsistency, imprecision, and evidence of dose-response. All included studies generally showed increased risk of ASD with increasing exposure to air pollution, although not consistently across all chemical components. CONCLUSION: After considering strengths and limitations of the body of research, we concluded that there is "limited evidence of toxicity" for the association between early life exposure to air pollution as a whole and diagnosis of ASD. The strongest evidence was between prenatal exposure to particulate matter and ASD. However, the small number of studies in the meta-analysis and unexplained statistical heterogeneity across the individual study estimates means that the effect could be larger or smaller (including not significant) than these studies estimate. Our research supports the need for health protective public policy to reduce exposures to harmful airborne contaminants among pregnant women and children and suggests opportunities for optimizing future research.