Developing a database of systematic reviews of animal studies.

Systematic reviews (SRs) are common practice in clinical and public health research, but less common in non-human animal research. Systematic reviews of animal studies can be valuable to inform clinical research, to evaluate the need for further animal experiments on a given topic, and to assess the hazard of an environmental exposure in the evaluation of toxicological studies. In the last 10 years, there has been an increase in the number of SRs of animal research, as well as several publications with detailed guidance on how to perform high-quality systematic reviews of experimental animal studies. In order to evaluate current analytical approaches used in SRs of animal studies, easily identify all systematic reviews on a specific topic, and subsequently the original animal studies and their results and promote awareness and understanding of these emerging approaches, we compiled a database of SRs of animal studies. The database was developed using a rigorous, systematic approach and covers a broad range of research fields: preclinical research, toxicology, environmental health, and veterinary medicine. The database currently includes 3113 SRs of animal studies (search date June 2019). In addition to bibliographical information, data on whether or not a risk of bias assessment and meta-analysis were conducted were extracted. For future users, the search features of the database provide users with a platform to identify and select SRs with a particular characteristic for export to Microsoft Word or Microsoft Excel. From there, users may perform additional data extraction to meet their research needs. The database is freely available at www.Mendeley.com (link). The database provides methodologists a comprehensive source that can be used to explore and advance the current methodology applied to SRs of animal studies, and can help researchers to easily identify all systematic reviews on a specific topic, and subsequently the original animal studies and their results and avoid duplication and unnecessary animal research.

Integrating Personal Air Sensor and GPS to Determine Microenvironment-Specific Exposures to Volatile Organic Compounds.

Personal exposure to volatile organic compounds (VOCs) from indoor sources including consumer products is an understudied public health concern. To develop and evaluate methods for monitoring personal VOC exposures, we performed a pilot study and examined time-resolved sensor-based measurements of geocoded total VOC (TVOC) exposures across individuals and microenvironments (MEs). We integrated continuous (1 min) data from a personal TVOC sensor and a global positioning system (GPS) logger, with a GPS-based ME classification model, to determine TVOC exposures in four MEs, including indoors at home (Home-In), indoors at other buildings (Other-In), inside vehicles (In-Vehicle), and outdoors (Out), across 45 participant-days for five participants. To help identify places with large emission sources, we identified high-exposure events (HEEs; TVOC > 500 ppb) using geocoded TVOC time-course data overlaid on Google Earth maps. Across the 45 participant-days, the MEs ranked from highest to lowest median TVOC were: Home-In (165 ppb), Other-In (86 ppb), In-Vehicle (52 ppb), and Out (46 ppb). For the two participants living in single-family houses with attached garages, the median exposures for Home-In were substantially higher (209, 416 ppb) than the three participant homes without attached garages: one living in a single-family house (129 ppb), and two living in apartments (38, 60 ppb). The daily average Home-In exposures exceeded the estimated Leadership in Energy and Environmental Design (LEED) building guideline of 108 ppb for 60% of the participant-days. We identified 94 HEEs across all participant-days, and 67% of the corresponding peak levels exceeded 1000 ppb. The MEs ranked from the highest to the lowest number of HEEs were: Home-In (60), Other-In (13), In-Vehicle (12), and Out (9). For Other-In and Out, most HEEs occurred indoors at fast food restaurants and retail stores, and outdoors in parking lots, respectively. For Home-In HEEs, the median TVOC emission and removal rates were 5.4 g h-1 and 1.1 h-1, respectively. Our study demonstrates the ability to determine individual sensor-based time-resolved TVOC exposures in different MEs, in support of identifying potential sources and exposure factors that can inform exposure mitigation strategies.

A national toxicology program systematic review of the evidence for long-term effects after acute exposure to sarin nerve agent.

Sarin is a highly toxic nerve agent that was developed for chemical warfare during World War II and is used in present conflicts. Immediate effects of acute sarin exposure are established; however, whether effects persist after initial signs have subsided is debated. The National Toxicology Program (NTP) conducted a systematic review to evaluate the evidence for long-term neurological effects following acute (<24 hour) exposure to sarin. The literature search and screening process identified 32 data sets within the 34 human studies and 47 data sets within the 51 animal studies (from 6837 potentially relevant references) that met the objective and the inclusion criteria. Four main health effect categories of neurological response were identified as having sufficient data to reach hazard conclusions: (1) cholinesterase levels; (2) visual and ocular effects; (3) effects on learning, memory, and intelligence; and (4) morphology and histopathology in nervous system tissues. NTP concluded that acute sarin exposure is known to be a neurological hazard to humans in the period following exposure up to 7 days and suspected to be a hazard week to years after exposure, given a lower level of evidence in later time periods. Effects included reduced cholinesterase, visual and ocular effects, impaired learning and memory, and altered nervous system morphology. Further mechanistic, targeted animal studies, translational research, and rapid research responses after human exposures may reduce uncertainties on long-term consequences of sarin.

Systematic reviews and meta-analyses of human and animal evidence of prenatal diethylhexyl phthalate exposure and changes in male anogenital distance.

Male reproductive alterations found in animals and humans following in utero phthalate exposure include decreased anogenital distance (AGD) and other reproductive-tract malformations. The aim of this investigation was to conduct systematic reviews of human and animal evidence of the effect of in utero exposure to diethylhexyl phthalate (DEHP) on anogenital distance (AGD) in males. PubMed, Embase, and Toxline were searched for relevant human and experimental animal studies on August 15, 2016. Search results were screened for relevance, and studies that met the inclusion criteria were evaluated for quality and data extracted for analysis. Confidence in the human and animal bodies of evidence was assessed and hazard conclusions reached by integrating evidence streams. The search yielded 6 relevant human studies and 19 animal studies. Meta-analysis of 5 human observational prospective cohort studies showed that increased maternal urinary concentrations of DEHP metabolites were associated with decreased AGD in boys (-4.07 [CI, -6.49 to -1.66] % decrease per log10 rise in DEHP metabolites). Meta-analysis and meta-regression of the 19 experimental animal studies found reduced AGD with DEHP treatment, with a dose-response gradient, and with heterogeneity explained by species and strain. There is a moderate level of evidence from human investigations and a high level of data from animal studies that in utero exposure to DEHP decreases AGD. Based upon the available human and animal evidence, and consideration of mechanistic data, DEHP is presumed to be a reproductive hazard to humans on the basis of effects on AGD.

Polybrominated diphenyl ether (PBDE) neurotoxicity: a systematic review and meta-analysis of animal evidence.

A recent systematic review (SR) and meta-analysis of human studies found an association between prenatal serum polybrominated diphenyl ethers (PBDE) concentrations and a decrease in the IQ of children. A SR of experimental developmental animal PBDE-mediated neurotoxicity studies was performed in the present study. Outcomes assessed included measures related to learning, memory, and attention, which parallel the intelligence-related outcomes evaluated in the human studies SR. PubMed, Embase, and Toxline were searched for relevant experimental non-human mammalian studies. Evaluation of risk of bias (RoB) and overall body of evidence followed guidance developed by the National Toxicology Program. Animal studies using varying designs and outcomes were available for BDEs 47, 99, 153, 203, 206, and 209 and the technical mixture DE-71. Study reporting of methods and results was often incomplete leading to concerns regarding RoB. A meta-analysis of 6 Morris water maze studies showed evidence of a significant increase in last trial latency (effect size of 25.8 [CI, 20.3 to 31.2]) in PBDE-exposed animals with low heterogeneity. For most endpoints, there were unexplained inconsistencies across studies and no consistent evidence of a dose-response relationship. There is a "moderate" level of evidence that exposure to BDEs 47, 99, and 209 affects learning. For other PBDEs and other endpoints, the level of evidence was "low" or "very low". The meta-analysis led to stronger conclusions than that based upon a qualitative review of the evidence. The SR also identified RoB concerns that might be remedied by better study reporting.

I-REFF diagrams: enhancing transparency in systematic review through interactive reference flow diagrams.

Systematic review methods are recognized for their rigor and transparency and are widely adapted to frameworks that cover review types such as systematic reviews, scoping reviews, and systematic evidence maps. Reporting guidelines help promote better systematic review practices and detailed documentation of the review process for different types of health research (e.g., PRISMA-Preferred Reporting Items for Systematic Reviews and Meta-Analyses; CONSORT-Consolidated Standards of Reporting Trials; and STROBE-Strengthening the Reporting of Observational Studies in Epidemiology). Transparency in the systematic review process and reporting of results is one of the key advantages of the methods and particularly important for hazard and risk assessments due to the high level of scrutiny these reviews face from scientific, political, and public communities. Data visualizations are important to clearly convey information from a review by helping readers perceive, understand, and assess the displayed information easily and quickly. The study flow diagram is a required element of a systematic review and maps out the number of included and excluded records identified, and the reasons for exclusion. Static literature flow diagrams help viewers readily understand the general review methodology and summarize the number of records included or excluded at each stage of the review. However, such diagrams can be time-consuming to develop and maintain during a systematic review or scoping review, and they provide limited summary-level information. We explored how the use of online systematic review tools such as DistillerSR coupled with visualization software such as Tableau can efficiently generate an Interactive REFerence Flow (I-REFF) diagram that is linked to the literature screening data, thus requiring minimal preparation, and resulting in a simplified process for updating the diagram. Furthermore, I-REFF diagrams enhance transparency and traceability by not only summarizing the records in the review but also allowing viewers to follow specific records throughout the review process. We present an example I-REFF diagram and discuss recommendations for key interactive elements to include in these diagrams and how this workflow can improve efficiency and result in an accessible and transparent interactive literature flow diagram without advanced programming.

A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

Becoming aWARE: The Development of a Web-Based Tool for Autism Research and the Environment.

A sharp rise in autism spectrum disorder (ASD) prevalence estimates, beginning in the 1990s, suggested factors additional to genetics were at play. This stimulated increased research investment in nongenetic factors, including the study of environmental chemical exposures, diet, nutrition, lifestyle, social factors, and maternal medical conditions. Consequently, both peer- and non-peer-reviewed bodies of evidence investigating environmental contributors to ASD etiology have grown significantly. The heterogeneity in the design and conduct of this research results in an inconclusive and unwieldy 'virtual stack' of publications. We propose to develop a Web-based tool for Autism Research and the Environment (aWARE) to comprehensively aggregate and assess these highly variable and often conflicting data. The interactive aWARE tool will use an approach for the development of systematic evidence maps (SEMs) to identify and display all available relevant published evidence, enabling users to explore multiple research questions within the scope of the SEM. Throughout tool development, listening sessions and workshops will be used to seek perspectives from the broader autism community. New evidence will be indexed in the tool annually, which will serve as a living resource to investigate the association between environmental factors and ASD.

Evaluating endocrine disrupting chemicals: A perspective on the novel assessments in CLARITY-BPA.

BACKGROUND: The Consortium Linking Academic and Regulatory Insights on Bisphenol A Toxicity (CLARITY-BPA) was a collaborative research effort to better link academic research with governmental guideline studies. This review explores the secondary goal of CLARITY-BPA: to identify endpoints or technologies from CLARITY-BPA and prior/concurrent literature from these laboratories that may enhance the capacity of rodent toxicity studies to detect endocrine disrupting chemicals (EDCs). METHODS: A systematic literature search was conducted with search terms for BPA and the CLARITY-BPA participants. Relevant studies employed a laboratory rodent model and reported results on 1 of the 10 organs/organ systems evaluated in CLARITY-BPA (brain and behavior, cardiac, immune, mammary gland, ovary, penile function, prostate gland and urethra, testis and epididymis, thyroid hormone and metabolism, and uterus). Study design and findings were summarized, and a risk-of-bias assessment was conducted. RESULTS: Several endpoints and methods were identified as potentially helpful to detect effects of EDCs. For example, molecular and quantitative morphological approaches were sensitive in detecting alterations in early postnatal development of the brain, ovary, and mammary glands. Hormone challenge studies mimicking human aging reported increased susceptibility of the prostate to disease following developmental BPA exposure. Statistical analyses for nonmonotonic dose responses, and computational approaches assessing multiple treatment-related outcomes concurrently in linked hormone-sensitive organ systems, reported effects at low BPA doses. CONCLUSIONS: This review provided an opportunity to evaluate the unique insights provided by nontraditional assessments in CLARITY-BPA to identify technologies and endpoints to enhance detection of EDCs in future studies.

Systematic evidence mapping informs a class-based approach to assessing personal care products and pubertal timing.

BACKGROUND: Personal care products (PCPs) contain many different compounds and are a source of exposure to endocrine disrupting chemicals (EDCs), including phthalates and phenols. Early-life exposure to EDCs commonly found in PCPs has been linked to earlier onset of puberty. OBJECTIVE: To characterize the human and animal evidence on the association between puberty-related outcomes and exposure to PCPs and their chemical constituents and, if there is sufficient evidence, identify groups of chemicals and outcomes to support a systematic review for a class-based hazard or risk assessment. METHODS: We followed the OHAT systematic review framework to characterize the human and animal evidence on the association between puberty-related health outcomes and exposure to PCPs and their chemical constituents. RESULTS: Ninety-eight human and 299 animal studies that evaluated a total of 96 different chemicals were identified and mapped by key concepts including chemical class, data stream, and puberty-related health outcome. Among these studies, phthalates and phenols were the most well-studied chemical classes. Most of the phthalate and phenol studies examined secondary sex characteristics and changes in estradiol and testosterone levels. Studies evaluating PCP use and other chemical classes (e.g., parabens) had less data. CONCLUSIONS: This systematic evidence map identified and mapped the published research evaluating the association between exposure to PCPs and their chemical constituents and puberty-related health outcomes. The resulting interactive visualization allows researchers to make evidence-based decisions on the available research by enabling them to search, sort, and filter the literature base of puberty-related studies by key concepts. This map can be used by researchers and regulators to prioritize and target future research and funding to reduce uncertainties and address data gaps. It also provides information to inform a class-based hazard or risk assessment on the association between phthalate and phenol exposures and puberty-related health outcomes.

Protocol for designing INVITES-IN, a tool for assessing the internal validity of in vitro studies.

This protocol describes the design and development of a tool for evaluation of the internal validity of in vitro studies, which is needed to include the data as evidence in systematic reviews and chemical risk assessments. The tool will be designed specifically to be applied to cell culture studies, including, but not restricted to, studies meeting the new approach methodology (NAM) definition. The tool is called INVITES-IN (IN VITro Experimental Studies INternal validity). In this protocol, three of the four studies that will be performed to create the release version of INVITES-IN are described. In the first study, evaluation of existing assessment tools will be combined with focus group discussions to identify how characteristics of the design or conduct of an in vitro study can affect its internal validity. Bias domains and items considered to be of relevance for in vitro studies will be identified. In the second study, group agreement on internal validity domains and items of importance for in vitro studies will be identified via a modified Delphi methodology. In the third study, the draft version of the tool will be created, based on the data on relevance and importance of bias domains and items collected in Studies 1 and 2. A separate protocol will be prepared for the fourth study, which includes the user testing and validation of the tool, and collection of users' experience.

Assessment of key characteristics, methodology, and effect size measures used in meta-analysis of human-health-related animal studies.

Since the early 1990s the number of systematic reviews (SR) of animal studies has steadily increased. There is, however, little guidance on when and how to conduct a meta-analysis of human-health-related animal studies. To gain insight about the methods that are currently used we created an overview of the key characteristics of published meta-analyses of animal studies, with a focus on the choice of effect size measures. An additional goal was to learn about the rationale behind the meta-analysis methods used by the review authors. We show that important details of the meta-analyses are not fully described, only a fraction of all human-health-related meta-analyses provided rationales for their decision to use specific effect size measures. In addition, our data may suggest that authors make post-hoc decisions to switch to another effect size measure during the course of their meta-analysis, and possibly search for significant effects. Based on analyses in this paper we recommend that review teams: 1) publish a review protocol before starting the conduct of a SR, prespecifying all methodological details (providing special attention to the planned meta-analysis including the effect size measure and the rational behind choosing a specific effect size, prespecifying subgroups and restricting the number of subgroup analyses), 2) always use the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) checklist to report your SR of animal studies, and 3) use the random effects model (REM) in human-health-related meta-analysis of animal studies, unless the assumptions for using the fixed effect model (FEM) are all met.

Identifying environmental factors that influence immune response to SARS-CoV-2: Systematic evidence map protocol.

BACKGROUND: Widespread environmental contamination can directly interact with human immune system functions. Environmental effects on the immune system may influence human susceptibility to respiratory infections as well as the severity of infectious diseases, such as the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Furthermore, the efficacy of vaccines to respiratory diseases may be impacted by environmental exposures through immune perturbations. Given the quick pace of research about COVID-19 and associated risk factors, it is critical to identify and curate the streams of evidence quickly and effectively. OBJECTIVE: We developed this systematic evidence map protocol to identify and organize existing human and animal literature on high-priority environmental chemical classes (Per- and polyfluoroalkyl substances, pesticides, phthalates, quaternary ammonium compounds, and air pollutants) and their potential to influence three key outcomes: (1) susceptibility to respiratory infection, including SARS-CoV-2 (2) severity of the resultant disease progression, and (3) impact on vaccine efficacy. The result of this project will be an online, interactive database which will show what evidence is currently available between involuntary exposures to select environmental chemicals and immune health effects, data gaps that require further research, and data rich areas that may support further analysis. SEARCH AND STUDY ELIGIBILITY: We will search PubMed for epidemiological or toxicological literature on select toxicants from each of the chemical classes and each of the three outcomes listed above. STUDY APPRAISAL AND SYNTHESIS OF METHODS: For each study, two independent reviewers will conduct title and abstract screening as well as full text review for data extraction of study characteristics. Study quality will not be evaluated in this evidence mapping. The main findings from the systematic evidence map will be visualized using a publicly available and interactive database hosted on Tableau Public.

Evaluation of a semi-automated data extraction tool for public health literature-based reviews: Dextr.

INTRODUCTION: There has been limited development and uptake of machine-learning methods to automate data extraction for literature-based assessments. Although advanced extraction approaches have been applied to some clinical research reviews, existing methods are not well suited for addressing toxicology or environmental health questions due to unique data needs to support reviews in these fields. OBJECTIVES: To develop and evaluate a flexible, web-based tool for semi-automated data extraction that: 1) makes data extraction predictions with user verification, 2) integrates token-level annotations, and 3) connects extracted entities to support hierarchical data extraction. METHODS: Dextr was developed with Agile software methodology using a two-team approach. The development team outlined proposed features and coded the software. The advisory team guided developers and evaluated Dextr's performance on precision, recall, and extraction time by comparing a manual extraction workflow to a semi-automated extraction workflow using a dataset of 51 environmental health animal studies. RESULTS: The semi-automated workflow did not appear to affect precision rate (96.0% vs. 95.4% manual, p = 0.38), resulted in a small reduction in recall rate (91.8% vs. 97.0% manual, p < 0.01), and substantially reduced the median extraction time (436 s vs. 933 s per study manual, p < 0.01) compared to a manual workflow. DISCUSSION: Dextr provides similar performance to manual extraction in terms of recall and precision and greatly reduces data extraction time. Unlike other tools, Dextr provides the ability to extract complex concepts (e.g., multiple experiments with various exposures and doses within a single study), properly connect the extracted elements within a study, and effectively limit the work required by researchers to generate machine-readable, annotated exports. The Dextr tool addresses data-extraction challenges associated with environmental health sciences literature with a simple user interface, incorporates the key capabilities of user verification and entity connecting, provides a platform for further automation developments, and has the potential to improve data extraction for literature reviews in this and other fields.

Consensus on the Key Characteristics of Immunotoxic Agents as a Basis for Hazard Identification.

BACKGROUND: Key characteristics (KCs), properties of agents or exposures that confer potential hazard, have been developed for carcinogens and other toxicant classes. KCs have been used in the systematic assessment of hazards and to identify assay and data gaps that limit screening and risk assessment. Many of the mechanisms through which pharmaceuticals and occupational or environmental agents modulate immune function are well recognized. Thus KCs could be identified for immunoactive substances and applied to improve hazard assessment of immunodulatory agents. OBJECTIVES: The goal was to generate a consensus-based synthesis of scientific evidence describing the KCs of agents known to cause immunotoxicity and potential applications, such as assays to measure the KCs. METHODS: A committee of 18 experts with diverse specialties identified 10 KCs of immunotoxic agents, namely, 1) covalently binds to proteins to form novel antigens, 2) affects antigen processing and presentation, 3) alters immune cell signaling, 4) alters immune cell proliferation, 5) modifies cellular differentiation, 6) alters immune cell-cell communication, 7) alters effector function of specific cell types, 8) alters immune cell trafficking, 9) alters cell death processes, and 10) breaks down immune tolerance. The group considered how these KCs could influence immune processes and contribute to hypersensitivity, inappropriate enhancement, immunosuppression, or autoimmunity. DISCUSSION: KCs can be used to improve efforts to identify agents that cause immunotoxicity via one or more mechanisms, to develop better testing and biomarker approaches to evaluate immunotoxicity, and to enable a more comprehensive and mechanistic understanding of adverse effects of exposures on the immune system. https://doi.org/10.1289/EHP10800.

An approach to quantifying the potential importance of residual confounding in systematic reviews of observational studies: A GRADE concept paper.

Small relative effect sizes are common in observational studies of exposure in environmental and public health. However, such effects can still have considerable policy importance when the baseline rate of the health outcome is high, and many persons are exposed. Assessing the certainty of the evidence based on these effect sizes is challenging because they can be prone to residual confounding due to the non-randomized nature of the evidence. When applying GRADE, a precise relative risk >2.0 increases the certainty in an existing effect because residual confounding is unlikely to explain the association. GRADE also suggests rating up when opposing plausible residual confounding exists for other effect sizes. In this concept paper, we propose using the E-value, defined as the smallest effect size of a confounder that still can reduce an observed RR to the null value, and a reference confounder to assess the likelihood of residual confounding. We propose a 4-step approach. 1. Assess the association of interest for relevant exposure levels. 2. Calculate the E-value for this observed association. 3. Choose a reference confounder with sufficient strength and information and assess its effect on the observed association using the E-value. 4. Assess how likely it is that residual confounding will still bias the observed RR. We present three case studies and discuss the feasibility of the approach.

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do.

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Applying evidence-based methods to the development and use of adverse outcome pathways

The workshop "Application of evidence-based methods to construct mechanistic frameworks for the development and use of non-animal toxicity tests" was organized by the Evidence-based Toxicology Collaboration and hosted by the Grading of Recommendations Assessment, Development and Evaluation Working Group on June 12, 2019. The purpose of the workshop was to bring together international regulatory bodies, risk assessors, academic scientists, and industry to explore how systematic review methods and the adverse outcome pathway framework could be combined to develop and use mechanistic test methods for predicting the toxicity of chemical substances in an evidence-based manner. The meeting covered the history of biological frameworks, the way adverse outcome pathways are currently developed, the basic principles of systematic methodology, including systematic reviews and evidence maps, and assessment of cer­tainty in models, and adverse outcome pathways in particular. Specific topics were discussed via case studies in small break-out groups. The group concluded that adverse outcome pathways provide an important framework to support mechanism-based assessment in environmental health. The process of their development has a few challenges that could be addressed with systematic methods and automation tools. Addressing these challenges will increase the transparency of the evidence behind adverse outcome pathways and the consistency with which they are defined; this in turn will increase their value for supporting public health decisions. It was suggested to explore the details of applying systematic methods to adverse outcome pathway development in a series of case studies and workshops.

Developing trustworthy recommendations as part of an urgent response (1-2 weeks): a GRADE concept paper.

OBJECTIVES: The aim of this study is to propose an approach for developing trustworthy recommendations as part of urgent responses (1-2 week) in the clinical, public health, and health systems fields. STUDY DESIGN AND SETTING: We conducted a review of the literature, outlined a draft approach, refined the concept through iterative discussions, a workshop by the Grading of Recommendations Assessment, Development and Evaluation Rapid Guidelines project group, and obtained feedback from the larger Grading of Recommendations Assessment, Development and Evaluation working group. RESULTS: A request for developing recommendations within 2 week is the usual trigger for an urgent response. Although the approach builds on the general principles of trustworthy guideline development, we highlight the following steps: (1) assess the level of urgency; (2) assess feasibility; (3) set up the organizational logistics; (4) specify the question(s); (5) collect the information needed; (6) assess the adequacy of identified information; (7) develop the recommendations using one of the 4 potential approaches: adopt existing recommendations, adapt existing recommendations, develop new recommendations using existing adequate systematic review, or develop new recommendations using expert panel input; and (8) consider an updating plan. CONCLUSION: An urgent response for developing recommendations requires building a cohesive, skilled, and highly motivated multidisciplinary team with the necessary clinical, scientific, and methodological expertise; adapting to shifting needs; complying with the principles of transparency; and properly managing conflicts of interest.