I-REFF diagrams: enhancing transparency in systematic review through interactive reference flow diagrams.

Systematic review methods are recognized for their rigor and transparency and are widely adapted to frameworks that cover review types such as systematic reviews, scoping reviews, and systematic evidence maps. Reporting guidelines help promote better systematic review practices and detailed documentation of the review process for different types of health research (e.g., PRISMA-Preferred Reporting Items for Systematic Reviews and Meta-Analyses; CONSORT-Consolidated Standards of Reporting Trials; and STROBE-Strengthening the Reporting of Observational Studies in Epidemiology). Transparency in the systematic review process and reporting of results is one of the key advantages of the methods and particularly important for hazard and risk assessments due to the high level of scrutiny these reviews face from scientific, political, and public communities. Data visualizations are important to clearly convey information from a review by helping readers perceive, understand, and assess the displayed information easily and quickly. The study flow diagram is a required element of a systematic review and maps out the number of included and excluded records identified, and the reasons for exclusion. Static literature flow diagrams help viewers readily understand the general review methodology and summarize the number of records included or excluded at each stage of the review. However, such diagrams can be time-consuming to develop and maintain during a systematic review or scoping review, and they provide limited summary-level information. We explored how the use of online systematic review tools such as DistillerSR coupled with visualization software such as Tableau can efficiently generate an Interactive REFerence Flow (I-REFF) diagram that is linked to the literature screening data, thus requiring minimal preparation, and resulting in a simplified process for updating the diagram. Furthermore, I-REFF diagrams enhance transparency and traceability by not only summarizing the records in the review but also allowing viewers to follow specific records throughout the review process. We present an example I-REFF diagram and discuss recommendations for key interactive elements to include in these diagrams and how this workflow can improve efficiency and result in an accessible and transparent interactive literature flow diagram without advanced programming.

Systematic evidence mapping informs a class-based approach to assessing personal care products and pubertal timing.

BACKGROUND: Personal care products (PCPs) contain many different compounds and are a source of exposure to endocrine disrupting chemicals (EDCs), including phthalates and phenols. Early-life exposure to EDCs commonly found in PCPs has been linked to earlier onset of puberty. OBJECTIVE: To characterize the human and animal evidence on the association between puberty-related outcomes and exposure to PCPs and their chemical constituents and, if there is sufficient evidence, identify groups of chemicals and outcomes to support a systematic review for a class-based hazard or risk assessment. METHODS: We followed the OHAT systematic review framework to characterize the human and animal evidence on the association between puberty-related health outcomes and exposure to PCPs and their chemical constituents. RESULTS: Ninety-eight human and 299 animal studies that evaluated a total of 96 different chemicals were identified and mapped by key concepts including chemical class, data stream, and puberty-related health outcome. Among these studies, phthalates and phenols were the most well-studied chemical classes. Most of the phthalate and phenol studies examined secondary sex characteristics and changes in estradiol and testosterone levels. Studies evaluating PCP use and other chemical classes (e.g., parabens) had less data. CONCLUSIONS: This systematic evidence map identified and mapped the published research evaluating the association between exposure to PCPs and their chemical constituents and puberty-related health outcomes. The resulting interactive visualization allows researchers to make evidence-based decisions on the available research by enabling them to search, sort, and filter the literature base of puberty-related studies by key concepts. This map can be used by researchers and regulators to prioritize and target future research and funding to reduce uncertainties and address data gaps. It also provides information to inform a class-based hazard or risk assessment on the association between phthalate and phenol exposures and puberty-related health outcomes.

Assessment of key characteristics, methodology, and effect size measures used in meta-analysis of human-health-related animal studies.

Since the early 1990s the number of systematic reviews (SR) of animal studies has steadily increased. There is, however, little guidance on when and how to conduct a meta-analysis of human-health-related animal studies. To gain insight about the methods that are currently used we created an overview of the key characteristics of published meta-analyses of animal studies, with a focus on the choice of effect size measures. An additional goal was to learn about the rationale behind the meta-analysis methods used by the review authors. We show that important details of the meta-analyses are not fully described, only a fraction of all human-health-related meta-analyses provided rationales for their decision to use specific effect size measures. In addition, our data may suggest that authors make post-hoc decisions to switch to another effect size measure during the course of their meta-analysis, and possibly search for significant effects. Based on analyses in this paper we recommend that review teams: 1) publish a review protocol before starting the conduct of a SR, prespecifying all methodological details (providing special attention to the planned meta-analysis including the effect size measure and the rational behind choosing a specific effect size, prespecifying subgroups and restricting the number of subgroup analyses), 2) always use the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) checklist to report your SR of animal studies, and 3) use the random effects model (REM) in human-health-related meta-analysis of animal studies, unless the assumptions for using the fixed effect model (FEM) are all met.

Developing a database of systematic reviews of animal studies.

Systematic reviews (SRs) are common practice in clinical and public health research, but less common in non-human animal research. Systematic reviews of animal studies can be valuable to inform clinical research, to evaluate the need for further animal experiments on a given topic, and to assess the hazard of an environmental exposure in the evaluation of toxicological studies. In the last 10 years, there has been an increase in the number of SRs of animal research, as well as several publications with detailed guidance on how to perform high-quality systematic reviews of experimental animal studies. In order to evaluate current analytical approaches used in SRs of animal studies, easily identify all systematic reviews on a specific topic, and subsequently the original animal studies and their results and promote awareness and understanding of these emerging approaches, we compiled a database of SRs of animal studies. The database was developed using a rigorous, systematic approach and covers a broad range of research fields: preclinical research, toxicology, environmental health, and veterinary medicine. The database currently includes 3113 SRs of animal studies (search date June 2019). In addition to bibliographical information, data on whether or not a risk of bias assessment and meta-analysis were conducted were extracted. For future users, the search features of the database provide users with a platform to identify and select SRs with a particular characteristic for export to Microsoft Word or Microsoft Excel. From there, users may perform additional data extraction to meet their research needs. The database is freely available at www.Mendeley.com (link). The database provides methodologists a comprehensive source that can be used to explore and advance the current methodology applied to SRs of animal studies, and can help researchers to easily identify all systematic reviews on a specific topic, and subsequently the original animal studies and their results and avoid duplication and unnecessary animal research.

GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence-An overview in the context of health decision-making.

OBJECTIVES: The objective of the study is to present the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) conceptual approach to the assessment of certainty of evidence from modeling studies (i.e., certainty associated with model outputs). STUDY DESIGN AND SETTING: Expert consultations and an international multidisciplinary workshop informed development of a conceptual approach to assessing the certainty of evidence from models within the context of systematic reviews, health technology assessments, and health care decisions. The discussions also clarified selected concepts and terminology used in the GRADE approach and by the modeling community. Feedback from experts in a broad range of modeling and health care disciplines addressed the content validity of the approach. RESULTS: Workshop participants agreed that the domains determining the certainty of evidence previously identified in the GRADE approach (risk of bias, indirectness, inconsistency, imprecision, reporting bias, magnitude of an effect, dose-response relation, and the direction of residual confounding) also apply when assessing the certainty of evidence from models. The assessment depends on the nature of model inputs and the model itself and on whether one is evaluating evidence from a single model or multiple models. We propose a framework for selecting the best available evidence from models: 1) developing de novo, a model specific to the situation of interest, 2) identifying an existing model, the outputs of which provide the highest certainty evidence for the situation of interest, either "off-the-shelf" or after adaptation, and 3) using outputs from multiple models. We also present a summary of preferred terminology to facilitate communication among modeling and health care disciplines. CONCLUSION: This conceptual GRADE approach provides a framework for using evidence from models in health decision-making and the assessment of certainty of evidence from a model or models. The GRADE Working Group and the modeling community are currently developing the detailed methods and related guidance for assessing specific domains determining the certainty of evidence from models across health care-related disciplines (e.g., therapeutic decision-making, toxicology, environmental health, and health economics).