Standardizing Extracted Data Using Automated Application of Controlled Vocabularies.

BACKGROUND: Extraction of toxicological end points from primary sources is a central component of systematic reviews and human health risk assessments. To ensure optimal use of these data, consistent language should be used for end point descriptions. However, primary source language describing treatment-related end points can vary greatly, resulting in large labor efforts to manually standardize extractions before data are fit for use. OBJECTIVES: To minimize these labor efforts, we applied an augmented intelligence approach and developed automated tools to support standardization of extracted information via application of preexisting controlled vocabularies. METHODS: We created and applied a harmonized controlled vocabulary crosswalk, consisting of Unified Medical Language System (UMLS) codes, German Federal Institute for Risk Assessment (BfR) DevTox harmonized terms, and The Organization for Economic Co-operation and Development (OECD) end point vocabularies, to roughly 34,000 extractions from prenatal developmental toxicology studies conducted by the National Toxicology Program (NTP) and 6,400 extractions from European Chemicals Agency (ECHA) prenatal developmental toxicology studies, all recorded based on the original study report language. RESULTS: We automatically applied standardized controlled vocabulary terms to 75% of the NTP extracted end points and 57% of the ECHA extracted end points. Of all the standardized extracted end points, about half (51%) required manual review for potential extraneous matches or inaccuracies. Extracted end points that were not mapped to standardized terms tended to be too general or required human logic to find a good match. We estimate that this augmented intelligence approach saved >350 hours of manual effort and yielded valuable resources including a controlled vocabulary crosswalk, organized related terms lists, code for implementing an automated mapping workflow, and a computationally accessible dataset. DISCUSSION: Augmenting manual efforts with automation tools increased the efficiency of producing a findable, accessible, interoperable, and reusable (FAIR) dataset of regulatory guideline studies. This open-source approach can be readily applied to other legacy developmental toxicology datasets, and the code design is customizable for other study types. https://doi.org/10.1289/EHP13215.

Enabling novel paradigms: a biological questions-based approach to human chemical hazard and drug safety assessment.

Throughput needs, costs of time and resources, and concerns about the use of animals in hazard and safety assessment studies are fueling a growing interest in adopting new approach methodologies for use in product development and risk assessment. However, current efforts to define "next-generation risk assessment" vary considerably across commercial and regulatory sectors, and an a priori definition of the biological scope of data needed to assess hazards is generally lacking. We propose that the absence of clearly defined questions that can be answered during hazard assessment is the primary barrier to the generation of a paradigm flexible enough to be used across varying product development and approval decision contexts. Herein, we propose a biological questions-based approach (BQBA) for hazard and safety assessment to facilitate fit-for-purpose method selection and more efficient evidence-based decision-making. The key pillars of this novel approach are bioavailability, bioactivity, adversity, and susceptibility. This BQBA is compared with current hazard approaches and is applied in scenarios of varying pathobiological understanding and/or regulatory testing requirements. To further define the paradigm and key questions that allow better prediction and characterization of human health hazard, a multidisciplinary collaboration among stakeholder groups should be initiated.

Epithelial ovarian cancer is infiltrated by activated effector T cells co-expressing CD39, PD-1, TIM-3, CD137 and interacting with cancer cells and myeloid cells.

Introduction: Despite predicted efficacy, immunotherapy in epithelial ovarian cancer (EOC) has limited clinical benefit and the prognosis of patients remains poor. There is thus a strong need for better identifying local immune dynamics and immune-suppressive pathways limiting T-cell mediated anti-tumor immunity. Methods: In this observational study we analyzed by immunohistochemistry, gene expression profiling and flow cytometry the antigenic landscape and immune composition of 48 EOC specimens, with a focus on tumor-infiltrating lymphocytes (TILs). Results: Activated T cells showing features of partial exhaustion with a CD137+CD39+PD-1+TIM-3+CD45RA-CD62L-CD95+ surface profile were exclusively present in EOC specimens but not in corresponding peripheral blood or ascitic fluid, indicating that the tumor microenvironment might sustain this peculiar phenotype. Interestingly, while neoplastic cells expressed several tumor-associated antigens possibly able to stimulate tumor-specific TILs, macrophages provided both co-stimulatory and inhibitory signals and were more abundant in TILs-enriched specimens harboring the CD137+CD39+PD-1+TIM-3+CD45RA-CD62L-CD95+ signature. Conclusion: These data demonstrate that EOC is enriched in CD137+CD39+PD-1+TIM-3+CD45RA-CD62L-CD95+ T lymphocytes, a phenotype possibly modulated by antigen recognition on neoplastic cells and by a combination of inhibitory and co-stimulatory signals largely provided by infiltrating myeloid cells. Furthermore, we have identified immunosuppressive pathways potentially hampering local immunity which might be targeted by immunotherapeutic approaches.

Workshop Report: Catalyzing Knowledge-Driven Discovery in Environmental Health Sciences through a Harmonized Language.

Harmonized language is essential to finding, sharing, and reusing large-scale, complex data. Gaps and barriers prevent the adoption of harmonized language approaches in environmental health sciences (EHS). To address this, the National Institute of Environmental Health Sciences and partners created the Environmental Health Language Collaborative (EHLC). The purpose of EHLC is to facilitate a community-driven effort to advance the development and adoption of harmonized language approaches in EHS. EHLC is a forum to pinpoint language harmonization gaps, to facilitate the development of, raise awareness of, and encourage the use of harmonization approaches and tools, and to develop new standards and recommendations. To ensure that EHLC's focus and structure would be sustainable long-term and meet the needs of the field, EHLC launched an inaugural workshop in September 2021 focused on "Developing Sustainable Language Solutions" and "Building a Sustainable Community". When the attendees were surveyed, 91% said harmonized language solutions would be of high value/benefit, and 60% agreed to continue contributing to EHLC efforts. Based on workshop discussions, future activities will focus on targeted collaborative use-case working groups in addition to offering education and training on ontologies, metadata, and standards, and developing an EHS language resource portal.

GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence-An overview in the context of health decision-making.

OBJECTIVES: The objective of the study is to present the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) conceptual approach to the assessment of certainty of evidence from modeling studies (i.e., certainty associated with model outputs). STUDY DESIGN AND SETTING: Expert consultations and an international multidisciplinary workshop informed development of a conceptual approach to assessing the certainty of evidence from models within the context of systematic reviews, health technology assessments, and health care decisions. The discussions also clarified selected concepts and terminology used in the GRADE approach and by the modeling community. Feedback from experts in a broad range of modeling and health care disciplines addressed the content validity of the approach. RESULTS: Workshop participants agreed that the domains determining the certainty of evidence previously identified in the GRADE approach (risk of bias, indirectness, inconsistency, imprecision, reporting bias, magnitude of an effect, dose-response relation, and the direction of residual confounding) also apply when assessing the certainty of evidence from models. The assessment depends on the nature of model inputs and the model itself and on whether one is evaluating evidence from a single model or multiple models. We propose a framework for selecting the best available evidence from models: 1) developing de novo, a model specific to the situation of interest, 2) identifying an existing model, the outputs of which provide the highest certainty evidence for the situation of interest, either "off-the-shelf" or after adaptation, and 3) using outputs from multiple models. We also present a summary of preferred terminology to facilitate communication among modeling and health care disciplines. CONCLUSION: This conceptual GRADE approach provides a framework for using evidence from models in health decision-making and the assessment of certainty of evidence from a model or models. The GRADE Working Group and the modeling community are currently developing the detailed methods and related guidance for assessing specific domains determining the certainty of evidence from models across health care-related disciplines (e.g., therapeutic decision-making, toxicology, environmental health, and health economics).

Novel text analytics approach to identify relevant literature for human health risk assessments: A pilot study with health effects of in utero exposures.

BACKGROUND: Systematic reviews involve mining literature databases to identify relevant studies. Identifying potentially relevant studies can be informed by computational tools comparing text similarity between candidate studies and selected key (i.e., seed) references. Challenge Using computational approaches to identify relevant studies for risk assessments is challenging, as these assessments examine multiple chemical effects across lifestages (e.g., human health risk assessments) or specific effects of multiple chemicals (e.g., cumulative risk). The broad scope of potentially relevant literature can make selection of seed references difficult. Approach We developed a generalized computational scoping strategy to identify human health relevant studies for multiple chemicals and multiple effects. We used semi-supervised machine learning to prioritize studies to review manually with training data derived from references cited in the hazard identification sections of several US EPA Integrated Risk Information System (IRIS) assessments. These generic training data or seed studies were clustered with the unclassified corpus to group studies based on text similarity. Clusters containing a high proportion of seed studies were prioritized for manual review. Chemical names were removed from seed studies prior to clustering resulting in a generic, chemical-independent method for identifying potentially human health relevant studies. We developed a case study that focused on identifying the array of chemicals that have been studied with respect to in utero exposure to test the recall of this novel literature searching strategy. We then evaluated the general strategy of using generic, chemical-independent training data with two previous IRIS assessments by comparing studies predicted relevant to those used in the assessments (i.e., total relevant). Outcome A keyword search designed to retrieve studies that examined the in utero effects of environmental chemicals identified over 54,000 candidate references. Clustering algorithms were applied using 1456 studies from multiple IRIS assessments with chemical names removed as training data or seeds (i.e., semi-supervised learning). Using a six-algorithm ensemble approach 2602 articles, or approximately 5% of candidate references, were "voted" relevant by four or more clustering algorithms and manual review confirmed nearly 50% of these studies were relevant. Further evaluations on two IRIS assessments, using a nine-algorithm ensemble approach and a set of generic, chemical-independent, externally-derived seed studies correctly identified 77-83% of hazard identification studies published in the assessments and eliminated the need to manually screen more than 75% of search results on average. Limitations The chemical-independent approach used to build the training literature set provides a broad and unbiased picture across a variety of endpoints and environmental exposures but does not systematically identify all available data. Variance between actual and predicted relevant studies will be greater because of the external and non-random origin of seed study selection. This approach depends on access to readily available generic training data that can be used to locate relevant references in an unclassified corpus. Impact A generic approach to identifying human health relevant studies could be an important first step in literature evaluation for risk assessments. This initial scoping approach could facilitate faster literature evaluation by focusing reviewer efforts, as well as potentially minimize reviewer bias in selection of key studies. Using externally-derived training data has applicability particularly for databases with very low search precision where identifying training data may be cost-prohibitive.

Evaluating potential human health risks from modeled inhalation exposures to volatile organic compounds emitted from oil and gas operations.

Some states and localities restrict siting of new oil and gas (O&G) wells relative to public areas. Colorado includes a 500-foot exception zone for building units, but it is unclear if that sufficiently protects public health from air emissions from O&G operations. To support reviews of setback requirements, this research examines potential health risks from volatile organic compounds (VOCs) released during O&G operations.We used stochastic dispersion modeling with published emissions for 47 VOCs (collected on-site during tracer experiments) to estimate outdoor air concentrations within 2,000 feet of hypothetical individual O&G facilities in Colorado. We estimated distributions of incremental acute, subchronic, and chronic inhalation non-cancer hazard quotients (HQs) and hazard indices (HIs), and inhalation lifetime cancer risks for benzene, by coupling modeled concentrations with microenvironmental penetration factors, human-activity diaries, and health-criteria levels.Estimated exposures to most VOCs were below health criteria at 500-2,000 feet. HQs were < 1 for 43 VOCs at 500 feet from facilities, with lowest values for chronic exposures during O&G production. Hazard estimates were highest for acute exposures during O&G development, with maximum acute HQs and HIs > 1 at most distances from facilities, particularly for exposures to benzene, 2- and 3-ethyltoluene, and toluene, and for hematological, neurotoxicity, and respiratory effects. Maximum acute HQs and HIs were > 10 for highest-exposed individuals 500 feet from eight of nine modeled facilities during O&G development (and 2,000 feet from one facility during O&G flowback); hematologic toxicity associated with benzene exposure was the critical toxic effect. Estimated cancer risks from benzene exposure were < 1.0 × 10-5 at 500 feet and beyond.Implications: Our stochastic use of emissions data from O&G facilities, along with activity-pattern exposure modeling, provides new information on potential public-health impacts due to emissions from O&G operations. The results will help in evaluating the adequacy of O&G setback distances. For an assessment of human-health risks from exposures to air emissions near individual O&G sites, we have utilized a unique dataset of tracer-derived emissions of VOCs detected at such sites in two regions of intense oil-and-gas development in Colorado. We have coupled these emission stochastically with local meteorological data and population and time-activity data to estimate the potential for acute, subchronic, and chronic exposures above health-criteria levels due to air emissions near individual sites. These results, along with other pertinent health and exposure data, can be used to inform setback distances to protect public health.

Python BMDS: A Python interface library and web application for the canonical EPA dose-response modeling software.

Several primary sources of publicly available, quantitative dose-response data from traditional toxicology study designs relevant to predictive toxicology applications are now available, including the redeveloped U.S. Environmental Protection Agency's Toxicity Reference Database (ToxRefDB v2.0), the Health Assessment Workspace Collaborative (HAWC), and the National Toxicology Program's Chemical Program's Chemical Effects in Biological Systems (CEBS). These resources provide effect level information but modeling these data to a curve may be more informative for predictive toxicology applications. Benchmark Dose Software (BMDS) has been recognized broadly and used for regulatory applications at multiple agencies. However, the current BMDS software was not amenable to modeling large datasets. Herein we describe development and use of a Python package that implements a wrapper around BMDS, a software that requires manual input in the dose-response modeling process (i.e., best-fitting model-selection, reporting, and dose-dropping). In the Python BMDS, users can select the BMDS version, customize model recommendation logic, and export summaries of the resultant BMDS output. Further, using the Python interface, a web-based application programming interface (API) has been developed for easy integration into other software systems, pipelines, or databases. Software utility was demonstrated via modeling nearly 28,000 datasets in ToxRefDB v2.0, re-creation of an existing, published large-scale analysis, and demonstration of usage in software such as CEBS and HAWC. Python BMDS enables rapid-batch processing of dose-response datasets using a modeling software with broad acceptance in the toxicology community, thereby providing an important tool for leveraging the publicly available quantitative toxicology data in a reproducible manner.

ToxRefDB version 2.0: Improved utility for predictive and retrospective toxicology analyses.

The Toxicity Reference Database (ToxRefDB) structures information from over 5000 in vivo toxicity studies, conducted largely to guidelines or specifications from the US Environmental Protection Agency and the National Toxicology Program, into a public resource for training and validation of predictive models. Herein, ToxRefDB version 2.0 (ToxRefDBv2) development is described. Endpoints were annotated (e.g. required, not required) according to guidelines for subacute, subchronic, chronic, developmental, and multigenerational reproductive designs, distinguishing negative responses from untested. Quantitative data were extracted, and dose-response modeling for nearly 28,000 datasets from nearly 400 endpoints using Benchmark Dose (BMD) Modeling Software were generated and stored. Implementation of controlled vocabulary improved data quality; standardization to guideline requirements and cross-referencing with United Medical Language System (UMLS) connects ToxRefDBv2 observations to vocabularies linked to UMLS, including PubMed medical subject headings. ToxRefDBv2 allows for increased connections to other resources and has greatly enhanced quantitative and qualitative utility for predictive toxicology.

A scoping review of the health and toxicological activity of bisphenol A (BPA) structural analogues and functional alternatives.

Recent studies report widespread usage or exposure to a variety of chemicals with structural or functional similarity to bisphenol A (BPA), referred to as BPA analogues or derivatives. These have been detected in foodstuffs, house dust, environmental samples, human urine or blood, and consumer products. Compared to BPA, relatively little is known about potential toxicity of these compounds. This scoping review aimed to summarize the human, animal, and mechanistic toxicity data for 24 BPA analogues of emerging interest to research and regulatory communities. PubMed was searched from March 1, 2015 to January 5, 2019 and combined with the results obtained from literature searches conducted through March 23, 2015, in The National Toxicology Program's Research Report 4 (NTP RR-04), "Biological Activity of Bisphenol A (BPA) Structural Analogues and Functional Alternatives". Study details are presented in interactive displays using Tableau Public. In total, 5748 records were screened for inclusion. One hundred sixty seven studies were included from NTP RR-04 and 175 studies were included from the updated literature search through January 2019. In total, there are 22, 117, and 221 human epidemiological, experimental animal, or in vitro studies included. The most frequently studied BPA analogues are bisphenol S (BPS), bisphenol F (4,4-BPF), and bisphenol AF (BPAF). Notable changes in the literature since 2015 include the growing body of human epidemiological studies and in vivo studies conducted in zebrafish. Numerous new endpoints were also evaluated across all three evidence streams including diabetes, obesity, and oxidative stress. However, few studies have addressed endpoints such as neurodevelopmental outcomes or impacts on the developing mammary or prostate glands, which are known to be susceptible to disruption by BPA. Further, there remains a critical need for better exposure information in order to prioritize experimental studies. Moving forward, researchers should also ensure that full dose responses are performed for all main effects in order to support hazard and risk characterization efforts. The evidence gathered here suggests that hazard and risk characterizations should expand beyond BPA in order to consider BPA structural and functional analogues.

Conditional Toxicity Value (CTV) Predictor: An In Silico Approach for Generating Quantitative Risk Estimates for Chemicals.

BACKGROUND: Human health assessments synthesize human, animal, and mechanistic data to produce toxicity values that are key inputs to risk-based decision making. Traditional assessments are data-, time-, and resource-intensive, and they cannot be developed for most environmental chemicals owing to a lack of appropriate data. OBJECTIVES: As recommended by the National Research Council, we propose a solution for predicting toxicity values for data-poor chemicals through development of quantitative structure-activity relationship (QSAR) models. METHODS: We used a comprehensive database of chemicals with existing regulatory toxicity values from U.S. federal and state agencies to develop quantitative QSAR models. We compared QSAR-based model predictions to those based on high-throughput screening (HTS) assays. RESULTS: QSAR models for noncancer threshold-based values and cancer slope factors had cross-validation-based Q2 of 0.25-0.45, mean model errors of 0.70-1.11 log10 units, and applicability domains covering >80% of environmental chemicals. Toxicity values predicted from QSAR models developed in this study were more accurate and precise than those based on HTS assays or mean-based predictions. A publicly accessible web interface to make predictions for any chemical of interest is available at http://toxvalue.org. CONCLUSIONS: An in silico tool that can predict toxicity values with an uncertainty of an order of magnitude or less can be used to quickly and quantitatively assess risks of environmental chemicals when traditional toxicity data or human health assessments are unavailable. This tool can fill a critical gap in the risk assessment and management of data-poor chemicals. https://doi.org/10.1289/EHP2998.

In vitro cardiotoxicity assessment of environmental chemicals using an organotypic human induced pluripotent stem cell-derived model.

An important target area for addressing data gaps through in vitro screening is the detection of potential cardiotoxicants. Despite the fact that current conservative estimates relate at least 23% of all cardiovascular disease cases to environmental exposures, the identities of the causative agents remain largely uncharacterized. Here, we evaluate the feasibility of a combinatorial in vitro/in silico screening approach for functional and mechanistic cardiotoxicity profiling of environmental hazards using a library of 69 representative environmental chemicals and drugs. Human induced pluripotent stem cell-derived cardiomyocytes were exposed in concentration-response for 30min or 24h and effects on cardiomyocyte beating and cellular and mitochondrial toxicity were assessed by kinetic measurements of intracellular Ca2+ flux and high-content imaging using the nuclear dye Hoechst 33342, the cell viability marker Calcein AM, and the mitochondrial depolarization probe JC-10. More than half of the tested chemicals exhibited effects on cardiomyocyte beating after 30min of exposure. In contrast, after 24h, effects on cell beating without concomitant cytotoxicity were observed in about one third of the compounds. Concentration-response data for in vitro bioactivity phenotypes visualized using the Toxicological Prioritization Index (ToxPi) showed chemical class-specific clustering of environmental chemicals, including pesticides, flame retardants, and polycyclic aromatic hydrocarbons. For environmental chemicals with human exposure predictions, the activity-to-exposure ratios between modeled blood concentrations and in vitro bioactivity were between one and five orders of magnitude. These findings not only demonstrate that some ubiquitous environmental pollutants might have the potential at high exposure levels to alter cardiomyocyte function, but also indicate similarities in the mechanism of these effects both within and among chemicals and classes.

Standardizing benchmark dose calculations to improve science-based decisions in human health assessments.

BACKGROUND: Benchmark dose (BMD) modeling computes the dose associated with a prespecified response level. While offering advantages over traditional points of departure (PODs), such as no-observed-adverse-effect-levels (NOAELs), BMD methods have lacked consistency and transparency in application, interpretation, and reporting in human health assessments of chemicals. OBJECTIVES: We aimed to apply a standardized process for conducting BMD modeling to reduce inconsistencies in model fitting and selection. METHODS: We evaluated 880 dose-response data sets for 352 environmental chemicals with existing human health assessments. We calculated benchmark doses and their lower limits [10% extra risk, or change in the mean equal to 1 SD (BMD/L10/1SD)] for each chemical in a standardized way with prespecified criteria for model fit acceptance. We identified study design features associated with acceptable model fits. RESULTS: We derived values for 255 (72%) of the chemicals. Batch-calculated BMD/L10/1SD values were significantly and highly correlated (R2 of 0.95 and 0.83, respectively, n = 42) with PODs previously used in human health assessments, with values similar to reported NOAELs. Specifically, the median ratio of BMDs10/1SD:NOAELs was 1.96, and the median ratio of BMDLs10/1SD:NOAELs was 0.89. We also observed a significant trend of increasing model viability with increasing number of dose groups. CONCLUSIONS: BMD/L10/1SD values can be calculated in a standardized way for use in health assessments on a large number of chemicals and critical effects. This facilitates the exploration of health effects across multiple studies of a given chemical or, when chemicals need to be compared, providing greater transparency and efficiency than current approaches.

Assessment of beating parameters in human induced pluripotent stem cells enables quantitative in vitro screening for cardiotoxicity.

Human induced pluripotent stem cell (iPSC)-derived cardiomyocytes show promise for screening during early drug development. Here, we tested a hypothesis that in vitro assessment of multiple cardiomyocyte physiological parameters enables predictive and mechanistically-interpretable evaluation of cardiotoxicity in a high-throughput format. Human iPSC-derived cardiomyocytes were exposed for 30 min or 24 h to 131 drugs, positive (107) and negative (24) for in vivo cardiotoxicity, in up to 6 concentrations (3 nM to 30 uM) in 384-well plates. Fast kinetic imaging was used to monitor changes in cardiomyocyte function using intracellular Ca(2+) flux readouts synchronous with beating, and cell viability. A number of physiological parameters of cardiomyocyte beating, such as beat rate, peak shape (amplitude, width, raise, decay, etc.) and regularity were collected using automated data analysis. Concentration-response profiles were evaluated using logistic modeling to derive a benchmark concentration (BMC) point-of-departure value, based on one standard deviation departure from the estimated baseline in vehicle (0.3% dimethyl sulfoxide)-treated cells. BMC values were used for cardiotoxicity classification and ranking of compounds. Beat rate and several peak shape parameters were found to be good predictors, while cell viability had poor classification accuracy. In addition, we applied the Toxicological Prioritization Index (ToxPi) approach to integrate and display data across many collected parameters, to derive "cardiosafety" ranking of tested compounds. Multi-parameter screening of beating profiles allows for cardiotoxicity risk assessment and identification of specific patterns defining mechanism-specific effects. These data and analysis methods may be used widely for compound screening and early safety evaluation in drug development.