Guidance for good practice in the application of machine learning in development of toxicological quantitative structure-activity relationships (QSARs).

Recent years have seen a substantial growth in the adoption of machine learning approaches for the purposes of quantitative structure-activity relationship (QSAR) development. Such a trend has coincided with desire to see a shifting in the focus of methodology employed within chemical safety assessment: away from traditional reliance upon animal-intensive in vivo protocols, and towards increased application of in silico (or computational) predictive toxicology. With QSAR central amongst techniques applied in this area, the emergence of algorithms trained through machine learning with the objective of toxicity estimation has, quite naturally, arisen. On account of the pattern-recognition capabilities of the underlying methods, the statistical power of the ensuing models is potentially considerable-appropriate for the handling even of vast, heterogeneous datasets. However, such potency comes at a price: this manifesting as the general practical deficits observed with respect to the reproducibility, interpretability and generalisability of the resulting tools. Unsurprisingly, these elements have served to hinder broader uptake (most notably within a regulatory setting). Areas of uncertainty liable to accompany (and hence detract from applicability of) toxicological QSAR have previously been highlighted, accompanied by the forwarding of suggestions for "best practice" aimed at mitigation of their influence. However, the scope of such exercises has remained limited to "classical" QSAR-that conducted through use of linear regression and related techniques, with the adoption of comparatively few features or descriptors. Accordingly, the intention of this study has been to extend the remit of best practice guidance, so as to address concerns specific to employment of machine learning within the field. In doing so, the impact of strategies aimed at enhancing the transparency (feature importance, feature reduction), generalisability (cross-validation) and predictive power (hyperparameter optimisation) of algorithms, trained upon real toxicity data through six common learning approaches, is evaluated.

Making in silico predictive models for toxicology FAIR.

In silico predictive models for toxicology include quantitative structure-activity relationship (QSAR) and physiologically based kinetic (PBK) approaches to predict physico-chemical and ADME properties, toxicological effects and internal exposure. Such models are used to fill data gaps as part of chemical risk assessment. There is a growing need to ensure in silico predictive models for toxicology are available for use and that they are reproducible. This paper describes how the FAIR (Findable, Accessible, Interoperable, Reusable) principles, developed for data sharing, have been applied to in silico predictive models. In particular, this investigation has focussed on how the FAIR principles could be applied to improved regulatory acceptance of predictions from such models. Eighteen principles have been developed that cover all aspects of FAIR. It is intended that FAIRification of in silico predictive models for toxicology will increase their use and acceptance.

Probabilistic modelling of developmental neurotoxicity based on a simplified adverse outcome pathway network.

In a century where toxicology and chemical risk assessment are embracing alternative methods to animal testing, there is an opportunity to understand the causal factors of neurodevelopmental disorders such as learning and memory disabilities in children, as a foundation to predict adverse effects. New testing paradigms, along with the advances in probabilistic modelling, can help with the formulation of mechanistically-driven hypotheses on how exposure to environmental chemicals could potentially lead to developmental neurotoxicity (DNT). This investigation aimed to develop a Bayesian hierarchical model of a simplified AOP network for DNT. The model predicted the probability that a compound induces each of three selected common key events (CKEs) of the simplified AOP network and the adverse outcome (AO) of DNT, taking into account correlations and causal relations informed by the key event relationships (KERs). A dataset of 88 compounds representing pharmaceuticals, industrial chemicals and pesticides was compiled including physicochemical properties as well as in silico and in vitro information. The Bayesian model was able to predict DNT potential with an accuracy of 76%, classifying the compounds into low, medium or high probability classes. The modelling workflow achieved three further goals: it dealt with missing values; accommodated unbalanced and correlated data; and followed the structure of a directed acyclic graph (DAG) to simulate the simplified AOP network. Overall, the model demonstrated the utility of Bayesian hierarchical modelling for the development of quantitative AOP (qAOP) models and for informing the use of new approach methodologies (NAMs) in chemical risk assessment.

Determination of "fitness-for-purpose" of quantitative structure-activity relationship (QSAR) models to predict (eco-)toxicological endpoints for regulatory use.

In silico models are used to predict toxicity and molecular properties in chemical safety assessment, gaining widespread regulatory use under a number of legislations globally. This study has rationalised previously published criteria to evaluate quantitative structure-activity relationships (QSARs) in terms of their uncertainty, variability and potential areas of bias, into ten assessment components, or higher level groupings. The components have been mapped onto specific regulatory uses (i.e. data gap filling for risk assessment, classification and labelling, and screening and prioritisation) identifying different levels of uncertainty that may be acceptable for each. Twelve published QSARs were evaluated using the components, such that their potential use could be identified. High uncertainty was commonly observed with the presentation of data, mechanistic interpretability, incorporation of toxicokinetics and the relevance of the data for regulatory purposes. The assessment components help to guide strategies that can be implemented to improve acceptability of QSARs through the reduction of uncertainties. It is anticipated that model developers could apply the assessment components from the model design phase (e.g. through problem formulation) through to their documentation and use. The application of the components provides the possibility to assess QSARs in a meaningful manner and demonstrate their fitness-for-purpose against pre-defined criteria.

Mapping Ligand-Shape Space for Protein-Ligand Systems: Distinguishing Key-in-Lock and Hand-in-Glove Proteins.

Many of the recently developed methods to study the shape of molecules permit one conformation of one molecule to be compared to another conformation of the same or a different molecule: a relative shape. Other methods provide an absolute description of the shape of a conformation that does not rely on comparisons or overlays. Any absolute description of shape can be used to generate a self-organizing map (shape map) that places all molecular shapes relative to one another; in the studies reported here, the shape fingerprint and ultrafast shape recognition methods are employed to create such maps. In the shape maps, molecules that are near one another have similar shapes, and the maps for the 102 targets in the DUD-E set have been generated. By examining the distribution of actives in comparison with their physical-property-matched decoys, we show that the proteins of key-in-lock type (relatively rigid receptor and ligand) can be distinguished from those that are more of a hand-in-glove type (more flexible receptor and ligand). These are linked to known differences in protein flexibility and binding-site size.

Comparison of the predictive nature of the Genomic Allergen Rapid Detection (GARD) assay with mammalian assays in determining the skin sensitisation potential of agrochemical active ingredients.

Alternatives to mammalian testing are highly desirable to predict the skin sensitisation potential of agrochemical active ingredients (AI). The GARD assay, a stimulated, dendritic cell-like, cell line measuring genomic signatures, was evaluated using twelve AIs (seven sensitisers and five non-sensitisers) and the results compared with historical results from guinea pig or local lymph node assay (LLNA) studies. Initial GARD results suggested 11/12 AIs were sensitisers and six concurred with mammalian data. Conformal predictions changed one AI to a non-sensitiser. An AI identified as non-sensitising in the GARD assay was considered a potent sensitiser in the LLNA. In total 7/12 GARD results corresponded with mammalian data. AI chemistries might not be comparable to the GARD training set in terms of applicability domains. Whilst the GARD assay can replace mammalian tests for skin sensitisation evaluation for compounds including cosmetic ingredients, further work in agrochemical chemistries is needed for this assay to be a viable replacement to animal testing. The work conducted here is, however, considered exploratory research and the methodology needs further development to be validated for agrochemicals. Mammalian and other alternative assays for regulatory safety assessments of AIs must provide confidence to assign the appropriate classification for human health protection.

A Review of In Silico Tools as Alternatives to Animal Testing: Principles, Resources and Applications.

Across the spectrum of industrial sectors, including pharmaceuticals, chemicals, personal care products, food additives and their associated regulatory agencies, there is a need to develop robust and reliable methods to reduce or replace animal testing. It is generally recognised that no single alternative method will be able to provide a one-to-one replacement for assays based on more complex toxicological endpoints. Hence, information from a combination of techniques is required. A greater understanding of the time and concentration-dependent mechanisms, underlying the interactions between chemicals and biological systems, and the sequence of events that can lead to apical effects, will help to move forward the science of reducing and replacing animal experiments. In silico modelling, in vitro assays, high-throughput screening, organ-on-a-chip technology, omics and mathematical biology, can provide complementary information to develop a complete picture of the potential response of an organism to a chemical stressor. Adverse outcome pathways (AOPs) and systems biology frameworks enable relevant information from diverse sources to be logically integrated. While individual researchers do not need to be experts across all disciplines, it is useful to have a fundamental understanding of what other areas of science have to offer, and how knowledge can be integrated with other disciplines. The purpose of this review is to provide those who are unfamiliar with predictive in silico tools, with a fundamental understanding of the underlying theory. Current applications, software, barriers to acceptance, new developments and the use of integrated approaches are all discussed, with additional resources being signposted for each of the topics.

Quantitative adverse outcome pathway (qAOP) models for toxicity prediction.

The quantitative adverse outcome pathway (qAOP) concept is gaining interest due to its potential regulatory applications in chemical risk assessment. Even though an increasing number of qAOP models are being proposed as computational predictive tools, there is no framework to guide their development and assessment. As such, the objectives of this review were to: (i) analyse the definitions of qAOPs published in the scientific literature, (ii) define a set of common features of existing qAOP models derived from the published definitions, and (iii) identify and assess the existing published qAOP models and associated software tools. As a result, five probabilistic qAOPs and ten mechanistic qAOPs were evaluated against the common features. The review offers an overview of how the qAOP concept has advanced and how it can aid toxicity assessment in the future. Further efforts are required to achieve validation, harmonisation and regulatory acceptance of qAOP models.

Development and analysis of an adverse outcome pathway network for human neurotoxicity.

An adverse outcome pathway (AOP) network is an attempt to represent the complexity of systems toxicology. This study illustrates how an AOP network can be derived and analysed in terms of its topological features to guide research and support chemical risk assessment. A four-step workflow describing general design principles and applied design principles was established and implemented. An AOP network linking nine linear AOPs was mapped and made available in AOPXplorer. The resultant AOP network was modelled and analysed in terms of its topological features, including level of degree, eccentricity and betweenness centrality. Several well-connected KEs were identified, and cell injury/death was established as the most hyperlinked KE across the network. The derived network expands the utility of linear AOPs to better understand signalling pathways involved in developmental and adult/ageing neurotoxicity. The results provide a solid basis to guide the development of in vitro test method batteries, as well as further quantitative modelling of key events (KEs) and key event relationships (KERs) in the AOP network, with an eventual aim to support hazard characterisation and chemical risk assessment.

Modelling changes in glutathione homeostasis as a function of quinone redox metabolism.

Redox cycling is an understated mechanism of toxicity associated with a plethora of xenobiotics, responsible for preventing the effective treatment of serious conditions such as malaria and cardiomyopathy. Quinone compounds are notorious redox cyclers, present in drugs such as doxorubicin, which is used to treat a host of human cancers. However, the therapeutic index of doxorubicin is undermined by dose-dependent cardiotoxicity, which may be a function of futile redox cycling. In this study, a doxorubicin-specific in silico quinone redox metabolism model is described. Doxorubicin-GSH adduct formation kinetics are thermodynamically estimated from its reduction potential, while the remainder of the model is parameterised using oxygen consumption rate data, indicative of hydroquinone auto-oxidation. The model is then combined with a comprehensive glutathione metabolism model, facilitating the simulation of quinone redox cycling, and adduct-induced GSH depletion. Simulations suggest that glutathione pools are most sensitive to exposure duration at pharmacologically and supra-pharmacologically relevant doxorubicin concentrations. The model provides an alternative method of investigating and quantifying redox cycling induced oxidative stress, circumventing the experimental difficulties of measuring and tracking radical species. This in silico framework provides a platform from which GSH depletion can be explored as a function of a compound's physicochemical properties.

A critical review of adverse effects to the kidney: mechanisms, data sources, and in silico tools to assist prediction.

INTRODUCTION: The kidney is a major target for toxicity elicited by pharmaceuticals and environmental pollutants. Standard testing which often does not investigate underlying mechanisms has proven not to be an adequate hazard assessment approach. As such, there is an opportunity for the application of computational approaches that utilize multiscale data based on the Adverse Outcome Pathway (AOP) paradigm, coupled with an understanding of the chemistry underpinning the molecular initiating event (MIE) to provide a deep understanding of how structural fragments of molecules relate to specific mechanisms of nephrotoxicity. Aims covered: The aim of this investigation was to review the current scientific landscape related to computational methods, including mechanistic data, AOPs, publicly available knowledge bases and current in silico models, for the assessment of pharmaceuticals and other chemicals with regard to their potential to elicit nephrotoxicity. A list of over 250 nephrotoxicants enriched with, where possible, mechanistic and AOP-derived understanding was compiled. Expert opinion: Whilst little mechanistic evidence has been translated into AOPs, this review identified a number of data sources of in vitro, in vivo, and human data that may assist in the development of in silico models which in turn may shed light on the interrelationships between nephrotoxicity mechanisms.

Development of a Decision Tree for Mitochondrial Dysfunction: Uncoupling of Oxidative Phosphorylation.

Mitochondrial dysfunction is the result of a number of processes including the uncoupling of oxidative phosphorylation. This study outlines the development of a decision tree-based profiling scheme capable of assigning chemicals to one of six confidence-based categories. The decision tree is based on a set of structural alerts and physicochemical boundaries identified from a detailed study of the literature. The physicochemical boundaries define a chemical relationship with both log P and p Ka. The study also outlines how the decision tree can be used to profile databases through an analysis of the publically available databases in the OECD QSAR Toolbox. This analysis enabled a set of additional structural alerts to be identified that are of concern for protonophoric ability. The decision tree will be incorporated in the OECD QSAR Toolbox V4.3. The intended usage is to group the chemicals into categories of chronic human health and environmental toxicological end points.

A Mechanistic Framework for Integrating Chemical Structure and High-Throughput Screening Results to Improve Toxicity Predictions.

Adverse Outcome Pathways (AOPs) establish a connection between a molecular initiating event (MIE) and an adverse outcome. Detailed understanding of the MIE provides the ideal data for determining chemical properties required to elicit the MIE. This study utilized high-throughput screening data from the ToxCast program, coupled with chemical structural information, to generate chemical clusters using three similarity methods pertaining to nine MIEs within an AOP network for hepatic steatosis. Three case studies demonstrate the utility of the mechanistic information held by the MIE for integrating biological and chemical data. Evaluation of the chemical clusters activating the glucocorticoid receptor identified activity differences in chemicals within a cluster. Comparison of the estrogen receptor results with previous work showed that bioactivity data and structural alerts can be combined to improve predictions in a customizable way where bioactivity data are limited. The aryl hydrocarbon receptor (AHR) highlighted that while structural data can be used to offset limited data for new screening efforts, not all ToxCast targets have sufficient data to define robust chemical clusters. In this context, an alternative to additional receptor assays is proposed where assays for proximal key events downstream of AHR activation could be used to enhance confidence in active calls. These case studies illustrate how the AOP framework can support an iterative process whereby in vitro toxicity testing and chemical structure can be combined to improve toxicity predictions. In vitro assays can inform the development of structural alerts linking chemical structure to toxicity. Consequently, structurally related chemical groups can facilitate identification of assays that would be informative for a specific MIE. Together, these activities form a virtuous cycle where the mechanistic basis for the in vitro results and the breadth of the structural alerts continually improve over time to better predict activity of chemicals for which limited toxicity data exist.

Turbocharging Matched Molecular Pair Analysis: Optimizing the Identification and Analysis of Pairs.

We have applied the two most commonly used methods for automatic matched pair identification, obtained the optimum settings, and discovered that the two methods are synergistic. A turbocharging approach to matched pair analysis is advocated in which a first round (a conservative categorical approach that uses an analogy with coin flips, heads corresponding to an increase in a measured property, tails to a decrease, and a biased coin to a structural change that reliably causes a change in that property) provides the settings for a second round (which uses the magnitude of the change in properties). Increased chemical specificity allows reliable knowledge to be extracted from smaller sets of pairs, and an assay-specific upper limit can be placed on the number of pairs required before adequate sampling of variability has been achieved.

Thresholds of Toxicological Concern for cosmetics-related substances: New database, thresholds, and enrichment of chemical space.

A new dataset of cosmetics-related chemicals for the Threshold of Toxicological Concern (TTC) approach has been compiled, comprising 552 chemicals with 219, 40, and 293 chemicals in Cramer Classes I, II, and III, respectively. Data were integrated and curated to create a database of No-/Lowest-Observed-Adverse-Effect Level (NOAEL/LOAEL) values, from which the final COSMOS TTC dataset was developed. Criteria for study inclusion and NOAEL decisions were defined, and rigorous quality control was performed for study details and assignment of Cramer classes. From the final COSMOS TTC dataset, human exposure thresholds of 42 and 7.9 µg/kg-bw/day were derived for Cramer Classes I and III, respectively. The size of Cramer Class II was insufficient for derivation of a TTC value. The COSMOS TTC dataset was then federated with the dataset of Munro and colleagues, previously published in 1996, after updating the latter using the quality control processes for this project. This federated dataset expands the chemical space and provides more robust thresholds. The 966 substances in the federated database comprise 245, 49 and 672 chemicals in Cramer Classes I, II and III, respectively. The corresponding TTC values of 46, 6.2 and 2.3 µg/kg-bw/day are broadly similar to those of the original Munro dataset.

In Silico Prediction of Organ Level Toxicity: Linking Chemistry to Adverse Effects.

In silico methods to predict toxicity include the use of (Quantitative) Structure-Activity Relationships ((Q)SARs) as well as grouping (category formation) allowing for read-across. A challenging area for in silico modelling is the prediction of chronic toxicity and the No Observed (Adverse) Effect Level (NO(A)EL) in particular. A proposed solution to the prediction of chronic toxicity is to consider organ level effects, as opposed to modelling the NO(A)EL itself. This review has focussed on the use of structural alerts to identify potential liver toxicants. In silico profilers, or groups of structural alerts, have been developed based on mechanisms of action and informed by current knowledge of Adverse Outcome Pathways. These profilers are robust and can be coded computationally to allow for prediction. However, they do not cover all mechanisms or modes of liver toxicity and recommendations for the improvement of these approaches are given.

Validation of a Fragment-Based Profiler for Thiol Reactivity for the Prediction of Toxicity: Skin Sensitization and Tetrahymena pyriformis.

This study outlines the use of a recently developed fragment-based thiol reactivity profiler for Michael acceptors to predict toxicity toward Tetrahymena pyriformis and skin sensitization potency as determined in the Local Lymph Node Assay (LLNA). The results showed that the calculated reactivity parameter from the profiler, -log RC50(calc), was capable of predicting toxicity for both end points with excellent statistics. However, the study highlighted the importance of a well-defined applicability domain for each end point. In terms of Tetrahymena pyriformis, this domain was defined in terms of how fast or slowly a given Michael acceptor reacts with thiol leading to two separate quantitative structure-activity models. The first, for fast reacting chemicals required only -log RC50(calc) as a descriptor, while the second required the addition of a descriptor for hydrophobicity. Modeling of the LLNA required only a single descriptor, -log RC50(calc), enabling potency to be predicted. The applicability domain excluded chemicals capable of undergoing polymerization and those that were predicted to be volatile. The modeling results for both end points, using the -log RC50(calc) value from the profiler, were in keeping with previously published studies that have utilized experimentally determined measurements of reactivity. These results demonstrate that the output from the fragment-based thiol reactivity profiler can be used to develop quantitative structure-activity relationship models where reactivity toward thiol is a driver of toxicity.

Development of a Fragment-Based in Silico Profiler for Michael Addition Thiol Reactivity.

The Adverse Outcome Pathway (AOP) paradigm details the existing knowledge that links the initial interaction between a chemical and a biological system, termed the molecular initiating event (MIE), through a series of intermediate events, to an adverse effect. An important example of a well-defined MIE is the formation of a covalent bond between a biological nucleophile and an electrophilic compound. This particular MIE has been associated with various toxicological end points such as acute aquatic toxicity, skin sensitization, and respiratory sensitization. This study has investigated the calculated parameters that are required to predict the rate of chemical bond formation (reactivity) of a dataset of Michael acceptors. Reactivity of these compounds toward glutathione was predicted using a combination of a calculated activation energy value (Eact, calculated using density functional theory (DFT) calculation at the B3YLP/6-31G+(d) level of theory, and solvent-accessible surface area values (SAS) at the α carbon. To further develop the method, a fragment-based algorithm was developed enabling the reactivity to be predicted for Michael acceptors without the need to perform the time-consuming DFT calculations. Results showed the developed fragment method was successful in predicting the reactivity of the Michael acceptors excluding two sets of chemicals: volatile esters with an extended substituent at the ß-carbon and chemicals containing a conjugated benzene ring as part of the polarizing group. Additionally the study also demonstrated the ease with which the approach can be extended to other chemical classes by the calculation of additional fragments and their associated Eact and SAS values. The resulting method is likely to be of use in regulatory toxicology tools where an understanding of covalent bond formation as a potential MIE is important within the AOP paradigm.

Adverse Outcome Pathway (AOP) Informed Modeling of Aquatic Toxicology: QSARs, Read-Across, and Interspecies Verification of Modes of Action.

Alternative approaches have been promoted to reduce the number of vertebrate and invertebrate animals required for the assessment of the potential of compounds to cause harm to the aquatic environment. A key philosophy in the development of alternatives is a greater understanding of the relevant adverse outcome pathway (AOP). One alternative method is the fish embryo toxicity (FET) assay. Although the trends in potency have been shown to be equivalent in embryo and adult assays, a detailed mechanistic analysis of the toxicity data has yet to be performed; such analysis is vital for a full understanding of the AOP. The research presented herein used an updated implementation of the Verhaar scheme to categorize compounds into AOP-informed categories. These were then used in mechanistic (quantitative) structure-activity relationship ((Q)SAR) analysis to show that the descriptors governing the distinct mechanisms of acute fish toxicity are capable of modeling data from the FET assay. The results show that compounds do appear to exhibit the same mechanisms of toxicity across life stages. Thus, this mechanistic analysis supports the argument that the FET assay is a suitable alternative testing strategy for the specified mechanisms and that understanding the AOPs is useful for toxicity prediction across test systems.

Development of an in Silico Profiler for Mitochondrial Toxicity.

This study outlines the analysis of mitochondrial toxicity for a variety of pharmaceutical drugs extracted from Zhang et al. ((2009) Toxicol. In Vitro, 23, 134-140). These chemicals were grouped into categories based upon structural similarity. Subsequently, mechanistic analysis was undertaken for each category to identify the molecular initiating event driving mitochondrial toxicity. The mechanistic information elucidated during the analysis enabled mechanism-based structural alerts to be developed and combined together to form an in silico profiler. This profiler is envisaged to be used to develop chemical categories based upon similar mechanisms as part of the adverse outcome pathway paradigm. Additionally, the profiler could be utilized in screening large data sets in order to identify chemicals with the potential to induce mitochondrial toxicity.