Evaluation of QSAR models for predicting mutagenicity: outcome of the Second Ames/QSAR international challenge project.

Quantitative structure-activity relationship (QSAR) models are powerful in silico tools for predicting the mutagenicity of unstable compounds, impurities and metabolites that are difficult to examine using the Ames test. Ideally, Ames/QSAR models for regulatory use should demonstrate high sensitivity, low false-negative rate and wide coverage of chemical space. To promote superior model development, the Division of Genetics and Mutagenesis, National Institute of Health Sciences, Japan (DGM/NIHS), conducted the Second Ames/QSAR International Challenge Project (2020-2022) as a successor to the First Project (2014-2017), with 21 teams from 11 countries participating. The DGM/NIHS provided a curated training dataset of approximately 12,000 chemicals and a trial dataset of approximately 1,600 chemicals, and each participating team predicted the Ames mutagenicity of each trial chemical using various Ames/QSAR models. The DGM/NIHS then provided the Ames test results for trial chemicals to assist in model improvement. Although overall model performance on the Second Project was not superior to that on the First, models from the eight teams participating in both projects achieved higher sensitivity than models from teams participating in only the Second Project. Thus, these evaluations have facilitated the development of QSAR models.

Evaluation of non-commercial models for genotoxicity and carcinogenicity in the assessment of EFSA's databases.

Over the past years, the European Food Safety Authority (EFSA) released to the public domain several databases, with the main objectives of collecting and storing hazard data on the substances considered in EFSA's risk assessment and secondly to serve as a basis for further development of in silico tools such as quantitative structure-activity relationship (QSAR) models. In this work, we evaluated the ability of freely available QSAR models to estimate genotoxicity and carcinogenicity properties and their possible use for screening purposes on three different EFSA's databases. With an accuracy close to 90%, the results showed good capabilities of QSAR models to predict genotoxicity in terms of bacterial reverse mutation test, while statistics for in vivo micronucleus test are not satisfactory (accuracy in the predictions close to 50%). Interestingly, results on the carcinogenicity assessment showed an accuracy in prediction close to 70% for the best models. In addition, an example of the potential application of in silico models is presented in order to provide a preliminary screening of genotoxicity properties of botanicals intended for use as food supplements.

Could deep learning in neural networks improve the QSAR models?

Assessing chemical toxicity is a multidisciplinary process, traditionally involving in vivo, in vitro and in silico tests. Currently, toxicological goal is to reduce new tests on chemicals, exploiting all information yet available. Recent advancements in machine learning and deep neural networks allow computers to automatically mine patterns and learn from data. This technology, applied to (Q)SAR model development, leads to discover by learning the structural-chemical-biological relationships and the emergent properties. Starting from Toxception, a deep neural network predicting activity from the chemical graph image, we designed SmilesNet, a recurrent neural network taking SMILES as the only input. We then integrated the two networks into C-Tox network to make the final classification. Results of our networks, trained on a ~20K molecule dataset with Ames test experimental values, match or even outperform the current state of the art. We also extract knowledge from the networks and compare it with the available mutagenic structural alerts. The advantage over traditional QSAR modelling is that our models automatically extract the features without using descriptors. Nevertheless, the model is successful if large numbers of examples are provided and computation is more complex than in classical methods.

A large comparison of integrated SAR/QSAR models of the Ames test for mutagenicity$.

Results from the Ames test are the first outcome considered to assess the possible mutagenicity of substances. Many QSAR models and structural alerts are available to predict this endpoint. From a regulatory point of view, the recommendation from international authorities is to consider the predictions of more than one model and to combine results in order to develop conclusions about the mutagenicity risk posed by chemicals. However, the results of those models are often conflicting, and the existing inconsistency in the predictions requires intelligent strategies to integrate them. In our study, we evaluated different strategies for combining results of models for Ames mutagenicity, starting from a set of 10 diverse individual models, each built on a dataset of around 6000 compounds. The novelty of our study is that we collected a much larger set of about 18,000 compounds and used the new data to build a family of integrated models. These integrations used probabilistic approaches, decision theory, machine learning, and voting strategies in the integration scheme. Results are discussed considering balanced or conservative perspectives, regarding the possible uses for different purposes, including screening of large collection of substances for prioritization.

Results of a round-robin exercise on read-across.

A round-robin exercise was conducted within the CALEIDOS LIFE project. The participants were invited to assess the hazard posed by a substance, applying in silico methods and read-across approaches. The exercise was based on three endpoints: mutagenicity, bioconcentration factor and fish acute toxicity. Nine chemicals were assigned for each endpoint and the participants were invited to complete a specific questionnaire communicating their conclusions. The interesting aspect of this exercise is the justification behind the answers more than the final prediction in itself. Which tools were used? How did the approach selected affect the final answer?

In silico exploratory study using structure-activity relationship models and metabolic information for prediction of mutagenicity based on the Ames test and rodent micronucleus assay.

The mutagenic potential of chemicals is a cause of growing concern, due to the possible impact on human health. In this paper we have developed a knowledge-based approach, combining information from structure-activity relationship (SAR) and metabolic triggers generated from the metabolic fate of chemicals in biological systems for prediction of mutagenicity in vitro based on the Ames test and in vivo based on the rodent micronucleus assay. In the first part of the work, a model was developed, which comprises newly generated SAR rules and a set of metabolic triggers. These SAR rules and metabolic triggers were further externally validated to predict mutagenicity in vitro, with metabolic triggers being used only to predict mutagenicity of chemicals, which were predicted unknown, by SARpy. Hence, this model has a higher accuracy than the SAR model, with an accuracy of 89% for the training set and 75% for the external validation set. Subsequently, the results of the second part of this work enlist a set of metabolic triggers for prediction of mutagenicity in vivo, based on the rodent micronucleus assay. Finally, the results of the third part enlist a list of metabolic triggers to find similarities and differences in the mutagenic response of chemicals in vitro and in vivo.