Machine Learning Strategies When Transitioning between Biological Assays.

Machine learning is widely used in drug development to predict activity in biological assays based on chemical structure. However, the process of transitioning from one experimental setup to another for the same biological endpoint has not been extensively studied. In a retrospective study, we here explore different modeling strategies of how to combine data from the old and new assays when training conformal prediction models using data from hERG and NaV assays. We suggest to continuously monitor the validity and efficiency of models as more data is accumulated from the new assay and select a modeling strategy based on these metrics. In order to maximize the utility of data from the old assay, we propose a strategy that augments the proper training set of an inductive conformal predictor by adding data from the old assay but only having data from the new assay in the calibration set, which results in valid (well-calibrated) models with improved efficiency compared to other strategies. We study the results for varying sizes of new and old assays, allowing for discussion of different practical scenarios. We also conclude that our proposed assay transition strategy is more beneficial, and the value of data from the new assay is higher, for the harder case of regression compared to classification problems.

Skin sensitization in silico protocol.

The assessment of skin sensitization has evolved over the past few years to include in vitro assessments of key events along the adverse outcome pathway and opportunistically capitalize on the strengths of in silico methods to support a weight of evidence assessment without conducting a test in animals. While in silico methods vary greatly in their purpose and format; there is a need to standardize the underlying principles on which such models are developed and to make transparent the implications for the uncertainty in the overall assessment. In this contribution, the relationship between skin sensitization relevant effects, mechanisms, and endpoints are built into a hazard assessment framework. Based on the relevance of the mechanisms and effects as well as the strengths and limitations of the experimental systems used to identify them, rules and principles are defined for deriving skin sensitization in silico assessments. Further, the assignments of reliability and confidence scores that reflect the overall strength of the assessment are discussed. This skin sensitization protocol supports the implementation and acceptance of in silico approaches for the prediction of skin sensitization.

Predicting Ames Mutagenicity Using Conformal Prediction in the Ames/QSAR International Challenge Project.

Valid and predictive models for classifying Ames mutagenicity have been developed using conformal prediction. The models are Random Forest models using signature molecular descriptors. The investigation indicates, on excluding not-strongly mutagenic compounds (class B), that the validity for mutagenic compounds is increased for the predictions based on both public and the Division of Genetics and Mutagenesis, National Institute of Health Sciences of Japan (DGM/NIHS) data while less so when using only the latter data source. The former models only result in valid predictions for the majority, non-mutagenic, class whereas the latter models are valid for both classes, i.e. mutagenic and non-mutagenic compounds. These results demonstrate the importance of data consistency manifested through the superior predictive quality and validity of the models based only on DGM/NIHS generated data compared to a combination of this data with public data sources.

Accurate Hit Estimation for Iterative Screening Using Venn-ABERS Predictors.

Iterative screening has emerged as a promising approach to increase the efficiency of high-throughput screening (HTS) campaigns in drug discovery. By learning from a subset of the compound library, inferences on what compounds to screen next can be made by predictive models. One of the challenges of iterative screening is to decide how many iterations to perform. This is mainly related to difficulties in estimating the prospective hit rate in any given iteration. In this article, a novel method based on Venn-ABERS predictors is proposed. The method provides accurate estimates of the number of hits retrieved in any given iteration during an HTS campaign. The estimates provide the necessary information to support the decision on the number of iterations needed to maximize the screening outcome. Thus, this method offers a prospective screening strategy for early-stage drug discovery.

Genetic toxicology in silico protocol.

In silico toxicology (IST) approaches to rapidly assess chemical hazard, and usage of such methods is increasing in all applications but especially for regulatory submissions, such as for assessing chemicals under REACH as well as the ICH M7 guideline for drug impurities. There are a number of obstacles to performing an IST assessment, including uncertainty in how such an assessment and associated expert review should be performed or what is fit for purpose, as well as a lack of confidence that the results will be accepted by colleagues, collaborators and regulatory authorities. To address this, a project to develop a series of IST protocols for different hazard endpoints has been initiated and this paper describes the genetic toxicity in silico (GIST) protocol. The protocol outlines a hazard assessment framework including key effects/mechanisms and their relationships to endpoints such as gene mutation and clastogenicity. IST models and data are reviewed that support the assessment of these effects/mechanisms along with defined approaches for combining the information and evaluating the confidence in the assessment. This protocol has been developed through a consortium of toxicologists, computational scientists, and regulatory scientists across several industries to support the implementation and acceptance of in silico approaches.

A novel multi-parametric high content screening assay in ciPTEC-OAT1 to predict drug-induced nephrotoxicity during drug discovery.

Drug-induced nephrotoxicity is a major concern in the clinic and hampers the use of available treatments as well as the development of innovative medicines. It is typically discovered late during drug development, which reflects a lack of in vitro nephrotoxicity assays available that can be employed readily in early drug discovery, to identify and hence steer away from the risk. Here, we report the development of a high content screening assay in ciPTEC-OAT1, a proximal tubular cell line that expresses several relevant renal transporters, using five fluorescent dyes to quantify cell health parameters. We used a validation set of 62 drugs, tested across a relevant concentration range compared to their exposure in humans, to develop a model that integrates multi-parametric data and drug exposure information, which identified most proximal tubular toxic drugs tested (sensitivity 75%) without any false positives (specificity 100%). Due to the relatively high throughput (straight-forward assay protocol, 96-well format, cost-effective) the assay is compatible with the needs in the early drug discovery setting to enable identification, quantification and subsequent mitigation of the risk for nephrotoxicity.

In silico toxicology protocols.

The present publication surveys several applications of in silico (i.e., computational) toxicology approaches across different industries and institutions. It highlights the need to develop standardized protocols when conducting toxicity-related predictions. This contribution articulates the information needed for protocols to support in silico predictions for major toxicological endpoints of concern (e.g., genetic toxicity, carcinogenicity, acute toxicity, reproductive toxicity, developmental toxicity) across several industries and regulatory bodies. Such novel in silico toxicology (IST) protocols, when fully developed and implemented, will ensure in silico toxicological assessments are performed and evaluated in a consistent, reproducible, and well-documented manner across industries and regulatory bodies to support wider uptake and acceptance of the approaches. The development of IST protocols is an initiative developed through a collaboration among an international consortium to reflect the state-of-the-art in in silico toxicology for hazard identification and characterization. A general outline for describing the development of such protocols is included and it is based on in silico predictions and/or available experimental data for a defined series of relevant toxicological effects or mechanisms. The publication presents a novel approach for determining the reliability of in silico predictions alongside experimental data. In addition, we discuss how to determine the level of confidence in the assessment based on the relevance and reliability of the information.

Predicting the Risk of Phospholipidosis with in Silico Models and an Image-Based in Vitro Screen.

The drug-induced accumulation of phospholipids in lysosomes of various tissues is predominantly observed in regular repeat dose studies, often after prolonged exposure, and further investigated in mechanistic studies prior to candidate nomination. The finding can cause delays in the discovery process inflicting high costs to the affected projects. This article presents an in vitro imaging-based method for early detection of phospholipidosis liability and a hybrid approach for early detection and risk mitigation of phospolipidosis utilizing the in vitro readout with in silico model prediction. A set of reference compounds with phospolipidosis annotation was used as an external validation set yielding accuracies between 77.6% and 85.3% for various in vitro and in silico models, respectively. By means of a small set of chemically diverse known drugs with in vivo phospholipidosis annotation, the advantages of combining different prediction methods to reach an overall improved phospholipidosis prediction will be discussed.

Applying Mondrian Cross-Conformal Prediction To Estimate Prediction Confidence on Large Imbalanced Bioactivity Data Sets.

Conformal prediction has been proposed as a more rigorous way to define prediction confidence compared to other application domain concepts that have earlier been used for QSAR modeling. One main advantage of such a method is that it provides a prediction region potentially with multiple predicted labels, which contrasts to the single valued (regression) or single label (classification) output predictions by standard QSAR modeling algorithms. Standard conformal prediction might not be suitable for imbalanced data sets. Therefore, Mondrian cross-conformal prediction (MCCP) which combines the Mondrian inductive conformal prediction with cross-fold calibration sets has been introduced. In this study, the MCCP method was applied to 18 publicly available data sets that have various imbalance levels varying from 1:10 to 1:1000 (ratio of active/inactive compounds). Our results show that MCCP in general performed well on bioactivity data sets with various imbalance levels. More importantly, the method not only provides confidence of prediction and prediction regions compared to standard machine learning methods but also produces valid predictions for the minority class. In addition, a compound similarity based nonconformity measure was investigated. Our results demonstrate that although it gives valid predictions, its efficiency is much worse than that of model dependent metrics.

Extending (Q)SARs to incorporate proprietary knowledge for regulatory purposes: A case study using aromatic amine mutagenicity.

Statistical-based and expert rule-based models built using public domain mutagenicity knowledge and data are routinely used for computational (Q)SAR assessments of pharmaceutical impurities in line with the approach recommended in the ICH M7 guideline. Knowledge from proprietary corporate mutagenicity databases could be used to increase the predictive performance for selected chemical classes as well as expand the applicability domain of these (Q)SAR models. This paper outlines a mechanism for sharing knowledge without the release of proprietary data. Primary aromatic amine mutagenicity was selected as a case study because this chemical class is often encountered in pharmaceutical impurity analysis and mutagenicity of aromatic amines is currently difficult to predict. As part of this analysis, a series of aromatic amine substructures were defined and the number of mutagenic and non-mutagenic examples for each chemical substructure calculated across a series of public and proprietary mutagenicity databases. This information was pooled across all sources to identify structural classes that activate or deactivate aromatic amine mutagenicity. This structure activity knowledge, in combination with newly released primary aromatic amine data, was incorporated into Leadscope's expert rule-based and statistical-based (Q)SAR models where increased predictive performance was demonstrated.