Assessing the utility of common arguments used in expert review of in silico predictions as part of ICH M7 assessments.

Expert review of two predictions, made by complementary (quantitative) structure-activity relationship models, to an overall conclusion is a key component of using in silico tools to assess the mutagenic potential of impurities as part of the ICH M7 guideline. In lieu of a specified protocol, numerous publications have presented best practise guides, often indicating the occurrence of common prediction scenarios and the evidence required to resolve them. A semi-automated expert review tool has been implemented in Lhasa Limited's Nexus platform following collation of these common arguments and assignment to the associated prediction scenarios made by Derek Nexus and Sarah Nexus. Using datasets primarily donated by pharmaceutical companies, an automated analysis of the frequency these prediction scenarios occur, and the likelihood of the associated arguments assigning the correct resolution, could then be conducted. This article highlights that a relatively small number of common arguments may be used to accurately resolve many prediction scenarios to a single conclusion. The use of a standardised method of argumentation and assessment of evidence for a given impurity is proposed to improve the efficiency and consistency of expert review as part of an ICH M7 submission.

Federated Learning in Computational Toxicology: An Industrial Perspective on the Effiris Hackathon.

In silico approaches have acquired a towering role in pharmaceutical research and development, allowing laboratories all around the world to design, create, and optimize novel molecular entities with unprecedented efficiency. From a toxicological perspective, computational methods have guided the choices of medicinal chemists toward compounds displaying improved safety profiles. Even if the recent advances in the field are significant, many challenges remain active in the on-target and off-target prediction fields. Machine learning methods have shown their ability to identify molecules with safety concerns. However, they strongly depend on the abundance and diversity of data used for their training. Sharing such information among pharmaceutical companies remains extremely limited due to confidentiality reasons, but in this scenario, a recent concept named "federated learning" can help overcome such concerns. Within this framework, it is possible for companies to contribute to the training of common machine learning algorithms, using, but not sharing, their proprietary data. Very recently, Lhasa Limited organized a hackathon involving several industrial partners in order to assess the performance of their federated learning platform, called "Effiris". In this paper, we share our experience as Roche in participating in such an event, evaluating the performance of the federated algorithms and comparing them with those coming from our in-house-only machine learning models. Our aim is to highlight the advantages of federated learning and its intrinsic limitations and also suggest some points for potential improvements in the method.

Increasing the Value of Data Within a Large Pharmaceutical Company Through In Silico Models.

The present contribution describes how in silico models and methods are applied at different stages of the drug discovery process in the pharmaceutical industry. A description of the most relevant computational methods and tools is given along with an evaluation of their performance in the assessment of potential genotoxic impurities and the prediction of off-target in vitro pharmacology. The challenges of predicting the outcome of highly complex in vivo studies are discussed followed by considerations on how novel ways to manage, store, exchange, and analyze data may advance knowledge and facilitate modeling efforts. In this context, the current status of broad data sharing initiatives, namely, eTOX and eTransafe, will be described along with related projects that could significantly reduce the use of animals in drug discovery in the future.

Preclinical Characterization of a Next-Generation Brain Permeable, Paradox Breaker BRAF Inhibitor.

PURPOSE: Disease progression in BRAF V600E/K positive melanomas to approved BRAF/MEK inhibitor therapies is associated with the development of resistance mediated by RAF dimer inducing mechanisms. Moreover, progressing disease after BRAFi/MEKi frequently involves brain metastasis. Here we present the development of a novel BRAF inhibitor (Compound Ia) designed to address the limitations of available BRAFi/MEKi. EXPERIMENTAL DESIGN: The novel, brain penetrant, paradox breaker BRAFi is comprehensively characterized in vitro, ex vivo, and in several preclinical in vivo models of melanoma mimicking peripheral disease, brain metastatic disease, and acquired resistance to first-generation BRAFi. RESULTS: Compound Ia manifested elevated potency and selectivity, which triggered cytotoxic activity restricted to BRAF-mutated models and did not induce RAF paradoxical activation. In comparison to approved BRAFi at clinical relevant doses, this novel agent showed a substantially improved activity in a number of diverse BRAF V600E models. In addition, as a single agent, it outperformed a currently approved BRAFi/MEKi combination in a model of acquired resistance to clinically available BRAFi. Compound Ia presents high central nervous system (CNS) penetration and triggered evident superiority over approved BRAFi in a macro-metastatic and in a disseminated micro-metastatic brain model. Potent inhibition of MAPK by Compound Ia was also demonstrated in patient-derived tumor samples. CONCLUSIONS: The novel BRAFi demonstrates preclinically the potential to outperform available targeted therapies for the treatment of BRAF-mutant tumors, thus supporting its clinical investigation.

Principles and Procedures for Assessment of Acute Toxicity Incorporating In Silico Methods.

Acute toxicity in silico models are being used to support an increasing number of application areas including (1) product research and development, (2) product approval and registration as well as (3) the transport, storage and handling of chemicals. The adoption of such models is being hindered, in part, because of a lack of guidance describing how to perform and document an in silico analysis. To address this issue, a framework for an acute toxicity hazard assessment is proposed. This framework combines results from different sources including in silico methods and in vitro or in vivo experiments. In silico methods that can assist the prediction of in vivo outcomes (i.e., LD50) are analyzed concluding that predictions obtained using in silico approaches are now well-suited for reliably supporting assessment of LD50-based acute toxicity for the purpose of GHS classification. A general overview is provided of the endpoints from in vitro studies commonly evaluated for predicting acute toxicity (e.g., cytotoxicity/cytolethality as well as assays targeting specific mechanisms). The increased understanding of pathways and key triggering mechanisms underlying toxicity and the increased availability of in vitro data allow for a shift away from assessments solely based on endpoints such as LD50, to mechanism-based endpoints that can be accurately assessed in vitro or by using in silico prediction models. This paper also highlights the importance of an expert review of all available information using weight-of-evidence considerations and illustrates, using a series of diverse practical use cases, how in silico approaches support the assessment of acute toxicity.

In Silico Approaches In Carcinogenicity Hazard Assessment: Current Status and Future Needs.

Historically, identifying carcinogens has relied primarily on tumor studies in rodents, which require enormous resources in both money and time. In silico models have been developed for predicting rodent carcinogens but have not yet found general regulatory acceptance, in part due to the lack of a generally accepted protocol for performing such an assessment as well as limitations in predictive performance and scope. There remains a need for additional, improved in silico carcinogenicity models, especially ones that are more human-relevant, for use in research and regulatory decision-making. As part of an international effort to develop in silico toxicological protocols, a consortium of toxicologists, computational scientists, and regulatory scientists across several industries and governmental agencies evaluated the extent to which in silico models exist for each of the recently defined 10 key characteristics (KCs) of carcinogens. This position paper summarizes the current status of in silico tools for the assessment of each KC and identifies the data gaps that need to be addressed before a comprehensive in silico carcinogenicity protocol can be developed for regulatory use.

Skin sensitization in silico protocol.

The assessment of skin sensitization has evolved over the past few years to include in vitro assessments of key events along the adverse outcome pathway and opportunistically capitalize on the strengths of in silico methods to support a weight of evidence assessment without conducting a test in animals. While in silico methods vary greatly in their purpose and format; there is a need to standardize the underlying principles on which such models are developed and to make transparent the implications for the uncertainty in the overall assessment. In this contribution, the relationship between skin sensitization relevant effects, mechanisms, and endpoints are built into a hazard assessment framework. Based on the relevance of the mechanisms and effects as well as the strengths and limitations of the experimental systems used to identify them, rules and principles are defined for deriving skin sensitization in silico assessments. Further, the assignments of reliability and confidence scores that reflect the overall strength of the assessment are discussed. This skin sensitization protocol supports the implementation and acceptance of in silico approaches for the prediction of skin sensitization.

Genotoxicity Assessment of Drug Metabolites in the Context of MIST and Beyond.

While there are dedicated guidelines for industry regarding the assessment of the genotoxic potential of new pharmaceuticals and impurities, and the general safety assessment of major drug metabolites, only limited guidance exists on the assessment of potential genotoxic minor drug metabolites. In this Perspective, we discuss challenges associated with assessing the genotoxic potential of human metabolites and share five case studies within the context of an "aware-avoid-assess" paradigm. A special focus is on a class of potentially genotoxic carcinogens, aromatic amines (arylamines and anilines). This compound class is frequently used as building blocks and may show up as impurities, metabolites, or degradants in pharmaceuticals. We propose several recommendations that should help project teams at different stages of pharmaceutical development. In most cases, proactive interactions with the relevant health authority should be considered to endorse the proposed genotoxicity assessment strategy for minor drug metabolites.

Genetic toxicology in silico protocol.

In silico toxicology (IST) approaches to rapidly assess chemical hazard, and usage of such methods is increasing in all applications but especially for regulatory submissions, such as for assessing chemicals under REACH as well as the ICH M7 guideline for drug impurities. There are a number of obstacles to performing an IST assessment, including uncertainty in how such an assessment and associated expert review should be performed or what is fit for purpose, as well as a lack of confidence that the results will be accepted by colleagues, collaborators and regulatory authorities. To address this, a project to develop a series of IST protocols for different hazard endpoints has been initiated and this paper describes the genetic toxicity in silico (GIST) protocol. The protocol outlines a hazard assessment framework including key effects/mechanisms and their relationships to endpoints such as gene mutation and clastogenicity. IST models and data are reviewed that support the assessment of these effects/mechanisms along with defined approaches for combining the information and evaluating the confidence in the assessment. This protocol has been developed through a consortium of toxicologists, computational scientists, and regulatory scientists across several industries to support the implementation and acceptance of in silico approaches.

Principles and procedures for handling out-of-domain and indeterminate results as part of ICH M7 recommended (Q)SAR analyses.

The International Council for Harmonization (ICH) M7 guideline describes a hazard assessment process for impurities that have the potential to be present in a drug substance or drug product. In the absence of adequate experimental bacterial mutagenicity data, (Q)SAR analysis may be used as a test to predict impurities' DNA reactive (mutagenic) potential. However, in certain situations, (Q)SAR software is unable to generate a positive or negative prediction either because of conflicting information or because the impurity is outside the applicability domain of the model. Such results present challenges in generating an overall mutagenicity prediction and highlight the importance of performing a thorough expert review. The following paper reviews pharmaceutical and regulatory experiences handling such situations. The paper also presents an analysis of proprietary data to help understand the likelihood of misclassifying a mutagenic impurity as non-mutagenic based on different combinations of (Q)SAR results. This information may be taken into consideration when supporting the (Q)SAR results with an expert review, especially when out-of-domain results are generated during a (Q)SAR evaluation.

In silico toxicology protocols.

The present publication surveys several applications of in silico (i.e., computational) toxicology approaches across different industries and institutions. It highlights the need to develop standardized protocols when conducting toxicity-related predictions. This contribution articulates the information needed for protocols to support in silico predictions for major toxicological endpoints of concern (e.g., genetic toxicity, carcinogenicity, acute toxicity, reproductive toxicity, developmental toxicity) across several industries and regulatory bodies. Such novel in silico toxicology (IST) protocols, when fully developed and implemented, will ensure in silico toxicological assessments are performed and evaluated in a consistent, reproducible, and well-documented manner across industries and regulatory bodies to support wider uptake and acceptance of the approaches. The development of IST protocols is an initiative developed through a collaboration among an international consortium to reflect the state-of-the-art in in silico toxicology for hazard identification and characterization. A general outline for describing the development of such protocols is included and it is based on in silico predictions and/or available experimental data for a defined series of relevant toxicological effects or mechanisms. The publication presents a novel approach for determining the reliability of in silico predictions alongside experimental data. In addition, we discuss how to determine the level of confidence in the assessment based on the relevance and reliability of the information.

The Use of In Silico Models Within a Large Pharmaceutical Company.

The present contribution describes how in silico models are applied at different stages of the drug discovery process in the pharmaceutical industry. A thorough description of the most relevant computational methods and tools is given along with an in-depth evaluation of their performance in the context of potential genotoxic impurities assessment.The challenges of predicting the outcome of highly complex studies are discussed followed by considerations on how novel ways to manage, store, share and analyze data may advance knowledge and facilitate modeling efforts.

Principles and procedures for implementation of ICH M7 recommended (Q)SAR analyses.

The ICH M7 guideline describes a consistent approach to identify, categorize, and control DNA reactive, mutagenic, impurities in pharmaceutical products to limit the potential carcinogenic risk related to such impurities. This paper outlines a series of principles and procedures to consider when generating (Q)SAR assessments aligned with the ICH M7 guideline to be included in a regulatory submission. In the absence of adequate experimental data, the results from two complementary (Q)SAR methodologies may be combined to support an initial hazard classification. This may be followed by an assessment of additional information that serves as the basis for an expert review to support or refute the predictions. This paper elucidates scenarios where additional expert knowledge may be beneficial, what such an expert review may contain, and how the results and accompanying considerations may be documented. Furthermore, the use of these principles and procedures to yield a consistent and robust (Q)SAR-based argument to support impurity qualification for regulatory purposes is described in this manuscript.

It's difficult, but important, to make negative predictions.

At the confluence of predictive and regulatory toxicologies, negative predictions may be the thin green line that prevents populations from being exposed to harm. Here, two novel approaches to making confident and robust negative in silico predictions for mutagenicity (as defined by the Ames test) have been evaluated. Analyses of 12 data sets containing >13,000 compounds, showed that negative predictivity is high (∼90%) for the best approach and features that either reduce the accuracy or certainty of negative predictions are identified as misclassified or unclassified respectively. However, negative predictivity remains high (and in excess of the prevalence of non-mutagens) even in the presence of these features, indicating that they are not flags for mutagenicity.

Evaluation of a statistics-based Ames mutagenicity QSAR model and interpretation of the results obtained.

The relative wealth of bacterial mutagenicity data available in the public literature means that in silico quantitative/qualitative structure activity relationship (QSAR) systems can readily be built for this endpoint. A good means of evaluating the performance of such systems is to use private unpublished data sets, which generally represent a more distinct chemical space than publicly available test sets and, as a result, provide a greater challenge to the model. However, raw performance metrics should not be the only factor considered when judging this type of software since expert interpretation of the results obtained may allow for further improvements in predictivity. Enough information should be provided by a QSAR to allow the user to make general, scientifically-based arguments in order to assess and overrule predictions when necessary. With all this in mind, we sought to validate the performance of the statistics-based in vitro bacterial mutagenicity prediction system Sarah Nexus (version 1.1) against private test data sets supplied by nine different pharmaceutical companies. The results of these evaluations were then analysed in order to identify findings presented by the model which would be useful for the user to take into consideration when interpreting the results and making their final decision about the mutagenic potential of a given compound.

Use of in silico systems and expert knowledge for structure-based assessment of potentially mutagenic impurities.

Genotoxicity hazard identification is part of the impurity qualification process for drug substances and products, the first step of which being the prediction of their potential DNA reactivity using in silico (quantitative) structure-activity relationship (Q)SAR models/systems. This white paper provides information relevant to the development of the draft harmonized tripartite guideline ICH M7 on potentially DNA-reactive/mutagenic impurities in pharmaceuticals and their application in practice. It explains relevant (Q)SAR methodologies as well as the added value of expert knowledge. Moreover, the predictive value of the different methodologies analyzed in two surveys conveyed in the US and European pharmaceutical industry is compared: most pharmaceutical companies used a rule-based expert system as their primary methodology, yielding negative predictivity values of ⩾78% in all participating companies. A further increase (>90%) was often achieved by an additional expert review and/or a second QSAR methodology. Also in the latter case, an expert review was mandatory, especially when conflicting results were obtained. Based on the available data, we concluded that a rule-based expert system complemented by either expert knowledge or a second (Q)SAR model is appropriate. A maximal transparency of the assessment process (e.g. methods, results, arguments of weight-of-evidence approach) achieved by e.g. data sharing initiatives and the use of standards for reporting will enable regulators to fully understand the results of the analysis. Overall, the procedures presented here for structure-based assessment are considered appropriate for regulatory submissions in the scope of ICH M7.

Comparative evaluation of in silico systems for ames test mutagenicity prediction: scope and limitations.

The predictive power of four commonly used in silico tools for mutagenicity prediction (DEREK, Toxtree, MC4PC, and Leadscope MA) was evaluated in a comparative manner using a large, high-quality data set, comprising both public and proprietary data (F. Hoffmann-La Roche) from 9,681 compounds tested in the Ames assay. Satisfactory performance statistics were observed on public data (accuracy, 66.4-75.4%; sensitivity, 65.2-85.2%; specificity, 53.1-82.9%), whereas a significant deterioration of sensitivity was observed in the Roche data (accuracy, 73.1-85.5%; sensitivity, 17.4-43.4%; specificity, 77.5-93.9%). As a general tendency, expert systems showed higher sensitivity and lower specificity when compared to QSAR-based tools, which displayed the opposite behavior. Possible reasons for the performance differences between the public and Roche data, relating to the experimentally inactive to active compound ratio and the different coverage of chemical space, are thoroughly discussed. Examples of peculiar chemical classes enriched in false negative or false positive predictions are given, and the results of the combined use of the prediction systems are described.

Comparative molecular dynamics simulations of HIV-1 integrase and the T66I/M154I mutant: binding modes and drug resistance to a diketo acid inhibitor.

HIV-1 IN is an essential enzyme for viral replication and an interesting target for the design of new pharmaceuticals for use in multidrug therapy of AIDS. L-731,988 is one of the most active molecules of the class of beta-diketo acids. Individual and combined mutations of HIV-1 IN at residues T66, S153, and M154 confer important degrees of resistance to one or more inhibitors belonging to this class. In an effort to understand the molecular mechanism of the resistance of T66I/M154I IN to the inhibitor L-731,988 and its specific binding modes, we have carried out docking studies, explicit solvent MD simulations, and binding free energy calculations. The inhibitor was docked against different protein conformations chosen from prior MD trajectories, resulting in 2 major orientations within the active site. MD simulations have been carried out for the T66I/M154I DM IN, DM IN in complex with L-731,988 in 2 different orientations, and 1QS4 IN in complex with L-731,988. The results of these simulations show a similar dynamical behavior between T66I/M154I IN alone and in complex with L-731,988, while significant differences are observed in the mobility of the IN catalytic loop (residues 138-149). Water molecules bridging the inhibitor to residues from the active site have been identified, and residue Gln62 has been found to play an important role in the interactions between the inhibitor and the protein. This work provides information about the binding modes of L-731,988, as well as insight into the mechanism of inhibitor-resistance in HIV-1 integrase.

Comparison of multiple molecular dynamics trajectories calculated for the drug-resistant HIV-1 integrase T66I/M154I catalytic domain.

HIV-1 integrase (IN) is an essential enzyme for the viral replication and an interesting target for the design of new pharmaceuticals for multidrug therapy of AIDS. Single and multiple mutations of IN at residues T66, S153, or M154 confer degrees of resistance to several inhibitors that prevent the enzyme from performing its normal strand transfer activity. Four different conformations of IN were chosen from a prior molecular dynamics (MD) simulation on the modeled IN T66I/M154I catalytic core domain as starting points for additional MD studies. The aim of this article is to understand the dynamic features that may play roles in the catalytic activity of the double mutant enzyme in the absence of any inhibitor. Moreover, we want to verify the influence of using different starting points on the MD trajectories and associated dynamical properties. By comparison of the trajectories obtained from these MD simulations we have demonstrated that the starting point does not affect the conformational space explored by this protein and that the time of the simulation is long enough to achieve convergence for this system.

HIV-1 integrase pharmacophore model derived from diverse classes of inhibitors.

A three-dimensional pharmacophore model has been generated for HIV-1 integrase (HIV-1 IN) from known inhibitors. A dataset consisting of 26 inhibitors was selected on the basis of the information content of the structures and activity data as required by the catalyst/HypoGen program. Our model was able to predict the activity of other known HIV-1 IN inhibitors not included in the model generation, and can be further used to identify structurally diverse compounds with desired biological activity by virtual screening.