Rethinking the applicability domain analysis in QSAR models.

Notwithstanding the wide adoption of the OECD principles (or best practices) for QSAR modeling, disparities between in silico predictions and experimental results are frequent, suggesting that model predictions are often too optimistic. Of these OECD principles, the applicability domain (AD) estimation has been recognized in several reports in the literature to be one of the most challenging, implying that the actual reliability measures of model predictions are often unreliable. Applying tree-based error analysis workflows on 5 QSAR models reported in the literature and available in the QsarDB repository, i.e., androgen receptor bioactivity (agonists, antagonists, and binders, respectively) and membrane permeability (highest membrane permeability and the intrinsic permeability), we demonstrate that predictions erroneously tagged as reliable (AD prediction errors) overwhelmingly correspond to instances in subspaces (cohorts) with the highest prediction error rates, highlighting the inhomogeneity of the AD space. In this sense, we call for more stringent AD analysis guidelines which require the incorporation of model error analysis schemes, to provide critical insight on the reliability of underlying AD algorithms. Additionally, any selected AD method should be rigorously validated to demonstrate its suitability for the model space over which it is applied. These steps will ultimately contribute to more accurate estimations of the reliability of model predictions. Finally, error analysis may also be useful in "rational" model refinement in that data expansion efforts and model retraining are focused on cohorts with the highest error rates.

The OECD Principles for (Q)SAR Models in the Context of Knowledge Discovery in Databases (KDD).

The steps followed in the knowledge discovery in databases (KDD) process are well documented and are widely used in different areas where exploration of data is used for decision making. In turn, while different workflows for developing quantitative structure-activity relationship (QSAR) models have been proposed, including combinatorial use of QSAR, there is now agreement on common requirements for building trustable predictive models. In this work, we analyze and confront the steps involved in KDD and QSAR and present how they comply with the OECD principles for the validation, for regulatory purposes, of QSAR models.