A Bayesian approach to toxicological testing.

INTRODUCTION: Testing for toxicities is an important activity in drug development. In an ideal world the tests applied would be definitive. In reality this is seldom the case. There are two types of power associated with a test. A test's discriminatory power is characterized by its sensitivity and specificity and tells the investigator the probability of obtaining a test positive in the presence (sensitivity) or a test negative in the absence (specificity) of the toxicity. A test's discriminatory power is an attribute of the test itself. The investigator is, however, more interested in a test's predictive power, which is the probability that the toxicity is present or absent in a novel molecule given the test result. A test's predictive power is a consequence of the test's discriminatory power and the context of its application. Unlike its discriminatory power, the predictive power of a test is not 'fixed' and varies with testing context. This means that tests and test context must be taken together to enable an investigator to achieve their desired predictive power. Our intent is to illustrate a broadly applicable approach to testing schemes designed to maximize a test's positive or negative predictive power. Rather than hypothetical tests and toxicities, we use as examples tests available for the prediction of a substance's liability to cause the cardiac arrhythmia torsade de pointes. METHODS: Owing to intense focus over the last two decades, the discriminatory powers of a number of tests for predicting a torsade de pointes liability are publicly available. Having randomly chosen an initial test (random although plausible as an early screening assessment), the inter-relationship between the prevalence of torsadogenic liability and the discriminatory power of potential follow-on tests were explored in a probability framework, based on Bayes Theorem, to show how testing schemes could be developed based on odds and likelihood ratios. Uncertainty around the prevalence of torsade liability and the discriminatory power of a test were addressed by varying these values and examining their impact on the test's predictive power. RESULTS: Overall, the analysis demonstrates that tests can be strategically combined to reach a desired level of predictive power. This is generally more easily achieved for negative predictive power given a low prevalence of the toxicity under scrutiny. For this work, we used a base prevalence of 10% for a substance to carry a tordsadogenic liability. Given uncertainty around a test's discriminatory power, a probabilistic rather than deterministic approach was recommended. Such an approach necessarily requires the investigator to define distributions around test characteristics as well as their desired probability of attaining a given predictive power. CONCLUSIONS: The proposed approach is easily implemented deterministically since values of the discriminatory power of the tests are readily and publicly available. The probabilistic implementation is also easily implemented, but requires that the uncertainty around the test performance and prevalence, and the targets for probability of attaining the desired predictive value all be made explicit rather than remain implicit as is often the case in 'integrated risk assessment' or 'totality of evidence' presentations. This general approach could form a basis for testing and decision-making that can be communicated and discussed in a consistent manner between scientists as well as between sponsors and regulators.

Real estate diagrams: A tool to quickly assess the utility of a new predictive technology.

INTRODUCTION: Scientists are increasingly in a position to ask whether or not to adopt new technologies. We present a visualization tool to help scientists swiftly evaluate the worth of new assays. METHODS: The parameters (prevalence, sensitivity and, specificity) relevant to use of a toxicity test have values between 0 and 1. The proper arrangement of the parameters can be used to define areas in a [0,1] × [0, 1] cross-space. Our analogy is a square plot of land subdivided into smaller lots. We call the resultant graphic a real estate diagram. RESULTS: We use the well studied example of predicting prolongation of the QT interval of the electrocardiogram to illustrate the diagrams. The experience in human clinical Thorough QT (TQT) studies has been described (Park et al., 2013). Within the data we chose two chronological sets: 2 blocks of two years (2005-2006 and 2011-2012). In the first block 13 of 29 (45%) submitted compounds had positive TQT studies; in the second block the prevalence was 4 of 42 (10%). In other studies, the hERG channel patch-clamp assay used in predicting TQT outcome had an expected sensitivity of 0.70 and expected specificity of 0.72. Real estate diagrams were constructed to yield insight into the positive and negative predictive value (PPV and NPV, respectively) of the TQT prediction. The structure of the real estate diagrams revealed that increasing assay sensitivity in the face of declining prevalence would have a trivial effect on PPV and NPV. DISCUSSION: Nonclinical safety scientists will be called upon to question whether a new technology has the potential to meaningfully increase the predictive value of testing regimens. The real-estate diagram is a useful tool in making that assessment.

A multiattribute model for evaluating the benefit-risk profiles of treatment alternatives.

The assessment of the benefits and risks associated with a medicine's use requires careful consideration of a wealth of information of varying format and quality, ranging from efficacy and safety data derived from randomized clinical trials to statistical results from health outcomes studies to spontaneously reported adverse events. Contrary to the expectations of patients, physicians, and regulators, the literature offers little guidance as to how to strike an appropriate balance between benefit and risk. Although a qualitative listing of a medicine's benefits and risks is useful, much could be gained from a systematic and transparent process to evaluate a medicine's pre- and postmarketing performance. The authors propose a representational model based on multicriteria decision analysis that can incorporate both evaluative judgments from different perspectives (e.g., physician, patient) and quantitative data to inform tradeoffs between multiple benefit and multiple risk elements in a logically consistent and transparent manner. The model is designed to highlight the relative merits and deficits of treatment alternatives in well-defined and specific contexts. It is intended to serve as a common platform to facilitate focused benefit-risk tradeoff discussions between scientists, physicians, regulatory authorities, and pharmaceutical companies, and to assist in the communication of clear and consistent messages regarding those tradeoffs.