Optimising the trade-off between type I and II error rates in the Bayesian context.

For any decision-making study, there are two sorts of errors that can be made, declaring a positive result when the truth is negative, and declaring a negative result when the truth is positive. Traditionally, the primary analysis of a study is a two-sided hypothesis test, the type I error rate will be set to 5% and the study is designed to give suitably low type II error - typically 10 or 20% - to detect a given effect size. These values are standard, arbitrary and, other than the choice between 10 and 20%, do not reflect the context of the study, such as the relative costs of making type I and II errors and the prior belief the drug will be placebo-like. Several authors have challenged this paradigm, typically for the scenario where the planned analysis is frequentist. When resource is limited, there will always be a trade-off between the type I and II error rates, and this article explores optimising this trade-off for a study with a planned Bayesian statistical analysis. This work provides a scientific basis for a discussion between stakeholders as to what type I and II error rates may be appropriate and some algebraic results for normally distributed data.

Pooling Different Safety Data Sources: Impact of Combining Solicited and Spontaneous Reports on Signal Detection In Pharmacovigilance.

INTRODUCTION: The volume of adverse events (AEs) collected, analysed, and reported has been increasing at a rapid rate for over the past 10 years, largely due to the growth of solicited programmes. The proportion of various forms of solicited case data has evolved over time, with the main relative volume increase coming from Patient Support Programmes. In this study, we sought to examine the impact of the pooling of AE report data from solicited sources with data from spontaneous sources to safety signal detection using disproportionality analysis methods. METHODS: Two conditions were explored in which disproportionality scores from hypothetical drugs were evaluated in a simulated safety database. The first condition held occurrence of events constant and varied solicited case volume, while the second condition varied both proportion of occurrence of events and solicited case volume. RESULTS: In the first setting, where all AE terms have the same probability to occur with any drug, increasing volumes of solicited cases while keeping occurrence of events constant leads to reduced variability in disproportionality scores, consequently reducing or eliminating identified signals of disproportionate reporting. In the second setting, varying both case volume and reporting rates can mask true safety signals and falsely identify signals where there are none. CONCLUSIONS: This analysis of simulated data suggests that pooling AE data from solicited sources with spontaneous case data may impact the results of disproportionality analyses, masking true safety signals and identifying false positives. Therefore, increased volumes of safety data do not necessarily correlate with improved safety signal detection.

Advantages of a wholly Bayesian approach to assessing efficacy in early drug development: a case study.

This paper illustrates how the design and statistical analysis of the primary endpoint of a proof-of-concept study can be formulated within a Bayesian framework and is motivated by and illustrated with a Pfizer case study in chronic kidney disease. It is shown how decision criteria for success can be formulated, and how the study design can be assessed in relation to these, both using the traditional approach of probability of success conditional on the true treatment difference and also using Bayesian assurance and pre-posterior probabilities. The case study illustrates how an informative prior on placebo response can have a dramatic effect in reducing sample size, saving time and resource, and we argue that in some cases, it can be considered unethical not to include relevant literature data in this way.

The translational challenge in the development of new and effective therapies for endometriosis: a review of confidence from published preclinical efficacy studies.

BACKGROUND: Endometriosis is a benign gynaecological condition that presents symptoms of chronic pelvic pain and the ectopic growth of endometrial lesions at sites on the peritoneum. Few new approaches to the management of the disease symptoms and progression have emerged in decades. The cornerstone of developing new therapies is the confidence and translational value placed in the preclinical models used to assess efficacy of emerging approaches. METHODS: We systematically reviewed preclinical efficacy data from rodent and non-human primates, evaluating the effects of an investigational agent or target reported in PubMed between 2000 and 2010. We evaluated the reports for which model and end-points had been used to determine efficacy, whether there was evidence of independent replication, whether techniques had been incorporated into the experimental design to eliminate potential bias and whether there was a confirmation of drug exposure or target engagement in the study. RESULTS: We identified 94 publications that met our criteria for review. Efficacy studies were conducted in a wider range of different models with no clear consensus on which model or end-point has the most translational value. The large majority of studies either did not report what measures were incorporated into the design to address potential bias or account for it or did not confirm whether the specified target was engaged. CONCLUSIONS: Greater scrutiny of the preclinical efficacy models, end-points and experimental designs is needed if the desire of translating novel treatment approaches is to be realized for women with endometriosis.

Designing compound subsets: comparison of random and rational approaches using statistical simulation.

Compound subsets, which may be screened where it is not feasible or desirable to screen all available compounds, may be designed using rational or random selection. Literature on the relative performance of random versus rational selection reports conflicting observations, possibly because some random subsets might be more representative than others and perform better than subsets designed by rational means, or vice versa. In order to address this likelihood, we simulated a large number of rationally designed subsets for evaluation against an equally large number of randomly generated subsets. We found that our rationally designed subsets give higher mean hit rates compared to those of the random ones. We also compared subsets comprising random plates with subsets of random compounds and found that, while the mean hit rate of both is the same, the former demonstrates more variation in the hit rate. The choice of compound file, rational subset method, and ratio of the subset size to the compound file size are key factors in the relative performance of random and rational selection, and statistical simulation is a viable way to identify the selection approach appropriate for a subset.

Pharmaco-metabonomic phenotyping and personalized drug treatment.

There is a clear case for drug treatments to be selected according to the characteristics of an individual patient, in order to improve efficacy and reduce the number and severity of adverse drug reactions. However, such personalization of drug treatments requires the ability to predict how different individuals will respond to a particular drug/dose combination. After initial optimism, there is increasing recognition of the limitations of the pharmacogenomic approach, which does not take account of important environmental influences on drug absorption, distribution, metabolism and excretion. For instance, a major factor underlying inter-individual variation in drug effects is variation in metabolic phenotype, which is influenced not only by genotype but also by environmental factors such as nutritional status, the gut microbiota, age, disease and the co- or pre-administration of other drugs. Thus, although genetic variation is clearly important, it seems unlikely that personalized drug therapy will be enabled for a wide range of major diseases using genomic knowledge alone. Here we describe an alternative and conceptually new 'pharmaco-metabonomic' approach to personalizing drug treatment, which uses a combination of pre-dose metabolite profiling and chemometrics to model and predict the responses of individual subjects. We provide proof-of-principle for this new approach, which is sensitive to both genetic and environmental influences, with a study of paracetamol (acetaminophen) administered to rats. We show pre-dose prediction of an aspect of the urinary drug metabolite profile and an association between pre-dose urinary composition and the extent of liver damage sustained after paracetamol administration.

Statistical evaluation of biomarkers as surrogate endpoints: a literature review.

A valid surrogate endpoint allows correct inference to be drawn regarding the effect of an intervention on the unobserved true clinical endpoint of interest. The perceived practical and ethical advantages of substituting a surrogate endpoint for a clinical endpoint have led to a considerable number of statistical methods being proposed for the evaluation of a biomarker as a surrogate endpoint. We review the main statistical schools of thought which have developed and consider how the validation process might be arranged within the regulatory and practical constraints of the drug development process. We conclude by assessing which of the candidate statistical methods offer the best approach for surrogate endpoint evaluation.