From innovative thinking to pharmaceutical industry implementation: Some success stories.

In industry, successful innovation involves not only developing new statistical methodology, but also ensuring that this methodology is implemented successfully. This includes enabling applied statisticians to understand the method, its benefits and limitations and empowering them to implement the new method. This will include advocacy, influencing in-house and external stakeholders, such that these stakeholders are receptive to the new methodology. In this paper, we describe some industry successes and focus on our colleague, Andy Grieve's role in these.

Optimising the trade-off between type I and II error rates in the Bayesian context.

For any decision-making study, there are two sorts of errors that can be made, declaring a positive result when the truth is negative, and declaring a negative result when the truth is positive. Traditionally, the primary analysis of a study is a two-sided hypothesis test, the type I error rate will be set to 5% and the study is designed to give suitably low type II error - typically 10 or 20% - to detect a given effect size. These values are standard, arbitrary and, other than the choice between 10 and 20%, do not reflect the context of the study, such as the relative costs of making type I and II errors and the prior belief the drug will be placebo-like. Several authors have challenged this paradigm, typically for the scenario where the planned analysis is frequentist. When resource is limited, there will always be a trade-off between the type I and II error rates, and this article explores optimising this trade-off for a study with a planned Bayesian statistical analysis. This work provides a scientific basis for a discussion between stakeholders as to what type I and II error rates may be appropriate and some algebraic results for normally distributed data.

Phosphodiesterase Type 5 Inhibition Reduces Albuminuria in Subjects with Overt Diabetic Nephropathy.

Diabetic nephropathy (DN) is the leading cause of ESRD worldwide. Reduced bioavailability or uncoupling of nitric oxide in the kidney, leading to decreased intracellular levels of the nitric oxide pathway effector molecule cyclic guanosine monophosphate (cGMP), has been implicated in the progression of DN. Preclinical studies suggest that elevating the cGMP intracellular pool through inhibition of the cGMP-hydrolyzing enzyme phosphodiesterase type 5 (PDE5) might exert renoprotective effects in DN. To test this hypothesis, the novel, highly specific, and long-acting PDE5 inhibitor, PF-00489791, was assessed in a multinational, multicenter, randomized, double-blind, placebo-controlled, parallel group trial of subjects with type 2 diabetes mellitus and overt nephropathy receiving angiotensin converting enzyme inhibitor or angiotensin receptor blocker background therapy. In total, 256 subjects with an eGFR between 25 and 60 ml/min per 1.73 m2 and macroalbuminuria defined by a urinary albumin-to-creatinine ratio >300 mg/g, were randomly assigned 3:1, respectively, to receive PF-00489791 (20 mg) or placebo orally, once daily for 12 weeks. Using the predefined primary assessment of efficacy (Bayesian analysis with informative prior), we observed a significant reduction in urinary albumin-to-creatinine ratio of 15.7% (ratio 0.843; 95% credible interval 0.73 to 0.98) in response to the 12-week treatment with PF-00489791 compared with placebo. PF-00489791 was safe and generally well tolerated in this patient population. Most common adverse events were mild in severity and included headache and upper gastrointestinal events. In conclusion, the safety and efficacy profile of PDE5 inhibitor PF-00489791 supports further investigation as a novel therapy to improve renal outcomes in DN.

Addressing potential prior-data conflict when using informative priors in proof-of-concept studies.

Bayesian methods are increasingly used in proof-of-concept studies. An important benefit of these methods is the potential to use informative priors, thereby reducing sample size. This is particularly relevant for treatment arms where there is a substantial amount of historical information such as placebo and active comparators. One issue with using an informative prior is the possibility of a mismatch between the informative prior and the observed data, referred to as prior-data conflict. We focus on two methods for dealing with this: a testing approach and a mixture prior approach. The testing approach assesses prior-data conflict by comparing the observed data to the prior predictive distribution and resorting to a non-informative prior if prior-data conflict is declared. The mixture prior approach uses a prior with a precise and diffuse component. We assess these approaches for the normal case via simulation and show they have some attractive features as compared with the standard one-component informative prior. For example, when the discrepancy between the prior and the data is sufficiently marked, and intuitively, one feels less certain about the results, both the testing and mixture approaches typically yield wider posterior-credible intervals than when there is no discrepancy. In contrast, when there is no discrepancy, the results of these approaches are typically similar to the standard approach. Whilst for any specific study, the operating characteristics of any selected approach should be assessed and agreed at the design stage; we believe these two approaches are each worthy of consideration.

Using Bayesian analysis in repeated preclinical in vivo studies for a more effective use of animals.

Whilst innovative Bayesian approaches are increasingly used in clinical studies, in the preclinical area Bayesian methods appear to be rarely used in the reporting of pharmacology data. This is particularly surprising in the context of regularly repeated in vivo studies where there is a considerable amount of data from historical control groups, which has potential value. This paper describes our experience with introducing Bayesian analysis for such studies using a Bayesian meta-analytic predictive approach. This leads naturally either to an informative prior for a control group as part of a full Bayesian analysis of the next study or using a predictive distribution to replace a control group entirely. We use quality control charts to illustrate study-to-study variation to the scientists and describe informative priors in terms of their approximate effective numbers of animals. We describe two case studies of animal models: the lipopolysaccharide-induced cytokine release model used in inflammation and the novel object recognition model used to screen cognitive enhancers, both of which show the advantage of a Bayesian approach over the standard frequentist analysis. We conclude that using Bayesian methods in stable repeated in vivo studies can result in a more effective use of animals, either by reducing the total number of animals used or by increasing the precision of key treatment differences. This will lead to clearer results and supports the "3Rs initiative" to Refine, Reduce and Replace animals in research. Copyright © 2016 John Wiley & Sons, Ltd.

Advantages of a wholly Bayesian approach to assessing efficacy in early drug development: a case study.

This paper illustrates how the design and statistical analysis of the primary endpoint of a proof-of-concept study can be formulated within a Bayesian framework and is motivated by and illustrated with a Pfizer case study in chronic kidney disease. It is shown how decision criteria for success can be formulated, and how the study design can be assessed in relation to these, both using the traditional approach of probability of success conditional on the true treatment difference and also using Bayesian assurance and pre-posterior probabilities. The case study illustrates how an informative prior on placebo response can have a dramatic effect in reducing sample size, saving time and resource, and we argue that in some cases, it can be considered unethical not to include relevant literature data in this way.

Pharmaco-metabonomic phenotyping and personalized drug treatment.

There is a clear case for drug treatments to be selected according to the characteristics of an individual patient, in order to improve efficacy and reduce the number and severity of adverse drug reactions. However, such personalization of drug treatments requires the ability to predict how different individuals will respond to a particular drug/dose combination. After initial optimism, there is increasing recognition of the limitations of the pharmacogenomic approach, which does not take account of important environmental influences on drug absorption, distribution, metabolism and excretion. For instance, a major factor underlying inter-individual variation in drug effects is variation in metabolic phenotype, which is influenced not only by genotype but also by environmental factors such as nutritional status, the gut microbiota, age, disease and the co- or pre-administration of other drugs. Thus, although genetic variation is clearly important, it seems unlikely that personalized drug therapy will be enabled for a wide range of major diseases using genomic knowledge alone. Here we describe an alternative and conceptually new 'pharmaco-metabonomic' approach to personalizing drug treatment, which uses a combination of pre-dose metabolite profiling and chemometrics to model and predict the responses of individual subjects. We provide proof-of-principle for this new approach, which is sensitive to both genetic and environmental influences, with a study of paracetamol (acetaminophen) administered to rats. We show pre-dose prediction of an aspect of the urinary drug metabolite profile and an association between pre-dose urinary composition and the extent of liver damage sustained after paracetamol administration.

The translational efficacy of a nonsteroidal progesterone receptor antagonist, 4-[3-cyclopropyl-1-(mesylmethyl)-5-methyl-1H-pyrazol-4-yl]oxy,-2,6-dimethylbenzonitrile (PF-02413873), on endometrial growth in macaque and human.

There is considerable ongoing investment in the research and development of selective progesterone receptor (PR) modulators for the treatment of gynecological conditions such as endometriosis. Here, we provide the first report on the clinical evaluation of a nonsteroidal progesterone receptor antagonist 4-[3-cyclopropyl-1-(mesylmethyl)-5-methyl-1H-pyrazol-4-yl]oxy,-2,6-dimethylbenzonitrile (PF-02413873) in healthy female subjects. In in vitro assays, PF-02413873 behaved as a selective and fully competitive PR antagonist, blocking progesterone binding and PR nuclear translocation. The pharmacological mode of action of PF-02413873 seems to differ from the founding member of the class of steroidal PR antagonists, 11ß-4-dimethylaminophenyl-17ß-hydroxy-17α-propinyl-4,9-estradiene-3-one (RU-486; mifepristone). Exposure-effect data from studies in the cynomolgus macaque, however, demonstrated that PF-02413873 reduced endometrial functionalis thickness to a comparable degree to RU-486 and this effect was accompanied by a decrease in proliferation rate (as measured by bromodeoxyuridine incorporation) for both RU-486 and high-dose PF-02413873. These data were used to underwrite a clinical assessment of PF-02413873 in a randomized, double-blinded, third-party open, placebo-controlled, dose-escalation study in healthy female volunteers with dosing for 14 days. PF-02413873 blocked the follicular phase increase in endometrial thickness, the midcycle lutenizing hormone surge, and elevation in estradiol in a dose-dependent fashion compared with placebo. This is the first report of translational efficacy data with a nonsteroidal PR antagonist in cynomolgus macaque and human subjects.

Real-World Evidence Utilization in Clinical Development Reflected by US Product Labeling: Statistical Review.

The US Food and Drug Administration (FDA) has shown scientific discretion in interpreting the substantial evidence requirement for the approval of new drugs with its considerations on the use of single controlled or uncontrolled trials (Federal Food, Drug, and Cosmetic Act § 505(d), 21 USC 355(d), 1962). With the passage of the 21st Centuries Cures Act (21st Century Cures-patients. House, Energy and Commerce Committee, Washington, DC, 2019 available at: https://energycommerce.house.gov/sites/republicans.energycommerce.house.gov/files/analysis/21stCenturyCures/20140516PatientsWhitePaper.pdf ), the FDA is mandated to expand the role of real-world evidence (RWE) in support of drug approval. This mandate further broadens the scope of scientific discretion to include data collected outside clinical trials. We summarize the agency's past acceptance of real-world data (RWD) sources for supporting drug approval in new indications which have been reflected in US labels. In our summary, we focus on the type of RWD and statistical methodologies presented in these labels. Furthermore, two labels were selected for in-depth assessment of the RWE presented in these labels. Through these examples, we demonstrate the issues that can be raised in data collection that could affect interpretation. In addition, a brief discussion of statistical methods that can be used to incorporate RWE to clinical development is presented.

Reducing Patient Burden in Clinical Trials Through the Use of Historical Controls: Appropriate Selection of Historical Data to Minimize Risk of Bias.

Historical data have been used to augment or replace control arms in some rare disease and pediatric clinical trials. With greater availability of historical data and new methodology such as dynamic borrowing, the inclusion of historical data in clinical trials is an increasingly appealing approach for larger disease areas as well, as this can result in increased power and precision and can minimize the burden on patients in clinical trials. However, sponsors must assess whether the potential biases incurred with this approach outweigh the benefits and discuss this trade-off with the regulatory agencies. This paper discusses important points for the appropriate selection of historical controls for inclusion in the analysis of primary and/or key secondary endpoint(s) in clinical trials. The general steps are as follows: (1) Assess whether a trial is a suitable candidate for this approach. (2) If it is, then carefully identify appropriate historical trials to minimize selection bias. (3) Refine the historical control set if appropriate, for example, by selecting subsets of studies or patients. Identification of trial settings that are amenable to historical borrowing and selection of appropriate historical data using the principles discussed in this paper has the potential to lead to more efficient estimation and decision making. Ultimately, this efficiency gain results in lower patient burden and gets effective drugs to patients more quickly.

Pooling Different Safety Data Sources: Impact of Combining Solicited and Spontaneous Reports on Signal Detection In Pharmacovigilance.

INTRODUCTION: The volume of adverse events (AEs) collected, analysed, and reported has been increasing at a rapid rate for over the past 10 years, largely due to the growth of solicited programmes. The proportion of various forms of solicited case data has evolved over time, with the main relative volume increase coming from Patient Support Programmes. In this study, we sought to examine the impact of the pooling of AE report data from solicited sources with data from spontaneous sources to safety signal detection using disproportionality analysis methods. METHODS: Two conditions were explored in which disproportionality scores from hypothetical drugs were evaluated in a simulated safety database. The first condition held occurrence of events constant and varied solicited case volume, while the second condition varied both proportion of occurrence of events and solicited case volume. RESULTS: In the first setting, where all AE terms have the same probability to occur with any drug, increasing volumes of solicited cases while keeping occurrence of events constant leads to reduced variability in disproportionality scores, consequently reducing or eliminating identified signals of disproportionate reporting. In the second setting, varying both case volume and reporting rates can mask true safety signals and falsely identify signals where there are none. CONCLUSIONS: This analysis of simulated data suggests that pooling AE data from solicited sources with spontaneous case data may impact the results of disproportionality analyses, masking true safety signals and identifying false positives. Therefore, increased volumes of safety data do not necessarily correlate with improved safety signal detection.

Minimizing Patient Burden Through the Use of Historical Subject-Level Data in Innovative Confirmatory Clinical Trials: Review of Methods and Opportunities.

The goal of clinical trial research is to deliver safe and efficacious new treatments to patients in need in a timely and cost-effective manner. There is precedent in using historical control data to reduce the number of concurrent control subjects required in developing medicines for rare diseases and other areas of unmet need. The purpose of this paper is to provide a review for a regulatory and industry audience of the current state of relevant statistical methods, and of the uptake of these approaches and the opportunities for broader use of historical data in confirmatory clinical trials. General principles to consider when incorporating historical control data in a new trial are presented. Bayesian and frequentist approaches are outlined including how the operating characteristics for such a trial can be obtained. Finally, examples of approved new treatments that incorporated historical controls in their confirmatory trials are presented.

The translational challenge in the development of new and effective therapies for endometriosis: a review of confidence from published preclinical efficacy studies.

BACKGROUND: Endometriosis is a benign gynaecological condition that presents symptoms of chronic pelvic pain and the ectopic growth of endometrial lesions at sites on the peritoneum. Few new approaches to the management of the disease symptoms and progression have emerged in decades. The cornerstone of developing new therapies is the confidence and translational value placed in the preclinical models used to assess efficacy of emerging approaches. METHODS: We systematically reviewed preclinical efficacy data from rodent and non-human primates, evaluating the effects of an investigational agent or target reported in PubMed between 2000 and 2010. We evaluated the reports for which model and end-points had been used to determine efficacy, whether there was evidence of independent replication, whether techniques had been incorporated into the experimental design to eliminate potential bias and whether there was a confirmation of drug exposure or target engagement in the study. RESULTS: We identified 94 publications that met our criteria for review. Efficacy studies were conducted in a wider range of different models with no clear consensus on which model or end-point has the most translational value. The large majority of studies either did not report what measures were incorporated into the design to address potential bias or account for it or did not confirm whether the specified target was engaged. CONCLUSIONS: Greater scrutiny of the preclinical efficacy models, end-points and experimental designs is needed if the desire of translating novel treatment approaches is to be realized for women with endometriosis.

Designing compound subsets: comparison of random and rational approaches using statistical simulation.

Compound subsets, which may be screened where it is not feasible or desirable to screen all available compounds, may be designed using rational or random selection. Literature on the relative performance of random versus rational selection reports conflicting observations, possibly because some random subsets might be more representative than others and perform better than subsets designed by rational means, or vice versa. In order to address this likelihood, we simulated a large number of rationally designed subsets for evaluation against an equally large number of randomly generated subsets. We found that our rationally designed subsets give higher mean hit rates compared to those of the random ones. We also compared subsets comprising random plates with subsets of random compounds and found that, while the mean hit rate of both is the same, the former demonstrates more variation in the hit rate. The choice of compound file, rational subset method, and ratio of the subset size to the compound file size are key factors in the relative performance of random and rational selection, and statistical simulation is a viable way to identify the selection approach appropriate for a subset.

Statistical evaluation of biomarkers as surrogate endpoints: a literature review.

A valid surrogate endpoint allows correct inference to be drawn regarding the effect of an intervention on the unobserved true clinical endpoint of interest. The perceived practical and ethical advantages of substituting a surrogate endpoint for a clinical endpoint have led to a considerable number of statistical methods being proposed for the evaluation of a biomarker as a surrogate endpoint. We review the main statistical schools of thought which have developed and consider how the validation process might be arranged within the regulatory and practical constraints of the drug development process. We conclude by assessing which of the candidate statistical methods offer the best approach for surrogate endpoint evaluation.

Gas chromatography/flame ionisation detection mass spectrometry for the detection of endogenous urine metabolites for metabonomic studies and its use as a complementary tool to nuclear magnetic resonance spectroscopy.

Metabonomics is a relatively new field of research in which the total pool of metabolites in body fluids or tissues from different patient groups is subjected to comparative analysis. Nuclear magnetic resonance (NMR) spectroscopy is the technology that is currently most widely used for the analysis of these highly complex metabolite mixtures, and hundreds of metabolites can be detected without any upfront separation. We have investigated in this study whether gas chromatography (GC) separation in combination with flame ionisation detection (FID) and mass spectrometry (MS) detection can be used for metabolite profiling from urine. We show that although GC sample preparation is much more involved than for NMR, hundreds of metabolites can reproducibly be detected and analysed by GC. We show that the data quality is sufficiently high--particularly if appropriate baseline correction and time-warping methods are applied--to allow for data comparison by chemometrics methods. A sample set of urines from eleven healthy human volunteers was analysed independently by GC and NMR, and subsequent chemometrics analysis of the two datasets showed some similar features. As judged by NIST database searches of the GC/MS data some of the major metabolites that are detected by NMR are also visible by GC/MS. Since in contrast to NMR every peak in GC corresponds to a single metabolite, the electron ionisation spectra can be used to quickly identify metabolites of interest if their reference spectra are present in a searchable database. In summary, we show that GC is a method that can be used as a complementary tool to NMR for metabolite profiling of urine samples.