Probability of success and group sequential designs.

In this article, I extend the use of probability of success calculations, previously developed for fixed sample size studies to group sequential designs (GSDs) both for studies planned to be analyzed by standard frequentist techniques or Bayesian approaches. The structure of GSDs lends itself to sequential learning which in turn allows us to consider how knowledge about the result of an interim analysis can influence our assessment of the study's probability of success. In this article, I build on work by Temple and Robertson who introduced the idea of conditional probability of success, an idea which I also treated in a recent monograph.

On implementing Jeffreys' substitution likelihood for Bayesian inference concerning the medians of unknown distributions.

When statisticians are uncertain as to which parametric statistical model to use to analyse experimental data, they will often resort to a non-parametric approach. The purpose of this paper is to provide insight into a simple approach to take when it is unclear as to the appropriate parametric model and plan to conduct a Bayesian analysis. I introduce an approximate, or substitution likelihood, first proposed by Harold Jeffreys in 1939 and show how to implement the approach combined with both a non-informative and an informative prior to provide a random sample from the posterior distribution of the median of the unknown distribution. The first example I use to demonstrate the approach is a within-patient bioequivalence design and then show how to extend the approach to a parallel group design.

Optimising the trade-off between type I and II error rates in the Bayesian context.

For any decision-making study, there are two sorts of errors that can be made, declaring a positive result when the truth is negative, and declaring a negative result when the truth is positive. Traditionally, the primary analysis of a study is a two-sided hypothesis test, the type I error rate will be set to 5% and the study is designed to give suitably low type II error - typically 10 or 20% - to detect a given effect size. These values are standard, arbitrary and, other than the choice between 10 and 20%, do not reflect the context of the study, such as the relative costs of making type I and II errors and the prior belief the drug will be placebo-like. Several authors have challenged this paradigm, typically for the scenario where the planned analysis is frequentist. When resource is limited, there will always be a trade-off between the type I and II error rates, and this article explores optimising this trade-off for a study with a planned Bayesian statistical analysis. This work provides a scientific basis for a discussion between stakeholders as to what type I and II error rates may be appropriate and some algebraic results for normally distributed data.

How to design a dose-finding study using the continual reassessment method.

INTRODUCTION: The continual reassessment method (CRM) is a model-based design for phase I trials, which aims to find the maximum tolerated dose (MTD) of a new therapy. The CRM has been shown to be more accurate in targeting the MTD than traditional rule-based approaches such as the 3 + 3 design, which is used in most phase I trials. Furthermore, the CRM has been shown to assign more trial participants at or close to the MTD than the 3 + 3 design. However, the CRM's uptake in clinical research has been incredibly slow, putting trial participants, drug development and patients at risk. Barriers to increasing the use of the CRM have been identified, most notably a lack of knowledge amongst clinicians and statisticians on how to apply new designs in practice. No recent tutorial, guidelines, or recommendations for clinicians on conducting dose-finding studies using the CRM are available. Furthermore, practical resources to support clinicians considering the CRM for their trials are scarce. METHODS: To help overcome these barriers, we present a structured framework for designing a dose-finding study using the CRM. We give recommendations for key design parameters and advise on conducting pre-trial simulation work to tailor the design to a specific trial. We provide practical tools to support clinicians and statisticians, including software recommendations, and template text and tables that can be edited and inserted into a trial protocol. We also give guidance on how to conduct and report dose-finding studies using the CRM. RESULTS: An initial set of design recommendations are provided to kick-start the design process. To complement these and the additional resources, we describe two published dose-finding trials that used the CRM. We discuss their designs, how they were conducted and analysed, and compare them to what would have happened under a 3 + 3 design. CONCLUSIONS: The framework and resources we provide are aimed at clinicians and statisticians new to the CRM design. Provision of key resources in this contemporary guidance paper will hopefully improve the uptake of the CRM in phase I dose-finding trials.

Response-adaptive clinical trials: case studies in the medical literature.

The past 15 years has seen many pharmaceutical sponsors consider and implement adaptive designs (AD) across all phases of drug development. Given their arrival at the turn of the millennium, we might think that they are a recent invention. That is not the case. The earliest idea of an AD predates Bradford Hill's MRC tuberculosis study, appearing in Biometrika in 1933. In this paper, we trace the development of response-ADs, designs in which the allocation to intervention arms depends on the responses of subjects already treated. We describe some statistical details underlying the designs, but our main focus is to describe and comment on ADs from the medical research literature. Copyright © 2016 John Wiley & Sons, Ltd.

Statistical issues in first-in-human studies on BIA 10-2474: Neglected comparison of protocol against practice.

By setting the regulatory-approved protocol for a suite of first-in-human studies on BIA 10-2474 against the subsequent French investigations, we highlight 6 key design and statistical issues, which reinforce recommendations by a Royal Statistical Society Working Party, which were made in the aftermath of cytokine release storm in 6 healthy volunteers in the United Kingdom in 2006. The 6 issues are dose determination, availability of pharmacokinetic results, dosing interval, stopping rules, appraisal by safety committee, and clear algorithm required if combining approvals for single and multiple ascending dose studies.

Idle thoughts of a 'well-calibrated' Bayesian in clinical drug development.

The use of Bayesian approaches in the regulated world of pharmaceutical drug development has not been without its difficulties or its critics. The recent Food and Drug Administration regulatory guidance on the use of Bayesian approaches in device submissions has mandated an investigation into the operating characteristics of Bayesian approaches and has suggested how to make adjustments in order that the proposed approaches are in a sense calibrated. In this paper, I present examples of frequentist calibration of Bayesian procedures and argue that we need not necessarily aim for perfect calibration but should be allowed to use procedures, which are well-calibrated, a position supported by the guidance.

Simulation-based sample-sizing and power calculations in logistic regression with partial prior information.

There have been many approximations developed for sample sizing of a logistic regression model with a single normally-distributed stimulus. Despite this, it has been recognised that there is no consensus as to the best method. In pharmaceutical drug development, simulation provides a powerful tool to characterise the operating characteristics of complex adaptive designs and is an ideal method for determining the sample size for such a problem. In this paper, we address some issues associated with applying simulation to determine the sample size for a given power in the context of logistic regression. These include efficient methods for evaluating the convolution of a logistic function and a normal density and an efficient heuristic approach to searching for the appropriate sample size. We illustrate our approach with three case studies. Copyright © 2016 John Wiley & Sons, Ltd.

The predictive distribution of the residual variability in the linear-fixed effects model for clinical cross-over trials.

In the linear model for cross-over trials, with fixed subject effects and normal i.i.d. random errors, the residual variability corresponds to the intraindividual variability. While population variances are in general unknown, an estimate can be derived that follows a gamma distribution, where the scale parameter is based on the true unknown variability. This gamma distribution is often used for the sample size calculation for trial planning with the precision approach, where the aim is to achieve in the next trial a predefined precision with a given probability. But then the imprecision in the estimated residual variability or, from a Bayesian perspective, the uncertainty of the unknown variability is not taken into account. Here, we present the predictive distribution for the residual variability, and we investigate a link to the F distribution. The consequence is that in the precision approach more subjects will be necessary than with the conventional calculation. For values of the intraindividual variability that are typical of human pharmacokinetics, that is a gCV of 17-36%, we would need approximately a sixth more subjects.

How to test hypotheses if you must.

Drug development is not the only industrial-scientific enterprise subject to government regulations. In some fields of ecology and environmental sciences, the application of statistical methods is also regulated by ordinance. Over the past 20years, ecologists and environmental scientists have argued against an unthinking application of null hypothesis significance tests. More recently, Canadian ecologists have suggested a new approach to significance testing, taking account of the costs of both type I and type II errors. In this paper, we investigate the implications of this for testing in drug development and demonstrate that its adoption leads directly to the likelihood principle and Bayesian approaches.

Risk and cumulative risk of stroke recurrence: a systematic review and meta-analysis.

BACKGROUND AND PURPOSE: Estimates of risk of stroke recurrence are widely variable and focused on the short- term. A systematic review and meta-analysis was conducted to estimate the pooled cumulative risk of stroke recurrence. METHODS: Studies reporting cumulative risk of recurrence after first-ever stroke were identified using electronic databases and by manually searching relevant journals and conference abstracts. Overall cumulative risks of stroke recurrence at 30 days and 1, 5, and 10 years after first stroke were calculated, and analyses for heterogeneity were conducted. A Weibull model was fitted to the risk of stroke recurrence of the individual studies and pooled estimates were calculated with 95% CI. RESULTS: Sixteen studies were identified, of which 13 studies reported cumulative risk of stroke recurrence in 9115 survivors. The pooled cumulative risk was 3.1% (95% CI, 1.7-4.4) at 30 days, 11.1% (95% CI, 9.0-13.3) at 1 year, 26.4% (95% CI, 20.1-32.8) at 5 years, and 39.2% (95% CI, 27.2-51.2) at 10 years after initial stroke. Substantial heterogeneity was found at all time points. This study also demonstrates a temporal reduction in 5-year risk of stroke recurrence from 32% to 16.2% across the studies. CONCLUSIONS: The cumulative risk of recurrence varies greatly up to 10 years. This may be explained by differences in case mix and changes in secondary prevention over time However, methodological differences are likely to play an important role and consensus on definitions would improve future comparability of estimates and characterization of groups of stroke survivors at increased risk of recurrence.

Acute Stroke Therapy by Inhibition of Neutrophils (ASTIN): an adaptive dose-response study of UK-279,276 in acute ischemic stroke.

BACKGROUND AND PURPOSE: UK-279,276 (neutrophil inhibitory factor) reduced infarct volume in a rat middle cerebral artery occlusion reperfusion model. ASTIN (Acute Stroke Therapy by Inhibition of Neutrophils) was an adaptive phase 2 dose-response-finding, proof-of-concept study to establish whether UK-279,276 improves recovery in acute ischemic stroke. The prime objective was to determine the dose that gave a clinically relevant effect in patients. METHODS: A Bayesian sequential design with real-time efficacy data capture and continuous reassessment of the dose response allowed double-blind, randomized, adaptive allocation to 1 of 15 doses (dose range, 10 to 120 mg) or placebo and early termination for efficacy or futility. The primary end point was change from baseline to day 90 on the Scandinavian Stroke Scale (DeltaSSS), adjusted for baseline SSS, aiming for a 3-point additional mean recovery above placebo. RESULTS: Nine hundred sixty-six acute stroke patients (887 ischemic, 204 cotreated with intravenous tissue plasminogen activator; mean baseline SSS score, 28; range, 10 to 40) were treated within 6 hours of symptom onset. Mean DeltaSSS was approximately +17 points of improvement on SSS for the overall evaluable population. There was no treatment effect for UK-279,276 (posterior probability of futility, 0.89). The trial was stopped early for futility. Post hoc analysis indicated a mean 1.6-point additional improvement on DeltaSSS in the tissue plasminogen activator-treated subset (credible interval=0.5, 2.6). UK-279,276 was generally well tolerated, with no increased incidence of infections. CONCLUSIONS: UK-279,276 did not improve recovery in acute ischemic stroke patients but was devoid of serious side effects. The adaptive design facilitated early termination for futility.

Within-person study designs had lower precision and greater susceptibility to bias because of trends in exposure than cohort and nested case-control designs.

OBJECTIVE: To compare precision and apparent bias between cohort, nested case-control, self-controlled case series, case-crossover, and case-time-control study designs. STUDY DESIGN AND SETTING: Study designs were implemented to evaluate the association between thiazolidinediones (TZDs) and heart failure, TZDs and fracture, and liver enzyme-inducing anticonvulsants and fracture. RESULTS: Effect estimates were similar for the cohort and case-control study; for the association between TZDs and fracture in women, the hazard ratio was 1.36 (1.18, 1.56) and odds ratio (OR) was 1.44 (1.21, 1.70). For this clinical example, the self-controlled case series gave upward bias when follow-up was censored at the outcome (incidence rate ratio [IRR], 7.08; 4.96, 10.09) but was otherwise unbiased (IRR, 1.41; 1.14, 1.75). The retrospective case-crossover OR was 3.24 (2.18, 4.80), which was reduced by either bidirectional sampling (OR, 1.20; 0.98, 1.46) or with the case-time-control design (OR, 1.40; 1.09, 1.81). Findings on apparent bias were similar for the other two clinical examples. In each clinical example, within-person designs had considerably lower precision than the cohort or case-control study designs. CONCLUSION: When long-term exposures are analyzed, within-person study designs may have lower precision and greater susceptibility to bias. Bias may be reduced by sampling follow-up both before and after the outcome or with the case-time-control study design.

Cluster randomised trial in the General Practice Research Database: 1. Electronic decision support to reduce antibiotic prescribing in primary care (eCRT study).

BACKGROUND: The purpose of this research is to develop and evaluate methods for conducting cluster randomised trials in a primary care database that contains electronic patient records for large numbers of family practices. Cluster randomised trials are trials in which the units allocated represent groups of individuals, in this case family practices and their registered patients. Cluster randomised trials often suffer from the limitation that they include too few clusters, leading to problems of insufficient power and only imprecise estimation of the intraclass correlation coefficient, a key design parameter. This difficulty might be overcome by utilising databases that already hold electronic patient records for large numbers of practices. The protocol describes one application: a study of antibiotic prescribing for acute respiratory infection; a second protocol outlines an intervention in a less frequent chronic condition of public health importance, stroke. METHODS/DESIGN: The objective of the study is to implement a cluster randomised trial to test the effectiveness of an electronic record-based intervention at achieving a reduction in antibiotic prescribing at consultations for respiratory illness in patients aged 18 and 59 years old in intervention family practices as compared with controls. Family practices will be recruited from the practices that presently contribute data to the UK General Practice Research Database (GPRD). Following randomisation, electronic prompts will be installed remotely at intervention practices to promote adherence with evidence-based standards of medical practice. The intervention was developed through qualitative research at non-intervention practices. Data for outcome assessment will be obtained from anonymised electronic patient records that are routinely collected into GPRD. This protocol outlines the proposed study designs, data sources, sample size requirements, analysis methods and dissemination plans. Ethical issues are also discussed. DISCUSSION: Results from this study will provide methodological evidence concerning the use of electronic patient records and databases for implementing cluster randomised trials in primary care. The study will also provide substantive findings in respect of electronic record-based interventions to reduce antibiotic prescribing in primary care. TRIAL REGISTRATION: Current Controlled Trials ISRCTN 47558792.

25 years of Bayesian methods in the pharmaceutical industry: a personal, statistical bummel.

Twenty-five years ago the use of Bayesian methods in Pharmaceutical R&D was non-existent. Today that is no longer true. In this paper I describe my own personal journey along the road of discovery of Bayesian methods to routine use in the pharmaceutical industry.

Flexible design and efficient implementation of adaptive dose-finding studies.

A dose-finding study with an adaptive design generates three computational problems: fitting the dose-response curve given the current data, identifying the dose to be given to the next patient that is optimal for learning about the dose-response curve, and pretrial simulation in order to establish operating characteristics of alternative designs. Identifying the 'optimal' dose is the rate-limiting step since conventional methods, estimating the full posterior predictive distribution of some utility function under each of the possible doses, are very slow. We explore a simpler strategy based on importance sampling, whereby the posterior mean of the utility at each candidate dose is estimated by taking its average across an empirical distribution for the model parameters from the current Markov chain Monte Carlo (MCMC) run, weighted according to the likelihood of one or more predicted observations. We identify appropriate settings for this algorithm and illustrate its application in the context of a normal dynamic linear model used in a dose-finding clinical trial of a neutrophil inhibitory factor in acute ischaemic stroke.

Adaptive designs in clinical drug development: opportunities, challenges, and scope reflections following PhRMA's November 2006 workshop.

This paper provides reflections on the opportunities, scope and challenges of adaptive design as discussed at PhRMA's workshop held in November 2006. We also provide a status report of workstreams within PhRMA's working group on adaptive designs, which were triggered by the November workshop. Rather than providing a comprehensive review of the presentations given, we limit ourselves to a selection of key statements. The authors reflect the position of PhRMA's working group on adaptive designs.

Implementation of a Bayesian adaptive design in a proof of concept study.

With increased costs of drug development the need for efficient studies has become critical. A key decision point on the development pathway has become the proof of concept study. These studies must provide clear information to the project teams to enable decision making about further developing a drug candidate but also to gain evidence that any effect size is sufficient to warrant this development given the current market environment. Our case study outlines one such proof of concept trial where a new candidate therapy for neuropathic pain was investigated to assess dose-response and to evaluate the magnitude of its effect compared to placebo. A Normal Dynamic Linear Model was used to estimate the dose-response--enforcing some smoothness in the dose-response, but allowing for the fact that the dose-response may be non-monotonic. A pragmatic, parallel group study design was used with interim analyses scheduled to allow the sponsor to drop ineffective doses or to stop the study. Simulations were performed to assess the operating characteristics of the study design. The study results are presented. Significant cost savings were made when it transpired that the new candidate drug did not show superior efficacy when compared placebo and the study was stopped.