Epistemic uncertainty in Bayesian predictive probabilities.

Bayesian predictive probabilities have become a ubiquitous tool for design and monitoring of clinical trials. The typical procedure is to average predictive probabilities over the prior or posterior distributions. In this paper, we highlight the limitations of relying solely on averaging, and propose the reporting of intervals or quantiles for the predictive probabilities. These intervals formalize the intuition that uncertainty decreases with more information. We present four different applications (Phase 1 dose escalation, early stopping for futility, sample size re-estimation, and assurance/probability of success) to demonstrate the practicality and generality of the proposed approach.

Promising zone two-stage design for a single-arm study with binary outcome.

Adaptive designs are increasingly used in clinical trials to assess the effectiveness of new drugs. For a single-arm study with a binary outcome, several adaptive designs were developed by using numerical search algorithms and the conditional power approach. The design based on numerical search algorithms is able to identify the global optimal design, but the computational intensity limits the usage of these designs. The conditional power approach searches for the optimal design without expensive computing time. In addition, promising zone strategy was proposed to move on drug development to the follow-up stages when the interim results are promising. We propose to develop two adaptive designs: One based on the conditional power approach, and the other based on the promising zone strategy. These two designs preserve types I and II error rates. It is preferable to satisfy the monotonic property for adaptive designs: The second stage sample size decreases as the first stage responses go up. We theoretically prove this important property for the two proposed designs. The proposed designs can be easily applied to real trials with limited computing resources.

A systematic review of the "promising zone" design.

INTRODUCTION: Sample size calculations require assumptions regarding treatment response and variability. Incorrect assumptions can result in under- or overpowered trials, posing ethical concerns. Sample size re-estimation (SSR) methods investigate the validity of these assumptions and increase the sample size if necessary. The "promising zone" (Mehta and Pocock, Stat Med 30:3267-3284, 2011) concept is appealing to researchers for its design simplicity. However, it is still relatively new in the application and has been a source of controversy. OBJECTIVES: This research aims to synthesise current approaches and practical implementation of the promising zone design. METHODS: This systematic review comprehensively identifies the reporting of methodological research and of clinical trials using promising zone. Databases were searched according to a pre-specified search strategy, and pearl growing techniques implemented. RESULTS: The combined search methods resulted in 270 unique records identified; 171 were included in the review, of which 30 were trials. The median time to the interim analysis was 60% of the original target sample size (IQR 41-73%). Of the 15 completed trials, 7 increased their sample size. Only 21 studies reported the maximum sample size that would be considered, for which the median increase was 50% (IQR 35-100%). CONCLUSIONS: Promising zone is being implemented in a range of trials worldwide, albeit in low numbers. Identifying trials using promising zone was difficult due to the lack of reporting of SSR methodology. Even when SSR methodology was reported, some had key interim analysis details missing, and only eight papers provided promising zone ranges.

Optimal promising zone designs.

Clinical trials with adaptive sample size reassessment based on an unblinded analysis of interim results are perhaps the most popular class of adaptive designs (see Elsäßer et al., 2007). Such trials are typically designed by prespecifying a zone for the interim test statistic, termed the promising zone, along with a decision rule for increasing the sample size within that zone. Mehta and Pocock (2011) provided some examples of promising zone designs and discussed several procedures for controlling their type-1 error. They did not, however, address how to choose the promising zone or the corresponding sample size reassessment rule, and proposed instead that the operating characteristics of alternative promising zone designs could be compared by simulation. Jennison and Turnbull (2015) developed an approach based on maximizing expected utility whereby one could evaluate alternative promising zone designs relative to a gold-standard optimal design. In this paper, we show how, by eliciting a few preferences from the trial sponsor, one can construct promising zone designs that are both intuitive and achieve the Jennison and Turnbull (2015) gold-standard for optimality.

Statistical inference following sample size adjustment based on the 50%-conditional-power principle.

Sample size adjustment at an interim analysis can mitigate the risk of failing to meet the study objective due to lower-than-expected treatment effect. Without modification to the conventional statistical methods, the type I error rate will be inflated, primarily caused by increasing sample size when the interim observed treatment effect is close to null or no treatment effect. Modifications to the conventional statistical methods, such as changing critical values or using weighted test statistics, have been proposed to address primarily such a scenario at the cost of flexibility or interpretability. In reality, increasing sample size when interim results indicate no or very small treatment effect could unnecessarily waste limited resource on an ineffective drug candidate. Such considerations lead to the recently increased interest in sample size adjustment based on promising interim results. The 50% conditional power principle allows sample size increase only when the unblinded interim results are promising or the conditional power is greater than 50%. The conventional unweighted test statistics and critical values can be used without inflation of type I error rate. In this paper, statistical inference following such a design is assessed. As shown in the numerical study, the bias of the conventional maximum likelihood estimate (MLE) and coverage error of its conventional confidence interval are generally small following sample size adjustment. We recommend use of conventional, MLE-based statistical inference when applying the 50% conditional power principle for sample size adjustment. In such a way, consistent statistics will be used in both hypothesis test and statistical inference.

Sample size adjustment designs with time-to-event outcomes: A caution.

BACKGROUND: Sample size adjustment designs, which allow increasing the study sample size based on interim analysis of outcome data from a randomized clinical trial, have been increasingly promoted in the biostatistical literature. Although it is recognized that group sequential designs can be at least as efficient as sample size adjustment designs, many authors argue that a key advantage of these designs is their flexibility; interim sample size adjustment decisions can incorporate information and business interests external to the trial. Recently, Chen et al. (Clinical Trials 2015) considered sample size adjustment applications in the time-to-event setting using a design (CDL) that limits adjustments to situations where the interim results are promising. The authors demonstrated that while CDL provides little gain in unconditional power (versus fixed-sample-size designs), there is a considerable increase in conditional power for trials in which the sample size is adjusted. METHODS: In time-to-event settings, sample size adjustment allows an increase in the number of events required for the final analysis. This can be achieved by either (a) following the original study population until the additional events are observed thus focusing on the tail of the survival curves or (b) enrolling a potentially large number of additional patients thus focusing on the early differences in survival curves. We use the CDL approach to investigate performance of sample size adjustment designs in time-to-event trials. RESULTS: Through simulations, we demonstrate that when the magnitude of the true treatment effect changes over time, interim information on the shape of the survival curves can be used to enrich the final analysis with events from the time period with the strongest treatment effect. In particular, interested parties have the ability to make the end-of-trial treatment effect larger (on average) based on decisions using interim outcome data. Furthermore, in "clinical null" cases where there is no benefit due to crossing survival curves, the sample size adjustment design is shown to increase the probability of recommending an ineffective therapy. CONCLUSION: Access to interim information on the shape of the survival curves may jeopardize the perceived integrity of trials using sample size adjustment designs. Therefore, given the lack of efficiency advantage over group sequential designs, sample size adjustment designs in time-to-event settings remain unjustified.

Adaptive designs from a Data Safety Monitoring Board perspective: Some controversies and some case studies.

This article describes vignettes concerning interactions with Data Safety Monitoring Boards during the design and monitoring of some clinical trials with an adaptive design. Most reflect personal experiences by the author.

Optimal flexible sample size design with robust power.

It is well recognized that sample size determination is challenging because of the uncertainty on the treatment effect size. Several remedies are available in the literature. Group sequential designs start with a sample size based on a conservative (smaller) effect size and allow early stop at interim looks. Sample size re-estimation designs start with a sample size based on an optimistic (larger) effect size and allow sample size increase if the observed effect size is smaller than planned. Different opinions favoring one type over the other exist. We propose an optimal approach using an appropriate optimality criterion to select the best design among all the candidate designs. Our results show that (1) for the same type of designs, for example, group sequential designs, there is room for significant improvement through our optimization approach; (2) optimal promising zone designs appear to have no advantages over optimal group sequential designs; and (3) optimal designs with sample size re-estimation deliver the best adaptive performance. We conclude that to deal with the challenge of sample size determination due to effect size uncertainty, an optimal approach can help to select the best design that provides most robust power across the effect size range of interest. Copyright © 2016 John Wiley & Sons, Ltd.

An objective re-evaluation of adaptive sample size re-estimation: commentary on 'Twenty-five years of confirmatory adaptive designs'.

Over the past 25 years, adaptive designs have gradually gained acceptance and are being used with increasing frequency in confirmatory clinical trials. Recent surveys of submissions to the regulatory agencies reveal that the most popular type of adaptation is unblinded sample size re-estimation. Concerns have nevertheless been raised that this type of adaptation is inefficient.We intend to show in our discussion that such concerns are greatly exaggerated in any practical setting and that the advantages of adaptive sample size re-estimation usually outweigh any minor loss of efficiency.

Adaptive sample size modification in clinical trials: start small then ask for more?

We consider sample size re-estimation in a clinical trial, in particular when there is a significant delay before the measurement of patient response. Mehta and Pocock have proposed methods in which sample size is increased when interim results fall in a 'promising zone' where it is deemed worthwhile to increase conditional power by adding more subjects. Our analysis reveals potential pitfalls in applying this approach. Mehta and Pocock use results of Chen, DeMets and Lan to identify when increasing sample size, but applying a conventional level α significance test at the end of the trial does not inflate the type I error rate: we have found the greatest gains in power per additional observation are liable to lie outside the region defined by this method. Mehta and Pocock increase sample size to achieve a particular conditional power, calculated under the current estimate of treatment effect: this leads to high increases in sample size for a small range of interim outcomes, whereas we have found it more efficient to make moderate increases in sample size over a wider range of cases. If the aforementioned pitfalls are avoided, we believe the broad framework proposed by Mehta and Pocock is valuable for clinical trial design. Working in this framework, we propose sample size rules that apply explicitly the principle of adding observations when they are most beneficial. The resulting trial designs are closely related to efficient group sequential tests for a delayed response proposed by Hampson and Jennison.