Exact confidence limits after a group sequential single arm binary trial.

Group sequential single arm designs are common in phase II trials as well as attribute testing and acceptance sampling. After the trial is completed, especially if the recommendation is to proceed to further testing, there is interest in full inference on treatment efficacy. For a binary response, there is the potential to construct exact upper and lower confidence limits, the first published method for which is Jennison and Turnbull (1983). We place their method within the modern theory of exact confidence limits and provide a new general result that ensures that the exact limits are consistent with the test result, an issue that has been largely ignored in the literature. Amongst methods based on the minimal sufficient statistic, we propose two exact methods that out-perform Jennison and Turnbull's method across 10 selected designs. One of these we prefer and recommend for practical and theoretical reasons. We also investigate a method based on inverting Fisher's combination test, as well as a pure tie-breaking variant of it. For the range of designs considered, neither of these methods result in large enough improvements in efficiency to justify violation of the sufficiency principle. For any nonadaptive sequential design, an R-package is provided to select a method and compute the inference from a given realization.

Tests for noninferiority trials with binomial endpoints: A guide to modern and quasi-exact methods for biomedical researchers.

Applied statisticians and pharmaceutical researchers are frequently involved in the design and analysis of clinical trials where at least one of the outcomes is binary. Treatments are judged by the probability of a positive binary response. A typical example is the noninferiority trial, where it is tested whether a new experimental treatment is practically not inferior to an active comparator with a prespecified margin δ. Except for the special case of δ = 0, no exact conditional test is available although approximate conditional methods (also called second-order methods) can be applied. However, in some situations, the approximation can be poor and the logical argument for approximate conditioning is not compelling. The alternative is to consider an unconditional approach. Standard methods like the pooled z-test are already unconditional although approximate. In this article, we review and illustrate unconditional methods with a heavy emphasis on modern methods that can deliver exact, or near exact, results. For noninferiority trials based on either rate difference or rate ratio, our recommendation is to use the so-called E-procedure, based on either the score or likelihood ratio statistic. This test is effectively exact, computationally efficient, and respects monotonicity constraints in practice. We support our assertions with a numerical study, and we illustrate the concepts developed in theory with a clinical example in pulmonary oncology; R code to conduct all these analyses is available from the authors.

Accurate p-values for adaptive designs with binary endpoints.

Adaptive designs encompass all trials allowing various types of design modifications over the course of the trial. A key requirement for confirmatory adaptive designs to be accepted by regulators is the strong control of the family-wise error rate. This can be achieved by combining the p-values for each arm and stage to account for adaptations (including but not limited to treatment selection), sample size adaptation and multiple stages. While the theory for this is novel and well-established, in practice, these methods can perform poorly, especially for unbalanced designs and for small to moderate sample sizes. The problem is that standard stagewise tests have inflated type I error rate, especially but not only when the baseline success rate is close to the boundary and this is carried over to the adaptive tests, seriously inflating the family-wise error rate. We propose to fix this problem by feeding the adaptive test with second-order accurate p-values, in particular bootstrap p-values. Secondly, an adjusted version of the Simes procedure for testing intersection hypotheses that reduces the built-in conservatism is suggested. Numerical work and simulations show that unlike their standard counterparts the new approach preserves the overall error rate, at or below the nominal level across the board, irrespective of the baseline rate, stagewise sample sizes or allocation ratio. Copyright © 2017 John Wiley & Sons, Ltd.

Accurate confidence limits for stratified clinical trials.

For stratified 2 × 2 tables, standard approximate confidence limits can perform poorly from a strict frequentist perspective, even for moderate-sized samples, yet they are routinely used. In this paper, I show how to use importance sampling to compute highly accurate limits in reasonable time. The methodology is very general and simple to implement, and orders of magnitude are faster than existing alternatives.

A comprehensive open-source library for exact required sample size in binary clinical trials.

We describe how we are creating a new and comprehensive R library solving the problem of exact sample size determination of RCTs. A crucial prerequisite for the trial protocol is a priori sample sizes that bound the test size below a target (often 5%) and the test power above a target (often 80%). Approximate formulas are available for binary trials but the target test size and power are often violated by standard methods for even quite large sample sizes. Moreover, adjusting standard tests to take account of their size bias can reduce power substantially. This has been well known for several decades. Exact and quasi-exact tests are now available and can be computed in a few seconds for a single data set. However, calculating the exact power and size of such tests requires computing them for all possible outcomes. Searching for minimum samples sizes that achieve a given target requires doing this for a wide range of sample sizes. This becomes computationally infeasible very quickly; to compute required sample sizes for a target size of 5% and power of 80% would, on a standard computer, take several months. Computation time increases as the size and clinically relevant difference decreases. After having presented the main operative challenges to creating this library, mainly due to the need of summarizing a very large amount of information, we put forward our innovative solutions to deal with this complex problem from a statistical viewpoint. The described library will be released in open source.

Bootstrap and second-order tests of risk difference.

Clinical trials data often come in the form of low-dimensional tables of small counts. Standard approximate tests such as score and likelihood ratio tests are imperfect in several respects. First, they can give quite different answers from the same data. Second, the actual type-1 error can differ significantly from nominal, even for quite large sample sizes. Third, exact inferences based on these can be strongly nonmonotonic functions of the null parameter and lead to confidence sets that are discontiguous. There are two modern approaches to small sample inference. One is to use so-called higher order asymptotics (Reid, 2003, Annal of Statistics 31, 1695-1731) to provide an explicit adjustment to the likelihood ratio statistic. The theory for this is complex but the statistic is quick to compute. The second approach is to perform an exact calculation of significance assuming the nuisance parameters equal their null estimate (Lee and Young, 2005, Statistic and Probability Letters 71, 143-153), which is a kind of parametric bootstrap. The purpose of this article is to explain and evaluate these two methods, for testing whether a difference in probabilities p(2) - p(1) exceeds a prechosen noninferiority margin δ(0) . On the basis of an extensive numerical study, we recommend bootstrap P-values as superior to all other alternatives. First, they produce practically identical answers regardless of the basic test statistic chosen. Second, they have excellent size accuracy and higher power. Third, they vary much less erratically with the null parameter value δ(0) .

A new exact and more powerful unconditional test of no treatment effect from binary matched pairs.

We consider the problem of testing for a difference in the probability of success from matched binary pairs. Starting with three standard inexact tests, the nuisance parameter is first estimated and then the residual dependence is eliminated by maximization, producing what I call an E+M P-value. The E+M P-value based on McNemar's statistic is shown numerically to dominate previous suggestions, including partially maximized P-values as described in Berger and Sidik (2003, Statistical Methods in Medical Research 12, 91-108). The latter method, however, may have computational advantages for large samples.

A new method of identifying target groups for pronatalist policy applied to Australia.

A country's total fertility rate (TFR) depends on many factors. Attributing changes in TFR to changes of policy is difficult, as they could easily be correlated with changes in the unmeasured drivers of TFR. A case in point is Australia where both pronatalist effort and TFR increased in lock step from 2001 to 2008 and then decreased. The global financial crisis or other unobserved confounders might explain both the reducing TFR and pronatalist incentives after 2008. Therefore, it is difficult to estimate causal effects of policy using econometric techniques. The aim of this study is to instead look at the structure of the population to identify which subgroups most influence TFR. Specifically, we build a stochastic model relating TFR to the fertility rates of various subgroups and calculate elasticity of TFR with respect to each rate. For each subgroup, the ratio of its elasticity to its group size is used to evaluate the subgroup's potential cost effectiveness as a pronatalist target. In addition, we measure the historical stability of group fertility rates, which measures propensity to change. Groups with a high effectiveness ratio and also high propensity to change are natural policy targets. We applied this new method to Australian data on fertility rates broken down by parity, age and marital status. The results show that targeting parity 3+ is more cost-effective than lower parities. This study contributes to the literature on pronatalist policies by investigating the targeting of policies, and generates important implications for formulating cost-effective policies.

Exact one-sided confidence bounds for the risk ratio in 2 x 2 tables with structural zero.

This paper examines exact one-sided confidence limits for the risk ratio in a 2 x 2 table with structural zero. Starting with four approximate lower and upper limits, we adjust each using the algorithm of Buehler (1957) to arrive at lower (upper) limits that have exact coverage properties and are as large (small) as possible subject to coverage, as well as an ordering, constraint. Different Buehler limits are compared by their mean size, since all are exact in their coverage. Buehler limits based on the signed root likelihood ratio statistic are found to have the best performance and recommended for practical use.

A more powerful exact test of noninferiority from binary matched-pairs data.

Assessing the therapeutic noninferiority of one medical treatment compared with another is often based on the difference in response rates from a matched binary pairs design. This paper develops a new exact unconditional test for noninferiority that is more powerful than available alternatives. There are two new elements presented in this paper. First, we introduce the likelihood ratio statistic as an alternative to the previously proposed score statistic of Nam (Biometrics 1997; 53:1422-1430). Second, we eliminate the nuisance parameter by estimation followed by maximization as an alternative to the partial maximization of Berger and Boos (Am. Stat. Assoc. 1994; 89:1012-1016) or traditional full maximization. Based on an extensive numerical study, we recommend tests based on the score statistic, the nuisance parameter being controlled by estimation followed by maximization.

Exact one-sided confidence limits for the difference between two correlated proportions.

We construct exact and optimal one-sided upper and lower confidence bounds for the difference between two probabilities based on matched binary pairs using well-established optimality theory of Buehler. Starting with five different approximate lower and upper limits, we adjust them to have coverage probability exactly equal to the desired nominal level and then compare the resulting exact limits by their mean size. Exact limits based on the signed root likelihood ratio statistic are preferred and recommended for practical use.

Unconditional efficient one-sided confidence limits for the odds ratio based on conditional likelihood.

We compare various one-sided confidence limits for the odds ratio in a 2 x 2 table. The first group of limits relies on first-order asymptotic approximations and includes limits based on the (signed) likelihood ratio, score and Wald statistics. The second group of limits is based on the conditional tilted hypergeometric distribution, with and without mid-P correction. All these limits have poor unconditional coverage properties and so we apply the general transformation of Buehler (J. Am. Statist. Assoc. 1957; 52:482-493) to obtain limits which are unconditionally exact. The performance of these competing exact limits is assessed across a range of sample sizes and parameter values by looking at their mean size. The results indicate that Buehler limits generated from the conditional likelihood have the best performance, with a slight preference for the mid-P version. This confidence limit has not been proposed before and is recommended for general use, especially when the underlying probabilities are not extreme.