Discussion on "Adaptive enrichment designs with a continuous biomarker" by Nigel Stallard.

We congratulate Dr. Nigel Stallard on his stimulating paper on adaptive enrichment designs with a continuous biomarker. Dr. Stallard details a framework for a large and interesting class of enrichment procedures. His work has motivated us to offer some thoughts in response. Dr. Stallard's strategy is to use the maximum of a test statistic over a set of possible threshold values to define the enriched population to be sampled in a second stage. This reminds us of procedures for identifying a change point, a biomarker value beyond which the effect of treatment is increased. For simplicity we focus our comments on Dr. Stallard's Rule 1 for selecting the second-stage sampling threshold. Using this rule, we present the likelihood ratio approach for adaptive testing and compare it to Dr. Stallard's approach for a few scenarios.

Design considerations for two-stage enrichment clinical trials.

When there is a predictive biomarker, enrichment can focus the clinical trial on a benefiting subpopulation. We describe a two-stage enrichment design, in which the first stage is designed to efficiently estimate a threshold and the second stage is a "phase III-like" trial on the enriched population. The goal of this paper is to explore design issues: sample size in Stages 1 and 2, and re-estimation of the Stage 2 sample size following Stage 1. By treating these as separate trials, we can gain insight into how the predictive nature of the biomarker specifically impacts the sample size. We also show that failure to adequately estimate the threshold can have disastrous consequences in the second stage. While any bivariate model could be used, we assume a continuous outcome and continuous biomarker, described by a bivariate normal model. The correlation coefficient between the outcome and biomarker is the key to understanding the behavior of the design, both for predictive and prognostic biomarkers. Through a series of simulations we illustrate the impact of model misspecification, consequences of poor threshold estimation, and requisite sample sizes that depend on the predictive nature of the biomarker. Such insight should be helpful in understanding and designing enrichment trials.

Understanding Research Methods: Up-and-down Designs for Dose-finding.

SUMMARY: For the task of estimating a target benchmark dose such as the ED50 (the dose that would be effective for half the population), an adaptive dose-finding design is more effective than the standard approach of treating equal numbers of patients at a set of equally spaced doses. Up-and-down is the most popular family of dose-finding designs and is in common use in anesthesiology. Despite its widespread use, many aspects of up-and-down are not well known, implementation is often misguided, and standard, up-to-date reference material about the design is very limited. This article provides an overview of up-and-down properties, recent methodologic developments, and practical recommendations, illustrated with the help of simulated examples. Additional reference material is offered in the Supplemental Digital Content.

Distribution Theory Following Blinded and Unblinded Sample Size Re-estimation under Parametric Models.

Asymptotic distribution theory for maximum likelihood estimators under fixed alternative hypotheses is reported in the literature even though the power of any realistic test converges to one under fixed alternatives. Under fixed alternatives, authors have established that nuisance parameter estimates are inconsistent when sample size re-estimation (SSR) follows blinded randomization. These results have helped to inhibit the use of SSR. In this paper, we argue for local alternatives to be used instead of fixed alternatives. Motivated by Gould and Shih (1998), we treat unavailable treatment assignments in blinded experiments as missing data and rely on single imputation from marginal distributions to fill in for missing data. With local alternatives, it is sufficient to proceed only with the first step of the EM algorithm mimicking imputation under the null hypothesis. Then, we show that blinded and unblinded estimates of the nuisance parameter σ θ 2 are consistent, and re-estimated sample sizes converge to their locally asymptotically optimal values. This theoretical finding is confirmed through Monte-Carlo simulation studies. Practical utility is illustrated through a multiple logistic regression example. We conclude that, for hypothesis testing with a predetermined minimally clinically relevant local effect size, both blinded and unblinded SSR procedures lead to similar sample sizes and power.

Most Powerful Test Sequences with Early Stopping Options.

We extended the application of uniformly most powerful tests to sequential tests with different stage-specific sample sizes and critical regions. In the one parameter exponential family, likelihood ratio sequential tests are shown to be uniformly most powerful for any predetermined α-spending function and stage-specific sample sizes. To obtain this result, the probability measure of a group sequential design is constructed with support for all possible outcome events, as is useful for designing an experiment prior to having data. This construction identifies impossible events that are not part of the support. The overall probability distribution is dissected into components determined by the stopping stage. These components are the sub-densities of interim test statistics first described by Armitage, McPherson and Rowe (1969) that are commonly used to create stopping boundaries given an α-spending function and a set of interim analysis times. Likelihood expressions conditional on reaching a stage are given to connect pieces of the probability anatomy together. The reduction of support caused by the adoption of an early stopping rule induces sequential truncation (not nesting) in the probability distributions of possible events. Multiple testing induces mixtures on the adapted support. Even asymptotic distributions of inferential statistics that are typically normal, are not. Rather they are derived from mixtures of truncated multivariate normal distributions.

Choosing Interim Sample Sizes in Group Sequential Designs.

This manuscript investigates sample sizes for interim analyses in group sequential designs. Traditional group sequential designs (GSD) rely on "information fraction" arguments to define the interim sample sizes. Then, interim maximum likelihood estimators (MLEs) are used to decide whether to stop early or continue the data collection until the next interim analysis. The possibility of early stopping changes the distribution of interim and final MLEs: possible interim decisions on trial stopping excludes some sample space elements. At each interim analysis the distribution of an interim MLE is a mixture of truncated and untruncated distributions. The distributional form of an MLE becomes more and more complicated with each additional interim analysis. Test statistics that are asymptotically normal without a possibly of early stopping, become mixtures of truncated normal distributions under local alternatives. Stage-specific information ratios are equivalent to sample size ratios for independent and identically distributed data. This equivalence is used to justify interim sample sizes in GSDs. Because stage-specific information ratios derived from normally distributed data differ from those derived from non-normally distributed data, the former equivalence is invalid when there is a possibility of early stopping. Tarima and Flournoy [3] have proposed a new GSD where interim sample sizes are determined by a pre-defined sequence of ordered alternative hypotheses, and the calculation of information fractions is not needed. This innovation allows researchers to prescribe interim analyses based on desired power properties. This work compares interim power properties of a classical one-sided three stage Pocock design with a one-sided three stage design driven by three ordered alternatives.

Bias induced by adaptive dose-finding designs.

There is a long literature on bias in maximum likelihood estimators. Here we demonstrate that adaptive dose-finding procedures (such as Continual Reassessment Methods, Up-and-Down and Interval Designs) themselves induce bias. In particular, with Bernoulli responses and dose assignments that depend on prior responses, we provide an explicit formula for the bias of observed response rates. We illustrate the patterns of bias for designs that aim to concentrate dose allocations around a target dose, which represents a specific quantile of a cumulative response-threshold distribution. For such designs, bias tends to be positive above the target dose and negative below it. To our knowledge, this property of dose-finding designs has not previously been recognized by design developers. We discuss the implications of this bias and suggest a simple shrinkage mitigation formula that improves estimation at doses away from the target.

Asymptotic properties of maximum likelihood estimators with sample size recalculation.

Consider an experiment in which the primary objective is to determine the significance of a treatment effect at a predetermined type I error and statistical power. Assume that the sample size required to maintain these type I error and power will be re-estimated at an interim analysis. A secondary objective is to estimate the treatment effect. Our main finding is that the asymptotic distributions of standardized statistics are random mixtures of distributions, which are non-normal except under certain model choices for sample size re-estimation (SSR). Monte-Carlo simulation studies and an illustrative example highlight the fact that asymptotic distributions of estimators with SSR may differ from the asymptotic distribution of the same estimators without SSR.

Comparison of Isotonic Designs for Dose-Finding.

We compare several decision rules for allocating subjects to dosages that are based on sequential isotonic estimates of a monotone dose-toxicity curve. We conclude that the decision rule in which the next assignment is to the dose having probability of toxicity closest to target does not work well. The best rule in our comparison is given by the cumulative cohort design. According to this design, the dose for the next subject is decreased, increased, or repeated depending on the distance between the estimated toxicity rate at the current dose and the target quantile.

Development and validation of a measure of dysmorphology: useful for autism subgroup classification.

Autism spectrum disorders (ASD) comprise a class of neurodevelopmental disorders that can originate from a variety of genetic and environmental causes. To delineate autism's heterogeneity we have looked for biologically-based phenotypes found in consistent proportions of ASD individuals. One informative phenotype is that of generalized dysmorphology, based on whole body examinations by medical geneticists trained in the nuances of anomalous embryologic development. We identified a need for a dysmorphology measure that could be completed by medical clinicians not extensively trained in dysmorphology that would still retain the level of sensitivity and specificity of the comprehensive dysmorphology examination. Based on expert-derived consensus dysmorphology designation of 222 autism patients and a classification validation study of 30 subjects by four dysmorphologists, we determined that dysmorphology designations based on body areas provided superior inter-rater reliability. Using 34 body area designations, we performed a classification and regression tree (CART) analysis to construct a scoring algorithm. Compared to the consensus classification, the model performed with 81% sensitivity and 99% specificity, and classification of a replication dataset of 31 ASD individuals performed well, with 82% sensitivity and 95% specificity. The autism dysmorphology measure (ADM) directs the clinician to score 12 body areas sequentially to arrive at a determination of "dysmorphic" or "nondysmorphic." We anticipate the ADM will permit clinicians to differentiate accurately between dysmorphic and nondysmorphic individuals-allowing better diagnostic classification, prognostication, recurrence risk assessment, and laboratory analysis decisions-and research scientists to better define more homogeneous autism subtypes.

Quantile estimation following non-parametric phase I clinical trials with ordinal response.

A non-parametric multi-dimensional isotonic regression estimator is developed for use in estimating a set of target quantiles from an ordinal toxicity scale. We compare this estimator to the standard parametric maximum likelihood estimator from a proportional odds model for extremely small data sets. A motivating example is from phase I oncology clinical trials, where various non-parametric designs have been proposed that lead to very small data sets, often with ordinal toxicity response data. Our comparison of estimators is performed in conjunction with three of these non-parametric sequential designs for ordinal response data, two from the literature and a new design based on a random walk rule. We also compare with a non-parametric design for binary response trials, by keeping track of ordinal data for estimation purposes, but dichotomizing the data in the design phase. We find that a multidimensional isotonic regression-based estimator far exceeds the others in terms of accuracy and efficiency. A rule by Simon et al. (J. Natl. Cancer Inst. 1997; 89:1138-1147) yields particularly efficient estimators, more so than the random walk rule, but has higher numbers of dose-limiting toxicity. A small data set from a leukemia clinical trial is analysed using our multidimensional isotonic regression-based estimator.

Estimating the probability of toxicity at the target dose following an up-and-down design.

One of the most important aspects of a phase I trial or other acute toxicity study is estimating accurately the probability of toxicity that is associated with the recommended dose. We use the biased coin up-and-down design to allocate and isotonic regression to estimate toxicity probabilities and determine the recommended dose. We then derive, using bootstrap methods, an estimate of the probability of toxicity at the recommended dose. Small sample properties of this estimator are also evaluated. Published in 2003 by John Wiley & Sons, Ltd.

Dose finding using the biased coin up-and-down design and isotonic regression.

We are interested in finding a dose that has a prespecified toxicity rate in the target population. In this article, we investigate five estimators of the target dose to be used with the up-and-down biased coin design (BCD) introduced by Durham and Flournoy (1994, Statistical Decision Theory and Related Topics). These estimators are derived using maximum likelihood, weighted least squares, sample averages, and isotonic regression. A linearly interpolated isotonic regression estimate is shown to be simple to derive and to perform as well as or better than the other target dose estimators in terms of mean square error and average number of subjects needed for convergence in most scenarios studied.