Constrained randomization and statistical inference for multi-arm parallel cluster randomized controlled trials.

A practical limitation of cluster randomized controlled trials (cRCTs) is that the number of available clusters may be small, resulting in an increased risk of baseline imbalance under simple randomization. Constrained randomization overcomes this issue by restricting the allocation to a subset of randomization schemes where sufficient overall covariate balance across comparison arms is achieved. However, for multi-arm cRCTs, several design and analysis issues pertaining to constrained randomization have not been fully investigated. Motivated by an ongoing multi-arm cRCT, we elaborate the method of constrained randomization and provide a comprehensive evaluation of the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs in multi-arm cRCTs, with varying combinations of design and analysis-based covariate adjustment strategies. In particular, as randomization-based tests have not been extensively studied in multi-arm cRCTs, we additionally develop most-powerful randomization tests under the linear mixed model framework for our comparisons. Our results indicate that under constrained randomization, both model-based and randomization-based analyses could gain power while preserving nominal type I error rate, given proper analysis-based adjustment for the baseline covariates. Randomization-based analyses, however, are more robust against violations of distributional assumptions. The choice of balance metrics and candidate set sizes and their implications on the testing of the pairwise and global hypotheses are also discussed. Finally, we caution against the design and analysis of multi-arm cRCTs with an extremely small number of clusters, due to insufficient degrees of freedom and the tendency to obtain an overly restricted randomization space.

A roadmap to using randomization in clinical trials.

BACKGROUND: Randomization is the foundation of any clinical trial involving treatment comparison. It helps mitigate selection bias, promotes similarity of treatment groups with respect to important known and unknown confounders, and contributes to the validity of statistical tests. Various restricted randomization procedures with different probabilistic structures and different statistical properties are available. The goal of this paper is to present a systematic roadmap for the choice and application of a restricted randomization procedure in a clinical trial. METHODS: We survey available restricted randomization procedures for sequential allocation of subjects in a randomized, comparative, parallel group clinical trial with equal (1:1) allocation. We explore statistical properties of these procedures, including balance/randomness tradeoff, type I error rate and power. We perform head-to-head comparisons of different procedures through simulation under various experimental scenarios, including cases when common model assumptions are violated. We also provide some real-life clinical trial examples to illustrate the thinking process for selecting a randomization procedure for implementation in practice. RESULTS: Restricted randomization procedures targeting 1:1 allocation vary in the degree of balance/randomness they induce, and more importantly, they vary in terms of validity and efficiency of statistical inference when common model assumptions are violated (e.g. when outcomes are affected by a linear time trend; measurement error distribution is misspecified; or selection bias is introduced in the experiment). Some procedures are more robust than others. Covariate-adjusted analysis may be essential to ensure validity of the results. Special considerations are required when selecting a randomization procedure for a clinical trial with very small sample size. CONCLUSIONS: The choice of randomization design, data analytic technique (parametric or nonparametric), and analysis strategy (randomization-based or population model-based) are all very important considerations. Randomization-based tests are robust and valid alternatives to likelihood-based tests and should be considered more frequently by clinical investigators.

ERDO - a framework to select an appropriate randomization procedure for clinical trials.

BACKGROUND: Randomization is considered to be a key feature to protect against bias in randomized clinical trials. Randomization induces comparability with respect to known and unknown covariates, mitigates selection bias, and provides a basis for inference. Although various randomization procedures have been proposed, no single procedure performs uniformly best. In the design phase of a clinical trial, the scientist has to decide which randomization procedure to use, taking into account the practical setting of the trial with respect to the potential of bias. Less emphasis has been placed on this important design decision than on analysis, and less support has been available to guide the scientist in making this decision. METHODS: We propose a framework that weights the properties of the randomization procedure with respect to practical needs of the research question to be answered by the clinical trial. In particular, the framework assesses the impact of chronological and selection bias on the probability of a type I error. The framework is applied to a case study with a 2-arm parallel group, single center randomized clinical trial with continuous endpoint, with no-interim analysis, 1:1 allocation and no adaptation in the randomization process. RESULTS: In so doing, we derive scientific arguments for the selection of an appropriate randomization procedure and develop a template which is illustrated in parallel by a case study. Possible extensions are discussed. CONCLUSION: The proposed ERDO framework guides the investigator through a template for the choice of a randomization procedure, and provides easy to use tools for the assessment. The barriers for the thorough reporting and assessment of randomization procedures could be further reduced in the future when regulators and pharmaceutical companies employ similar, standardized frameworks for the choice of a randomization procedure.

A SAS Macro for Covariate-Constrained Randomization of General Cluster-Randomized and Unstratified Designs.

Ivers et al. (2012) have recently stressed the importance to both statistical power and face validity of balancing allocations to study arms on relevant covariates. While several techniques exist (e.g., minimization, pair-matching, stratification), the covariate-constrained randomization (CCR) approach proposed by Moulton (2004) is favored when clusters can be recruited prior to randomization. CCRA V1.0, a macro published by Chaudhary and Moulton (2006), provides a SAS implementation of CCR for a particular subset of possible designs (those with two arms, small numbers of strata and clusters, an equal number of clusters within each stratum, and constraints that can be expressed as absolute mean differences between arms). This paper presents a more comprehensive macro, CCR, that is applicable across a wider variety of designs and provides statistics describing the range of possible allocations meeting the constraints in addition to performing the actual random assignment.

On the use of randomization tests following adaptive designs.

Randomization tests (sometimes referred to as "re-randomization" tests) are used in clinical trials, either as an assumption-free confirmation of parametric analyses, or as an independent analysis based on the principle of randomization-based inference. In the context of adaptive randomization, either restricted or response-adaptive procedures, it is unclear how accurate such Monte Carlo approximations are, or how many Monte Carlo sequences to generate. In this paper, we describe several randomization procedures for which there is a known exact or asymptotic distribution of the randomization test. For a special class of procedures, called [Formula: see text], and binary responses, the exact test statistic has a simple closed form. For the limited subset of existing procedures with known exact and asymptotic distributions, we can use these as a benchmark for the accuracy of Monte Carlo randomization techniques. We conclude that Monte Carlo tests are very accurate, and require minimal computation time. For simple tests with binary response in the class of [Formula: see text] procedures, the exact distribution provides the best test, but Monte Carlo approximations can be used when the exact distribution is difficult to compute.

Inference for blocked randomization under a selection bias model.

We provide an asymptotic test to analyze randomized clinical trials that may be subject to selection bias. For normally distributed responses, and under permuted block randomization, we derive a likelihood ratio test of the treatment effect under a selection bias model. A likelihood ratio test of the presence of selection bias arises from the same formulation. We prove that the test is asymptotically chi-square on one degree of freedom. These results correlate well with the likelihood ratio test of Ivanova et al. (2005, Statistics in Medicine 24, 1537-1546) for binary responses, for which they established by simulation that the asymptotic distribution is chi-square. Simulations also show that the test is robust to departures from normality and under another randomization procedure. We illustrate the test by reanalyzing a clinical trial on retinal detachment.

Exact optimum coin bias in Efron's randomization procedure.

Efron's biased coin design is a restricted randomization procedure that has very favorable balancing properties, yet it is fully randomized, in that subjects are always randomized to one of two treatments with a probability less than 1. The parameter of interest is the bias p of the coin, which can range from 0.5 to 1. In this note, we propose a compound optimization strategy that selects p based on a subjected weighting of the relative importance of the two fundamental criteria of interest for restricted randomization mechanisms, namely balance between the treatment assignments and allocation randomness. We use exact and asymptotic distributional properties of Efron's coin to find the optimal p under compound criteria involving imbalance variability, expected imbalance, selection bias, and accidental bias, for both small/moderate trials and large samples.

Brick tunnel randomization and the momentum of the probability mass.

The allocation space of an unequal-allocation permuted block randomization can be quite wide. The development of unequal-allocation procedures with a narrower allocation space, however, is complicated by the need to preserve the unconditional allocation ratio at every step (the allocation ratio preserving (ARP) property). When the allocation paths are depicted on the K-dimensional unitary grid, where allocation to the l-th treatment is represented by a step along the l-th axis, l = 1 to K, the ARP property can be expressed in terms of the center of the probability mass after i allocations. Specifically, for an ARP allocation procedure that randomizes subjects to K treatment groups in w1 :⋯:wK ratio, w1 +⋯+wK =1, the coordinates of the center of the mass are (w1 i,…,wK i). In this paper, the momentum with respect to the center of the probability mass (expected imbalance in treatment assignments) is used to compare ARP procedures in how closely they approximate the target allocation ratio. It is shown that the two-arm and three-arm brick tunnel randomizations (BTR) are the ARP allocation procedures with the tightest allocation space among all allocation procedures with the same allocation ratio; the two-arm BTR is the minimum-momentum two-arm ARP allocation procedure. Resident probabilities of two-arm and three-arm BTR are analytically derived from the coordinates of the center of the probability mass; the existence of the respective transition probabilities is proven. Probability of deterministic assignments with BTR is found generally acceptable. Copyright © 2015 John Wiley & Sons, Ltd.

Randomization, matching, and propensity scores in the design and analysis of experimental studies with measured baseline covariates.

In many experimental situations, researchers have information on a number of covariates prior to randomization. This information can be used to balance treatment assignment with respect to these covariates as well as in the analysis of the outcome data. In this paper, we investigate the use of propensity scores in both of these roles. We also introduce a randomization procedure in which the balance of all measured covariates is approximately indexed by the variance of the empirical propensity scores and randomization is restricted to those permutations with the least variable propensity scores. This procedure is compared with recently proposed methods in terms of resulting covariate balance and estimation efficiency. Properties of the estimators resulting from each procedure are compared with estimates which incorporate the propensity score in the analysis stage. Simulation results show that analytical adjustment for the propensity score yields results on par with those obtained through restricted randomization procedures and can be used in conjunction with such procedures to further improve inferential efficiency.

Simple compared to covariate-constrained randomization methods in balancing baseline characteristics: a case study of randomly allocating 72 hemodialysis centers in a cluster trial.

BACKGROUND AND AIM: Some parallel-group cluster-randomized trials use covariate-constrained rather than simple randomization. This is done to increase the chance of balancing the groups on cluster- and patient-level baseline characteristics. This study assessed how well two covariate-constrained randomization methods balanced baseline characteristics compared with simple randomization. METHODS: We conducted a mock 3-year cluster-randomized trial, with no active intervention, that started April 1, 2014, and ended March 31, 2017. We included a total of 11,832 patients from 72 hemodialysis centers (clusters) in Ontario, Canada. We randomly allocated the 72 clusters into two groups in a 1:1 ratio on a single date using individual- and cluster-level data available until April 1, 2013. Initially, we generated 1000 allocation schemes using simple randomization. Then, as an alternative, we performed covariate-constrained randomization based on historical data from these centers. In one analysis, we restricted on a set of 11 individual-level prognostic variables; in the other, we restricted on principal components generated using 29 baseline historical variables. We created 300,000 different allocations for the covariate-constrained randomizations, and we restricted our analysis to the 30,000 best allocations based on the smallest sum of the penalized standardized differences. We then randomly sampled 1000 schemes from the 30,000 best allocations. We summarized our results with each randomization approach as the median (25th and 75th percentile) number of balanced baseline characteristics. There were 156 baseline characteristics, and a variable was balanced when the between-group standardized difference was ≤ 10%. RESULTS: The three randomization techniques had at least 125 of 156 balanced baseline characteristics in 90% of sampled allocations. The median number of balanced baseline characteristics using simple randomization was 147 (142, 150). The corresponding value for covariate-constrained randomization using 11 prognostic characteristics was 149 (146, 151), while for principal components, the value was 150 (147, 151). CONCLUSION: In this setting with 72 clusters, constraining the randomization using historical information achieved better balance on baseline characteristics compared with simple randomization; however, the magnitude of benefit was modest.

The randomization-induced risk of a trial failing to attain its target power: assessment and mitigation.

Health researchers are familiar with the concept of trial power, a number that prior to the start of a trial is intended to describe the probability that the results of the trial will correctly conclude that the intervention has an effect. Trial power, as calculated using standard software, is an expected power that arises from averaging hypothetical trial results over all possible treatment allocations that could be generated by the randomization algorithm. However, in the trial that ultimately is conducted, only one treatment allocation will occur, and the corresponding attained power (conditional on the allocation that occurred) is not guaranteed to be equal to the expected power and may be substantially lower. We provide examples illustrating this issue, discuss some circumstances when this issue is a concern, define and advocate the examination of the pre-randomization power distribution for evaluating the risk of obtaining unacceptably low attained power, and suggest the use of randomization restrictions to reduce this risk. In trials that randomize only a modest number of units, we recommend that trial designers evaluate the risk of getting low attained power and, if warranted, modify the randomization algorithm to reduce this risk.

Use and reporting of restricted randomization: a review.

RATIONALE, AIMS AND OBJECTIVES: Restricted randomization, such as blocking or minimization, allows for the creation of balanced groups and even distribution of covariates, but it increases the risk of selection bias and technical error. Various methods are available to reduce these risks but there is limited evidence about their current usage, and there are also indications that reporting of these methods may not be adequate. This review aims to identify how frequently different methods of restriction are being used and to assess the reporting of these methods against established reporting standards. METHODS: 82 reports of randomized controlled trial were reviewed. For each trial, the reported method of randomization was recorded and the reporting of randomization was assessed. Where the method of randomization was not clear from the main paper, protocols and other published materials were also reviewed, and authors were contacted for further information. RESULTS: For 11% of trials the method of randomization was not reported in either the paper or a published protocol, and in a further 39% of cases the report omitted key details so that the predictability of the method could not be evaluated. In total, 88% of trials appear to have used some form of restricted randomization, and all of those that report the exact methods used either blocking or minimization. 15% of trials reported using blocks of six or less and 4% used minimization with no random element reported, both of which are highly predictable. CONCLUSION: Our results indicate that the majority of trials use some form of restriction, with many using relatively predictable methods that put them at greater risk of selection bias and technical error. Reporting of randomization methods often falls short of the minimum requirements set out by the CONSORT statement, leaving the reader unable to make an informed judgement about the risk of bias.

A review of the use of covariates in cluster randomized trials uncovers marked discrepancies between guidance and practice.

OBJECTIVES: Reviews of the handling of covariates in trials have explicitly excluded cluster randomized trials (CRTs). In this study, we review the use of covariates in randomization, the reporting of covariates, and adjusted analyses in CRTs. STUDY DESIGN AND SETTING: We reviewed a random sample of 300 CRTs published between 2000 and 2008 across 150 English language journals. RESULTS: Fifty-eight percent of trials used covariates in randomization. Only 69 (23%) included tables of cluster- and individual-level covariates. Fifty-eight percent reported significance tests of baseline balance. Of 207 trials that reported baseline measures of the primary outcome, 155 (75%) subsequently adjusted for these in analyses. Of 174 trials that used covariates in randomization, 30 (17%) included an analysis adjusting for all those covariates. Of 219 trial reports that included an adjusted analysis of the primary outcome, only 71 (32%) reported that covariates were chosen a priori. CONCLUSION: There are some marked discrepancies between practice and guidance on the use of covariates in the design, analysis, and reporting of CRTs. It is essential that researchers follow guidelines on the use and reporting of covariates in CRTs, promoting the validity of trial conclusions and quality of trial reports.