Analysis of adaptive platform trials using a network approach.

BACKGROUND: Adaptive platform trials allow randomized controlled comparisons of multiple treatments using a common infrastructure and the flexibility to adapt key design features during the study. Nonetheless, they have been criticized due to the potential for time trends in the underlying risk level of the population. Such time trends lead to confounding between design features and risk level, which may introduce bias favoring one or more treatments. This is particularly true when experimental treatments are not all randomized during the same time period as the control, leading to the potential for bias from non-concurrent controls. METHODS: Two analysis methods addressing this bias are stratification and adjustment. Stratification uses only comparisons between treatment cohorts randomized during identical time periods and does not use non-concurrent randomizations. Adjustment uses a modeled analysis including time period adjustment, allowing all data to be used, even from periods without concurrent randomization. We show that these competing approaches may be embedded in a common framework using network meta-analysis principles. We interpret the stages between adaptations in a platform trial as separate fixed design trials. This allows platform trials to be viewed as networks of direct randomized comparisons and indirect non-randomized comparisons. Network meta-analysis methodology can be re-purposed to aggregate the total information from a platform trial and to transparently decompose this total information into direct randomized evidence and indirect non-randomized evidence. This allows sensitivity to indirect information to be assessed and the two analysis methods to be clearly compared. RESULTS: Simulations of platform trials were analyzed using a network approach implemented in the netmeta package in R. The results demonstrated bias of unadjusted methods in the presence of time trends in risk level. Adjustment and stratification were both unbiased when direct evidence and indirect evidence were consistent. Network tests of inconsistency may be used to diagnose inconsistency when it exists. In an illustrative network analysis of one of the treatment comparisons from the STAMPEDE platform trial in metastatic prostate cancer, indirect comparisons using non-concurrent controls were inconsistent with the information from direct randomized comparisons. This supports the primary analysis approach of STAMPEDE, which used only direct randomized comparisons. CONCLUSION: Network meta-analysis provides a natural methodology for analyzing the network of direct and indirect treatment comparisons from a platform trial. Such analyses provide transparent separation of direct and indirect evidence, allowing assessment of the impact of non-concurrent controls. We recommend time-stratified analysis of concurrently controlled comparisons for primary analyses, with time-adjusted analyses incorporating non-concurrent controls reserved for secondary analyses. However, regardless of which methodology is used, a network analysis provides a useful supplement to the primary analysis.

Design of clinical trials involving multiple hypothesis tests with a common control.

Randomized clinical trials comparing several treatments to a common control are often reported in the medical literature. For example, multiple experimental treatments may be compared with placebo, or in combination therapy trials, a combination therapy may be compared with each of its constituent monotherapies. Such trials are typically designed using a balanced approach in which equal numbers of individuals are randomized to each arm, however, this can result in an inefficient use of resources. We provide a unified framework and new theoretical results for optimal design of such single-control multiple-comparator studies. We consider variance optimal designs based on D-, A-, and E-optimality criteria, using a general model that allows for heteroscedasticity and a range of effect measures that include both continuous and binary outcomes. We demonstrate the sensitivity of these designs to the type of optimality criterion by showing that the optimal allocation ratios are systematically ordered according to the optimality criterion. Given this sensitivity to the optimality criterion, we argue that power optimality is a more suitable approach when designing clinical trials where testing is the objective. Weighted variance optimal designs are also discussed, which, like power optimal designs, allow the treatment difference to play a major role in determining allocation ratios. We illustrate our methods using two real clinical trial examples taken from the medical literature. Some recommendations on the use of optimal designs in single-control multiple-comparator trials are also provided.

Methods for exploring treatment effect heterogeneity in subgroup analysis: an application to global clinical trials.

Multi-country randomised clinical trials (MRCTs) are common in the medical literature, and their interpretation has been the subject of extensive recent discussion. In many MRCTs, an evaluation of treatment effect homogeneity across countries or regions is conducted. Subgroup analysis principles require a significant test of interaction in order to claim heterogeneity of treatment effect across subgroups, such as countries in an MRCT. As clinical trials are typically underpowered for tests of interaction, overly optimistic expectations of treatment effect homogeneity can lead researchers, regulators and other stakeholders to over-interpret apparent differences between subgroups even when heterogeneity tests are insignificant. In this paper, we consider some exploratory analysis tools to address this issue. We present three measures derived using the theory of order statistics, which can be used to understand the magnitude and the nature of the variation in treatment effects that can arise merely as an artefact of chance. These measures are not intended to replace a formal test of interaction but instead provide non-inferential visual aids, which allow comparison of the observed and expected differences between regions or other subgroups and are a useful supplement to a formal test of interaction. We discuss how our methodology differs from recently published methods addressing the same issue. A case study of our approach is presented using data from the Study of Platelet Inhibition and Patient Outcomes (PLATO), which was a large cardiovascular MRCT that has been the subject of controversy in the literature. An R package is available that implements the proposed methods.

Meta-analysis of clinical trials with early stopping: an investigation of potential bias.

Clinical trials that stop early for benefit have a treatment difference that overestimates the true effect. The consequences of this fact have been extensively debated in the literature. Some researchers argue that early stopping, or truncation, is an important source of bias in treatment effect estimates, particularly when truncated studies are incorporated into meta-analyses. Such claims are bound to lead some systematic reviewers to consider excluding truncated studies from evidence synthesis. We therefore investigated the implications of this strategy by examining the properties of sequentially monitored studies conditional on reaching the final analysis. As well as estimation bias, we studied information bias measured by the difference between standard measures of statistical information, such as sample size, and the actual information based on the conditional sampling distribution. We found that excluding truncated studies leads to underestimation of treatment effects and overestimation of information. Importantly, the information bias increases with the estimation bias, meaning that greater estimation bias is accompanied by greater overweighting in a meta-analysis. Simulations of meta-analyses confirmed that the bias from excluding truncated studies can be substantial. In contrast, when meta-analyses included truncated studies, treatment effect estimates were essentially unbiased. Previous analyses comparing treatment effects in truncated and non-truncated studies are shown not to be indicative of bias in truncated studies. We conclude that early stopping of clinical trials is not a substantive source of bias in meta-analyses and recommend that all studies, both truncated and non-truncated, be included in evidence synthesis.

Long-term inhaled dry powder mannitol in cystic fibrosis: an international randomized study.

RATIONALE: New treatment strategies are needed to improve airway clearance and reduce the morbidity and the time burden associated with cystic fibrosis (CF). OBJECTIVES: To determine whether long-term treatment with inhaled mannitol, an osmotic agent, improves lung function and morbidity. METHODS: Double-blind, randomized, controlled trial of inhaled mannitol, 400 mg twice a day (n = 192, "treated" group) or 50 mg twice a day (n = 126, "control" group) for 26 weeks, followed by 26 weeks of open-label treatment. MEASUREMENTS AND MAIN RESULTS: The primary endpoint was absolute change in FEV(1) from baseline in treated versus control groups, averaged over the study period. Secondary endpoints included other spirometric measurements, pulmonary exacerbations, and hospitalization. Clinical, microbiologic, and laboratory safety were assessed. The treated group had a mean improvement in FEV(1) of 105 ml (8.2% above baseline). The treated group had a relative improvement in FEV(1) of 3.75% (P = 0.029) versus the control group. Adverse events and sputum microbiology were similar in both treatment groups. Exacerbation rates were low, but there were fewer in the treated group (hazard ratio, 0.74; 95% confidence interval, 0.42-1.32; P = 0.31), although this was not statistically significant. In the 26-week open-label extension study, FEV(1) was maintained in the original treated group, and improved in the original control group to the same degree. CONCLUSIONS: Inhaled mannitol, 400 mg twice a day, resulted in improved lung function over 26 weeks, which was sustained after an additional 26 weeks of treatment. The safety profile was also acceptable, demonstrating the potential role for this chronic therapy for CF. Clinical trial registered with www.clinicaltrials.gov (NCT 00630812).

Reacting to prognostic covariate imbalance in randomised controlled trials.

Clinical trialists may regard an observed imbalance on a prognostic covariate as sufficiently troubling to warrant action. OBJECTIVE: To elucidate the issues associated with selecting, and switching between, an unadjusted versus an adjusted analysis in response to an observed covariate imbalance. STUDY DESIGN AND SETTING: Simulation study performed under the null hypothesis of no treatment effect using data from a large secondary prevention trial of statin therapy. The operating characteristics of three reaction strategies to baseline imbalances observed post-hoc were assessed. RESULTS: Unadjusted analyses produced valid p-values irrespective of chance imbalance on a prognostic covariate. Switching to an adjusted analysis introduced no bias when the decision was made without knowledge of the direction of the imbalance. When the decision was based on the direction of the imbalance, the risk of incorrectly declaring the experimental treatment superior was inflated (by up to 48% in the scenarios investigated). CONCLUSION: Overreaction to baseline imbalances observed post-hoc is unwarranted and we support adherence to the ICH guideline recommendations on the use of covariates. A legitimate case for switching to an adjusted analysis prior to finalisation of the statistical analysis plan (SAP) could nevertheless be potentially made provided that the direction of an observed covariate imbalance is unknown. Investigators should avoid reviewing the distribution of baseline characteristics across randomised groups in an unblinded fashion, for open-label and blinded studies alike, prior to finalisation of the SAP. WHAT IS NEW: ICH guidelines on adjustment for covariates in RCT analyses appropriately advise against overreaction to baseline imbalances observed post-hoc. CONSORT reporting guidelines nevertheless place an emphasis on comparability of baseline characteristics across randomised groups. We demonstrate through a series of simulation studies why the ICH guidance is sound, but that a switch to an adjusted analysis in reaction to an observed prognostic covariate imbalance could legitimately be made provided that, when reaching the decision, treatment allocation is masked, and the direction of the imbalance is unknown. Trialists should therefore consider preserving the masking of actual treatment assignment when assessing the distribution of baseline characteristics across randomised groups.

Sensitivity analyses assessing the impact of early stopping on systematic reviews: Recommendations for interpreting guidelines.

The CONSORT Statement says that data-driven early stopping of a clinical trial is likely to weaken the inferences that can be drawn from the trial. The GRADE guidelines go further, saying that early stopping is a study limitation that carries the risk of bias, and recommending sensitivity analyses in which trials stopped early are omitted from evidence synthesis. Despite extensive debate in the literature over these issues, the existence of clear recommendations in high profile guidelines makes it inevitable that systematic reviewers will consider sensitivity analyses investigating the impact of early stopping. The purpose of this article is to assess methodologies for conducting such sensitivity analyses, and to make recommendations about how the guidelines should be interpreted. We begin with a clarifying overview of the impacts of early stopping on treatment effect estimation in single studies and meta-analyses. We then warn against naive approaches for conducting sensitivity analyses, including simply omitting trials stopped early from meta-analyses. This approach underestimates treatment effects, which may have serious implications if cost-effectiveness analyses determine whether treatments are made widely available. Instead, we discuss two unbiased approaches to sensitivity analysis, one of which is straightforward but statistically inefficient, and the other of which achieves greater statistical efficiency by making use of recent methodological developments in the analysis of clinical trials. We end with recommendations for interpreting: (a) the CONSORT Statement on reporting of reasons for early stopping, and (b) the GRADE guidelines on sensitivity analyses assessing the impact of early stopping.

Underestimation of treatment effects in sequentially monitored clinical trials that did not stop early for benefit.

In recent years, there has been a prominent discussion in the literature about the potential for overestimation of the treatment effect when a clinical trial stops at an interim analysis due to the experimental treatment showing a benefit over the control. However, there has been much less attention paid to the converse issue, namely, that sequentially monitored clinical trials which did not stop early for benefit tend to underestimate the treatment effect. In meta-analyses of many studies, these two sources of bias will tend to balance each other to produce an unbiased estimate of the treatment effect. However, for the interpretation of a single study in isolation, underestimation due to interim analysis may be an important consideration. In this paper, we discuss the nature of this underestimation, including theoretical and simulation results demonstrating that it can be substantial in some contexts. Furthermore, we show how a conditional approach to estimation, in which we condition on the study reaching its final analysis, may be used to validly inflate the observed treatment difference from a sequentially monitored clinical trial. Expressions for the conditional bias and information are derived, and these are used in supplied R code that computes the bias-adjusted estimate and an associated confidence interval. As well as simulation results demonstrating the validity of the methods, we present a data analysis example from a pivotal clinical trial in cardiovascular disease. The methods will be most useful when an unbiased treatment effect estimate is critical, such as in cost-effectiveness analysis or risk prediction.