Estimating treatment effect in randomized trial after control to treatment crossover using external controls.

In clinical trials, it is common to design a study that permits the administration of an experimental treatment to participants in the placebo or standard of care group post primary endpoint. This is often seen in the open-label extension phase of a phase III, pivotal study of the new medicine, where the focus is on assessing long-term safety and efficacy. With the availability of external controls, proper estimation and inference of long-term treatment effect during the open-label extension phase in the absence of placebo-controlled patients are now feasible. Within the framework of causal inference, we propose several difference-in-differences (DID) type methods and a synthetic control method (SCM) for the combination of randomized controlled trials and external controls. Our realistic simulation studies demonstrate the desirable performance of the proposed estimators in a variety of practical scenarios. In particular, DID methods outperform SCM and are the recommended methods of choice. An empirical application of the methods is demonstrated through a phase III clinical trial in rare disease.

A cautionary tale: an evaluation of the performance of treatment switching adjustment methods in a real world case study.

BACKGROUND: Treatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82-1.22) for all patients, and 0.99 (95% CI 0.75-1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus). METHODS: We tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76-0.77 (95% CI 0.60-0.98) for all patients, and 0.55-0.73 (95% CI 0.41-0.93) for patients with wild-type KRAS. RESULTS: Some applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals. CONCLUSIONS: Adjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability.

PubPredict: Prediction of progression and survival in oncology leveraging publications and early efficacy data.

In oncology/hematology early phase clinical trials, efficacies were often observed in terms of response rate, depth, timing, and duration. However, the true clinical benefits that eventually support registrational purpose are progression-free survival (PFS) and/or overall survival (OS), the follow-up of which are typically not long enough in early phase trials. This gap imposes challenges in strategies for late phase drug development. In this article, we tackle the question by leveraging published study to establish a quantitative link between early efficacy outcomes and late phase efficacy endpoints. We used solid tumor cancer as disease model. We modeled the disease course of a RECISTv1.1 assessed solid tumor with a continuous Markov chain (CMC) model. We parameterize the transition intensity matrix of a CMC model based on published aggregate-level summary statistics, and then simulate subject-level time-to-event data. The simulated data is shown to have good approximation to published studies. PFS and/or OS could be predicted with the transition intensity matrix modified given clinical knowledge to reflect various assumptions on response rate, depth, timing, and duration. The authors have built a R shiny application named PubPredict, the tool implements the algorithm described above and allows customized features including multiple response levels, treatment crossover and varying follow-up duration. This toolset has been applied to advise phase 3 trial design when only early efficacy data are available from phase 1 or 2 studies.

Structural cumulative survival models for estimation of treatment effects accounting for treatment switching in randomized experiments.

Treatment switching in a randomized controlled trial occurs when a patient in one treatment arm switches to another arm during follow-up. This can occur at the point of disease progression, whereby patients in the control arm may be offered the experimental treatment. It is widely known that failure to account for treatment switching can seriously bias the estimated treatment causal effect. In this paper, we aim to account for the potential impact of treatment switching in a reanalysis evaluating the treatment effect of nucleoside reverse transcriptase inhibitors (NRTIs) on a safety outcome (time to first severe or worse sign or symptom) in participants receiving a new antiretroviral regimen that either included or omitted NRTIs in the optimized treatment that includes or omits NRTIs trial. We propose an estimator of a treatment causal effect for a censored time to event outcome under a structural cumulative survival model that leverages randomization as an instrumental variable to account for selective treatment switching. We establish that the proposed estimator is uniformly consistent and asymptotically Gaussian, with a consistent variance estimator and confidence intervals given, whose finite-sample performance is evaluated via extensive simulations. An R package 'ivsacim' implementing all proposed methods is freely available on R CRAN. Results indicate that adding NRTIs versus omitting NRTIs to a new optimized treatment regime may increase the risk for a safety outcome.

Adjusting for treatment crossover in the MAVORIC trial: survival in advanced mycosis fungoides and Sézary syndrome.

Background: Relative overall survival (OS) estimates reported in the MAVORIC trial are potentially confounded by a high proportion of patients randomized to vorinostat switching to mogamulizumab; furthermore, vorinostat is not used in clinical practice in the UK. Methods: Three methods were considered for crossover adjustment. Survival post-crossover adjustment was compared with data from the Hospital Episode Statistics (HES) to contextualize estimates. Results: Following adjustment, the OS hazard ratio for mogamulizumab versus vorinostat was 0.42 (95% CI: 0.18, 0.98) using the method considered most appropriate based on an assessment of assumptions and comparison with HES. Conclusions: OS of mogamulizumab relative to vorinostat may be underestimated in MAVORIC due to the presence of crossover. The HES database was used to validate this adjustment.

Improved two-stage estimation to adjust for treatment switching in randomised trials: g-estimation to address time-dependent confounding.

In oncology trials, control group patients often switch onto the experimental treatment during follow-up, usually after disease progression. In this case, an intention-to-treat analysis will not address the policy question of interest - that of whether the new treatment represents an effective and cost-effective use of health care resources, compared to the standard treatment. Rank preserving structural failure time models (RPSFTM), inverse probability of censoring weights (IPCW) and two-stage estimation (TSE) have often been used to adjust for switching to inform treatment reimbursement policy decisions. TSE has been applied using a simple approach (TSEsimp), assuming no time-dependent confounding between the time of disease progression and the time of switch. This is problematic if there is a delay between progression and switch. In this paper we introduce TSEgest, which uses structural nested models and g-estimation to account for time-dependent confounding, and compare it to TSEsimp, RPSFTM and IPCW. We simulated scenarios where control group patients could switch onto the experimental treatment with and without time-dependent confounding being present. We varied switching proportions, treatment effects and censoring proportions. We assessed adjustment methods according to their estimation of control group restricted mean survival times that would have been observed in the absence of switching. All methods performed well in scenarios with no time-dependent confounding. TSEgest and RPSFTM continued to perform well in scenarios with time-dependent confounding, but TSEsimp resulted in substantial bias. IPCW also performed well in scenarios with time-dependent confounding, except when inverse probability weights were high in relation to the size of the group being subjected to weighting, which occurred when there was a combination of modest sample size and high switching proportions. TSEgest represents a useful addition to the collection of methods that may be used to adjust for treatment switching in trials in order to address policy-relevant questions.

Two-stage estimation to adjust for treatment switching in randomised trials: a simulation study investigating the use of inverse probability weighting instead of re-censoring.

BACKGROUND: Treatment switching is common in randomised trials of oncology treatments, with control group patients switching onto the experimental treatment during follow-up. This distorts an intention-to-treat comparison of the treatments under investigation. Two-stage estimation (TSE) can be used to estimate counterfactual survival times for patients who switch treatments - that is, survival times that would have been observed in the absence of switching. However, when switchers do not die during the study, counterfactual censoring times are estimated, inducing informative censoring. Re-censoring is usually applied alongside TSE to resolve this problem, but results in lost longer-term information - a major concern if the objective is to estimate long-term treatment effects, as is usually the case in health technology assessment. Inverse probability of censoring weights (IPCW) represents an alternative technique for addressing informative censoring but has not before been combined with TSE. We aim to determine whether combining TSE with IPCW (TSEipcw) represents a valid alternative to re-censoring. METHODS: We conducted a simulation study to compare TSEipcw to TSE with and without re-censoring. We simulated 48 scenarios where control group patients could switch onto the experimental treatment, with switching affected by prognosis. We investigated various switching proportions, treatment effects, survival function shapes, disease severities and switcher prognoses. We assessed the alternative TSE applications according to their estimation of control group restricted mean survival (RMST) that would have been observed in the absence of switching up to the end of trial follow-up. RESULTS: TSEipcw performed well when its weights had a low coefficient of variation, but performed poorly when the coefficient of variation was high. Re-censored analyses usually under-estimated control group RMST, whereas non-re-censored analyses usually produced over-estimates, with bias more serious when the treatment effect was high. In scenarios where TSEipcw performed well, it produced low bias that was often between the two extremes associated with the re-censoring and non-recensoring options. CONCLUSIONS: Treatment switching adjustment analyses using TSE should be conducted with re-censoring, without re-censoring, and with IPCW to explore the sensitivity in results to these application options. This should allow analysts and decision-makers to better interpret the results of adjustment analyses.

Causal inference for long-term survival in randomised trials with treatment switching: Should re-censoring be applied when estimating counterfactual survival times?

Treatment switching often has a crucial impact on estimates of effectiveness and cost-effectiveness of new oncology treatments. Rank preserving structural failure time models (RPSFTM) and two-stage estimation (TSE) methods estimate 'counterfactual' (i.e. had there been no switching) survival times and incorporate re-censoring to guard against informative censoring in the counterfactual dataset. However, re-censoring causes a loss of longer term survival information which is problematic when estimates of long-term survival effects are required, as is often the case for health technology assessment decision making. We present a simulation study designed to investigate applications of the RPSFTM and TSE with and without re-censoring, to determine whether re-censoring should always be recommended within adjustment analyses. We investigate a context where switching is from the control group onto the experimental treatment in scenarios with varying switch proportions, treatment effect sizes, treatment effect changes over time, survival function shapes, disease severity and switcher prognosis. Methods were assessed according to their estimation of control group restricted mean survival that would be observed in the absence of switching, up to the end of trial follow-up. We found that analyses which re-censored usually produced negative bias (i.e. underestimating control group restricted mean survival and overestimating the treatment effect), whereas analyses that did not re-censor consistently produced positive bias which was often smaller in magnitude than the bias associated with re-censored analyses, particularly when the treatment effect was high and the switching proportion was low. The RPSFTM with re-censoring generally resulted in increased bias compared to the other methods. We believe that analyses should be conducted with and without re-censoring, as this may provide decision-makers with useful information on where the true treatment effect is likely to lie. Incorporating re-censoring should not always represent the default approach when the objective is to estimate long-term survival times and treatment effects.

Design and monitoring of survival trials in complex scenarios.

This paper proposes an approach to design and monitor survival trials accounting for complex scenarios such as delayed treatment effect, treatment dilution, and treatment crossover. These scenarios often lead to non-proportional hazards, making study design and monitoring more difficult. We demonstrate that, with event times following piecewise exponential distributions, the log-rank statistic as well as its variance-covariance structure can be easily computed, which greatly simplifies study design and monitoring. As the number of pieces in the exponential distributions can be arbitrary, this approach can handle a wide range of scenarios. Three hypothetical examples are used to demonstrate its potential use.

Assessing methods for dealing with treatment switching in clinical trials: A follow-up simulation study.

When patients randomised to the control group of a randomised controlled trial are allowed to switch onto the experimental treatment, intention-to-treat analyses of the treatment effect are confounded because the separation of randomised groups is lost. Previous research has investigated statistical methods that aim to estimate the treatment effect that would have been observed had this treatment switching not occurred and has demonstrated their performance in a limited set of scenarios. Here, we investigate these methods in a new range of realistic scenarios, allowing conclusions to be made based upon a broader evidence base. We simulated randomised controlled trials incorporating prognosis-related treatment switching and investigated the impact of sample size, reduced switching proportions, disease severity, and alternative data-generating models on the performance of adjustment methods, assessed through a comparison of bias, mean squared error, and coverage, related to the estimation of true restricted mean survival in the absence of switching in the control group. Rank preserving structural failure time models, inverse probability of censoring weights, and two-stage methods consistently produced less bias than the intention-to-treat analysis. The switching proportion was confirmed to be a key determinant of bias: sample size and censoring proportion were relatively less important. It is critical to determine the size of the treatment effect in terms of an acceleration factor (rather than a hazard ratio) to provide information on the likely bias associated with rank-preserving structural failure time model adjustments. In general, inverse probability of censoring weight methods are more volatile than other adjustment methods.

Adjusting for treatment switching in randomised controlled trials - A simulation study and a simplified two-stage method.

Estimates of the overall survival benefit of new cancer treatments are often confounded by treatment switching in randomised controlled trials (RCTs) - whereby patients randomised to the control group are permitted to switch onto the experimental treatment upon disease progression. In health technology assessment, estimates of the unconfounded overall survival benefit associated with the new treatment are needed. Several switching adjustment methods have been advocated in the literature, some of which have been used in health technology assessment. However, it is unclear which methods are likely to produce least bias in realistic RCT-based scenarios. We simulated RCTs in which switching, associated with patient prognosis, was permitted. Treatment effect size and time dependency, switching proportions and disease severity were varied across scenarios. We assessed the performance of alternative adjustment methods based upon bias, coverage and mean squared error, related to the estimation of true restricted mean survival in the absence of switching in the control group. We found that when the treatment effect was not time-dependent, rank preserving structural failure time models (RPSFTM) and iterative parameter estimation methods produced low levels of bias. However, in the presence of a time-dependent treatment effect, these methods produced higher levels of bias, similar to those produced by an inverse probability of censoring weights method. The inverse probability of censoring weights and structural nested models produced high levels of bias when switching proportions exceeded 85%. A simplified two-stage Weibull method produced low bias across all scenarios and provided the treatment switching mechanism is suitable, represents an appropriate adjustment method.

TREATMENT SWITCHING: STATISTICAL AND DECISION-MAKING CHALLENGES AND APPROACHES.

OBJECTIVES: Treatment switching refers to the situation in a randomized controlled trial where patients switch from their randomly assigned treatment onto an alternative. Often, switching is from the control group onto the experimental treatment. In this instance, a standard intention-to-treat analysis does not identify the true comparative effectiveness of the treatments under investigation. We aim to describe statistical methods for adjusting for treatment switching in a comprehensible way for nonstatisticians, and to summarize views on these methods expressed by stakeholders at the 2014 Adelaide International Workshop on Treatment Switching in Clinical Trials. METHODS: We describe three statistical methods used to adjust for treatment switching: marginal structural models, two-stage adjustment, and rank preserving structural failure time models. We draw upon discussion heard at the Adelaide International Workshop to explore the views of stakeholders on the acceptability of these methods. RESULTS: Stakeholders noted that adjustment methods are based on assumptions, the validity of which may often be questionable. There was disagreement on the acceptability of adjustment methods, but consensus that when these are used, they should be justified rigorously. The utility of adjustment methods depends upon the decision being made and the processes used by the decision-maker. CONCLUSIONS: Treatment switching makes estimating the true comparative effect of a new treatment challenging. However, many decision-makers have reservations with adjustment methods. These, and how they affect the utility of adjustment methods, require further exploration. Further technical work is required to develop adjustment methods to meet real world needs, to enhance their acceptability to decision-makers.

Treatment switching in oncology trials and the acceptability of adjustment methods.

Treatment switching has become an important issue in the development and approval of new drugs, particularly in oncology. Randomized controlled trials (RCTs) represent the gold standard for evaluating the effectiveness of interventions, but often patients randomized to the control group are permitted to switch onto the experimental treatment at some point during the trial. This is important, because standard statistical approaches used to analyze RCTs compare groups as randomized, based upon an intention-to-treat principle. When patients in both groups receive the new drug, such analyses do not provide an accurate estimate of the comparative effectiveness of the two treatments. This may lead to inappropriate decision-making - cost-effective drugs may not be approved. Limited healthcare finances may be used inefficiently. Health-related quality-of-life and lives may be lost.

Adjusting survival time estimates to account for treatment switching in randomized controlled trials--an economic evaluation context: methods, limitations, and recommendations.

BACKGROUND: Treatment switching commonly occurs in clinical trials of novel interventions in the advanced or metastatic cancer setting. However, methods to adjust for switching have been used inconsistently and potentially inappropriately in health technology assessments (HTAs). OBJECTIVE: We present recommendations on the use of methods to adjust survival estimates in the presence of treatment switching in the context of economic evaluations. METHODS: We provide background on the treatment switching issue and summarize methods used to adjust for it in HTAs. We discuss the assumptions and limitations associated with adjustment methods and draw on results of a simulation study to make recommendations on their use. RESULTS: We demonstrate that methods used to adjust for treatment switching have important limitations and often produce bias in realistic scenarios. We present an analysis framework that aims to increase the probability that suitable adjustment methods can be identified on a case-by-case basis. We recommend that the characteristics of clinical trials, and the treatment switching mechanism observed within them, should be considered alongside the key assumptions of the adjustment methods. Key assumptions include the "no unmeasured confounders" assumption associated with the inverse probability of censoring weights (IPCW) method and the "common treatment effect" assumption associated with the rank preserving structural failure time model (RPSFTM). CONCLUSIONS: The limitations associated with switching adjustment methods such as the RPSFTM and IPCW mean that they are appropriate in different scenarios. In some scenarios, both methods may be prone to bias; "2-stage" methods should be considered, and intention-to-treat analyses may sometimes produce the least bias. The data requirements of adjustment methods also have important implications for clinical trialists.