A cautionary tale: an evaluation of the performance of treatment switching adjustment methods in a real world case study.

BACKGROUND: Treatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82-1.22) for all patients, and 0.99 (95% CI 0.75-1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus). METHODS: We tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76-0.77 (95% CI 0.60-0.98) for all patients, and 0.55-0.73 (95% CI 0.41-0.93) for patients with wild-type KRAS. RESULTS: Some applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals. CONCLUSIONS: Adjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability.

Adjusting Overall Survival Estimates of Macitentan in Pulmonary Arterial Hypertension After Treatment Switching: Results from the SERAPHIN Study.

INTRODUCTION: Evaluating overall survival in randomized controlled trials (RCTs) can often be confounded by bias introduced by treatment switching. SERAPHIN was a large RCT that evaluated the effects of long-term treatment with the endothelin receptor antagonist macitentan in patients with pulmonary arterial hypertension. In an intent-to-treat (ITT) analysis, a non-significant decrease in the risk of all-cause mortality up to study closure was reported with macitentan 10 mg versus placebo. As patients could switch treatment when experiencing symptoms of disease progression, this analysis attempts to adjust for the confounding effects on overall survival. METHODS: The inverse probability of censoring weighted (IPCW) and rank-preserving structural failure time (RPSFT) models were used to estimate the treatment effect on overall mortality had there been no treatment switching in SERAPHIN. Time to all-cause death was evaluated up to study closure. Treatment switching was defined as patients in the placebo group switching to open-label macitentan 10 mg, and patients in the macitentan 10 mg group prematurely discontinuing macitentan. RESULTS: By study closure, 73.2% (183/250) of patients in the placebo group had switched to macitentan 10 mg. Among these patients, exposure time to macitentan 10 mg represented 28.2% of total study treatment exposure (cumulative exposure 134.6 patient-years). At study closure, 24.8% (60/242) of patients in the macitentan 10 mg group were not receiving open-label macitentan; mean time not receiving macitentan was 44.3 weeks. The adjusted hazard ratios (HR) for overall survival using the IPCW and RPSFT methods were lower (HR 0.42, 95% confidence interval [CI] 0.22, 0.81; p = 0.009, and HR 0.33, 95% CI 0.04, 2.83, respectively) than the ITT unadjusted HR (0.80, 95% CI 0.51, 1.24). CONCLUSION: These results from the current analyses indicate that in SERAPHIN, the standard ITT analysis was confounded by treatment switching resulting in an underestimation of the benefit of macitentan 10 mg on overall survival. By adjusting for switching, the IPCW and RPSFT models estimated a 58% and 67% reduction in risk of mortality, respectively, with macitentan 10 mg versus placebo. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT00660179.

Assessing the Influence of Subsequent Immunotherapy on Overall Survival in Patients with Unresectable Stage III Non-Small Cell Lung Cancer from the PACIFIC Study.

BACKGROUND: Historically, the standard of care for patients with unresectable, Stage III non-small cell lung cancer had been concurrent chemoradiotherapy. However, outcomes had been poor, with approximately 15% to 32% of patients alive at 5 years. In the placebo-controlled Phase III A PACIFIC trial, consolidation treatment with durvalumab after concurrent chemoradiotherapy significantly improved overall survival (OS) and progression-free survival in patients with unresectable, Stage III non-small cell lung cancer, establishing this regimen as a new standard of care in this setting. In the PACIFIC trial, crossover between treatment arms (durvalumab or placebo) was not permitted. However, after discontinuation from study treatment, patients from both arms of PACIFIC could switch to subsequent anticancer therapy, including durvalumab and other immunotherapies, which is known to influence standard intention-to-treat analysis of OS, potentially underestimating the effect of an experimental drug. Moreover, the introduction of immunotherapies has demonstrated marked improvements in the postprogression, metastatic non-small cell lung cancer setting. OBJECTIVE: To examine the influence of subsequent immunotherapy on OS in the PACIFIC trial. METHODS: Both a Rank Preserving Structural Failure Time Model (RPSFTM) and modified 2-stage method were used. RPSFTM assumes that a patient's survival time with no immunotherapy (counterfactual survival time) is equal to the observed time influenced by immunotherapy, multiplied by an acceleration factor, plus the time not influenced. The modified 2-stage method estimates the effect of immunotherapy by comparing postsubsequent-treatment-initiation survival times between patients with and without subsequent immunotherapy. In both models, OS was adjusted to reflect a hypothetical scenario in which no patients received subsequent immunotherapy. RPSFTM was also used for scenarios in which subsequent immunotherapy was received by increasing proportions of placebo patients but none of the durvalumab patients. RESULTS: In the intention-to-treat analysis (3-year follow-up), durvalumab improved OS versus placebo (stratified hazard ratio = 0.69; 95% CI, 0.55-0.86). Overall, 10% and 27% of durvalumab and placebo patients, respectively, received subsequent immunotherapy. With subsequent immunotherapy removed from both arms, estimated hazard ratio was 0.66 (95% CI, 0.53-0.84) with RPSFTM and 0.68 (95% CI, 0.54-0.85) with the modified 2-stage method. With subsequent immunotherapy removed from the durvalumab arm only (RPSFTM), estimated hazard ratio increased as the proportion of placebo patients receiving subsequent immunotherapy increased, up to 0.75 (95% CI, 0.60-0.94) maximum (assuming all placebo patients with subsequent treatment received immunotherapy). CONCLUSIONS: Results were consistent with the intention-to-treat analysis, supporting the conclusion that durvalumab after chemoradiotherapy provides substantial OS benefit in patients with Stage III, unresectable non-small cell lung cancer. ClinicalTrials.gov identifier: NCT02125461 (Curr Ther Res Clin Exp. 2021; 82:XXX-XXX).

Assessing the Long-Term Effectiveness of Cladribine vs. Placebo in the Relapsing-Remitting Multiple Sclerosis CLARITY Randomized Controlled Trial and CLARITY Extension Using Treatment Switching Adjustment Methods.

OBJECTIVES: Treatment switching adjustment methods are often used to adjust for switching in oncology randomized controlled trials (RCTs). In this exploratory analysis, we apply these methods to adjust for treatment changes in the setting of an RCT followed by an extension study in relapsing-remitting multiple sclerosis. METHODS: The CLARITY trial evaluated cladribine tablets versus placebo over 96 weeks. In the 96-week CLARITY Extension, patients who received placebo in CLARITY received cladribine tablets; patients who received cladribine tablets in CLARITY were re-randomized to placebo or cladribine tablets. End points were time to first qualifying relapse (FQR) and time to 3- and 6-month confirmed disability progression (3mCDP, 6mCDP). We aimed to compare the effectiveness of cladribine tablets with placebo over CLARITY and the extension. The rank-preserving structural failure time model (RPSFTM) and iterative parameter estimation (IPE) were used to estimate what would have happened if patients had received placebo in CLARITY and the extension versus patients that received cladribine tablets and switched to placebo. To gauge whether treatment effect waned after the 96 weeks of CLARITY, we compared hazard ratios (HRs) from the adjustment analysis with HRs from CLARITY. RESULTS: The RPSFTM resulted in an HR of 0.48 [95% confidence interval (CI) 0.36-0.62] for FQR, 0.62 (95% CI 0.46-0.84) for 3mCDP and 0.62 (95% CI 0.44-0.88) for 6mCDP. IPE algorithm results were similar. CLARITY HRs were 0.44 (95% CI 0.34-0.58), 0.60 (95% CI 0.41-0.87) and 0.58 (95% CI 0.40-0.83) for FQR, 3mCDP and 6mCDP, respectively. CONCLUSIONS: Treatment switching adjustment methods are applicable in non-oncology settings. Adjusted CLARITY plus CLARITY Extension HRs were similar to the CLARITY HRs, demonstrating significant treatment benefits associated with cladribine tablets versus placebo. FUNDING: EMD Serono, Inc. (a business of Merck KGaA, Darmstadt, Germany).

Impact of Nonrandomized Dropout on Treatment Switching Adjustment in the Relapsing-Remitting Multiple Sclerosis CLARITY Trial and the CLARITY Extension Study.

OBJECTIVES: Statistical methods to adjust for treatment switching are commonly applied to randomized controlled trials (RCTs) in oncology. Nevertheless, RCTs with extension studies incorporating nonrandomized dropout require consideration of alternative adjustment methods. The current study used a recognized method and a novel method to adjust for treatment switching in relapsing-remitting multiple sclerosis (MS). METHODS: The Cladribine Tablets Treating Multiple Sclerosis Orally (CLARITY) RCT evaluated the efficacy of cladribine versus placebo over 96 weeks. Many (but not all) CLARITY participants enrolled in the 96-week CLARITY extension study; placebo-treated patients from CLARITY received cladribine (PP→LL), and cladribine-treated patients were re-randomized to placebo (LL→PP) or continued cladribine (LL→LL). End points were time to first qualifying relapse (FQR) and time to 3-month and 6-month confirmed disability progression (3mCDP, 6mCDP). We aimed to estimate the effectiveness of the LL→PP treatment strategy compared with a counterfactual (unobserved) PP→PP strategy. We applied the commonly used rank-preserving structural failure time model (RPSFTM) and a novel approach that combined propensity score matching (PSM) with inverse probability of censoring weights (IPCW). RESULTS: The RPSFTM resulted in LL→PP versus PP→PP hazard ratios (HRs) of 0.48 (95% confidence interval [CI] 0.36-0.62) for FQR, 0.62 (95% CI 0.46-0.84) for 3mCDP, and 0.62 (95% CI 0.44-0.88) for 6mCDP. The PSM+IPCW resulted in HRs of 0.47 (95% CI 0.38-0.63) for FQR, 0.61 (95% CI 0.43-0.86) for 3mCDP, and 0.63 (95% CI 0.40-0.87) for 6mCDP. CONCLUSIONS: The PSM+IPCW HRs were consistent with those from the RPSFTM, suggesting that the results were not substantially biased by informative dropout, assuming that all relevant confounders were controlled for. There was no statistical evidence of a reduction in the cladribine treatment effect during the extension period.

A Validation Study of the Rank-Preserving Structural Failure Time Model: Confidence Intervals and Unique, Multiple, and Erroneous Solutions.

BACKGROUND: The rank-preserving structural failure time model (RPSFTM) is used for health technology assessment submissions to adjust for switching patients from reference to investigational treatment in cancer trials. It uses counterfactual survival (survival when only reference treatment would have been used) and assumes that, at randomization, the counterfactual survival distribution for the investigational and reference arms is identical. Previous validation reports have assumed that patients in the investigational treatment arm stay on therapy throughout the study period. OBJECTIVES: To evaluate the validity of the RPSFTM at various levels of crossover in situations in which patients are taken off the investigational drug in the investigational arm. METHODS: The RPSFTM was applied to simulated datasets differing in percentage of patients switching, time of switching, underlying acceleration factor, and number of patients, using exponential distributions for the time on investigational and reference treatment. RESULTS: There were multiple scenarios in which two solutions were found: one corresponding to identical counterfactual distributions, and the other to two different crossing counterfactual distributions. The same was found for the hazard ratio (HR). Unique solutions were observed only when switching patients were on investigational treatment for <40% of the time that patients in the investigational arm were on treatment. LIMITATIONS: Distributions other than exponential could have been used for time on treatment. CONCLUSIONS: An HR equal to 1 is a necessary but not always sufficient condition to indicate acceleration factors associated with equal counterfactual survival. Further assessment to distinguish crossing counterfactual curves from equal counterfactual curves is especially needed when the time that switchers stay on investigational treatment is relatively long compared to the time direct starters stay on investigational treatment.

On an enhanced rank-preserving structural failure time model to handle treatment switch, crossover, and dropout.

It is very challenging to estimate the comparative treatment effect between a treatment therapy and a control therapy on overall survival in the presence of treatment crossover, switch to an alternative non-study therapy, and non-random patient dropout. Existing methods (e.g., intent-to-treat and per-protocol) are known to be biased. We proposed two new estimators to address these analytical challenges and evaluated their performance via a comprehensive simulation study. The new estimators were constructed by combining an enhanced rank-preserving structural failure time model and the inverse probability censoring weighting approach. In the simulation study, we assessed and compared the performance of the two new estimators with four estimators from existing methods. The simulation results show that the new estimators have much better performance in almost all considered settings compared with the existing estimators. Copyright © 2017 John Wiley & Sons, Ltd.

Sunitinib in pancreatic neuroendocrine tumors: updated progression-free survival and final overall survival from a phase III randomized study.

BACKGROUND: In a phase III trial in patients with advanced, well-differentiated, progressive pancreatic neuroendocrine tumors, sunitinib 37.5 mg/day improved investigator-assessed progression-free survival (PFS) versus placebo (11.4 versus 5.5 months; HR, 0.42; P < 0.001). Here, we present PFS using retrospective blinded independent central review (BICR) and final median overall survival (OS), including an assessment highlighting the impact of patient crossover from placebo to sunitinib. PATIENTS AND METHODS: In this randomized, double-blind, placebo-controlled study, cross-sectional imaging from patients was evaluated retrospectively by blinded third-party radiologists using a two-reader, two-time-point lock, followed by a sequential locked-read, batch-mode paradigm. OS was summarized using the Kaplan-Meier method and Cox proportional hazards model. Crossover-adjusted OS effect was derived using rank-preserving structural failure time (RPSFT) analyses. RESULTS: Of 171 randomized patients (sunitinib, n = 86; placebo, n = 85), 160 (94%) had complete scan sets/time points. By BICR, median (95% confidence interval [CI]) PFS was 12.6 (11.1-20.6) months for sunitinib and 5.8 (3.8-7.2) months for placebo (HR, 0.32; 95% CI 0.18-0.55; P = 0.000015). Five years after study closure, median (95% CI) OS was 38.6 (25.6-56.4) months for sunitinib and 29.1 (16.4-36.8) months for placebo (HR, 0.73; 95% CI 0.50-1.06; P = 0.094), with 69% of placebo patients having crossed over to sunitinib. RPSFT analysis confirmed an OS benefit for sunitinib. CONCLUSIONS: BICR confirmed the doubling of PFS with sunitinib compared with placebo. Although the observed median OS improved by nearly 10 months, the effect estimate did not reach statistical significance, potentially due to crossover from placebo to sunitinib. TRIAL REGISTRATION NUMBER: NCT00428597.

Long-Term Extrapolation of Clinical Benefits Among Patients With Mild Heart Failure Receiving Cardiac Resynchronization Therapy: Analysis of the 5-Year Follow-Up From the REVERSE Study.

OBJECTIVES: This study sought to assess the lifelong extrapolated patient outcomes with cardiac resynchronization therapy (CRT) in mild heart failure (HF), beyond the follow-up of randomized clinical trials (RCTs). BACKGROUND: RCTs have demonstrated short-term survival and HF hospitalization benefits of CRT in mild HF. We used data from the 5-year follow-up of the REVERSE (REsynchronization reVErses Remodeling in Systolic left vEntricular dysfunction) study to extrapolate survival and HF hospitalizations. We compared CRT-ON versus CRT-OFF and CRT defibrillators (CRT-D) versus CRT pacemakers (CRT-P). METHODS: Multivariate regression models were used to estimate treatment-specific all-cause mortality, disease progression, and HF-related hospitalization rates. Rank-preserving structural failure time (RPSFT) models were used to adjust for protocol-mandated crossover in the survival analysis. RESULTS: CRT-ON was predicted to increase survival by 22.8% (CRT-ON 52.5% vs. CRT-OFF 29.7%; hazard ratio [HR]: 0.45; p = 0.21), leading to an expected survival of 9.76 years (CRT-ON) versus 7.5 years (CRT-OFF). CRT-D showed a significant improvement in survival compared with CRT-P (HR: 0.47; 95% confidence interval [CI]: 0.25 to 0.88; p = 0.02) and were predicted to offer 2.77 additional life-years. New York Heart Association (NYHA) functional class II patients had a 30.6% higher HF hospitalization risk than class I (I vs. II incident rate ratio [IRR]: 0.69; 95% CI: 0.57 to 0.85; p < 0.001) and 3 times lower rate compared with class III (III vs. II IRR: 2.98; 95% CI: 2.29 to 3.87; p < 0.001). CONCLUSIONS: RPSFT estimates yielded results demonstrating clinically important long-term benefit of CRT in mild HF. CRT was predicted to reduce mortality, with CRT-D prolonging life more than CRT-P. NYHA functional class I/II patients were shown to have a significantly reduced risk of HF hospitalization compared with class III, leading to CRT reducing HF hospitalization rates.

Assessing the treatment effect in a randomized controlled trial with extensive non-adherence: the EVOLVE trial.

Intention-to-treat (ITT) analysis is widely used to establish efficacy in randomized clinical trials. However, in a long-term outcomes study where non-adherence to study drug is substantial, the on-treatment effect of the study drug may be underestimated using the ITT analysis. The analyses presented herein are from the EVOLVE trial, a double-blind, placebo-controlled, event-driven cardiovascular outcomes study conducted to assess whether a treatment regimen including cinacalcet compared with placebo in addition to other conventional therapies reduces the risk of mortality and major cardiovascular events in patients receiving hemodialysis with secondary hyperparathyroidism. Pre-specified sensitivity analyses were performed to assess the impact of non-adherence on the estimated effect of cinacalcet. These analyses included lag-censoring, inverse probability of censoring weights (IPCW), rank preserving structural failure time model (RPSFTM) and iterative parameter estimation (IPE). The relative hazard (cinacalcet versus placebo) of mortality and major cardiovascular events was 0.93 (95% confidence interval 0.85, 1.02) using the ITT analysis; 0.85 (0.76, 0.95) using lag-censoring analysis; 0.81 (0.70, 0.92) using IPCW; 0.85 (0.66, 1.04) using RPSFTM and 0.85 (0.75, 0.96) using IPE. These analyses, while not providing definitive evidence, suggest that the intervention may have an effect while subjects are receiving treatment. The ITT method remains the established method to evaluate efficacy of a new treatment; however, additional analyses should be considered to assess the on-treatment effect when substantial non-adherence to study drug is expected or observed.

Correcting for non-compliance in randomized non-inferiority trials with active and placebo control using structural models.

The three-arm clinical trial design, which includes a test treatment, an active reference, and placebo control, is the gold standard for the assessment of non-inferiority. In the presence of non-compliance, one common concern is that an intent-to-treat (ITT) analysis (which is the standard approach to non-inferiority trials), tends to increase the chances of erroneously concluding non-inferiority, suggesting that the per-protocol (PP) analysis may be preferable for non-inferiority trials despite its inherent bias. The objective of this paper was to develop statistical methodology for dealing with non-compliance in three-arm non-inferiority trials for censored, time-to-event data. Changes in treatment were here considered the only form of non-compliance. An approach using a three-arm rank preserving structural failure time model and G-estimation analysis is here presented. Using simulations, the impact of non-compliance on non-inferiority trials was investigated in detail using ITT, PP analyses, and the present proposed method. Results indicate that the proposed method shows good characteristics, and that neither ITT nor PP analyses can always guarantee the validity of the non-inferiority conclusion. A Statistical Analysis System program for the implementation of the proposed test procedure is available from the authors upon request.

Adjusting overall survival for treatment switches: commonly used methods and practical application.

In parallel group trials, long-term efficacy endpoints may be affected if some patients switch or cross over to the alternative treatment arm prior to the event. In oncology trials, switch to the experimental treatment can occur in the control arm following disease progression and potentially impact overall survival. It may be a clinically relevant question to estimate the efficacy that would have been observed if no patients had switched, for example, to estimate 'real-life' clinical effectiveness for a health technology assessment. Several commonly used statistical methods are available that try to adjust time-to-event data to account for treatment switching, ranging from naive exclusion and censoring approaches to more complex inverse probability of censoring weighting and rank-preserving structural failure time models. These are described, along with their key assumptions, strengths, and limitations. Best practice guidance is provided for both trial design and analysis when switching is anticipated. Available statistical software is summarized, and examples are provided of the application of these methods in health technology assessments of oncology trials. Key considerations include having a clearly articulated rationale and research question and a well-designed trial with sufficient good quality data collection to enable robust statistical analysis. No analysis method is universally suitable in all situations, and each makes strong untestable assumptions. There is a need for further research into new or improved techniques. This information should aid statisticians and their colleagues to improve the design and analysis of clinical trials where treatment switch is anticipated.