Examining the Effect of Missing Data and Unmeasured Confounding on External Comparator Studies: Case Studies and Simulations.

BACKGROUND AND OBJECTIVE: Missing data and unmeasured confounding are key challenges for external comparator studies. This work evaluates bias and other performance characteristics depending on missingness and unmeasured confounding by means of two case studies and simulations. METHODS: Two case studies were constructed by taking the treatment arms from two randomised controlled trials and an external real-world data source that exhibited substantial missingness. The indications of the randomised controlled trials were multiple myeloma and metastatic hormone-sensitive prostate cancer. Overall survival was taken as the main endpoint. The effects of missing data and unmeasured confounding were assessed for the case studies by reporting estimated external comparator versus randomised controlled trial treatment effects. Based on the two case studies, simulations were performed broadening the settings by varying the underlying hazard ratio, the sample size, the sample size ratio between the experimental arm and the external comparator, the number of missing covariates and the percentage of missingness. Thereby, bias and other performance metrics could be quantified dependent on these factors. RESULTS: For the multiple myeloma external comparator study, results were in line with the randomised controlled trial, despite missingness and potential unmeasured confounding, while for the metastatic hormone-sensitive prostate cancer case study missing data led to a low sample size, leading overall to inconclusive results. Furthermore, for the metastatic hormone-sensitive prostate cancer study, missing data in important eligibility criteria led to further limitations. Simulations were successfully applied to gain a quantitative understanding of the effects of missing data and unmeasured confounding. CONCLUSIONS: This exploratory study confirmed external comparator strengths and limitations by quantifying the impact of missing data and unmeasured confounding using case studies and simulations. In particular, missing data in key eligibility criteria were seen to limit the ability to derive the external comparator target analysis population accurately, while simulations demonstrated the magnitude of bias to expect for various settings.

Benefits and risks of clofarabine in adult acute lymphoblastic leukemia investigated in depth by multi-state modeling.

BACKGROUND: We recently reported results of the prospective, open-label HOVON-100 trial in 334 adult patients with acute lymphoblastic leukemia (ALL) randomized to first-line treatment with or without clofarabine (CLO). No improvement of event-free survival (EFS) was observed, while a higher proportion of patients receiving CLO obtained minimal residual disease (MRD) negativity. AIM: In order to investigate the effects of CLO in more depth, two multi-state models were developed to identify why CLO did not show a long-term survival benefit despite more MRD-negativity. METHODS: The first model evaluated the effect of CLO on going off-protocol (not due to refractory disease/relapse, completion or death) as a proxy of severe treatment-related toxicity, while the second model evaluated the effect of CLO on obtaining MRD negativity. The subsequent impact of these intermediate events on death or relapsed/refractory disease was assessed in both models. RESULTS: Overall, patients receiving CLO went off-protocol more frequently than control patients (35/168 [21%] vs. 18/166 [11%], p = 0.019; HR 2.00 [1.13-3.52], p = 0.02), especially during maintenance (13/44 [30%] vs. 6/56 [11%]; HR 2.85 [95%CI 1.08-7.50], p = 0.035). Going off-protocol was, however, not associated with more relapse or death. Patients in the CLO arm showed a trend towards an increased rate of MRD-negativity compared with control patients (HR MRD-negativity: 1.35 [0.95-1.91], p = 0.10), which did not translate into a significant survival benefit. CONCLUSION: We conclude that the intermediate states, i.e., going off-protocol and MRD-negativity, were affected by adding CLO, but these transitions were not associated with subsequent survival estimates, suggesting relatively modest antileukemic activity in ALL.

Bayesian interim analysis for prospective randomized studies: reanalysis of the acute myeloid leukemia HOVON 132 clinical trial.

Randomized controlled trials (RCTs) are the gold standard to establish the benefit-risk ratio of novel drugs. However, the evaluation of mature results often takes many years. We hypothesized that the addition of Bayesian inference methods at interim analysis time points might accelerate and enforce the knowledge that such trials may generate. In order to test that hypothesis, we retrospectively applied a Bayesian approach to the HOVON 132 trial, in which 800 newly diagnosed AML patients aged 18 to 65 years were randomly assigned to a "7 + 3" induction with or without lenalidomide. Five years after the first patient was recruited, the trial was negative for its primary endpoint with no difference in event-free survival (EFS) between experimental and control groups (hazard ratio [HR] 0.99, p = 0.96) in the final conventional analysis. We retrospectively simulated interim analyses after the inclusion of 150, 300, 450, and 600 patients using a Bayesian methodology to detect early lack of efficacy signals. The HR for EFS comparing the lenalidomide arm with the control treatment arm was 1.21 (95% CI 0.81-1.69), 1.05 (95% CI 0.86-1.30), 1.00 (95% CI 0.84-1.19), and 1.02 (95% CI 0.87-1.19) at interim analysis 1, 2, 3 and 4, respectively. Complete remission rates were lower in the lenalidomide arm, and early deaths more frequent. A Bayesian approach identified that the probability of a clinically relevant benefit for EFS (HR < 0.76, as assumed in the statistical analysis plan) was very low at the first interim analysis (1.2%, 0.6%, 0.4%, and 0.1%, respectively). Similar observations were made for low probabilities of any benefit regarding CR. Therefore, Bayesian analysis significantly adds to conventional methods applied for interim analysis and may thereby accelerate the performance and completion of phase III trials.

How to develop new systemic treatments in ultra-rare cancers with high unmet needs? The case of alveolar soft-part sarcoma.

Developing new drugs or generating evidence for existing drugs in new indications for ultra-rare cancers is complex and carries a high-risk of failure. This gets even harder in ultra-rare tumours, which have an annual incidence of 1 per 1,000,000 population or less. Here, we illustrate the problem of adequate evidence generation in ultra-rare tumours, using Alveolar Soft-Part Sarcomas (ASPS) - an ultra-rare sarcoma newly diagnosed in approximately 60 persons a year in the European Union - as an exemplar case showing challenges in development despite being potentially relevant for classes of agents. We discuss some possible approaches for addressing such challenges, especially focussing on constructive collaboration between academic groups, patients and advocates, drug manufacturers, and regulators to optimise drug development in ultra-rare cancers. This article, written by various European stakeholders, proposes a way forward to ultimately get better options for patients with ultra-rare cancers.

Attitudes of healthcare professionals and drug regulators about progression-free survival as endpoint in the advanced cancer setting.

PURPOSE: To describe the attitudes of healthcare professionals and drug regulators about progression-free survival (PFS) as efficacy endpoint in clinical trials with patients with advanced cancer and to explore to what extent these attitudes influence the willingness to trade between PFS and toxicity. METHODS: Cross-sectional survey with regulators from the European Medicines Agency (EMA), and healthcare professionals (HCP) from the "Stichting Hemato-Oncologie voor Volwassenen Nederland" (HOVON) collaborative group and the European Organisation for Research and Treatment of Cancer (EORTC). Attitudes towards PFS were elicited using 5-point Likert items. The respondents' willingness to trade between PFS and grade 3 or 4 (G34) toxicity was assessed using the threshold technique and quantified in terms of their maximum acceptable risk (MAR). RESULTS: Responses were collected from 287 HCPs and 64 regulators with mainly clinical expertise. Attitudes towards PFS were often spread out in both groups and related to beliefs about PFS being a likely surrogate for clinical benefit, being an intrinsic benefit to be distinguished from OS, or on the importance given to OS. Being a regulator or holding stronger beliefs about PFS being a likely surrogate or an intrinsic benefit were associated with a higher MAR. Presence of a supportive trend in OS was stated as important but was not associated with MAR. There was agreement on the need to address bias in the adjudication of PFS and the need for improving communication to patients about meaning, strengths, and limitations of improvements in PFS. CONCLUSION: Attitudes towards PFS were spread out and were associated with individual differences in the willingness to trade between toxicity and PFS. There was agreement on the need to address bias in the adjudication of PFS and improving communication to patients.