A Comparison of Seven Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies.

OBJECTIVES: The development of accelerated approval programs for high morbidity and unmet need conditions has driven the use of single-arm studies in drug development. Regulatory and health technology assessment (HTA) agencies are recognizing that high-quality external control arms (ECAs), built using real-world data, can reduce uncertainties arising from single-arm studies. This review compared 7 case studies of regulatory and HTA agencies' evaluations of oncology ECAs. METHODS: Food and Drug Administration multidisciplinary reviews for oncology submissions from 2014 to 2021 were screened to identify 7 cases (2 blinatumomab indications, avelumab, and erdafitinib, entrectinib, trastuzumab deruxtecan, and idecabtagene vicleucel) with ECAs to support efficacy claims. Regulatory (Food and Drug Administration, European Medicines Agency, Health Canada) and HTA (pan-Canadian Oncology Drug Review, National Institute for Health and Care Excellence, Federal Joint Committee, Haute Autorité de Santé, and Pharmaceutical Benefits Advisory Committee) submissions for these cases were reviewed. The decision makers' ECA critiques and the level of influence on the decision were analyzed and categorized. RESULTS: Across case studies, selection bias and confounding were the most common ECA critiques. Nevertheless, agreement in critiques between and among regulators and HTA bodies was low. ECA influence on agencies' decisions also varied. CONCLUSIONS: Evaluating the same ECA evidence, agencies focused on methodologic issues (ie, selection bias and confounding), but were often not aligned on their critiques. Further research is needed to fully characterize how agencies evaluate ECAs. This study is a first step in critically evaluating agencies' critiques of ECAs and highlights the need for future guidance development around ECA design and generation.

Real-world evidence to support regulatory decision making: New or expanded medical product indications.

There is increasing interest in utilizing real-world data (RWD) to produce real-world evidence (RWE) on the benefits and risks of medical products that could support regulatory approval decisions. The field of pharmacoepidemiology has a long history of focusing on data and evidence that would now be termed "real-world," including evidence from healthcare claims, registries, and electronic health records. However, several emerging trends over the past decade are converging to support the use of these and other RWD sources for approval decisions, and there are several recent examples and ongoing research that demonstrate how RWE may be used to support regulatory approval of new or expanded indications. The goal of this article is to review the current landscape and future directions of the use of RWE in this context. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE).

How to choose a time zero for patients in external control arms.

When a sponsor carries out a single-arm trial of a novel oncology compound, it may wish to assess the efficacy of the compound via comparison of overall survival to an external control arm, constructed using patients included in some retrospective registry. If efficacy of the novel compound is compared to efficacy of physician's choice of chemotherapy, patients in the retrospective registry might qualify for inclusion in the external control arm at multiple different points in time, when they receive different chemotherapy treatments. For example, a patient might qualify at the start of their second, third and fourth lines of therapy. From the start of which line of therapy should this patient's survival be compared to survival of participants in the single-arm trial? Some sponsors have elected to include patients in the external control arm from the last available line of therapy in the retrospective database. Another possibility is to randomly select a line of therapy for each external control arm patient from among those available. In this paper, we show, via probabilistic arguments and also via simulation based on real data, that both of these methods give rise to a bias in favor of the single-arm trial. We further show that this bias can be avoided by instead including external control arm patients multiple times in the external control arm, once for each time they receive qualifying treatment.

Quantitative bias analysis for external control arms using real-world data in clinical trials: a primer for clinical researchers.

Development of medicines in rare oncologic patient populations are growing, but well-powered randomized controlled trials are typically extremely challenging or unethical to conduct in such settings. External control arms using real-world data are increasingly used to supplement clinical trial evidence where no or little control arm data exists. The construction of an external control arm should always aim to match the population, treatment settings and outcome measurements of the corresponding treatment arm. Yet, external real-world data is typically fraught with limitations including missing data, measurement error and the potential for unmeasured confounding given a nonrandomized comparison. Quantitative bias analysis (QBA) comprises a collection of approaches for modelling the magnitude of systematic errors in data which cannot be addressed with conventional statistical adjustment. Their applications can range from simple deterministic equations to complex hierarchical models. QBA applied to external control arm represent an opportunity for evaluating the validity of the corresponding comparative efficacy estimates. We provide a brief overview of available QBA approaches and explore their application in practice. Using a motivating example of a comparison between pralsetinib single-arm trial data versus pembrolizumab alone or combined with chemotherapy real-world data for RET fusion-positive advanced non-small cell lung cancer (aNSCLC) patients (1-2% among all NSCLC), we illustrate how QBA can be applied to external control arms. We illustrate how QBA is used to ascertain robustness of results despite a large proportion of missing data on baseline ECOG performance status and suspicion of unknown confounding. The robustness of findings is illustrated by showing that no meaningful change to the comparative effect was observed across several 'tipping-point' scenario analyses, and by showing that suspicion of unknown confounding was ruled out by use of E-values. Full R code is also provided.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 11.

In this latest update we highlight a study from the REPEAT initiative that evaluates the reproducibility of real-world data studies, the publication of the HARPER Protocol Template developed by a joint ISPE/ISPOR taskforce, and discuss recent US FDA guidance on external control arms.