Analytical Methods for Comparing Uncontrolled Trials With External Controls From Real-World Data: A Systematic Literature Review and Comparison With European Regulatory and Health Technology Assessment Practice.

OBJECTIVES: This study aimed to provide an overview of analytical methods in scientific literature for comparing uncontrolled medicine trials with external controls from individual patient data real-world data (IPD-RWD) and to compare these methods with recommendations made in guidelines from European regulatory and health technology assessment (HTA) organizations and with their evaluations described in assessment reports. METHODS: A systematic literature review (until March 1, 2023) in PubMed and Connected Papers was performed to identify analytical methods for comparing uncontrolled trials with external controls from IPD-RWD. These methods were compared descriptively with methods recommended in method guidelines and encountered in assessment reports of the European Medicines Agency (2015-2020) and 4 European HTA organizations (2015-2023). RESULTS: Thirty-four identified scientific articles described analytical methods for comparing uncontrolled trial data with IPD-RWD-based external controls. The various methods covered controlling for confounding and/or dependent censoring, correction for missing data, and analytical comparative modeling methods. Seven guidelines also focused on research design, RWD quality, and transparency aspects, and 4 of those recommended analytical methods for comparisons with IPD-RWD. The methods discussed in regulatory (n = 15) and HTA (n = 35) assessment reports were often based on aggregate data and lacked transparency owing to the few details provided. CONCLUSIONS: Literature and guidelines suggest a methodological approach to comparing uncontrolled trials with external controls from IPD-RWD similar to target trial emulation, using state-of-the-art methods. External controls supporting regulatory and HTA decision making were rarely in line with this approach. Twelve recommendations are proposed to improve the quality and acceptability of these methods.

Model-based standardization using multiple imputation.

BACKGROUND: When studying the association between treatment and a clinical outcome, a parametric multivariable model of the conditional outcome expectation is often used to adjust for covariates. The treatment coefficient of the outcome model targets a conditional treatment effect. Model-based standardization is typically applied to average the model predictions over the target covariate distribution, and generate a covariate-adjusted estimate of the marginal treatment effect. METHODS: The standard approach to model-based standardization involves maximum-likelihood estimation and use of the non-parametric bootstrap. We introduce a novel, general-purpose, model-based standardization method based on multiple imputation that is easily applicable when the outcome model is a generalized linear model. We term our proposed approach multiple imputation marginalization (MIM). MIM consists of two main stages: the generation of synthetic datasets and their analysis. MIM accommodates a Bayesian statistical framework, which naturally allows for the principled propagation of uncertainty, integrates the analysis into a probabilistic framework, and allows for the incorporation of prior evidence. RESULTS: We conduct a simulation study to benchmark the finite-sample performance of MIM in conjunction with a parametric outcome model. The simulations provide proof-of-principle in scenarios with binary outcomes, continuous-valued covariates, a logistic outcome model and the marginal log odds ratio as the target effect measure. When parametric modeling assumptions hold, MIM yields unbiased estimation in the target covariate distribution, valid coverage rates, and similar precision and efficiency than the standard approach to model-based standardization. CONCLUSION: We demonstrate that multiple imputation can be used to marginalize over a target covariate distribution, providing appropriate inference with a correctly specified parametric outcome model and offering statistical performance comparable to that of the standard approach to model-based standardization.

Indirect Treatment Comparisons of Lenacapavir Plus Optimized Background Regimen Versus Other Treatments for Multidrug-Resistant Human Immunodeficiency Virus.

BACKGROUND/AIMS: Heavily treatment-experienced (HTE) people with human immunodeficiency virus (HIV) (PWH) may not achieve virologic suppression (VS) with combination antiretroviral therapy due to multidrug resistance (MDR), intolerance, and safety concerns. These PWH often receive highly individualized treatment regimens, but these regimens may not enable PWH to achieve VS, thereby halting disease progression. Novel medications are required for treating individuals with MDR HIV. Lenacapavir (LEN), a first-in-class HIV capsid inhibitor, is under investigation for the treatment of HTE individuals with MDR HIV in the phase 2/3 CAPELLA study. This study aimed to compare LEN plus optimized background regimen (OBR) with fostemsavir (FTR) + OBR, ibalizumab (IBA) + OBR, and OBR alone in terms of VS, CD4 cell count change from baseline, immunologic recovery, and discontinuation due to adverse events, using indirect treatment comparisons. METHODS: A systematic review identified clinical evidence on HIV-1 treatments in HTE PWH. A feasibility assessment evaluated the identified studies for indirect treatment comparison analyses based on population characteristics, interventions, comparators, and outcomes of interest. Unanchored simulated treatment comparisons of LEN + OBR versus comparators were conducted. RESULTS: LEN + OBR had 6.57 times higher odds of VS at weeks 24 to 28 than FTR + OBR (95% confidence interval [CI] 1.34-32.28), 8.93 times higher odds of VS than IBA + OBR (95% CI 2.07-38.46), and 12.74 times higher odds of VS than OBR alone (95% CI 1.70-95.37). Change from baseline in CD4 cell count was similar across LEN + OBR, FTR + OBR, and IBA + OBR. CONCLUSION: LEN + OBR has statistically significantly greater odds of VS at weeks 24 to 28 than its comparators and represents a novel treatment for people with MDR HIV.

A Comparison of Relative-Efficacy Estimate(S) Derived From Both Matching-Adjusted Indirect Comparisons and Standard Anchored Indirect Treatment Comparisons: A Review of Matching-Adjusted Indirect Comparisons.

OBJECTIVES: We present an empirical comparison of relative-efficacy estimate(s) from matching-adjusted indirect comparisons (MAICs) with estimates from corresponding standard anchored indirect treatment comparisons. METHODS: A total of 80 comparisons were identified from 17 publications through a systematic rapid review. A standardized metric that used reported relative treatment efficacy estimates and their associated uncertainty was used to compare the methods across different treatment indications and outcome measures. RESULTS: On aggregate, MAICs presented for connected networks tended to report a more favorable relative-efficacy estimate for the treatment for which individual-level patient data were available relative to the reported indirect treatment comparison estimate. CONCLUSIONS: Although we recognize the importance of MAIC and other population adjustment methods in certain situations, we recommend that results from these analyses are interpreted with caution. Researchers and analysts should carefully consider if MAICs are appropriate where presented and whether MAICs would have added value where omitted.

Interpreting and assessing confidence in network meta-analysis results: an introduction for clinicians.

PURPOSE: We aimed to provide clinicians with introductory guidance for interpreting and assessing confidence in on Network meta-analysis (NMA) results. METHODS: We reviewed current literature on NMA and summarized key points. RESULTS: Network meta-analysis (NMA) is a statistical method for comparing the efficacy of three or more interventions simultaneously in a single analysis by synthesizing both direct and indirect evidence across a network of randomized clinical trials. It has become increasingly popular in healthcare, since direct evidence (head-to-head randomized clinical trials) are not always available. NMA methods are categorized as either Bayesian or frequentist, and while the two mostly provide similar results, the two approaches are theoretically different and require different interpretations of the results. CONCLUSIONS: We recommend a careful approach to interpreting NMA results and the validity of an NMA depends on its underlying statistical assumptions and the quality of the evidence used in the NMA.

Indirect comparison of anti-interleukin 17 targeted biological treatments for moderate-to-severe psoriasis.

WHAT IS KNOWN AND OBJECTIVE: Psoriasis is an inflammatory skin disease with an important disease burden worldwide and its treatment includes systemic therapies which have advanced over time to target specific immune cytokines such as interleukin-17. The main objective of this study was to compare the relative efficacy of brodalumab, ixekizumab and secukinumab (three anti-interleukin-17 drugs) through adjusted indirect treatment comparisons (ITCs). METHODS: A search was carried out in June 2019, consulting these databases: MEDLINE, EMBASE, Web of Science and the Cochrane Library. Studies including patients with moderate to severe psoriasis randomized to receive treatment with anti-interleukin-17 drugs or ustekinumab and with outcomes such as Psoriasis Area and Severity Index (PASI) 75, PASI 90 and PASI 100 scores or static Physician's Global Assessment (sPGA), were included. ITCs were carried out using the method proposed by Bucher et al RESULTS AND DISCUSSION: Five randomized clinical trials were included. Analysing short-term data, there were no statistically significant differences between any pair of drugs in terms of PASI 75, PASI 100 or sPGA/sIGA 0/1. Analysing long-term data, statistically significant differences were only observed for secukinumab versus brodalumab in terms of PASI 100 (Absolute risk reduction -12.9%; 95% confidence interval -22.7% to -3.1%). WHAT IS NEW AND CONCLUSION: The ITCs indicated no efficacy differences between anti-interleukin-17 drugs, except for secukinumab versus brodalumab 52-week analysis in terms of achieving PASI 100. All three drugs appear to act as equivalent clinical treatments for psoriasis. However, independent head-to-head trials should be carried out.

Indirect comparisons of treatment effects: Network meta-analyses.

Many therapeutic options are now available for patients with multiple sclerosis. While the efficacy of each drug has been assessed against placebo or, more recently, against interferon, no direct comparisons of these new therapies have been conducted in randomized clinical trials. Therefore, indirect treatment comparisons are needed to inform clinical decisions. In this brief report, some basic concepts about network meta-analyses that are the formal methods used to run multiple indirect treatment comparisons are reviewed when applied in the context of multiple sclerosis studies.

Generalizing some key results from "alternative weighting schemes when performing matching-adjusted indirect comparisons".

A recent paper proposed an alternative weighting scheme when performing matching-adjusted indirect comparisons. This alternative approach follows the conventional one in matching the covariate means across two studies but differs in that it maximizes the effective sample size when doing so. The appendix of this paper showed, assuming there is one covariate and negative weights are permitted, that the resulting weights are linear in the covariates. This explains how the alternative method achieves a larger effective sample size and results in a metric that quantifies the difficulty of matching on particular covariates. We explain how these key results generalize to the case where there are multiple covariates, giving rise to a new metric that can be used to quantify the impact of matching on multiple covariates.

Four alternative methodologies for simulated treatment comparison: How could the use of simulation be re-invigorated?

Simulated treatment comparison (STC) is an established method for performing population adjustment for the indirect comparison of two treatments, where individual patient data (IPD) are available for one trial but only aggregate level information is available for the other. The most commonly used method is what we call 'standard STC'. Here we fit an outcome model using data from the trial with IPD, and then substitute mean covariate values from the trial where only aggregate level data are available, to predict what the first of these trial's outcomes would have been if its population had been the same as the second. However, this type of STC methodology does not involve simulation and can result in bias when the link function used in the outcome model is non-linear. An alternative approach is to use the fitted outcome model to simulate patient profiles in the trial for which IPD are available, but in the other trial's population. This stochastic alternative presents additional challenges. We examine the history of STC and propose two new simulation-based methods that resolve many of the difficulties associated with the current stochastic approach. A virtue of the simulation-based STC methods is that the marginal estimands are then clearly targeted. We illustrate all methods using a numerical example and explore their use in a simulation study.

'Twenty years of network meta-analysis: Continuing controversies and recent developments': A health technology assessment perspective.

We respond to some of the methodological issues raised in a recent review of network meta-analysis (NMA). We also provide a health technology developer's perspective and consider the future application of NMA to health technology assessment.

Network meta-analysis of CAR T-Cell therapy for the treatment of 3L+ R/R LBCL after using published comparative studies.

INTRODUCTION: Studies have compared chimeric antigen receptor (CAR) T-cell therapies and salvage chemotherapy in relapsed/refractory large B-cell lymphoma (LBCL) patients, but further evidence of their relative effectiveness is warranted. METHODS: Our systematic review identified studies comparing efficacy and safety outcomes of axicabtagene ciloleucel (axi-cel), lisocabtagene maraleucel (liso-cel) and tisagenlecleucel (tisa-cel) trials to salvage chemotherapy cohorts in LBCL patients with ≥2 prior lines of treatment; and an extended evidence network included indirect comparisons comparing CAR T-cell therapies. We conducted network meta-analyzes using Bayesian hierarchical modeling. RESULTS: Three studies comparing ZUMA-1 (axi-cel), TRANSCEND (liso-cel) and JULIET (tisa-cel) trials to salvage chemotherapy within the SCHOLAR-1 cohort were identified. Axi-cel (odds ratio [OR]:5.63; 95% credible interval [CrI]:2.66-12.42) and liso-cel (OR:4.26; 95%CrI:2.33-7.93) showed a significant increased overall response rate compared to tisa-cel, but not to one-another. Axi-cel demonstrated significant improvements in overall survival relative to liso-cel (hazard ratio [HR]:0.54; 95%CrI:0.37-0.79) and tisa-cel (HR:0.47; 95%CrI:0.26-0.88). Higher rates of grade ≥3 neurological events were observed with axi-cel than with tisa-cel and liso-cel. CONCLUSIONS: We highlight important differences in clinical outcomes between CAR T-cell therapies. Axi-cel demonstrated improved overall survival compared to tisa-cel and liso-cel, and both axi-cel and liso-cel showed higher response rates compared to tisa-cel.

Feasibility of Indirect Treatment Comparisons Between Niraparib Plus Abiraterone Acetate and Other First-Line Poly ADP-Ribose Polymerase Inhibitor Treatment Regimens for Patients with BRCA1/2 Mutation-Positive Metastatic Castration-Resistant Prostate Cancer.

INTRODUCTION: Poly(ADP-ribose) polymerase inhibitors (PARPi) are a novel option to treat patients with metastatic castration-resistant prostate cancer (mCRPC). Niraparib plus abiraterone acetate and prednisone (AAP) is indicated for BRCA1/2 mutation-positive mCRPC. Niraparib plus AAP demonstrated safety and efficacy in the phase 3 MAGNITUDE trial (NCT03748641). In the absence of head-to-head studies comparing PARPi regimens, the feasibility of conducting indirect treatment comparisons (ITC) to inform decisions for patients with first-line BRCA1/2 mutation-positive mCRPC has been explored. METHODS: A systematic literature review was conducted to identify evidence from randomized controlled trials on relevant comparators to inform the feasibility of conducting ITCs via network meta-analysis (NMA) or population-adjusted indirect comparisons (PAIC). Feasibility was assessed based on network connectivity, data availability in the BRCA1/2 mutation-positive population, and degree of within- and between-study heterogeneity or bias. RESULTS: NMAs between niraparib plus AAP and other PARPi regimens (olaparib monotherapy, olaparib plus AAP, and talazoparib plus enzalutamide) were inappropriate due to the disconnected network, differences in trial populations related to effect modifiers, or imbalances within BRCA1/2 mutation-positive subgroups. The latter issue, coupled with the lack of a common comparator (except for olaparib plus AAP), also rendered anchored PAICs infeasible. Unanchored PAICs were either inappropriate due to lack of population overlap (vs. olaparib monotherapy) or were restricted by unmeasured confounders and small sample size (vs. olaparib plus AAP). PAIC versus talazoparib plus enzalutamide was not possible due to lack of published arm-level baseline characteristics and sufficient efficacy outcome data in the relevant population. CONCLUSION: The current randomized controlled trial evidence network does not permit robust comparisons between niraparib plus AAP and other PARPi regimens for patients with 1L BRCA-positive mCRPC. Decision-makers should scrutinize any ITC results in light of their limitations. Real-world evidence combined with clinical experience should inform treatment recommendations in this indication.

Population adjusted-indirect comparisons in health technology assessment: A methodological systematic review.

In health technology assessment (HTA), population-adjusted indirect comparisons (PAICs) are increasingly considered to adjust for the difference in the target population between studies. We aim to assess the conduct and reporting of PAICs in recent HTA practice, by performing, a methodological systematic review of studies implementing PAICs from PubMed, EMBASE Classic, Embase/Ovid Medline All, and Cochrane databases from January 1, 2010 to Feb 13, 2023. Four independent researchers screened the titles, abstracts, and full-texts of the identified records, then extracted data on methodological and reporting characteristics of 106 eligible articles. Most PAIC analyses (96.9%, n = 157) were conducted by (or received funding from) pharmaceutical companies. Prior to adjustment, 44.5% of analyses (n = 72) (partially) aligned the eligibility criteria of different studies to enhance the similarity of their target populations. In 37.0% of analyses (n = 60), the clinical and methodological heterogeneity across studies were extensively assessed. In 9.3% of analyses (n = 15), the quality (or bias) of individual studies was evaluated. Among 18 analyses using methods that required an outcome model specification, results of the model fitting procedure were adequately reported in three analyses (16.7%). These findings suggest that the conduct and reporting of PAICs are remarkably heterogeneous and suboptimal in current practice. More recommendations and guidelines on PAICs are thus warranted to enhance the quality of these analyses in the future.

Bayesian hierarchical model-based network meta-analysis to overcome survival extrapolation challenges caused by data immaturity.

Aim: This research evaluated standard Weibull mixture cure (WMC) network meta-analysis (NMA) with Bayesian hierarchical (BH) WMC NMA to inform long-term survival of therapies. Materials & methods: Four trials in previously treated metastatic non-small-cell lung cancer with PD-L1 >1% were used comparing docetaxel with nivolumab, pembrolizumab and atezolizumab. Cure parameters related to a certain treatment class were assumed to share a common distribution. Results: Standard WMC NMA predicted cure rates were 0.03 (0.01; 0.07), 0.18 (0.12; 0.24), 0.07 (0.02; 0.15) and 0.03 (0.00; 0.09) for docetaxel, nivolumab, pembrolizumab and atezolizumab, respectively, with corresponding incremental life years (LY) of 3.11 (1.65; 4.66), 1.06 (0.41; 2.37) and 0.42 (-0.57; 1.68). The Bayesian hierarchical-WMC-NMA rates were 0.06 (0.03; 0.10), 0.17 (0.11; 0.23), 0.12 (0.05; 0.20) and 0.12 (0.03; 0.23), respectively, with incremental LY of 2.35 (1.04; 3.93), 1.67 (0.68; 2.96) and 1.36 (-0.05; 3.64). Conclusion: BH-WMC-NMA impacts incremental mean LYs and cost-effectiveness ratios, potentially affecting reimbursement decisions.

Comparison of indirect treatment methods in migraine prevention to address differences in mode of administration.

Aim: Indirect treatment comparisons (ITCs) are anchored on a placebo comparator, and the placebo response may vary according to drug administration route. Migraine preventive treatment studies were used to evaluate ITCs and determine whether mode of administration influences placebo response and the overall study findings. Materials & methods: Change from baseline in monthly migraine days produced by monoclonal antibody treatments (subcutaneous, intravenous) was compared using fixed-effects Bayesian network meta-analysis (NMA), network meta-regression (NMR), and unanchored simulated treatment comparison (STC). Results: NMA and NMR provide mixed, rarely differentiated results between treatments, whereas unanchored STC strongly favors eptinezumab over other preventive treatments. Conclusion: Further investigations are needed to determine which ITC best reflects the impact of mode of administration on placebo.

Network Meta-Analysis.

There are often multiple potential interventions to treat a disease; therefore, we need a method for simultaneously comparing and ranking all of these available interventions. In contrast to pairwise meta-analysis, which allows for the comparison of one intervention to another based on head-to-head data from randomized trials, network meta-analysis (NMA) facilitates simultaneous comparison of the efficacy or safety of multiple interventions that may not have been directly compared in a randomized trial. NMAs help researchers study important and previously unanswerable questions, which have contributed to a rapid rise in the number of NMA publications in the biomedical literature. However, the conduct and interpretation of NMAs are more complex than pairwise meta-analyses: there are additional NMA model assumptions (i.e., network connectivity, homogeneity, transitivity, and consistency) and outputs (e.g., network plots and surface under the cumulative ranking curves [SUCRAs]). In this chapter, we will: (1) explore similarities and differences between pairwise and network meta-analysis; (2) explain the differences between direct, indirect, and mixed treatment comparisons; (3) describe how treatment effects are derived from NMA models; (4) discuss key criteria predicating completion of NMA; (5) interpret NMA outputs; (6) discuss areas of ongoing methodological research in NMA; (7) outline an approach to conducting a systematic review and NMA; (8) describe common problems that researchers encounter when conducting NMAs and potential solutions; and (9) outline an approach to critically appraising a systematic review and NMA.

Efficacy and Safety of LAMA/LABA Fixed-Dose Combination Therapies in Chronic Obstructive Pulmonary Disease: A Systematic Review of Direct and Indirect Treatment Comparisons.

Background: This literature review assessed comparative efficacy and safety of long-acting muscarinic antagonist/long-acting ß2-agonist (LAMA/LABA) fixed-dose combinations (FDCs) in patients with COPD and moderate-to-very severe airflow limitation, using evidence from direct (head-to-head) and indirect treatment comparisons. Methods: Two systematic literature reviews were conducted to identify direct comparisons (head-to-head randomized controlled trials [RCTs]) and indirect comparisons (network meta-analyses [NMAs]; indirect treatment comparisons; meta-analyses) in patients with COPD with moderate-to-very severe airflow limitation. Study/Analysis characteristics, eligibility criteria, patient characteristics, and overall conclusions were extracted from relevant publications. The review of indirect comparisons focused on NMAs reporting efficacy outcomes at 12 and 24 weeks of treatment (established durations of symptomatic studies in COPD recommended by regulators). Results: Direct comparisons: Four RCTs that provided head-to-head comparisons of LAMA/LABA FDCs were identified, and these varied in their study design, included patient population and reported endpoints. While some differences in lung function outcomes were noted, where assessed, LAMA/LABA FDCs had comparable efficacy in improving symptoms, health status, exacerbations, and comparable safety profiles. However, the differences in study methodology and patient characteristics between these studies made it difficult to draw generalizable conclusions regarding the comparative effectiveness of LAMA/LABA FDCs from the direct comparisons alone. Indirect comparisons: Six NMAs were identified that reported indirect comparisons between LAMA/LABA FDCs; five of these were within the pre-defined scope of this review. Although the scope of each NMA varied, all five concluded that LAMA/LABA FDCs were generally comparable in terms of lung function improvements, patient-reported outcomes, and safety (where assessed). Conclusion: Although there were some inconsistencies between the outcomes of RCTs and NMAs for lung function, the totality of lung function, symptoms, exacerbations, and safety data suggests that currently available LAMA/LABA FDCs have comparable efficacy and safety in patients with COPD and moderate-to-very severe airflow limitation.

Treatment sequence network meta-analysis in Crohn's disease: a methodological case study.

OBJECTIVE: Several biologic therapies are available for the treatment of mild-to-moderate Crohn's disease (CD). This network meta-analysis (NMA) aimed to assess the comparative efficacy of ustekinumab, adalimumab, vedolizumab and infliximab in the maintenance of clinical response and remission after 1 year of treatment. METHODS: A systematic literature search was performed to identify relevant randomized controlled trials (RCTs). Key outcomes of interest were clinical response (CD activity index [CDAI] reduction of 100 points; CDAI-100) and remission (CDAI score under 150 points; CDAI < 150). A treatment sequence Bayesian NMA was conducted to account for the re-randomization of patients based on different clinical definitions, the lack of similarity of the common comparator for each trial and the full treatment pathway from the induction phase onwards. RESULTS: Thirteen RCTs were identified. Ustekinumab 90 mg q8w was associated with statistically significant improvement in clinical response relative to placebo and vedolizumab 300 mg. For clinical remission, ustekinumab 90 mg q8w was associated with statistically significant improvement relative to placebo and vedolizumab 300 mg q8w. Findings from sub-population analyses had similar results but were not statistically significant. CONCLUSIONS: The NMA suggest that ustekinumab is associated with the highest likelihood of reaching response or remission at 1 year compared with placebo, adalimumab and vedolizumab. Results should be interpreted with caution because this is a novel methodology; however, the treatment sequence analysis may be the most methodologically sound analysis to derive estimates of comparative efficacy in CD in the absence of head-to-head evidence.

Network meta-analysis: an introduction for pharmacists.

Network meta-analysis is a new tool used to summarize and compare studies for multiple interventions, irrespective of whether these interventions have been directly evaluated against each other. Network meta-analysis is quickly becoming the standard in conducting therapeutic reviews and clinical guideline development. However, little guidance is available to help pharmacists review network meta-analysis studies in their practice. Major institutions such as the Cochrane Collaboration, Agency for Healthcare Research and Quality, Canadian Agency for Drugs and Technologies in Health, and National Institute for Health and Care Excellence Decision Support Unit have endorsed utilizing network meta-analysis to establish therapeutic evidence and inform decision making. Our objective is to introduce this novel technique to pharmacy practitioners, and highlight key assumptions behind network meta-analysis studies.