A Comparison of Relative-Efficacy Estimate(S) Derived From Both Matching-Adjusted Indirect Comparisons and Standard Anchored Indirect Treatment Comparisons: A Review of Matching-Adjusted Indirect Comparisons.

OBJECTIVES: We present an empirical comparison of relative-efficacy estimate(s) from matching-adjusted indirect comparisons (MAICs) with estimates from corresponding standard anchored indirect treatment comparisons. METHODS: A total of 80 comparisons were identified from 17 publications through a systematic rapid review. A standardized metric that used reported relative treatment efficacy estimates and their associated uncertainty was used to compare the methods across different treatment indications and outcome measures. RESULTS: On aggregate, MAICs presented for connected networks tended to report a more favorable relative-efficacy estimate for the treatment for which individual-level patient data were available relative to the reported indirect treatment comparison estimate. CONCLUSIONS: Although we recognize the importance of MAIC and other population adjustment methods in certain situations, we recommend that results from these analyses are interpreted with caution. Researchers and analysts should carefully consider if MAICs are appropriate where presented and whether MAICs would have added value where omitted.

Cost-utility and value of information analysis of tisagenlecleucel for relapsed/refractory diffuse large B-cell lymphoma in the Irish healthcare setting.

Background: The evidence base of tisagenlecleucel is uncertain. Objective: To evaluate the cost-effectiveness of tisagenlecleucel. To conduct expected value of perfect information (EVPI) and partial EVPI (EVPPI) analyses. Study Design: A three-state partitioned survival model. A short-term decision tree partitioned patients in the tisagenlecleucel arm according to infusion status. Survival was extrapolated to 5 years; general population mortality with a standardised mortality ratio was then applied. EVPI and EVPPI were scaled up to population according to the incidence of the decision. Setting: Irish healthcare payer. Participants: Patients with relapsed/refractory diffuse large B-cell lymphoma (R/R DLBCL). Interventions: Tisagenlecleucel versus Salvage Chemotherapy (with or without haematopoietic stem cell transplant). Main Outcome Measure: Incremental cost-effectiveness ratio (ICER). Population EVPI and EVPPI. Results: At list prices, the ICER was €119,509 per quality-adjusted life year (QALY) (incremental costs €218,092; incremental QALYs 1.82). Probability of cost-effectiveness, at a €45,000 per QALY threshold, was 0%. Population EVPI was €0.00. Population EVPI, at the price of tisagenlecleucel that reduced the ICER to €45,000 per QALY, was €3,989,438. Here, survival analysis had the highest population EVPPI (€1,128,053). Conclusion: Tisagenlecleucel is not cost-effective, versus salvage chemotherapy (with or without haematopoietic stem cell transplant), for R/R DLBCL in Ireland. At list prices, further research to decrease decision uncertainty may not be of value.

Tisagenlecleucel for relapsed/refractory acute lymphoblastic leukemia in the Irish healthcare setting: cost-effectiveness and value of information analysis.

OBJECTIVES: This study evaluates the cost-effectiveness of tisagenlecleucel (a CAR T-cell therapy), versus blinatumomab, for the treatment of pediatric and young adult patients with relapsed/refractory acute lymphoblastic leukemia (R/R ALL) in the Irish healthcare setting. The value of conducting further research, to investigate the value of uncertainty associated with the decision problem, is assessed by means of expected value of perfect information (EVPI) and partial EVPI (EVPPI) analyses. METHODS: A three-state partitioned survival model was developed. A short-term decision tree partitioned patients in the tisagenlecleucel arm according to infusion status. Survival was extrapolated to 60 months; general population mortality with a standardized mortality ratio was then applied. Estimated EVPI and EVPPI were scaled up to population according to the incidence of the decision. RESULTS: At list prices, the incremental cost-effectiveness ratio was EUR 73,086 per quality-adjusted life year (QALY) (incremental costs EUR 156,928; incremental QALYs 2.15). The probability of cost-effectiveness, at the willingness-to-pay threshold of EUR 45,000 per QALY, was 16 percent. At this threshold, population EVPI was EUR 314,455; population EVPPI was below EUR 100,000 for each parameter category. CONCLUSIONS: Tisagenlecleucel is not cost effective, versus blinatumomab, for the treatment of pediatric and young adult patients with R/R ALL in Ireland (at list prices). Further research to decrease decision (parameter) uncertainty, at the defined willingness-to-pay threshold, may not be of value. However, there is a high degree of uncertainty underpinning the analysis, which may not be captured by EVPI analysis.

Network Meta-analysis on Disconnected Evidence Networks When Only Aggregate Data Are Available: Modified Methods to Include Disconnected Trials and Single-Arm Studies while Minimizing Bias.

BACKGROUND: Network meta-analysis (NMA) requires a connected network of randomized controlled trials (RCTs) and cannot include single-arm studies. Regulators or academics often have only aggregate data. Two aggregate data methods for analyzing disconnected networks are random effects on baseline and aggregate-level matching (ALM). ALM has been used only for single-arm studies, and both methods may bias effect estimates. METHODS: We modified random effects on baseline to separate RCTs connected to and disconnected from the reference and any single-arm studies, minimizing the introduction of bias. We term our modified method reference prediction. We similarly modified ALM and extended it to include RCTs disconnected from the reference. We tested these methods using constructed data and a simulation study. RESULTS: In simulations, bias for connected treatments for ALM ranged from -0.0158 to 0.051 and for reference prediction from -0.0107 to 0.083. These were low compared with the true mean effect of 0.5. Coverage ranged from 0.92 to 1.00. In disconnected treatments, bias of ALM ranged from -0.16 to 0.392 and of reference prediction from -0.102 to 0.40, whereas coverage of ALM ranged from 0.30 to 0.82 and of reference prediction from 0.64 to 0.94. Under fixed study effects for disconnected evidence, bias was similar, but coverage was 0.81 to 1.00 for reference prediction and 0.18 to 0.76 for ALM. Trends of similar bias but greater coverage for reference prediction with random study effects were repeated in constructed data. CONCLUSIONS: Both methods with random study effects seem to minimize bias in treatment connected to the reference. They can estimate treatment effects for disconnected treatments but may be biased. Reference prediction has greater coverage and may be recommended overall. HIGHLIGHTS: Two methods were modified for network meta-analysis on disconnected networks and for including single-arm observational or interventional studies in network meta-analysis using only aggregate data and for minimizing the bias of effect estimates for treatments only in trials connected to the reference.Reference prediction was developed as a modification of random effects on baseline that keeps analyses of trials connected to the reference separately from those disconnected from the reference and from single-arm studies. The method was further modified to account for correlation in trials with more than 2 arms and, under random study effects, to estimate variance in heterogeneity separately in connected and disconnected evidence.Aggregate-level matching was extended to include trials disconnected from the reference, rather than only single-arm studies. The method was further modified to separately estimate treatment effects and heterogeneity variance in the connected and disconnected evidence and to account for the correlation between arms in trials with more than 2 arms.Performance was assessed using a constructed data example and simulation study.The methods were found to have similar, and sometimes low, bias when estimating the relative effects for disconnected treatments, but reference prediction with random study effects had the greatest coverage.The use of reference prediction with random study effects for disconnected networks is recommended if no individual patient data or alternative real-world evidence is available.

Coronavirus Disease 2019: Considerations for Health Technology Assessment From the National Centre for Pharmacoeconomics Review Group.

It is expected that the coronavirus disease 2019 (COVID-19) pandemic will leave large deficits in the budgets of many jurisdictions. Funding for other treatments, in particular new treatments, may become more constrained than previously expected. Therefore, a robust health technology assessment (HTA) system is vital. Many clinical trials carried out during the pandemic may have been temporarily halted, while others may have had to change their protocols. Even trials that continue as normal may experience external changes as other aspects of the healthcare service may not be available to the patients in the trial, or the patients themselves may contract COVID-19. Consequently, many limitations are likely to arise in the provision of robust HTAs, which could have profound consequences on the availability of new treatments. Therefore, the National Centre for Pharmacoeconomics Review Group wishes to discuss these issues and make recommendations for applicants submitting to HTA agencies, in ample time for these HTAs to be prepared and assessed. We discuss how the pandemic may affect the estimation of the treatment effect, costs, life-years, utilities, discontinuation rates, and methods of evidence synthesis and extrapolation. In particular, we note that trials conducted during the pandemic will be subject to a higher degree of uncertainty than before. It is vital that applicants clearly identify any parameters that may be affected by the pandemic. These parameters will require considerably more scenario and sensitivity analyses to account for this increase in uncertainty.

Comparative efficacy of exercise and anti-hypertensive pharmacological interventions in reducing blood pressure in people with hypertension: A network meta-analysis.

AIMS: This analysis aims to estimate the comparative efficacy of anti-hypertensive medications and exercise interventions on systolic and diastolic blood pressure reduction in people with hypertension. METHODS: A systematic review was conducted focusing on randomised controlled trials (RCTs) of exercise interventions and first-line anti-hypertensives where blood pressure reduction was the primary outcome in those with hypertension. Network meta-analyses were conducted to generate estimates of comparative efficacy. RESULTS: We identified 93 RCTs (N = 32,404, mean age in RCTs: 39-70 years) which compared placebo or usual care with first-line antihypertensives including angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, calcium channel blockers and thiazide-like diuretics and exercise interventions including aerobic training and dynamic resistance training. Of these, there were 81 (87%) trials related to medications (n = 31,347, 97%) and 12 (13%) trials related to exercise (n = 1057, 3%). The point estimates suggested that antihypertensive medications were more effective than exercise but there was insufficient evidence to suggest that first-line medications significantly reduced blood pressure to a greater extent than did the exercise interventions. Of the first-line treatments, angiotensin receptor blockers and calcium channel blockers had the highest treatment ranking, while exercise had the second lowest treatment ranking, followed by control conditions. CONCLUSION: The current evidence base with a bias towards medication research may partly explain the circumspection around the efficacy of exercise in guidelines and practice. Clinicians may justifiably consider exercise for low risk hypertension patients who confirm a preference for such an approach.

Assessing the impact of a matching-adjusted indirect comparison in a Bayesian network meta-analysis.

If IPD is available for some or all trials in a network meta-analysis (NMA), then incorporating this IPD into an NMA is routinely considered to be preferable. However, the situation often arises where a researcher has IPD for trials concerning a particular treatment (eg, from a sponsor) but none for other trials. Therefore, one can reweight the IPD so that the covariate characteristics in the IPD trials match that of the aggregate data (AgD) trials, using a matching-adjusted indirect comparison (MAIC). We assess the impact of using the reweighted aggregated data, obtained by the MAIC, in a Bayesian NMA for a connected treatment network. We apply this method to a network of multiple myeloma treatments in newly diagnosed patients (ndMM), where the outcome is progression free survival. We investigate the reliability of the methods and results through a simulation study. The ndMM network consists of three IPD studies comparing lenalidomide to placebo (Len-Placebo), one AgD study comparing Len-Placebo, and one AgD study comparing thalidomide to placebo (Thal-Placebo). We therefore investigate two options of weighting the covariates: (a) All three studies are weighted separately to match the AgD Thal-Placebo trial. (b) Patients are weighted across all three IPD studies to match the AgD Thal-Placebo trial, but the NMA considers each trial separately. We observe limited benefit to MAIC in the full network population. While MAIC can be beneficial as a sensitivity analysis to confirm results across patient populations, we advise that MAIC is used and interpreted with caution.

Difficulties arising in reimbursement recommendations on new medicines due to inadequate reporting of population adjustment indirect comparison methods.

Indirect treatment comparisons are useful to estimate relative treatment effects when head-to-head studies are not conducted. Statisticians at the National Centre for Pharmacoeconomics Ireland (NCPE) and Scottish Medicines Consortium (SMC) assess the clinical and cost-effectiveness of new medicines as part of multidisciplinary teams. We describe some shared observations on areas where reporting of population-adjustment indirect comparison methods is causing uncertainty in our recommendations to decision-making committees when assessing reimbursement of medicines.

Incorporating single-arm evidence into a network meta-analysis using aggregate level matching: Assessing the impact.

Increasingly, single-armed evidence is included in health technology assessment submissions when companies are seeking reimbursement for new drugs. While it is recognized that randomized controlled trials provide a higher standard of evidence, these are not available for many new agents that have been granted licenses in recent years. Therefore, it is important to examine whether alternative strategies for assessing this evidence may be used. In this work, we examine approaches to incorporating single-armed evidence formally in the evaluation process. We consider matching aggregate level covariates to comparator arms or trials and including this evidence in a network meta-analysis. We consider two methods of matching: (i) we include the chosen matched arm in the data set itself as a comparator for the single-arm trial; (ii) we use the baseline odds of an event in a chosen matched trial to use as a plug-in estimator for the single-arm trial. We illustrate that the synthesis of evidence resulting from such a setup is sensitive to the between-study variability, formulation of the prior for the between-design effect, weight given to the single-arm evidence, and extent of the bias in single-armed evidence. We provide a flowchart for the process involved in such a synthesis and highlight additional sensitivity analyses that should be carried out. This work was motivated by a hepatitis C data set, where many agents have only been examined in single-arm studies. We present the results of our methods applied to this data set.

The impact of individual patient data in a network meta-analysis: An investigation into parameter estimation and model selection.

The use of individual patient data (IPD) in network meta-analysis (NMA) is becoming increasingly popular. However, as most studies do not report IPD, most NMAs are performed using aggregate data for at least some, if not all, of the studies. We investigate the benefits of including varying proportions of IPD studies in an NMA. Several models have previously been developed for including both aggregate data and IPD in the same NMA. We performed a simulation study based on these models to examine the impact of additional IPD studies on the accuracy and precision of the estimates of both the treatment effect and the covariate effect. We also compared the deviance information criterion (DIC) between models to assess model fit. An increased proportion of IPD resulted in more accurate and precise estimates for most models and datasets. However, the coverage probability sometimes decreased when the model was misspecified. The use of IPD leads to greater differences in DIC, which allows us choose the correct model more often. We analysed a Hepatitis C network consisting of 3 IPD observational studies. The ranking of treatments remained the same for all models and datasets. We observed similar results to the simulation study: The use of IPD leads to differences in DIC and more precise estimates for the covariate effect. However, IPD sometimes increased the posterior SD of the treatment effect estimate, which may indicate between study heterogeneity. We recommend that IPD should be used where possible, especially for assessing model fit.

The use of single armed observational data to closing the gap in otherwise disconnected evidence networks: a network meta-analysis in multiple myeloma.

BACKGROUND: Network meta-analysis (NMA) allows for the estimation of comparative effectiveness of treatments that have not been studied in head-to-head trials; however, relative treatment effects for all interventions can only be derived where available evidence forms a connected network. Head-to-head evidence is limited in many disease areas, regularly resulting in disconnected evidence structures where a large number of treatments are available. This is also the case in the evidence of treatments for relapsed or refractory multiple myeloma. METHODS: Randomised controlled trials (RCTs) identified in a systematic literature review form two disconnected evidence networks. Standard Bayesian NMA models are fitted to obtain estimates of relative effects within each network. Observational evidence was identified to fill the evidence gap. Single armed trials are matched to act as each other's control group based on a distance metric derived from covariate information. Uncertainty resulting from including this evidence is incorporated by analysing the space of possible matches. RESULTS: Twenty five randomised controlled trials form two disconnected evidence networks; 12 single armed observational studies are considered for bridging between the networks. Five matches are selected to bridge between the networks. While significant variation in the ranking is observed, daratumumab in combination with dexamethasone and either lenalidomide or bortezomib, as well as triple therapy of carfilzomib, ixazomib and elozumatab, in combination with lenalidomide and dexamethasone, show the highest effects on progression free survival, on average. CONCLUSIONS: The analysis shows how observational data can be used to fill gaps in the existing networks of RCT evidence; allowing for the indirect comparison of a large number of treatments, which could not be compared otherwise. Additional uncertainty is accounted for by scenario analyses reducing the risk of over confidence in interpretation of results.