Challenges of Incorporating Life Cycle Drug Pricing in Cost-Effectiveness Models: A Review of Methods and Modeling Suggestions.

OBJECTIVES: This study aimed to conduct a review of existing methods used to incorporate life cycle drug pricing (LCDP) in cost-effectiveness analyses (CEAs), identify common methodological challenges, and suggest modeling approaches for prospectively implementing LCDP in CEA. METHODS: Two complementary searches were conducted in PubMed, combined with hand searching and reference mining, to identify English language full-text articles that explored (1) how drug prices change over time and (2) methods used to apply dynamic pricing in cost-effectiveness models (CEMs). Relevant articles were reviewed, and authors discussed the common methodological practices used in the literature and their associated challenges on prospectively implementing LCDP in CEMs. For each key challenge identified, we provide modeling suggestions to address the issue. RESULTS: We screened 1200 studies based on title and abstract; 117 were reviewed for eligibility, and 47 individual studies were included across both searches. Variations in prices over a product's life cycle are complex and multifactorial, and models applying LCDP in CEA varied in their methodology. We identified 4 key challenges to modeling LCDP in CEA, including how to model price trends before and after loss of exclusivity, how to capture the effect of price changes on future patient cohorts, and how to report results. CONCLUSION: Accurately quantifying the impact of LCDP requires careful consideration of multiple aspects pertaining to both the evolution of drug prices and how to reflect these in CEA. Although uncertainties remain, our findings can aid implementation and evaluation of LCDP in economic evaluations.

A Comparison of Alternative Network Meta-Analysis Methods in the Presence of Nonproportional Hazards: A Case Study in First-Line Advanced or Metastatic Renal Cell Carcinoma.

OBJECTIVES: Network meta-analysis (NMA) of time-to-event outcomes based on constant hazard ratios can result in biased findings when the proportional hazards (PHs) assumption does not hold in a subset of trials. We aimed to summarize the published non-PH NMA methods for time-to-event outcomes, demonstrate their application, and compare their results. METHODS: The following non-PH NMA methods were compared through an illustrative case study in oncology of 4 randomized controlled trials in terms of progression-free survival and overall survival: (1) 1-step or (2) 2-step multivariate NMAs based on traditional survival distributions or fractional polynomials, (3) NMAs with restricted cubic splines for baseline hazard, and (4) restricted mean survival NMA. RESULTS: For progression-free survival, the PH assumption did not hold across trials and non-PH NMA methods better reflected the relative treatment effects over time. The most flexible models (fractional polynomials and restricted cubic splines) fit better to the data than the other approaches. Estimated hazard ratios obtained with different non-PH NMA methods were similar at 5 years of follow-up but differed thereafter in the extrapolations. Although there was no strong evidence of PH violation for overall survival, non-PH NMA methods captured this uncertainty in the relative treatment effects over time. CONCLUSIONS: When the PH assumption is questionable in a subset of the randomized controlled trials, we recommend assessing alternative non-PH NMA methods to estimate relative treatment effects for time-to-event outcomes. We propose a transparent and explicit stepwise model selection process considering model fit, external constraints, and clinical validity. Given inherent uncertainty, sensitivity analyses are suggested.

A Guide to Selecting Flexible Survival Models to Inform Economic Evaluations of Cancer Immunotherapies.

OBJECTIVES: Parametric models are routinely used to estimate the benefit of cancer drugs beyond trial follow-up. The advent of immune checkpoint inhibitors has challenged this paradigm, and emerging evidence suggests that more flexible survival models, which can better capture the shapes of complex hazard functions, might be needed for these interventions. Nevertheless, there is a need for an algorithm to help analysts decide whether flexible models are required and, if so, which should be chosen for testing. This position article has been produced to bridge this gap. METHODS: A virtual advisory board comprising 7 international experts with in-depth knowledge of survival analysis and health technology assessment was held in summer 2021. The experts discussed 24 questions across 6 topics: the current survival model selection procedure, data maturity, heterogeneity of treatment effect, cure and mortality, external evidence, and additions to existing guidelines. Their responses culminated in an algorithm to inform selection of flexible survival models. RESULTS: The algorithm consists of 8 steps and 4 questions. Key elements include the systematic identification of relevant external data, using clinical expert input at multiple points in the selection process, considering the future and the observed hazard functions, assessing the potential for long-term survivorship, and presenting results from all plausible models. CONCLUSIONS: This algorithm provides a systematic, evidence-based approach to justify the selection of survival extrapolation models for cancer immunotherapies. If followed, it should reduce the risk of selecting inappropriate models, partially addressing a key area of uncertainty in the economic evaluation of these agents.

A Review of Overall Survival Extrapolations of Immune-Checkpoint Inhibitors Used in Health Technology Assessments by the French Health Authorities.

OBJECTIVES: Extrapolation is often required to inform cost-effectiveness (CE) evaluations of immune-checkpoint inhibitors (ICIs) since survival data from pivotal clinical trials are seldom complete. The objectives of this study were to evaluate the accuracy of estimates of long-term overall survival (OS) predicted in French CE assessment reports of ICIs, and to identify models presenting the best fit to the observed long-term survival data. METHODS: A systematic review of French assessment reports of ICIs in the metastatic setting since inception until May 2020 was performed. A targeted literature review was conducted to collect associated extended follow-up of randomized controlled trials (RCTs) used in the CE assessment reports. Difference between projected and observed OS was calculated. A range of standard parametric and spline-based models were applied to the extended follow-up data from the RCT to determine the best-fitting survival models. RESULTS: Of the 121 CE assessment reports published, 11 reports met the inclusion criteria. OS was underestimated in 73 percent of the CE assessment reports. The mean relative difference between each source was -13 percent (median: -15 percent; IQR: -0.4 to 26 percent). Models providing the best fit were those that could reflect nonmonotonic hazards. CONCLUSIONS: Based on the available data at the time of submission, longer-term survival of ICIs was not fully captured by the extrapolation models used in CE assessments. Standard and flexible parametric models which can capture nonmonotonic hazard functions provided the best fit to the extended follow-up data. However, these models may still have performed poorly if fitted to survival data available at the time of submission to the French National Authority for Health.

Assessment of technical errors and validation processes in economic models submitted by the company for NICE technology appraisals.

BACKGROUND: Economic models play a central role in the decision-making process of the National Institute for Health and Care Excellence (NICE). Inadequate validation methods allow for errors to be included in economic models. These errors may alter the final recommendations and have a significant impact on outcomes for stakeholders. OBJECTIVE: To describe the patterns of technical errors found in NICE submissions and to provide an insight into the validation exercises carried out by the companies prior to submission. METHODS: All forty-one single technology appraisals (STAs) completed in 2017 by NICE were reviewed and all were on medicines. The frequency of errors and information on their type, magnitude, and impact was extracted from publicly available NICE documentation along with the details of model validation methods used. RESULTS: Two STAs (5 percent) had no reported errors, nineteen (46 percent) had between one and four errors, sixteen (39 percent) had between five and nine errors, and four (10 percent) had more than ten errors. The most common errors were transcription errors (29 percent), logic errors (29 percent), and computational errors (25 percent). All STAs went through at least one type of validation. Moreover, errors that were notable enough were reported in the final appraisal document (FAD) in eight (20 percent) of the STAs assessed but each of these eight STAs received positive recommendations. CONCLUSIONS: Technical errors are common in the economic models submitted to NICE. Some errors were considered important enough to be reported in the FAD. Improvements are needed in the model development process to ensure technical errors are kept to a minimum.

An Evaluation of an Algorithm for the Selection of Flexible Survival Models for Cancer Immunotherapies: Pass or Fail?

BACKGROUND AND OBJECTIVE: Accurately extrapolating survival beyond trial follow-up is essential in a health technology assessment where model choice often substantially impacts estimates of clinical and cost effectiveness. Evidence suggests standard parametric models often provide poor fits to long-term data from immuno-oncology trials. Palmer et al. developed an algorithm to aid the selection of more flexible survival models for these interventions. We assess the usability of the algorithm, identify areas for improvement and evaluate whether it effectively identifies models capable of accurate extrapolation. METHODS: We applied the Palmer algorithm to the CheckMate-649 trial, which investigated nivolumab plus chemotherapy versus chemotherapy alone in patients with gastroesophageal adenocarcinoma. We evaluated the algorithm's performance by comparing survival estimates from identified models using the 12-month data cut to survival observed in the 48-month data cut. RESULTS: The Palmer algorithm offers a systematic procedure for model selection, encouraging detailed analyses and ensuring that crucial stages in the selection process are not overlooked. In our study, a range of models were identified as potentially appropriate for extrapolating survival, but only flexible parametric non-mixture cure models provided extrapolations that were plausible and accurately predicted subsequently observed survival. The algorithm could be improved with minor additions around the specification of hazard plots and setting out plausibility criteria. CONCLUSIONS: The Palmer algorithm provides a systematic framework for identifying suitable survival models, and for defining plausibility criteria for extrapolation validity. Using the algorithm ensures that model selection is based on explicit justification and evidence, which could reduce discordance in health technology appraisals.

Heterogeneity in Survival with Immune Checkpoint Inhibitors and Its Implications for Survival Extrapolations: A Case Study in Advanced Melanoma.

Background: Survival heterogeneity and limited trial follow-up present challenges for estimating lifetime benefits of oncology therapies. This study used CheckMate 067 (NCT01844505) extended follow-up data to assess the predictive accuracy of standard parametric and flexible models in estimating the long-term overall survival benefit of nivolumab plus ipilimumab (an immune checkpoint inhibitor combination) in advanced melanoma. Methods: Six sets of survival models (standard parametric, piecewise, cubic spline, mixture cure, parametric mixture, and landmark response models) were independently fitted to overall survival data for treatments in CheckMate 067 (nivolumab plus ipilimumab, nivolumab, and ipilimumab) using successive data cuts (28, 40, 52, and 60 mo). Standard parametric models allow survival extrapolation in the absence of a complex hazard. Piecewise and cubic spline models allow additional flexibility in fitting the hazard function. Mixture cure, parametric mixture, and landmark response models provide flexibility by explicitly incorporating survival heterogeneity. Sixty-month follow-up data, external ipilimumab data, and clinical expert opinion were used to evaluate model estimation accuracy. Lifetime survival projections were compared using a 5% discount rate. Results: Standard parametric, piecewise, and cubic spline models underestimated overall survival at 60 mo for the 28-mo data cut. Compared with other models, mixture cure, parametric mixture, and landmark response models provided more accurate long-term overall survival estimates versus external data, higher mean survival benefit over 20 y for the 28-mo data cut, and more consistent 20-y mean overall survival estimates across data cuts. Conclusion: This case study demonstrates that survival models explicitly incorporating survival heterogeneity showed greater accuracy for early data cuts than standard parametric models did, consistent with similar immune checkpoint inhibitor survival validation studies in advanced melanoma. Research is required to assess generalizability to other tumors and disease stages. Highlights: Given that short clinical trial follow-up periods and survival heterogeneity introduce uncertainty in the health technology assessment of oncology therapies, this study evaluated the suitability of conventional parametric survival modeling approaches as compared with more flexible models in the context of immune checkpoint inhibitors that have the potential to provide lasting survival benefits.This study used extended follow-up data from the phase III CheckMate 067 trial (NCT01844505) to assess the predictive accuracy of standard parametric models in comparison with more flexible methods for estimating the long-term survival benefit of the immune checkpoint inhibitor combination of nivolumab plus ipilimumab in advanced melanoma.Mixture cure, parametric mixture, and landmark response models provided more accurate estimates of long-term overall survival versus external data than other models tested.In this case study with immune checkpoint inhibitor therapies in advanced melanoma, extrapolation models that explicitly incorporate differences in cancer survival between observed or latent subgroups showed greater accuracy with both early and later data cuts than other approaches did.

Non-Price-Related Determinants of Value and Access for Novel Non-small Cell Lung Cancer Treatments: A Cross-Country Review of HTA Decision Making.

INTRODUCTION: Access and funding for newly approved treatments for non-small cell lung cancer (NSCLC) are often dependent on Health Technology Assessment (HTA) involving cost-effectiveness analysis. Whilst methods used by HTA agencies share many similarities, final decisions may differ. This may be the result, not just of price considerations, but also of variation in value judgements by different agencies. The aim of this study was to review international HTA evaluations to identify determinants of value and access for NSCLC treatments. METHODS: A targeted review and analysis was undertaken of published HTAs for NSCLC across HTA agencies in six countries (Australia, Canada, England, France, Ireland and Scotland). Analysis of extracted data consisted of three stages: descriptive analysis, bivariate analysis and multivariable analysis. RESULTS: The analysis included 163 HTAs that assessed oncological treatments for NSCLC from 2003 to 2019. The majority of HTA decisions (67.5%) were positive. However, some evidence of heterogeneity in HTA decisions and the factors informing them were identified. The most influential factors included in the multivariate model related to the HTA agency conducting the appraisal, the year of market authorisation, treatment type and the line of treatment. CONCLUSION: Heterogenous decision-making frameworks can present a challenge to developing HTA submissions. This research contributes to understanding decision-making factors and why countries make different decisions.

What Did Time Tell Us? A Comparison and Retrospective Validation of Different Survival Extrapolation Methods for Immuno-Oncologic Therapy in Advanced or Metastatic Renal Cell Carcinoma.

BACKGROUND: The immuno-oncologic (IO) mechanism of action may lead to an overall survival (OS) hazard that changes over time, producing shapes that standard parametric extrapolation methods may struggle to reflect. Furthermore, selection of the most appropriate extrapolation method for health technology assessment is often based on trial data with limited follow-up. OBJECTIVE: To examine this problem, we fitted a range of extrapolation methods to patient-level survival data from CheckMate 025 (NCT01668784, CM-025), a phase III trial comparing nivolumab with everolimus for previously treated advanced renal cell carcinoma (aRCC), to assess their predictive accuracy over time. METHODS: Six extrapolation methods were examined: standard parametric models, natural cubic splines, piecewise models combining Kaplan-Meier data with an exponential or non-exponential distribution, response-based landmark models, and parametric mixture models. We produced three database locks (DBLs) at minimum follow-ups of 15, 27, and 39 months to align with previously published CM-025 data. A three-step evaluation process was adopted: (1) selection of the distribution family for each method in each of the three DBLs, (2) internal validation comparing extrapolation-based landmark and mean survival with the latest CM-025 dataset (minimum follow-up, 64 months), and (3) external validation of survival projections using clinical expert opinion and long-term follow-up data from other nivolumab studies in aRCC (CheckMate 003 and CheckMate 010). RESULTS: All extrapolation methods, with the exception of mixture models, underestimated landmark and mean OS for nivolumab compared with CM-025 long-term follow-up data. OS estimates for everolimus tended to be more accurate, with four of the six methods providing landmark OS estimates within the 95% confidence interval of observed OS as per the latest dataset. The predictive accuracy of survival extrapolation methods fitted to nivolumab also showed greater variation than for everolimus. The proportional hazards assumption held for all DBLs, and a dependent log-logistic model provided reliable estimates of longer-term survival for both nivolumab and everolimus across the DBLs. Although mixture models and response-based landmark models provided reasonable estimates of OS based on the 39-month DBL, this was not the case for the two earlier DBLs. The piecewise exponential models consistently underestimated OS for both nivolumab and everolimus at clinically meaningful pre-specified landmark time points. CONCLUSIONS: This aRCC case study identified marked differences in the predictive accuracy of survival extrapolation methods for nivolumab but less so for everolimus. The dependent log-logistic model did not suffer from overfitting to early DBLs to the same extent as more complex methods. Methods that provide more degrees of freedom may accurately represent survival for IO therapy, particularly if data are more mature or external data are available to inform the long-term extrapolations.

Risk Prediction Using Bayesian Networks: An Immunotherapy Case Study in Patients With Metastatic Renal Cell Carcinoma.

PURPOSE: To address the need for more accurate risk stratification models for cancer immuno-oncology, this study aimed to develop a machine-learned Bayesian network model (BNM) for predicting outcomes in patients with metastatic renal cell carcinoma (mRCC) being treated with immunotherapy. METHODS: Patient-level data from the randomized, phase III CheckMate 025 clinical trial comparing nivolumab with everolimus for second-line treatment in patients with mRCC were used to develop the BNM. Outcomes of interest were overall survival (OS), all-cause adverse events, and treatment-related adverse events (TRAE) over 36 months after treatment initiation. External validation of the model's predictions for OS was conducted using data from select centers from the International Metastatic Renal Cell Carcinoma Database Consortium (IMDC). RESULTS: Areas under the receiver operating characteristic curve (AUCs) for BNM-based classification of OS using baseline data were 0.74, 0.71, and 0.68 over months 12, 24, and 36, respectively. AUC for OS at 12 months increased to 0.86 when treatment response and progression status in year 1 were included as predictors; progression and response at 12 months were highly prognostic of all outcomes over the 36-month period. AUCs for adverse events and treatment-related adverse events were approximately 0.6 at 12 months but increased to approximately 0.7 by 36 months. Sensitivity analysis comparing the BNM with machine learning classifiers showed comparable performance. Test AUC on IMDC data for 12-month OS was 0.71 despite several variable imbalances. Notably, the BNM outperformed the IMDC risk score alone. CONCLUSION: The validated BNM performed well at prediction using baseline data, particularly with the inclusion of response and progression at 12 months. Additionally, the results suggest that 12 months of follow-up data alone may be sufficient to inform long-term survival projections in patients with mRCC.

Current challenges for assessing the long-term clinical benefit of cancer immunotherapy: a multi-stakeholder perspective.

Immuno-oncologics (IOs) differ from chemotherapies as they prime the patient's immune system to attack the tumor, rather than directly destroying cancer cells. The IO mechanism of action leads to durable responses and prolonged survival in some patients. However, providing robust evidence of the long-term benefits of IOs at health technology assessment (HTA) submission presents several challenges for manufacturers. The aim of this article was to identify, analyze, categorize, and further explore the key challenges that regulators, HTA agencies, and payers commonly encounter when assessing the long-term benefits of IO therapies. Insights were obtained from an international, multi-stakeholder steering committee (SC) and expert panels comprising of payers, economists, and clinicians. The selected individuals were tasked with developing a summary of challenges specific to IOs in demonstrating their long-term benefits at HTA submission. The SC and expert panels agreed that standard methods used to assess the long-term benefit of anticancer drugs may have limitations for IO therapies. Three key areas of challenges were identified: (1) lack of a disease model that fully captures the mechanism of action and subsequent patient responses; (2) estimation of longer-term outcomes, including a lack of agreement on ideal methods of survival analyses and extrapolation of survival curves; and (3) data limitations at the time of HTA submission, for which surrogate survival end points and real-world evidence could prove useful. A summary of the key challenges facing manufacturers when submitting evidence at HTA submission was developed, along with further recommendations for manufacturers in what evidence to produce. Despite almost a decade of use, there remain significant challenges around how best to demonstrate the long-term benefit of checkpoint inhibitor-based IOs to HTA agencies, clinicians, and payers. Manufacturers can potentially meet or mitigate these challenges with a focus on strengthening survival analysis methodology. Approaches to doing this include identifying reliable biomarkers, intermediate and surrogate end points, and the use of real-world data to inform and validate long-term survival projections. Wider education across all stakeholders-manufacturers, payers, and clinicians-in considering the long-term survival benefit with IOs is also important.

Challenges in economic modeling of anticancer therapies: an example of modeling the survival benefit of olaparib maintenance therapy for patients with BRCA-mutated platinum-sensitive relapsed ovarian cancer.

OBJECTIVE: The aim of this paper is to describe a four health-state, semi-Markov model structure with health states defined by initiation of subsequent treatment, designed to make best possible use of the data available from a phase 2 clinical trial. METHOD: The approach is illustrated using data from a sub-group of patients enrolled in a phase 2 clinical trial of olaparib maintenance therapy in patients with platinum-sensitive relapsed ovarian cancer and a BRCA mutation (NCT00753545). A semi-Markov model was developed with four health states: progression-free survival (PFS), first subsequent treatment (FST), second subsequent treatment (SST), and death. Transition probabilities were estimated by fitting survival curves to trial data for time from randomization to FST, time from FST to SST, and time from SST to death. RESULTS: Survival projections generated by the model are broadly consistent with the outcomes observed in the clinical trial. However, limitations of the trial data (small sample size, immaturity of the PFS and overall survival [OS] end-points, and treatment switching) create uncertainty in estimates of survival. CONCLUSION: The model framework offers a promising approach to evaluating cost-effectiveness of a maintenance therapy for patients with cancer, which may be generalizable to other chronic diseases.

Estimating health-state utility values for patients with recurrent ovarian cancer using Functional Assessment of Cancer Therapy - General mapping algorithms.

OBJECTIVES: In the absence of EuroQol 5D data, mapping algorithms can be used to predict health-state utility values (HSUVs) for use in economic evaluation. In a placebo-controlled Phase II study of olaparib maintenance therapy (NCT00753545), health-related quality of life was measured using the Functional Assessment of Cancer Therapy - Ovarian (FACT-O) questionnaire. Our objective was to generate HSUVs from the FACT-O data using published mapping algorithms. MATERIALS AND METHODS: Algorithms were identified from a review of the literature. Goodness-of-fit and patient characteristics were compared to select the best-performing algorithm, and this was used to generate base-case HSUVs for the intention-to-treat population of the olaparib study and for patients with breast cancer antigen mutations. RESULTS: Four FACT - General (the core component of FACT-O) mapping algorithms were identified and compared. Under the preferred algorithm, treatment-related adverse events had no statistically significant effect on HSU (P>0.05). Discontinuation of the study treatment and breast cancer antigen mutation status were both associated with a reduction in HSUVs (-0.06, P=0.0009; and -0.03, P=0.0511, respectively). The mean HSUV recorded at assessment visits was 0.786. CONCLUSION: FACT - General mapping generated credible HSUVs for an economic evaluation of olaparib. As reported in other studies, different algorithms may produce significantly different estimates of HSUV. For this reason, it is important to test whether the choice of a specific algorithm changes the conclusions of an economic evaluation.