An Evaluation of an Algorithm for the Selection of Flexible Survival Models for Cancer Immunotherapies: Pass or Fail?

BACKGROUND AND OBJECTIVE: Accurately extrapolating survival beyond trial follow-up is essential in a health technology assessment where model choice often substantially impacts estimates of clinical and cost effectiveness. Evidence suggests standard parametric models often provide poor fits to long-term data from immuno-oncology trials. Palmer et al. developed an algorithm to aid the selection of more flexible survival models for these interventions. We assess the usability of the algorithm, identify areas for improvement and evaluate whether it effectively identifies models capable of accurate extrapolation. METHODS: We applied the Palmer algorithm to the CheckMate-649 trial, which investigated nivolumab plus chemotherapy versus chemotherapy alone in patients with gastroesophageal adenocarcinoma. We evaluated the algorithm's performance by comparing survival estimates from identified models using the 12-month data cut to survival observed in the 48-month data cut. RESULTS: The Palmer algorithm offers a systematic procedure for model selection, encouraging detailed analyses and ensuring that crucial stages in the selection process are not overlooked. In our study, a range of models were identified as potentially appropriate for extrapolating survival, but only flexible parametric non-mixture cure models provided extrapolations that were plausible and accurately predicted subsequently observed survival. The algorithm could be improved with minor additions around the specification of hazard plots and setting out plausibility criteria. CONCLUSIONS: The Palmer algorithm provides a systematic framework for identifying suitable survival models, and for defining plausibility criteria for extrapolation validity. Using the algorithm ensures that model selection is based on explicit justification and evidence, which could reduce discordance in health technology appraisals.

MPES-R: Multi-Parameter Evidence Synthesis in R for Survival Extrapolation-A Tutorial.

Survival extrapolation often plays an important role in health technology assessment (HTA), and there are a range of different approaches available. Approaches that can leverage external evidence (i.e. data or information collected outside the main data source of interest) may be helpful, given the extent of uncertainty often present when determining a suitable survival extrapolation. One of these methods is the multi-parameter evidence synthesis (MPES) approach, first proposed for use in HTA by Guyot et al., and more recently by Jackson. While MPES has potential benefits over conventional extrapolation approaches (such as simple or flexible parametric models), it is more computationally complex and requires use of specialist software. This tutorial presents an introduction to MPES for HTA, alongside a user-friendly, publicly available operationalisation of Guyot's original MPES that can be executed using the statistical software package R. Through two case studies, both Guyot's and Jackson's MPES approaches are explored, along with sensitivity analyses relevant to HTA. Finally, the discussion section of the tutorial details important considerations for analysts considering use of an MPES approach, along with potential further developments. MPES has not been used often in HTA, and so there are limited examples of how it has been used and perceived. However, this tutorial may aid future research efforts exploring the use of MPES further.

Treatment Effect Waning in Immuno-oncology Health Technology Assessments: A Review of Assumptions and Supporting Evidence with Proposals to Guide Modelling.

Treatment effect waning (TEW) refers to the attenuation of treatment effects over time. Assumptions of a sustained immuno-oncologic treatment effect have been a source of contention in health technology assessment (HTA). We review how TEW has been addressed in HTA and in the wider scientific literature. We analysed company submissions to English language HTA agencies and summarised methods and assumptions used. We subsequently reviewed TEW-related work in the ISPOR Scientific Presentations Database and conducted a targeted literature review (TLR) for evidence of the maintenance of immuno-oncology (IO) treatment effects post-treatment discontinuation. We found no standardised approach adopted by companies in submissions to HTA agencies, with immediate TEW most used in scenario analyses. Independently fitted survival models do however suggest TEW may often be implicitly modelled. Materials in the ISPOR scientific database suggest gradual TEW is more plausible than immediate TEW. The TLR uncovered evidence of durable survival in patients treated with IOs but no evidence that directly addresses the presence or absence of TEW. Our HTA review shows the need for a consistent and appropriate implementation of TEW in oncology appraisals. However, the TLR highlights the absence of direct evidence on TEW in literature, as TEW is defined in terms of relative treatment effects-not absolute survival. We propose a sequence of steps for analysts to use when assessing whether a TEW scenario is necessary and appropriate to present in appraisals of IOs.

Mixture and Non-mixture Cure Models for Health Technology Assessment: What You Need to Know.

There is increasing interest in the use of cure modelling to inform health technology assessment (HTA) due to the development of new treatments that appear to offer the potential for cure in some patients. However, cure models are often not included in evidence dossiers submitted to HTA agencies, and they are relatively rarely relied upon to inform decision-making. This is likely due to a lack of understanding of how cure models work, what they assume, and how reliable they are. In this tutorial we explain why and when cure models may be useful for HTA, describe the key characteristics of mixture and non-mixture cure models, and demonstrate their use in a range of scenarios, providing Stata code. We highlight key issues that must be taken into account by analysts when fitting these models and by reviewers and decision-makers when interpreting their predictions. In particular, we note that flexible parametric non-mixture cure models have not been used in HTA, but they offer advantages that make them well suited to an HTA context when a cure assumption is valid but follow-up is limited.

Challenges and Opportunities in Interdisciplinary Research and Real-World Data for Treatment Sequences in Health Technology Assessments.

With an ever-increasing number of treatment options, the assessment of treatment sequences has become crucial in health technology assessment (HTA). This review systematically explores the multifaceted challenges inherent in evaluating sequences, delving into their interplay and nuances that go beyond economic model structures. We synthesised a 'roadmap' of literature from key methodological studies, highlighting the evolution of recent advances and emerging research themes. These insights were compared against HTA guidelines to identify potential avenues for future research. Our findings reveal a spectrum of challenges in sequence evaluation, encompassing selecting appropriate decision-analytic modelling approaches and comparators, deriving appropriate clinical effectiveness evidence in the face of data scarcity, scrutinising effectiveness assumptions and statistical adjustments, considering treatment displacement, and optimising model computations. Integrating methodologies from diverse disciplines-statistics, epidemiology, causal inference, operational research and computer science-has demonstrated promise in addressing these challenges. An updated review of application studies is warranted to provide detailed insights into the extent and manner in which these methodologies have been implemented. Data scarcity on the effectiveness of treatment sequences emerged as a dominant concern, especially because treatment sequences are rarely compared in clinical trials. Real-world data (RWD) provide an alternative means for capturing evidence on effectiveness and future research should prioritise harnessing causal inference methods, particularly Target Trial Emulation, to evaluate treatment sequence effectiveness using RWD. This approach is also adaptable for analysing trials harbouring sequencing information and adjusting indirect comparisons when collating evidence from heterogeneous sources. Such investigative efforts could lend support to reviews of HTA recommendations and contribute to synthesising external control arms involving treatment sequences.

The feasibility and acceptability of an app-based intervention with brief behavioural support (APPROACH) to promote brisk walking in people diagnosed with breast, prostate and colorectal cancer in the UK.

INTRODUCTION: Increased moderate to vigorous physical activity (MVPA) can improve clinical and psychosocial outcomes for people living with and beyond cancer (LWBC). This study aimed to assess the feasibility and acceptability of trial procedures in a pilot randomised controlled trial (RCT) of a theory-driven app-based intervention with behavioural support focused on promoting brisk walking (a form of MVPA) in people LWBC (APPROACH). METHODS: Participants diagnosed with breast, prostate or colorectal cancer were recruited from a single UK hospital site. Assessments at baseline and 3 months included online questionnaires, device-measured brisk walking (activPAL accelerometer) and self-reported weight and height. Participants were randomised to intervention or control (care as usual). The intervention comprised a non-cancer-specific app to promote brisk walking (National Health Service 'Active 10') augmented with print information about habit formation, a walking planner and two behavioural support telephone calls. Feasibility and acceptability of trial procedures were explored. Initial estimates for physical activity informed a power calculation for a phase III RCT. A preliminary health economics analysis was conducted. RESULTS: Of those medically eligible, 369/577 (64%) were willing to answer further eligibility questions and 90/148 (61%) of those eligible were enrolled. Feasibility outcomes, including retention (97%), assessment completion rates (>86%) and app download rates in the intervention group (96%), suggest that the trial procedures are acceptable and that the intervention is feasible. The phase III RCT will require 472 participants to be randomised. As expected, the preliminary health economic analyses indicate a high level of uncertainty around the cost-effectiveness of the intervention. CONCLUSIONS: This pilot study demonstrates that a large trial of the brisk walking intervention with behavioural support is both feasible and acceptable to people LWBC. The results support progression onto a confirmatory phase III trial to determine the efficacy and cost-effectiveness of the intervention.

A cautionary tale: an evaluation of the performance of treatment switching adjustment methods in a real world case study.

BACKGROUND: Treatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82-1.22) for all patients, and 0.99 (95% CI 0.75-1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus). METHODS: We tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76-0.77 (95% CI 0.60-0.98) for all patients, and 0.55-0.73 (95% CI 0.41-0.93) for patients with wild-type KRAS. RESULTS: Some applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals. CONCLUSIONS: Adjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability.

Evaluation of Flexible Parametric Relative Survival Approaches for Enforcing Long-Term Constraints When Extrapolating All-Cause Survival.

OBJECTIVES: Parametric models are used to estimate the lifetime benefit of an intervention beyond the range of trial follow-up. Recent recommendations have suggested more flexible survival approaches and the use of external data when extrapolating. Both of these can be realized by using flexible parametric relative survival modeling. The overall aim of this article is to introduce and contrast various approaches for applying constraints on the long-term disease-related (excess) mortality including cure models and evaluate the consequent implications for extrapolation. METHODS: We describe flexible parametric relative survival modeling approaches. We then introduce various options for constraining the long-term excess mortality and compare the performance of each method in simulated data. These methods include fitting a standard flexible parametric relative survival model, enforcing statistical cure, and forcing the long-term excess mortality to converge to a constant. We simulate various scenarios, including where statistical cure is reasonable and where the long-term excess mortality persists. RESULTS: The compared approaches showed similar survival fits within the follow-up period. However, when extrapolating the all-cause survival beyond trial follow-up, there is variation depending on the assumption made about the long-term excess mortality. Altering the time point from which the excess mortality is constrained enables further flexibility. CONCLUSIONS: The various constraints can lead to applying explicit assumptions when extrapolating, which could lead to more plausible survival extrapolations. The inclusion of general population mortality directly into the model-building process, which is possible for all considered approaches, should be adopted more widely in survival extrapolation in health technology assessment.

Modeling the multi-state natural history of rare diseases with heterogeneous individual patient data: A simulation study.

Multi-state survival models are used to represent the natural history of a disease, forming the basis of a health technology assessment comparing a novel treatment to current practice. Constructing such models for rare diseases is problematic, since evidence sources are typically much sparser and more heterogeneous. This simulation study investigated different one-stage and two-stage approaches to meta-analyzing individual patient data (IPD) in a multi-state survival setting when the number and size of studies being meta-analyzed are small. The objective was to assess methods of different complexity to see when they are accurate, when they are inaccurate and when they struggle to converge due to the sparsity of data. Biologically plausible multi-state IPD were simulated from study- and transition-specific hazard functions. One-stage frailty and two-stage stratified models were estimated, and compared to a base case model that did not account for study heterogeneity. Convergence and the bias/coverage of population-level transition probabilities to, and lengths of stay in, each state were used to assess model performance. A real-world application to Duchenne Muscular Dystrophy, a neuromuscular rare disease, was conducted, and a software demonstration is provided. Models not accounting for study heterogeneity were consistently out-performed by two-stage models. Frailty models struggled to converge, particularly in scenarios of low heterogeneity, and predictions from models that did converge were also subject to bias. Stratified models may be better suited to meta-analyzing disparate sources of IPD in rare disease natural history/economic modeling, as they converge more consistently and produce less biased predictions of lengths of stay.

Perils of Randomized Controlled Trial Survival Extrapolation Assuming Treatment Effect Waning: Why the Distinction Between Marginal and Conditional Estimates Matters.

OBJECTIVES: A long-term, constant, protective treatment effect is a strong assumption when extrapolating survival beyond clinical trial follow-up; hence, sensitivity to treatment effect waning is commonly assessed for economic evaluations. Forcing a hazard ratio (HR) to 1 does not necessarily estimate loss of individual-level treatment effect accurately because of HR selection bias. A simulation study was designed to explore the behavior of marginal HRs under a waning conditional (individual-level) treatment effect and demonstrate bias in forcing a marginal HR to 1 when the estimand is "survival difference with individual-level waning". METHODS: Data were simulated under 4 parameter combinations (varying prognostic strength of heterogeneity and treatment effect). Time-varying marginal HRs were estimated in scenarios where the true conditional HR attenuated to 1. Restricted mean survival time differences, estimated having constrained the marginal HR to 1, were compared with true values to assess bias induced by marginal constraints. RESULTS: Under loss of conditional treatment effect, the marginal HR took a value >1 because of covariate imbalances. Constraining this value to 1 lead to restricted mean survival time difference bias of up to 0.8 years (57% increase). Inflation of effect size estimates also increased with the magnitude of initial protective treatment effect. CONCLUSIONS: Important differences exist between survival extrapolations assuming marginal versus conditional treatment effect waning. When a marginal HR is constrained to 1 to assess efficacy under individual-level treatment effect waning, the survival benefits associated with the new treatment will be overestimated, and incremental cost-effectiveness ratios will be underestimated.

Survival Extrapolation Incorporating General Population Mortality Using Excess Hazard and Cure Models: A Tutorial.

BACKGROUND: Different parametric survival models can lead to widely discordant extrapolations and decision uncertainty in cost-effectiveness analyses. The use of excess hazard (EH) methods, which incorporate general population mortality data, has the potential to reduce model uncertainty. This review highlights key practical considerations of EH methods for estimating long-term survival. METHODS: Demonstration of methods used a case study of 686 patients from the German Breast Cancer Study Group, followed for a maximum of 7.3 y and divided into low (1/2) and high (3) grade cancers. Seven standard parametric survival models were fit to each group separately. The same 7 distributions were then used in an EH framework, which incorporated general population mortality rates, and fitted both with and without a cure parameter. Survival extrapolations, restricted mean survival time (RMST), and difference in RMST between high and low grades were compared up to 30 years along with Akaike information criterion goodness-of-fit and cure fraction estimates. The sensitivity of the EH models to lifetable misspecification was investigated. RESULTS: In our case study, variability in survival extrapolations was extensive across the standard models, with 30-y RMST ranging from 7.5 to 14.3 y. Incorporation of general population mortality rates using EH cure methods substantially reduced model uncertainty, whereas EH models without cure had less of an effect. Long-term treatment effects approached the null for most models but at varying rates. Lifetable misspecification had minimal effect on RMST differences. CONCLUSIONS: EH methods may be useful for survival extrapolation, and in cancer, EHs may decrease over time and be easier to extrapolate than all-cause hazards. EH cure models may be helpful when cure is plausible and likely to result in less extrapolation variability. HIGHLIGHTS: In health economic modeling, to help anchor long-term survival extrapolation, it has been recommended that survival models incorporate background mortality rates using excess hazard (EH) methods.We present a thorough description of EH methods with and without the assumption of cure and demonstrate user-friendly software to aid researchers wishing to use these methods.EH models are applied to a case study, and we demonstrate that EHs are easier to extrapolate and that the use of the EH cure model, when cure is plausible, can reduce extrapolation variability.EH methods are relatively robust to lifetable misspecification.

A Systematic Review of Methods to Incorporate External Evidence into Trial-Based Survival Extrapolations for Health Technology Assessment.

BACKGROUND: External evidence is commonly used to inform survival modeling for health technology assessment (HTA). While there are a range of methodological approaches that have been proposed, it is unclear which methods could be used and how they compare. PURPOSE: This review aims to identify, describe, and categorize established methods to incorporate external evidence into survival extrapolation for HTA. DATA SOURCES: Embase, MEDLINE, EconLit, and Web of Science databases were searched to identify published methodological studies, supplemented by hand searching and citation tracking. STUDY SELECTION: Eligible studies were required to present a novel extrapolation approach incorporating external evidence (i.e., data or information) within survival model estimation. DATA EXTRACTION: Studies were classified according to how the external evidence was integrated as a part of model fitting. Information was extracted concerning the model-fitting process, key requirements, assumptions, software, application contexts, and presentation of comparisons with, or validation against, other methods. DATA SYNTHESIS: Across 18 methods identified from 22 studies, themes included use of informative prior(s) (n = 5), piecewise (n = 7), and general population adjustment (n = 9), plus a variety of "other" (n = 8) approaches. Most methods were applied in cancer populations (n = 13). No studies compared or validated their method against another method that also incorporated external evidence. LIMITATIONS: As only studies with a specific methodological objective were included, methods proposed as part of another study type (e.g., an economic evaluation) were excluded from this review. CONCLUSIONS: Several methods were identified in this review, with common themes based on typical data sources and analytical approaches. Of note, no evidence was found comparing the identified methods to one another, and so an assessment of different methods would be a useful area for further research.HighlightsThis review aims to identify methods that have been used to incorporate external evidence into survival extrapolations, focusing on those that may be used to inform health technology assessment.We found a range of different approaches, including piecewise methods, Bayesian methods using informative priors, and general population adjustment methods, as well as a variety of "other" approaches.No studies attempted to compare the performance of alternative methods for incorporating external evidence with respect to the accuracy of survival predictions. Further research investigating this would be valuable.

Is Intention to Treat Still the Gold Standard or Should Health Technology Assessment Agencies Embrace a Broader Estimands Framework?: Insights and Perspectives From the National Institute for Health and Care Excellence and Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen on the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use E9 (R1) Addendum.

OBJECTIVES: The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E9 (R1) addendum will have an important impact on the design and analysis of randomized controlled clinical trials, which represent crucial sources of evidence in health technology assessments, and on the intention-to-treat (ITT) principle in particular. This article brings together a task force of health economists and statisticians in academic institutes and the pharmaceutical industry, to examine the implications of the addendum from the perspective of the National Institute for Health and Care Excellence (NICE) and the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) and to address the question of whether the ITT principle should be considered the gold standard for estimating treatment effects. METHODS: We review the ITT principle, as introduced in the ICH E9 guideline. We then present an overview of the ICH E9 (R1) addendum and its estimand framework, highlighting its premise and the proposed strategies for handling intercurrent events, and examine some cases among submissions to IQWiG and NICE. RESULTS: IQWiG and NICE appear to have diverging perspectives around the relevance of the ITT principle and, in particular, the acceptance of hypothetical strategies for estimating treatment effects, as suggested by examples where the sponsor proposed an alternative approach to the ITT principle when accounting for treatment switching for interventional oncology trials. CONCLUSIONS: The ICH E9 (R1) addendum supports the use of methods that depart from the ITT principle. The relevance of estimands using these methods depends on the perspectives and objectives of payers. It is challenging to design a study that meets all stakeholders' research questions. Different estimands may serve to answer different relevant questions or decision problems.

Propensity Score and Instrumental Variable Techniques in Observational Transplantation Studies: An Overview and Worked Example Relating to Pre-Transplant Cardiac Screening.

Inferring causality from observational studies is difficult due to inherent differences in patient characteristics between treated and untreated groups. The randomised controlled trial is the gold standard study design as the random allocation of individuals to treatment and control arms should result in an equal distribution of known and unknown prognostic factors at baseline. However, it is not always ethically or practically possible to perform such a study in the field of transplantation. Propensity score and instrumental variable techniques have theoretical advantages over conventional multivariable regression methods and are increasingly being used within observational studies to reduce the risk of confounding bias. An understanding of these techniques is required to critically appraise the literature. We provide an overview of propensity score and instrumental variable techniques for transplant clinicians, describing their principles, assumptions, strengths, and weaknesses. We discuss the different patient populations included in analyses and how to interpret results. We illustrate these points using data from the Access to Transplant and Transplant Outcome Measures study examining the association between pre-transplant cardiac screening in kidney transplant recipients and post-transplant cardiac events.

Challenges in valuing and paying for combination regimens in oncology: reporting the perspectives of a multi-stakeholder, international workshop.

BACKGROUND: It is increasingly common for two or more treatments for cancer to be combined as a single regimen. Determining value and appropriate payment for such regimens can be challenging. This study discusses these challenges, and possible solutions. METHODS: Stakeholders from around the world attended a 2-day workshop, supported by a background paper. This study captures key outcomes from the discussion, but is not a consensus statement. RESULTS: Workshop attendees agreed that combining on-patent treatments can result in affordability and value for money challenges that delay or deny patient access to clinically effective treatments in many health systems. Options for addressing these challenges include: (i) Increasing the value of combination therapies through improved clinical development; (ii) Willingness to pay more for combinations than for single drugs offering similar benefit, or; (iii) Aligning the cost of constituent therapies with their value within a regimen. Workshop attendees felt that (i) and (iii) merited further discussion, whereas (ii) was unlikely to be justifiable. Views differed on the feasibility of (i). Key to (iii) would be systems allowing different prices to apply to different uses of a drug. CONCLUSIONS: Common ground was identified on immediate actions to improve access to combination regimens. These include an exploration of the legal challenges associated with price negotiations, and ensuring that pricing systems can support implementation of negotiated prices for specific uses. Improvements to clinical development and trial design should be pursued in the medium and longer term.

Estimation of Transition Probabilities for State-Transition Models: A Review of NICE Appraisals.

State transition models are used to inform health technology reimbursement decisions. Within state transition models, the movement of patients between the model health states over discrete time intervals is determined by transition probabilities (TPs). Estimating TPs presents numerous issues, including missing data for specific transitions, data incongruence and uncertainty around extrapolation. Inappropriately estimated TPs could result in biased models. There is limited guidance on how to address common issues associated with TP estimation. To assess current methods for estimating TPs and to identify issues that may introduce bias, we reviewed National Institute for Health and Care Excellence Technology Appraisals published from 1 January, 2019 to 27 May, 2020. Twenty-eight models (from 26 Technology Appraisals) were included in the review. Several methods for estimating TPs were identified: survival analysis (n = 11); count method (n = 9); multi-state modelling (n = 7); logistic regression (n = 2); negative binomial regression (n = 2); Poisson regression (n = 1); and calibration (n = 1). Evidence Review Groups identified several issues relating to TP estimation within these models, including important transitions being excluded (n = 5); potential selection bias when estimating TPs for post-randomisation health states (n = 2); issues concerning the use of multiple data sources (n = 4); potential biases resulting from the use of data from different populations (n = 2), and inappropriate assumptions around extrapolation (n = 3). These issues remained unresolved in almost every instance. Failing to address these issues may bias model results and lead to sub-optimal decision making. Further research is recommended to address these methodological problems.

Prevalence of Immature Survival Data for Anti-Cancer Drugs Presented to the National Institute for Health and Care Excellence and Impact on Decision Making.

OBJECTIVES: This research aims to explore how often the National Institute for Health and Care Excellence (NICE) uses immature overall survival data to inform reimbursement decisions on cancer treatments, and the implications of this for resource allocation decisions. METHODS: NICE cancer technology appraisals published between 2015 and 2017 were reviewed to determine the prevalence of using immature survival data. A case study was used to demonstrate the potential impact of basing decisions on immature data. The economic model submitted by the company was reconstructed and was populated first using survival data available at the time of the appraisal, and then using data from an updated data cut published after the appraisal concluded. The incremental cost-effectiveness ratios (ICERs) obtained using the different data cuts were compared. Probabilistic sensitivity analysis was undertaken and expected value of perfect information estimated. RESULTS: Forty-one percent of NICE cancer technology appraisals used immature data to inform reimbursement decisions. In the case study, NICE gave a positive recommendation for a limited patient subgroup, with ICERs too high in the complete patient population. ICERs were dramatically lower when the final data cut was used, irrespective of the parametric model used to model survival. Probabilistic sensitivity analysis and expected value of perfect information may not have fully characterized uncertainty, because as they did not account for structural uncertainty. CONCLUSION: Analyses of cancer treatments using immature survival data may result in incorrect estimates of survival benefit and cost-effectiveness, potentially leading to inappropriate funding decisions. This research highlights the importance of revisiting past decisions when updated data cuts become available.