Comprehensive Review of Methods to Assess Uncertainty in Health Economic Evaluations.

Uncertainty assessment is a cornerstone in model-based health economic evaluations (HEEs) that inform reimbursement decisions. No comprehensive overview of available uncertainty assessment methods currently exists. We aimed to review methods for uncertainty assessment for use in model-based HEEs, by conducting a snowballing review. We categorised all methods according to their stage of use relating to uncertainty assessment (identification, analysis, communication). Additionally, we classified identification methods according to sources of uncertainty, and subdivided analysis and communication methods according to their purpose. The review identified a total of 80 uncertainty methods: 30 identification, 28 analysis, and 22 communication methods. Uncertainty identification methods exist to address uncertainty from different sources. Most identification methods were developed with the objective to assess related concepts such as validity, model quality, and relevance. Almost all uncertainty analysis and communication methods required uncertainty to be quantified and inclusion of uncertainties in probabilistic analysis. Our review can help analysts and decision makers in selecting uncertainty assessment methods according to their aim and purpose of the assessment. We noted a need for further clarification of terminology and guidance on the use of (combinations of) methods to identify uncertainty and related concepts such as validity and quality. A key finding is that uncertainty assessment relies heavily on quantification, which may necessitate increased use of expert elicitation and/or the development of methods to assess unquantified uncertainty.

Headroom Analysis for Early Economic Evaluation: A Systematic Review.

OBJECTIVES: The headroom analysis is an early economic evaluation that quantifies the highest price at which an intervention may still be cost effective. Currently, there is no comprehensive review on how it is applied. This study investigated the application of the headroom analysis, specifically (1) how the headroom analysis is framed (2) the analytical approach and sources of evidence used, and (3) how expert judgement is used and reported. METHODS: A systematic search was conducted in PubMed, Embase, Web of Science, EconLit, and Google Scholar on 28 April 2022. Studies were eligible if they reported an application of the headroom analysis. Data were presented in tabular form and summarised descriptively. RESULTS: We identified 42 relevant papers. The headroom analysis was applied to medicines (29%), diagnostic or screening tests (29%), procedures, programmes and systems (21%), medical devices (19%), and a combined test and device (2%). All studies used model-based analyses, with 40% using simple models and 60% using more comprehensive models. Thirty-three percent of the studies assumed perfect effectiveness of the health technology, while 67% adopted realistic assumptions. Ten percent of the studies calculated an effectiveness-seeking headroom instead of a cost-seeking headroom. Expert judgement was used in 71% of the studies; 23 studies (55%) used expert opinion, 6 studies (14%) used expert elicitation, and 1 study (2%) used both. CONCLUSIONS: Because the application of the headroom analysis varies considerably, we recommend its appropriate use and clear reporting of analytical approaches, level of evidence available, and the use of expert judgement.

Oral Azacitidine for Maintenance Treatment of Acute Myeloid Leukaemia After Induction Therapy: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Celgene) of oral azacitidine (ONUREG), as part of the Single Technology Appraisal (STA) process, to submit evidence for the clinical effectiveness and cost-effectiveness of oral azacitidine for maintenance treatment of acute myeloid leukaemia (AML) after induction therapy compared with watch-and-wait plus best supportive care (BSC) and midostaurin. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review on the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations and describes the development of the NICE guidance by the Appraisal Committee. In the QUAZAR AML-001 trial, oral azacitidine significantly improved overall survival (OS) versus placebo: median OS gain of 9.9 months (24.7 months versus 14.8 months; hazard ratio (HR) 0.69 (95% CI 0.55-0.86), p < 0.001). The median time to relapse was also better for oral azacitidine, and the incidences of TEAEs were similar for the two arms. The company excluded two of the comparators listed in the scope, low-dose cytarabine and subcutaneous azacitidine, informed only by clinical expert opinion, leaving only best supportive care (BSC) and midostaurin for the FLT3-ITD and/or FLT3-TKD (FLT3 mutation)-positive subgroup. An ITC comparing oral azacitidine to midostaurin as maintenance therapy in the appropriate subgroup demonstrated that the OS and relapse-free survival (RFS) HRs were favourable for oral azacitidine when compared with midostaurin. However, in the only available trial of midostaurin as maintenance treatment in AML that was used for this ITC, subjects were not randomised at the maintenance phase, but at induction, which posed a substantial risk of bias. The revised and final probabilistic incremental cost-effectiveness ratio (ICER) presented by the company, including a commercial arrangement, was £32,480 per quality-adjusted life year (QALY) gained for oral azacitidine versus watch-and-wait plus BSC. Oral azacitidine was dominant versus midostaurin in the FLT-3 subgroup. The ERG's concerns included the approach of modelling haematopoietic stem cell transplantation (HSCT), the generalisability of the population and the number of cycles of consolidation therapy pre-treatment in the QUAZAR AML-001 trial to UK clinical practice, and uncertainty in the relapse utility. The revised and final ERG base case resulted in a similar probabilistic ICER of £33,830 per QALY gained versus watch-and-wait plus BSC, but with remaining uncertainty. Oral azacitidine remained dominant versus midostaurin in the FLT-3 subgroup. After the second NICE appraisal committee meeting, the NICE Appraisal Committee recommended oral azacitidine (according to the commercial arrangement), within its marketing authorisation, as an option for maintenance treatment for AML in adults who are in complete remission, or complete remission with incomplete blood count recovery, after induction therapy with or without consolidation treatment, and cannot have or do not want HSCT.

Fenfluramine for Treating Dravet Syndrome: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

Fenfluramine, tradename Fintepla®, was appraised within the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process as Technology Appraisal 808. Within the STA process, the company (Zogenix International) provided NICE with a written submission and a mathematical health economic model, summarising the company's estimates of the clinical effectiveness and cost-effectiveness of fenfluramine for patients with Dravet syndrome (DS). This company submission (CS) was reviewed by an evidence review group (ERG) independent of NICE. The ERG, Kleijnen Systematic Reviews in collaboration with Maastricht University Medical Centre, produced an ERG report. This paper presents a summary of the ERG report and the development of the NICE guidance. The CS included a systematic review of the evidence for fenfluramine. From this review the company identified and presented evidence from two randomised trials (Study 1 and Study 1504), an open-label extension study (Study 1503) and 'real world evidence' from a prospective and retrospective study. Both randomised trials were conducted in patients up to 18 years of age with DS, whose seizures were incompletely controlled with previous anti-epileptic drugs. A Bayesian network meta-analysis was performed to compare fenfluramine with cannabidiol plus clobazam. There was no evidence of a difference between any doses of fenfluramine and cannabidiol in the mean convulsive seizure frequency (CSF) rate during treatment. However, fenfluramine increased the number of patients achieving ≥ 50% reduction in CSF frequency from baseline compared to cannabidiol. The company used an individual-patient state-transition model (R version 3.5.2) to model cost-effectiveness of fenfluramine. The CSF and convulsive seizure-free days were estimated using patient-level data from the placebo arm of the fenfluramine registration studies. Subsequently, a treatment effect of either fenfluramine or cannabidiol was applied. Utility values for the economic model were obtained by mapping Pediatric Quality of Life Inventory data from the registration studies to EuroQol-5D-3L Youth (EQ-5D-Y-3L). The company included caregiver utilities in their base-case, as the severe needs of patients with DS have a major impact on parents and caregivers. There were several key issues. First, the company included caregiver utilities in the model in a way that when patients in the economic model died, the corresponding caregiver utility was also set to zero. Second, the model was built in R statistical software, resulting in transparency issues. Third, the company assumed the same percentage reduction for convulsive seizure days as was estimated for CSF. Fourth, during the final appraisal committee meeting, influential changes were made to the model that were not in line with the ERG's preferences (but were accepted by the appraisal committee). The company's revised and final incremental cost effectiveness ratio (ICER) in line with committee preferences resulted in fenfluramine dominating cannabidiol. Fenfluramine was recommended as an add-on to other antiepileptic medicines for treating seizures associated with DS in people aged 2 years and older in the National Health Service (NHS).

Finding the way forward: COVID-19 vaccination progress in Germany, Austria and Switzerland.

Objectives: : This paper presents an overview of the procurement and deployment of COVID-19 vaccinations in Germany, Austria and Switzerland (DACH) from the success of the first vaccine trials until the end of August 2021. Data regarding vaccination procurement and deployment is presented, followed by an analysis of the challenges these countries face in improving their vaccination rates. Methods: : A review and analysis of available data from the DACH countries was conducted. Data was collected from official government sources whenever possible and supplemented by information from international databases and local reports. The data was analyzed to identify common patterns as well as divergences across the DACH region, especially as they relate to vaccine hesitancy and health policy. Results: : Following initial global supply problems, the DACH countries were largely successful at administering vaccinations to their populations. However, by the end of August 2021 their vaccination progress had plateaued. This was primarily due to vaccine hesitancy in the region, which is correlated with a multitude of complex factors. These factors need to be better understood before this issue can be effectively addressed. Unlike other countries, the DACH countries have not (yet) used financial incentives or mandates to increase vaccination rates. Conclusions: : The DACH countries displayed effective governance in their deployment of COVID-19 vaccines, but vaccine hesitancy is slowing progress. Due to various social and political factors, Germany, Austria and Switzerland have not been able to employ effective policy levers to overcome this barrier and a more nuanced strategy will have to be developed. Public Interest Summary: : Germany, Austria and Switzerland (DACH) were largely successful at procuring COVID-19 vaccine doses and administering them to their populations. After the first doses were acquired, their vaccination rates continued to steadily rise, but progress began to slow down substantially by August 2021 due in part to vaccine hesitancy. Unlike in other countries, the DACH governments have not been able to implement vaccine mandates to try and overcome this issue due to their specific political and social circumstances. A deeper understanding of the factors driving vaccine hesitancy in the region will be required before effective solutions can be found.

Belimumab for Treating Active Autoantibody-Positive Systemic Lupus Erythematosus: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

As part of its Single Technology Appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited the manufacturer (GlaxoSmithKline [GSK]) of Benlysta (belimumab) to submit evidence regarding its clinical and cost effectiveness, for the review and possible extension of a previously conditionally approved intravenous formulation of belimumab for the treatment of active autoantibody-positive systemic lupus erythematosus (SLE). Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the NICE Appraisal Committee.This appraisal is different to the previous appraisal in three ways: (1). This appraisal expands its definition of 'high disease activity'. (2). In TA397, belimumab was approved, with a managed access arrangement (MAA), for adults only. This appraisal includes subjects aged 5 years or older. (3). The original appraisal included an intravenous formulation only, but the current appraisal also includes a new subcutaneous formulation in the form of a prefilled pen.The company was required to collect real-world data from the British Isles Lupus Assessment Group Biologics Register (BILAG-BR), including data on the efficacy, safety, and effect on health-related quality of life of belimumab versus rituximab. This appraisal considers these data as well as additional clinical trial evidence presented in the company's updated submission to address uncertainties identified during the original appraisal. The ERG identified three major concerns with the evidence presented on the clinical effectiveness in the current submission; namely, short follow-up in the main comparative trials (BLISS-SC, BLISS-52 and BLISS-76); using the propensity score-matching (PSM) analysis in calibrating the cost-effectiveness model can severely bias the results in favour of belimumab; and BILAG-BR data are not suitable for a comparison of belimumab with rituximab.The main issue in the economic analysis was the uncertainty about long-term disease activity progression and resulting organ damage. The company's approach of calibrating modelled organ damage to longer-term data analysed using the PSM analysis was methodologically inappropriate. The final analysis comparing belimumab with standard treatment for the intravenous formulation resulted in an incremental cost-effectiveness ratio of £12,335 per quality-adjusted life-year (QALY) gained and £30,278 per QALY gained in the company's and ERG's base-case analyses, respectively. For the subcutaneous formulation, the final analysis resulted in £8480 per QALY gained and £29,313 per QALY gained in the company's and ERG's base-case analyses, respectively. NICE recommended belimumab in both intravenous and subcutaneous formulations as an add-on treatment option for active autoantibody-positive SLE in the HDA-2 subgroup.

State of the ART? Two New Tools for Risk Communication in Health Technology Assessments.

PURPOSE: Outcomes of health technology assessments (HTA) are uncertain, and decision-making is associated with a risk. This risk, consisting of the probability of making a wrong decision and its impact, is rarely considered in HTA. This hampers transparent and consistent risk assessment and management. The aim of this study was to develop risk communication tools in the context of health technology decision-making under uncertainty. METHODS: We performed a scoping review of tools for uncertainty and risk communication within HTA using citation pearl-growing. We developed two tools, drawing on existing publications on risk and uncertainty communication for inspiration. Individual semi-structured interviews with HTA stakeholders were performed to identify potential improvements in usefulness, user-friendliness, and information adequacy. Tools were amended and further evaluated in a real-world HTA and workshop with HTA stakeholders. RESULTS: The identified risk communication tools did not include non-quantified uncertainties, and did not link to risk management strategies. We developed two tools: the Assessment of Risk Table (ART), for a summary of quantified and non-quantified uncertainties and the resulting risk assessment, and the Appraisal of Risk Chart (ARCH), for linking net benefit and risk outcomes to appropriate risk management strategies. Stakeholders appreciated the usefulness of the tools. They also highlighted that more information on local policy options was required for optimal risk management use, and HTA processes may need adapting. CONCLUSION: The risk communication tools presented here can help assess risk, facilitate communication between analysts and decision-makers, and guide the appropriate use of available risk management strategies.

COVID-19 pandemic in the United States.

OBJECTIVES: The paper highlights US health policy and technology responses to the COVID-19 pandemic from January 1, 2020 - August 9, 2020. METHODS: A review of primary data sources in the US was conducted. The data were summarized to describe national and state-level trends in the spread of COVID-19 and in policy and technology solutions. RESULTS: COVID-19 cases and deaths initially peaked in late March and April, but after a brief reduction in June cases and deaths began rising again during July and continued to climb into early August. The US policy response is best characterized by its federalist, decentralized nature. The national government has led in terms of economic and fiscal response, increasing funding for scientific research into testing, treatment, and vaccines, and in creating more favorable regulations for the use of telemedicine. State governments have been responsible for many of the containment, testing, and treatment responses, often with little federal government support. Policies that favor economic re-opening are often followed by increases in state-level case numbers, which are then followed by stricter containment measures, such as mask wearing or pausing re-opening plans. CONCLUSIONS: While all US states have begun to "re-open" economic activities, this trend appears to be largely driven by social tensions and economic motivations rather than an ability to effectively test and surveil populations.