Disease severity-based subgrouping of type 2 diabetes does not parallel differences in quality of life: the Maastricht Study.

AIMS/HYPOTHESIS: Type 2 diabetes is a highly heterogeneous disease for which new subgroups ('clusters') have been proposed based on disease severity: moderate age-related diabetes (MARD), moderate obesity-related diabetes (MOD), severe insulin-deficient diabetes (SIDD) and severe insulin-resistant diabetes (SIRD). It is unknown how disease severity is reflected in terms of quality of life in these clusters. Therefore, we aimed to investigate the cluster characteristics and cluster-wise evolution of quality of life in the previously defined clusters of type 2 diabetes. METHODS: We included individuals with type 2 diabetes from the Maastricht Study, who were allocated to clusters based on a nearest centroid approach. We used logistic regression to evaluate the cluster-wise association with diabetes-related complications. We plotted the evolution of HbA1c levels over time and used Kaplan-Meier curves and Cox regression to evaluate the cluster-wise time to reach adequate glycaemic control. Quality of life based on the Short Form 36 (SF-36) was also plotted over time and adjusted for age and sex using generalised estimating equations. The follow-up time was 7 years. Analyses were performed separately for people with newly diagnosed and already diagnosed type 2 diabetes. RESULTS: We included 127 newly diagnosed and 585 already diagnosed individuals. Already diagnosed people in the SIDD cluster were less likely to reach glycaemic control than people in the other clusters, with an HR compared with MARD of 0.31 (95% CI 0.22, 0.43). There were few differences in the mental component score of the SF-36 in both newly and already diagnosed individuals. In both groups, the MARD cluster had a higher physical component score of the SF-36 than the other clusters, and the MOD cluster scored similarly to the SIDD and SIRD clusters. CONCLUSIONS/INTERPRETATION: Disease severity suggested by the clusters of type 2 diabetes is not entirely reflected in quality of life. In particular, the MOD cluster does not appear to be moderate in terms of quality of life. Use of the suggested cluster names in practice should be carefully considered, as the non-neutral nomenclature may affect disease perception in individuals with type 2 diabetes and their healthcare providers.

Uncertainty in Long-Term Relative Effectiveness of Medicines in Health Technology Assessment.

OBJECTIVES: Uncertainty regarding the long-term relative effectiveness is an important factor in health technology assessment (HTA) of medicines. This study investigated how different HTA bodies address this uncertainty in their assessments. METHODS: A total of 49 HTA reports from 6 national HTA bodies, assessing 9 medicines for spinal muscular atrophy, cystic fibrosis, and hypercholesterolemia, were included. In these reports, 81 relative effectiveness assessments and 45 cost-effectiveness assessments were performed on an indication level. We collected information on included trials, assessment outcomes, uncertainty regarding the long-term effectiveness, proposed managed entry agreements, and reassessments. RESULTS: Uncertainty regarding the long-term effectiveness was an important consideration in almost all cost-effectiveness assessments (91%) and three-quarters of relative effectiveness assessments (74%), despite differences in methodologies among HTA bodies. There were considerable differences in the amount and type of long-term effectiveness data included by HTA bodies due to timing and inclusion criteria. In total 23 managed entry agreements were proposed of which 14 were linked to uncertainty regarding the long-term effectiveness. In addition, 13 reassessments were performed of which 4 led to an increase in patient access because of more available long-term effectiveness data. CONCLUSIONS: Uncertainty regarding the long-term effectiveness is an important challenge for HTA bodies. There are large differences in the acceptance of evidence among HTA bodies, which leads to heterogeneity in the inclusion of available long-term effectiveness data for decision making. In cases with large uncertainty regarding the long-term effectiveness, outcome-based agreements and reassessments are used by HTA bodies, but differently between HTA bodies and indications.

Analytical Methods for Comparing Uncontrolled Trials With External Controls From Real-World Data: A Systematic Literature Review and Comparison With European Regulatory and Health Technology Assessment Practice.

OBJECTIVES: This study aimed to provide an overview of analytical methods in scientific literature for comparing uncontrolled medicine trials with external controls from individual patient data real-world data (IPD-RWD) and to compare these methods with recommendations made in guidelines from European regulatory and health technology assessment (HTA) organizations and with their evaluations described in assessment reports. METHODS: A systematic literature review (until March 1, 2023) in PubMed and Connected Papers was performed to identify analytical methods for comparing uncontrolled trials with external controls from IPD-RWD. These methods were compared descriptively with methods recommended in method guidelines and encountered in assessment reports of the European Medicines Agency (2015-2020) and 4 European HTA organizations (2015-2023). RESULTS: Thirty-four identified scientific articles described analytical methods for comparing uncontrolled trial data with IPD-RWD-based external controls. The various methods covered controlling for confounding and/or dependent censoring, correction for missing data, and analytical comparative modeling methods. Seven guidelines also focused on research design, RWD quality, and transparency aspects, and 4 of those recommended analytical methods for comparisons with IPD-RWD. The methods discussed in regulatory (n = 15) and HTA (n = 35) assessment reports were often based on aggregate data and lacked transparency owing to the few details provided. CONCLUSIONS: Literature and guidelines suggest a methodological approach to comparing uncontrolled trials with external controls from IPD-RWD similar to target trial emulation, using state-of-the-art methods. External controls supporting regulatory and HTA decision making were rarely in line with this approach. Twelve recommendations are proposed to improve the quality and acceptability of these methods.

Opportunities and Challenges for Decentralized Clinical Trial Approaches: European Health Technology Assessment Perspective.

OBJECTIVES: Decentralized clinical trial (DCT) approaches are clinical trials in which some or all trial activities take place closer to participants' proximities instead of a traditional investigative site. Data from DCTs may be used for clinical and economic evaluations by health technology assessment (HTA) bodies to support reimbursement decision making. This study aimed to explore the opportunities and challenges for DCT approaches from an HTA perspective by interviewing representatives from European HTA bodies. METHODS: We conducted semistructured interviews with 25 European HTA representatives between September 2022 and February 2023, and transcripts were analyzed after thematic analysis. RESULTS: Two main themes were identified from the data relating to (1) DCT approaches in HTA and (2) trial-level acceptance and relevance. Experience with assessing DCTs was limited and a variety of knowledge about DCTs was observed. The respondents recognized the opportunity of DCTs to reduce recall bias when participant-reported outcome data can be collected more frequently and conveniently from home. Concerns were expressed about the data quality when participants become responsible for data collection. Despite this challenge, the respondents recognized the potential of DCTs to increase the generalizability of results because data can be collected in a setting reflective of the everyday situation potentially from a more diverse participant group. CONCLUSIONS: DCTs could generate relevant results for HTA decision making when data are collected in a real-world setting from a diverse participant group. Increased awareness of the opportunities and challenges could help HTA assessors in their appraisal of DCT approaches.

Illustrating the Financial Consequences of Outcome-Based Payment Models From a Payers Perspective: The Case of Autologous Gene Therapy Atidarsagene Autotemcel (Libmeldy®).

OBJECTIVES: To illustrate the financial consequences of implementing different managed entry agreements (managed entry agreements for the Dutch healthcare system for autologous gene therapy atidarsagene autotemcel [Libmeldy]), while also providing a first systematic guidance on how to construct managed entry agreements to aid future reimbursement decision making and create patient access to high-cost, one-off potentially curative therapies. METHODS: Three payment models were compared: (1) an arbitrary 60% price discount, (2) an outcome-based spread payment with discounts, and (3) an outcome-based spread payment linked to a willingness to pay model with discounts. Financial consequences were estimated for full responders (A), patients responding according to the predicted clinical pathway presented in health technology assessment reports (B), and unstable responders (C). The associated costs for an average patient during the time frame of the payment agreement, the total budget impact, and associated benefits expressed in quality-adjusted life-years of the patient population were calculated. RESULTS: When patients responded according to the predicted clinical pathway presented in health technology assessment reports (scenario B), implementing outcome-based reimbursement models (models 2 and 3) had lower associated budget impacts while gaining similar benefits compared with the discount (scenario 1, €8.9 million to €6.6 million vs €9.2 million). In the case of unstable responders (scenario C), costs for payers are lower in the outcome-based scenarios (€4.1 million and €3.0 million, scenario 2C and 3C, respectively) compared with implementing the discount (€9.2 million, scenario 1C). CONCLUSIONS: Outcome-based models can mitigate the financial risk of reimbursing atidarsagene autotemcel. This can be considerably beneficial over simple discounts when clinical performance was similar to or worse than predicted.

A blueprint for health technology assessment capacity building: lessons learned from Malta.

OBJECTIVES: The development and strengthening of health technology assessment (HTA) capacity on the individual and organizational level and the wider environment is relevant for cooperation on HTAs. Based on the Maltese case, we provide a blueprint for building HTA capacity. METHODS: A set of activities were developed based on Pichler et al.'s framework and the starting HTA capacity in Malta. Individual level activities focused on strengthening epidemiological and health economic skills through online and in-person training. On the organizational level, a new HTA framework was developed which was subsequently utilized in a shadow assessment. Awareness campaign activities raised awareness and support in the wider environment where HTAs are conducted and utilized. RESULTS: The time needed to build HTA capacity exceeded the planned two years accommodating the learning progress of the assessors. In addition to the planned trainings, webinars supplemented the online courses, allowing for more knowledge exchange. The advanced online course was extended over time to facilitate learning next to the assessors' daily tasks. Training sessions were added to implement the new economic evaluation framework, which was utilized in a second shadow assessment. Awareness by decision-makers was achieved with reports, posters, and an article on the current and developing HTA capacity. CONCLUSIONS: It takes time and much (hands-on) training to build skills for conducting complex assessment such as HTAs. Facilitating exchange with knowledgeable parties is crucial for succeeding as well as the buy-in of local managers motivating staff. Decision-makers need to be on-boarded for the continued success of HTA capacity building.

Navigating the path towards successful implementation of the EU HTA Regulation: key takeaways from the 2023 Spring Convention of the European Access Academy.

BACKGROUND: The European Regulation on Health Technology Assessment (EU HTA R), effective since January 2022, aims to harmonize and improve the efficiency of common HTA across Member States (MS), with a phased implementation from January 2025. At "midterms" of the preparation phase for the implementation of the Regulation our aim was to identify and prioritize tangible action points to move forward. METHODS: During the 2023 Spring Convention of the European Access Academy (EAA), participants from different nationalities and stakeholder backgrounds discussed readiness and remaining challenges for the Regulation's implementation and identified and prioritized action points. For this purpose, participants were assigned to four working groups: (i) Health Policy Challenges, (ii) Stakeholder Readiness, (iii) Approach to Uncertainty and (iv) Challenges regarding Methodology. Top four action points for each working group were identified and subsequently ranked by all participants during the final plenary session. RESULTS: Overall "readiness" for the Regulation was perceived as neutral. Prioritized action points included the following: Health Policy, i.e. assess adjustability of MS laws and health policy processes; Stakeholders, i.e. capacity building; Uncertainty, i.e. implement HTA guidelines as living documents; Methodology, i.e. clarify the Population, Intervention, Comparator(s), Outcomes (PICO) identification process. CONCLUSIONS: At "midterms" of the preparation phase, the focus for the months to come is on executing the tangible action points identified at EAA's Spring Convention. All action points centre around three overarching themes: harmonization and standardization, capacity building and collaboration, uncertainty management and robust data. These themes will ultimately determine the success of the EU HTA R in the long run.

HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force.

PROBLEM: Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias. WHAT WE DID: The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The overarching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility. STRATEGIES TO DISSEMINATE AND FACILITATE USE: Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years. CONCLUSION: The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

Regulatory, health technology assessment and company interactions: the current landscape and future ecosystem for drug development, review and reimbursement.

BACKGROUND: Multi-stakeholder interactions have evolved at product and policy levels. There is a need to assess the current and future landscape of interactions between companies, and regulatory and HTA agencies to address challenges and identify areas for improvement. OBJECTIVES: The aims of this study were to review the current interactions within and across regulatory and HTA agencies, and companies' experiences in engaging in these activities; to assess the added value of interactions as well as limitations; to explore the future ecosystem for stakeholder interactions. METHOD: Three separate questionnaires were developed for companies, regulators and HTA agencies, respectively, to assess their experiences and perceptions. The responses were analyzed using descriptive statistics and discussed at a multi-stakeholder workshop. Key outcomes from the surveys and workshop discussion were reported. RESULTS: All seven regulators and seven HTA agencies in the survey indicated that they had stakeholder interactions. More formal collaboration occurred with regulators compared with HTA agencies. All nine companies have taken early advice but indicated the need for future prioritization. Success indicators can be built at the product and therapy levels, with the added value of faster patient access. Four principles were proposed for the future ecosystem: separate remit and functions between regulators and HTA; align processes; converge evidence requirements where possible; increase transparency. CONCLUSIONS: This research brought together regulators, HTA agencies, companies to examine how they interact with one another. We propose measures of value and make recommendations on future evolution to enable better evidence generation and improve regulatory and HTA decision-making.

Health technology assessment of tests for SARS-CoV-2 and treatments for COVID-19: A proposed approach and best-practice recommendations.

OBJECTIVES: To develop best-practice guidance for health technology assessment (HTA) agencies when appraising diagnostic tests for SARS-CoV-2 and treatments for COVID-19. METHODS: We used a policy sandbox approach to develop best-practice guidance for HTA agencies to approach known challenges associated with assessing tests and treatments for COVID-19. The guidance was developed by a multi-stakeholder workshop of twenty-one participants representing HTA agencies, clinical and patient experts, academia, industry, and a payer, from across Europe and North America. The workshop was supported by extensive background work to identify the key challenges, including: targeted reviews of existing COVID-related methods guidance for assessing interventions and clinical guidelines, engagement with clinical experts, a survey and workshop of HTA agencies, a systematic review of published economic evaluations, and a workshop of health economic modelers. RESULTS: We suggest HTA agencies should consider using other types of evidence (e.g., real world) where high-quality randomized controlled trials may be lacking and healthcare systems would value timely HTA outputs. A "living" HTA approach may be useful, given the context of an evolving disease, scientific understanding and evidence base, allowing for decisions to be efficiently revisited in response to new information; particularly, if supported by a common "disease model" for COVID-19. Innovative ways of engaging with the public and clinicians, and early engagement with regulators and payers, are recommended. CONCLUSIONS: HTA agencies should consider the elements of this guidance that are most suited to their existing processes to enable them to assess the effectiveness and value of interventions for COVID-19.

Uncertainty management in regulatory and health technology assessment decision-making on drugs: guidance of the HTAi-DIA Working Group.

OBJECTIVES: Uncertainty is a fundamental component of decision making regarding access to and pricing and reimbursement of drugs. The context-specific interpretation and mitigation of uncertainty remain major challenges for decision makers. Following the 2021 HTAi Global Policy Forum, a cross-sectoral, interdisciplinary HTAi-DIA Working Group (WG) was initiated to develop guidance to support stakeholder deliberation on the systematic identification and mitigation of uncertainties in the regulatory-HTA interface. METHODS: Six online discussions among WG members (Dec 2021-Sep 2022) who examined the output of a scoping review, two literature-based case studies and a survey; application of the initial guidance to a real-world case study; and two international conference panel discussions. RESULTS: The WG identified key concepts, clustered into twelve building blocks that were collectively perceived to define uncertainty: "unavailable," "inaccurate," "conflicting," "not understandable," "random variation," "information," "prediction," "impact," "risk," "relevance," "context," and "judgment." These were converted into a checklist to explain and define whether any issue constitutes a decision-relevant uncertainty. A taxonomy of domains in which uncertainty may exist within the regulatory-HTA interface was developed to facilitate categorization. The real-world case study was used to demonstrate how the guidance may facilitate deliberation between stakeholders and where additional guidance development may be needed. CONCLUSIONS: The systematic approach taken for the identification of uncertainties in this guidance has the potential to facilitate understanding of uncertainty and its management across different stakeholders involved in drug development and evaluation. This can improve consistency and transparency throughout decision processes. To further support uncertainty management, linkage to suitable mitigation strategies is necessary.

CODE-EHR best practice framework for the use of structured electronic healthcare records in clinical research.

Big data is central to new developments in global clinical science aiming to improve the lives of patients. Technological advances have led to the routine use of structured electronic healthcare records with the potential to address key gaps in clinical evidence. The covid-19 pandemic has demonstrated the potential of big data and related analytics, but also important pitfalls. Verification, validation, and data privacy, as well as the social mandate to undertake research are key challenges. The European Society of Cardiology and the BigData@Heart consortium have brought together a range of international stakeholders, including patient representatives, clinicians, scientists, regulators, journal editors and industry. We propose the CODE-EHR Minimum Standards Framework as a means to improve the design of studies, enhance transparency and develop a roadmap towards more robust and effective utilisation of healthcare data for research purposes.

Reported Challenges in Health Technology Assessment of Complex Health Technologies.

OBJECTIVES: With complex health technologies entering the market, methods for health technology assessment (HTA) may require changes. This study aimed to identify challenges in HTA of complex health technologies. METHODS: A survey was sent to European HTA organizations participating in European Network for HTA (EUnetHTA). The survey contained open questions and used predefined potentially complex health technologies and 7 case studies to identify types of complex health technologies and challenges faced during HTA. The survey was validated, tested for reliability by an expert panel, and pilot tested before dissemination. RESULTS: A total of 22 HTA organizations completed the survey (67%). Advanced therapeutic medicinal products (ATMPs) and histology-independent therapies were considered most challenging based on the predefined complex health technologies and case studies. For the case studies, more than half of the reported challenges were "methodological," equal in relative effectiveness assessments as in cost-effectiveness assessments. Through the open questions, we found that most of these challenges actually rooted in data unavailability. Data were reported as "absent," "insufficient," "immature," or "low quality" by 18 of 20 organizations (90%), in particular data on quality of life. Policy and organizational challenges and challenges because of societal or political pressure were reported by 8 (40%) and 4 organizations (20%), respectively. Modeling issues were reported least often (n = 2, 4%). CONCLUSIONS: Most challenges in HTA of complex health technologies root in data insufficiencies rather than in the complexity of health technologies itself. As the number of complex technologies grows, the urgency for new methods and policies to guide HTA decision making increases.

Key Considerations in the Health Technology Assessment of Advanced Therapy Medicinal Products in Scotland, The Netherlands, and England.

OBJECTIVES: Advanced therapy medicinal products (ATMPs) are highly innovative therapies. Their costs and uncertain value claims have raised concerns among health technology assessment (HTA) bodies and payers. Little is known about how underlying considerations in HTA of ATMPs shape assessment and reimbursement recommendations. We aim to identify and assess key considerations that played a role in HTA of ATMPs underlying reimbursement recommendations. METHODS: A review of HTA reports was conducted of all authorized ATMPs in Scotland, The Netherlands, and England. Considerations were extracted and categorized into EUnetHTA Core Model domains. Per jurisdiction, considerations were aggregated and key considerations identified (defined as occurring in >1/assessment per jurisdiction). A narrative analysis was conducted comparing key considerations between jurisdictions and different reimbursement recommendations. RESULTS: We identified 15 ATMPs and 18 HTA reports. In The Netherlands and England most key considerations were identified in clinical effectiveness (EFF) and cost- and economic effectiveness (ECO) domains. In Scotland, the social aspects domain yielded most key considerations, followed by ECO and EFF. More uncertainty in evidence and assessment outcomes was accepted when orphan or end-of-life criteria were applied. A higher percentage of considerations supporting recommendations were identified for products with positive recommendations compared with restricted and negative recommendations. CONCLUSIONS: This is the first empirical review of HTA's using the EUnetHTA Core Model to identify and structure key considerations retrospectively. It provides insights in supporting and opposing considerations for reimbursement of individual products and differences between jurisdictions. Besides the EFF and ECO domain, the social, ethical, and legal domains seem to bear considerable weight in assessment of ATMPs.

HARmonized Protocol Template to Enhance Reproducibility of Hypothesis Evaluating Real-World Evidence Studies on Treatment Effects: A Good Practices Report of a Joint ISPE/ISPOR Task Force.

OBJECTIVES: Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias. METHODS: The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The over-arching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility. STRATEGIES TO DISSEMINATE AND FACILITATE USE: Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years. CONCLUSION: The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

Understanding innovation of health technology assessment methods: the IHTAM framework.

Adequate methods are urgently needed to guarantee the good practice of health technology assessment (HTA) for technologies with novel properties. The aim of the study was to construct a conceptual framework to help understand the innovation of HTA methods (IHTAM). The construction of the IHTAM framework was based on two scoping reviews, one on the current practice of innovating methods, that is existing HTA frameworks, and one on theoretical foundations for innovating methods outside the HTA discipline. Both aimed to identify and synthesize concepts of innovation (i.e., innovation processes and roles of stakeholders in innovation). Using these concepts, the framework was developed in iterative brainstorming sessions and subsequent discussions with representatives from various stakeholder groups. The framework was constructed based on twenty documents on innovating HTA frameworks and fourteen guidelines from three scientific disciplines. It includes a generic innovation process consisting of three phases ("Identification," "Development," and "Implementation") and nine subphases. In the framework, three roles that HTA stakeholders can play in innovation ("Developers," "Practitioners," and "Beneficiaries") are defined, and a process on how the stakeholders innovate HTA methods is included. The IHTAM framework visualizes systematically which elements and stakeholders are important to the development and implementation of novel HTA methods. The framework could be used by all stakeholders involved in HTA innovation to learn how to engage dynamically and collaborate effectively throughout the innovation process. HTA stakeholders in practice have welcomed the framework, though additional testing of its applicability and acceptance is essential.

Addressing uncertainty in relative effectiveness assessments by HTA organizations.

This study outlines the ways in which different health technology assessment (HTA) organizations deal with uncertainty in relative effectiveness assessments (REAs), using the GRADE framework as a common reference. Guidelines regarding REA and uncertainty assessment methods and three most recent HTA reports (as of April 2020) of seven HTA organizations in Germany, England and Wales, France, the Netherlands, Europe (EUnetHTA), the USA, and Canada were included. First, it was analyzed how each organization addressed uncertainty on the following levels of evidence: (i) individual studies, (ii) body of evidence for one outcome, (iii) body of evidence across all outcomes, and (iv) added net benefit. Second, the extent to which HTA organizations considered the eight domains of certainty of evidence defined by GRADE was assessed. For individual studies, checklists were the most common approach to express uncertainty (4/7 organizations). Uncertainty in the body of evidence for all outcomes and in added benefit was combined in a single conclusion by five organizations. All organizations reported on at least 4/5 downgrading domains of GRADE, while the three upgrading domains were reported less. The operationalization of the assessment of multiple domains was unclear due to vague or absent guidelines. HTA organizations consider most domains of the GRADE framework, but approaches to assess uncertainty within REAs on different levels of evidence differ substantially between organizations. More alignment and guidance on the best methods to deal with uncertainty within HTA could lead to more clarity for stakeholders and to more aligned reimbursement recommendations.

Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials.

BACKGROUND: There have been ongoing efforts to understand when and how data from observational studies can be applied to clinical and regulatory decision making. The objective of this review was to assess the comparability of relative treatment effects of pharmaceuticals from observational studies and randomized controlled trials (RCTs). METHODS: We searched PubMed and Embase for systematic literature reviews published between January 1, 1990, and January 31, 2020, that reported relative treatment effects of pharmaceuticals from both observational studies and RCTs. We extracted pooled relative effect estimates from observational studies and RCTs for each outcome, intervention-comparator, or indication assessed in the reviews. We calculated the ratio of the relative effect estimate from observational studies over that from RCTs, along with the corresponding 95% confidence interval (CI) for each pair of pooled RCT and observational study estimates, and we evaluated the consistency in relative treatment effects. RESULTS: Thirty systematic reviews across 7 therapeutic areas were identified from the literature. We analyzed 74 pairs of pooled relative effect estimates from RCTs and observational studies from 29 reviews. There was no statistically significant difference (based on the 95% CI) in relative effect estimates between RCTs and observational studies in 79.7% of pairs. There was an extreme difference (ratio < 0.7 or > 1.43) in 43.2% of pairs, and, in 17.6% of pairs, there was a significant difference and the estimates pointed in opposite directions. CONCLUSIONS: Overall, our review shows that while there is no significant difference in the relative risk ratios between the majority of RCTs and observational studies compared, there is significant variation in about 20% of comparisons. The source of this variation should be the subject of further inquiry to elucidate how much of the variation is due to differences in patient populations versus biased estimates arising from issues with study design or analytical/statistical methods.

Early Cost-Effectiveness of Onasemnogene Abeparvovec-xioi (Zolgensma) and Nusinersen (Spinraza) Treatment for Spinal Muscular Atrophy I in The Netherlands With Relapse Scenarios.

OBJECTIVES: Onasemnogene Abeparvovec-xioi (AVXS-101) is a gene therapy intended for curative treatment of spinal muscular atrophy (SMA) with an expected price of around €2 000 000. The goal of this study is to perform a cost-effectiveness analysis of treatment of SMA I patients with AVXS-101 in The Netherlands including relapse scenarios. METHODS: An individual-based state-transition model was used to model treatment effect and survival of SMA I patients treated with AVXS-101, nusinersen and best supportive care (BSC). The model included five health states: three health states according to SMA types, one for permanent ventilation and one for death. Deterministic and probabilistic sensitivity analyses were performed. Effects of relapsing to lower health states in the years following treatment was explored. RESULTS: The base-case incremental cost-effectiveness ratio (ICER) for AVXS-101 versus BSC is €138 875/QALY, and €53 447/QALY for AVXS-101 versus nusinersen. If patients relapse within 10 years after treatment with AVXS-101, the ICER can increase up to 6-fold, with effects diminishing thereafter. Only relapses occurring later than 50 years after treatment have a negligible effect on the ICER. To comply with Dutch willingness-to-pay reference values, the price of AVXS-101 must decrease to €680 000. CONCLUSIONS: Based on this model, treatment with AVXS-101 is unlikely to be cost-effective under Dutch willingness-to-pay reference values. Uncertainty regarding the long-term curative properties of AVXS-101 can result in multiplication of the ICER. Decision-makers are advised to appropriately balance these uncertainties against the price they are willing to pay now.

Bridging health technology assessment and healthcare quality improvement using international consortium of health outcomes measurement standard sets.

OBJECTIVE: Although health technology assessment (HTA) and healthcare quality improvement are distinct processes, a greater level of alignment in outcome measures used may increase the quality and efficiency of data collection. This study evaluates the agreement in outcome measures used in oncology for healthcare quality improvement and HTAs, and how these align to the International Consortium for Health Outcomes Measurement (ICHOM) standard sets. METHODS: We conducted a cross-sectional comparative analysis of ICHOM sets focusing on oncological indications and publicly available measures for healthcare quality and HTA reports published by the National Health Care Institute from the Netherlands and the National Institute for Health and Care Excellence from the United Kingdom. RESULTS: All ICHOM sets and HTAs used overall survival, whereas quality improvement used different survival estimates. Different progression estimates for cancer were used in HTAs, ICHOM sets, and quality improvement. Data on health-related quality of life (HRQoL) was recommended in all ICHOM sets and all HTAs, but selectively for quality improvement. In HTAs, generic HRQoL questionnaires were preferred, whereas, in quality improvement and ICHOM sets, disease-specific questionnaires were recommended. Unfavorable outcomes were included in all HTAs and all ICHOM sets, but not always for quality improvement. CONCLUSIONS: Although HTA and quality improvement use outcome measures from the same domains, a greater level of alignment seems possible. ICHOM may provide input on standardized outcome measures to support this alignment. However, residual discrepancies will remain due to the different objectives of HTA and quality improvement.