Cost-utility analysis of TAVI compared with surgery in patients with severe aortic stenosis at low risk of surgical mortality in the Netherlands.

BACKGROUND: There is growing evidence to support the benefits of transcatheter aortic valve implantation (TAVI) over surgical aortic valve replacement (SAVR) in patients with symptomatic severe aortic stenosis (sSAS) who are at high- or intermediate-risk of surgical mortality. The PARTNER 3 trial showed clinical benefits with SAPIEN 3 TAVI compared with SAVR in patients at low risk of surgical mortality. Whether TAVI is also cost-effective compared with SAVR for low-risk patients in the Dutch healthcare system remains uncertain. This article presents an analysis using PARTNER 3 outcomes and costs data from the Netherlands to inform a cost-utility model and examine cost implications of TAVI over SAVR in a Dutch low-risk population. METHODS: A two-stage cost-utility analysis was performed using a published and validated health economic model based on adverse events with both TAVI and SAVR interventions from a published randomized low risk trial dataset, and a Markov model that captured lifetime healthcare costs and patient outcomes post-intervention. The model was adapted using Netherlands-specific cost data to assess the cost-effectiveness of TAVI and SAVR. Uncertainty was addressed using deterministic and probabilistic sensitivity analyses. RESULTS: TAVI generated 0.89 additional quality-adjusted life years (QALYs) at a €4742 increase in costs per patient compared with SAVR over a lifetime time horizon, representing an incremental cost-effectiveness ratio (ICER) of €5346 per QALY gained. Sensitivity analyses confirm robust results, with TAVI remaining cost-effective across several sensitivity analyses. CONCLUSIONS: Based on the model results, compared with SAVR, TAVI with SAPIEN 3 appears cost-effective for the treatment of Dutch patients with sSAS who are at low risk of surgical mortality. Qualitative data suggest broader societal benefits are likely and these findings could be used to optimize appropriate intervention selection for this patient population.

Assessing medical devices: a qualitative study from the validate perspective.

OBJECTIVES: Our objective was to explore procedures and methods used at health technology assessment (HTA) agencies for assessing medical devices and the underlying views of HTA practitioners about appropriate methodology to identify challenges in adopting new methodologies for assessing devices. We focused on the role of normative commitments of HTA practitioners in the adoption of new methods. METHODS: An online survey, including questions on procedures, scoping, and assessments of medical devices, was sent to members of the International Network of Agencies for Health Technology Assessment. Interviews were conducted with survey respondents and HTA practitioners involved in assessments of transcatheter aortic valve implantation to gain an in-depth understanding of choices made and views about assessing medical devices. Survey and interview questions were inspired by the "values in doing assessments of health technologies" approach towards HTA, which states that HTA addresses value-laden questions and information. RESULTS: The current practice of assessing medical devices at HTA agencies is predominantly based on procedures, methods, and epistemological principles developed for assessments of drugs. Both practical factors (available time, demands of decision-makers, existing legal frameworks, and HTA guidelines), as well as commitments of HTA practitioners to principles of evidence-based medicine, make the adoption of a new methodology difficult. CONCLUSIONS: There is a broad recognition that assessments of medical devices may need changes in HTA methodology. In order to realize this, the HTA community may require both a discussion on the role, responsibility, and goals of HTA, and resulting changes in institutional context to adopt new methodologies.

How to strengthen societal impact of research and innovation? Lessons learned from an explanatory research-on-research study on participatory knowledge infrastructures funded by the Netherlands Organization for Health Research and Development.

BACKGROUND: Scientific research and innovation can generate societal impact via different pathways. Productive interactions, such as collaboration between researchers and relevant stakeholders, play an important role and have increasingly gained interest of health funders around the globe. What works, how and why in research partnerships to generate societal impact in terms of knowledge utilisation is still not well-known. To explore these issues, the Netherlands Organization for Health Research and Development (ZonMw) initiated an exploratory research-on-research study with a focus on participatory knowledge infrastructures (PKIs) that they fund in the field of public health and healthcare. PKIs are sustainable infrastructures in which knowledge production, dissemination and utilisation takes place via committed collaboration between researchers and stakeholders from policy, practice and/or education. Examples are learning networks, academic collaborative centres, care networks and living labs. The aim of the study was twofold: to gain insights in what constitutes effective collaboration in PKIs; and to learn and improve the research governance, particularly of ZonMw as part of their dissemination and implementation activities. METHODS: During 2020-2022, we conducted a literature review on long-term research partnerships, analysed available documentation of twenty ZonMw-funded PKIs, surveyed participants of the 2021 European Implementation Event, interviewed steering committee members, organized a Group Decision Room with lecturers, and validated the findings with key experts. RESULTS: We identified eight mechanisms ('how and why') that are conditional for effective collaboration in PKIs: transdisciplinary collaboration; defining a shared ambition; doing justice to everyone's interests; investing in personal relationships; a professional organisation or structure; a meaningful collaborative process; mutual trust, sufficient time for and continuity of collaboration. Several factors ('what') may hinder (e.g., lack of ownership or structural funding) or facilitate (e.g., stakeholder commitment, embeddedness in an organisation or policy) effective collaboration in research partnerships. CONCLUSION: To use the study results in policy, practice, education, and/or (further) research, cultural and behavioural change of all stakeholders is needed. To facilitate this, we provide recommendations for funding organisations, particularly ZonMw and its partners within the relevant knowledge ecosystem. It is meant as a roadmap towards the realisation and demonstration of societal impact of (health) research and innovation in the upcoming years.

Understanding the Normativity of Health Technology Assessment: Ontological, Moral, and Epistemological Commitments.

The inherent normativity of HTA can be conceptualized as a result of normative commitments, a concept that we further specify to encompass moral, epistemological and ontological commitments at play in the practice of HTA. Based on examples from literature, and an analysis of the example of assessing Non-Invasive Prenatal Testing (NIPT), we will show that inevitable normative decisions in conducting an assessment commits the HTA practitioner to moral (regarding what makes a health technology desirable), ontological (regarding which effects of health technology are conceivable), and epistemological (regarding how to obtain reliable information about health technology) norms. This highlights and supports the need for integrating normative analysis and stakeholder participation, providing guidance to HTA practitioners when making normative choices. This will foster a shared understanding between those who conduct, use, or are impacted by assessments regarding what are conceivable and desirable outcomes of using health technology, and how to collect reliable information to assess whether these outcomes are (going to be) realized. It also provides more insight into the implications of different normative choices.

The use of cost-effectiveness analysis for health benefit package design - should countries follow a sectoral, incremental or hybrid approach?

BACKGROUND: Countries around the world are increasingly rethinking the design of their health benefit package to achieve universal health coverage. Countries can periodically revise their packages on the basis of sectoral cost-effectiveness analyses, i.e. by evaluating a broad set of services against a 'doing nothing' scenario using a budget constraint. Alternatively, they can use incremental cost-effectiveness analyses, i.e. to evaluate specific services against current practice using a threshold. In addition, countries may employ hybrid approaches which combines elements of sectoral and incremental cost-effectiveness analysis - a country may e.g. not evaluate the comprehensive set of all services but rather relatively small sets of services targeting a certain condition. However, there is little practical guidance for countries as to which kind of approach they should follow. METHODS: The present study was based on expert consultation. We refined the typology of approaches of cost-effectiveness analysis for benefit package design, identified factors that should be considered in the choice of approach, and developed recommendations. We reached consensus among experts over the course of several review rounds. RESULTS: Sectoral cost-effectiveness analysis is especially suited in contexts with large allocative inefficiencies in current service provision and can, in theory, realize large efficiency gains. However, it may be challenging to implement a comprehensive redesign of the package in practice. Incremental cost-effectiveness analysis is especially relevant in contexts where specific new services may impact the sustainability of the health system. It may potentially support efficiency improvement, but its focus has typically been on new services while existing inefficiencies remain unchallenged. The use of hybrid approach may be a way forward to address the strengths and weaknesses of sectoral and incremental analysis areas. Such analysis may be especially useful to target disease areas with suspected high inefficiencies in service provision, and would then make good use of the available research capacity and be politically rewarding. However, disease-specific analyses bear the risk of not addressing resource allocation inefficiencies across disease areas. CONCLUSIONS: Countries should carefully select their approach of cost-effectiveness analyses for benefit package design, based on their decision-making context.

Uncertainty management in regulatory and health technology assessment decision-making on drugs: guidance of the HTAi-DIA Working Group.

OBJECTIVES: Uncertainty is a fundamental component of decision making regarding access to and pricing and reimbursement of drugs. The context-specific interpretation and mitigation of uncertainty remain major challenges for decision makers. Following the 2021 HTAi Global Policy Forum, a cross-sectoral, interdisciplinary HTAi-DIA Working Group (WG) was initiated to develop guidance to support stakeholder deliberation on the systematic identification and mitigation of uncertainties in the regulatory-HTA interface. METHODS: Six online discussions among WG members (Dec 2021-Sep 2022) who examined the output of a scoping review, two literature-based case studies and a survey; application of the initial guidance to a real-world case study; and two international conference panel discussions. RESULTS: The WG identified key concepts, clustered into twelve building blocks that were collectively perceived to define uncertainty: "unavailable," "inaccurate," "conflicting," "not understandable," "random variation," "information," "prediction," "impact," "risk," "relevance," "context," and "judgment." These were converted into a checklist to explain and define whether any issue constitutes a decision-relevant uncertainty. A taxonomy of domains in which uncertainty may exist within the regulatory-HTA interface was developed to facilitate categorization. The real-world case study was used to demonstrate how the guidance may facilitate deliberation between stakeholders and where additional guidance development may be needed. CONCLUSIONS: The systematic approach taken for the identification of uncertainties in this guidance has the potential to facilitate understanding of uncertainty and its management across different stakeholders involved in drug development and evaluation. This can improve consistency and transparency throughout decision processes. To further support uncertainty management, linkage to suitable mitigation strategies is necessary.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force.

OBJECTIVES: Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap. METHODS: The joint Task Force consisted of 15 members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to 2 rounds of peer review. RESULTS: A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation. CONCLUSIONS: The guidance and the checklist contain a series of questions, grouped by 6 phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Health technology assessment: A matter of facts and values.

The unique task and contribution of health technology assessment (HTA) is to help identify those health technologies and their uses that are most likely to preserve and restore a population's health in a way that is consonant with its values, including, for example, equity and access to high-quality care (1). Such a task is challenging for at least two reasons. First, because of the vast and constantly evolving number and diversity of health technologies and their applications. Second, because of the usual wide variety of competing views within communities and their stakeholders regarding what strategies are likely to be conducive to the goal of preserving and restoring population health. Although perhaps tempting, it would be a grave mistake to hold that the controversies that result from such competing views can be resolved by taking recourse to the facts only. For such controversies are usually fueled by different notions of health and disease and different specifications of values such as equity and individual and collective responsibility for health. For this reason, they cannot be resolved in a satisfactory way without also addressing those normative issues and their interplay with empirical analysis.

Integrating Empirical Analysis and Normative Inquiry in Health Technology Assessment: The Values in Doing Assessments of Health Technologies Approach.

Health technology assessment (HTA) aims, through empirical analysis, to shed light on the value of health technologies (O'Rourke et al. [2020, International Journal of Technology Assessment in Health Care 36, 187-90]). HTA is, then, where facts and values meet. But how, where, and when do facts and values meet in HTA? Currently, HTA is usually portrayed as a sequential process, starting with empirical analysis (assessment), followed by a deliberation on the implications of the findings for a judgment of a health technology's value (appraisal). In this paper, we will argue that in HTA, empirical analysis and normative inquiry are much more closely entwined. In fact, as we hope to show, normative commitments act as an indispensable guide for the collection and interpretation of empirical evidence. Drawing on policy sciences, we will suggest a concrete methodology that can help HTA practitioners to integrate empirical analysis and normative inquiry in a transparent way. The proposed methodology can be conceived as a concrete means for conducting a scoping exercise in HTA. Moreover, it offers a distinct way of giving stakeholders a structural and constructive role in HTA. This paper outlines the approach developed by the values in doing assessments of health technologies project, a project funded by the Erasmus+ program (contract number 2018-1-NL01-KA203-038960), which is the European Union's program to support education, training, youth, and sport in Europe. The project has resulted in an E-learning course, an accompanying handbook, and a consensus statement, all freely available from the project's website www.validatehta.eu.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force.

OBJECTIVES: Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap. METHODS: The joint Task Force consisted of fifteen members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to two rounds of peer review. RESULTS: A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation. CONCLUSIONS: The guidance and the checklist contain a series of questions, grouped by six phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Learning and practicing more value-reflective, problem-setting health technology assessment: experiences and lessons from the VALIDATE project.

OBJECTIVES: To conduct a formative evaluation of applying the VALIDATE approach in practice by (i) assessing how students appreciated the e-learning course, (ii) exploring how, for what purposes and with what outcomes the acquired VALIDATE competences subsequently were used in internships in different institutional contexts, and how this was shaped by these contexts, and (iii) what this shows on real-world use of VALIDATE. METHODS: Comparative discussion of experiences of applying the VALIDATE approach via a semistructured survey and oral feedback from e-course students; final reports on internships in health technology assessment (HTA) practice, followed by semistructured interviews with interns and supervisors to complement and interpret results. RESULTS: All students considered the VALIDATE approach an enlightening and important addition to current HTA knowledge, especially regarding understanding the relation between empirical analysis and normative inquiry, identifying policy relevant questions and using the method of reconstructing interpretive frames for scoping. The latter appeared intellectually challenging and requiring some prior HTA knowledge. The use the VALIDATE approach in practice shows that interns productively redefined the HTA problem, based on appreciation of different stakeholders' definition of the issue; they experienced constraints from retrieving all relevant perspectives from existing literature as well as from institutional rules and routines. CONCLUSIONS: Some challenges in applying the VALIDATE approach deserve attention for its future use: currently used research approaches often assume a problem as "given"; and the data needed on different perspectives is often not reported in scientific literature. Finally, data gathering on and evaluation of value dimensions was experienced as challenging.

Case-studies of displacement effects in Dutch hospital care.

BACKGROUND: Under a constrained health care budget, cost-increasing technologies may displace funds from existing health services. However, it is unknown what services are displaced and how such displacement takes place in practice. The aim of our study was to investigate how the Dutch hospital sector has dealt with the introduction of cost-increasing health technologies, and to present evidence of the relative importance of three main options to deal with cost-increases in health care: increased spending, increased efficiency, or displacement of other services. METHODS: We conducted six case-studies and interviewed 84 professionals with various roles and responsibilities (practitioners, heads of clinical department, board of directors, insurers, and others) to investigate how they experienced decision making in response to the cost pressure of cost-increasing health technologies. Transcripts were analyzed thematically in Atlas.ti on the basis of an item list. RESULTS: Direct displacement of high-value care due to the introduction of new technologies was not observed; respondents primarily pointed to increased spending and efficiency measures to accommodate the introduction of the cost-increasing technologies. Respondents found it difficult to identify the opportunity costs; partly due to limited transparency in the internal allocation of funds within a hospital. Furthermore, respondents experienced the entry of new technologies and cost-containment as two parallel processes that are generally not causally linked: cost containment was experienced as a permanent issue to level costs and revenues, independent from entry of new technologies. Furthermore, the way of financing was found important in displacement in the Netherlands, especially as there is a separate budget for expensive drugs. This budget pressure was found to be reallocated horizontally across departments, whereas the budget pressure of other services is primarily reallocated vertically within departments or divisions. Respondents noted that hospitals have reacted to budget pressures primarily through a narrowing in the portfolio of their services, and a range of (other) efficiency measures. The board of directors is central in these processes, insurers are involved only to a limited extent. CONCLUSIONS: Our findings indicate that new technologies were generally accommodated by greater efficiency and increased spending, and that hospitals sought savings or efficiency measures in response to cumulative cost pressures rather than in response to single cost-increasing technologies.

Implementing evidence-informed deliberative processes in health technology assessment: a low income country perspective.

The purpose of this paper is to discuss the potential feasibility and utility of evidence-informed deliberative processes (EPDs) in low income country (LIC) contexts. EDPs are implemented in high and middle income countries and thought to improve the quality, consistency, and transparency of decisions informed by health technology assessment (HTA). Together these would ultimately improve the legitimacy of any decision making process. We argue-based on our previous work and in light of the priority setting literature-that EDPs are relevant and feasible within LICs. The extreme lack of resources necessitates making tough decisions which may mean depriving populations of potentially valuable health technologies. It is critical that the decisions and the decision making bodies are perceived as fair and legitimate by the people that are most affected by the decisions. EDPs are well aligned with the political infrastructure in some LICs, which encourages public participation in decision making. Furthermore, many countries are committed to evidence-informed decision making. However, the application of EDPs may be hampered by the limited availability of evidence of good quality, lack of interest in transparency and accountability (in some LICs), limited capacity to conduct HTA, as well as limited time and financial resources to invest in a deliberative process. While EDPs would potentially benefit many LICs, mitigating the identified potential barriers would strengthen their applicability. We believe that implementation studies in LICs, documenting the contextualized enablers and barriers will facilitate the development of context specific improvement strategies for EDPs.

Application of evidence-informed deliberative processes in health technology assessment in low- and middle-income countries.

OBJECTIVES: Evidence-informed deliberative processes (EDPs) were introduced to guide health technology assessment (HTA) agencies to improve their processes toward more legitimate decision making. A survey among members of the International Network of Agencies for HTA (INAHTA) showed that EDPs can also be relevant for countries that have not (yet) established such an agency. Therefore, we explored to what extent low- and middle-income countries (LMIC) applied the steps and elements stipulated in the EDP framework and their need for guidance. METHODS: The survey among INAHTA members was slightly adapted to address LMIC context and sent to 416 experts identified through several HTA sources. The questions focused on contextual factors and the EDP steps (installation of an appraisal committee, selecting technologies and criteria, assessment, appraisal, communication and appeal). Data collection took place between 21 May and 1 September 2019. Descriptive statistics and qualitative analyses were used to summarize the findings. RESULTS: We received sixty-six meaningful responses from experts in thirty-two LMIC. We found that contextual factors to support HTA development are overall not present or only present to some extent. Respondents indicated that guidance was needed for specific elements related to selecting technologies and criteria, assessment, appraisal, as well as communication and appeal. CONCLUSIONS: EDPs have the potential to provide steps for improving HTA processes. The results of this study can serve as a baseline measurement for future monitoring and evaluation of EDP application in the responding LMIC. This could support the countries in improving their processes and enhancing legitimate decision making when using HTA.

The new definition of health technology assessment: A milestone in international collaboration.

BACKGROUND: An international joint task group co-led by the International Network of Agencies for Health Technology Assessment (INAHTA) and Health Technology Assessment International (HTAi) has developed a new and internationally accepted definition of HTA. METHODS: The task group, consisting of representatives of leading HTA networks, societies and global organizations, developed guiding principles for the process and followed an established consultation plan with the broader HTA community to develop the definition. RESULTS: The consensus achieved by the international joint task group brings the collective weight of the participating networks, societies, and organizations behind the new definition. CONCLUSION: The new definition of HTA is an historic achievement and it is offered to the current and emerging HTA world as a cornerstone reference for today and into the future.

Toward a common understanding of competencies for health technology assessment: enhancing educational and training programs around the globe.

BACKGROUND: Depending on the health system context and the demands of relevant stakeholders in countries, the need, organizational structure, and prerequisites for enabling capacity building and development in health technology assessment (HTA) will vary. Core competencies are instrumental in this and include essential knowledge, skills, and attitudes (KSAs). They provide building blocks for delivering high-quality and effective practices of HTA. We aimed to systematically explore and develop an overview of the core competencies necessary for HTA. METHODS: This study was conducted during 2016-19 using different methods in a structured manner. We drew concepts of KSAs from various literature sources, surveyed universities and HTA professionals, and conducted expert workshops to arrive at a common understanding of the required competencies. RESULTS: The terminology for KSAs defining competencies in HTA programs has been clarified. In addition, a list of competencies offered through different educational and training programs has been created. The surveys provided clarity on a common understanding of KSAs among HTA stakeholders. Thereafter, a set of competencies was described and classified according to the HTA domains. CONCLUSIONS: Our study shows that there is diversity in HTA programs offered by educational institutions. The content of the programs varies due to differences between countries regarding the level of HTA development and the need for HTA, including the understanding of what HTA is. The preparation of a competency checklist or a "menu" of options mirroring the diversity of HTA will ensure that the specific needs of the HTA community will be covered.

Core competencies for ethics experts in health technology assessment.

OBJECTIVES: There is no consensus on who might be qualified to conduct ethical analysis in the field of health technology assessment (HTA). Is there a specific expertise or skill set for doing this work? The aim of this article is to (i) clarify the concept of ethics expertise and, based on this, (ii) describe and specify the characteristics of ethics expertise in HTA. METHODS: Based on the current literature and experiences in conducting ethical analysis in HTA, a group of members of the Health Technology Assessment International (HTAi) Interest Group on Ethical Issues in HTA critically analyzed the collected information during two face-to-face workshops. On the basis of the analysis, working definitions of "ethics expertise" and "core competencies" of ethics experts in HTA were developed. This paper reports the output of the workshop and subsequent revisions and discussions online among the authors. RESULTS: Expertise in a domain consists of both explicit and tacit knowledge and is acquired by formal training and social learning. There is a ubiquitous ethical expertise shared by most people in society; nevertheless, some people acquire specialist ethical expertise. To become an ethics expert in the field of HTA, one needs to acquire general knowledge about ethical issues as well as specific knowledge of the ethical domain in HTA. The core competencies of ethics experts in HTA consist of three fundamental elements: knowledge, skills, and attitudes. CONCLUSIONS: The competencies described here can be used by HTA agencies and others involved in HTA to call attention to and strengthen ethical analysis in HTA.

Multicriteria Decision Analysis to Support Health Technology Assessment Agencies: Benefits, Limitations, and the Way Forward.

OBJECTIVE: Recent years have witnessed an increased interest in the use of multicriteria decision analysis (MCDA) to support health technology assessment (HTA) agencies for setting healthcare priorities. However, its implementation to date has been criticized for being "entirely mechanistic," ignoring opportunity costs, and not following best practice guidelines. This article provides guidance on the use of MCDA in this context. METHODS: The present study was based on a systematic review and consensus development. We developed a typology of MCDA studies and good implementation practice. We reviewed 36 studies over the period 1990 to 2018 on their compliance with good practice and developed recommendations. We reached consensus among authors over the course of several review rounds. RESULTS: We identified 3 MCDA study types: qualitative MCDA, quantitative MCDA, and MCDA with decision rules. The types perform differently in terms of quality, consistency, and transparency of recommendations on healthcare priorities. We advise HTA agencies to always include a deliberative component. Agencies should, at a minimum, undertake qualitative MCDA. The use of quantitative MCDA has additional benefits but also poses design challenges. MCDA with decision rules, used by HTA agencies in The Netherlands and the United Kingdom and typically referred to as structured deliberation, has the potential to further improve the formulation of recommendations but has not yet been subjected to broad experimentation and evaluation. CONCLUSION: MCDA holds large potential to support HTA agencies in setting healthcare priorities, but its implementation needs to be improved.

Addressing Health System Values in Health Technology Assessment: The Use of Evidence-Informed Deliberative Processes.

Health technology assessment (HTA) is increasingly used around the globe to inform resource allocation decisions. Furthermore, the importance of using explicit and transparent criteria for coverage decision making in line with health system values has been acknowledged. However, the values of a health system are often not explicitly taken into account in the HTA process. This situation influences the allocation of scarce resources and could lead to a discord between the HTA outcome and the values of the health system. We describe how evidence-informed deliberative processes (EDPs) can help to improve this situation. EDPs are integrating two theoretical frameworks; multi-criteria decision-analysis and accountability for reasonableness. Through the use of EDPs, HTA agencies can ensure that health system values are more explicitly and consistently taken into account in the HTA process, enhancing the legitimacy of coverage decisions.

How to Deal with the Inevitable: Generating Real-World Data and Using Real-World Evidence for HTA Purposes - From Theory to Action.

OBJECTIVES: Real-world evidence (RWE), derived from real-world data (RWD), is already used, to some extent, for health technology assessment (HTA) purposes. With the increased availability of RWD, there is potential for more widespread use but also challenges ensuring reliable RWE for HTA. Opportunities to overcome key challenges, identified at a scoping meeting, were discussed during the 2019 HTA international (HTAi) Global Policy Forum (GPF). METHODS: Reflection of discussions using Design Thinking (an interactive process aimed to solve complex problems) between seventy-three representatives from not-for-profit, for-profit organizations, and HTAi leadership. The discussions were informed by a background paper, and presentations from three invited keynote speakers and eleven GPF members. RESULTS: Several options were listed for addressing the identified key challenges: quality and acceptability, governance and accountability, transferability, and informing decision making. The GPF emphasized that the HTA community should first understand what questions could be answered with RWE. Additionally, more clarity on methods, standards, streamlining RWD collection, data sharing across jurisdictions, replication of RWD, and expert analysis were mentioned as important priorities. CONCLUSIONS: The HTA community is currently standing at a cross-road as it is not yet fully equipped to address these key challenges. It is, therefore, time for action. The community should start aligning on what is the best source of evidence according to purpose and how the data should be collected to create reliable evidence. It should also initiate the development of actions and guidance to properly develop and manage RWD/RWE to inform decision making across the technology lifecycle.