Overcoming Barriers in Hospital-Based Health Technology Assessment (HB-HTA): International Expert Panel Consensus.

The purpose of this article is to investigate the common facilitators and barriers associated with the implementation of hospital-based health technology assessment (HB-HTA) across diverse hospital settings in seven countries. Through a two-round Delphi study, insights were gathered from a panel of 15 HTA specialists from France, Hungary, Italy, Kazakhstan, Poland, Switzerland, and Ukraine. Experts initially conducted a comprehensive review of the HB-HTA implementation in their respective countries, identifying the barriers and facilitators through descriptive analysis. Subsequently, panel experts ranked these identified barriers and facilitators on a seven-point Likert scale. A median agreement score ≥ 6 and interquartile range (IQR) ≤ 1 was accepted as reaching a consensus. Out of the 12 statements categorized as external and internal barriers and facilitators, the expert panel reached consensus on six statements (two barriers and four facilitators). The external barrier, which achieved consensus, was the lack of the formal recognition of the role of HB-HTA in national or regional legislations. The internal barrier reaching consensus was the limited availability of human resources dedicated to HB-HTA. This qualitative study indicates that HB-HTA still has progress to make before being formally accepted and integrated across most countries, although by building on the facilitating factors we identified there may be an opportunity for the implementation of internationally developed strategies to strengthen HB-HTA practices.

Frailty prevalence in 42 European countries by age and gender: development of the SHARE Frailty Atlas for Europe.

Comparative frailty prevalence data across European countries is sparse due to heterogeneous measurement methods. The Survey of Health, Ageing and Retirement (SHARE) initiative conducted interviews with probability sampling of non-institutionalized elderly people in several European countries. Previous frailty analyses of SHARE datasets were limited to initial SHARE countries and did not provide age- and gender-stratified frailty prevalence. Our aim was to provide age- and gender-stratified frailty prevalence estimates in all European countries, with predictions where necessary. From 29 SHARE participating countries, 311,915 individual surveys were analyzed. Frailty prevalence was estimated by country and gender in 5-year age bands using the SHARE Frailty Instrument and a frailty index. Association of frailty prevalence with age, gender, and GDP per capita (country-specific economic indicator for predictions) was investigated in multivariate mixed logistic regression models with or without multiple imputation. Female gender and increasing age were significantly associated with higher frailty prevalence. Higher GDP per capita, with or without purchasing power parity adjustment, was significantly associated with lower frailty prevalence in the 65-79 age groups in all analyses. Observed and predicted data on frailty rates by country are provided in the interactive SHARE Frailty Atlas for Europe. Our study provides age- and gender-stratified frailty prevalence estimates for all European countries, revealing remarkable between-country heterogeneity. Higher frailty prevalence is strongly associated with lower GDP per capita, underlining the importance of investigating transferability of evidence across countries at different developmental levels and calling for improved policies to reduce inequity in risk of developing frailty across European countries.

Multistakeholder Perceptions of Additional Value Elements for United States Value Assessment of Health Interventions.

OBJECTIVES: Limitations in conventional cost-effectiveness methods have led to calls for incorporation of additional value elements in assessments of health technologies. However, gaps remain in how additional value elements may inform decision making. This study aimed to prioritize additional value elements from the perspective of healthy individuals without a specific condition or indicated for a specific treatment in the United States among a multistakeholder panel and compare the importance of perspective-specific value elements. METHODS: Additional value elements were prioritized in 2 phases: (1) we identified and categorized additional value elements in a targeted literature review, and (2) we convened a multistakeholder group-based preference elicitation study (N = 28) to evaluate the description of each value element and rank and generate normalized weights of each value element for its significance in value assessment. The importance of additional value elements was also weighted relative to patient-centric value elements. RESULTS: The rank and weight of contextual value elements among 28 stakeholders were "severity of the disease" (26.2%), "disadvantaged and vulnerable target populations highly represented" (21.8%), "broader economic impact" (17.3%), "risk protection" (13.8%), "rarity of the disease" (11.3%), and "novel mechanism of action" (9.7%). Relative weight of the additional value elements versus patient-centric value elements was 52% and 48%, respectively. CONCLUSIONS: Study findings may inform priority setting for value frameworks and emerging US government assessments. The group-based elicitation method is repeatable and useful for structured deliberative processes in value assessment and may help improve the consistency and predictability of what is important to stakeholders.

Recommendations for patient involvement in health technology assessment in Central and Eastern European countries.

Introduction: Meaningful patient involvement in health technology assessment (HTA) is essential in ensuring that the interests of the affected patient population, their families, and the general public are accurately reflected in coverage and reimbursement decisions. Central and Eastern European (CEE) countries are generally at less advanced stages of implementing HTA, which is particularly true for patient involvement activities. As part of the Horizon2020 HTx project, this research aimed to form recommendations for critical barriers to patient involvement in HTA in CEE countries. Methods: Built on previous research findings on potential barriers, a prioritisation survey was conducted online with CEE stakeholders. Recommendations for prioritised barriers were formed through a face-to-face workshop by CEE stakeholders and HTx experts. Results: A total of 105 stakeholders from 13 CEE countries completed the prioritisation survey and identified 12 of the 22 potential barriers as highly important. The workshop had 36 participants representing 9 CEE countries, and 5 Western European countries coming together to discuss solutions in order to form recommendations based on best practices, real-life experience, and transferability aspects. Stakeholder groups involved in both phases included HTA organisation representatives, payers, patients, caregivers, patient organisation representatives, patient experts, health care providers, academic and non-academic researchers, health care consultants and health technology manufacturers/providers. As a result, 12 recommendations were formed specified to the CEE region's context, but potentially useful for a broader geographic audience. Conclusion: In this paper, we present 12 recommendations for meaningful, systematic, and sustainable patient involvement in HTA in CEE countries. Our hope is that engaging more than a hundred CEE stakeholders in the study helped to spread awareness of the importance and potential of patient involvement and that the resulting recommendations provide tangible steps for the way forward. Future studies shall focus on country-specific case studies of the implemented recommendations.

Recommendations to overcome barriers to the use of artificial intelligence-driven evidence in health technology assessment.

Background: Artificial intelligence (AI) has attracted much attention because of its enormous potential in healthcare, but uptake has been slow. There are substantial barriers that challenge health technology assessment (HTA) professionals to use AI-generated evidence for decision-making from large real-world databases (e.g., based on claims data). As part of the European Commission-funded HTx H2020 (Next Generation Health Technology Assessment) project, we aimed to put forward recommendations to support healthcare decision-makers in integrating AI into the HTA processes. The barriers, addressed by the paper, are particularly focusing on Central and Eastern European (CEE) countries, where the implementation of HTA and access to health databases lag behind Western European countries. Methods: We constructed a survey to rank the barriers to using AI for HTA purposes, completed by respondents from CEE jurisdictions with expertise in HTA. Using the results, two members of the HTx consortium from CEE developed recommendations on the most critical barriers. Then these recommendations were discussed in a workshop by a wider group of experts, including HTA and reimbursement decision-makers from both CEE countries and Western European countries, and summarized in a consensus report. Results: Recommendations have been developed to address the top 15 barriers in areas of (1) human factor-related barriers, focusing on educating HTA doers and users, establishing collaborations and best practice sharing; (2) regulatory and policy-related barriers, proposing increasing awareness and political commitment and improving the management of sensitive information for AI use; (3) data-related barriers, suggesting enhancing standardization and collaboration with data networks, managing missing and unstructured data, using analytical and statistical approaches to address bias, using quality assessment tools and quality standards, improving reporting, and developing better conditions for the use of data; and (4) technological barriers, suggesting sustainable development of AI infrastructure. Conclusion: In the field of HTA, the great potential of AI to support evidence generation and evaluation has not yet been sufficiently explored and realized. Raising awareness of the intended and unintended consequences of AI-based methods and encouraging political commitment from policymakers is necessary to upgrade the regulatory and infrastructural environment and knowledge base required to integrate AI into HTA-based decision-making processes better.

Health technology assessment of tests for SARS-CoV-2 and treatments for COVID-19: A proposed approach and best-practice recommendations.

OBJECTIVES: To develop best-practice guidance for health technology assessment (HTA) agencies when appraising diagnostic tests for SARS-CoV-2 and treatments for COVID-19. METHODS: We used a policy sandbox approach to develop best-practice guidance for HTA agencies to approach known challenges associated with assessing tests and treatments for COVID-19. The guidance was developed by a multi-stakeholder workshop of twenty-one participants representing HTA agencies, clinical and patient experts, academia, industry, and a payer, from across Europe and North America. The workshop was supported by extensive background work to identify the key challenges, including: targeted reviews of existing COVID-related methods guidance for assessing interventions and clinical guidelines, engagement with clinical experts, a survey and workshop of HTA agencies, a systematic review of published economic evaluations, and a workshop of health economic modelers. RESULTS: We suggest HTA agencies should consider using other types of evidence (e.g., real world) where high-quality randomized controlled trials may be lacking and healthcare systems would value timely HTA outputs. A "living" HTA approach may be useful, given the context of an evolving disease, scientific understanding and evidence base, allowing for decisions to be efficiently revisited in response to new information; particularly, if supported by a common "disease model" for COVID-19. Innovative ways of engaging with the public and clinicians, and early engagement with regulators and payers, are recommended. CONCLUSIONS: HTA agencies should consider the elements of this guidance that are most suited to their existing processes to enable them to assess the effectiveness and value of interventions for COVID-19.

Guidance on using real-world evidence from Western Europe in Central and Eastern European health policy decision making.

Aim: Real-world data and real-world evidence (RWE) are becoming more important for healthcare decision making and health technology assessment. We aimed to propose solutions to overcome barriers preventing Central and Eastern European (CEE) countries from using RWE generated in Western Europe. Materials & methods: To achieve this, following a scoping review and a webinar, the most important barriers were selected through a survey. A workshop was held with CEE experts to discuss proposed solutions. Results: Based on survey results, we selected the nine most important barriers. Multiple solutions were proposed, for example, the need for a European consensus, and building trust in using RWE. Conclusion: Through collaboration with regional stakeholders, we proposed a list of solutions to overcome barriers on transferring RWE from Western Europe to CEE countries.

Framework for Patient Experience Value Elements in Rare Disease: A Case Study Demonstrating the Applicability of Combined Qualitative and Quantitative Methods.

BACKGROUND AND OBJECTIVE: Several novel methods have been suggested to extend a conventional value assessment to capture a more comprehensive perspective of value from a patient perspective. The objective of this research was to demonstrate a framework for implementing a combined qualitative and quantitative method to elicit and prioritize patient experience value elements in rare diseases. Neuromyelitis optica spectrum disorder was used as a case study. METHODS: The method for eliciting and prioritizing patient experience value elements involved a three-step process: (1) collecting potential patient experience value elements from existing literature sources followed by deliberation by a multi-stakeholder research team; (2) a pre-workshop webinar and survey to identify additional patient-reported value elements; and (3) a workshop to discuss, prioritize the value elements using a swing weighting method. Outcomes were prioritized value elements with normalized weights for patients considering a treatment for neuromyelitis optica spectrum disorder. RESULTS: A literature review and deliberation resulted in the following initial value elements: ability to reach important personal milestones, patient's financial burden, value of hope/balance or timing of risks and benefits, Uncertainty about long-term benefits and safety of the treatment, Patient empowerment through therapeutic advancement and technology, Caregiver/family's financial burden, patient experience related to treatment regimen, Therapeutic options, and Caregiver/family's quality of life. Eight patients with neuromyelitis optica spectrum disorder participated in the case study. In the online survey, participants found the nine proposed patient experience value elements both understandable and important with no additions. During the workshop, 'Uncertainty about long-term benefits and safety,' 'Patient experience related to treatment regimen,' and 'Patient's financial burden' were found to be the most important patient experience value elements, with a respective weight of 25%, 19.2%, and 14.4% (out of total 100%). CONCLUSIONS: This case study provides a framework for eliciting and prioritizing patient experience value elements using direct patient input. Although elements/weights may differ by disease, and even in neuromyelitis optica spectrum disorder, additional research is needed, value frameworks, researchers, and manufacturers can use this practical method to generate patient experience value elements and evaluate their impact on treatment selection.

Reimbursement and payment models in Central and Eastern European as well as Middle Eastern countries: A survey of their current use and future outlook.

There is growing interest in innovative reimbursement and payment models in Central and Eastern European (CEE) and Middle Eastern (ME) countries. A questionnaire was sent to payers from CEE and ME countries regarding the current use of, future preferences for and perceived barriers with these models. Twenty-seven healthcare payers from 11 countries completed the survey. Results showed participants preferred using outcome-based reimbursement models and delayed payment models more often; however, currently they are rarely applied. Barriers hindering implementation were mostly related to IT and data infrastructure, measurement issues, transaction costs and the administrative burden. Given these barriers highlighted in our study, policymakers should focus on the development of an implementation framework with contract templates for the preferred reimbursement and payment schemes to aid the feasibility of a successful implementation.

Systematic literature review of health economic models developed for multiple myeloma to support future analyses.

AIMS: The goal of this study was to review the economic evaluations of health technologies in multiple myeloma (MM) and provide guidance and recommendations for future health economic analyses. MATERIALS AND METHODS: A systemic literature review (SLR) was conducted on original economic assessment studies and structured review papers focusing on the studies in MM. The search was limited to English language papers published from 1 January 2000 onwards. Publications not applying any type of modelling methodology to describe disease progression and patient pathways over a specific time horizon were excluded. RESULTS: A total of 2,643 publications were initially identified, of which 148 were eligible to be included in the full-text review phase. From these, 49 publications were included in the final analysis. Most published health economic analyses supported by models came from high-income countries. Evaluations from middle-income countries were rarely published. Diagnostic technologies were rarely modelled and integrated care had not been modelled. Very few models investigated MM treatments from a societal perspective and there was a relative lack of evaluations regarding minimal residual disease (MRD). LIMITATIONS: Limitations of the publications included differences between trial populations and modelled populations, justification of methods, lack of confounder analyses, and small trial populations. Limitations of our study included the infeasibility of comparing MM economic evaluations due to the significant variance in modelled therapeutic lines and indications, and the relative scarcity of published economic evaluations from non-high-income countries. CONCLUSIONS: As published economic models lacked many of the elements of the complex and heterogeneous patient pathways in MM and they focused on single decision problems, a thorough, open-source economic whole disease modelling framework is needed to assess the economic value of a wide range of technologies across countries with various income levels with a more detailed view on MM, by including patient-centric and societal aspects.

A multicriteria decision analysis (MCDA) tool to purchase implantable medical devices in Egypt.

BACKGROUND: With the availability of several similar medical devices performing the same function, choosing one for reimbursement is not easy, especially if purchased for a large number of patients. The objective of this project was to create a multicriteria decision analysis (MCDA) tool, that captures and compares all implantable medical devices' attributes, to provide an objective method for choosing among the available options in Egypt. METHOD: We conducted a systematic review and expert interviews, to identify the relevant criteria for inclusion in the tool. Subsequently, a workshop was conducted, that involved experts in procuring and tendering medical devices. Experts chose the criteria, ranked them, assigned weights and scoring functions for each criterion, and then created the draft tool. A pilot phase followed; then, another workshop was conducted to fine-tune the tool. We readjusted the tool based on experts' experience with the draft tool. RESULTS: The final tool included eight criteria, arranged according to their weightage: technical characteristics (29.4%), country of origin (19.5%), use in reference countries (14.9%), supply reliability (11.7%), previous use in tenders (9.0%), instant replacement within product variety (6.9%), pharmacovigilance (4.6%), and refund or replacement (4.0%). Each medical device was assessed on these eight criteria to achieve a final score, that was compared to the alternative devices' scores. Price is not included in the MCDA tool, but it will be added in the financial evaluation phase. CONCLUSION: Decisionmakers could use the MCDA tool, to make evidence-based and objective decisions for purchasing implantable devices, in the Egyptian public sector. Post price evaluation, the product with the best value will be chosen for reimbursement. HIGHLIGHTS: We created an MCDA tool to help decision makers choose between alternative implantable medical devices in Egypt. The MCDA tool includes eight criteria, where price is evaluated as a separate step. "Technical characteristics" and "country of origin" criteria carried the highest weights, thus representing approximately 50% of the decision.

Outcome-based reimbursement in Central-Eastern Europe and Middle-East.

Outcome-based reimbursement models can effectively reduce the financial risk to health care payers in cases when there is important uncertainty or heterogeneity regarding the clinical value of health technologies. Still, health care payers in lower income countries rely mainly on financial based agreements to manage uncertainties associated with new therapies. We performed a survey, an exploratory literature review and an iterative brainstorming in parallel about potential barriers and solutions to outcome-based agreements in Central and Eastern Europe (CEE) and in the Middle East (ME). A draft list of recommendations deriving from these steps was validated in a follow-up workshop with payer experts from these regions. 20 different barriers were identified in five groups, including transaction costs and administrative burden, measurement issues, information technology and data infrastructure, governance, and perverse policy outcomes. Though implementing outcome-based reimbursement models is challenging, especially in lower income countries, those challenges can be mitigated by conducting pilot agreements and preparing for predictable barriers. Our guidance paper provides an initial step in this process. The generalizability of our recommendations can be improved by monitoring experiences from pilot reimbursement models in CEE and ME countries and continuing the multistakeholder dialogue at national levels.

Barriers to Use Artificial Intelligence Methodologies in Health Technology Assessment in Central and East European Countries.

The aim of this paper is to identify the barriers that are specifically relevant to the use of Artificial Intelligence (AI)-based evidence in Central and Eastern European (CEE) Health Technology Assessment (HTA) systems. The study relied on two main parallel sources to identify barriers to use AI methodologies in HTA in CEE, including a scoping literature review and iterative focus group meetings with HTx team members. Most of the other selected articles discussed AI from a clinical perspective (n = 25), and the rest are from regulatory perspective (n = 13), and transfer of knowledge point of view (n = 3). Clinical areas studied are quite diverse-from pediatric, diabetes, diagnostic radiology, gynecology, oncology, surgery, psychiatry, cardiology, infection diseases, and oncology. Out of all 38 articles, 25 (66%) describe the AI method and the rest are more focused on the utilization barriers of different health care services and programs. The potential barriers could be classified as data related, methodological, technological, regulatory and policy related, and human factor related. Some of the barriers are quite similar, especially concerning the technologies. Studies focusing on the AI usage for HTA decision making are scarce. AI and augmented decision making tools are a novel science, and we are in the process of adapting it to existing needs. HTA as a process requires multiple steps, multiple evaluations which rely on heterogenous data. Therefore, the observed range of barriers come as a no surprise, and experts in the field need to give their opinion on the most important barriers in order to develop recommendations to overcome them and to disseminate the practical application of these tools.

Potential Barriers of Patient Involvement in Health Technology Assessment in Central and Eastern European Countries.

Patients' perspectives are important to identify preferences, estimate values and appreciate unmet medical needs in the process of research and development and subsequent assessment of new health technologies. Patient and public involvement in health technology assessment (HTA) is essential in understanding and assessing wider implications of coverage and reimbursement decisions for patients, their relatives, caregivers, and the general population. There are two approaches to incorporating the patients' voice in HTA, preferably used in a mix. In the first one, patients, caregivers and/or their representatives directly participate at discussions in different stages of the HTA process, often at the same table with other stakeholders. Secondly, patient involvement activities can be supported by evidence on patient value and experience collected directly from patients, caregivers and/or their representatives often by patient groups Patient involvement practices, however, are limited in Central and Eastern European (CEE) countries without clear methodology or regulatory mechanisms to guide patient involvement in the HTA process. This poses the question of transferability of practices used in other countries, and might call for the development of new CEE-specific guidelines and methods. In this study we aim to map potential barriers of patient involvement in HTA in countries of the CEE region.

Delayed payment schemes in Central-Eastern Europe and Middle-East.

The need for innovative payment models for health technologies with high upfront costs has emerged due to affordability concerns across the world. Early technology adopter countries have been experimenting with delayed payment schemes. Our objective included listing potential barriers for implementing delayed payment models and recommendations on how to address these barriers in lower income countries of Central and Eastern Europe (CEE) and the Middle East (ME). We conducted a survey, an exploratory literature review and an iterative brainstorming about potential barriers and solutions to implement delayed payment models in these two regions. A draft list of recommendations was validated in a virtual workshop with payer experts from the two regions. Eight barriers were identified in 4 areas, including transaction costs and administrative burden, payment schedule, information technology and data infrastructure, and governance. Fifteen practical recommendations were prepared to address these barriers, including recommendations that are specific to lower income countries, and recommendations that can be applied more universally, but are more crucial in countries with severe budget constraints. Conclusions of this policy research can be considered as an initial step in a multistakeholder dialogue about implementing delayed payment schemes in CEE and ME countries.

Using real-world evidence in healthcare from Western to Central and Eastern Europe: a review of existing barriers.

As part of the HTx (Next Generation Health Technology Assessment) project, this study was aimed at identifying the main barriers for application of real-world evidence (RWE) for the purposes of health technology assessment in the Central and Eastern European countries. A mixed methods approach was employed to identify the main barriers: a scoping review of the literature and a series of discussions with stakeholders. Based on the applied approaches, we attempted to summarize the main barriers and challenges related to transferability of RWE in five main groups: technical, regulatory, clinical, scientific and perceptional barriers. Further research should pursue the development of detailed, consensus-based guidelines to improve the harmonization and standardization of RWE.

Should Hungary Pay More for a QALY Gain than Higher-Income Western European Countries?

OBJECTIVES: Cost-effectiveness thresholds (CETs) play a particularly important role in the reimbursement decisions of health technologies in countries with limited healthcare resources. Our goal is to develop a scientifically solid proposal for a revised cost-effectiveness threshold, as part of the planned review of the Hungarian health economic guidance. METHODS: The Threshold Working Group of the Hungarian Health Economics Association performed a targeted review on CETs in European countries. International trends on CETs served as a basis for our recommendation, which was discussed at the Association's workshop and deliberated at an expert committee meeting with representatives from the national health technology assessment (HTA) and healthcare payer bodies, and academic HTA centres. RESULTS: The current Hungarian CET is one of the highest among European countries relative to GDP per capita, and even higher in nominal value than the CET applied by NICE. As opposed to the current, single Hungarian threshold, other European countries apply multiple thresholds. The Working Group recommends that Hungary should also apply multiple CETs in the range of 1.5-3 times GDP per capita with stratification according to the relative quality-adjusted life-year (QALY) gain of the new technology. In addition, multiple CETs in the range of 3-10 times GDP per capita is recommended for technologies in rare diseases. CONCLUSIONS: CETs should be aligned with the country's economic performance and should reflect societal preferences. Our recommendation may increase the efficiency of healthcare resource allocation in Hungary by strengthening the role of HTA in the reimbursement decisions and favouring new technologies with higher QALY gain.

Country score tool to assess readiness and guide evidence generation of immunization programs in aging adults in Europe.

Objectives: Delaying of policies for immunization of aging adults, low vaccine uptake, and the lack of supportive evidence at the national level could diminish the value in health and economics of such programs. This study aims to develop a "country score tool" to assess readiness and to facilitate evidence generation for aging adult immunization programs in Europe, and examine the comprehensiveness, relevance, acceptability, and feasibility of the tool. Methods: The tool was developed in two phases. First, a modified Delphi process was used to construct the tool. The process included a literature review, stakeholder consultations, and a three-round Delphi study. The Delphi panel included researchers, supra-national and national decision-makers of immunization programs recruited from five countries, using snowball sampling method. The consensus was predefined at the agreement rate of 70%. Pilot testing of the tool was conducted in the Netherlands, Germany, Serbia, and Hungary involving researchers in the field of health technology assessment. After assessing the countries' readiness, researchers evaluated four features, namely comprehensiveness, relevance, acceptability, and feasibility of the tool via an online survey that included 5-scale Likert questions. The percentages of affirmative answers including "agree" and "totally agree" choices were presented. Results: The review identified 16 tools and frameworks that formed the first version of our tool with 14 items. Eight experts were involved in the Delphi panel. Through three Delphi rounds, four items were added, one was dropped, and all others were amended. The consensus was achieved on the tool with 17 items divided into decision-making and implementation parts. Each item has a guiding question, corresponding to explanations and rationales to inform assessment with readiness scores. Eight researchers completed the pilot testing. The tool was rated as comprehensive (75%), relevant (100%), acceptable (75%), and feasible (88%) by participants. Conclusion: Through a thorough and transparent process, a country score tool was developed helping to identify strengths, weaknesses, and evidential requirements for decision-making and implementation of immunization programs of aging adults. The tool is relevant for different European contexts and shows good comprehensiveness, acceptability, and feasibility.

Criteria and Scoring Functions Used in Multi-criteria Decision Analysis and Value Frameworks for the Assessment of Rare Disease Therapies: A Systematic Literature Review.

BACKGROUND: Traditionally, the economic value of health technologies is assessed with cost-effectiveness (CE) and budget impact (BI) analyses. However, the evaluation of rare disease therapies often considers novel value criteria. Multi-criteria decision analysis (MCDA) is a promising tool in the assessment of value criteria that typically cannot be captured with traditional approaches. OBJECTIVES: The objective of this research was to investigate the criteria and scoring functions applied in value frameworks and MCDA tools relevant to the evaluation of rare disease therapies. The aim was to gain a better understanding of the domains and measurement of commonly referenced novel value criteria. METHODS: A systematic literature review was performed covering the period from 2013 to 2019. MCDA or value framework articles and structured review papers on orphan-drug-specific MCDA articles were reviewed. Information sources included MEDLINE, Embase, Scopus, and 26 other gray literature sources. A descriptive review of identified criteria and scoring functions was performed, with special focus on "novel" value criteria that are traditionally not considered in CE or BI analyses. RESULTS: In total, 15 relevant value frameworks and MCDA tools were identified. These studies included a large number (n = 56) of individual value criteria. The most commonly included novel criteria were unmet medical need, severity of disease, and reduction in uncertainty. The identified scoring functions (measurement methods) for novel criteria were highly heterogeneous and tailored. Standardized scoring functions were not observed. Additionally, the studies did not provide their rationale for choosing a specific scoring function for a criterion. CONCLUSIONS: MCDA is a promising tool to include novel value criteria into the health technology assessment of therapies for rare diseases. To support the development of a transparent and justified evaluation process, scoring functions should be further investigated.