An overview of the perspectives used in health economic evaluations.

The term 'perspective' in the context of economic evaluations and costing studies in healthcare refers to the viewpoint that an analyst has adopted to define the types of costs and outcomes to consider in their studies. However, there are currently notable variations in terms of methodological recommendations, definitions, and applications of different perspectives, depending on the objective or intended user of the study. This can make it a complex area for stakeholders when interpreting these studies. Consequently, there is a need for a comprehensive overview regarding the different types of perspectives employed in such analyses, along with the corresponding implications of their use. This is particularly important, in the context of low-and-middle-income countries (LMICs), where practical guidelines may be less well-established and infrastructure for conducting economic evaluations may be more limited. This article addresses this gap by summarising the main types of perspectives commonly found in the literature to a broad audience (namely the patient, payer, health care providers, healthcare sector, health system, and societal perspectives), providing their most established definitions and outlining the corresponding implications of their uses in health economic studies, with examples particularly from LMIC settings. We then discuss important considerations when selecting the perspective and present key arguments to consider when deciding whether the societal perspective should be used. We conclude that there is no one-size-fits-all answer to what perspective should be used and the perspective chosen will be influenced by the context, policymakers'/stakeholders' viewpoints, resource/data availability, and intended use of the analysis. Moving forward, considering the ongoing issues regarding the variation in terminology and practice in this area, we urge that more standardised definitions of the different perspectives and the boundaries between them are further developed to support future studies and guidelines, as well as to improve the interpretation and comparison of health economic evidence.

Telemedicine Utilization Patterns and Implications Amidst COVID-19 Outbreaks in Thailand Under Public Universal Coverage Scheme.

During COVID-19 pandemic, telemedicine was a strategy to facilitate healthcare service delivery minimizing the risk of direct exposure among people. In Thailand, the National Health Security Office has included telemedicine services under the Universal Coverage Scheme to support social distancing policies to reduce the spread of COVID-19. This study aimed to determine the patterns of telemedicine service use during major COVID-19 outbreaks including Alpha, Delta, and Omicron in Thailand. We retrospectively analyzed a dataset of telemedicine e-claims from the National Health Security Office, which covers services reimbursed under the Universal Coverage Scheme between December 2020 and August 2022. An interrupted time-series analysis, Pearson correlation analysis and binary logistic regression were performed. Almost 70% of the patients using telemedicine services were over 40 years old. Most patients used services for mental health problems (25.6%) and major noncommunicable diseases, including essential hypertension (12.6%) and diabetes mellitus (9.2%). The daily number of using telemedicine service was strongly correlated with the number of COVID-19 new cases detected. An immediate change in the trend of using telemedicine was detected at the onset of outbreaks along with the surge of infection. The follow-up use of telemedicine services was not substantial among female, older adults patients and those with non-communicable diseases except mental health problems, and infectious diseases. Strategies need to be developed to reinforced healthcare resources for telemedicine during the surge of outbreaks and sustain the use of telemedicine services for chronic and infectious diseases, regardless of the pandemic, and promote the efficiency of healthcare systems.

Cost-effectiveness analysis of typhoid vaccination in Lao PDR.

BACKGROUND: Typhoid vaccination has been shown to be an effective intervention to prevent enteric fever and is under consideration for inclusion in the national immunization program in Lao PDR. METHODS: A cost-utility analysis was performed using an age-structured static decision tree model to estimate the costs and health outcomes of introducing TCV. Vaccination strategies combined with five delivery approaches in different age groups compared to no vaccination were considered from the societal perspective, using the Gavi price of 1.5 USD per dose. The vaccination program was considered to be cost-effective if the incremental cost-effectiveness ratio was less than a threshold of 1 GDP per capita for Lao PDR, equivalent to USD 2,535 in 2020. RESULTS: In the model, we estimated 172.2 cases of enteric fever, with 1.3 deaths and a total treatment cost of USD 7,244, based on a birth cohort of 164,662 births without TCV vaccination that was followed over their lifetime. To implement a TCV vaccination program over the lifetime horizon, the estimated cost of the vaccine and administration costs would be between USD 470,934 and USD 919,186. Implementation of the TCV vaccination program would prevent between 14 and 106 cases and 0.1 to 0.8 deaths. None of the vaccination programs appeared to be cost-effective. CONCLUSIONS: Inclusion of TCV in the national vaccination program in Lao PDR would only be cost-effective if the true typhoid incidence is 25-times higher than our current estimate.

Expanding access to high-cost medicines under the Universal Health Coverage scheme in Thailand: review of current practices and recommendations.

BACKGROUND: There has been an increasing demand to reimburse high-cost medicines, through public health insurance schemes in Thailand. METHODS: A mixed method approach was employed. First, a rapid review of select high-income countries was conducted, followed by expert consultations and an in-depth review of three countries: Australia, England and Republic of Korea to understand reimbursement mechanisms of high-cost medicines. In Thailand, current pathways for reimbursing high-cost medicines reviewed, the potential opportunity cost estimated, and stakeholder consultations were conducted to identify context specific considerations. RESULTS: High-income countries reviewed have implemented a variety of pathways and mechanisms for reimbursing high-cost medicines under specific eligibility criteria, listing processes, varying cost-effectiveness thresholds and special funding arrangements. In Thailand, high-cost medicines that do not offer good value-for-money are excluded from the reimbursement process. A framework for reimbursing high-cost medicines that are not cost-effective at the current willingness-to-pay threshold was proposed for Thailand. Under this framework, specific criteria are proposed to determine their eligibility for reimbursement such life-saving nature, treatment of conditions with no alternative treatment options, and affordability. CONCLUSION: High-cost medicines may become eligible for reimbursement through alternative mechanisms based on specific criteria which depend on each context. The application of HTA methods and processes is important in guiding these decisions to support sustainable access to affordable healthcare in pursuit of Universal Health Coverage (UHC).

The impact of universal health coverage and COVID-19 pandemic on out-of-pocket expenses in Thailand: an analysis of household survey from 1994 to 2021.

OBJECTIVES: After Thailand achieved Universal Health Coverage (UHC) in 2002, the extent of financial risk protection has not been assessed in the long term, especially after the COVID-19 pandemic. Therefore, this study aims to revisit the impact of UHC on out-of-pocket expenses (OOPE) for health and to descriptively explore the impact of COVID-19 on OOPE. METHODS: This study was a secondary data analysis and used data from the Socio-Economic Survey from 1994 to 2021 in Thailand. The effect of UHC on the percentage of OOPE in total health expenditures (THE) from 1994 to 2019 was investigated with an interrupted time-series analysis. Descriptive analyses of OOPE in absolute value during the COVID-19 were conducted. RESULTS: The percentage of OOPE in THE significantly decreased both before (ß -2.02%; 95% CI: -2.70% to - 1.33%) and during (ß 1.41%; 95% CI: 0.70% to 2.11%) the UHC period. During the pandemic, total household OOPE for medical equipment was found to have rapidly increased from 643 million THB in 2019 to 9.4 billion THB in 2020. CONCLUSIONS: The trend of providing financial risk protection (measured by OOPE/THE) in Thailand continues until 2019. Providing medical equipment in sufficient and equally accessible manners should be prioritized during the future pandemic.

Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore.

Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots' significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

Economic Analysis of Border Control Policies during COVID-19 Pandemic: A Modelling Study to Inform Cross-Border Travel Policy between Singapore and Thailand.

With countries progressing towards high COVID-19 vaccination rates, strategies for border reopening are required. This study focuses on Thailand and Singapore, two countries that share significant tourism visitation, to illustrate a framework for optimizing COVID-19 testing and quarantine policies for bilateral travel with a focus on economic recovery. The timeframe is the month of October 2021, when Thailand and Singapore were preparing to reopen borders for bilateral travel. This study was conducted to provide evidence for the border reopening policy decisions. Incremental net benefit (INB) compared to the pre-opening period was quantified through a willingness-to-travel model, a micro-simulation COVID-19 transmission model and an economic model accounting for medical and non-medical costs/benefits. Multiple testing and quarantine policies were examined, and Pareto optimal (PO) policies and the most influential components were identified. The highest possible INB for Thailand is US $125.94 million, under a PO policy with no quarantine but with antigen rapid tests (ARTs) pre-departure and upon arrival to enter both countries. The highest possible INB for Singapore is US $29.78 million, under another PO policy with no quarantine on both sides, no testing to enter Thailand, and ARTs pre-departure and upon arrival to enter Singapore. Tourism receipts and costs/profits of testing and quarantine have greater economic impacts than that from COVID-19 transmission. Provided healthcare systems have sufficient capacity, great economic benefits can be gained for both countries by relaxing border control measures.

Which interventions for alcohol use should be included in a universal healthcare benefit package? An umbrella review of targeted interventions to address harmful drinking and dependence.

BACKGROUND: This study aimed to identify targeted interventions for the prevention and treatment of harmful alcohol use. Umbrella review methodology was used to summarise the effectiveness across a broad range of interventions, in order to identify which interventions should be considered for inclusion within universal health coverage schemes in low- and middle-income countries. METHODS AND FINDINGS: We included systematic reviews with meta-analysis of randomised controlled trials (RCTs) on targeted interventions addressing alcohol use in harmful drinkers or individuals with alcohol use disorder. We only included outcomes related to alcohol consumption, heavy drinking, binge drinking, abstinence, or alcohol-attributable accident, injury, morbidity or mortality. PubMed, Embase, PsycINFO, Cochrane Database of Systematic Reviews, and the International HTA Database were searched from inception to 3 September 2021. Risk of bias of reviews was assessed using the AMSTAR2 tool. After reviewing the abstracts of 9,167 articles, results were summarised narratively and certainty in the body of evidence for each intervention was assessed using GRADE. In total, 86 studies met the inclusion criteria, of which the majority reported outcomes for brief intervention (30 studies) or pharmacological interventions (29 studies). Overall, methodological quality of included studies was low. CONCLUSIONS: For harmful drinking, brief interventions, cognitive behavioural therapy, and motivational interviewing showed a small effect, whereas mentoring in adolescents and children may have a significant long-term effect. For alcohol use disorder, social network approaches and acamprosate showed evidence of a significant and durable effect. More evidence is required on the effectiveness of gamma-hydroxybutyric acid (GHB), nalmefene, and quetiapine, as well as optimal combinations of pharmacological and psychosocial interventions. As an umbrella review, we were unable to identify the extent to which variation between studies stemmed from differences in intervention delivery or variation between country contexts. Further research is required on applicability of findings across settings and best practice for implementation. Funded by the Thai Health Promotion Foundation, grant number 61-00-1812.

Fifteen Lessons from Fifteen Years of the Health Intervention and Technology Assessment Program in Thailand.

The Health Intervention and Technology Assessment Program (HITAP) was established in 2007. This article highlights 15 lessons from over 15 years of experience, noting five achievements about what HITAP has done well, five areas that it is currently working on, and five aims for work in the future. HITAP built capacity for HTA and linked research to policy and practice in Thailand. With collaborators from academic and policy spheres, HITAP has mobilized regional and global support, and developed global public goods to enhance the field of HTA. HITAP's semi-autonomous structure has facilitated these changes, though they have not been without their challenges. HITAP aims to continue its work on HTA for public health interventions and disinvestments, effectively engaging with stakeholders and strategically managing its human resources. Moving forward, HITAP will develop and update global public goods on HTA, work on emerging topics such as early HTA, address issues in digital health, real-world evidence and equity, support HTA development globally, particularly in low-income settings, and seek to engage more effectively with the public. HITAP seeks to learn from its experience and invest in the areas identified so that it can grow sustainably. Its journey may be relevant to other countries and institutions that are interested in developing HTA programs.

International Partnerships to Develop Evidence-informed Priority Setting Institutions: Ten Years of Experience from the International Decision Support Initiative (iDSI).

All health systems must set priorities. Evidence-informed priority-setting (EIPS) is a specific form of systematic priority-setting which involves explicit consideration of evidence to determine the healthcare interventions to be provided. The international Decision Support Initiative (iDSI) was established in 2013 as a collaborative platform to catalyze faster progress on EIPS, particularly in low- and middle-income countries. This article summarizes the successes, challenges, and lessons learned from ten years of iDSI partnering with countries to develop EIPS institutions and processes. This is a thematic documentary analysis, structured by iDSI's theory of change, extracting successes, challenges, and lessons from three external evaluations and 19 internal reports to funders. We identified three phases of iDSI's work-inception (2013-15), scale-up (2016-2019), and focus on Africa (2019-2023). iDSI has established a global platform for coordinating EIPS, advanced the field, and supported regional networks in Asia and Africa. It has facilitated progress in securing high-level commitment to EIPS, strengthened EIPS institutions, and developed capacity for health technology assessments. This has resulted in improved decisions on service provision, procurement, and clinical care. Major lessons learned include the importance of sustained political will to develop EIPS; a clear EIPS mandate; inclusive governance structures appropriate to health financing context; politically sensitive and country-led support to EIPS, taking advantage of policy windows for EIPS reforms; regional networks for peer support and long-term sustainability; utilization of context appropriate methods such as adaptive HTA; and crucially, donor-funded global health initiatives supporting and integrating with national EIPS systems, not undermining them.

How can health technology assessment support our response to public health emergencies?

Public health emergencies (PHEs), such as the COVID-19 crisis, are threats to global health and public order. We recommend that countries bolster their PHE responses by investing in health technology assessment (HTA), defined as a systematic process of gathering pertinent information on and evaluating health technologies from a medical, economic, social and ethical standpoint. We present examples of how HTA organizations in low- and middle-income countries have adapted to supporting PHE-related decisions during COVID-19 and describe the ways HTA can help the response to a PHE. In turn, we advocate for HTA capacity to be further developed globally and for increased institutional acceptance of these methods as a building block for preparedness and response to future PHEs. Finally, the long-term potential of HTA in strengthening health systems and embedding confidence and transparency into scientific policy should be recognized.

The Relative Importance of Vulnerability and Efficiency in COVID-19 Contact Tracing Programmes: A Discrete Choice Experiment.

Objectives: This study aims to assess the trade-offs between vulnerability and efficiency attributes of contact tracing programmes based on preferences of COVID-19 contact tracing practitioners, researchers and other relevant stakeholders at the global level. Methods: We conducted an online discrete choice experiment (DCE). Respondents were recruited globally to explore preferences according to country income level and the prevailing epidemiology of COVID-19 in the local setting. The DCE attributes represented efficiency (timeliness, completeness, number of contacts), vulnerability (vulnerable population), cooperation and privacy. A mixed-logit model and latent class analysis were used. Results: The number of respondents was 181. Timeliness was the most important attribute regardless of country income level and COVID-19 epidemiological condition. Vulnerability of contacts was the second most important attribute for low-to-lower-middle-income countries and third for upper-middle-to-high income countries. When normalised against conditional relative importance of timeliness, conditional relative importance of vulnerability ranged from 0.38 to 0.42. Conclusion: Vulnerability and efficiency criteria were both considered to be important attributes of contact tracing programmes. However, the relative values placed on these criteria varied significantly between epidemiological and economic context.

What Can We Learn From Others to Develop a Regional Centre for Infectious Diseases in ASEAN? Comment on "Operationalising Regional Cooperation for Infectious Disease Control: A Scoping Review of Regional Disease Control Bodies and Networks".

The coronavirus disease 2019 (COVID-19) pandemic has brought the need for regional collaboration on disease prevention and control to the fore. The review by Durrance-Bagale et al offers insights on the enablers, barriers and lessons learned from the experience of various regional initiatives. Translating these lessons into action, however, remains a challenge. The Association of Southeast Asian Nations (ASEAN) planned to establish a regional centre for disease control; however, many factors have slowed the realisation of these efforts. Going forward, regional initiatives should be able to address the complexity of emerging infectious diseases through a One Health approach, assess the social and economic impact of diseases on the region and study the real-world effectiveness of regional collaborations. The initiatives should seek to be inclusive of stakeholders including those from the private sector and should identify innovative measures for financing. This advancement will enable regions such as ASEAN to effectively prepare for the next pandemic.

What is the value of explicit priority setting for health interventions? A simulation study.

Many countries seek to secure efficiency in health spending through establishing explicit priority setting institutions (PSIs). Since such institutions divert resources from frontline services which benefit patients directly, it is legitimate and reasonable to ask whether they are worth the money. We address this question by comparing, through simulation, the health benefits and costs from implementing two alternative funding approaches - one scenario in which an active PSI enables cost-effectiveness-threshold based funding decisions, and a counterfactual scenario where there is no PSI. We present indicative results for one dataset from the United Kingdom (published in 2015) and one from Malawi (published in 2018), which show that the threshold rule reliably resulted in decreased health system costs, improved health benefits, or both. Our model is implemented in Microsoft Excel and designed to be user-friendly, and both the model and a user guide are made publicly available, in order to enable others to parameterise the model based on the local setting. Although inevitably stylised, we believe that our modelling and results offer a valid perspective on the added value of explicit PSIs.

Factors associated with the opposition to COVID-19 vaccination certificates: A multi-country observational study from Asia.

BACKGROUND: There are ongoing calls to harmonise and increase the use of COVID-19 vaccination certificates (CVCs) in Asia. Identifying groups in Asian societies who oppose CVCs and understanding their reasons can help formulate an effective CVCs policy in the region. However, no formal studies have explored this issue in Asia. METHOD: The COVID-19 Vaccination Policy Research and Decision-Support Initiative in Asia (CORESIA) was established to address policy questions related to CVCs. An online cross-sectional survey was conducted from June to October 2021 in nine Asian countries. Multivariable logistical regression analyses were performed to identify potential opposers of CVCs. RESULTS: Six groups were identified as potential opposers of CVCs: (i) unvaccinated (Odd Ratio (OR): 2.01, 95% Confidence Interval (CI): 1.65-2.46); vaccine hesitant and those without access to COVID-19 vaccines; (ii) those not wanting existing NPIs to continue (OR: 2.97, 95% CI: 2.51-3.53); (iii) those with low level of trust in governments (OR: 1.25, 95% CI: 1.02-2.52); (iv) those without travel plans (OR: 1.58, 95% CI: 1.31-1.90); (v) those expecting no financial gains from CVCs (OR: 2.35, 95% CI: 1.98-2.78); and (vi) those disagreeing to use CVCs for employment, education, events, hospitality, and domestic travel. CONCLUSIONS: Addressing recurring public health bottlenecks such as vaccine hesitancy and equitable access, adherence to policies, public trust, and changing the narrative from 'societal-benefit' to 'personal-benefit' may be necessary and may help increase wider adoption of CVCs in Asia.

Monitoring and Evaluation of National Vaccination Implementation: A Scoping Review of How Frameworks and Indicators Are Used in the Public Health Literature.

An effective Monitoring and Evaluation (M&E) framework helps vaccination programme managers determine progress and effectiveness for agreed indicators against clear benchmarks and targets. We aimed to identify the literature on M&E frameworks and indicators used in national vaccination programmes and synthesise approaches and lessons to inform development of future frameworks. We conducted a scoping review using Arksey and O'Malley's six-stage framework to identify and synthesise sources on monitoring or evaluation of national vaccination implementation that described a framework or indicators. The findings were summarised thematically. We included 43 eligible sources of 4291 screened. Most (95%) were in English and discussed high-income (51%) or middle-income (30%) settings, with 13 in Europe (30%), 10 in Asia-Pacific (23%), nine in Africa (21%), and eight in the Americas (19%), respectively, while three crossed regions. Only five (12%) specified the use of an M&E framework. Most (32/43; 74%) explicitly or implicitly included vaccine coverage indicators, followed by 12 including operational (28%), five including clinical (12%), and two including cost indicators (5%). The use of M&E frameworks was seldom explicit or clearly defined in our sources, with indicators rarely fully defined or benchmarked against targets. Sources focused on ways to improve vaccination programmes without explicitly considering ways to improve assessment. Literature on M&E framework and indicator use in national vaccination programmes is limited and focused on routine childhood vaccination. Therefore, documentation of more experiences and lessons is needed to better inform vaccination M&E beyond childhood.

Recalibrating the notion of modelling for policymaking during pandemics.

COVID-19 disease models have aided policymakers in low-and middle-income countries (LMICs) with many critical decisions. Many challenges remain surrounding their use, from inappropriate model selection and adoption, inadequate and untimely reporting of evidence, to the lack of iterative stakeholder engagement in policy formulation and deliberation. These issues can contribute to the misuse of models and hinder effective policy implementation. Without guidance on how to address such challenges, the true potential of such models may not be realised. The COVID-19 Multi-Model Comparison Collaboration (CMCC) was formed to address this gap. CMCC is a global collaboration between decision-makers from LMICs, modellers and researchers, and development partners. To understand the limitations of existing COVID-19 disease models (primarily from high income countries) and how they could be adequately support decision-making in LMICs, a desk review of modelling experience during the COVID-19 and past disease outbreaks, two online surveys, and regular online consultations were held among the collaborators. Three key recommendations from CMCC include: A 'fitness-for-purpose' flowchart, a tool that concurrently walks policymakers (or their advisors) and modellers through a model selection and development process. The flowchart is organised around the following: policy aims, modelling feasibility, model implementation, model reporting commitment. Holmdahl and Buckee (2020) A 'reporting standards trajectory', which includes three gradually increasing standard of reports, 'minimum', 'acceptable', and 'ideal', and seeks collaboration from funders, modellers, and decision-makers to enhance the quality of reports over time and accountability of researchers. Malla et al. (2018) A framework for "collaborative modelling for effective policy implementation and evaluation" which extends the definition of stakeholders to funders, ground-level implementers, public, and other researchers, and outlines how each can contribute to modelling. We advocate for standardisation of modelling processes and adoption of country-owned model through iterative stakeholder participation and discuss how they can enhance trust, accountability, and public ownership to decisions.

Avoiding health technology assessment: a global survey of reasons for not using health technology assessment in decision making.

INTRODUCTION: Despite the documented benefits of using health technology assessments (HTA) to inform resource allocation in health care systems, HTA remains underused, especially in low- and middle-income countries. A survey of global health practitioners was conducted to reveal the top reasons ("excuses") that they had heard from colleagues, policymakers or other stakeholders for not using HTA in their settings. METHODS: There were 193 respondents to the survey. Most responses were from individuals in research organisations (37%), ministries of health (27%) and other government agencies (14%). Participants came from Southeast Asia (40%), the Western Pacific (30%), Africa (15%), Europe (7%), the Americas (7%) and the Eastern Mediterranean region (2%). RESULTS: The top five reasons encountered by respondents related to lack of data, lack of technical skills for HTA, the technocratic nature of the work, the lack of explicit decision rules and the perception that HTA puts a "price on life". CONCLUSIONS: This study aimed to understand and address the top reasons for not using HTA. They fall into three categories: (1) misconceptions about HTA; (2) feasibility issues; and (3) values, attitudes and politics. Previous literature has shown that these reasons can be addressed when identified, and even imperfect HTA analyses can provide useful information to a decision-maker.

Assessing the performance of health technology assessment (HTA) agencies: developing a multi-country, multi-stakeholder, and multi-dimensional framework to explore mechanisms of impact.

BACKGROUND: Health technology assessment (HTA) agencies have an important role to play in managing the rising demands on health systems. However, creating and running such agencies potentially diverts resources from frontline services. A large number of studies address the question of 'what is the impact of HTA?'. Several points of heterogeneity in this literature include: purpose of the study, definition of HTA, definition of impact, and scope and rigour of evaluations. Our study seeks to address several limitations in this literature. This study aims to explore the mechanisms of impact of an HTA agency. In doing so, we consider HTA as an institution rather than a knowledge product to build an impact evaluation framework from an international, multi-stakeholder and multi-dimensional perspective. METHODS: We conducted 9 key informant interviews with experts from the international HTA community. We addressed several questions, informed by existing frameworks of impact within the literature, to understand their perspectives on the mechanisms of impact of an HTA agency. We analyse data using logic modelling and impact mapping, as tools to understand and visualise mechanisms of change. FINDINGS: Our impact mapping highlights several distinct, but not necessarily mutually exclusive, mechanisms through which the overall impact of an HTA agency is achieved. These are: the effective conduct of HTA studies; effective use of HTA in agenda-setting and policy formulation processes; effective engagement and external communications; good institutional reputation and fit within the healthcare and policy-making system; effective use of HTA as a tool for the negotiation of health technology prices; and the effective implementation of policy change regarding health technologies. We also identify indicators of these effects. CONCLUSIONS: Our findings and resulting evaluation framework complement and add to existing literature by offering a new perspective on the mechanisms by which HTA agencies generate impact. This new perspective considers HTA as an institution rather than a knowledge product, is international, multi-dimensional, and includes multi-stakeholder views. We hope the analysis will be useful to countries interested in managing HTA performance.