Toward the Implementation of a Value Framework for Diagnostic Technologies: Operationalization, Piloting, and Validation in Latin America.

OBJECTIVES: In 2020, a group of 30 stakeholders from Latin America established 15 criteria for a diagnostic technologies value framework (D-VF) to help assess and inform decisions on diagnostic technologies. This article aims to present the operationalization, piloting, and initial validation of the framework for its implementation. METHODS: This work was carried out collaboratively with a variety of stakeholders. Three sequential phases were undertaken: (1) operationalization of the D-VF through a literature search for conceptual definitions and assessment tools, (2) piloting of the D-VF through a rapid health technology assessment document applying the methodology of the framework, and (3) a face validation process conducted through a virtual workshop, where usefulness and implementation aspects of the framework were assessed. RESULTS: The operationalization of the framework was conducted, and a methodological user guide was published. The D-VF criteria were applied in a health technology assessment document on human papilloma virus testing in cervical cancer screening. Also, an open-access training program was developed. Stakeholders agreed on the usefulness of the D-VF for assessment and decision-making stages of diagnostic technologies. However, they highlighted the need to improve technical capacities and the potential for added complexity when applying a D-VF with many criteria. The absence of an established value framework for diagnostic technologies in Latin America and the potential for strengthening technical capacities made the project valuable to those involved. CONCLUSIONS: The diagnostic technologies value framework was shown to be fit for implementation in real-life decision-making settings after the operationalization, piloting, and initial validation phases. Further experiences are important to support its implementation.

Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force.

This ISPOR Good Practices report provides a framework for assessing the suitability of electronic health records data for use in health technology assessments (HTAs). Although electronic health record (EHR) data can fill evidence gaps and improve decisions, several important limitations can affect its validity and relevance. The ISPOR framework includes 2 components: data delineation and data fitness for purpose. Data delineation provides a complete understanding of the data and an assessment of its trustworthiness by describing (1) data characteristics; (2) data provenance; and (3) data governance. Fitness for purpose comprises (1) data reliability items, ie, how accurate and complete the estimates are for answering the question at hand and (2) data relevance items, which assess how well the data are suited to answer the particular question from a decision-making perspective. The report includes a checklist specific to EHR data reporting: the ISPOR SUITABILITY Checklist. It also provides recommendations for HTA agencies and policy makers to improve the use of EHR-derived data over time. The report concludes with a discussion of limitations and future directions in the field, including the potential impact from the substantial and rapid advances in the diffusion and capabilities of large language models and generative artificial intelligence. The report's immediate audiences are HTA evidence developers and users. We anticipate that it will also be useful to other stakeholders, particularly regulators and manufacturers, in the future.

Real-world evidence: experiences and challenges for decision making in Latin America.

OBJECTIVE: The Health Technology Assessment (HTA) process aims to optimize health system funding of technologies. In recent years there has been an increase in what is known as Real-World Evidence (RWE) as a complement to clinical trials. The objective of Health Technology Assessment International's Latin American Policy Forum 2022 was to explore the utility of incorporating RWE into HTA and decision-making processes in the region. METHODS: This article is based on a background document, survey, and the deliberative work of the country representatives who participated in the Forum. RESULTS: There is a growing interest in the use of Real-World Data / Real-World Evidence in HTA processes in Latin America, although currently there are no specific local guidelines for RWE use by HTA agencies. At present, its use is limited to certain areas such as adding context to HTA reports, the evaluation of adverse events, or cost estimation.Potential future uses of RWE were identified, including the creation of risk-sharing agreements, the assessment of technology performance in routine practice, providing information on outcomes that are not so easily evaluated in clinical trials (e.g., the identification of specific subpopulations or quality of life), and the estimation of input parameters for economic evaluations. CONCLUSIONS: The participants agreed that there are several areas presenting significant potential to expand the application of RWD/RWE and that the development of normative frameworks for its use could be helpful.

Evidence-Informed Update of Argentina's Health Benefit Package: Application of a Rapid Review Methodology.

OBJECTIVES: Argentina has a fragmented healthcare system with social security covering almost two thirds of the population. Its benefit package-called compulsory medical program (PMO; by its Spanish acronym Programa Médico Obligatorio)-has not been formally and widely updated since 2005. However, laws, clinical practice guidelines (CPGs), and a high-cost technology reimbursement fund complement it. Our objective was to comprehensively review such a PMO and propose an update considering the corresponding complementary sources. METHODS: We followed four steps: (i) identification of health technologies from the current PMO and complementary sources, (ii) prioritization, (iii) assessment through rapid health technology assessment (HTA), and (iv) appraisal and recommendations. We evaluated three value domains: quality of evidence, net benefit, and economics, which were summarized in a five-category recommendation traffic-light scale ranging from a strong recommendation in favor of inclusion to a strong recommendation for exclusion. RESULTS: Eight hundred fifty technologies were identified; 164 of those, considered as high priority, were assessed through rapid HTAs. Those technologies mentioned in laws and CPGs were mostly outpatient essential medicines, whereas those from the reimbursement system were mostly high-cost drugs; of these 101 technologies, 50 percent were recommended to be kept in the PMO. The other 63 (identified by the Superintendence of Health Services, technology producers, and patients) were mostly medical procedures and high-cost drugs; only 25 percent of those resulted in a favorable recommendation. CONCLUSIONS: A methodology based on four clearly identified steps was used to carry out a comprehensive review of an outdated and fragmented benefit package. The use of rapid HTAs and a traffic-light recommendation framework facilitated the deliberative evidence-based update.

Deliberative processes in decision making informed by health technology assessment in Latin America.

OBJECTIVE: The objective of Health Technology Assessment International's 6th Latin America Policy Form, held in 2021, was to explore the implementation of deliberative processes in the framework of health technology assessment (HTA) and how agencies in the region could involve stakeholders in this process. METHODS: This paper is based on a preparatory survey, a background document, and the deliberative work of participants at the virtual Forum conducted in 2021. There were ninety-one participants in the open session and fifty-two in the closed sessions, representing twelve countries and diverse areas of the health sector. RESULTS: While there are mechanisms in most countries in Latin America to consider stakeholder involvement to some degree, it remains reduced or limited to a consultative role, making true participative involvement rare. There are significant barriers and structural and contextual limitations that have impeded or slowed progress toward deliberative processes. Relatively low levels of institutionalization and knowledge about HTA, as well as the lack of trust among stakeholders are important challenges. This situation has impacted health systems by diminishing the legitimacy of decisions and the very structures and processes of HTA. CONCLUSION: The Forum's broad group of participants identified barriers, facilitators, and recommendations to improve the use of deliberative processes in Latin America to foster improved fairness and reasonableness in HTA and decision making.

A Value Framework for the Assessment of Diagnostic Technologies: A Proposal Based on a Targeted Systematic Review and a Multistakeholder Deliberative Process in Latin America.

OBJECTIVES: there are very few value frameworks (VFs) to assess health technologies that are focused on diagnostic tests; they usually do not reflect a multistakeholder process; and they are all developed in high-income countries. Our project performed a targeted systematic review, with the objective of proposing an evidence-based, up-to-date VF informed by a multinational multistakeholder group working in the health technology assessment (HTA) space. METHODS: (1) A targeted systematic review, with the aim to identify existing VFs and their dimensions; and (2) generation a VF proposal through a mixed-methods, qualitative-quantitative approach. RESULTS: From 73 citations identified, 20 met our inclusion criteria and served to provide the initial list of dimensions for our VF. An initial list of criteria and subcriteria for a preliminary VF was proposed. After a full-day deliberative face-to-face meeting with 30 relevant stakeholders from seven Latin American countries and the United Kingdom, the final VF was defined, consisting of 15 criteria: five "essential or core," six highly relevant, three moderately relevant, and one of low relevance. Barriers and facilitators of value assessment of diagnostic technologies were also discussed. CONCLUSIONS: We propose a VF oriented to diagnostic technologies based on a targeted systematic review and a participatory process with key HTA stakeholders. It is the first to be produced in a lower and middle income setting but can also be potentially useful in other contexts aimed to assist decision-making processes with these particularly complex health technologies.

Identification and selection of health technologies for assessment by agencies in support of reimbursement decisions in Latin America.

OBJECTIVE: There is no health system that has the resources to evaluate all technologies. The presence of a clear process to prioritize health technologies for assessment by health technology assessment (HTA) agencies is a good practice principle recognized at the international level. The objective of Health Technology Assessment International's 2020 Latin American Policy Forum (LatamPF) was to explore how to improve the way HTA agencies in Latin America identify and prioritize technologies for assessment. METHODS: This paper is based on a background document, a survey, and the deliberations of the members of the LatamPF (forty-six participants from eleven countries) using a design thinking methodology. RESULTS: Participants agreed that a lack of clear prioritization mechanisms results in HTA processes and decisions that are perceived to be of low transparency and overly exposed to political or interest group pressures. The LatamPF identified barriers and recommended actions to improve HTA prioritization mechanisms in Latin America. The criteria identified as the most important to be taken into consideration by HTA agencies in the region when prioritizing a technology for assessment were: the burden of illness, the potential clinical benefit, the alignment with national health priorities, the potential impact on equity, a lack of treatment alternatives for patients, and the potential economic impact. CONCLUSIONS: Forum participants agreed that the establishment of transparent prioritization processes is a key element for all health systems. Improvements in these processes will strengthen HTA and provide greater legitimacy to decision making.

[Mecanismos de identificación y priorización de las tecnologías sanitarias a ser evaluadas por las Agencias de ETESA para decisiones de cobertura en Latino America].

OBJETIVO: Ningún sistema de salud cuenta con los recursos necesarios para evaluar todas las tecnologías. Contar con un proceso claro para priorizar qué tecnologías serán evaluadas por las agencias de evaluación de tecnologías sanitarias (ETESA) constituye un principio de buena práctica reconocido a nivel internacional. El objetivo del Foro de Políticas en Latino América (LatamPF) 2020 de Health Technology Assessment International fue explorar cómo puede mejorarse la forma en que las agencias de ETESA de Latino América identifican y priorizan las tecnologías a ser evaluadas. MÉTODOS: Este manuscrito está basado en un documento base, una encuesta, y en el trabajo deliberativo realizado por los miembros (cuarenta y seis participantes, once países) que participaron del LatamPF, a través de la metodología design-thinking. RESULTADOS: Los participantes coincidieron en que la falta de mecanismos claros de priorización trae como consecuencia una falta de legitimidad de las decisiones y procesos de ETESA, que son percibidos como poco transparentes y demasiado expuestos a presiones políticas o de grupos de interés. También se identificaron barreras y acciones para mejorar los mecanismos de priorización de ETESA en América Latina. Los criterios identificados como más importantes para ser tenidos en cuenta por las agencias de ETESA de la región al momento de priorizar una tecnología para ser evaluada fueron la carga de enfermedad, el potencial beneficio clínico, la alineación con prioridades de salud nacionales, el potencial impacto en la equidad, ausencia de otras alternativas para los pacientes, y el potencial impacto económico. CONCLUSIONES: Los participantes del Foro coincidieron en que el establecimiento de procesos transparentes de priorización es un elemento clave para todos los sistemas de salud. Las mejoras en este proceso fortalecerán la ETESA en Latino América y darán mayor legitimidad a sus decisiones.

Health and Economic Impact of Health Warnings and Plain Tobacco Packaging in Seven Latin American Countries: Results of a Simulation Model.

INTRODUCTION: The burden of disease attributable to tobacco use in Latin America is very high. Our objective was to evaluate the 10-year potential impact of current legislation related to cigarette packaging and warnings and expected effects of moving to a higher level of strategies implementing cigarette plain packaging on health and cost outcomes in Argentina, Bolivia, Brazil, Chile, Colombia, Mexico, and Peru, using a microsimulation model. AIMS AND METHODS: We used a probabilistic state-transition microsimulation model, considering natural history, costs, and quality of life losses associated with main tobacco-related diseases. We followed up individuals in hypothetical cohorts and calculated health outcomes annually to obtain aggregated long-term population health outcomes and costs. We performed a literature review to estimate effects and analyzed studies and information from ministries, relevant organizations, and national surveys. We calibrated the model comparing the predicted disease-specific mortality rates with local statistics. RESULTS: Current graphic warnings already in place in each country could avert, during 10 years, 69 369 deaths and 638 295 disease events, adding 1.2 million years of healthy life and saving USD 5.3 billion in the seven countries. If these countries implemented plain packaging strategies, additional 155 857 premature deaths and 4 133 858 events could be averted, adding 4.1 million healthy years of life and saving USD 13.6 billion in direct health care expenses of diseases attributable to smoking. CONCLUSIONS: Latin American countries should not delay the implementation of this strategy that will alleviate part of the enormous health and financial burden that tobacco poses on their economies and health care systems. IMPLICATIONS: Tobacco smoking is the single most preventable and premature mortality cause in the world. The Framework Convention on Tobacco Control, supported by the World Health Organization, introduced a package of evidence-based measures for tobacco control. This study adds evidence on the potential health effects and savings of implementing cigarette plain packaging in countries representing almost 80% of the Latin American population; findings are valuable resources for policy makers in the region.

The link between health technology assessment and decision making for the allocation of health resources in Latin America.

OBJECTIVE: One of the good practice principles for health technology assessment (HTA) is having a clear link between the assessment and decision making. The objective of the 2019 Latin American Policy Forum (LatamPF) of Health Technology Assessment International was to explore different models of connection between HTA and decision making and to discuss the potential applicability of such models in Latin America. METHODS: This paper is based on a background document and the deliberations of the members of the LatamPF (fifty-four participants from twelve countries) where a design-thinking methodology was used. RESULTS: The participants agreed that insufficient links between HTA and decision making undermine the legitimacy of decisions, expose the HTA process to excessive political and judicial influence, and promote the exclusion of some stakeholders from participating in the assessment process and decision making. High priority aspects of the HTA process that could feasibly be improved and which hold the greatest potential to generate positive changes in the health systems in the region were identified. The majority of these aspects were associated with the appropriate institutionalization of HTA, a greater degree of participation by different stakeholders, and improved transparency in the HTA process. CONCLUSIONS: The LatamPF identified barriers and recommended actions to strengthen the link between HTA and decision making. Participants emphasized that there is now a window of opportunity in the region as many societal actors see this as a priority. For this reason, health system stakeholders must take this opportunity to increase efforts toward strengthening the link between HTA and decision making.

[Vínculo Entre La Evaluación De Tecnologías Sanitarias y La Toma De Decisión Para La Asignación De Recursos Sanitarios en Latino-América].

OBJETIVOS: Un vínculo claro entre la evaluación y la toma de decisión constituye un principio de buena práctica en evaluación de tecnologías sanitarias (ETESA) reconocido a nivel internacional. El objetivo del Foro de Políticas en Latino-América (LatamPF) 2019 de Health Technology Assessment International fue explorar los diferentes modelos que vinculan la ETESA y la toma de decisión y discutir su potencial aplicabilidad en Latino-América. MÉTODOS: Este manuscrito está basado en un documento base y en el trabajo deliberativo realizado por los miembros (54 participantes, 12 países) que asistieron al LatamPF, a través de la metodología design thinking. RESULTADOS: Los participantes coincidieron en que la relación inapropiada entre la ETESA y la toma de decisión atenta hoy contra la legitimidad de las decisiones, expone al proceso de ETESA a una excesiva influencia política y judicial, y condiciona que algunos actores se sientan relegados del proceso de evaluación y toma de decisión. Se identificaron los atributos del proceso de ETESA más prioritarios y factibles de ser mejorados en la región, y con el mayor potencial para generar un cambio positivo en los sistemas de salud. La mayor parte de estos están vinculados con la apropiada institucionalización de la ETESA, ampliar la participación de los diferentes actores y mejorar la transparencia de los procesos de ETESA. CONCLUSIONES: El LatamPF ha identificado barreras y recomendado acciones para reforzar el vínculo entre ETESA y la decisión. A su vez, existe en estos momentos una ventana de oportunidad en la región, ya que el tema es visualizado como una prioridad por gran parte de los actores de la sociedad. Por ello, los diferentes actores de los sistemas sanitarios deberían ahora tomar esta oportunidad para avanzar en el fortalecimiento del vínculo entre ETESA y toma de decisión.

Real-World Evidence in Healthcare Decision Making: Global Trends and Case Studies From Latin America.

BACKGROUND: Real-world evidence (RWE) is increasingly used to inform health technology assessments for resource allocation, which are valuable tools for emerging economies such as in America. Nevertheless, the characteristics and uses in South America are unknown. OBJECTIVES: To identify sources, characteristics, and uses of RWE in Argentina, Brazil, Colombia, and Chile, and evaluate the context-specific challenges. The implications for future regulation and responsible management of RWE in the region are also considered. METHODS: A systematic literature review, database mapping, and targeted gray literature search were conducted to identify the sources and characteristics of RWE. Findings were validated by key opinion leaders attending workshops in 4 South American countries. RESULTS: A database mapping exercise revealed 407 unique databases. Geographic scope, database type, population, and outcomes captured were reported. Characteristics of national health information systems show efforts to collect interoperable data from service providers, insurers, and government agencies, but that initiatives are hampered by fragmentation, lack of stewardship, and resources. In South America, RWE is mainly used for pharmacovigilance and as pure academic research, but less so for health technology assessment decision making or pricing negotiations and not at all to inform early access schemes. CONCLUSIONS: The quality of real-world data in the case study countries vary and RWE is not consistently used in healthcare decision making. Authors recommend that future studies monitor the impact of digitalization and the potential effects of access to RWE on the quality of patient care.

Definiendo el valor de las tecnologías sanitarias en Latino-América: desarrollo de marcos de valor para informar la priorización de recursos sanitarios.

ResumenObjetivosEl reciente desarrollo de marcos de valor responde a una demanda por hacer más explícito e inclusivo el proceso de toma de decisiones sobre asignación de recursos sanitarios. El objetivo del Foro de Políticas en Latino América 2018 de Health Technology Assessment International fue explorar las experiencias internacionales y discutir la potencial aplicación de marcos de valor en la región.MétodosEste manuscrito está basado en un documento base y en las presentaciones y discusiones mantenidas por los miembros del Foro (43 participantes, 12 países) durante el Foro del 2018.ResultadosLos participantes coincidieron en que un proceso de evaluación y toma de decisión basado en marcos de valor más inclusivos y abarcativos podría mejorar la efectividad, eficiencia, sustentabilidad y equidad; promover la transparencia y una evaluación más completa de las tecnologías, mejorar la rendición de cuentas de las decisiones y la participación de actores. Se identificaron como criterios esenciales para ser incluidos en marcos de valor en la región a la carga de enfermedad y severidad de la condición, la efectividad y seguridad de la tecnología, la calidad de la evidencia, la costo-efectividad e impacto presupuestario. Los potenciales desafíos para su implementación identificados incluyeron la escasez de recursos humanos y las demoras en el proceso de evaluación que se podrían producir.ConclusionesLos participantes remarcaron que los próximos pasos deberían ser identificar procesos y metodologías apropiadas, adaptadas al contexto de cada país, para facilitar la aplicación de marcos de valor y mejorar el vínculo entre la evaluación y la toma de decisiones.

Defining the Value of Health Technologies in Latin America: Developments in Value Frameworks to Inform the Allocation of Healthcare Resources.

OBJECTIVES: The recent development of value frameworks to inform healthcare resource allocation responds to a demand to make the decision-making process more inclusive and explicit. The objectives of the 2018 Latin American (LAtam) Health Technology Assessment International (HTAi) Policy Forum were to explore the current international experiences and to discuss the potential application of value frameworks in Latin America. METHODS: A background paper, presentations, and group discussions of Policy Forum members (43 participants, 12 LAtam countries represented) at the 2018 HTAi Policy Forum meeting informed this paper. RESULTS: Participants agreed that HTA and decision making based on more comprehensive and inclusive value frameworks could improve health system effectiveness, efficiency, sustainability, and equity; promote transparency in the decision process; sustain a more comprehensive assessment of technologies; and facilitate stakeholder participation as well as accountability of decisions. Criteria that were identified as essential to be included in a value framework for LAtam were burden of illness and severity of the disease, effectiveness and safety of the technology, quality of the evidence, cost-effectiveness, and budget impact. Potential challenges identified for the application of value frameworks in LAtam, included scarcity of human resources and delays in the assessment process. CONCLUSIONS: Forum participants agreed that the next steps should be to identify appropriate processes and methodologies, adapted to the context of each country, regarding the application of value frameworks to improve the link between HTA and decision making.

HEALTH TECHNOLOGY ASSESSMENT FOR DECISION MAKING IN LATIN AMERICA: GOOD PRACTICE PRINCIPLES.

OBJECTIVES: The aim of this study was to identify good practice principles for health technology assessment (HTA) that are the most relevant and of highest priority for application in Latin America and to identify potential barriers to their implementation in the region. METHODS: HTA good practice principles proposed at the international level were identified and then explored during a deliberative process in a forum of assessors, funders, and product manufacturers. RESULTS: Forty-two representatives from ten Latin American countries participated. Good practice principles proposed at the international level were considered valid and potentially relevant to Latin America. Five principles were identified as priority and with the greatest potential to be strengthened at this time: transparency in the production of HTA, involvement of relevant stakeholders in the HTA process, mechanisms to appeal decisions, clear priority-setting processes in HTA, and a clear link between HTA and decision making. The main challenge identified was to find a balance between the application of these principles and the available resources in a way that would not detract from the production of reports and adaptation to the needs of decision makers. CONCLUSIONS: The main recommendation was to progress gradually in strengthening HTA and its link to decision making by developing appropriate processes for each country, without trying to impose, in the short-term, standards taken from examples at the international level without adequate adaptation of these to local contexts.

[Health technology assessment for decision-making in Latin America: good practice principles]. / Evaluación de tecnologías sanitarias para la toma de decisiones en Latinoamérica: principios de buenas prácticas.

OBJECTIVE: Identify the most relevant, applicable, and priority good practice principles in health technology assessment (HTA) in Latin America, and potential barriers to implementing them in the region. METHODS: HTA good practice principles postulated worldwide were identified and then explored through a deliberative process in a forum of evaluators, funders, and technology producers. RESULTS: Forty-two representatives from ten Latin American countries participated in the forum. The good practice principles postulated at the international level were considered valid and potentially applicable in Latin America. Five principles were identified as priorities and as having greater potential to be expanded at this time: transparency in carrying out HTA; involvement of stakeholders in the HTA process; existence of mechanisms to appeal decisions; existence of clear mechanisms for HTA priority-setting; and existence of a clear link between assessment and decision-making. The main challenge identified was to find a balance between application of these principles and available resources, to prevent the planned improvements from jeopardizing report production times and failing to meet decision-makers' needs. CONCLUSIONS: The main recommendation was to gradually advance in improving HTA and its link to decision-making by developing appropriate processes for each country, without attempting to impose, in the short term, standards taken from examples at the international level without adequate adaptation to the local context.

Governance arrangements for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, as well as the involvement of stakeholders in decision-making. Changes in governance arrangements can affect health and related goals in numerous ways, generally through changes in authority, accountability, openness, participation and coherence. A broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems and improving the governance of their health systems. OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of governance arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview. METHODS: We searched Health Systems Evidence in November 2010 and PDQ Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of governance arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out-of-pocket payments, cost-effectiveness), healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty, employment) and that were published after April 2005. We excluded reviews with limitations that were important enough to compromise the reliability of the findings of the review. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 systematic reviews and included 21 of them in this overview (19 primary reviews and 2 supplementary reviews). We focus here on the results of the 19 primary reviews, one of which had important methodological limitations. The other 18 were reliable (with only minor limitations).We grouped the governance arrangements addressed in the reviews into five categories: authority and accountability for health policies (three reviews); authority and accountability for organisations (two reviews); authority and accountability for commercial products (three reviews); authority and accountability for health professionals (seven reviews); and stakeholder involvement (four reviews).Overall, we found desirable effects for the following interventions on at least one outcome, with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects. Decision-making about what is covered by health insurance- Placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate-certainty evidence). Stakeholder participation in policy and organisational decisions- Participatory learning and action groups for women probably improve newborn survival (moderate-certainty evidence).- Consumer involvement in preparing patient information probably improves the quality of the information and patient knowledge (moderate-certainty evidence). Disclosing performance information to patients and the public- Disclosing performance data on hospital quality to the public probably encourages hospitals to implement quality improvement activities (moderate-certainty evidence).- Disclosing performance data on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate-certainty evidence). AUTHORS' CONCLUSIONS: Investigators have evaluated a wide range of governance arrangements that are relevant for low-income countries using sound systematic review methods. These strategies have been targeted at different levels in health systems, and studies have assessed a range of outcomes. Moderate-certainty evidence shows desirable effects (with no undesirable effects) for some interventions. However, there are important gaps in the availability of systematic reviews and primary studies for the all of the main categories of governance arrangements.

Implementation strategies for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: A key function of health systems is implementing interventions to improve health, but coverage of essential health interventions remains low in low-income countries. Implementing interventions can be challenging, particularly if it entails complex changes in clinical routines; in collaborative patterns among different healthcare providers and disciplines; in the behaviour of providers, patients or other stakeholders; or in the organisation of care. Decision-makers may use a range of strategies to implement health interventions, and these choices should be based on evidence of the strategies' effectiveness. OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of implementation strategies for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on alternative implementation strategies and informing refinements of the framework for implementation strategies presented in the overview. METHODS: We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of implementation strategies on professional practice and patient outcomes and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the review findings. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 systematic reviews and included 39 of them in this overview. An additional four reviews provided supplementary information. Of the 39 reviews, 32 had only minor limitations and 7 had important methodological limitations. Most studies in the reviews were from high-income countries. There were no studies from low-income countries in eight reviews.Implementation strategies addressed in the reviews were grouped into four categories - strategies targeting:1. healthcare organisations (e.g. strategies to change organisational culture; 1 review);2. healthcare workers by type of intervention (e.g. printed educational materials; 14 reviews);3. healthcare workers to address a specific problem (e.g. unnecessary antibiotic prescription; 9 reviews);4. healthcare recipients (e.g. medication adherence; 15 reviews).Overall, we found the following interventions to have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects.1.Strategies targeted at healthcare workers: educational meetings, nutrition training of health workers, educational outreach, practice facilitation, local opinion leaders, audit and feedback, and tailored interventions.2.Strategies targeted at healthcare workers for specific types of problems: training healthcare workers to be more patient-centred in clinical consultations, use of birth kits, strategies such as clinician education and patient education to reduce antibiotic prescribing in ambulatory care settings, and in-service neonatal emergency care training.3. Strategies targeted at healthcare recipients: mass media interventions to increase uptake of HIV testing; intensive self-management and adherence, intensive disease management programmes to improve health literacy; behavioural interventions and mobile phone text messages for adherence to antiretroviral therapy; a one time incentive to start or continue tuberculosis prophylaxis; default reminders for patients being treated for active tuberculosis; use of sectioned polythene bags for adherence to malaria medication; community-based health education, and reminders and recall strategies to increase vaccination uptake; interventions to increase uptake of cervical screening (invitations, education, counselling, access to health promotion nurse and intensive recruitment); health insurance information and application support. AUTHORS' CONCLUSIONS: Reliable systematic reviews have evaluated a wide range of strategies for implementing evidence-based interventions in low-income countries. Most of the available evidence is focused on strategies targeted at healthcare workers and healthcare recipients and relates to process-based outcomes. Evidence of the effects of strategies targeting healthcare organisations is scarce.

Delivery arrangements for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: Delivery arrangements include changes in who receives care and when, who provides care, the working conditions of those who provide care, coordination of care amongst different providers, where care is provided, the use of information and communication technology to deliver care, and quality and safety systems. How services are delivered can have impacts on the effectiveness, efficiency and equity of health systems. This broad overview of the findings of systematic reviews can help policymakers and other stakeholders identify strategies for addressing problems and improve the delivery of services. OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of delivery arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on delivery arrangements and informing refinements of the framework for delivery arrangements outlined in the review. METHODS: We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of delivery arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty or employment) and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the findings. Two overview authors independently screened reviews, extracted data, and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence), and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 systematic reviews and included 51 of them in this overview. We judged 6 of the 51 reviews to have important methodological limitations and the other 45 to have only minor limitations. We grouped delivery arrangements into eight categories. Some reviews provided more than one comparison and were in more than one category. Across these categories, the following intervention were effective; that is, they have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects. Who receives care and when: queuing strategies and antenatal care to groups of mothers. Who provides care: lay health workers for caring for people with hypertension, lay health workers to deliver care for mothers and children or infectious diseases, lay health workers to deliver community-based neonatal care packages, midlevel health professionals for abortion care, social support to pregnant women at risk, midwife-led care for childbearing women, non-specialist providers in mental health and neurology, and physician-nurse substitution. Coordination of care: hospital clinical pathways, case management for people living with HIV and AIDS, interactive communication between primary care doctors and specialists, hospital discharge planning, adding a service to an existing service and integrating delivery models, referral from primary to secondary care, physician-led versus nurse-led triage in emergency departments, and team midwifery. Where care is provided: high-volume institutions, home-based care (with or without multidisciplinary team) for people living with HIV and AIDS, home-based management of malaria, home care for children with acute physical conditions, community-based interventions for childhood diarrhoea and pneumonia, out-of-facility HIV and reproductive health services for youth, and decentralised HIV care. Information and communication technology: mobile phone messaging for patients with long-term illnesses, mobile phone messaging reminders for attendance at healthcare appointments, mobile phone messaging to promote adherence to antiretroviral therapy, women carrying their own case notes in pregnancy, interventions to improve childhood vaccination. Quality and safety systems: decision support with clinical information systems for people living with HIV/AIDS. Complex interventions (cutting across delivery categories and other health system arrangements): emergency obstetric referral interventions. AUTHORS' CONCLUSIONS: A wide range of strategies have been evaluated for improving delivery arrangements in low-income countries, using sound systematic review methods in both Cochrane and non-Cochrane reviews. These reviews have assessed a range of outcomes. Most of the available evidence focuses on who provides care, where care is provided and coordination of care. For all the main categories of delivery arrangements, we identified gaps in primary research related to uncertainty about the applicability of the evidence to low-income countries, low- or very low-certainty evidence or a lack of studies.

CAPACITY BUILDING IN AGENCIES FOR EFFICIENT AND EFFECTIVE HEALTH TECHNOLOGY ASSESSMENT.

OBJECTIVES: Health technology assessment (HTA) yields information that can be ideally used to address deficiencies in health systems and to create a wider understanding of the impact of different policy considerations around technology reimbursement and use. The structure of HTA programs varies across different jurisdictions according to decision-maker needs. Moreover, conducting HTA requires specialized skills. Effective decision making should include multiple criteria (medical, economic, technical, ethical, social, legal, and cultural) and requires multi-disciplinary teams of experts working together to produce these assessments. A workshop explored the multi-disciplinary skills and competencies required to build an effective and efficient HTA team, with a focus on low- and middle-income settings. METHODS: This proceeding summarizes main points from a workshop on capacity building, drawing on presentations and group discussions among attendees including different points of view. RESULTS AND CONCLUSIONS: The workshop and thus this study would have benefited from a larger variety of stakeholders. Therefore, the conclusions arising from the workshop are not the opinion of a representative sample of HTA professionals. Nonetheless, organizations and speakers were carefully selected to provide a valuable approach to this theme. Thus, these proceedings highlight some of the gaps and needs in the education and training programs offered worldwide and calls for further investigation.