A taxonomy of demand-driven questions for use by evidence producers, intermediaries and decision-makers: results from a cross-sectional survey.

BACKGROUND: Globally, a growing number of calls to formalize and strengthen evidence-support systems have been released, all of which emphasize the importance of evidence-informed decision making. To achieve this, it is critical that evidence producers and decision-makers interact, and that decision-makers' evidence needs can be efficiently translated into questions to which evidence producers can respond. This paper aims to create a taxonomy of demand-driven questions for use by evidence producers, intermediaries (i.e., people working in between researchers and decision-makers) and decision-makers. METHODS: We conducted a global cross-sectional survey of units providing some type of evidence support at the explicit request of decision-makers. Unit representatives were invited to answer an online questionnaire where they were asked to provide a list of the questions that they have addressed through their evidence-support mechanism. Descriptive analyses were used to analyze the survey responses, while the questions collected from each unit were iteratively analyzed to create a mutually exclusive and collectively exhaustive list of types of questions that can be answered with some form of evidence. RESULTS: Twenty-nine individuals completed the questionnaire, and more than 250 submitted questions were analysed to create a taxonomy of 41 different types of demand-driven questions. These 41 questions were organized by the goal to be achieved, and the goals were grouped in the four decision-making stages (i) clarifying a societal problem, its causes and potential impacts; (ii) finding and selecting options to address a problem; (iii) implementing or scaling-up an option; and (iv) monitoring implementation and evaluating impacts. CONCLUSION: The mutually exclusive and collectively exhaustive list of demand-driven questions will help decision-makers (to ask and prioritize questions), evidence producers (to organize and present their work), and evidence-intermediaries (to connect evidence needs with evidence supply).

Enhancing the delivery of comprehensive care for people living with HIV in Canada: insights from citizen panels and a national stakeholder dialogue.

BACKGROUND: People living with human immunodeficiency virus (HIV) are living longer with health-related disability associated with ageing, including complex conditions. However, health systems in Canada have not adapted to meet these comprehensive care needs. METHODS: We convened three citizen panels and a national stakeholder dialogue. The panels were informed by a plain-language citizen brief that outlined data and evidence about the challenge/problem, elements of an approach for addressing it and implementation considerations. The national dialogue was informed by a more detailed version of the same brief that included a thematic analysis of the findings from the panels. RESULTS: The 31 citizen panel participants emphasized the need for more prevention, testing and social supports, increased public education to address stigma and access to more timely data to inform system changes. The 21 system leaders emphasized the need to enhance person-centred care and for implementing learning and improvement across provinces, territories and Indigenous communities. Citizens and system leaders highlighted that policy actions need to acknowledge that HIV remains unique among conditions faced by Canadians. CONCLUSIONS: Action will require a national learning collaborative to support spread and scale of successful prevention, care and support initiatives. Such a collaborative should be grounded in a rapid-learning and improvement approach that is anchored on the needs, perspectives and aspirations of people living with HIV; driven by timely data and evidence; supported by appropriate decision supports and aligned governance, financial and delivery arrangements; and enabled with a culture of and competencies for rapid learning and improvement.

Matching the right study design to decision-maker questions: Results from a Delphi study.

Research evidence can play an important role in each stage of decision-making, evidence-support systems play a key role in aligning the demand for and supply of evidence. This paper provides guidance on what type of study designs most suitably address questions asked by decision-makers. This study used a two-round online Delphi approach, including methodological experts in different areas, disciplines, and geographic locations. Participants prioritized study designs for each of 40 different types of question, with a Kendall's W greater than 0.6 and reaching statistical significance (p<0.05) considered as a consensus. For each type of question, we sorted the final rankings based on their median ranks and interquartile ranges, and listed the four study designs with the highest median ranks. Participants provided 29 answers in the two rounds of the Delphi, and reached a consensus for 28 (out of the 40) questions (eight in the first round and 20 in the second). Participants achieved a consensus for 8 of 15 questions in stage I (clarifying a societal problem, its causes, and potential impacts), 12 of 13 in stage II (finding options to address a problem) and four of six in each of stages III (implementing or scaling-up an option) and IV (monitoring implementation and evaluating impact). This paper provides guidance on what study designs are more suitable to give insights on 28 different types of questions. Decision-makers, evidence intermediaries (, researchers and funders can use this guidance to make better decisions on what type of study design to commission, use or fund when answering specific needs.

User-experience testing of an evidence-to-decision framework for selecting essential medicines.

Essential medicine lists (EMLs) are important medicine prioritization tools used by the World Health Organization (WHO) EML and over 130 countries. The criteria used by WHO's Expert Committee on the Selection and Use of Essential Medicines has parallels to the GRADE Evidence-to-Decision (EtD) frameworks. In this study, we explored the EtD frameworks and a visual abstract as adjunctive tools to strengthen the integrate evidence and improve the transparency of decisions of EML applications. We conducted user-experience testing interviews of key EML stakeholders using Morville's honeycomb model. Interviews explored multifaceted dimensions (e.g., usability) on two EML applications for the 2021 WHO EML-long-acting insulin analogues for diabetes and immune checkpoint inhibitors for lung cancer. Using a pre-determined coding framework and thematic analysis we iteratively improved both the EtD framework and the visual abstract. We coded the transcripts of 17 interviews with 13 respondents in 103 locations of the interview texts across all dimensions of the user-experience honeycomb. Respondents felt the EtD framework and visual abstract presented complementary useful and findable adjuncts to the traditional EML application. They felt this would increase transparency and efficiency in evidence assessed by EML committees. As EtD frameworks are also used in health practice guidelines, including those by the WHO, respondents articulated that the adoption of the EtD by EML applications represents a tangible mechanism to align EMLs and guidelines, decrease duplication of work and improve coordination. Improvements were made to clarify instructions for the EtD and visual abstract, and to refine the design and content included. 'Availability' was added as an additional criterion for EML applications to highlight this criterion in alignment with WHO EML criteria. EtD frameworks and visual abstracts present additional important tools to communicate evidence and support decision-criteria in EML applications, which have global health impact. Access to essential medicines is important for achieving universal health coverage, and the development of essential medicine lists should be as evidence-based and trustworthy as possible.

GRADE Concept 7: Issues and Insights Linking Guideline Recommendations to Trustworthy Essential Medicine Lists.

OBJECTIVES: Guidelines and essential medicine lists (EMLs) bear similarities and differences in the process that lead to decisions. Access to essential medicines is central to achieve universal health coverage. The World Health Organization (WHO) EML has guided prioritization of essential medicines globally for nearly 50 years, and national EMLs (NEMLs) exist in over 130 countries. Guideline and EML decisions, at WHO or national levels, are not always coordinated and aligned. We sought to explore challenges, and potential solutions, for decision-making to support trustworthy medicine selection for EMLs from a Grading of Recommendations, Assessment, Development and Evaluations (GRADE) Working Group perspective. We primarily focus on the WHO EML; however, our findings may be applicable to NEML decisions as well. STUDY DESIGN AND SETTING: We identified key challenges in connecting the EML to health guidelines by involving a broad group of stakeholders and assessing case studies including real applications to the WHO EML, South Africa NEML, and a multiple sclerosis guideline connected to a WHO EML application for multiple sclerosis treatments. To address challenges, we utilized the results of a survey and feedback from the stakeholders, and iteratively met as a project group. We drafted a conceptual framework of challenges and potential solutions. We presented a summary of the results for feedback to all attendees of the GRADE Working Group meetings in November 2022 (approximately 120 people) and in May 2023 (approximately 100 people) before finalizing the framework. RESULTS: We prioritized issues and insights/solutions that addressed the connections between the EML and health guidelines. Our suggested solutions include early planning alignment of guideline groups and EMLs, considering shared participation to strengthen linkage, further clarity on price/cost considerations, and using explicit shared criteria to make guideline and EML decisions. We also provide recommendations to strengthen the connection between WHO EML and NEMLs including through contextualization methods. CONCLUSION: This GRADE concept article, jointly developed by key stakeholders from the guidelines and EMLs field, identified key conceptual issues and potential solutions to support the continued advancement of trustworthy EMLs. Adopting structured decision criteria that can be linked to guideline recommendations bears the potential to advance health equity and gaps in availability of essential medicines within and between countries.

Living evidence and adaptive policy: perfect partners?

BACKGROUND: While there has been widespread global acceptance of the importance of evidence-informed policy, many opportunities to inform health policy with research are missed, often because of a mismatch between when and where reliable evidence is needed, and when and where it is available. 'Living evidence' is an approach where systematic evidence syntheses (e.g. living reviews, living guidelines, living policy briefs, etc.) are continually updated to incorporate new relevant evidence as it becomes available. Living evidence approaches have the potential to overcome a major barrier to evidence-informed policy, making up-to-date systematic summaries of policy-relevant research available at any time that policy-makers need them. These approaches are likely to be particularly beneficial given increasing calls for policy that is responsive, and rapidly adaptive to changes in the policy context. We describe the opportunities presented by living evidence for evidence-informed policy-making and highlight areas for further exploration. DISCUSSION: There are several elements of living approaches to evidence synthesis that might support increased and improved use of evidence to inform policy. Reviews are explicitly prioritised to be 'living' by partnerships between policy-makers and researchers based on relevance to decision-making, as well as uncertainty of existing evidence, and likelihood that new evidence will arise. The ongoing nature of the work means evidence synthesis teams can be dynamic and engage with policy-makers in a variety of ways over time; and synthesis topics, questions and methods can be adapted as policy interests or contextual factors shift. Policy-makers can sign-up to be notified when relevant new evidence is found, and can be confident that living syntheses are up-to-date and contain all research whenever they access them. The always up-to-date nature of living evidence syntheses means producers can rapidly demonstrate availability of relevant, reliable evidence when it is needed, addressing a frequently cited barrier to evidence-informed policymaking. CONCLUSIONS: While there are challenges to be overcome, living evidence provides opportunities to enable policy-makers to access up-to-date evidence whenever they need it and also enable researchers to respond to the issues of the day with up-to-date research; and update policy-makers on changes in the evidence base as they arise. It also provides an opportunity to build flexible partnerships between researchers and policy-makers to ensure that evidence syntheses reflect the changing needs of policy-makers.

International law's effects on health and its social determinants: protocol for a systematic review, meta-analysis, and meta-regression analysis

Background: In recent years, there have been numerous calls for global institutions to develop and enforce new international laws. International laws are, however, often blunt instruments with many uncertain benefits, costs, risks of harm, and trade-offs. Thus, they are probably not always appropriate solutions to global health challenges. Given these uncertainties and international law's potential importance for improving global health, the paucity of synthesized evidence addressing whether international laws achieve their intended effects or whether they are superior in comparison to other approaches is problematic. Methods: Ten electronic bibliographic databases were searched using predefined search strategies, including MEDLINE, Global Health, CINAHL, Applied Social Sciences Index and Abstracts, Dissertations and Theses, International Bibliography of Social Sciences, International Political Science Abstracts, Social Sciences Abstracts, Social Sciences Citation Index, PAIS International, and Worldwide Political Science Abstracts. Two reviewers will independently screen titles and abstracts using predefined inclusion criteria. Pairs of reviewers will then independently screen the full-text of articles for inclusion using predefined inclusion criteria and then independently extract data and assess risk of bias for included studies. Where feasible, results will be pooled through subgroup analyses, meta-analyses, and meta-regression techniques. Discussion: The findings of this review will contribute to a better understanding of the expected benefits and possible harms of using international law to address different kinds of problems, thereby providing important evidence-informed guidance on when and how it can be effectively introduced and implemented by countries and global institutions.

Rol de la evidencia científica en las decisiones políticas relacionadas con los sistemas de salud / The role of research-based evidence in health system policy decision-making

RESUMEN Existen diferentes modelos para explicar cómo la evidencia de la investigación se utiliza en los procesos de formulación de políticas sobre los sistemas de salud. En este artículo argumentamos que los modelos que se desarrollaron desde el contexto clínico, como el de políticas basadas en la evidencia, pueden ser útiles en algunas decisiones políticas. Sin embargo, debido a su "silencio" sobre el contexto político, estos modelos son incompatibles con las decisiones relacionadas con la modificación de los arreglos de los sistemas de salud. Otros modelos, generados desde las ciencias políticas, son más útiles para entender que la investigación es uno solo de los factores que afecta la toma de decisiones y que diferentes tipos de evidencia científica pueden ser utilizados de manera instrumental, conceptual o estratégica en diferentes etapas del proceso de formulación de políticas.

ABSTRACT Different models may be used for explaining how research-based evidence is used in healthcare system policy-making. It is argued that models arising from a clinical setting (i.e. evidence-based policy-making model) could be useful regarding some types of healthcare system decision-making. However, such models are "silent" concerning the influence of political contextual factors on healthcare policy-making and are thus inconsistent with decision-making regarding the modification of healthcare system arrangements. Other political science-based models would seem to be more useful for understanding that research is just one factor affecting decision-making and that different types of research-based evidence can be used instrumentally, conceptual or strategically during different policy-making stages.

SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking?

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers. In this article, we discuss the following three questions: What is evidence? What is the role of research evidence in informing health policy decisions? What is evidence-informed policymaking? Evidence-informed health policymaking is an approach to policy decisions that aims to ensure that decision making is well-informed by the best available research evidence. It is characterised by the systematic and transparent access to, and appraisal of, evidence as an input into the policymaking process. The overall process of policymaking is not assumed to be systematic and transparent. However, within the overall process of policymaking, systematic processes are used to ensure that relevant research is identified, appraised and used appropriately. These processes are transparent in order to ensure that others can examine what research evidence was used to inform policy decisions, as well as the judgements made about the evidence and its implications. Evidenceinformed policymaking helps policymakers gain an understanding of these processes.