International treaties have mostly failed to produce their intended effects.

There are over 250,000 international treaties that aim to foster global cooperation. But are treaties actually helpful for addressing global challenges? This systematic field-wide evidence synthesis of 224 primary studies and meta-analysis of the higher-quality 82 studies finds treaties have mostly failed to produce their intended effects. The only exceptions are treaties governing international trade and finance, which consistently produced intended effects. We also found evidence that impactful treaties achieve their effects through socialization and normative processes rather than longer-term legal processes and that enforcement mechanisms are the only modifiable treaty design choice with the potential to improve the effectiveness of treaties governing environmental, human rights, humanitarian, maritime, and security policy domains. This evidence synthesis raises doubts about the value of international treaties that neither regulate trade or finance nor contain enforcement mechanisms.

Enhancing the delivery of comprehensive care for people living with HIV in Canada: insights from citizen panels and a national stakeholder dialogue.

BACKGROUND: People living with human immunodeficiency virus (HIV) are living longer with health-related disability associated with ageing, including complex conditions. However, health systems in Canada have not adapted to meet these comprehensive care needs. METHODS: We convened three citizen panels and a national stakeholder dialogue. The panels were informed by a plain-language citizen brief that outlined data and evidence about the challenge/problem, elements of an approach for addressing it and implementation considerations. The national dialogue was informed by a more detailed version of the same brief that included a thematic analysis of the findings from the panels. RESULTS: The 31 citizen panel participants emphasized the need for more prevention, testing and social supports, increased public education to address stigma and access to more timely data to inform system changes. The 21 system leaders emphasized the need to enhance person-centred care and for implementing learning and improvement across provinces, territories and Indigenous communities. Citizens and system leaders highlighted that policy actions need to acknowledge that HIV remains unique among conditions faced by Canadians. CONCLUSIONS: Action will require a national learning collaborative to support spread and scale of successful prevention, care and support initiatives. Such a collaborative should be grounded in a rapid-learning and improvement approach that is anchored on the needs, perspectives and aspirations of people living with HIV; driven by timely data and evidence; supported by appropriate decision supports and aligned governance, financial and delivery arrangements; and enabled with a culture of and competencies for rapid learning and improvement.

Quality improvement strategies for diabetes care: Effects on outcomes for adults living with diabetes.

BACKGROUND: There is a large body of evidence evaluating quality improvement (QI) programmes to improve care for adults living with diabetes. These programmes are often comprised of multiple QI strategies, which may be implemented in various combinations. Decision-makers planning to implement or evaluate a new QI programme, or both, need reliable evidence on the relative effectiveness of different QI strategies (individually and in combination) for different patient populations. OBJECTIVES: To update existing systematic reviews of diabetes QI programmes and apply novel meta-analytical techniques to estimate the effectiveness of QI strategies (individually and in combination) on diabetes quality of care. SEARCH METHODS: We searched databases (CENTRAL, MEDLINE, Embase and CINAHL) and trials registers (ClinicalTrials.gov and WHO ICTRP) to 4 June 2019. We conducted a top-up search to 23 September 2021; we screened these search results and 42 studies meeting our eligibility criteria are available in the awaiting classification section. SELECTION CRITERIA: We included randomised trials that assessed a QI programme to improve care in outpatient settings for people living with diabetes. QI programmes needed to evaluate at least one system- or provider-targeted QI strategy alone or in combination with a patient-targeted strategy. - System-targeted: case management (CM); team changes (TC); electronic patient registry (EPR); facilitated relay of clinical information (FR); continuous quality improvement (CQI). - Provider-targeted: audit and feedback (AF); clinician education (CE); clinician reminders (CR); financial incentives (FI). - Patient-targeted: patient education (PE); promotion of self-management (PSM); patient reminders (PR). Patient-targeted QI strategies needed to occur with a minimum of one provider or system-targeted strategy. DATA COLLECTION AND ANALYSIS: We dual-screened search results and abstracted data on study design, study population and QI strategies. We assessed the impact of the programmes on 13 measures of diabetes care, including: glycaemic control (e.g. mean glycated haemoglobin (HbA1c)); cardiovascular risk factor management (e.g. mean systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), proportion of people living with diabetes that quit smoking or receiving cardiovascular medications); and screening/prevention of microvascular complications (e.g. proportion of patients receiving retinopathy or foot screening); and harms (e.g. proportion of patients experiencing adverse hypoglycaemia or hyperglycaemia). We modelled the association of each QI strategy with outcomes using a series of hierarchical multivariable meta-regression models in a Bayesian framework. The previous version of this review identified that different strategies were more or less effective depending on baseline levels of outcomes. To explore this further, we extended the main additive model for continuous outcomes (HbA1c, SBP and LDL-C) to include an interaction term between each strategy and average baseline risk for each study (baseline thresholds were based on a data-driven approach; we used the median of all baseline values reported in the trials). Based on model diagnostics, the baseline interaction models for HbA1c, SBP and LDL-C performed better than the main model and are therefore presented as the primary analyses for these outcomes. Based on the model results, we qualitatively ordered each QI strategy within three tiers (Top, Middle, Bottom) based on its magnitude of effect relative to the other QI strategies, where 'Top' indicates that the QI strategy was likely one of the most effective strategies for that specific outcome. Secondary analyses explored the sensitivity of results to choices in model specification and priors.  Additional information about the methods and results of the review are available as Appendices in an online repository. This review will be maintained as a living systematic review; we will update our syntheses as more data become available. MAIN RESULTS: We identified 553 trials (428 patient-randomised and 125 cluster-randomised trials), including a total of 412,161 participants. Of the included studies, 66% involved people living with type 2 diabetes only. Participants were 50% female and the median age of participants was 58.4 years. The mean duration of follow-up was 12.5 months. HbA1c was the commonest reported outcome; screening outcomes and outcomes related to cardiovascular medications, smoking and harms were reported infrequently. The most frequently evaluated QI strategies across all study arms were PE, PSM and CM, while the least frequently evaluated QI strategies included AF, FI and CQI. Our confidence in the evidence is limited due to a lack of information on how studies were conducted.  Four QI strategies (CM, TC, PE, PSM) were consistently identified as 'Top' across the majority of outcomes. All QI strategies were ranked as 'Top' for at least one key outcome. The majority of effects of individual QI strategies were modest, but when used in combination could result in meaningful population-level improvements across the majority of outcomes. The median number of QI strategies in multicomponent QI programmes was three.  Combinations of the three most effective QI strategies were estimated to lead to the below effects:  - PR + PSM + CE: decrease in HbA1c by 0.41% (credibility interval (CrI) -0.61 to -0.22) when baseline HbA1c < 8.3%; - CM + PE + EPR: decrease in HbA1c by 0.62% (CrI -0.84 to -0.39) when baseline HbA1c > 8.3%;  - PE + TC + PSM: reduction in SBP by 2.14 mmHg (CrI -3.80 to -0.52) when baseline SBP < 136 mmHg; - CM + TC + PSM: reduction in SBP by 4.39 mmHg (CrI -6.20 to -2.56) when baseline SBP > 136 mmHg;  - TC + PE + CM: LDL-C lowering of 5.73 mg/dL (CrI -7.93 to -3.61) when baseline LDL < 107 mg/dL; - TC + CM + CR: LDL-C lowering by 5.52 mg/dL (CrI -9.24 to -1.89) when baseline LDL > 107 mg/dL. Assuming a baseline screening rate of 50%, the three most effective QI strategies were estimated to lead to an absolute improvement of 33% in retinopathy screening (PE + PR + TC) and 38% absolute increase in foot screening (PE + TC + Other). AUTHORS' CONCLUSIONS: There is a significant body of evidence about QI programmes to improve the management of diabetes. Multicomponent QI programmes for diabetes care (comprised of effective QI strategies) may achieve meaningful population-level improvements across the majority of outcomes. For health system decision-makers, the evidence summarised in this review can be used to identify strategies to include in QI programmes. For researchers, this synthesis identifies higher-priority QI strategies to examine in further research regarding how to optimise their evaluation and effects. We will maintain this as a living systematic review.

Living evidence and adaptive policy: perfect partners?

BACKGROUND: While there has been widespread global acceptance of the importance of evidence-informed policy, many opportunities to inform health policy with research are missed, often because of a mismatch between when and where reliable evidence is needed, and when and where it is available. 'Living evidence' is an approach where systematic evidence syntheses (e.g. living reviews, living guidelines, living policy briefs, etc.) are continually updated to incorporate new relevant evidence as it becomes available. Living evidence approaches have the potential to overcome a major barrier to evidence-informed policy, making up-to-date systematic summaries of policy-relevant research available at any time that policy-makers need them. These approaches are likely to be particularly beneficial given increasing calls for policy that is responsive, and rapidly adaptive to changes in the policy context. We describe the opportunities presented by living evidence for evidence-informed policy-making and highlight areas for further exploration. DISCUSSION: There are several elements of living approaches to evidence synthesis that might support increased and improved use of evidence to inform policy. Reviews are explicitly prioritised to be 'living' by partnerships between policy-makers and researchers based on relevance to decision-making, as well as uncertainty of existing evidence, and likelihood that new evidence will arise. The ongoing nature of the work means evidence synthesis teams can be dynamic and engage with policy-makers in a variety of ways over time; and synthesis topics, questions and methods can be adapted as policy interests or contextual factors shift. Policy-makers can sign-up to be notified when relevant new evidence is found, and can be confident that living syntheses are up-to-date and contain all research whenever they access them. The always up-to-date nature of living evidence syntheses means producers can rapidly demonstrate availability of relevant, reliable evidence when it is needed, addressing a frequently cited barrier to evidence-informed policymaking. CONCLUSIONS: While there are challenges to be overcome, living evidence provides opportunities to enable policy-makers to access up-to-date evidence whenever they need it and also enable researchers to respond to the issues of the day with up-to-date research; and update policy-makers on changes in the evidence base as they arise. It also provides an opportunity to build flexible partnerships between researchers and policy-makers to ensure that evidence syntheses reflect the changing needs of policy-makers.

Future leaders in a Learning Health System: Exploring the Health System Impact Fellowship.

The Canadian health system is reeling following the COVID-19 pandemic. Strains have become growing cracks, with long emergency department wait times, shortage of human health resources, and growing dissatisfaction from both clinicians and patients. To address long needed health system reform in Canada, a modernization of training is required for the next generation health leaders. The Canadian Institutes of Health Research Health System Impact Fellowship is an example of a well-funded and connected training program which prioritizes embedded research and embedding technically trained scholars with health system partners. The program has been successful in the scope and impact of its training outcomes as well as providing health system partners with a pool of connected and capable scholars. Looking forward, integrating aspects of evidence synthesis from both domestic and international sources and adapting a general contractor approach to implementation within the HSIF could help catalyze Learning Health System reform in Canada.

Facilitators, barriers and strategies for health-system guidance implementation: a critical interpretive synthesis protocol.

BACKGROUND: As systematically developed statements regarding possible courses of action, health system guidance (HSG) can assist with making decisions about addressing problems or achieving goals in health systems. However, there are conceptual and methodological challenges in HSG implementation due to the complexity of health-system policy-making, the diversity of available evidence and vast differences in contexts. To address these gaps, we aim to develop a theoretical framework for supporting HSG implementation as part of a broader effort to promote evidence-informed policy-making in health systems. METHODS: To develop a theoretical framework about facilitators, barriers and strategies for HSG implementation, we will apply a critical interpretive synthesis (CIS) approach to synthesize the findings from a range of relevant literature. We will search 11 electronic databases and seven organizational websites to identify relevant published and grey literature. We will check the references of included studies and contact experts to identify additional eligible papers. Finally, we will conduct purposively sampling of the literature to fill any identified conceptual gaps. We will use relevance and five quality criteria to assess included papers. A standardized form will be developed for extracting information. We will use an interpretive analytic approach to synthesize the findings, including a constant comparative method throughout the analysis. Two independent reviewers will conduct the literature screening and relevance assessment, and disagreements will be resolved through discussion. The principal investigator will conduct data extraction and synthesis, and a second reviewer will check the sample of extracted data for consistency and accuracy. DISCUSSION: A new theoretical framework about facilitators, barriers and strategies for HSG implementation will be developed using a CIS approach. The HSG implementation framework could be widely used for supporting the implementation of HSG covering varied topics and in different contexts (including low-, middle- and high-income countries). In later work, we will develop a tool for supporting HSG implementation based on the theoretical framework. Registration PROSPERO CRD42020214072. Date of Registration: 14 December 2020.

Development of a STandard reporting guideline for Evidence briefs for Policy (STEP): context and study protocol.

BACKGROUND: Evidence briefs for policy (EBP) draw on best-available data and research evidence (e.g., systematic reviews) to help clarify policy problems, frame options for addressing them, and identify implementation considerations for policymakers in a given context. An increasing number of governments, non-governmental organizations and research groups have been developing EBP on a wide variety of topics. However, the reporting characteristics of EBP vary across organizations due to a lack of internationally accepted standard reporting guidelines. This project aims to develop a STandard reporting guideline of Evidence briefs for Policy (STEP), which will encompass a reporting checklist and a STEP statement and a user manual. METHODS: We will refer to and adapt the methods recommended by the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network. The key actions include: (1) developing a protocol; (2) establishing an international multidisciplinary STEP working group (consisting of a Coordination Team and a Delphi Panel); (3) generating an initial draft of the potential items for the STEP reporting checklist through a comprehensive review of EBP-related literature and documents; (4) conducting a modified Delphi process to select and refine the reporting checklist; (5) using the STEP to evaluate published policy briefs in different countries; (6) finalizing the checklist; (7) developing the STEP statement and the user manual (8) translating the STEP into different languages; and (9) testing the reliability through real world use. DISCUSSION: Our protocol describes the development process for STEP. It will directly address what and how information should be reported in EBP and contribute to improving their quality. The decision-makers, researchers, journal editors, evaluators, and other stakeholders who support evidence-informed policymaking through the use of mechanisms like EBP will benefit from the STEP. Registration We registered the protocol on the EQUATOR network. ( https://www.equator-network.org/library/reporting-guidelines-under-development/#84 ).

Enhancing the use of technology in the long-term care sector in Canada: Insights from citizen panels and a national stakeholder dialogue.

Enhancing the use of technology in long-term care has been identified as a key part of broader efforts to strengthen the sector in the wake of the COVID-19 pandemic. To inform such efforts, we convened a series of citizen panels, followed by a national stakeholder dialogue with system leaders focused on reimagining the long-term care sector using technology. Key actions prioritized through the deliberations convened included: developing an innovation roadmap/agenda (including national standards and guidelines); using co-design approaches for the strengthening the long-term care sector and for technological innovation; identifying and coordinating existing innovation projects to support scale and spread; enabling rapid-learning and improvement cycles to support the development, evaluation, and implementation of new technologies; and using funding models that enable the flexibility needed for such rapid-learning cycles.

Understanding the conditions that influence the roles of midwives in Ontario, Canada's health system: an embedded single-case study.

BACKGROUND: Despite the significant variability in the role and integration of midwifery across provincial and territorial health systems, there has been limited scholarly inquiry into whether, how and under what conditions midwifery has been assigned roles and integrated into Canada's health systems. METHODS: We use Yin's (2014) embedded single-case study design, which allows for an in-depth exploration to qualitatively assess how, since the regulation of midwives in 1994, the Ontario health system has assigned roles to and integrated midwives as a service delivery option. Kingdon's agenda setting and 3i + E theoretical frameworks are used to analyze two recent key policy directions (decision to fund freestanding midwifery-led birth centres and the Patients First primary care reform) that presented opportunities for the integration of midwives into the health system. Data were collected from key informant interviews and documents. RESULTS: Nineteen key informant interviews were conducted, and 50 documents were reviewed in addition to field notes taken during the interviews. Our findings suggest that while midwifery was created as a self-regulated profession in 1994, health-system transformation initiatives have restricted the profession's integration into Ontario's health system. The policy legacies of how past decisions influence the decisions possible today have the most explanatory power to understand why midwives have had limited integration into interprofessional maternity care. The most important policy legacies to emerge from the analyses were related to payment mechanisms. In the medical model, payment mechanisms privilege physician-provided and hospital-based services, while payment mechanisms in the midwifery model have imposed unintended restrictions on the profession's ability to practice in interprofessional environments. CONCLUSIONS: This is the first study to explain why midwives have not been fully integrated into the Ontario health system, as well as the limitations placed on their roles and scope of practice. The study also builds a theoretical understanding of the integration process of healthcare professions within health systems and how policy legacies shape service delivery options.

Conceptualising characteristics of resources withdrawal from medical services: a systematic qualitative synthesis.

BACKGROUND: Terms used to describe government-led resource withdrawal from ineffective and unsafe medical services, including 'rationing' and 'disinvestment', have tended to be used interchangeably, despite having distinct characteristics. This lack of descriptive precision for arguably distinct terms contributes to the obscurity that hinders effective communication and the achievement of evidence-based decision-making. The objectives of this study are to (1) identify the various terms used to describe resource withdrawal and (2) propose definitions for the key or foundational terms, which includes a clear description of the unique characteristics of each. METHODS: This is a systematic qualitative synthesis of characteristics and terms found through a search of the academic and grey literature. This approach involved identifying commonly used resource withdrawal terms, extracting data about resource withdrawal characteristics associated with each term and conducting a comparative analysis by categorising elements as antecedents, attributes or outcomes. RESULTS: Findings from an analysis of 106 documents demonstrated that terms used to describe resource withdrawal are inconsistently defined and applied. The characteristics associated with these terms, mainly antecedents and attributes, are used interchangeably by many authors but are differentiated by others. Our analysis resulted in the development of a framework that organises these characteristics to demonstrate the unique attributes associated with each term. To enhance precision, these terms were classified as either policy options or patient health outcomes and refined definitions for rationing and disinvestment were developed. Rationing was defined as resource withdrawal that denies, on average, patient health benefits. Disinvestment was defined as resource withdrawal that results in, on average, improved or no change in health benefits. CONCLUSION: Agreement on the definition of various resource withdrawal terms and their key characteristics is required for transparent government decision-making regarding medical service withdrawal. This systematic qualitative synthesis presents the proposed definitions of resource withdrawal terms that will promote consistency, benefit public policy dialogue and enhance the policy-making process for health systems.

Supporting the use of research evidence in decision-making in crisis zones in low- and middle-income countries: a critical interpretive synthesis.

BACKGROUND: Decision-makers in crisis zones are faced with the challenge of having to make health-related decisions under limited time and resource constraints and in light of the many factors that can influence their decisions, of which research evidence is just one. To address a key gap in the research literature about how best to support the use of research evidence in such situations, we conducted a critical interpretive synthesis approach to develop a conceptual framework that outlines the strategies that leverage the facilitators and address the barriers to evidence use in crisis zones. METHODS: We systematically reviewed both empirical and non-empirical literature and used an interpretive analytic approach to synthesise the results and develop the conceptual framework. We used a 'compass' question to create a detailed search strategy and conducted electronic searches in CINAHL, EMBASE, MEDLINE, SSCI and Web of Science. A second reviewer was assigned to a representative sample of articles. We purposively sampled additional papers to fill in conceptual gaps. RESULTS: We identified 21 eligible papers to be analysed and purposively sampled an additional 6 to fill conceptual gaps. The synthesis resulted in a conceptual framework that focuses on evidence use in crisis zones examined through the lens of four systems - political, health, international humanitarian aid and health research. Within each of the four systems, the framework identifies the most actionable strategies that leverage the facilitators and address the barriers to evidence use. CONCLUSIONS: This study presents a new conceptual framework that outlines strategies that leverage the facilitators and address the barriers to evidence use in crisis zones within different systems. This study expands on the literature pertaining to evidence-informed decision-making.

A critical interpretive synthesis of the roles of midwives in health systems.

BACKGROUND: Midwives' roles in sexual and reproductive health and rights continues to evolve. Understanding the profession's role and how midwives can be integrated into health systems is essential in creating evidence-informed policies. Our objective was to develop a theoretical framework of how political system factors and health systems arrangements influence the roles of midwives within the health system. METHODS: A critical interpretive synthesis was used to develop the theoretical framework. A range of electronic bibliographic databases (CINAHL, EMBASE, Global Health database, HealthSTAR, Health Systems Evidence, MEDLINE and Web of Science) was searched through to 14 May 2020 as were policy and health systems-related and midwifery organisation websites. A coding structure was created to guide the data extraction. RESULTS: A total of 4533 unique documents were retrieved through electronic searches, of which 4132 were excluded using explicit criteria, leaving 401 potentially relevant records, in addition to the 29 records that were purposively sampled through grey literature. A total of 100 documents were included in the critical interpretive synthesis. The resulting theoretical framework identified the range of political and health system components that can work together to facilitate the integration of midwifery into health systems or act as barriers that restrict the roles of the profession. CONCLUSIONS: Any changes to the roles of midwives in health systems need to take into account the political system where decisions about their integration will be made as well as the nature of the health system in which they are being integrated. The theoretical framework, which can be thought of as a heuristic, identifies the core contextual factors that governments can use to best leverage their position when working to improve sexual and reproductive health and rights.

A framework for explaining the role of values in health policy decision-making in Latin America: a critical interpretive synthesis.

BACKGROUND: Although values underpin the goals pursued in health systems, including how health systems benefit the population, it is often not clear how values are incorporated into policy decision-making about health systems. The challenge is to encompass social/citizen values, health system goals, and financial realities and to incorporate them into the policy-making process. This is a challenge for all health systems and of particular importance for Latin American (LA) countries. Our objective was to understand how and under what conditions societal values inform decisions about health system financing in LA countries. METHODS: A critical interpretive synthesis approach was utilised for this work. We searched 17 databases in December 2016 to identify articles written in English, Spanish or Portuguese that focus on values that inform the policy process for health system financing in LA countries at the macro and meso levels. Two reviewers independently screened records and assessed them for inclusion. One researcher conceptually mapped the included articles, created structured summaries of key findings from each, and selected a purposive sample of articles to thematically synthesise the results across the domains of agenda-setting/prioritisation, policy development and implementation. RESULTS: We identified 5925 references, included 199 papers, and synthesised 68 papers. We identified 116 values and developed a framework to explain how values have been used to inform policy decisions about financing in LA countries. This framework has four categories - (1) goal-related values (i.e. guiding principles of the health system); (2) technical values (those incorporated into the instruments adopted by policy-makers to ensure a sustainable and efficient health system); (3) governance values (those applied in the policy process to ensure a transparent and accountable process of decision-making); and (4) situational values (a broad category of values that represent competing strategies to make decisions in the health systems, their influence varying according to the four factors). CONCLUSIONS: It is an effort to consolidate and explain how different social values are considered and how they support policy decision-making about health system financing. This can help policy-makers to explicitly incorporate values into the policy process and understand how values are supporting the achievement of policy goals in health system financing. TRIAL REGISTRATION: The protocol was registered with PROSPERO, ID=CRD42017057049 .

Lessons learned from descriptions and evaluations of knowledge translation platforms supporting evidence-informed policy-making in low- and middle-income countries: a systematic review.

BACKGROUND: Knowledge translation (KT) platforms are organisations, initiatives and networks that focus on supporting evidence-informed policy-making at least in part about the health-system arrangements that determine whether the right programmes, services and products get to those who need them. Many descriptions and evaluations of KT platforms in low- and middle-income countries have been produced but, to date, they have not been systematically reviewed. METHODS: We identified potentially relevant studies through a search of five electronic databases and a variety of approaches to identify grey literature. We used four criteria to select eligible empirical studies. We extracted data about seven characteristics of included studies and about key findings. We used explicit criteria to assess study quality. In synthesising the findings, we gave greater attention to themes that emerged from multiple studies, higher-quality studies and different contexts. RESULTS: Country was the most common jurisdictional focus of KT platforms, EVIPNet the most common name and high turnover among staff a common infrastructural feature. Evidence briefs and deliberative dialogues were the activities/outputs that were the most extensively studied and viewed as helpful, while rapid evidence services were the next most studied but only in a single jurisdiction. None of the summative evaluations used a pre-post design or a control group and, with the exception of the evaluations of the influence of briefs and dialogues on intentions to act, none of the evaluations achieved a high quality score. CONCLUSIONS: A large and growing volume of research evidence suggests that KT platforms offer promise in supporting evidence-informed policy-making in low- and middle-income countries. KT platforms should consider as next steps expanding their current, relatively limited portfolio of activities and outputs, building bridges to complementary groups, and planning for evaluations that examine 'what works' for 'what types of issues' in 'what types of contexts'.

Understanding the supports needed for policy implementation: a comparative analysis of the placement of intermediaries across three mental health systems.

BACKGROUND: Intermediaries are organisations or programmes that work between policy-makers and service providers to facilitate effective implementation of evidence-informed policies, programmes and practices. A number of intermediaries now exist in well-established mental health systems; however, research on them, and how they may be optimised to support implementation is lacking. This research seeks to understand the puzzling variation in the system placement of intermediaries supporting policy implementation in the mental health systems of Canada (Ontario), New Zealand and Scotland. METHODS: Using a comparative case study approach, the analytic goal was to compare intermediaries across jurisdictions and explain differences in their placement using explanatory frameworks from political science. Data for this analysis were derived from several sources, including key informant interviews, a literature search of published and grey literature on intermediaries and on policy implementation in mental health systems, a review of relevant policy documents and websites, as well as documents and websites relating to the various intermediaries and other interest groups within each system. RESULTS: Through the analysis, we argue that the placement of intermediaries supporting policy implementation can be explained through an understanding of the political structures, the policy legacies leading to the current public/private mix of mental health service delivery, and the differing administrative capacities of mental health systems. CONCLUSIONS: This research contributes to our growing understanding of policy-related intermediaries supporting implementation at scale and how we might build appropriate infrastructure in systems to support the implementation of policy and achieve better outcomes for citizens.

Stakeholders' experiences with the evidence aid website to support 'real-time' use of research evidence to inform decision-making in crisis zones: a user testing study.

BACKGROUND: Humanitarian action in crisis zones is fraught with many challenges, including lack of timely and accessible research evidence to inform decision-making about humanitarian interventions. Evidence websites have the potential to address this challenge. Evidence Aid is the only evidence website designed for crisis zones that focuses on providing research evidence in the form of systematic reviews. The objective of this study is to explore stakeholders' views of Evidence Aid, contributing further to our understanding of the use of research evidence in decision-making in crisis zones. METHODS: We designed a qualitative user-testing study to collect interview data from stakeholders about their impressions of Evidence Aid. Eligible stakeholders included those with and without previous experience of Evidence Aid. All participants were either currently working or have worked within the last year in a crisis zone. Participants were asked to perform the same user experience-related tasks and answer questions about this experience and their knowledge needs. Data were analysed using a deductive framework analysis approach drawing on Morville's seven facets of the user experience - findability, usability, usefulness, desirability, accessibility, credibility and value. RESULTS: A total of 31 interviews were completed with senior decision-makers (n = 8), advisors (n = 7), field managers (n = 7), analysts/researchers (n = 5) and healthcare providers (n = 4). Participant self-reported knowledge needs varied depending on their role. Overall, participants did not identify any 'major' problems (highest order) and identified only two 'big' problems (second highest order) with using the Evidence Aid website, namely the lack of a search engine on the home page and that some full-text articles linked to/from the site require a payment. Participants identified seven specific suggestions about how to improve Evidence Aid, many of which can also be applied to other evidence websites. CONCLUSIONS: Stakeholders in crisis zones found Evidence Aid to be useful, accessible and credible. However, they experienced some problems with the lack of a search engine on the home page and the requirement for payment for some full-text articles linked to/from the site.

Developing evidence briefs for policy: a qualitative case study comparing the process of using a guidance-contextualization workbook in Peru and Uganda.

BACKGROUND: Translating research evidence from global guidance into policy can help strengthen health systems. A workbook was developed to support the contextualization of the WHO's 'Optimizing health worker roles to improve maternal and newborn health' (OptimizeMNH) guidance. This study evaluated the use of the workbook for the development of evidence briefs in two countries - Peru and Uganda. Findings surrounding contextual factors, steps in the process and evaluation of the workbook are presented. METHODS: A qualitative embedded case study was used. The case was the process of using the workbook to support the contextualization of global health systems guidance, with local evidence, to develop evidence briefs. Criterion sampling was used to select the countries, participants for interviews and documents included in the study. A template-organizing style and constant comparison were used for data analysis. RESULTS: A total of 19 participant-observation sessions and 8 interviews were conducted, and 50 documents were reviewed. Contextual factors, including the cadres, or groups, of health workers available in each country, the way the problem and its causes were framed, potential policy options to address the problem, and implementation considerations for these policy options, varied substantially between Peru and Uganda. However, many similarities were found in the process of using the workbook. Overall, the workbook was viewed positively and participants in both countries would use it again for other topics. CONCLUSIONS: Organizations that produce global guidance, such as WHO, need to consider institutionalizing the application of the workbook into their guidance development processes to help users at the national/subnational level create actionable and context-relevant policies. Feedback mechanisms also need to be established so that the evidence briefs and health policies arising from global guidance are tracked and the findings coming out of such guideline contextualization processes can be taken into consideration during future guidance development and research priority-setting.

Addressing the affordability of cancer drugs: using deliberative public engagement to inform health policy.

BACKGROUND: Health system expenditure on cancer drugs is rising rapidly in many OECD countries given the costly new treatments and increased rates of use due to a growing and ageing population. These factors put considerable strain on the sustainability of health systems worldwide, sparking public debate among clinicians, pharmaceutical companies, policy-makers and citizens on issues of affordability and equity. We engaged Canadians through a series of deliberative public engagement events to determine their priorities for making cancer drug funding decisions fair and sustainable in Canada's publicly financed health system. METHODS: An approach to deliberation was developed based on the McMaster Health Forum's citizen panels and the established Burgess and O'Doherty model of deliberative public engagement. Six deliberations were held across Canada in 2016. Transcripts were coded in NVivo and analysed to determine where participants' views converged and diverged. Recommendations were grouped thematically. RESULTS: A total of 115 Canadians participated in the deliberative events and developed 86 recommendations. Recommendations included the review and regular re-review of approved drugs using 'real-world' evidence on effectiveness and cost-effectiveness; prioritisation of treatments that restore patients' independence, mental health and general well-being; ensuring that decision processes, results and their rationales are transparent; and commitment to people with similar needs receiving the same care regardless of where in Canada they live. CONCLUSIONS: The next steps for policy-makers should be to develop mechanisms for (1) re-reviewing effectiveness and cost-effectiveness data for all cancer drugs; (2) making disinvestments in cancer drugs that satisfy requirements relating to grandfathering and compassionate access; (3) ensuring fair and equitable access to cancer drugs for all Canadians; and (4) fostering a pan-Canadian approach to cancer drug funding decisions.

Evaluating user experiences of a clearing house for health policy and systems.

BACKGROUND: Timely access to evidence increases the prospects for evidence informed decision making. We evaluated user experiences of a Clearing House for Health Policy with the aim of increasing access to evidence about the Uganda health system and interventions. METHODS: We conducted in-depth interviews with 15 potential users including policymakers, health policy advisors, health managers and researchers to provide evidence on their experience with the clearinghouse. On average participants took 20 minutes to first navigate the site and 45 minutes to perform search tasks and complete the interview. RESULTS: Most respondents successfully searched for information with accuracy and completeness in a short time. Participants commended the performance and expressed high regard for the credibility of the clearinghouse. The majority felt that using the resource was worth their effort. The clearinghouse provided appropriate functionalities for information searching. Navigating and finding information from the site was achievable. However, inadequate background information about the site and lack of current information were widely reported. CONCLUSION: Our paper provides insights on the issues that can be addressed to improve online resources for health policy and system information in a limited resource setting. Users' experience of such resources can be improved by regularly appraising and appropriately indexing the contents, and optimising the capacity to customise information.

Levers for addressing medical underuse and overuse: achieving high-value health care.

The preceding papers in this Series have outlined how underuse and overuse of health-care services occur within a complex system of health-care production, with a multiplicity of causes. Because poor care is ubiquitous and has considerable consequences for the health and wellbeing of billions of people around the world, remedying this problem is a morally and politically urgent task. Universal health coverage is a key step towards achieving the right care. Therefore, full consideration of potential levers of change must include an upstream perspective-ie, an understanding of the system-level factors that drive overuse and underuse, as well as the various incentives at work during a clinical encounter. One example of a system-level factor is the allocation of resources (eg, hospital beds and clinicians) to meet the needs of a local population to minimise underuse or overuse. Another example is priority setting using tools such as health technology assessment to guide the optimum diffusion of safe, effective, and cost-effective health-care services. In this Series paper we investigate a range of levers for eliminating medical underuse and overuse. Some levers could operate effectively (and be politically viable) across many different health and political systems (eg, increase patient activation with decision support) whereas other levers must be tailored to local contexts (eg, basing coverage decisions on a particular cost-effectiveness ratio). Ideally, policies must move beyond the purely incremental; that is, policies that merely tinker at the policy edges after underuse or overuse arises. In this regard, efforts to increase public awareness, mobilisation, and empowerment hold promise as universal methods to reset all other contexts and thereby enhance all other efforts to promote the right care.