User-experience testing of an evidence-to-decision framework for selecting essential medicines.

Essential medicine lists (EMLs) are important medicine prioritization tools used by the World Health Organization (WHO) EML and over 130 countries. The criteria used by WHO's Expert Committee on the Selection and Use of Essential Medicines has parallels to the GRADE Evidence-to-Decision (EtD) frameworks. In this study, we explored the EtD frameworks and a visual abstract as adjunctive tools to strengthen the integrate evidence and improve the transparency of decisions of EML applications. We conducted user-experience testing interviews of key EML stakeholders using Morville's honeycomb model. Interviews explored multifaceted dimensions (e.g., usability) on two EML applications for the 2021 WHO EML-long-acting insulin analogues for diabetes and immune checkpoint inhibitors for lung cancer. Using a pre-determined coding framework and thematic analysis we iteratively improved both the EtD framework and the visual abstract. We coded the transcripts of 17 interviews with 13 respondents in 103 locations of the interview texts across all dimensions of the user-experience honeycomb. Respondents felt the EtD framework and visual abstract presented complementary useful and findable adjuncts to the traditional EML application. They felt this would increase transparency and efficiency in evidence assessed by EML committees. As EtD frameworks are also used in health practice guidelines, including those by the WHO, respondents articulated that the adoption of the EtD by EML applications represents a tangible mechanism to align EMLs and guidelines, decrease duplication of work and improve coordination. Improvements were made to clarify instructions for the EtD and visual abstract, and to refine the design and content included. 'Availability' was added as an additional criterion for EML applications to highlight this criterion in alignment with WHO EML criteria. EtD frameworks and visual abstracts present additional important tools to communicate evidence and support decision-criteria in EML applications, which have global health impact. Access to essential medicines is important for achieving universal health coverage, and the development of essential medicine lists should be as evidence-based and trustworthy as possible.

Quality improvement strategies for diabetes care: Effects on outcomes for adults living with diabetes.

BACKGROUND: There is a large body of evidence evaluating quality improvement (QI) programmes to improve care for adults living with diabetes. These programmes are often comprised of multiple QI strategies, which may be implemented in various combinations. Decision-makers planning to implement or evaluate a new QI programme, or both, need reliable evidence on the relative effectiveness of different QI strategies (individually and in combination) for different patient populations. OBJECTIVES: To update existing systematic reviews of diabetes QI programmes and apply novel meta-analytical techniques to estimate the effectiveness of QI strategies (individually and in combination) on diabetes quality of care. SEARCH METHODS: We searched databases (CENTRAL, MEDLINE, Embase and CINAHL) and trials registers (ClinicalTrials.gov and WHO ICTRP) to 4 June 2019. We conducted a top-up search to 23 September 2021; we screened these search results and 42 studies meeting our eligibility criteria are available in the awaiting classification section. SELECTION CRITERIA: We included randomised trials that assessed a QI programme to improve care in outpatient settings for people living with diabetes. QI programmes needed to evaluate at least one system- or provider-targeted QI strategy alone or in combination with a patient-targeted strategy. - System-targeted: case management (CM); team changes (TC); electronic patient registry (EPR); facilitated relay of clinical information (FR); continuous quality improvement (CQI). - Provider-targeted: audit and feedback (AF); clinician education (CE); clinician reminders (CR); financial incentives (FI). - Patient-targeted: patient education (PE); promotion of self-management (PSM); patient reminders (PR). Patient-targeted QI strategies needed to occur with a minimum of one provider or system-targeted strategy. DATA COLLECTION AND ANALYSIS: We dual-screened search results and abstracted data on study design, study population and QI strategies. We assessed the impact of the programmes on 13 measures of diabetes care, including: glycaemic control (e.g. mean glycated haemoglobin (HbA1c)); cardiovascular risk factor management (e.g. mean systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), proportion of people living with diabetes that quit smoking or receiving cardiovascular medications); and screening/prevention of microvascular complications (e.g. proportion of patients receiving retinopathy or foot screening); and harms (e.g. proportion of patients experiencing adverse hypoglycaemia or hyperglycaemia). We modelled the association of each QI strategy with outcomes using a series of hierarchical multivariable meta-regression models in a Bayesian framework. The previous version of this review identified that different strategies were more or less effective depending on baseline levels of outcomes. To explore this further, we extended the main additive model for continuous outcomes (HbA1c, SBP and LDL-C) to include an interaction term between each strategy and average baseline risk for each study (baseline thresholds were based on a data-driven approach; we used the median of all baseline values reported in the trials). Based on model diagnostics, the baseline interaction models for HbA1c, SBP and LDL-C performed better than the main model and are therefore presented as the primary analyses for these outcomes. Based on the model results, we qualitatively ordered each QI strategy within three tiers (Top, Middle, Bottom) based on its magnitude of effect relative to the other QI strategies, where 'Top' indicates that the QI strategy was likely one of the most effective strategies for that specific outcome. Secondary analyses explored the sensitivity of results to choices in model specification and priors.  Additional information about the methods and results of the review are available as Appendices in an online repository. This review will be maintained as a living systematic review; we will update our syntheses as more data become available. MAIN RESULTS: We identified 553 trials (428 patient-randomised and 125 cluster-randomised trials), including a total of 412,161 participants. Of the included studies, 66% involved people living with type 2 diabetes only. Participants were 50% female and the median age of participants was 58.4 years. The mean duration of follow-up was 12.5 months. HbA1c was the commonest reported outcome; screening outcomes and outcomes related to cardiovascular medications, smoking and harms were reported infrequently. The most frequently evaluated QI strategies across all study arms were PE, PSM and CM, while the least frequently evaluated QI strategies included AF, FI and CQI. Our confidence in the evidence is limited due to a lack of information on how studies were conducted.  Four QI strategies (CM, TC, PE, PSM) were consistently identified as 'Top' across the majority of outcomes. All QI strategies were ranked as 'Top' for at least one key outcome. The majority of effects of individual QI strategies were modest, but when used in combination could result in meaningful population-level improvements across the majority of outcomes. The median number of QI strategies in multicomponent QI programmes was three.  Combinations of the three most effective QI strategies were estimated to lead to the below effects:  - PR + PSM + CE: decrease in HbA1c by 0.41% (credibility interval (CrI) -0.61 to -0.22) when baseline HbA1c < 8.3%; - CM + PE + EPR: decrease in HbA1c by 0.62% (CrI -0.84 to -0.39) when baseline HbA1c > 8.3%;  - PE + TC + PSM: reduction in SBP by 2.14 mmHg (CrI -3.80 to -0.52) when baseline SBP < 136 mmHg; - CM + TC + PSM: reduction in SBP by 4.39 mmHg (CrI -6.20 to -2.56) when baseline SBP > 136 mmHg;  - TC + PE + CM: LDL-C lowering of 5.73 mg/dL (CrI -7.93 to -3.61) when baseline LDL < 107 mg/dL; - TC + CM + CR: LDL-C lowering by 5.52 mg/dL (CrI -9.24 to -1.89) when baseline LDL > 107 mg/dL. Assuming a baseline screening rate of 50%, the three most effective QI strategies were estimated to lead to an absolute improvement of 33% in retinopathy screening (PE + PR + TC) and 38% absolute increase in foot screening (PE + TC + Other). AUTHORS' CONCLUSIONS: There is a significant body of evidence about QI programmes to improve the management of diabetes. Multicomponent QI programmes for diabetes care (comprised of effective QI strategies) may achieve meaningful population-level improvements across the majority of outcomes. For health system decision-makers, the evidence summarised in this review can be used to identify strategies to include in QI programmes. For researchers, this synthesis identifies higher-priority QI strategies to examine in further research regarding how to optimise their evaluation and effects. We will maintain this as a living systematic review.

Understanding the Role of Values in Health Policy Decision-Making From the Perspective of Policy-Makers and Stakeholders: A Multiple-Case Embedded Study in Chile and Colombia.

BACKGROUND: Chile and Colombia are examples of Latin American countries with health systems shaped by similar values. Recently, both countries have crafted policies to regulate the participation of private for-profit insurance companies in their health systems, but through very different mechanisms. This study asks: what values are important in the decision-making processes that crafted these policies? And how and why are they used? METHODS: An embedded multiple-case study design was carried out for 2 specific decisions in each country: (1) in Chile, the development of the Universal Plan of Explicit Entitlements -AUGE/GES - and mandating universal coverage of treatments for high-cost diseases; and (2) in Colombia, the declaration of health as a fundamental right and a mechanism to explicitly exclude technologies that cannot be publicly funded. We interviewed key informants involved in one or more of the decisions and/or in the policy analysis and development process that contributed to the eventual decision. The data analysis involved a constant comparative approach and thematic analysis for each case study. RESULTS: From the 40 individuals who were invited, 28 key informants participated. A tension between 2 important values was identified for each decision (eg, solidarity vs. individualism for the AUGE/GES plan in Chile; human dignity vs. sustainability for the declaration of the right to health in Colombia). Policy-makers used values in the decisionmaking process to frame problems in meaningful ways, to guide policy development, as a pragmatic instrument to make decisions, and as a way to legitimize decisions. In Chile, values such as individualism and free choice were incorporated in decision-making because attaining private health insurance was seen as an indicator of improved personal economic status. In Colombia, human dignity was incorporated as the core value because the Constitutional Court asserted its importance in its use of judicial activism as a check on the power of the executive and legislative branches. CONCLUSION: There is an opportunity to open further exploration of the role of values in different health decisions, political sectors besides health, and even other jurisdictions.

Addressing the affordability of cancer drugs: using deliberative public engagement to inform health policy.

BACKGROUND: Health system expenditure on cancer drugs is rising rapidly in many OECD countries given the costly new treatments and increased rates of use due to a growing and ageing population. These factors put considerable strain on the sustainability of health systems worldwide, sparking public debate among clinicians, pharmaceutical companies, policy-makers and citizens on issues of affordability and equity. We engaged Canadians through a series of deliberative public engagement events to determine their priorities for making cancer drug funding decisions fair and sustainable in Canada's publicly financed health system. METHODS: An approach to deliberation was developed based on the McMaster Health Forum's citizen panels and the established Burgess and O'Doherty model of deliberative public engagement. Six deliberations were held across Canada in 2016. Transcripts were coded in NVivo and analysed to determine where participants' views converged and diverged. Recommendations were grouped thematically. RESULTS: A total of 115 Canadians participated in the deliberative events and developed 86 recommendations. Recommendations included the review and regular re-review of approved drugs using 'real-world' evidence on effectiveness and cost-effectiveness; prioritisation of treatments that restore patients' independence, mental health and general well-being; ensuring that decision processes, results and their rationales are transparent; and commitment to people with similar needs receiving the same care regardless of where in Canada they live. CONCLUSIONS: The next steps for policy-makers should be to develop mechanisms for (1) re-reviewing effectiveness and cost-effectiveness data for all cancer drugs; (2) making disinvestments in cancer drugs that satisfy requirements relating to grandfathering and compassionate access; (3) ensuring fair and equitable access to cancer drugs for all Canadians; and (4) fostering a pan-Canadian approach to cancer drug funding decisions.

Rol de la evidencia científica en las decisiones políticas relacionadas con los sistemas de salud / The role of research-based evidence in health system policy decision-making

RESUMEN Existen diferentes modelos para explicar cómo la evidencia de la investigación se utiliza en los procesos de formulación de políticas sobre los sistemas de salud. En este artículo argumentamos que los modelos que se desarrollaron desde el contexto clínico, como el de políticas basadas en la evidencia, pueden ser útiles en algunas decisiones políticas. Sin embargo, debido a su "silencio" sobre el contexto político, estos modelos son incompatibles con las decisiones relacionadas con la modificación de los arreglos de los sistemas de salud. Otros modelos, generados desde las ciencias políticas, son más útiles para entender que la investigación es uno solo de los factores que afecta la toma de decisiones y que diferentes tipos de evidencia científica pueden ser utilizados de manera instrumental, conceptual o estratégica en diferentes etapas del proceso de formulación de políticas.

ABSTRACT Different models may be used for explaining how research-based evidence is used in healthcare system policy-making. It is argued that models arising from a clinical setting (i.e. evidence-based policy-making model) could be useful regarding some types of healthcare system decision-making. However, such models are "silent" concerning the influence of political contextual factors on healthcare policy-making and are thus inconsistent with decision-making regarding the modification of healthcare system arrangements. Other political science-based models would seem to be more useful for understanding that research is just one factor affecting decision-making and that different types of research-based evidence can be used instrumentally, conceptual or strategically during different policy-making stages.

Interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers and clinicians.

BACKGROUND: Systematic reviews provide a transparent and robust summary of existing research. However, health system managers, national and local policy makers and healthcare professionals can face several obstacles when attempting to utilise this evidence. These include constraints operating within the health system, dealing with a large volume of research evidence and difficulties in adapting evidence from systematic reviews so that it is locally relevant. In an attempt to increase the use of systematic review evidence in decision-making a number of interventions have been developed. These include summaries of systematic review evidence that are designed to improve the accessibility of the findings of systematic reviews (often referred to as information products) and changes to organisational structures, such as employing specialist groups to synthesise the evidence to inform local decision-making. OBJECTIVES: To identify and assess the effects of information products based on the findings of systematic review evidence and organisational supports and processes designed to support the uptake of systematic review evidence by health system managers, policy makers and healthcare professionals. SEARCH METHODS: We searched The Cochrane Library, MEDLINE, EMBASE, CINAHL, Web of Science, and Health Economic Evaluations Database. We also handsearched two journals (Implementation Science and Evidence and Policy), Cochrane Colloquium abstracts, websites of key organisations and reference lists of studies considered for inclusion. Searches were run from 1992 to March 2011 on all databases, an update search to March 2012 was run on MEDLINE only. SELECTION CRITERIA: Randomised controlled trials (RCTs), interrupted time-series (ITS) and controlled before-after studies (CBA) of interventions designed to aid the use of systematic reviews in healthcare decision-making were considered. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted the data and assessed the study quality. We extracted the median value across similar outcomes for each study and reported the range of values for each median value. We calculated the median of the two middlemost values if an even number of outcomes were reported. MAIN RESULTS: We included eight studies evaluating the effectiveness of different interventions designed to support the uptake of systematic review evidence. The overall quality of the evidence was very low to moderate.Two cluster RCTs evaluated the effectiveness of multifaceted interventions, which contained access to systematic reviews relevant to reproductive health, to change obstetric care; the high baseline performance in some of the key clinical indicators limited the findings of these studies. There were no statistically significant effects on clinical practice for all but one of the clinical indicators in selected obstetric units in Thailand (median effect size 4.2%, range -11.2% to 18.2%) and none in Mexico (median effect size 3.5%, range 0.1% to 19.0%). In the second cluster RCT there were no statistically significant differences in selected obstetric units in the UK (median effect RR 0.92; range RR 0.57 to RR 1.10). One RCT evaluated the perceived understanding and ease of use of summary of findings tables in Cochrane Reviews. The median effect of the differences in responses for the acceptability of including summary of findings tables in Cochrane Reviews versus not including them was 16%, range 1% to 28%. One RCT evaluated the effect of an analgesic league table, derived from systematic review evidence, and there was no statistically significant effect on self-reported pain. Only one RCT evaluated an organisational intervention (which included a knowledge broker, access to a repository of systematic reviews and provision of tailored messages), and reported no statistically significant difference in evidence informed programme planning.Three interrupted time series studies evaluated the dissemination of printed bulletins based on evidence from systematic reviews. A statistically significant reduction in the rates of surgery for glue ear in children under 10 years (mean annual decline of -10.1%; 95% CI -7.9 to -12.3) and in children under 15 years (quarterly reduction -0.044; 95% CI -0.080 to -0.011) was reported. The distribution to general practitioners of a bulletin on the treatment of depression was associated with a statistically significant lower prescribing rate each quarter than that predicted by the rates of prescribing observed before the distribution of the bulletin (8.2%; P = 0.005). AUTHORS' CONCLUSIONS: Mass mailing a printed bulletin which summarises systematic review evidence may improve evidence-based practice when there is a single clear message, if the change is relatively simple to accomplish, and there is a growing awareness by users of the evidence that a change in practice is required. If the intention is to develop awareness and knowledge of systematic review evidence, and the skills for implementing this evidence, a multifaceted intervention that addresses each of these aims may be required, though there is insufficient evidence to support this approach.

Do provincial drug benefit initiatives create an effective policy lab? The evidence from Canada.

Although the costs of doctors' visits and hospital stays in Canada are covered by national public health insurance, the cost of outpatient prescription drugs is not. To solve problems of access, Canadian provinces have introduced provincial prescription drug benefit programs. This study analyzes the prescription drug policymaking process in five Canadian provinces between 1992 and 2004 with a view to (1) determining the federal government's role in the area of prescription drugs; (2) describing the policymaking process; (3) identifying factors in each province's choice of a policy; (4) identifying patterns in those factors across the five provinces; and (5) assessing the federal government's influence on the policies chosen. Analysis shows that despite significant differences in policy choices, the ideological motivations of the provinces were unexpectedly similar. The findings also highlight the importance of institutional factors, for example, in provinces' decision to compete rather than to collaborate. We conclude that, to date, Canada's federalism laboratory has only partly benefited the Canadian public. Cost pressures may, however, eventually overcome barriers to cooperation between the provincial and the federal governments, enabling them to capitalize on Canada's federal structure to improve the accessibility and affordability of drugs.

The Use of Research Evidence in Two International Organizations' Recommendations about Health Systems.

BACKGROUND: Little is known about the extent to which research evidence informs the development of recommendations by international organizations. METHODS: We identified specific World Health Organization (WHO) and World Bank recommendations on five topics (contracting, healthcare financing, health human resources, tuberculosis control and tobacco control), catalogued the related systematic reviews and assessed the recommendations to determine their consistency with the systematic reviews that were available at the time of their formulation. FINDINGS: Only two of the eight publications examined were found to cite systematic reviews, and only five of 14 WHO and two of seven World Bank recommendations were consistent with both the direction and nature of effect claims from systematic reviews. Ten of 14 WHO and five of seven World Bank recommendations were consistent with the direction of effect claims only. CONCLUSION: WHO and the World Bank - working with donor agencies and national governments - can improve their use of (or at least, their reporting about their use of) research evidence. Decision-makers and clinicians should critically evaluate the quality and local applicability of recommendations from any source, including international organizations, prior to their implementation.

Parallel payers and preferred access: how Canada's Workers' Compensation Boards expedite care for injured and ill workers.

Canada's workers' compensation boards (WCBs) finance healthcare for injured and ill workers in parallel with provincial health insurance plans. Parallel systems of healthcare finance can create preferred access for some. WCBs have in recent years pursued a number of strategies to expedite or improve the quality of care for injured or ill workers, including in-house provision in WCB-owned facilities; contracting with private, for-profit clinics; contracting with publicly funded hospitals and clinics for use of facilities "off-hours"; and supporting specialized clinics within publicly funded hospitals. Many of these strategies incorporate incentive payments to physicians and facilities for treating WCB cases more quickly than patients covered by provincial plans. In this paper we document the development of these strategies and discuss their implications for physicians, patients, government and the provincial public insurance plans.

A systematic review of studies comparing health outcomes in Canada and the United States.

BACKGROUND: Differences in medical care in the United States compared with Canada, including greater reliance on private funding and for-profit delivery, as well as markedly higher expenditures, may result in different health outcomes. OBJECTIVES: To systematically review studies comparing health outcomes in the United States and Canada among patients treated for similar underlying medical conditions. METHODS: We identified studies comparing health outcomes of patients in Canada and the United States by searching multiple bibliographic databases and resources. We masked study results before determining study eligibility. We abstracted study characteristics, including methodological quality and generalizability. RESULTS: We identified 38 studies comparing populations of patients in Canada and the United States. Studies addressed diverse problems, including cancer, coronary artery disease, chronic medical illnesses and surgical procedures. Of 10 studies that included extensive statistical adjustment and enrolled broad populations, 5 favoured Canada, 2 favoured the United States, and 3 showed equivalent or mixed results. Of 28 studies that failed one of these criteria, 9 favoured Canada, 3 favoured the United States, and 16 showed equivalent or mixed results. Overall, results for mortality favoured Canada (relative risk 0.95, 95% confidence interval 0.92-0.98, p= 0.002) but were very heterogeneous, and we failed to find convincing explanations for this heterogeneity. The only condition in which results consistently favoured one country was end-stage renal disease, in which Canadian patients fared better. INTERPRETATION: Available studies suggest that health outcomes may be superior in patients cared for in Canada versus the United States, but differences are not consistent.

Assessing country-level efforts to link research to action.

We developed a framework for assessing country-level efforts to link research to action. The framework has four elements. The first element assesses the general climate (how those who fund research, universities, researchers and users of research support or place value on efforts to link research to action). The second element addresses the production of research (how priority setting ensures that users' needs are identified and how scoping reviews, systematic reviews and single studies are undertaken to address these needs). The third element addresses the mix of four clusters of activities used to link research to action. These include push efforts (how strategies are used to support action based on the messages arising from research), efforts to facilitate "user pull" (how "one-stop shopping" is provided for optimally packaged high-quality reviews either alone or as part of a national electronic library for health, how these reviews are profiled during "teachable moments" such as intense media coverage, and how rapid-response units meet users' needs for the best research), "user pull" efforts undertaken by those who use research (how users assess their capacity to use research and how structures and processes are changed to support the use of research) and exchange efforts (how meaningful partnerships between researchers and users help them to jointly ask and answer relevant questions). The fourth element addresses approaches to evaluation (how support is provided for rigorous evaluations of efforts to link research to action).

Research, public policymaking, and knowledge-translation processes: Canadian efforts to build bridges.

Public policymakers must contend with a particular set of institutional arrangements that govern what can be done to address any given issue, pressure from a variety of interest groups about what they would like to see done to address any given issue, and a range of ideas (including research evidence) about how best to address any given issue. Rarely do processes exist that can get optimally packaged high-quality and high-relevance research evidence into the hands of public policymakers when they most need it, which is often in hours and days, not months and years. In Canada, a variety of efforts have been undertaken to address the factors that have been found to increase the prospects for research use, including the production of systematic reviews that meet the shorter term (but not urgent) needs of public policymakers and encouraging partnerships between researchers and policymakers that allow for their interaction around the tasks of asking and answering relevant questions. Much less progress has been made in making available research evidence to inform the urgent needs of public policymakers and in addressing attitudinal barriers and capacity limitations. In the future, knowledge-translation processes, particularly push efforts and efforts to facilitate user pull, should be undertaken on a sufficiently large scale and with a sufficiently rigorous evaluation so that robust conclusions can be drawn about their effectiveness.

Canadians confront health care reform.

In 2002 Canadians were less anxious about the state of their health care system than they were a few years earlier, when perceptions peaked that the system needed major reform. They expressed strong support in 2002 for maintaining the status quo on health care financing (that is, no user fees and no two-tier care) within the traditional domains of physician and hospital care. But they appeared more receptive to two-tier care and for-profit delivery for the newer and rapidly expanding domains of home care and high-tech care.