Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

The effects of provider contract types and fiscal decentralization on the efficiency of the Polish hospital sector: A data envelopment analysis across 16 health regions.

BACKGROUND: Expenditures on hospitals constitute a large proportion of total health expenditures. In Poland the share of hospital spending is higher than the European Union average. The efficiency of the Polish hospital sector merits investigation. RESEARCH GOAL: The goal of this study is to estimate the relative technical efficiencies of Polish hospital regions, and to measure the impacts of provider contract types, fiscal decentralization, and the 2017 reform on the relative efficiencies. METHODS: Using data from 16 hospital regions in Poland for the years 2007 to 2019, we estimate relative technical efficiencies and their determinants using a two-stage approach. In the first stage, we apply Data Envelopment Analysis, in which we assume variable returns to scale and an output orientation. In the second stage we use a truncated regression with double bootstrapping. RESULTS: Our findings are threefold. First, fiscal decentralization may reduce technical efficiency, but the results was not statistically significant. Second, efficiency tended to be higher in regions where a greater proportion of nurses and midwives were offered employment as opposed to consulting contracts. Contract types offered to physicians were not statistically significant. Third, the 2017 reforms seem to have had a positive impact on efficiency to date. CONCLUSION: Policy makers may wish to consider offering less fiscal autonomy and control to regions and encouraging employment contracts with nurses and midwives.

Study protocol for CELLAR (COVID-related Eating Limitations and Latent dietary effects in the Atlantic Region): population-based observational study to monitor dietary intakes and purchasing during COVID-19 in four Atlantic Canadian provinces.

INTRODUCTION: Poor diet is a leading preventable risk for the global burden of non-communicable disease. Robust measurement is needed to determine the effect of COVID-19 on dietary intakes and consumer purchasing, given the widespread changes to consumer food environments and economic precarity. The research objectives are as follows: (1) describe dietary intakes of foods, beverages and nutrients of concern during the COVID-19 pandemic; (2) quantify change in diet during COVID-19 as compared with prepandemic, previously captured in the provincial samples of the population-representative 2015 Canadian Community Health Survey-Nutrition and (3) examine how household purchasing practices predict dietary intakes during COVID-19. METHODS AND ANALYSES: Observational study of diet, using a population-based stratified probability sampling strategy allocated via dual-frame (landline and cellphone) calls to random-digit dialled numbers, followed by age-sex group quotas. The base population comprises the four provinces of the Atlantic region of Canada, jurisdictions with an excess burden of pre-existing dietary risk, compared with the rest of Canada. Our aim is n=1000 to obtain reliable estimates at a regional level to describe intakes and compare with prepandemic baseline. Data collection entails 12 weeks participation: (1) enrolment with sociodemographics (key dietary risk predictors such as age, sex, gender, pre-COVID-19 income, employment, household composition, receipt of economic relief, rural residence); (2) two 24hour diet recalls using the online ASA-24 Canada 2018 tool; and (3) online uploads of household food purchase receipts over the 12 weeks enrolled. Participation incentives will be offered. ETHICS AND DISSEMINATION: This research protocol received funding from the Canadian Institutes of Health Research (FRN VR5 172691) and ethics review approval from the Dalhousie University Research Ethics Board. Study protocol and instruments and a de-identified dataset will be made publicly available. We will submit the findings to peer-reviewed journals, as well as conferences geared towards scientific and decision-maker audiences.

The most important facilitators and barriers to the use of Health Technology Assessment in Canada: a best-worst scaling approach.

BACKGROUND: Health Technology Assessment (HTA), which can support public drug reimbursement decisions will play a core function in the planned national Pharmacare program in Canada. To address existing barriers to the use of HTA, these must be ranked in order of priority. The goal of this study was to access the relative importance of known facilitators and barriers to the use of HTA in the context of the Canadian health care system, with attention to differences between regions and stakeholder groups. METHODS: We used the best-worst scaling object case approach to elicit a quantitative ranking of a list of 20 facilitators and 22 barriers. A sample of 68 Canadian HTA stakeholders, including members of expert committees, decision/policymakers, researchers/academics, and others participated in the study. Their task was to identify the most important and the least important item in 12 sub-sets of five facilitators and 14 sub-sets of five barriers. FINDINGS: Relative Importance Scores derived via hierarchical Bayes analysis revealed relations, engagement, and contact between stakeholders as most important on both the barrier and facilitator sides. Other top-ranked facilitators included the availably of credible and relevant research. Other top-ranked barriers included inconsistencies in the evidence and limited generalizability. The availability of HTA guidelines did not rank highly on either side. The main limitation of the study was the challenge with reaching the relevant respondents; this was mitigated by involving the national HTA agency in the research. CONCLUSION: Canadian stakeholders consider the relationships within the HTA network among the most important. Policies should focus on strengthening these relationships. Future research should focus on the connectivity and distribution of knowledge and power within the HTA network.

Uncertainty tolerance among experts involved in drug reimbursement recommendations: Qualitative evidence from HTA committees in Canada and Poland.

OBJECTIVES: Drug reimbursement decisions often rely on health technology assessment (HTA). Increasingly, new drugs have limited clinical evidence and uncertain clinical benefit. Our goal was to describe how members of drug advisory committees and other stakeholders conceptualize and tolerate uncertainty and how they rationalize uncertainty tolerance. METHODS: Our triangulated parallel design applied two qualitative methods. We interviewed 31 members of drug advisory committees in Canada and Poland about their information needs and included hypothetical scenarios with uncertain clinical benefits. Respondents speculated about their likely reimbursement recommendation. We analyzed written recommendations of the pan Canadian Oncology Drug Review for drugs with uncertain benefit and compared initial recommendations to the responses from patient and clinician groups. RESULTS: Uncertainty tolerance varied among committee members and across jurisdictions. In the scenario analysis, 7 Canadian and 11 Polish respondents leaned against recommending a hypothetical drug with uncertain clinical benefit, whereas 5 Canadian and 5 Polish respondents leaned in favour. Those against rationalized that uncertainty increases potential harm; those in favour rationalized that patients often have no alternatives. The document analysis revealed that patients had higher uncertainty tolerance in general. CONCLUSIONS: Uncertainty tolerance varies among committee members and other stakeholders depending on their backgrounds and on the decision contexts. We argue that policy guidance around uncertainty management could improve the transparency and consistency of recommendations.

Ranking the Criteria Used in the Appraisal of Drugs for Reimbursement: A Stated Preferences Elicitation With Health Technology Assessment Stakeholders Across Jurisdictional Contexts.

OBJECTIVES: Our goal was to estimate the relative importance assigned to health technology assessment (HTA) criteria by stakeholders involved in the HTA process. HTA is an increasingly common framework used in the appraisal of drugs for public reimbursement. It identifies clinical, economic, social, and organizational criteria to be considered. The criteria can vary across jurisdictions and are typically appraised by multidisciplinary expert committees. Guidance on the relative weighing of criteria is often absent. METHODS: We elicited stakeholders' preferences using a single-scenario discrete choice experiment and a best-worst scaling model with conviction scores to assess the weights assigned to selected criteria by HTA stakeholders. We recruited 111 HTA stakeholders across multiple jurisdictions, including members of expert committees, clinical and economic experts, patients, and public payer representatives. Each judged twelve hypothetical cancer drug profiles for suitability for public funding and identified which characteristics were best and worst. In addition to standard discrete choice experiment and best-worst scaling models, we estimated a hybrid model to obtain a ranking of criteria by importance they played in the appraisal. RESULTS: A strong clinical benefit proved the most important criterion, followed by cost considerations, presence of adverse events, and availability of other treatments. The importance of clinical benefit was moderated by unmet need, adverse events, and number of patients. CONCLUSION: Policymakers might want to consider providing an explicit weighing scheme, or moving to a 2-stage selection process with an assessment of the quality of clinical evidence as a gatekeeping step for a full HTA review.

Implications of interprofessional primary care team characteristics for health services and patient health outcomes: A systematic review with narrative synthesis.

Interprofessional primary care (IPPC) teams are promoted as an alternative to single profession physician practices in primary care with focus on preventive care and chronic disease management. Characteristics of teams can have an impact on their performance. We synthesized quantitative, qualitative or mixed-methods evidence addressing the design of IPPC teams. We searched Ovid MEDLINE, Embase, CINAHL, and PAIS using search terms focused on IPPC teams. Studies were included if they discussed the influence of team structure, organization, financial arrangements, or policies and procedures, or either health care processes or outputs, health outcomes, or costs, and were conducted in Australia, Canada, the United Kingdom or New Zealand between 2003 and 2016. We screened 11,707 titles, 5366 abstracts, and selected 77 full text articles (38 qualitative, 31 quantitative and 8 mixed-methods). Literature focused on the implications of team characteristics on team processes, such as teamwork, collaboration, or satisfaction of patients or providers. Despite heterogeneity of contexts, some trends are observable: shared space, common vision and goals, clear definitions of roles, and leadership as important to good teamwork. The impacts of these on health care outputs or patient health are not clear. To move the state of knowledge beyond perception of what works well for IPPC teams, researchers should focus on quantitative causal inference about the linkages between team characteristics and patient health.

Drug attributes associated with the selection of drugs for reimbursement: a pilot stated preferences experiment with Canadian stakeholders.

INTRODUCTION: Health Technologies Assessment requires that evidence about clinical, economic, social, and organizational aspects be considered and weighted in the selection of drugs for reimbursement. We investigate how evidence is balanced by committee members in Canada, where neither explicit weighing schemes nor thresholds are provided. METHODS: Thirty-six past and present members of cancer drug appraisal committees participated in an online stated preferences experiment. The experiment included a ranking of drug attributes, a discrete choice experiment asking to vote in favor or against the funding of drugs described using five attributes, and a best-worst scaling experiment using the same drug descriptions. RESULTS: Respondents focused on the clinical attributes of drugs, particularly on the survival benefit relative to a comparator. As a second criterion, respondents either consider economic attributes or they consider patient relevant attributes, depending on how questions are framed. The small sample size is a limitation to generalizability. CONCLUSION: Understanding how individuals involved in HTA weigh evidence is important to the development of policy guidelines for the drug selection process. Our pilot results suggest that non-clinical criteria can become marginalized in the appraisal process in the absence of clear guidelines to their use. Avenues for further research are discussed.

Threats to the value of Health Technology Assessment: Qualitative evidence from Canada and Poland.

BACKGROUND: Health Technology Assessment is used to support the process of drug appraisal and reimbursement decisions in a variety of health systems. Examples can be found in mature Western countries, such as Canada, and in emerging economies of Central and Eastern Europe, such as Poland. The value of HTA in the process is influenced by the evidence used and the stakeholders involved. METHODS: Qualitative interviews with 29 members of two appraisal committees were held in Canada and Poland between July 2017 and March 2018. An a priori thematic framework was applied and supplemented with emergent themes. RESULTS: We report on the results of a core emergent theme - threats identified by respondents to the value of HTA in the formulary process. We classified these into internal threats that arise due to undue influence on the individuals involved in appraisal, and external threats that arise due to undue influence on the production of evidence. DISCUSSION: Findings align with previous evidence regarding political and corporate pressures on the process, and a perception of declining quality of evidence. We contribute to the discussion by highlighting the importance of motivation of experts involved in the appraisal process. CONCLUSIONS: The recognition of internal and external threats lays the groundwork for a discussion of policies used to mitigate them. We offer suggestions about potential policy responses.

Funding models and medical dominance in interdisciplinary primary care teams: qualitative evidence from three Canadian provinces.

BACKGROUND: Primary care in Canada is the first point of entry for patients needing specialized services, the fundamental source of care for those living with chronic illness, and the main supplier of preventive services. Increased pressures on the system lead to changes such as an increased reliance on interdisciplinary teams, which are advocated to have numerous advantages. The functioning of teams largely depends on inter-professional relationships that can be supported or strained by the financial arrangements within teams. We assess which types of financial environments perpetuate and which reduce the challenge of medical dominance. METHODS: Using qualitative interview data from 19 interdisciplinary teams/networks in three Canadian provinces, as well as related policy documents, we develop a typology of financial environments along two dimensions, financial hierarchy and multiplicity of funding sources. A financial hierarchy is created when the incomes of some providers are a function of the incomes of other providers. A multiplicity of funding sources is created when team funding is provided by several funders and a team faces multiple lines of accountability. RESULTS: We argue that medical dominance is perpetuated with higher degrees of financial hierarchy and higher degrees of multiplicity. We show that the financial environments created in the three provinces have not supported a reduction in medical dominance. The longstanding Community Health Centre model, however, displays the least financial hierarchy and the least multiplicity-an environment least fertile for medical dominance. CONCLUSIONS: The functioning of interdisciplinary primary care teams can be negatively affected by the unique positioning of the medical profession. The financial environment created for teams is an important consideration in policy development, as it plays an important role in establishing inter-professional relationships. Policies that reduce financial hierarchies and funding multiplicities are optimal in this regard.