Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

Development of a Multi-Criteria Decision Analysis Rating Tool to Prioritize Real-World Evidence Questions for the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

The Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration developed an MCDA rating tool to assess and prioritize potential post-market real-world evidence (RWE) questions/uncertainties emerging from public drug funding decisions in Canada. In collaboration with a group of multidisciplinary stakeholders from across Canada, the rating tool was developed following a three-step process: (1) selection of criteria to assess the importance and feasibility of an RWE question; (2) development of rating scales, application of weights and calculating aggregate scores; and (3) validation testing. An initial MCDA rating tool was developed, composed of seven criteria, divided into two groups. Group A criteria assess the importance of an RWE question by examining the (1) drug's perceived clinical benefit, (2) magnitude of uncertainty identified, and (3) relevance of the uncertainty to decision-makers. Group B criteria assess the feasibility of conducting an RWE analysis including the (1) feasibility of identifying a comparator, (2) ability to identify cases, (3) availability of comprehensive data, and (4) availability of necessary expertise and methodology. Future directions include partnering with the Canadian Agency for Drugs and Technology in Health's Provincial Advisory Group for further tool refinement and to gain insight into incorporating the tool into drug funding deliberations.

Engaging Patients in the Canadian Real-World Evidence for Value in Cancer Drugs (CanREValue) Initiative: Processes and Lessons Learned.

The Canadian Real-world Evidence for Value in Cancer Drugs (CanREValue) Collaboration established the Engagement Working Group (WG) to ensure that all key stakeholders had an opportunity to provide input into the development and implementation of the CanREValue Real-World Evidence (RWE) Framework. Two consultations were held in 2021 to solicit patient perspectives on key policy and data access issues identified in the interim policy and data WG reports. Over 30 individuals, representing patients, caregivers, advocacy leaders, and individuals engaged in patient research were invited to participate. The consultations provided important feedback and valuable lessons in patient engagement. Patient leaders actively shaped the process and content of the consultation. Breakout groups facilitated by patient advocacy leaders gave the opportunity for open and thoughtful contributions from all participants. Important recommendations were made: the RWE framework should not impede access to new drugs; it should be used to support conditional approvals; patient relevant endpoints should be captured in provincial datasets; access to data to conduct RWE should be improved; and privacy issues must be considered. The manuscript documents the CanREValue experience of engaging patients in a consultative process and the useful contributions that can be achieved when the processes to engage are guided by patients themselves.