Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

Development of a Multi-Criteria Decision Analysis Rating Tool to Prioritize Real-World Evidence Questions for the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

The Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration developed an MCDA rating tool to assess and prioritize potential post-market real-world evidence (RWE) questions/uncertainties emerging from public drug funding decisions in Canada. In collaboration with a group of multidisciplinary stakeholders from across Canada, the rating tool was developed following a three-step process: (1) selection of criteria to assess the importance and feasibility of an RWE question; (2) development of rating scales, application of weights and calculating aggregate scores; and (3) validation testing. An initial MCDA rating tool was developed, composed of seven criteria, divided into two groups. Group A criteria assess the importance of an RWE question by examining the (1) drug's perceived clinical benefit, (2) magnitude of uncertainty identified, and (3) relevance of the uncertainty to decision-makers. Group B criteria assess the feasibility of conducting an RWE analysis including the (1) feasibility of identifying a comparator, (2) ability to identify cases, (3) availability of comprehensive data, and (4) availability of necessary expertise and methodology. Future directions include partnering with the Canadian Agency for Drugs and Technology in Health's Provincial Advisory Group for further tool refinement and to gain insight into incorporating the tool into drug funding deliberations.

Engaging Patients in the Canadian Real-World Evidence for Value in Cancer Drugs (CanREValue) Initiative: Processes and Lessons Learned.

The Canadian Real-world Evidence for Value in Cancer Drugs (CanREValue) Collaboration established the Engagement Working Group (WG) to ensure that all key stakeholders had an opportunity to provide input into the development and implementation of the CanREValue Real-World Evidence (RWE) Framework. Two consultations were held in 2021 to solicit patient perspectives on key policy and data access issues identified in the interim policy and data WG reports. Over 30 individuals, representing patients, caregivers, advocacy leaders, and individuals engaged in patient research were invited to participate. The consultations provided important feedback and valuable lessons in patient engagement. Patient leaders actively shaped the process and content of the consultation. Breakout groups facilitated by patient advocacy leaders gave the opportunity for open and thoughtful contributions from all participants. Important recommendations were made: the RWE framework should not impede access to new drugs; it should be used to support conditional approvals; patient relevant endpoints should be captured in provincial datasets; access to data to conduct RWE should be improved; and privacy issues must be considered. The manuscript documents the CanREValue experience of engaging patients in a consultative process and the useful contributions that can be achieved when the processes to engage are guided by patients themselves.

Mapping Canadian Data Assets to Generate Real-World Evidence: Lessons Learned from Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration's RWE Data Working Group.

Canadian provinces routinely collect patient-level data for administrative purposes. These real-world data (RWD) can be used to generate real-world evidence (RWE) to inform clinical care and healthcare policy. The CanREValue Collaboration is developing a framework for the use of RWE in cancer drug funding decisions. A Data Working Group (WG) was established to identify data assets across Canada for generating RWE of oncology drugs. The mapping exercise was conducted using an iterative scan with informant surveys and teleconference. Data experts from ten provinces convened for a total of three teleconferences and two in-person meetings from March 2018 to September 2019. Following each meeting, surveys were developed and shared with the data experts which focused on identifying databases and data elements, as well as a feasibility assessment of conducting RWE studies using existing data elements and resources. Survey responses were compiled into an interim data report, which was used for public stakeholder consultation. The feedback from the public consultation was used to update the interim data report. We found that databases required to conduct real-world studies are often held by multiple different data custodians. Ninety-seven databases were identified across Canada. Provinces held on average 9 distinct databases (range: 8-11). An Essential RWD Table was compiled that contains data elements that are necessary, at a minimal, to conduct an RWE study. An Expanded RWD Table that contains a more comprehensive list of potentially relevant data elements was also compiled and the availabilities of these data elements were mapped. While most provinces have data on patient demographics (e.g., age, sex) and cancer-related variables (e.g., morphology, topography), the availability and linkability of data on cancer treatment, clinical characteristics (e.g., morphology and topography), and drug costs vary among provinces. Based on current resources, data availability, and access processes, data experts in most provinces noted that more than 12 months would be required to complete an RWE study. The CanREValue Collaboration's Data WG identified key data holdings, access considerations, as well as gaps in oncology treatment-specific data. This data catalogue can be used to facilitate future oncology-specific RWE analyses across Canada.

Building a National Reassessment Process for Oncology Drugs: Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration through a Simulated Reassessment Exercise.

The CanREValue Collaboration established the Reassessment & Uptake Working Group to develop a preliminary process to reassess funded cancer drugs in Canada. A simulated exercise was conducted to evaluate the proposed reassessment process using a real-world case. We invited 32 attendees including representatives from Health Canada and Health Technology Assessment (HTA) agencies, along with payers, clinicians, academics, and patient representatives. A case was developed using a real-world study on a publicly funded cancer drug. In facilitated group sessions, participants were asked to deliberate upon the evidence presented in the case to issue reassessment recommendations. Several themes were identified through the deliberation discussions. While the generalizability of real-world evidence (RWE) is perceived as a strength, trust in the RWE depends largely on the source of the real-world data. The attendees suggested several improvements to the proposed reassessment process including evidence requirement for reassessment, recommendation categories, and a priori study protocols. This exercise generated important insights on the evidence required for conducting reassessment and considerations for improvements of the proposed reassessment process. Building upon lessons from this exercise, future work would continue to refine the reassessment process as part of the overall CanREValue framework.

Considerations for Developing a Reassessment Process: Report from the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration's Reassessment and Uptake Working Group.

The Canadian Real-world Evidence for Value in Cancer Drugs (CanREValue) Collaboration was established to develop a framework for generating and using real-world evidence (RWE) to inform the reassessment of cancer drugs following initial health technology assessment (HTA). The Reassessment and Uptake Working Group (RWG) is one of the five established CanREValue Working Groups. The RWG aims to develop considerations for incorporating RWE for HTA reassessment and strategies for using RWE to reassess drug funding decisions. Between February 2018 and December 2019, the RWG attended four teleconferences (with follow-up surveys) and two in-person meetings to discuss recommendations for the development of a reassessment process and potential barriers and facilitators. Modified Delphi methods were used to gather input. A draft report of recommendations (to December 2018) was shared for public consultation (December 2019 to January 2020). Initial considerations for developing a reassessment process were proposed. Specifically, reassessment can be initiated by diverse stakeholders, including decision makers from public drug plans or industry stakeholders. The reassessment process should be modelled after existing deliberation and recommendation frameworks used by HTA agencies. Proposed reassessment outcome categories include maintaining status quo, revisiting funding criteria, renegotiating price, or disinvesting. Overall, these initial considerations will serve as the basis for future advancements by the Collaboration.

Distance-limited walk tests post-stroke: A systematic review of measurement properties.

BACKGROUND: Improving walking capacity is a key objective of post-stroke rehabilitation. Evidence describing the quality and protocols of standardized tools for assessing walking capacity can facilitate their implementation. OBJECTIVE: To synthesize existing literature describing test protocols and measurement properties of distance-limited walk tests in people post-stroke. METHODS: Electronic database searches were completed in 2017. Records were screened and appraised for quality. RESULTS: Data were extracted from 43 eligible articles. Among the 12 walk tests identified, the 10-metre walk test (10mWT) at a comfortable pace was most commonly evaluated. Sixty-three unique protocols at comfortable and fast paces were identified. Walking pace and walkway surface, but not walkway length, influenced walking speed. Intraclass correlation coefficients for test-retest reliability ranged from 0.80-0.99 across walk tests. Measurement error values ranged from 0.04-0.40 and 0.06 to 0.20 for the 10mWT at comfortable and fast and paces, respectively. Across walk tests, performance was most frequently correlated with measures of strength, balance, and physical activity (r = 0.26-0.8, p < 0.05). CONCLUSIONS: The 10mWT has the most evidence of reliability and validity. Findings indicate that studies that include people with severe walking deficits, in acute and subacute phases of recovery, with improved quality of reporting, are needed.

Considerations for the Selection of Time-Limited Walk Tests Poststroke: A Systematic Review of Test Protocols and Measurement Properties.

BACKGROUND AND PURPOSE: Systematic reviews of research evidence describing the quality and methods for administering standardized outcome measures are essential to developing recommendations for their clinical application. The purpose of this systematic review was to synthesize the research literature describing test protocols and measurement properties of time-limited walk tests in people poststroke. METHODS: Following an electronic search of 7 bibliographic data-bases, 2 authors independently screened titles and abstracts. One author identified eligible articles, and performed quality appraisal and data extraction. RESULTS: Of 12 180 records identified, 43 articles were included. Among 5 walk tests described, the 6-minute walk test (6MWT) was most frequently evaluated (n = 36). Only 5 articles included participants in the acute phase (<1 month) poststroke. Within tests, protocols varied. Walkway length and walking aid, but not turning direction, influenced 6MWT performance. Intraclass correlation coefficients for reliability were 0.68 to 0.71 (12MWT) and 0.80 to 1.00 (2-, 3-, 5- and 6MWT). Minimal detectable change values at the 90% confidence level were 11.4 m (2MWT), 24.4 m (5MWT), and 27.7 to 52.1 m (6MWT; n = 6). Moderate-to-strong correlations (≥0.5) between 6MWT distance and balance, motor function, walking speed, mobility, and stair capacity were consistently observed (n = 33). Moderate-to-strong correlations between 5MWT performance and walking speed/independence (n = 1), and between 12MWT performance and balance, motor function, and walking speed (n = 1) were reported. DISCUSSION AND CONCLUSIONS: Strong evidence of the reliability and construct validity of using the 6MWT poststroke exists; studies in the acute phase are lacking. Because protocol variations influence performance, a standardized 6MWT protocol poststroke for use across the care continuum is needed.Video Abstract available for more insights from the authors (see Supplemental Digital Content 1, http://links.lww.com/JNPT/A150).

Reference values for standardized tests of walking speed and distance: a systematic review.

OBJECTIVE: To provide an overview of the reference values and methodology used to obtain them for time- and distance-limited walk tests. METHODS: We performed a systematic review and searched PubMed, MEDLINE (Ovid), EMBASE, CINAHL, Scopus, PEDro, and The Cochrane Library from 1946 to May 2013. Full-text peer-reviewed articles written in English, French or Spanish were considered eligible. Two authors independently screened titles and abstracts. One author determined eligibility of full-text articles, appraised methodological quality, and extracted data. A second author independently verified the accuracy of extracted data. RESULTS: Of the 41 eligible studies reviewed, 25 failed to describe the method used to select participants and 10 had an inadequate sample size. Twenty-five studies provided reference values for one time-limited walk test (6-min walk test (6 MWT)) and 18 studies provided reference values for 15 distance-limited walk tests. Across studies, walk test distances ranged from 3m to 40m. Descriptive values and reference equations for the 6 MWT were reported in 15 and 20 studies, respectively. Across 43 regression equations (median R(2)=0.46), age (98%) and sex (91%) were most frequently included. The equation yielding the maximum R(2) value (0.78) included age, height, weight and percentage of predicted maximum heart rate. Among six unique regression equations for distance-limited walk tests (median R(2)=0.17), sex (83%), age (67%) and weight (67%) were most frequently included. The equation yielding the maximum R(2) value (0.25) included age and sex. CONCLUSIONS: Reference values reported for these tests provide a basis for classifying walking capacity as within normal limits, determining the magnitude of deficit, educating clients, setting rehabilitation goals, and planning studies.

Speed and distance requirements for community ambulation: a systematic review.

OBJECTIVE: To provide an overview of the research literature on distance and speed requirements for adults to walk outside the home. DATA SOURCES: We conducted a systematic review and searched PubMed, MEDLINE (Ovid), EMBASE, CINAHL, Scopus, PEDro, and The Cochrane Library from 1948 to May 2012, and other sources. Search terms included communities, walk, ambulation, and neighborhood. STUDY SELECTION: Full-text peer-reviewed articles written in English, French, or Spanish reporting distance and/or speed requirements for individuals walking outside the home were considered eligible. Two authors independently screened titles and abstracts. One author reviewed full-text articles to determine inclusion. Of the 3191 titles and abstracts screened, 15 studies (.47%) were selected for detailed review. One author appraised methodological quality. Inadequate description of the reliability of the measurement methods and the population of the town/city assessed was noted. DATA EXTRACTION: One author extracted data from included studies. A second reviewer independently verified extracted data for accuracy. DATA SYNTHESIS: Seven studies examining 24 community sites and crosswalks in the United States, Australia, and Singapore were included. Three sites with the largest mean distance requirements for adults to walk were club warehouses (677m), superstores (183-607m), and hardware stores (566m). Three sites with the lowest mean distance requirements were walking at the front (16m) and back (19m) of the house, and at cemeteries (18m). The average speed required to cross the street in the time of a walk signal varied from .44 to 1.32m/s. CONCLUSIONS: Distance and speed requirements for adults to walk in the community environment vary widely. Findings are relevant to judging capacity for community ambulation to carry out essential activities of daily living, educating patients, and setting rehabilitation goals.