Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

Comparative Effectiveness and Safety of Trastuzumab Biosimilars to Herceptin for Adjuvant Treatment of HER2+ Breast Cancer.

Background: Ontario publicly funds reference trastuzumab (Herceptin) and four biosimilar trastuzumab products for adjuvant treatment of HER2+ breast cancer. We assessed the real-world safety and effectiveness of biosimilar trastuzumab compared to Herceptin for adjuvant treatment of patients with HER2+ breast cancer. Methods: This was a population-based, retrospective study comparing the safety and effectiveness of biosimilar trastuzumab and Herceptin for neoadjuvant/adjuvant treatment of HER2+ breast cancer from 2016 to 2021. Treatment patients started biosimilar trastuzumab from November 2019 to June 2021; historical comparator patients started Herceptin from June 2016 to October 2019. Safety outcomes death within 30 days of last dose of trastuzumab, direct hospitalization, emergency department visit leading to hospitalization, early treatment discontinuation, and in-patient admission for congestive heart failure were measured using logistic/negative binomial regression. Overall survival (OS) was measured using Kaplan-Meier methods and Cox proportional hazards regression. Propensity score matching was applied. Results: From June 2016 to 2021, 5071 patients with breast cancer were treated with neoadjuvant/adjuvant trastuzumab. The rate of direct hospitalization (RR: 0.85, 95% CI: 0.74-0.98, p-value: 0.032) was significantly lower in biosimilar compared to Herceptin patients. OS (log-rank test p = 0.98) and risk of mortality (HR: 1.29, 95% CI: 0.72-2.30, p-value = 0.39) did not significantly differ between treatment groups. Conclusions: Biosimilar trastuzumab demonstrated similar safety and effectiveness to Herceptin. The findings can help improve confidence in and use of biosimilars and demonstrate the value of real-world evidence generation for supporting biosimilar implementations and reassessments.

Evaluation of the reproducibility of responses to nasal allergen challenge and effects of inhaled nasal corticosteroids.

BACKGROUND: Similar immune responses in the nasal and bronchial mucosa implies that nasal allergen challenge (NAC) is a suitable early phase experimental model for drug development targeting allergic rhinitis (AR) and asthma. We assessed NAC reproducibility and the effects of intranasal corticosteroids (INCS) on symptoms, physiology, and inflammatory mediators. METHODS: 20 participants with mild atopic asthma and AR underwent three single blinded nasal challenges each separated by three weeks (NCT03431961). Cohort A (n = 10) underwent a control saline challenge, followed by two allergen challenges. Cohort B (n = 10) underwent a NAC with no treatment intervention, followed by NAC with 14 days pre-treatment with saline nasal spray (placebo), then NAC with 14 days pre-treatment with INCS (220 µg triamcinolone acetonide twice daily). Nasosorption, nasal lavage, blood samples, forced expiratory volume 1 (FEV1), total nasal symptom score (TNSS), peak nasal inspiratory flow (PNIF) were collected up to 24 h after NAC. Total and active tryptase were measured as early-phase allergy biomarkers (≤30 min) and IL-13 and eosinophil cell counts as late-phase allergy biomarkers (3-7 h) in serum and nasal samples. Period-period reproducibility was assessed by intraclass correlation coefficients (ICC), and sample size estimates were performed using effect sizes measured after INCS. RESULTS: NAC significantly induced acute increases in nasosorption tryptase and TNSS and reduced PNIF, and induced late increases in nasosorption IL-13 with sustained reductions in PNIF. Reproducibility across NACs varied for symptoms and biomarkers, with total tryptase 5 min post NAC having the highest reproducibility (ICC = 0.91). Treatment with INCS inhibited NAC-induced IL-13 while blunting changes in TNSS and PNIF. For a similar crossover study, 7 participants per treatment arm are needed to detect treatment effects comparable to INCS for TNSS. CONCLUSION: NAC-induced biomarkers and symptoms are reproducible and responsive to INCS. NAC is suitable for assessing pharmacodynamic activity and proof of mechanism for drugs targeting allergic inflammation.

Factors associated with intention for revaccination among patients with adverse events following immunization.

OBJECTIVES: Individuals and healthcare providers may be uncertain about the safety of revaccination after an adverse event following immunization (AEFI). We identified factors associated with physician recommendation for revaccination and participant intention to be revaccinated among patients with adverse events following immunization (AEFIs) assessed in the Canadian Special Immunization Clinic (SIC) Network from 2013 to 2019. METHODS: This prospective observational study included patients assessed in the Canadian Special Immunization Clinic Network from 2013 to 2019 for an AEFI who required additional doses of the vaccine temporally associated with their AEFI. Participants underwent standardized assessment and data collection. Physician recommendations regarding revaccination and participant intent for revaccination were recorded. AEFI impact on daily activities and need for medical attention was captured as low, moderate, high impact and serious (e.g., requiring hospitalization). Multivariable logistic regression analysis identified factors associated with physician recommendation and participant intention for revaccination, controlling for province of assessment. RESULTS: Physician recommendation was significantly associated with the type of AEFI and AEFI impact. Compared to large local reaction, physician recommendation for revaccination was reduced for immediate hypersensitivity (aOR: 0.24 [95% CI: 0.08-0.76]) and new onset autoimmune disease (aOR: 0.16; 95% CI: 0.04-0.69). Compared to low impact AEFIs, physician recommendation was reduced for moderate (aOR: 0.22 [95% CI: 0.07-0.65]), high impact (aOR: 0.08 [95% CI: 0.02-0.30]), and serious AEFIs (aOR: 0.11 [95% CI: 0.03-0.37]). Participant intention for revaccination was significantly associated with AEFI impact, with reduced odds for high versus low impact AEFIs (aOR: 0.12 [95% CI: 0.04-0.42]). CONCLUSION: Physicians appear to use AEFI type and impact to guide recommendations while patients use primarily AEFI impact to form intentions for revaccination. The findings may help inform counselling for patients with AEFIs.

Development of a Multi-Criteria Decision Analysis Rating Tool to Prioritize Real-World Evidence Questions for the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

The Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration developed an MCDA rating tool to assess and prioritize potential post-market real-world evidence (RWE) questions/uncertainties emerging from public drug funding decisions in Canada. In collaboration with a group of multidisciplinary stakeholders from across Canada, the rating tool was developed following a three-step process: (1) selection of criteria to assess the importance and feasibility of an RWE question; (2) development of rating scales, application of weights and calculating aggregate scores; and (3) validation testing. An initial MCDA rating tool was developed, composed of seven criteria, divided into two groups. Group A criteria assess the importance of an RWE question by examining the (1) drug's perceived clinical benefit, (2) magnitude of uncertainty identified, and (3) relevance of the uncertainty to decision-makers. Group B criteria assess the feasibility of conducting an RWE analysis including the (1) feasibility of identifying a comparator, (2) ability to identify cases, (3) availability of comprehensive data, and (4) availability of necessary expertise and methodology. Future directions include partnering with the Canadian Agency for Drugs and Technology in Health's Provincial Advisory Group for further tool refinement and to gain insight into incorporating the tool into drug funding deliberations.

Revaccination and Adverse Event Recurrence in Patients with Adverse Events following Immunization.

OBJECTIVES: To estimate the risk of recurrence of adverse events following immunization (AEFIs) upon revaccination and to determine among patients with suspected vaccine allergy whether allergy skin test positivity was associated with AEFI recurrence. STUDY DESIGN: This prospective observational study included patients assessed in the Canadian Special Immunization Clinic Network from 2013 to 2019 with AEFIs who required revaccination with the vaccine temporally associated with their AEFI. Participants underwent standardized assessment and data collection. Special Immunization Clinic physicians used guidelines to inform their recommendations. Participants were followed up after revaccination to capture AEFI recurrences. Data were transferred to a central database for descriptive analysis. RESULTS: Overall, 588 participants were assessed for 627 AEFIs; 570 (91%) AEFIs occurred in children <18 years of age. AEFIs included immediate hypersensitivity (130/627; 21%), large local reactions (110/627; 18%), nonurticarial rash (51/627; 8%), seizures (26/627; 4%), and thrombocytopenia (11/627; 2%). Revaccination was recommended to 513 of 588 (87%) participants. Among participants recommended and due for revaccination during the study period, 63% (299/477) were revaccinated. AEFI recurrence was 10% (31/299) overall, 31% (15/49) for large local reactions, and 7% (5/66) for immediate hypersensitivity. No recurrence was serious. Among 92 participants with suspected vaccine allergy who underwent skin testing and were revaccinated, the negative predictive value of skin testing for AEFI recurrence was 96% (95% CI 92.5%-99.5%). CONCLUSIONS: Most individuals with AEFIs were safely revaccinated. Among those with suspected vaccine allergy, skin testing may help determine the safety of revaccination.

Engaging Patients in the Canadian Real-World Evidence for Value in Cancer Drugs (CanREValue) Initiative: Processes and Lessons Learned.

The Canadian Real-world Evidence for Value in Cancer Drugs (CanREValue) Collaboration established the Engagement Working Group (WG) to ensure that all key stakeholders had an opportunity to provide input into the development and implementation of the CanREValue Real-World Evidence (RWE) Framework. Two consultations were held in 2021 to solicit patient perspectives on key policy and data access issues identified in the interim policy and data WG reports. Over 30 individuals, representing patients, caregivers, advocacy leaders, and individuals engaged in patient research were invited to participate. The consultations provided important feedback and valuable lessons in patient engagement. Patient leaders actively shaped the process and content of the consultation. Breakout groups facilitated by patient advocacy leaders gave the opportunity for open and thoughtful contributions from all participants. Important recommendations were made: the RWE framework should not impede access to new drugs; it should be used to support conditional approvals; patient relevant endpoints should be captured in provincial datasets; access to data to conduct RWE should be improved; and privacy issues must be considered. The manuscript documents the CanREValue experience of engaging patients in a consultative process and the useful contributions that can be achieved when the processes to engage are guided by patients themselves.

Mapping Canadian Data Assets to Generate Real-World Evidence: Lessons Learned from Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration's RWE Data Working Group.

Canadian provinces routinely collect patient-level data for administrative purposes. These real-world data (RWD) can be used to generate real-world evidence (RWE) to inform clinical care and healthcare policy. The CanREValue Collaboration is developing a framework for the use of RWE in cancer drug funding decisions. A Data Working Group (WG) was established to identify data assets across Canada for generating RWE of oncology drugs. The mapping exercise was conducted using an iterative scan with informant surveys and teleconference. Data experts from ten provinces convened for a total of three teleconferences and two in-person meetings from March 2018 to September 2019. Following each meeting, surveys were developed and shared with the data experts which focused on identifying databases and data elements, as well as a feasibility assessment of conducting RWE studies using existing data elements and resources. Survey responses were compiled into an interim data report, which was used for public stakeholder consultation. The feedback from the public consultation was used to update the interim data report. We found that databases required to conduct real-world studies are often held by multiple different data custodians. Ninety-seven databases were identified across Canada. Provinces held on average 9 distinct databases (range: 8-11). An Essential RWD Table was compiled that contains data elements that are necessary, at a minimal, to conduct an RWE study. An Expanded RWD Table that contains a more comprehensive list of potentially relevant data elements was also compiled and the availabilities of these data elements were mapped. While most provinces have data on patient demographics (e.g., age, sex) and cancer-related variables (e.g., morphology, topography), the availability and linkability of data on cancer treatment, clinical characteristics (e.g., morphology and topography), and drug costs vary among provinces. Based on current resources, data availability, and access processes, data experts in most provinces noted that more than 12 months would be required to complete an RWE study. The CanREValue Collaboration's Data WG identified key data holdings, access considerations, as well as gaps in oncology treatment-specific data. This data catalogue can be used to facilitate future oncology-specific RWE analyses across Canada.

Assessing cough symptom severity in refractory or unexplained chronic cough: findings from patient focus groups and an international expert panel.

Background: Cough symptom severity represents an important subjective end-point to assess the impact of therapies for patients with refractory or unexplained chronic cough (RCC/UCC). As existing instruments assessing the severity of cough are neither widely available nor tested for measurement properties, we aim to develop a new patient-reported outcome measure addressing cough severity. Objective: The aim of this study was to establish items and domains that would inform development of a new cough severity instrument. Methods: Three focus groups involving 16 adult patients with RCC/UCC provided data that we analysed using directed content analysis. Discussions led to consensus among an international panel of 15 experts on candidate items and domains to assess cough severity. Results: The patient focus group provided 48 unique items arranged under broad domains of urge-to-cough sensations and cough symptom. Feedback from expert panel members confirmed the appropriateness of items and domains, and provided an additional subdomain related to cough triggers. The final conceptual framework comprised 51 items in the following domains: urge-to-cough sensations (subdomains: frequency and intensity) and cough symptom (subdomains: triggers, control, frequency, fit/bout duration, intensity, quality and associated features/sequelae). Conclusions: Consensus findings from patients and international experts established domains of urge-to-cough and cough symptom with associated subdomains and relevant items. The results support item generation and content validity for a novel patient-reported outcome measure for use in health research and clinical practice.

Parenchyma-to-hydronephrosis Area Ratio Is a Promising Outcome Measure to Quantify Upper Tract Changes in Infants With High-grade Prenatal Hydronephrosis.

OBJECTIVE: To explore the value of renal parenchyma-to-hydronephrosis area ratio (PHAR) in detecting trends of hydronephrosis (HN) improvement or worsening and response to surgical intervention. METHODS: Initial and follow-up sagittal renal ultrasound images of patients entered into a prenatal HN database from 2008 to 2016, with baseline Society for Fetal Urology (SFU) grades III and IV HN and without vesicoureteral reflux, were evaluated using National Institutes of Health-sponsored image-processing software. Renal parenchymal area, hydronephrosis area (HA), PHAR, anteroposterior diameter (APd), and SFU grade were captured at baseline and most recent visit. Data were analyzed based on the need for surgical intervention to address obstruction. RESULTS: Out of 193 infants (159 boys; 135 left side), 58 (30%) underwent surgery. Patients managed surgically compared with those managed nonsurgically had worse baseline HN severity markers: SFU grade (3.6 ± 0.5 vs 3.1 ± 0.4; P <.001), urinary tract dilation classification (2.7 ± 0.5 vs 2.2 ± 0.4; P <.001), APd (20.3 ± 10.1 vs 12.8 ± 8.0; P <.001), HA (10.0 ± 6.6 vs 4.7 ± 2.8; P <.001), and PHAR (1.3 ± 1.0 vs 3.0 ± 2.9; P <.001); but both patient groups had similar renal parenchymal area (9.4 ± 3.5 vs 9.7 ± 2.8; P = .5). At last follow-up, the following discrepancies persisted: SFU grade (2.3 ± 1.0 vs 1.7 ± 1.0; P <.001), urinary tract dilation classification (1.5 ± 0.7 vs 1.0 ± 0.7; P <.001), APd (11.7 ± 8.0 vs 7.7 ± 5.7; P <.001), and HA (6.4 ± 5.1 vs 3.6 ± 2.7; P <.001); however, PHAR was equalized for both groups (7.2 ± 14.0 vs 7.1 ± 6.1; P = .9). CONCLUSION: By concurrently considering changes in renal parenchyma and degree of HN, we found that PHAR appears to be a promising parameter that reflects similarities between patients managed surgically and those managed nonsurgically, despite initial discrepancies. Our data suggest that this variable may provide reassurance and a more objective assessment of improvement after surgery compared with other traditional ultrasound outcome measures.