Exploiting relationships between outcomes in Bayesian multivariate network meta-analysis with an application to relapsing-remitting multiple sclerosis.

In multivariate network meta-analysis (NMA), the piecemeal nature of the evidence base means that there may be treatment-outcome combinations for which no data is available. Most existing multivariate evidence synthesis models are either unable to estimate the missing treatment-outcome combinations, or can only do so under particularly strong assumptions, such as perfect between-study correlations between outcomes or constant effect size across outcomes. Many existing implementations are also limited to two treatments or two outcomes, or rely on model specification that is heavily tailored to the dimensions of the dataset. We present a Bayesian multivariate NMA model that estimates the missing treatment-outcome combinations via mappings between the population mean effects, while allowing the study-specific effects to be imperfectly correlated. The method is designed for aggregate-level data (rather than individual patient data) and is likely to be useful when modeling multiple sparsely reported outcomes, or when varying definitions of the same underlying outcome are adopted by different studies. We implement the model via a novel decomposition of the treatment effect variance, which can be specified efficiently for an arbitrary dataset given some basic assumptions regarding the correlation structure. The method is illustrated using data concerning the efficacy and liver-related safety of eight active treatments for relapsing-remitting multiple sclerosis. The results indicate that fingolimod and interferon beta-1b are the most efficacious treatments but also have some of the worst effects on liver safety. Dimethyl fumarate and glatiramer acetate perform reasonably on all of the efficacy and safety outcomes in the model.

Recommendations for benefit-risk assessment methodologies and visual representations.

PURPOSE: The purpose of this study is to draw on the practical experience from the PROTECT BR case studies and make recommendations regarding the application of a number of methodologies and visual representations for benefit-risk assessment. METHODS: Eight case studies based on the benefit-risk balance of real medicines were used to test various methodologies that had been identified from the literature as having potential applications in benefit-risk assessment. Recommendations were drawn up based on the results of the case studies. RESULTS: A general pathway through the case studies was evident, with various classes of methodologies having roles to play at different stages. Descriptive and quantitative frameworks were widely used throughout to structure problems, with other methods such as metrics, estimation techniques and elicitation techniques providing ways to incorporate technical or numerical data from various sources. Similarly, tree diagrams and effects tables were universally adopted, with other visualisations available to suit specific methodologies or tasks as required. Every assessment was found to follow five broad stages: (i) Planning, (ii) Evidence gathering and data preparation, (iii) Analysis, (iv) Exploration and (v) Conclusion and dissemination. CONCLUSIONS: Adopting formal, structured approaches to benefit-risk assessment was feasible in real-world problems and facilitated clear, transparent decision-making. Prior to this work, no extensive practical application and appraisal of methodologies had been conducted using real-world case examples, leaving users with limited knowledge of their usefulness in the real world. The practical guidance provided here takes us one step closer to a harmonised approach to benefit-risk assessment from multiple perspectives.

A Bayesian approach to probabilistic sensitivity analysis in structured benefit-risk assessment.

Quantitative decision models such as multiple criteria decision analysis (MCDA) can be used in benefit-risk assessment to formalize trade-offs between benefits and risks, providing transparency to the assessment process. There is however no well-established method for propagating uncertainty of treatment effects data through such models to provide a sense of the variability of the benefit-risk balance. Here, we present a Bayesian statistical method that directly models the outcomes observed in randomized placebo-controlled trials and uses this to infer indirect comparisons between competing active treatments. The resulting treatment effects estimates are suitable for use within the MCDA setting, and it is possible to derive the distribution of the overall benefit-risk balance through Markov Chain Monte Carlo simulation. The method is illustrated using a case study of natalizumab for relapsing-remitting multiple sclerosis.

A case study using the PrOACT-URL and BRAT frameworks for structured benefit risk assessment.

While benefit-risk assessment is a key component of the drug development and maintenance process, it is often described in a narrative. In contrast, structured benefit-risk assessment builds on established ideas from decision analysis and comprises a qualitative framework and quantitative methodology. We compare two such frameworks, applying multi-criteria decision-analysis (MCDA) within the PrOACT-URL framework and weighted net clinical benefit (wNCB), within the BRAT framework. These are applied to a case study of natalizumab for the treatment of relapsing remitting multiple sclerosis. We focus on the practical considerations of applying these methods and give recommendations for visual presentation of results. In the case study, we found structured benefit-risk analysis to be a useful tool for structuring, quantifying, and communicating the relative benefit and safety profiles of drugs in a transparent, rational and consistent way. The two frameworks were similar. MCDA is a generic and flexible methodology that can be used to perform a structured benefit-risk in any common context. wNCB is a special case of MCDA and is shown to be equivalent to an extension of the number needed to treat (NNT) principle. It is simpler to apply and understand than MCDA and can be applied when all outcomes are measured on a binary scale.

Benefit-risk assessment in a post-market setting: a case study integrating real-life experience into benefit-risk methodology.

PURPOSE: Difficulties may be encountered when undertaking a benefit-risk assessment for an older product with well-established use but with a benefit-risk balance that may have changed over time. This case study investigates this specific situation by applying a formal benefit-risk framework to assess the benefit-risk balance of warfarin for primary prevention of patients with atrial fibrillation. METHODS: We used the qualitative framework BRAT as the starting point of the benefit-risk analysis, bringing together the relevant available evidence. We explored the use of a quantitative method (stochastic multi-criteria acceptability analysis) to demonstrate how uncertainties and preferences on multiple criteria can be integrated into a single measure to reduce cognitive burden and increase transparency in decision making. RESULTS: Our benefit-risk model found that warfarin is favourable compared with placebo for the primary prevention of stroke in patients with atrial fibrillation. This favourable benefit-risk balance is fairly robust to differences in preferences. The probability of a favourable benefit-risk for warfarin against placebo is high (0.99) in our model despite the high uncertainty of randomised clinical trial data. In this case study, we identified major challenges related to the identification of relevant benefit-risk criteria and taking into account the diversity and quality of evidence available to inform the benefit-risk assessment. CONCLUSION: The main challenges in applying formal methods for medical benefit-risk assessment for a marketed drug are related to outcome definitions and data availability. Data exist from many different sources (both randomised clinical trials and observational studies), and the variability in the studies is large.

A proposal for using benefit-risk methods to improve the prominence of adverse event results when reporting trials.

Adverse events suffer from poor reporting within randomised controlled trials, despite them being crucial to the evaluation of a treatment. A recent update to the CONSORT harms checklist aims to improve reporting by providing structure and consistency to the information presented. We propose an extension wherein harms would be reported in conjunction with effectiveness outcome(s) rather than in silo to provide a more complete picture of the evidence acquired within a trial. Benefit-risk methods are designed to simultaneously consider both benefits and risks, and therefore, we believe these methods could be implemented to improve the prominence of adverse events when reporting trials. The aim of this article is to use case studies to demonstrate the practical utility of benefit-risk methods to present adverse events results alongside effectiveness results. Two randomised controlled trials have been selected as case studies, the Option-DM trial and the SANAD II trial. Using a previous review, a shortlist of 17 benefit-risk methods which could potentially be used for reporting RCTs was created. From this shortlist, three benefit-risk methods are applied across the two case studies. We selected these methods for their usefulness to achieve the aim of this paper and which are commonly used in the literature. The methods selected were the Benefit-Risk Action Team (BRAT) Framework, net clinical benefit (NCB), and the Outcome Measures in Rheumatology (OMERACT) 3 × 3 table. Results using the benefit-risk method added further context and detail to the clinical summaries made from the trials. In the case of the SANAD II trial, the clinicians concluded that despite the primary outcome being improved by the treatment, the increase in adverse events negated the improvement and the treatment was therefore not recommended. The benefit-risk methods applied to this case study outlined the data that this decision was based on in a clear and transparent way. Using benefit-risk methods to report the results of trials can increase the prominence of adverse event results by presenting them alongside the primary efficacy/effectiveness outcomes. This ensures that all the factors which would be used to determine whether a treatment would be recommended are transparent to the reader.

Challenges and Opportunities of Real-World Data: Statistical Analysis Plan for the Optimise:MS Multicenter Prospective Cohort Pharmacovigilance Study.

Introduction: Optimise:MS is an observational pharmacovigilance study aimed at characterizing the safety profile of disease-modifying therapies (DMTs) for multiple sclerosis (MS) in a real world population. The study will categorize and quantify the occurrence of serious adverse events (SAEs) in a cohort of MS patients recruited from clinical sites around the UK. The study was motivated particularly by a need to establish the safety profile of newer DMTs, but will also gather data on outcomes among treatment-eligible but untreated patients and those receiving established DMTs (interferons and glatiramer acetate). It will also explore the impact of treatment switching. Methods: Causal pathway confounding between treatment selection and outcomes, together with the variety and complexity of treatment and disease patterns observed among MS patients in the real world, present statistical challenges to be addressed in the analysis plan. We developed an approach for analysis of the Optimise:MS data that will include disproportionality-based signal detection methods adapted to the longitudinal structure of the data and a longitudinal time-series analysis of a cohort of participants receiving second-generation DMT for the first time. The time-series analyses will use a number of exposure definitions in order to identify temporal patterns, carryover effects and interactions with prior treatments. Time-dependent confounding will be allowed for via inverse-probability-of-treatment weighting (IPTW). Additional analyses will examine rates and outcomes of pregnancies and explore interactions of these with treatment type and duration. Results: To date 14 hospitals have joined the study and over 2,000 participants have been recruited. A statistical analysis plan has been developed and is described here. Conclusion: Optimise:MS is expected to be a rich source of data on the outcomes of DMTs in real-world conditions over several years of follow-up in an inclusive sample of UK MS patients. Analysis is complicated by the influence of confounding factors including complex treatment histories and a highly variable disease course, but the statistical analysis plan includes measures to mitigate the biases such factors can introduce. It will enable us to address key questions that are beyond the reach of randomized controlled trials.

Evaluating the feasibility of a real world pharmacovigilance study (OPTIMISE:MS).

BACKGROUND: Clinical trial populations do not fully reflect routine practice. The power of routinely collected data to inform clinical practice is increasingly recognised. METHODS: The OPTIMISE:MS pharmacovigilance study is a prospective, pragmatic observational study, conducted across 13 UK MS centres. Data were collected at the time of routine clinical visits. The first participant was recruited on 24th May 2019; data were extracted on 11th November 2021. RESULTS: 2112 participants were included (median age 44.0 years; 1570 (72%) female; 1981 (94%) relapsing-remitting MS). 639 (30%) were untreated at study entry, 205 (10%) taking interferon beta/copaxone, 1004 (47%) second/third generation DMT first line and 264 (13%) had escalated from a platform DMT. 342 clinical events were reported, of which 108 infections. There was an increased risk of adverse events in people taking second/third generation DMT (RR 3.45, 95%CI 1.57-7.60, p<0.01 vs no DMT). Unadjusted Poisson regression demonstrated increased incident adverse events in people taking natalizumab (IRR 5.28, 95%CI 1.41-19.74, p<0.05), ocrelizumab (IRR 3.24, 95%CI 1.22-8.62, p<0.05), and GA biosimilar (Brabio) (IRR 4.89, 95%CI 1.31-18.21, p<0.05) vs no DMT. CONCLUSIONS: Routinely collected healthcare data can be used to evaluate DMT safety in people with MS. These data highlight the potential of pragmatic studies to guide understanding of risks and benefits associated with DMT.

OPTIMISE: MS study protocol: a pragmatic, prospective observational study to address the need for, and challenges with, real world pharmacovigilance in multiple sclerosis.

INTRODUCTION: The power of 'real world' data to improve our understanding of the clinical aspects of multiple sclerosis (MS) is starting to be realised. Disease modifying therapy (DMT) use across the UK is driven by national prescribing guidelines. As such, the UK provides an ideal country in which to gather MS outcomes data. A rigorously conducted observational study with a focus on pharmacovigilance has the potential to provide important data to inform clinicians and patients while testing the reliability of estimates from pivotal trials when applied to patients in the UK. METHODS AND ANALYSIS: The primary aim of this study is to characterise the incidence and compare the risk of serious adverse events in people with MS treated with DMTs. The OPTIMISE:MS database enables electronic data capture and secure data transfer. Selected clinical data, clinical histories and patient-reported outcomes are collected in a harmonised fashion across sites at the time of routine clinical visits. The first patient was recruited to the study on 24 May 2019. As of January 2021, 1615 individuals have baseline data recorded; follow-up data are being captured and will be reported in due course. ETHICS AND DISSEMINATION: This study has ethical permission (London City and East; Ref 19/LO/0064). Potential concerns around data storage and sharing are mitigated by the separation of identifiable data from all other clinical data, and limiting access to any identifiable data. The results of this study will be disseminated via publication. Participants provide consent for anonymised data to be shared for further research use, further enhancing the value of the study.