Quantifying the benefit-risk trade-off for individual patients in a clinical trial: principles and antithrombotic case study.

BACKGROUND: A treatment's overall favorable benefit-risk profile does not imply that every individual patient will benefit from the treatment. OBJECTIVES: To describe a statistical methodology for quantifying the benefit-risk trade-off in individual patients. METHODS: The method requires a large randomized controlled trial containing a primary efficacy outcome and a primary safety outcome, for instance, the Thrombin Receptor Antagonist in Secondary Prevention of Atherothrombotic Ischemic Events-Thrombolysis in Myocardial Infarction 50 placebo-controlled trial of vorapaxar in 17 779 patients following myocardial infarction. Multivariate regression models predict each individual patient's risk of ischemic events (benefit) and major bleeding events (harm) based on their profile. Hence, each patient's predicted benefit from vorapaxar (reduction in ischemic events) and predicted risk (increase in bleeding events) were estimated. The relative importance of ischemic and bleeding events based on links to all-cause mortality was quantified, although the limitations of such weightings are noted. RESULTS: Overall results demonstrated both clear benefit and harm from vorapaxar. Substantial interindividual variation in both benefit and risk facilitated distinguishing patients with a favorable benefit-risk trade-off from those who did not. Such findings were applied to recommend vorapaxar in as many as 98.3% of patients in which a favorable mortality-weighted benefit-risk trade-off was present, in 77.2% of patients with ischemic benefit 20% greater than bleeding risk, or in as few as 45.5% of patients if an annual decrease in ischemic risk of ≥0.5% was also required. CONCLUSION: While overall randomized controlled trials of treatment benefit vs risk are valuable, models determining each individual patient's estimated absolute benefit and risk provide more useful insight regarding patient-specific benefit-risk trade-offs to better enable personalized therapeutic decision-making.

An indirect treatment comparison (ITC) and matching-adjusted indirect comparison (MAIC) between a 15-valent (V114) and a 20-valent (PCV20) pneumococcal conjugate vaccine among healthy infants.

OBJECTIVES: Immunogenicity between 15-valent V114 (PCV15) and 20-valent PCV20 pneumococcal conjugate vaccines in healthy infants is compared in an indirect treatment comparison and matching-adjusted indirect comparison. Hypotheses: immunogenicity of V114 is non-inferior to PCV20 for all PCV13 serotypes, and superior to PCV20 for serotype 3 based on lower bound margins. METHODS: Two phase 3 pivotal studies on 3 + 1 pediatric vaccination schedule at age 2, 4, 6, and 12-15 months compared V114 (N = 858) to PCV13 (N = 856) and PCV20 (N = 1001) to PCV13 (N = 987). Infant's age and race in V114 study were matched to those in PCV20 study. Primary endpoints were serotype-specific Immunoglobulin G (IgG) response rate difference (RRD) 30 days post-dose (PD)3; IgG geometric mean concentration (GMC) ratios 30 days PD3 and PD4. RESULTS: V114 was non-inferior (marginRRD>-10%-point; marginGMCratio >0.5) to PCV20 (p-value <0.001) for all endpoints. V114 was superior (marginRRD >0%-point; marginGMCratio >1.2) to PCV20 (p-value <0.001) for serotype 3: RRD was 34.5% (95%CI 27.9%-41.1%) PD3, and IgG GMC ratios were 2.39 (95%CI 2.12-2.68) PD3 and 2.15 (95%CI 1.90-2.41) PD4. CONCLUSION: Immune response to V114 administered in a 3 + 1 schedule in healthy infants was considered non-inferior to PCV20 for all 13 PCV13 serotypes and superior for serotype 3 PD3 and PD4. CLINICAL TRIAL REGISTRATION: www.clinicaltrials.gov identifiers NCT03893448, NCT04382326.

Regulatory benefit-risk assessment of oncology drugs: A systematic review of FDA and EMA approvals.

The European Medicines Agency (EMA) and FDA have policy goals of strengthening benefit-risk (B-R) capabilities; but how this has been translating into regulatory practice is unclear. A systematic review of oncology drug approvals between 2015 and 2020 was conducted with approvals identified through review of FDA and EMA annual reports, with extraction of information on submission, clinical program and B-R assessment from publicly available review documents. Data were extracted from 236 reviews (EMA: 66 new submissions, 100 label extensions; FDA: 70 new submissions). The standard of evidence for B-R assessments seems to have diversified over time; yet, despite policy targets to extend their use, these assessments rarely include patient experience or real-world data.

Design of paediatric trials with benefit-risk endpoints using a composite score of adverse events of interest (AEI) and win-statistics.

A fundamental problem in the regulatory evaluation of a therapy is assessing whether the benefit outweighs the associated risks. This work proposes designing a trial that assesses a composite endpoint consisting of benefit and risk, hence, making the core of the design of the study, to assess benefit and risk. The proposed benefit risk measure consists of efficacy measure(s) and a risk measure that is based on a composite score obtained from pre-defined adverse events of interest (AEI). This composite score incorporates full aspects of adverse events of interest (i.e. the incidence, severity, and duration of the events). We call this newly proposed score the AEI composite score. After specifying the priorities between the components of the composite endpoint, a win-statistic (i.e. win ratio, win odds, or net benefit) is used to assess the difference between treatments in this composite endpoint. The power and sample size requirements of such a trial design are explored via simulation. Finally, using Dupixent published adult study results, we show how we can design a paediatric trial where the primary outcome is a composite of prioritized outcomes consisting of efficacy endpoints and the AEI composite score endpoint. The resulting trial design can potentially substantially reduce sample size compared to a trial designed to assess the co-primary efficacy endpoints, therefore it may address the challenge of slow enrollment and patient availability for paediatric studies.

Matching-adjusted indirect comparison of pneumococcal vaccines V114 and PCV20.

BACKGROUND: V114 (15-valent pneumococcal conjugate vaccine [PCV15]) and a 20-valent PCV (PCV20) are approved for adults (≥18 years) in the United States. We present methodologies to indirectly compare immune responses to V114 versus PCV20. RESEARCH DESIGN AND METHODS: Indirect treatment comparison and matching-adjusted indirect comparison (MAIC) were performed to estimate opsonophagocytic activity (OPA) geometric mean titer (GMT) ratios of V114/PCV20 at 30 days post-vaccination with PCV13 as common comparator for 13 serotypes (STs) shared with a 13-valent PCV (PCV13) among pneumococcal vaccine-naïve adults aged ≥60 years. Data from three V114 studies were pooled (V114, N = 2,196; PCV13, N = 843). In the MAIC analysis, data were reweighted, matching participant age and sex in NCT03760146 (PCV20, N = 1,507; PCV13, N = 1,490). RESULTS: The lower bound of V114/PCV20 OPA GMT ratio for all PCV13 STs is greater than the prespecified  0.5 non-inferiority margin and those for five PCV13 STs (3, 6A, 6B, 18C, and 23F) are greater than the prespecified 1.2 superiority margin. V114 was associated with 77% greater OPA GMT for ST3 versus PCV20. CONCLUSION: V114 was non-inferior to PCV20 for all PCV13 STs and statistically superior for five PCV13 STs.

Impact of Letermovir Use for Cytomegalovirus Prophylaxis on Re-Hospitalization Following Allogeneic Hematopoietic Stem Cell Transplantation: An Analysis of a Phase III Randomized Clinical Trial.

BACKGROUND: Allogeneic hematopoietic stem cell transplantation (HSCT) is associated with substantial healthcare resource use, particularly when recipients develop cytomegalovirus (CMV) infection. Letermovir reduced post-HSCT CMV infection risk compared with placebo in a previous phase III trial. This analysis evaluated letermovir's impact on re-hospitalization post-transplant. METHODS: Using data from a phase III, multicenter, randomized clinical trial (NCT02137772, registered May 14, 2014), this study assessed CMV-associated and all-cause re-hospitalizations at weeks 14, 24, and 48 post-transplant among recipients of letermovir versus placebo. Unstandardized re-hospitalization rates and days were reported; standardized rates and days were estimated accounting for censoring due to death or early study discontinuation. RESULTS: Unstandardized rates (95% confidence interval [CI]) of all-cause re-hospitalization in letermovir versus placebo recipients at weeks 14, 24, and 48 were 36.6% (31.4-42.1) versus 47.6% (39.9-55.4), 49.2% (43.7-54.8) versus 55.9% (48.1-63.5), and 55.7% (50.1-61.2) versus 60.6% (52.8-68.0), respectively. Unstandardized mean total duration (95% CI) of re-hospitalization with letermovir versus placebo at weeks 14, 24, and 48 were 7.6 (5.9-9.8) versus 11.3 (8.6-14.8), 13.9 (11.2-17.2) versus 15.5 (11.9-20.1), and 18.0 (14.8-21.9) versus 20.7 (15.8-27.1) days, respectively. Similar results were found in CMV-associated re-hospitalization outcomes and standardized rates and days of all-cause re-hospitalizations. CONCLUSIONS: In this post-hoc analysis, letermovir was associated with lower rates of CMV-associated and all-cause re-hospitalizations with a shorter length of stay (especially within the first 14 weeks post-transplant).

Bayesian credible subgroup identification for treatment effectiveness in time-to-event data.

Due to differential treatment responses of patients to pharmacotherapy, drug development and practice in medicine are concerned with personalized medicine, which includes identifying subgroups of population that exhibit differential treatment effect. For time-to-event data, available methods only focus on detecting and testing treatment-by-covariate interactions and may not consider multiplicity. In this work, we introduce the Bayesian credible subgroups approach for time-to-event endpoints. It provides two bounding subgroups for the true benefiting subgroup: one which is likely to be contained by the benefiting subgroup and one which is likely to contain the benefiting subgroup. A personalized treatment effect is estimated by two common measures of survival time: the hazard ratio and restricted mean survival time. We apply the method to identify benefiting subgroups in a case study of prostate carcinoma patients and a simulated large clinical dataset.

A Bayesian approach for individual-level drug benefit-risk assessment.

In existing benefit-risk assessment (BRA) methods, benefit and risk criteria are usually identified and defined separately based on aggregated clinical data and therefore ignore the individual-level differences as well as the association among the criteria. We proposed a Bayesian multicriteria decision-making method for BRA of drugs using individual-level data. We used a multidimensional latent trait model to account for the heterogeneity of treatment effects with latent variables introducing the dependencies among outcomes. We then applied the stochastic multicriteria acceptability analysis approach for BRA incorporating imprecise and heterogeneous patient preference information. We adopted an efficient Markov chain Monte Carlo algorithm when implementing the proposed method. We applied our method to a case study to illustrate how individual-level benefit-risk profiles could inform decision-making.

Periodic benefit-risk assessment using Bayesian stochastic multi-criteria acceptability analysis.

Benefit-risk (BR) assessment is essential to ensure the best decisions are made for a medical product in the clinical development process, regulatory marketing authorization, post-market surveillance, and coverage and reimbursement decisions. One challenge of BR assessment in practice is that the benefit and risk profile may keep evolving while new evidence is accumulating. Regulators and the International Conference on Harmonization (ICH) recommend performing periodic benefit-risk evaluation report (PBRER) through the product's lifecycle. In this paper, we propose a general statistical framework for periodic benefit-risk assessment, in which Bayesian meta-analysis and stochastic multi-criteria acceptability analysis (SMAA) will be combined to synthesize the accumulating evidence. The proposed approach allows us to compare the acceptability of different drugs dynamically and effectively and accounts for the uncertainty of clinical measurements and imprecise or incomplete preference information of decision makers. We apply our approaches to two real examples in a post-hoc way for illustration purpose. The proposed method may easily be modified for other pre and post market settings, and thus be an important complement to the current structured benefit-risk assessment (sBRA) framework to improve the transparent and consistency of the decision-making process.

Challenges and methodology in the incorporation of biomarkers in cancer clinical trials.

Biomarkers can be used to establish more homogeneous groups using the genetic makeup of the tumour to inform the selection of treatment for each individual patient. However, proper preclinical work and stringent validation are needed before taking forward biomarkers into confirmatory studies. Despite the challenges, incorporation of biomarkers into clinical trials could better target appropriate patients, and potentially be lifesaving. The authors conducted a systematic review to describe marker-based and adaptive design methodology for their integration in clinical trials, and to further describe the associated practical challenges. Studies published between 1990 to November 2015 were searched on PubMed. Titles, abstracts and full text articles were reviewed to identify relevant studies. Of the 4438 studies examined, 57 studies were included. The authors conclude that the proposed approaches may readily help researchers to design biomarker trials, but novel approaches are still needed.

An International Study of the Ability and Cost-Effectiveness of Advertising Methods to Facilitate Study Participant Self-Enrolment Into a Pilot Pharmacovigilance Study During Early Pregnancy.

BACKGROUND: Knowledge of the fetal effects of maternal medication use in pregnancy is often inadequate and current pregnancy pharmacovigilance (PV) surveillance methods have important limitations. Patient self-reporting may be able to mitigate some of these limitations, providing an adequately sized study sample can be recruited. OBJECTIVE: To compare the ability and cost-effectiveness of several direct-to-participant advertising methods for the recruitment of pregnant participants into a study of self-reported gestational exposures and pregnancy outcomes. METHODS: The Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium (PROTECT) pregnancy study is a non-interventional, prospective pilot study of self-reported medication use and obstetric outcomes provided by a cohort of pregnant women that was conducted in Denmark, the Netherlands, Poland, and the United Kingdom. Direct-to-participant advertisements were provided via websites, emails, leaflets, television, and social media platforms. RESULTS: Over a 70-week recruitment period direct-to-participant advertisements engaged 43,234 individuals with the study website or telephone system; 4.78% (2065/43,234) of which were successfully enrolled and provided study data. Of these 90.4% (1867/2065) were recruited via paid advertising methods, 23.0% (475/2065) of whom were in the first trimester of pregnancy. The overall costs per active recruited participant were lowest for email (€23.24) and website (€24.41) advertisements and highest for leaflet (€83.14) and television (€100.89). Website adverts were substantially superior in their ability to recruit participants during their first trimester of pregnancy (317/668, 47.5%) in comparison with other advertising methods (P<.001). However, we identified international variations in both the cost-effectiveness of the various advertisement methods used and in their ability to recruit participants in early pregnancy. CONCLUSIONS: Recruitment of a pregnant cohort using direct-to-participant advertisement methods is feasible, but the total costs incurred are not insubstantial. Future research is needed to identify advertising strategies capable of recruiting large numbers of demographically representative pregnant women, preferentially in early pregnancy.

Recommendations for benefit-risk assessment methodologies and visual representations.

PURPOSE: The purpose of this study is to draw on the practical experience from the PROTECT BR case studies and make recommendations regarding the application of a number of methodologies and visual representations for benefit-risk assessment. METHODS: Eight case studies based on the benefit-risk balance of real medicines were used to test various methodologies that had been identified from the literature as having potential applications in benefit-risk assessment. Recommendations were drawn up based on the results of the case studies. RESULTS: A general pathway through the case studies was evident, with various classes of methodologies having roles to play at different stages. Descriptive and quantitative frameworks were widely used throughout to structure problems, with other methods such as metrics, estimation techniques and elicitation techniques providing ways to incorporate technical or numerical data from various sources. Similarly, tree diagrams and effects tables were universally adopted, with other visualisations available to suit specific methodologies or tasks as required. Every assessment was found to follow five broad stages: (i) Planning, (ii) Evidence gathering and data preparation, (iii) Analysis, (iv) Exploration and (v) Conclusion and dissemination. CONCLUSIONS: Adopting formal, structured approaches to benefit-risk assessment was feasible in real-world problems and facilitated clear, transparent decision-making. Prior to this work, no extensive practical application and appraisal of methodologies had been conducted using real-world case examples, leaving users with limited knowledge of their usefulness in the real world. The practical guidance provided here takes us one step closer to a harmonised approach to benefit-risk assessment from multiple perspectives.

A case study using the PrOACT-URL and BRAT frameworks for structured benefit risk assessment.

While benefit-risk assessment is a key component of the drug development and maintenance process, it is often described in a narrative. In contrast, structured benefit-risk assessment builds on established ideas from decision analysis and comprises a qualitative framework and quantitative methodology. We compare two such frameworks, applying multi-criteria decision-analysis (MCDA) within the PrOACT-URL framework and weighted net clinical benefit (wNCB), within the BRAT framework. These are applied to a case study of natalizumab for the treatment of relapsing remitting multiple sclerosis. We focus on the practical considerations of applying these methods and give recommendations for visual presentation of results. In the case study, we found structured benefit-risk analysis to be a useful tool for structuring, quantifying, and communicating the relative benefit and safety profiles of drugs in a transparent, rational and consistent way. The two frameworks were similar. MCDA is a generic and flexible methodology that can be used to perform a structured benefit-risk in any common context. wNCB is a special case of MCDA and is shown to be equivalent to an extension of the number needed to treat (NNT) principle. It is simpler to apply and understand than MCDA and can be applied when all outcomes are measured on a binary scale.

A Bayesian approach to probabilistic sensitivity analysis in structured benefit-risk assessment.

Quantitative decision models such as multiple criteria decision analysis (MCDA) can be used in benefit-risk assessment to formalize trade-offs between benefits and risks, providing transparency to the assessment process. There is however no well-established method for propagating uncertainty of treatment effects data through such models to provide a sense of the variability of the benefit-risk balance. Here, we present a Bayesian statistical method that directly models the outcomes observed in randomized placebo-controlled trials and uses this to infer indirect comparisons between competing active treatments. The resulting treatment effects estimates are suitable for use within the MCDA setting, and it is possible to derive the distribution of the overall benefit-risk balance through Markov Chain Monte Carlo simulation. The method is illustrated using a case study of natalizumab for relapsing-remitting multiple sclerosis.

Structured Benefit-risk assessment: a review of key publications and initiatives on frameworks and methodologies.

Introduction The conduct of structured benefit-risk assessment (BRA) of pharmaceutical products is a key area of interest for regulatory agencies and the pharmaceutical industry. However, the acceptance of a standardized approach and implementation are slow. Statisticians play major roles in these organizations, and have a great opportunity to be involved and drive the shaping of future BRA. Method We performed a literature search of recent reviews and initiatives assessing BRA methodologies, and grouped them to assist those new to BRA in learning, understanding, and choosing methodologies. We summarized the key points and discussed the impact of this emerging field on various stakeholders, particularly statisticians in the pharmaceutical industry. Results We provide introductory, essential, special interest, and further information and initiatives materials that direct readers to the most relevant materials, which were published between 2000 and 2013. Based on recommendations in these materials we supply a toolkit of advocated BRA methodologies. Discussion Despite initiatives promoting these methodologies, there are still barriers, one of which being the lack of a consensus on the most appropriate methodologies among stakeholders. However, this opens up opportunities, for statisticians in the pharmaceutical industry especially, to champion appropriate BRA methodology use throughout the pharmaceutical product lifecycle. Conclusions This article may serve as a starting point for discussions and to reach a mutual consensus for methodology selection in a particular situation. Regulators and pharmaceutical industry should continue to collaborate to develop and take forward BRA methodologies, and by clear communication develop a mutual understanding of the key issues. Copyright © 2015 John Wiley & Sons, Ltd.

Literature review of visual representation of the results of benefit-risk assessments of medicinal products.

BACKGROUND: The PROTECT Benefit-Risk group is dedicated to research in methods for continuous benefit-risk monitoring of medicines, including the presentation of the results, with a particular emphasis on graphical methods. METHODS: A comprehensive review was performed to identify visuals used for medical risk and benefit-risk communication. The identified visual displays were grouped into visual types, and each visual type was appraised based on five criteria: intended audience, intended message, knowledge required to understand the visual, unintentional messages that may be derived from the visual and missing information that may be needed to understand the visual. RESULTS: Sixty-six examples of visual formats were identified from the literature and classified into 14 visual types. We found that there is not one single visual format that is consistently superior to others for the communication of benefit-risk information. In addition, we found that most of the drawbacks found in the visual formats could be considered general to visual communication, although some appear more relevant to specific formats and should be considered when creating visuals for different audiences depending on the exact message to be communicated. CONCLUSION: We have arrived at recommendations for the use of visual displays for benefit-risk communication. The recommendation refers to the creation of visuals. We outline four criteria to determine audience-visual compatibility and consider these to be a key task in creating any visual. Next we propose specific visual formats of interest, to be explored further for their ability to address nine different types of benefit-risk analysis information.

Prokinetics prescribing in paediatrics: evidence on cisapride, domperidone, and metoclopramide.

OBJECTIVES: Domperidone and metoclopramide are prokinetics commonly prescribed off-label to infants and younger children in an attempt to treat gastro-oesophageal reflux symptoms. Another prokinetic drug, cisapride, was used but withdrawn in 2000 in the United Kingdom because of serious arrhythmic adverse events. Medicines and Healthcare Products Regulatory Agency issued safety warnings for domperidone in May 2012 and restricted its indications. We report here national primary care prescribing trends and safety signals of these drugs in children. METHODS: We used data from the General Practice Research Database between 1990 and 2006 for children <18 years. Descriptive statistics and Poisson regressions were performed to characterise prescribing trends. We examined safety signals in nested case-control studies. RESULTS: The proportion of children <2 years old being prescribed one of the medications doubled during the study period. Prescriptions of domperidone increased 10-fold, mainly following the withdrawal of cisapride in 2000. Prescriptions of metoclopramide did not change significantly. Despite the increase in prescriptions of domperidone, no new safety signals were identified. CONCLUSIONS: These data showed dramatic changes in prescribing of cisapride and domperidone despite the lack of good-quality supporting evidence. It is possible that these prescribing trends were influenced by published guidelines. Even if produced without robust efficacy and safety evidence, published guidelines can influence clinicians and consequently affect prescribing. Therefore, improving the evidence base on prokinetics to inform future guidelines is vital. The lack of new safety signals during this period would support the development of suitable powered clinical studies.

Balancing the interests of patient data protection and medication safety monitoring in a public-private partnership.

Obtaining data without the intervention of a health care provider represents an opportunity to expand understanding of the safety of medications used in difficult-to-study situations, like the first trimester of pregnancy when women may not present for medical care. While it is widely agreed that personal data, and in particular medical data, needs to be protected from unauthorized use, data protection requirements for population-based studies vary substantially by country. For public-private partnerships, the complexities are enhanced. The objective of this viewpoint paper is to illustrate the challenges related to data protection based on our experiences when performing relatively straightforward direct-to-patient noninterventional research via the Internet or telephone in four European countries. Pregnant women were invited to participate via the Internet or using an automated telephone response system in Denmark, the Netherlands, Poland, and the United Kingdom. Information was sought on medications, other factors that may cause birth defects, and pregnancy outcome. Issues relating to legal controllership of data were most problematic; assuring compliance with data protection requirements took about two years. There were also inconsistencies in the willingness to accept nonwritten informed consent. Nonetheless, enrollment and data collection have been completed, and analysis is in progress. Using direct reporting from consumers to study the safety of medicinal products allows researchers to address a myriad of research questions relating to everyday clinical practice, including treatment heterogeneity in population subgroups not traditionally included in clinical trials, like pregnant women, children, and the elderly. Nonetheless, there are a variety of administrative barriers relating to data protection and informed consent, particularly within the structure of a public-private partnership.

Effects of targeted delivery of propionate to the human colon on appetite regulation, body weight maintenance and adiposity in overweight adults.

OBJECTIVE: The colonic microbiota ferment dietary fibres, producing short chain fatty acids. Recent evidence suggests that the short chain fatty acid propionate may play an important role in appetite regulation. We hypothesised that colonic delivery of propionate would increase peptide YY (PYY) and glucagon like peptide-1 (GLP-1) secretion in humans, and reduce energy intake and weight gain in overweight adults. DESIGN: To investigate whether propionate promotes PYY and GLP-1 secretion, a primary cultured human colonic cell model was developed. To deliver propionate specifically to the colon, we developed a novel inulin-propionate ester. An acute randomised, controlled cross-over study was used to assess the effects of this inulin-propionate ester on energy intake and plasma PYY and GLP-1 concentrations. The long-term effects of inulin-propionate ester on weight gain were subsequently assessed in a randomised, controlled 24-week study involving 60 overweight adults. RESULTS: Propionate significantly stimulated the release of PYY and GLP-1 from human colonic cells. Acute ingestion of 10 g inulin-propionate ester significantly increased postprandial plasma PYY and GLP-1 and reduced energy intake. Over 24 weeks, 10 g/day inulin-propionate ester supplementation significantly reduced weight gain, intra-abdominal adipose tissue distribution, intrahepatocellular lipid content and prevented the deterioration in insulin sensitivity observed in the inulin-control group. CONCLUSIONS: These data demonstrate for the first time that increasing colonic propionate prevents weight gain in overweight adult humans. TRIAL REGISTRATION NUMBER: NCT00750438.