Assurance methods for designing a clinical trial with a delayed treatment effect.

An assurance calculation is a Bayesian alternative to a power calculation. One may be performed to aid the planning of a clinical trial, specifically setting the sample size or to support decisions about whether or not to perform a study. Immuno-oncology is a rapidly evolving area in the development of anticancer drugs. A common phenomenon that arises in trials of such drugs is one of delayed treatment effects, that is, there is a delay in the separation of the survival curves. To calculate assurance for a trial in which a delayed treatment effect is likely to be present, uncertainty about key parameters needs to be considered. If uncertainty is not considered, the number of patients recruited may not be enough to ensure we have adequate statistical power to detect a clinically relevant treatment effect and the risk of an unsuccessful trial is increased. We present a new elicitation technique for when a delayed treatment effect is likely and show how to compute assurance using these elicited prior distributions. We provide an example to illustrate how this can be used in practice and develop open-source software to implement our methods. Our methodology has the potential to improve the success rate and efficiency of Phase III trials in immuno-oncology and for other treatments where a delayed treatment effect is expected to occur.

Effectiveness of tisagenlecleucel versus real-world standard of care in relapsed/refractory follicular lymphoma.

Aim: To contextualize the effectiveness of tisagenlecleucel versus real-world standard of care (SoC) in relapsed/refractory follicular lymphoma. Materials & methods: A retrospective indirect matched comparison study using data from the phase II ELARA trial and the US Flatiron Health Research Database. Results: Complete response rate was 69.1 versus 17.7% and the overall response rate was 85.6 versus 58.1% in tisagenlecleucel versus SoC, post weighting by odds. For overall survival, an estimated reduction in the risk of death was observed in favor of tisagenlecleucel over SoC. The hazard ratio for progression-free survival was 0.45 (95% CI: 0.26, 0.88), and for time-to-next treatment was 0.34 (95% CI: 0.15, 0.78) with tisagenlecleucel versus SoC. Conclusion: A consistent trend toward improved efficacy end points was observed in favor of tisagenlecleucel versus SoC.

Eliciting judgements about dependent quantities of interest: The SHeffield ELicitation Framework extension and copula methods illustrated using an asthma case study.

Pharmaceutical companies regularly need to make decisions about drug development programs based on the limited knowledge from early stage clinical trials. In this situation, eliciting the judgements of experts is an attractive approach for synthesising evidence on the unknown quantities of interest. When calculating the probability of success for a drug development program, multiple quantities of interest-such as the effect of a drug on different endpoints-should not be treated as unrelated. We discuss two approaches for establishing a multivariate distribution for several related quantities within the SHeffield ELicitation Framework (SHELF). The first approach elicits experts' judgements about a quantity of interest conditional on knowledge about another one. For the second approach, we first elicit marginal distributions for each quantity of interest. Then, for each pair of quantities, we elicit the concordance probability that both lie on the same side of their respective elicited medians. This allows us to specify a copula to obtain the joint distribution of the quantities of interest. We show how these approaches were used in an elicitation workshop that was performed to assess the probability of success of the registrational program of an asthma drug. The judgements of the experts, which were obtained prior to completion of the pivotal studies, were well aligned with the final trial results.

A New Comprehensive Approach to Assess the Probability of Success of Development Programs Before Pivotal Trials.

The point at which clinical development programs transition from early phase to pivotal trials is a critical milestone. Substantial uncertainty about the outcome of pivotal trials may remain even after seeing positive early phase data, and companies may need to make difficult prioritization decisions for their portfolio. The probability of success (PoS) of a program, a single number expressed as a percentage reflecting the multitude of risks that may influence the final program outcome, is a key decision-making tool. Despite its importance, companies often rely on crude industry benchmarks that may be "adjusted" by experts based on undocumented criteria and which are typically misaligned with the definition of success used to drive commercial forecasts, leading to overly optimistic expected net present value calculations. We developed a new framework to assess the PoS of a program before pivotal trials begin. Our definition of success encompasses the successful outcome of pivotal trials, regulatory approval and meeting the requirements for market access as outlined in the target product profile. The proposed approach is organized in four steps and uses an innovative Bayesian approach to synthesize all relevant evidence. The new PoS framework is systematic and transparent. It will help organizations to make more informed decisions. In this paper, we outline the rationale and elaborate on the structure of the proposed framework, provide examples, and discuss the benefits and challenges associated with its adoption.

Improving the assessment of the probability of success in late stage drug development.

There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may start with industry-wide success rates and then apply to these subjective adjustments to reflect program-specific information. However, this approach lacks transparency and fails to make full use of data from previous clinical trials. We describe a quantitative Bayesian approach for calculating the probability of success (PoS) at the end of phase II which incorporates internal clinical data from one or more phase IIb studies, industry-wide success rates, and expert opinion or external data if needed. Using an example, we illustrate how PoS can be calculated accounting for differences between the phase II data and future phase III trials, and discuss how the methods can be extended to accommodate accelerated drug development pathways.

Mycophenolate Mofetil Versus Cyclophosphamide for Remission Induction in Childhood Polyarteritis Nodosa: An Open-Label, Randomized, Bayesian Noninferiority Trial.

OBJECTIVE: Cyclophosphamide (CYC) is used in clinical practice off-label for the induction of remission in childhood polyarteritis nodosa (PAN). Mycophenolate mofetil (MMF) might offer a less toxic alternative. This study was undertaken to explore the relative effectiveness of CYC and MMF treatment in a randomized controlled trial (RCT). METHODS: This was an international, open-label, Bayesian RCT to investigate the relative effectiveness of CYC and MMF for remission induction in childhood PAN. Eleven patients with newly diagnosed childhood PAN were randomized (1:1) to receive MMF or intravenous CYC; all patients received the same glucocorticoid regimen. The primary end point was remission within 6 months while compliant with glucocorticoid taper. Bayesian distributions for remission rates were established a priori for MMF and CYC by experienced clinicians and updated to posterior distributions on trial completion. RESULTS: Baseline disease activity and features were similar between the 2 treatment groups. The primary end point was met in 4 of 6 patients (67%) in the MMF group and 4 of 5 patients (80%) in the CYC group. Time to remission was shorter in the MMF group compared to the CYC group (median 7.1 weeks versus 17.6 weeks). No relapses occurred in either group within 18 months. Two serious infections were found to be likely linked to MMF treatment. Physical and psychosocial quality-of-life scores were superior in the MMF group compared to the CYC group at 6 months and 18 months. Combining the prior expert opinion with results from the present study provided posterior estimates of remission of 71% for MMF (90% credibility interval [90% CrI] 51, 83) and 75% for CYC (90% CrI 57, 86). CONCLUSION: The present results, taken together with prior opinion, indicate that rates of remission induction in childhood PAN are similar with MMF treatment and CYC treatment, and MMF treatment might be associated with better health-related quality of life than CYC treatment.

Study to evaluate the optimal dose of remifentanil required to ensure apnea during magnetic resonance imaging of the heart under general anesthesia.

BACKGROUND: Magnetic resonance (MRI) scanning of the heart is an established part of the investigation of cardiovascular conditions in children. In young children, sedation is likely to be needed, and multiple controlled periods of apnea are often required to allow image acquisition. Suppression of spontaneous ventilation is possible with remifentanil; however, the dose required is uncertain. AIMS: To establish the dose of remifentanil, by infusion, required to suppress ventilation sufficiently to allow a 30-s apnea during MRI imaging of the heart. METHOD: Patients aged 1-6 years were exposed to different doses of remifentanil, and the success in achieving a 30-s apnea was recorded. A dose recommendation was made for each patient, informed by responses of previous patients using an adaptive Bayesian dose-escalation design. Other aspects of anesthesia were standardized. A final estimate of the dose needed to achieve a successful outcome in 80% of patients (ED80) was made using logistic regression. RESULTS: 38 patients were recruited, and apnea achieved in 31 patients. The estimate of the ED80 was 0.184 µg/kg/min (95% CI 0.178-0.190). Post hoc analysis revealed that higher doses were required in younger patients. CONCLUSION: The ED80 for this indication was 0.184 µg/kg/min (95% CI 0.178-0.190). This is different from optimal dosing identified for other indications and dosing of remifentanil should be specific to the clinical context in which it is used.

Bridging across patient subgroups in phase I oncology trials that incorporate animal data.

In this paper, we develop a general Bayesian hierarchical model for bridging across patient subgroups in phase I oncology trials, for which preliminary information about the dose-toxicity relationship can be drawn from animal studies. Parameters that re-scale the doses to adjust for intrinsic differences in toxicity, either between animals and humans or between human subgroups, are introduced to each dose-toxicity model. Appropriate priors are specified for these scaling parameters, which capture the magnitude of uncertainty surrounding the animal-to-human translation and bridging assumption. After mapping data onto a common, 'average' human dosing scale, human dose-toxicity parameters are assumed to be exchangeable either with the standardised, animal study-specific parameters, or between themselves across human subgroups. Random-effects distributions are distinguished by different covariance matrices that reflect the between-study heterogeneity in animals and humans. Possibility of non-exchangeability is allowed to avoid inferences for extreme subgroups being overly influenced by their complementary data. We illustrate the proposed approach with hypothetical examples, and use simulation to compare the operating characteristics of trials analysed using our Bayesian model with several alternatives. Numerical results show that the proposed approach yields robust inferences, even when data from multiple sources are inconsistent and/or the bridging assumptions are incorrect.

A Bayesian decision-theoretic approach to incorporate preclinical information into phase I oncology trials.

Leveraging preclinical animal data for a phase I oncology trial is appealing yet challenging. In this paper, we use animal data to improve decision-making in a model-based dose-escalation procedure. We make a proposal for how to measure and address a prior-data conflict in a sequential study with a small sample size. Animal data are incorporated via a robust two-component mixture prior for the parameters of the human dose-toxicity relationship. The weights placed on each component of the prior are chosen empirically and updated dynamically as the trial progresses and more data accrue. After completion of each cohort, we use a Bayesian decision-theoretic approach to evaluate the predictive utility of the animal data for the observed human toxicity outcomes, reflecting the degree of agreement between dose-toxicity relationships in animals and humans. The proposed methodology is illustrated through several data examples and an extensive simulation study.

Exposure-response modelling approaches for determining optimal dosing rules in children.

Within paediatric populations, there may be distinct age groups characterised by different exposure-response relationships. Several regulatory guidance documents have suggested general age groupings. However, it is not clear whether these categorisations will be suitable for all new medicines and in all disease areas. We consider two model-based approaches to quantify how exposure-response model parameters vary over a continuum of ages: Bayesian penalised B-splines and model-based recursive partitioning. We propose an approach for deriving an optimal dosing rule given an estimate of how exposure-response model parameters vary with age. Methods are initially developed for a linear exposure-response model. We perform a simulation study to systematically evaluate how well the various approaches estimate linear exposure-response model parameters and the accuracy of recommended dosing rules. Simulation scenarios are motivated by an application to epilepsy drug development. Results suggest that both bootstrapped model-based recursive partitioning and Bayesian penalised B-splines can estimate underlying changes in linear exposure-response model parameters as well as (and in many scenarios, better than) a comparator linear model adjusting for a categorical age covariate with levels following International Conference on Harmonisation E11 groupings. Furthermore, the Bayesian penalised B-splines approach consistently estimates the intercept and slope more accurately than the bootstrapped model-based recursive partitioning. Finally, approaches are extended to estimate Emax exposure-response models and are illustrated with an example motivated by an in vitro study of cyclosporine.

A robust Bayesian meta-analytic approach to incorporate animal data into phase I oncology trials.

Before a first-in-man trial is conducted, preclinical studies are performed in animals to help characterise the safety profile of the new medicine. We propose a robust Bayesian hierarchical model to synthesise animal and human toxicity data, using scaling factors to translate doses administered to different animal species onto an equivalent human scale. After scaling doses, the parameters of dose-toxicity models intrinsic to different animal species can be interpreted on a common scale. A prior distribution is specified for each translation factor to capture uncertainty about differences between toxicity of the drug in animals and humans. Information from animals can then be leveraged to learn about the relationship between dose and risk of toxicity in a new phase I trial in humans. The model allows human dose-toxicity parameters to be exchangeable with the study-specific parameters of animal species studied so far or non-exchangeable with any of them. This leads to robust inferences, enabling the model to give greatest weight to the animal data with parameters most consistent with human parameters or discount all animal data in the case of non-exchangeability. The proposed model is illustrated using a case study and simulations. Numerical results suggest that our proposal improves the precision of estimates of the toxicity rates when animal and human data are consistent, while it discounts animal data in cases of inconsistency.

Designing and evaluating dose-escalation studies made easy: The MoDEsT web app.

BACKGROUND/AIMS: Dose-escalation studies are essential in the early stages of developing novel treatments, when the aim is to find a safe dose for administration in humans. Despite their great importance, many dose-escalation studies use study designs based on heuristic algorithms with well-documented drawbacks. Bayesian decision procedures provide a design alternative that is conceptually simple and methodologically sound, but very rarely used in practice, at least in part due to their perceived statistical complexity. There are currently very few easily accessible software implementations that would facilitate their application. METHODS: We have created MoDEsT, a free and easy-to-use web application for designing and conducting single-agent dose-escalation studies with a binary toxicity endpoint, where the objective is to estimate the maximum tolerated dose. MoDEsT uses a well-established Bayesian decision procedure based on logistic regression. The software has a user-friendly point-and-click interface, makes changes visible in real time, and automatically generates a range of graphs, tables, and reports. It is aimed at clinicians as well as statisticians with limited expertise in model-based dose-escalation designs, and does not require any statistical programming skills to evaluate the operating characteristics of, or implement, the Bayesian dose-escalation design. RESULTS: MoDEsT comes in two parts: a 'Design' module to explore design options and simulate their operating characteristics, and a 'Conduct' module to guide the dose-finding process throughout the study. We illustrate the practical use of both modules with data from a real phase I study in terminal cancer. CONCLUSION: Enabling both methodologists and clinicians to understand and apply model-based study designs with ease is a key factor towards their routine use in early-phase studies. We hope that MoDEsT will enable incorporation of Bayesian decision procedures for dose escalation at the earliest stage of clinical trial design, thus increasing their use in early-phase trials.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop.

BACKGROUND: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. OBJECTIVE: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. METHODS: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. RESULTS: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. CONCLUSIONS: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items. FUNDING: Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

This Difference ELicitation in TriAls² (DELTA²) advice and recommendations document aims to help researchers choose the 'target difference' in a type of research study called a randomised controlled trial. The number of people needed to be involved in a study ­ the sample size ­ is usually based on a calculation aimed to ensure that the difference in benefit between treatments is likely to be detected. The calculation also accounts for the risk of a false-positive finding. No more patients than necessary should be involved. Choosing a 'target difference' is an important step in calculating the sample size. The target difference is defined as the amount of difference in the participants' response to the treatments that we wish to detect. It is probably the most important piece of information used in the sample size calculation. How we decide what the target difference should be depends on various factors. One key decision to make is how we should measure the benefits that treatments offer. For example, if we are evaluating a treatment for high blood pressure, the obvious thing to focus on would be blood pressure. We could then proceed to consider what an important difference in blood pressure between treatments would be, based on experts' views or evidence from previous research studies. This document seeks to provide assistance to researchers on how to choose the target difference when designing a trial. It also provides advice to help them clearly present what was done and why, when writing up the study proposal or reporting the study's findings. The document is also intended to be read by those who decide whether or not a proposed study should be funded. Clarifying a study's aim and getting a sensible sample size is important. It can affect not only those involved in the study, but also future patients who will receive treatment.

Correction to: Choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial - the development of the DELTA2 guidance.

Following publication of the original article [1], we have been notified of a few mistakes.

Sarcome-13/OS2016 trial protocol: a multicentre, randomised, open-label, phase II trial of mifamurtide combined with postoperative chemotherapy for patients with newly diagnosed high-risk osteosarcoma.

INTRODUCTION: The controversial results on the mifamurtide efficacy associated with chemotherapy, issued from the American INT-0133-study, in localised osteosarcomas, and the underpowered analysis performed separately in metastatic patients, should be clarified to homogenise international use of this promising drug. The European Commission has granted a marketing authorisation to mifamurtide combined with postoperative chemotherapy in localised osteosarcomas but not in metastatic patients, while the Food and Drug Administration (FDA) has denied this authorisation. METHODS AND ANALYSIS: Sarcome-13/OS2016 trial is a multicentre randomised open-label phase II trial evaluating the survival benefit of mifamurtide administered during 36 weeks in combination with postoperative chemotherapy versus chemotherapy alone, in patients >2 and ≤50 years with newly diagnosed high-risk localised or metastatic osteosarcoma. The main objective is to evaluate the impact on event-free survival (EFS) of mifamurtide on intention-to-treat population. The secondary objectives are to evaluate the impact of mifamurtide on overall survival, to evaluate the feasibility and toxicity of the planned treatment, to correlate biology/immunology with the mifamurtide efficacy/toxicity. With a total of 126 enrolled patients and 51 events, the power is 80% if mifamurtide is associated with an 18% improvement of the 3-year EFS (52%vs70%, equivalent to an HR=0.55), with a one-sided logrank test alpha=10%. As relevant historical data are available (aggregate treatment effect from the INT-0133 trial and individual data from the control group of the Sarcome-09/OS2006 trial), a Bayesian analysis is also planned. ETHICS AND DISSEMINATION: This study was approved by the 'Comité de Protection des Personnes Ile de France I' (12/06/2018), complies with the Declaration of Helsinki and French laws and regulations, and follows the International Conference on Harmonisation E6 Guideline for Good Clinical Practice. The trial results, even if they are inconclusive, as well as biological ancillary studies will be presented at appropriate international congresses and published in international peer-review journals. TRIAL REGISTRATION NUMBER: EudraCT 2017-001165-24, NCT03643133.

Incorporating individual historical controls and aggregate treatment effect estimates into a Bayesian survival trial: a simulation study.

BACKGROUND: Performing well-powered randomised controlled trials (RCTs) of new treatments for rare diseases is often infeasible. However, with the increasing availability of historical data, incorporating existing information into trials with small sample sizes is appealing in order to increase the power. Bayesian approaches enable one to incorporate historical data into a trial's analysis through a prior distribution. METHODS: Motivated by a RCT intended to evaluate the impact on event-free survival of mifamurtide in patients with osteosarcoma, we performed a simulation study to evaluate the impact on trial operating characteristics of incorporating historical individual control data and aggregate treatment effect estimates. We used power priors derived from historical individual control data for baseline parameters of Weibull and piecewise exponential models, while we used a mixture prior to summarise aggregate information obtained on the relative treatment effect. The impact of prior-data conflicts, both with respect to the parameters and survival models, was evaluated for a set of pre-specified weights assigned to the historical information in the prior distributions. RESULTS: The operating characteristics varied according to the weights assigned to each source of historical information, the variance of the informative and vague component of the mixture prior and the level of commensurability between the historical and new data. When historical and new controls follow different survival distributions, we did not observe any advantage of choosing a piecewise exponential model compared to a Weibull model for the new trial analysis. However, we think that it remains appealing given the uncertainty that will often surround the shape of the survival distribution of the new data. CONCLUSION: In the setting of Sarcome-13 trial, and other similar studies in rare diseases, the gains in power and accuracy made possible by incorporating different types of historical information commensurate with the new trial data have to be balanced against the risk of biased estimates and a possible loss in power if data are not commensurate. The weights allocated to the historical data have to be carefully chosen based on this trade-off. Further simulation studies investigating methods for incorporating historical data are required to generalise the findings.

Estimands and the Patient Journey: Addressing the Right Question in Oncology Clinical Trials.

The diversity of patient journeys can raise fundamental questions regarding the evaluation of drug effects in clinical trials to inform clinical practice. When defining the treatment effect of interest in a trial, the researcher needs to account for events occurring after treatment initiation, such as the start of a new therapy, before observing the end point. We review the newly introduced estimand framework to structure discussions on the relationship between patient journeys and the treatment effect of interest in oncology trials. In 2017, the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use released a draft addendum to its E9 guideline. The addendum introduces the concept of an estimand to precisely describe the treatment effect of interest. This estimand framework provides a structured approach to discuss how to account for intercurrent events that occur after random assignment and may affect the assessment or interpretation of the treatment effect. The framework is expected to improve coherence between trial objectives, design, analysis, and interpretation, as illustrated by examples in oncology disease settings. The estimand framework was applied to design a trial for a chimeric antigen receptor T-cell therapy. The treatment effect of interest was carefully defined considering the range of patient journeys expected for this particular indication and treatment. The trial design was developed accordingly to assess that treatment effect. All parties involved in the design of clinical trials need to consider possible patient journeys to define appropriate treatment effects and corresponding trial designs and analysis strategies. The estimand framework provides a common language to address the complexity introduced by varied patient journeys.

DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial.

BACKGROUND: A key step in the design of a RCT is the estimation of the number of participants needed in the study. The most common approach is to specify a target difference between the treatments for the primary outcome and then calculate the required sample size. The sample size is chosen to ensure that the trial will have a high probability (adequate statistical power) of detecting a target difference between the treatments should one exist. The sample size has many implications for the conduct and interpretation of the study. Despite the critical role that the target difference has in the design of a RCT, the way in which it is determined has received little attention. In this article, we summarise the key considerations and messages from new guidance for researchers and funders on specifying the target difference, and undertaking and reporting a RCT sample size calculation. This article on choosing the target difference for a randomised controlled trial (RCT) and undertaking and reporting the sample size calculation has been dual published in the BMJ and BMC Trials journals METHODS: The DELTA2 (Difference ELicitation in TriAls) project comprised five major components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a two-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of a guidance document (stage 5). RESULTS AND DISCUSSION: The key messages from the DELTA2 guidance on determining the target difference and sample size calculation for a randomised caontrolled trial are presented. Recommendations for the subsequent reporting of the sample size calculation are also provided.