Interim decision making in seamless trial designs: An application in an adaptive dose-finding study in a rare kidney disease.

Adaptive seamless trial designs, combining the learning and confirming cycles of drug development in a single trial, have gained popularity in recent years. Adaptations may include dose selection, sample size re-estimation and enrichment of the study population. Despite methodological advances and recognition of the potential efficiency gains such designs offer, their implementation, including how to enable efficient decision making on the adaptations in interim analyzes, remains a key challenge in their adoption. This manuscript uses a case study of an adaptive seamless proof-of-concept (Phase 2a)/dose-finding (Phase 2b) to showcase potential adaptive features that can be implemented in trial designs at earlier development stages and the role of simulations in assessing the design operating characteristics and specifying the decision rules for the adaptations. It further outlines the elements needed to support successful interim analysis decision making on the adaptations while safeguarding study integrity, including the role of different stakeholders, interactive simulation-based tools to facilitate decision making and operational aspects requiring preplanning. The benefits of the adaptive Phase 2a/2b design chosen compared to following the traditional two separate studies (2a and 2b) paradigm are discussed. With careful planning and appreciation of their complexity and components needed for their implementation, seamless adaptive designs have the potential to yield significant savings both in terms of time and resources.

Predicting subgroup treatment effects for a new study: Motivations, results and learnings from running a data challenge in a pharmaceutical corporation.

We present the motivation, experience, and learnings from a data challenge conducted at a large pharmaceutical corporation on the topic of subgroup identification. The data challenge aimed at exploring approaches to subgroup identification for future clinical trials. To mimic a realistic setting, participants had access to 4 Phase III clinical trials to derive a subgroup and predict its treatment effect on a future study not accessible to challenge participants. A total of 30 teams registered for the challenge with around 100 participants, primarily from Biostatistics organization. We outline the motivation for running the challenge, the challenge rules, and logistics. Finally, we present the results of the challenge, the participant feedback as well as the learnings. We also present our view on the implications of the results on exploratory analyses related to treatment effect heterogeneity.

Bayesian inference for a principal stratum estimand on recurrent events truncated by death.

Recurrent events are often important endpoints in randomized clinical trials. For example, the number of recurrent disease-related hospitalizations may be considered as a clinically meaningful endpoint in cardiovascular studies. In some settings, the recurrent event process may be terminated by an event such as death, which makes it more challenging to define and estimate a causal treatment effect on recurrent event endpoints. In this paper, we focus on the principal stratum estimand, where the treatment effect of interest on recurrent events is defined among subjects who would be alive regardless of the assigned treatment. For the estimation of the principal stratum effect in randomized clinical trials, we propose a Bayesian approach based on a joint model of the recurrent event and death processes with a frailty term accounting for within-subject correlation. We also present Bayesian posterior predictive check procedures for assessing the model fit. The proposed approaches are demonstrated in the randomized Phase III chronic heart failure trial PARAGON-HF (NCT01920711).

Sensitivity analyses for the principal ignorability assumption using multiple imputation.

In the context of clinical trials, there is interest in the treatment effect for subpopulations of patients defined by intercurrent events, namely disease-related events occurring after treatment initiation that affect either the interpretation or the existence of endpoints. With the principal stratum strategy, the ICH E9(R1) guideline introduces a formal framework in drug development for defining treatment effects in such subpopulations. Statistical estimation of the treatment effect can be performed based on the principal ignorability assumption using multiple imputation approaches. Principal ignorability is a conditional independence assumption that cannot be directly verified; therefore, it is crucial to evaluate the robustness of results to deviations from this assumption. As a sensitivity analysis, we propose a joint model that multiply imputes the principal stratum membership and the outcome variable while allowing different levels of violation of the principal ignorability assumption. We illustrate with a simulation study that the joint imputation model-based approaches are superior to naive subpopulation analyses. Motivated by an oncology clinical trial, we implement the sensitivity analysis on a time-to-event outcome to assess the treatment effect in the subpopulation of patients who discontinued due to adverse events using a synthetic dataset. Finally, we explore the potential usage and provide interpretation of such analyses in clinical settings, as well as possible extension of such models in more general cases.

Identifying treatment effect heterogeneity in dose-finding trials using Bayesian hierarchical models: Identifying treatment effect heterogeneity in dose-finding trials.

An important task in drug development is to identify patients, which respond better or worse to an experimental treatment. Identifying predictive covariates, which influence the treatment effect and can be used to define subgroups of patients, is a key aspect of this task. Analyses of treatment effect heterogeneity are however known to be challenging, since the number of possible covariates or subgroups is often large, while samples sizes in earlier phases of drug development are often small. In addition, distinguishing predictive covariates from prognostic covariates, which influence the response independent of the given treatment, can often be difficult. While many approaches for these types of problems have been proposed, most of them focus on the two-arm clinical trial setting, where patients are given either the treatment or a control. In this article we consider parallel groups dose-finding trials, in which patients are administered different doses of the same treatment. To investigate treatment effect heterogeneity in this setting we propose a Bayesian hierarchical dose-response model with covariate effects on dose-response parameters. We make use of shrinkage priors to prevent overfitting, which can easily occur, when the number of considered covariates is large and sample sizes are small. We compare several such priors in simulations and also investigate dependent modeling of prognostic and predictive effects to better distinguish these two types of effects. We illustrate the use of our proposed approach using a Phase II dose-finding trial and show how it can be used to identify predictive covariates and subgroups of patients with increased treatment effects.

Improving the assessment of the probability of success in late stage drug development.

There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may start with industry-wide success rates and then apply to these subjective adjustments to reflect program-specific information. However, this approach lacks transparency and fails to make full use of data from previous clinical trials. We describe a quantitative Bayesian approach for calculating the probability of success (PoS) at the end of phase II which incorporates internal clinical data from one or more phase IIb studies, industry-wide success rates, and expert opinion or external data if needed. Using an example, we illustrate how PoS can be calculated accounting for differences between the phase II data and future phase III trials, and discuss how the methods can be extended to accommodate accelerated drug development pathways.

Eliciting judgements about dependent quantities of interest: The SHeffield ELicitation Framework extension and copula methods illustrated using an asthma case study.

Pharmaceutical companies regularly need to make decisions about drug development programs based on the limited knowledge from early stage clinical trials. In this situation, eliciting the judgements of experts is an attractive approach for synthesising evidence on the unknown quantities of interest. When calculating the probability of success for a drug development program, multiple quantities of interest-such as the effect of a drug on different endpoints-should not be treated as unrelated. We discuss two approaches for establishing a multivariate distribution for several related quantities within the SHeffield ELicitation Framework (SHELF). The first approach elicits experts' judgements about a quantity of interest conditional on knowledge about another one. For the second approach, we first elicit marginal distributions for each quantity of interest. Then, for each pair of quantities, we elicit the concordance probability that both lie on the same side of their respective elicited medians. This allows us to specify a copula to obtain the joint distribution of the quantities of interest. We show how these approaches were used in an elicitation workshop that was performed to assess the probability of success of the registrational program of an asthma drug. The judgements of the experts, which were obtained prior to completion of the pivotal studies, were well aligned with the final trial results.

Comparing algorithms for characterizing treatment effect heterogeneity in randomized trials.

The identification and estimation of heterogeneous treatment effects in biomedical clinical trials are challenging, because trials are typically planned to assess the treatment effect in the overall trial population. Nevertheless, the identification of how the treatment effect may vary across subgroups is of major importance for drug development. In this work, we review some existing simulation work and perform a simulation study to evaluate recent methods for identifying and estimating the heterogeneous treatments effects using various metrics and scenarios relevant for drug development. Our focus is not only on a comparison of the methods in general, but on how well these methods perform in simulation scenarios that reflect real clinical trials. We provide the R package benchtm that can be used to simulate synthetic biomarker distributions based on real clinical trial data and to create interpretable scenarios to benchmark methods for identification and estimation of treatment effect heterogeneity.

Connecting Instrumental Variable methods for causal inference to the Estimand Framework.

Causal inference methods are gaining increasing prominence in pharmaceutical drug development in light of the recently published addendum on estimands and sensitivity analysis in clinical trials to the E9 guideline of the International Council for Harmonisation. The E9 addendum emphasises the need to account for post-randomization or 'intercurrent' events that can potentially influence the interpretation of a treatment effect estimate at a trial's conclusion. Instrumental Variables (IV) methods have been used extensively in economics, epidemiology, and academic clinical studies for 'causal inference,' but less so in the pharmaceutical industry setting until now. In this tutorial article we review the basic tools for causal inference, including graphical diagrams and potential outcomes, as well as several conceptual frameworks that an IV analysis can sit within. We discuss in detail how to map these approaches to the Treatment Policy, Principal Stratum and Hypothetical 'estimand strategies' introduced in the E9 addendum, and provide details of their implementation using standard regression models. Specific attention is given to discussing the assumptions each estimation strategy relies on in order to be consistent, the extent to which they can be empirically tested and sensitivity analyses in which specific assumptions can be relaxed. We finish by applying the methods described to simulated data closely matching two recent pharmaceutical trials to further motivate and clarify the ideas.

Principal stratum strategy: Potential role in drug development.

A randomized trial allows estimation of the causal effect of an intervention compared to a control in the overall population and in subpopulations defined by baseline characteristics. Often, however, clinical questions also arise regarding the treatment effect in subpopulations of patients, which would experience clinical or disease related events post-randomization. Events that occur after treatment initiation and potentially affect the interpretation or the existence of the measurements are called intercurrent events in the ICH E9(R1) guideline. If the intercurrent event is a consequence of treatment, randomization alone is no longer sufficient to meaningfully estimate the treatment effect. Analyses comparing the subgroups of patients without the intercurrent events for intervention and control will not estimate a causal effect. This is well known, but post-hoc analyses of this kind are commonly performed in drug development. An alternative approach is the principal stratum strategy, which classifies subjects according to their potential occurrence of an intercurrent event on both study arms. We illustrate with examples that questions formulated through principal strata occur naturally in drug development and argue that approaching these questions with the ICH E9(R1) estimand framework has the potential to lead to more transparent assumptions as well as more adequate analyses and conclusions. In addition, we provide an overview of assumptions required for estimation of effects in principal strata. Most of these assumptions are unverifiable and should hence be based on solid scientific understanding. Sensitivity analyses are needed to assess robustness of conclusions.

A multiple comparison procedure for dose-finding trials with subpopulations.

Identifying subgroups of patients with an enhanced response to a new treatment has become an area of increased interest in the last few years. When there is knowledge about possible subpopulations with an enhanced treatment effect before the start of a trial it might be beneficial to set up a testing strategy, which tests for a significant treatment effect not only in the full population, but also in these prespecified subpopulations. In this paper, we present a parametric multiple testing approach for tests in multiple populations for dose-finding trials. Our approach is based on the MCP-Mod methodology, which uses multiple comparison procedures (MCPs) to test for a dose-response signal, while considering multiple possible candidate dose-response shapes. Our proposed methods allow for heteroscedastic error variances between populations and control the family-wise error rate over tests in multiple populations and for multiple candidate models. We show in simulations that the proposed multipopulation testing approaches can increase the power to detect a significant dose-response signal over the standard single-population MCP-Mod, when the specified subpopulation has an enhanced treatment effect.

Subgroup identification in dose-finding trials via model-based recursive partitioning.

An important task in early-phase drug development is to identify patients, which respond better or worse to an experimental treatment. While a variety of different subgroup identification methods have been developed for the situation of randomized clinical trials that study an experimental treatment and control, much less work has been done in the situation when patients are randomized to different dose groups. In this article, we propose new strategies to perform subgroup analyses in dose-finding trials and discuss the challenges, which arise in this new setting. We consider model-based recursive partitioning, which has recently been applied to subgroup identification in 2-arm trials, as a promising method to tackle these challenges and assess its viability using a real trial example and simulations. Our results show that model-based recursive partitioning can be used to identify subgroups of patients with different dose-response curves and improves estimation of treatment effects and minimum effective doses compared to models ignoring possible subgroups, when heterogeneity among patients is present.

Likelihood ratio tests for a dose-response effect using multiple nonlinear regression models.

We consider the problem of testing for a dose-related effect based on a candidate set of (typically nonlinear) dose-response models using likelihood-ratio tests. For the considered models this reduces to assessing whether the slope parameter in these nonlinear regression models is zero or not. A technical problem is that the null distribution (when the slope is zero) depends on non-identifiable parameters, so that standard asymptotic results on the distribution of the likelihood-ratio test no longer apply. Asymptotic solutions for this problem have been extensively discussed in the literature. The resulting approximations however are not of simple form and require simulation to calculate the asymptotic distribution. In addition, their appropriateness might be doubtful for the case of a small sample size. Direct simulation to approximate the null distribution is numerically unstable due to the non identifiability of some parameters. In this article, we derive a numerical algorithm to approximate the exact distribution of the likelihood-ratio test under multiple models for normally distributed data. The algorithm uses methods from differential geometry and can be used to evaluate the distribution under the null hypothesis, but also allows for power and sample size calculations. We compare the proposed testing approach to the MCP-Mod methodology and alternative methods for testing for a dose-related trend in a dose-finding example data set and simulations.

Phase II dose-response trials: A simulation study to compare analysis method performance under design considerations.

Phase II trials are intended to provide information about the dose-response relationship and to support the choice of doses for a pivotal phase III trial. Recently, new analysis methods have been proposed to address these objectives, and guidance is needed to select the most appropriate analysis method in specific situations. We set up a simulation study to evaluate multiple performance measures of one traditional and three more recent dose-finding approaches under four design options and illustrate the investigated analysis methods with an example from clinical practice. Our results reveal no general recommendation for a particular analysis method across all design options and performance measures. However, we also demonstrate that the new analysis methods are worth the effort compared to the traditional ANOVA-based approach.

Model averaging for treatment effect estimation in subgroups.

In many clinical trials, biological, pharmacological, or clinical information is used to define candidate subgroups of patients that might have a differential treatment effect. Once the trial results are available, interest will focus on subgroups with an increased treatment effect. Estimating a treatment effect for these groups, together with an adequate uncertainty statement is challenging, owing to the resulting "random high" / selection bias. In this paper, we will investigate Bayesian model averaging to address this problem. The general motivation for the use of model averaging is to realize that subgroup selection can be viewed as model selection, so that methods to deal with model selection uncertainty, such as model averaging, can be used also in this setting. Simulations are used to evaluate the performance of the proposed approach. We illustrate it on an example early-phase clinical trial.

Model selection versus model averaging in dose finding studies.

A key objective of Phase II dose finding studies in clinical drug development is to adequately characterize the dose response relationship of a new drug. An important decision is then on the choice of a suitable dose response function to support dose selection for the subsequent Phase III studies. In this paper, we compare different approaches for model selection and model averaging using mathematical properties as well as simulations. We review and illustrate asymptotic properties of model selection criteria and investigate their behavior when changing the sample size but keeping the effect size constant. In a simulation study, we investigate how the various approaches perform in realistically chosen settings. Finally, the different methods are illustrated with a recently conducted Phase II dose finding study in patients with chronic obstructive pulmonary disease. Copyright © 2016 John Wiley & Sons, Ltd.

Viewpoint: model selection uncertainty, pre-specification, and model averaging.

Scientific progress in all empirical sciences relies on selecting models and performing inferences from selected models. Standard statistical properties (e.g., repeated sampling coverage probability of confidence intervals) cannot be guaranteed after a model selection. This viewpoint reviews this dilemma, puts the role that pre-specification can play into perspective and illustrates model averaging as a way to relax the problem of model selection uncertainty.

Unimodal regression using Bernstein-Schoenberg splines and penalties.

Research in the field of nonparametric shape constrained regression has been intensive. However, only few publications explicitly deal with unimodality although there is need for such methods in applications, for example, in dose-response analysis. In this article, we propose unimodal spline regression methods that make use of Bernstein-Schoenberg splines and their shape preservation property. To achieve unimodal and smooth solutions we use penalized splines, and extend the penalized spline approach toward penalizing against general parametric functions, instead of using just difference penalties. For tuning parameter selection under a unimodality constraint a restricted maximum likelihood and an alternative Bayesian approach for unimodal regression are developed. We compare the proposed methodologies to other common approaches in a simulation study and apply it to a dose-response data set. All results suggest that the unimodality constraint or the combination of unimodality and a penalty can substantially improve estimation of the functional relationship.

Model-based dose finding under model uncertainty using general parametric models.

The statistical methodology for the design and analysis of clinical Phase II dose-response studies, with related software implementation, is well developed for the case of a normally distributed, homoscedastic response considered for a single timepoint in parallel group study designs. In practice, however, binary, count, or time-to-event endpoints are encountered, typically measured repeatedly over time and sometimes in more complex settings like crossover study designs. In this paper, we develop an overarching methodology to perform efficient multiple comparisons and modeling for dose finding, under uncertainty about the dose-response shape, using general parametric models. The framework described here is quite broad and can be utilized in situations involving for example generalized nonlinear models, linear and nonlinear mixed effects models, Cox proportional hazards models, with the main restriction being that a univariate dose-response relationship is modeled, that is, both dose and response correspond to univariate measurements. In addition to the core framework, we also develop a general purpose methodology to fit dose-response data in a computationally and statistically efficient way. Several examples illustrate the breadth of applicability of the results. For the analyses, we developed the R add-on package DoseFinding, which provides a convenient interface to the general approach adopted here.

A practical guide to Bayesian group sequential designs.

Bayesian approaches to the monitoring of group sequential designs have two main advantages compared with classical group sequential designs: first, they facilitate implementation of interim success and futility criteria that are tailored to the subsequent decision making, and second, they allow inclusion of prior information on the treatment difference and on the control group. A general class of Bayesian group sequential designs is presented, where multiple criteria based on the posterior distribution can be defined to reflect clinically meaningful decision criteria on whether to stop or continue the trial at the interim analyses. To evaluate the frequentist operating characteristics of these designs, both simulation methods and numerical integration methods are proposed, as implemented in the corresponding R package gsbDesign. Normal approximations are used to allow fast calculation of these characteristics for various endpoints. The practical implementation of the approach is illustrated with several clinical trial examples from different phases of drug development, with various endpoints, and informative priors.