Statistical considerations in model-based dose finding for binary responses under model uncertainty.

The statistical methodology for model-based dose finding under model uncertainty has attracted increasing attention in recent years. While the underlying principles are simple and easy to understand, developing and implementing an efficient approach for binary responses can be a formidable task in practice. Motivated by the statistical challenges encountered in a phase II dose finding study, we explore several key design and analysis issues related to the hybrid testing-modeling approaches for binary responses. The issues include candidate model selection and specifications, optimal design and efficient sample size allocations, and, notably, the methods for dose-response testing and estimation. Specifically, we consider a class of generalized linear models suited for the candidate set and establish D-optimal designs for these models. Additionally, we propose using permutation-based tests for dose-response testing to avoid asymptotic normality assumptions typically required for contrast-based tests. We perform trial simulations to enhance our understanding of these issues.

Combination MCP-Mod for two-drug combination dose-ranging studies.

Combination therapies with multiple mechanisms of action can offer improved efficacy and/or safety profiles when compared to a single therapy with one mechanism of action. Consequently, the number of combination therapy studies have increased multi-fold, both in oncology and non-oncology indications. However, identifying the optimal doses of each drug in a combination therapy can require a large sample size and prolong study timelines, especially when full factorial designs are used. In this paper, we extend the MCP-Mod design of Bretz, Pinheiro, and Branson to a three-dimensional space to model the dose-response surface of a two-drug combination under the framework of Combination (Comb) MCP-Mod. The resulting model yields a set of dosages for each drug in the combination that elicits the target response so that an optimal dose for the combination can be selected for pivotal studies. We construct three-dimensional dose-response models for the combination and formulate the contrast test statistic to select the best model, which can then be used to select the optimal dose. Guidance to calculate power and sample size calculations are provided to assist study design. Simulation studies show that Comb MCP-Mod performs as well as the conventional multiple comparisons approach in controlling the family-wise error rate at the desired alpha level. However, Comb MCP-Mod is more powerful than the classical multiple comparisons approach in detecting dose-response relationships when treatment is non-null. The probability of correctly identifying the underlying dose-response relationship is generally higher when using Comb MCP-Mod than when using the multiple comparisons approach.

Calculation of Phase 2 dose-finding study sample size for reliable Phase 3 dose selection.

Sample sizes of Phase 2 dose-finding studies, usually determined based on a power requirement to detect a significant dose-response relationship, will generally not provide adequate precision for Phase 3 target dose selection. We propose to calculate the sample size of a dose-finding study based on the probability of successfully identifying the target dose within an acceptable range (e.g., 80%-120% of the target) using the multiple comparison and modeling procedure (MCP-Mod). With the proposed approach, different design options for the Phase 2 dose-finding study can also be compared. Due to inherent uncertainty around an assumed true dose-response relationship, sensitivity analyses to assess the robustness of the sample size calculations to deviations from modeling assumptions are recommended. Planning for a hypothetical Phase 2 dose-finding study is used to illustrate the main points. Codes for the proposed approach is available at https://github.com/happysundae/posMCPMod.

Improved inference for MCP-Mod approach using time-to-event endpoints with small sample sizes.

The Multiple Comparison Procedures with Modeling Techniques (MCP-Mod) framework has been recently approved by the U.S. Food, Administration, and European Medicines Agency as fit-for-purpose for phase II studies. Nonetheless, this approach relies on the asymptotic properties of Maximum Likelihood (ML) estimators, which might not be reasonable for small sample sizes. In this paper, we derived improved ML estimators and correction for their covariance matrices in the censored Weibull regression model based on the corrective and preventive approaches. We performed two simulation studies to evaluate ML and improved ML estimators with their covariance matrices in (i) a regression framework (ii) the Multiple Comparison Procedures with Modeling Techniques framework. We have shown that improved ML estimators are less biased than ML estimators yielding Wald-type statistics that controls type I error without loss of power in both frameworks. Therefore, we recommend the use of improved ML estimators in the MCP-Mod approach to control type I error at nominal value for sample sizes ranging from 5 to 25 subjects per dose.

A MCP-Mod approach to designing and analyzing survival trials with potential non-proportional hazards.

Non-proportional hazards have been observed in many studies especially in immuno-oncology clinical trials. Traditional analysis using the combined approach with log-rank test as the significance test and Cox model for treatment effect estimation becomes questionable as this approach relies heavily on the proportional hazards assumption. Inspired by the MCP-Mod (multiple comparisons and modeling approach) that has been widely used in dose-finding studies, we propose a similar approach to handle non-proportional hazards. Using this approach, efficacy signal is first established by a max-combo test, after which hazard ratios across time will be estimated using a logically nested splines model. Simulations studies and real-data examples are used to illustrate the use of this approach.

Confirmatory efficacy testing for individual dose-placebo comparisons using serial gatekeeping procedure in dose-finding trials with multiple comparison procedures-modeling.

Dose-finding trials play a key role in the entire drug development process to determine optimal doses for regulatory approval. We address confirmatory efficacy testing for individual dose-placebo comparisons in the context of a dose-finding trial designed with multiple comparison procedures-modeling (MCP-Mod). An extension of the MCP-Mod, called closed MCP-Mod, has been proposed to carry out the MCP-Mod in conjunction with pairwise dose-placebo comparisons; however, an issue associated with the misspecification of candidate dose-response models remains. We consider another way to combine the MCP-Mod and the individual dose-placebo comparisons using serial gatekeeping procedures with fixed sequence, Holm, Hochberg, and step-down Dunnett procedure. The method controls the family-wise error rate in the strong sense and is simple enough to be implemented by existing software. Simulation studies suggested that the serial gatekeeping procedure was comparable with the closed MCP-Mod in terms of statistical power to detect the efficacy of at least one dose, and both methods were capable of pursuing the efficacy claim rather than just establishing the dose-response signal with less than a 20% increase in sample size when assuming monotonic dose-response shapes. The serial gatekeeping procedure would have advantages in the simplicity of implementation and ease of interpretation. The dose-finding trials aiming to declare the dose-response signal, as well as the efficacy of individual doses, would be worth considering as an option to accelerate the drug development program in certain situations.

Model selection characteristics when using MCP-Mod for dose-response gene expression data.

We extend the scope of application for MCP-Mod (Multiple Comparison Procedure and Modeling) to in vitro gene expression data and assess its characteristics regarding model selection for concentration gene expression curves. Precisely, we apply MCP-Mod on single genes of a high-dimensional gene expression data set, where human embryonic stem cells were exposed to eight concentration levels of the compound valproic acid (VPA). As candidate models we consider the sigmoid Emax$E_{\max }$ (four-parameter log-logistic), linear, quadratic, Emax$E_{\max }$ , exponential, and beta model. Through simulations we investigate the impact of omitting one or more models from the candidate model set to uncover possibly superfluous models and to evaluate the precision and recall rates of selected models. Each model is selected according to Akaike information criterion (AIC) for a considerable number of genes. For less noisy cases the popular sigmoid Emax$E_{\max }$ model is frequently selected. For more noisy data, often simpler models like the linear model are selected, but mostly without relevant performance advantage compared to the second best model. Also, the commonly used standard Emax$E_{\max }$ model has an unexpected low performance.

Adaptive dose-response studies to establish proof-of-concept in learning-phase clinical trials.

In learning-phase clinical trials in drug development, adaptive designs can be efficient and highly informative when used appropriately. In this article, we extend the multiple comparison procedures with modeling techniques (MCP-Mod) procedure with generalized multiple contrast tests (GMCTs) to two-stage adaptive designs for establishing proof-of-concept. The results of an interim analysis of first-stage data are used to adapt the candidate dose-response models and the dosages studied in the second stage. GMCTs are used in both stages to obtain stage-wise p -values, which are then combined to determine an overall p -value. An alternative approach is also considered that combines the t -statistics across stages, employing the conditional rejection probability principle to preserve the Type I error probability. Simulation studies demonstrate that the adaptive designs are advantageous compared to the corresponding tests in a nonadaptive design if the selection of the candidate set of dose-response models is not well informed by evidence from preclinical and early-phase studies.

cLRT-Mod: An efficient methodology for pharmacometric model-based analysis of longitudinal phase II dose finding studies under model uncertainty.

Within the challenging context of phase II dose-finding trials, longitudinal analyses may increase drug effect detection power compared to an end-of-treatment analysis. This work proposes cLRT-Mod, a pharmacometric adaptation of the MCP-Mod methodology, which allows the use of nonlinear mixed effect models to first detect a dose-response signal and then identify the doses for the confirmatory phase while accounting for model structure uncertainty. The method was evaluated through extensive clinical trial simulations of a hypothetical phase II dose-finding trial using different scenarios and comparing different methods such as MCP-Mod. The results show an increase in power using cLRT with longitudinal data compared to an EOT multiple contrast tests for scenarios with small sample size and weak drug effect while maintaining pre-specifiability of the models prior to data analysis and the nominal type I error. This work shows how model averaging provides better coverage probability of the drug effect in the prediction step, and avoids under-estimation of the size of the confidence interval. Finally, for illustration purpose cLRT-Mod was applied to the analysis of a real phase II dose-finding trial.

Generalized multiple contrast tests in dose-response studies.

In the process of developing drugs, proof-of-concept studies can be helpful in determining whether there is any evidence of a dose-response relationship. A global test for this purpose that has gained popularity is a component of the multiple comparisons procedure with modeling techniques (MCP-Mod), which involves the specification of a candidate set of several plausible dose-response models. For each model, a test is performed for significance of an optimally chosen contrast among the sample means. An overall P-value is obtained from the distribution of the maximum of the contrast statistics. This is equivalent to basing the test on the minimum of the P-values arising from these contrast statistics and, hence, can be viewed as a method for combining dependent P-values. We generalize this idea to the use of different statistics for combining the dependent P-values, such as Fisher's combination method or the inverse normal combination method. Simulation studies show that the generalized multiple contrast tests (GMCTs) based on the Fisher and inverse normal methods are generally more powerful than the MCP-Mod procedure based on the minimum of the P-values except for cases where the true dose-response model is, in a sense, near the extremes of the candidate set of dose-response models. The proposed GMCTs can also be used for model selection and dosage selection by employing a closed testing procedure.

A multiple comparison procedure for dose-finding trials with subpopulations.

Identifying subgroups of patients with an enhanced response to a new treatment has become an area of increased interest in the last few years. When there is knowledge about possible subpopulations with an enhanced treatment effect before the start of a trial it might be beneficial to set up a testing strategy, which tests for a significant treatment effect not only in the full population, but also in these prespecified subpopulations. In this paper, we present a parametric multiple testing approach for tests in multiple populations for dose-finding trials. Our approach is based on the MCP-Mod methodology, which uses multiple comparison procedures (MCPs) to test for a dose-response signal, while considering multiple possible candidate dose-response shapes. Our proposed methods allow for heteroscedastic error variances between populations and control the family-wise error rate over tests in multiple populations and for multiple candidate models. We show in simulations that the proposed multipopulation testing approaches can increase the power to detect a significant dose-response signal over the standard single-population MCP-Mod, when the specified subpopulation has an enhanced treatment effect.

Power and sample size for dose-finding studies with survival endpoints under model uncertainty.

Multiple comparison procedures combined with modeling techniques (MCP-Mod) (Bretz et al., 2005) is an efficient and robust statistical methodology for the model-based design and analysis of dose-finding studies with an unknown dose-response model. With this approach, multiple comparison methods are used to identify statistically significant contrasts corresponding to a set of candidate dose-response models, and the best model is then used to estimate the target dose. Power and sample size calculations for this methodology require knowledge of the covariance matrix for the estimators of the (placebo-adjusted) mean responses among the dose groups. In this article, we consider survival endpoints and derive an analytic form of the covariance matrix for the estimators of the log hazard ratios as a function of the total number of events in the study. We then use this closed-form expression of the covariance matrix to derive the power and sample size formulas. We discuss practical considerations in the application of these formulas. In addition, we provide an illustration with a motivating example on chronic obstructive pulmonary disease. Finally, we demonstrate through simulation studies that the proposed formulas are accurate enough for practical use.

Comparison of different approaches for dose response analysis.

Characterizing an appropriate dose-response relationship and identifying the right dose in a clinical trial are two main goals of early drug-development. MCP-Mod is one of the pioneer approaches developed within the last 10 years that combines the modeling techniques with multiple comparison procedures to address the above goals in clinical drug development. The MCP-Mod approach begins with a set of potential dose-response models, tests for a significant dose-response effect (proof of concept, PoC) using multiple linear contrasts tests and selects the "best" model among those with a significant contrast test. A disadvantage of the method is that the parameter values of the candidate models need to be fixed a priori for the contrasts tests. This may lead to a loss in power and unreliable model selection. For this reason, several variations of the MCP-Mod approach and a hierarchical model selection approach have been suggested where the parameter values need not be fixed in the proof of concept testing step and can be estimated after the model selection step. This paper provides a numerical comparison of the different MCP-Mod variants and the hierarchical model selection approach with regard to their ability of detecting the dose-response trend, their potential to select the correct model and their accuracy in estimating the dose response shape and minimum effective dose. Additionally, as one of the approaches is based on two-sided model comparisons only, we make it more consistent with the common goals of a PoC study, by extending it to one-sided comparisons between the constant and alternative candidate models in the proof of concept step.

Understanding MCP-MOD dose finding as a method based on linear regression.

MCP-MOD is a testing and model selection approach for clinical dose finding studies. During testing, contrasts of dose group means are derived from candidate dose response models. A multiple-comparison procedure is applied that controls the alpha level for the family of null hypotheses associated with the contrasts. Provided at least one contrast is significant, a corresponding set of "good" candidate models is identified. The model generating the most significant contrast is typically selected. There have been numerous publications on the method. It was endorsed by the European Medicines Agency. The MCP-MOD procedure can be alternatively represented as a method based on simple linear regression, where "simple" refers to the inclusion of an intercept and a single predictor variable, which is a transformation of dose. It is shown that the contrasts are equal to least squares linear regression slope estimates after a rescaling of the predictor variables. The test for each contrast is the usual t statistic for a null slope parameter, except that a variance estimate with fewer degrees of freedom is used in the standard error. Selecting the model corresponding to the most significant contrast P value is equivalent to selecting the predictor variable yielding the smallest residual sum of squares. This criteria orders the models like a common goodness-of-fit test, but it does not assure a good fit. Common inferential methods applied to the selected model are subject to distortions that are often present following data-based model selection.

Testing effect of a drug using multiple nested models for the dose-response.

During development of a drug, typically the choice of dose is based on a Phase II dose-finding trial, where selected doses are included with placebo. Two common statistical dose-finding methods to analyze such trials are separate comparisons of each dose to placebo (using a multiple comparison procedure) or a model-based strategy (where a dose-response model is fitted to all data). The first approach works best when patients are concentrated on few doses, but cannot conclude on doses not tested. Model-based methods allow for interpolation between doses, but the validity depends on the correctness of the assumed dose-response model. Bretz et al. (2005, Biometrics 61, 738-748) suggested a combined approach, which selects one or more suitable models from a set of candidate models using a multiple comparison procedure. The method initially requires a priori estimates of any non-linear parameters of the candidate models, such that there is still a degree of model misspecification possible and one can only evaluate one or a few special cases of a general model. We propose an alternative multiple testing procedure, which evaluates a candidate set of plausible dose-response models against each other to select one final model. The method does not require any a priori parameter estimates and controls the Type I error rate of selecting a too complex model.

A flexible Bayesian approach for modeling monotonic dose-response relationships in drug development trials.

Clinical trials often involve comparing 2-4 doses or regimens of an experimental therapy with a control treatment. These studies might occur early in a drug development process, where the aim might be to demonstrate a basic level of proof (the so-called proof of concept (PoC) studies), at a later stage, to help establish a dose or doses that should be used in phase III trials (dose-finding), or even in confirmatory studies, where the registration of several doses might be considered. When a small number of doses are examined, the ability to implement parametric modeling is somewhat limited. As an alternative, in this paper, a flexible Bayesian model is suggested. In particular, we draw on the idea of using Bayesian model averaging (BMA) to exploit an assumed monotonic dose-response relationship, without using strong parametric assumptions. The approach is exemplified by assessing operating characteristics in the design of a PoC study examining a new treatment for psoriatic arthritis and a post hoc data analysis involving three confirmatory clinical trials, which examined an adjunctive treatment for partial epilepsy. Key difficulties, such as prior specification and computation, are discussed. A further extension, based on combining the flexible modeling with a classical multiple comparisons procedure, known as MCP-MOD, is examined. The benefit of this extension is a potential reduction in the number of simulations that might be needed to investigate operating characteristics of the statistical analysis.

Model-based dose finding under model uncertainty using general parametric models.

The statistical methodology for the design and analysis of clinical Phase II dose-response studies, with related software implementation, is well developed for the case of a normally distributed, homoscedastic response considered for a single timepoint in parallel group study designs. In practice, however, binary, count, or time-to-event endpoints are encountered, typically measured repeatedly over time and sometimes in more complex settings like crossover study designs. In this paper, we develop an overarching methodology to perform efficient multiple comparisons and modeling for dose finding, under uncertainty about the dose-response shape, using general parametric models. The framework described here is quite broad and can be utilized in situations involving for example generalized nonlinear models, linear and nonlinear mixed effects models, Cox proportional hazards models, with the main restriction being that a univariate dose-response relationship is modeled, that is, both dose and response correspond to univariate measurements. In addition to the core framework, we also develop a general purpose methodology to fit dose-response data in a computationally and statistically efficient way. Several examples illustrate the breadth of applicability of the results. For the analyses, we developed the R add-on package DoseFinding, which provides a convenient interface to the general approach adopted here.

A Bayesian design for finding optimal biological dose with mixed types of responses of toxicity and efficacy.

For molecularly targeted therapy and immunotherapy, the targeted dose in the early phase clinical trial has been shifted from the maximum tolerated dose for the cytotoxic drug to the optimal biological dose where both toxicity and efficacy are considered. In this paper, we consider the situation that the responses of toxicity and efficacy are mixed in binary and continuous types, respectively, where the continuous endpoint bears more magnitude information than the binary endpoint after dichotomization. We propose combining two model-based designs to sequentially identify the most efficacious and tolerably safe dose. The employed designs both take the dose level information into account to achieve high estimation efficiency. We demonstrate the superiority of the proposed method to some existing methods by simulation.

Statistical considerations on implementing the MCP-Mod method for binary endpoints in clinical trials.

The Multiple Comparison Procedure - Modelling (MCP-Mod) method was qaulified by regulatory agencies (e.g., EMA in 2014 and FDA in 2016) as an efficient statistical method for Phase 2 dose-finding studies when there is uncertainty about dose-response relationship. As this is a relatively new approach, there is limited literature providing practical guidance on its application. In this paper, we evaluated the performance of the MCP-Mod method for clinical trials with a binary primary endpoint, focusing on (1) the impact of sample size, data variability and treatment effect size on the performance of the MCP-Mod, (2) the impact of candidate model mis-specification, and (3) optimal sample allocation under a fixed sample size. The evaluation was performed via simulations under different scenarios.

Strong dose response after immunotherapy with PQ grass using conjunctival provocation testing.

BACKGROUND: Pollinex Quattro Grass (PQ Grass) is an effective, well-tolerated, short pre-seasonal subcutaneous immunotherapy to treat seasonal allergic rhinoconjunctivitis (SAR) due to grass pollen. In this Phase II study, 4 cumulative doses of PQ Grass and placebo were evaluated to determine its optimal cumulative dose. METHODS: Patients with grass pollen-induced SAR were randomised to either a cumulative dose of PQ Grass (5100, 14400, 27600 and 35600 SU) or placebo, administered as 6 weekly subcutaneous injections over 31-41 days (EudraCT number 2017-000333-31). Standardized conjunctival provocation tests (CPT) using grass pollen allergen extract were performed at screening, baseline and post-treatment to determine the total symptom score (TSS) assessed approximately 4 weeks after dosing. Three models were pre-defined (Emax, logistic, and linear in log-dose model) to evaluate a dose response relationship. RESULTS: In total, 95.5% of the 447 randomized patients received all 6 injections. A highly statistically significant (p < 0.0001), monotonic dose response was observed for all three pre-specified models. All treatment groups showed a statistically significant decrease from baseline in TSS compared to placebo, with the largest decrease observed after 27600 SU (p < 0.0001). The full course of 6 injections was completed by 95.5% of patients. Treatment-emergent adverse events were similar across PQ Grass groups, and mostly mild and transient in nature. CONCLUSIONS: PQ Grass demonstrated a strong curvilinear dose response in TSS following CPT without compromising its safety profile.