Using principal stratification in analysis of clinical trials.

The ICH E9(R1) addendum (2019) proposed principal stratification (PS) as one of five strategies for dealing with intercurrent events. Therefore, understanding the strengths, limitations, and assumptions of PS is important for the broad community of clinical trialists. Many approaches have been developed under the general framework of PS in different areas of research, including experimental and observational studies. These diverse applications have utilized a diverse set of tools and assumptions. Thus, need exists to present these approaches in a unifying manner. The goal of this tutorial is threefold. First, we provide a coherent and unifying description of PS. Second, we emphasize that estimation of effects within PS relies on strong assumptions and we thoroughly examine the consequences of these assumptions to understand in which situations certain assumptions are reasonable. Finally, we provide an overview of a variety of key methods for PS analysis and use a real clinical trial example to illustrate them. Examples of code for implementation of some of these approaches are given in Supplemental Materials.

Choosing Estimands in Clinical Trials: Putting the ICH E9(R1) Into Practice.

The National Research Council (NRC) Expert Panel Report on Prevention and Treatment of Missing Data in Clinical Trials highlighted the need for clearly defining objectives and estimands. That report sparked considerable discussion and literature on estimands and how to choose them. Importantly, consideration moved beyond missing data to include all postrandomization events that have implications for estimating quantities of interest (intercurrent events, aka ICEs). The ICH E9(R1) draft addendum builds on that research to outline key principles in choosing estimands for clinical trials, primarily with focus on confirmatory trials. This paper provides additional insights, perspectives, details, and examples to help put ICH E9(R1) into practice. Specific areas of focus include how the perspectives of different stakeholders influence the choice of estimands; the role of randomization and the intention-to-treat principle; defining the causal effects of a clearly defined treatment regimen, along with the implications this has for trial design and the generalizability of conclusions; detailed discussion of strategies for handling ICEs along with their implications and assumptions; estimands for safety objectives, time-to-event endpoints, early-phase and one-arm trials, and quality of life endpoints; and realistic examples of the thought process involved in defining estimands in specific clinical contexts.

Defining Efficacy Estimands in Clinical Trials: Examples Illustrating ICH E9(R1) Guidelines.

This paper provides examples of defining estimands in real-world scenarios following ICH E9(R1) guidelines. Detailed discussions on choosing the estimands and estimators can be found in our companion papers. Three scenarios of increasing complexity are illustrated. The first example is a proof-of-concept trial in major depressive disorder where the estimand is chosen to support the sponsor decision on whether to continue development. The second and third examples are confirmatory trials in severe asthma and rheumatoid arthritis respectively. We discuss the intercurrent events expected during each trial and how they can be handled so as to be consistent with the study objectives. The estimands discussed in these examples are not the only acceptable choices for their respective scenarios. The intent is to illustrate the key concepts rather than focus on specific choices. Emphasis is placed on following a study development process where estimands link the study objectives with data collection and analysis in a coherent manner, thereby avoiding disconnect between objectives, estimands, and analyses.

Points to consider for analyzing efficacy outcomes in long-term extension clinical trials.

This article focuses on 2 objectives in the analysis of efficacy in long-term extension studies of chronic diseases: (1) defining and discussing estimands of interest in such studies and (2) evaluating the performance of several multiple imputation methods that may be useful in estimating some of these estimands. Specifically, 4 estimands are defined and their clinical utility and inferential ramifications discussed. The performance of several multiple imputation methods and approaches were evaluated using simulated data. Results suggested that when interest is in a binary outcome derived from an underlying continuous measurement, it is preferable to impute the underlying continuous value that is subsequently dichotomized rather than to directly impute the binary outcome. Results also demonstrated that multivariate Gaussian models with Markov chain Monte Carlo imputation and sequential regression have minimal bias and the anticipated confidence interval coverage, even in settings with ordinal data where departures from normality are a concern. These approaches are further illustrated using a long-term extension study in psoriasis.

An evaluation of the trimmed mean approach in clinical trials with dropout.

The trimmed mean is a method of dealing with patient dropout in clinical trials that considers early discontinuation of treatment a bad outcome rather than leading to missing data. The present investigation is the first comprehensive assessment of the approach across a broad set of simulated clinical trial scenarios. In the trimmed mean approach, all patients who discontinue treatment prior to the primary endpoint are excluded from analysis by trimming an equal percentage of bad outcomes from each treatment arm. The untrimmed values are used to calculated means or mean changes. An explicit intent of trimming is to favor the group with lower dropout because having more completers is a beneficial effect of the drug, or conversely, higher dropout is a bad effect. In the simulation study, difference between treatments estimated from trimmed means was greater than the corresponding effects estimated from untrimmed means when dropout favored the experimental group, and vice versa. The trimmed mean estimates a unique estimand. Therefore, comparisons with other methods are difficult to interpret and the utility of the trimmed mean hinges on the reasonableness of its assumptions: dropout is an equally bad outcome in all patients, and adherence decisions in the trial are sufficiently similar to clinical practice in order to generalize the results. Trimming might be applicable to other inter-current events such as switching to or adding rescue medicine. Given the well-known biases in some methods that estimate effectiveness, such as baseline observation carried forward and non-responder imputation, the trimmed mean may be a useful alternative when its assumptions are justifiable.

Evaluation of sPGA × BSA as an Outcome Measure and Treatment Target for Clinical Practice.

Clinical outcome measures are becoming more important in psoriasis treatment. Reliable and standardized measures of severity feasible for clinical practice are needed. Our objective was to investigate body surface area (BSA) and the product of BSA and static Physician Global Assessment (sPGA) (ie, BSA × sPGA) as potential proxy measures for PASI scores. Data were pooled from three multicenter, randomized, double-blind, placebo-controlled, phase 3 trials of ixekizumab in patients with moderate to severe psoriasis (UNCOVER-1, -2, -3; N = 3,866). Assessments included the Psoriasis Area and Severity Index (PASI), BSA, and BSA × sPGA. Rank correlations between BSA × sPGA and PASI were stronger than between BSA and PASI (baseline, r = 0.759 vs. r = 0.707; week 12, r = 0.959 vs. r = 0.924). Week 12 concordance rates with PASI responses were as follows: for 75% reduction in PASI: BSA, 86.2%; BSA × sPGA, 93.8%; for 90% reduction in PASI: BSA, 86.9%; BSA × sPGA, 88.2%. The 75% reduction in PASI positive and negative predictive values were higher for BSA × sPGA versus BSA; for 90% reduction in PASI, positive predictive value was lower and negative predictive value was higher for BSA × sPGA versus BSA. Receiver operating characteristic curve analyses identified the most accurate percentage changes in BSA and BSA × sPGA as 66% and 83% for a 75% reduction in PASI cutoff and 84% and 94% for a 90% reduction in PASI, respectively. These results suggest that BSA and BSA × sPGA are viable tools for use as a PASI proxy by real-world practitioners and may be appropriate measurements for use in clinical practice for treat-to-target strategies.

Improvements in psoriasis within different body regions vary over time following treatment with ixekizumab.

BACKGROUND: Ixekizumab is a high-affinity monoclonal antibody that selectively targets interleukin-17 A. OBJECTIVE: Examine the efficacy of ixekizumab in clearing psoriasis within different body regions. METHODS: Data from 3 placebo- (PBO) or PBO- and etanercept (ETN)-controlled trials were integrated. Patients with moderate-to-severe psoriasis were randomized to 12 weeks of PBO (UNCOVER-1, -2, -3, N = 792; UNCOVER-2, -3, N = 361), 50 mg ETN twice weekly (N = 740), or 80 mg ixekizumab every 2 (IXE Q2W; N = 1169; N = 736) or 4 weeks (IXE Q4W; N = 1165; N = 733) after a 160-mg starting dose. RESULTS: Mean percent improvements in regional Psoriasis Area and Severity Index (PASI) were noted at Week 1 and increased through Week 12 in the IXE Q2W (approved dosing regimen) group for each body region. Week 12 improvements were 91.4% (head/neck); 92.8% (trunk); 89.9% (arms); and 88.7% (legs) (all regions p < .001 vs. PBO). Mean regional PASI improvements at Week 12 were ≥84.2% for ixekizumab versus ≤70.9% for ETN in all regions (p < .001). Scaling and thickness reduced faster than erythema. CONCLUSIONS: Within 12 weeks of ixekizumab treatment, all signs of psoriasis across all body regions reached clinically meaningful improvements, with the head/neck and trunk responding quicker than psoriasis of the arms and legs, especially with reduced scaling and thickness.

Choosing estimands in clinical trials with missing data.

Recent research has fostered new guidance on preventing and treating missing data. Consensus exists that clear objectives should be defined along with the causal estimands; trial design and conduct should maximize adherence to the protocol specified interventions; and a sensible primary analysis should be used along with plausible sensitivity analyses. Two general categories of estimands are effects of the drug as actually taken (de facto, effectiveness) and effects of the drug if taken as directed (de jure, efficacy). Motivated by examples, we argue that no single estimand is likely to meet the needs of all stakeholders and that each estimand has strengths and limitations. Therefore, stakeholder input should be part of an iterative study development process that includes choosing estimands that are consistent with trial objectives. To this end, an example is used to illustrate the benefit from assessing multiple estimands in the same study. A second example illustrates that maximizing adherence reduces sensitivity to missing data assumptions for de jure estimands but may reduce generalizability of results for de facto estimands if efforts to maximize adherence in the trial are not feasible in clinical practice. A third example illustrates that whether or not data after initiation of rescue medication should be included in the primary analysis depends on the estimand to be tested and the clinical setting. We further discuss the sample size and total exposure to placebo implications of including post-rescue data in the primary analysis. Copyright © 2016 John Wiley & Sons, Ltd.

Nonparametric covariate adjustment in estimating hazard ratios.

In randomized clinical trials with time-to-event outcomes, the hazard ratio is commonly used to quantify the treatment effect relative to a control. The Cox regression model is commonly used to adjust for relevant covariates to obtain more accurate estimates of the hazard ratio between treatment groups. However, it is well known that the treatment hazard ratio based on a covariate-adjusted Cox regression model is conditional on the specific covariates and differs from the unconditional hazard ratio that is an average across the population. Therefore, covariate-adjusted Cox models cannot be used when the unconditional inference is desired. In addition, the covariate-adjusted Cox model requires the relatively strong assumption of proportional hazards for each covariate. To overcome these challenges, a nonparametric randomization-based analysis of covariance method was proposed to estimate the covariate-adjusted hazard ratios for multivariate time-to-event outcomes. However, empirical evaluations of the performance (power and type I error rate) of the method have not been studied. Although the method is derived for multivariate situations, for most registration trials, the primary endpoint is a univariate outcome. Therefore, this approach is applied to univariate outcomes, and performance is evaluated through a simulation study in this paper. Stratified analysis is also investigated. As an illustration of the method, we also apply the covariate-adjusted and unadjusted analyses to an oncology trial.

To adjust or not to adjust for baseline when analyzing repeated binary responses? The case of complete data when treatment comparison at study end is of interest.

The benefits of adjusting for baseline covariates are not as straightforward with repeated binary responses as with continuous response variables. Therefore, in this study, we compared different methods for analyzing repeated binary data through simulations when the outcome at the study endpoint is of interest. Methods compared included chi-square, Fisher's exact test, covariate adjusted/unadjusted logistic regression (Adj.logit/Unadj.logit), covariate adjusted/unadjusted generalized estimating equations (Adj.GEE/Unadj.GEE), covariate adjusted/unadjusted generalized linear mixed model (Adj.GLMM/Unadj.GLMM). All these methods preserved the type I error close to the nominal level. Covariate adjusted methods improved power compared with the unadjusted methods because of the increased treatment effect estimates, especially when the correlation between the baseline and outcome was strong, even though there was an apparent increase in standard errors. Results of the Chi-squared test were identical to those for the unadjusted logistic regression. Fisher's exact test was the most conservative test regarding the type I error rate and also with the lowest power. Without missing data, there was no gain in using a repeated measures approach over a simple logistic regression at the final time point. Analysis of results from five phase III diabetes trials of the same compound was consistent with the simulation findings. Therefore, covariate adjusted analysis is recommended for repeated binary data when the study endpoint is of interest.

Comparison of several multiple imputation strategies for repeated measures analysis of clinical scales: to truncate or not to?

We evaluated via a simulation study several strategies for imputing missing ordinal outcomes in a longitudinal clinical trial, contrasting methods that involve truncation of imputed values outside plausible ranges with those that do not. Our aim was to identify a preferred imputation strategy for estimating treatment difference at study endpoint. Plausible data were simulated via resampling of existing placebo data sets and adding treatment effect; then different imputation strategies were evaluated under missingness at random (MAR) and varying dropout rates. Our conclusion is that imputation methods based on rounding and truncation lead to larger bias than strategies based on simple methods based on (nontruncated) multivariate normal distribution.

A multiple-imputation-based approach to sensitivity analyses and effectiveness assessments in longitudinal clinical trials.

It is important to understand the effects of a drug as actually taken (effectiveness) and when taken as directed (efficacy). The primary objective of this investigation was to assess the statistical performance of a method referred to as placebo multiple imputation (pMI) as an estimator of effectiveness and as a worst reasonable case sensitivity analysis in assessing efficacy. The pMI method assumes the statistical behavior of placebo- and drug-treated patients after dropout is the statistical behavior of placebo-treated patients. Thus, in the effectiveness context, pMI assumes no pharmacological benefit of the drug after dropout. In the efficacy context, pMI is a specific form of a missing not at random analysis expected to yield a conservative estimate of efficacy. In a simulation study with 18 scenarios, the pMI approach generally provided unbiased estimates of effectiveness and conservative estimates of efficacy. However, the confidence interval coverage was consistently greater than the nominal coverage rate. In contrast, last and baseline observation carried forward (LOCF and BOCF) were conservative in some scenarios and anti-conservative in others with respect to efficacy and effectiveness. As expected, direct likelihood (DL) and standard multiple imputation (MI) yielded unbiased estimates of efficacy and tended to overestimate effectiveness in those scenarios where a drug effect existed. However, in scenarios with no drug effect, and therefore where the true values for both efficacy and effectiveness were zero, DL and MI yielded unbiased estimates of efficacy and effectiveness.

A phase 2, placebo-controlled study of the opioid receptor antagonist LY2196044 for the treatment of alcohol dependence.

BACKGROUND: Endogenous opioid-mediated reward pathways may play a role in the development and maintenance of alcohol dependence. This study tested whether LY2196044, an opioid receptor antagonist, in combination with medical management would reduce drinking in alcohol-dependent patients. METHODS: This was a multicenter, outpatient, randomized, double-blind, parallel, and placebo-controlled trial with a 16-week treatment period. Patients (N = 375) were alcohol-dependent, treatment-seeking adults. Patients were randomly assigned to once-daily LY2196044 (final doses of 125 or 250 mg/d) or placebo. DNA samples were collected at baseline. At each visit, patients underwent safety assessments, laboratory testing, efficacy measures, and medical management. Blood samples were also obtained for pharmacokinetic testing. The primary measure was the change from baseline in the percent heavy drinking days (HDD). Secondary efficacy measures were percent days abstinent per month and number of drinks per day. RESULTS: The treatment difference in change from baseline in % HDD between LY2196044 and placebo was not statistically significant (-43.02 vs. -38.72%, respectively; p = 0.12). There was a trend toward greater change from baseline in the percent days abstinent per month for the LY2196044 group compared with the placebo group (33.49 vs. 28.12%, respectively; p = 0.051). The decrease from baseline for mean number of drinks per day was statistically significantly greater in the LY2196044 group compared with the placebo group (-5.37 vs. -4.66 drinks per day, respectively; p = 0.013). LY2196044-treated patients who were dopamine receptor type 4-variable number tandem repeat L carriers had greater reductions in % HDD (p = 0.0565), increased percent days abstinent (p = 0.0496), and reduced drinks per day (p = 0.0069) than placebo-treated L carriers. The safety profile for LY2196044 appeared similar to that of other opioid antagonists. CONCLUSIONS: The results from this proof-of-concept clinical trial warrant further evaluation of LY2196044 for the treatment of alcohol dependence.

The challenges of evaluating dose response in flexible-dose trials using marginal structural models.

Assessing dose response from flexible-dose clinical trials is problematic. The true dose effect may be obscured and even reversed in observed data because dose is related to both previous and subsequent outcomes. To remove selection bias, we propose marginal structural models, inverse probability of treatment-weighting (IPTW) methodology. Potential clinical outcomes are compared across dose groups using a marginal structural model (MSM) based on a weighted pooled repeated measures analysis (generalized estimating equations with robust estimates of standard errors), with dose effect represented by current dose and recent dose history, and weights estimated from the data (via logistic regression) and determined as products of (i) inverse probability of receiving dose assignments that were actually received and (ii) inverse probability of remaining on treatment by this time. In simulations, this method led to almost unbiased estimates of true dose effect under various scenarios. Results were compared with those obtained by unweighted analyses and by weighted analyses under various model specifications. The simulation showed that the IPTW MSM methodology is highly sensitive to model misspecification even when weights are known. Practitioners applying MSM should be cautious about the challenges of implementing MSM with real clinical data. Clinical trial data are used to illustrate the methodology.

A portfolio-based approach to optimize proof-of-concept clinical trials.

Improving proof-of-concept (PoC) studies is a primary lever for improving drug development. Since drug development is often done by institutions that work on multiple drugs simultaneously, the present work focused on optimum choices for rates of false positive (α) and false negative (ß) results across a portfolio of PoC studies. Simple examples and a newly derived equation provided conceptual understanding of basic principles regarding optimum choices of α and ß in PoC trials. In examples that incorporated realistic development costs and constraints, the levels of α and ß that maximized the number of approved drugs and portfolio value varied by scenario. Optimum choices were sensitive to the probability the drug was effective and to the proportion of total investment cost prior to establishing PoC. Results of the present investigation agree with previous research in that it is important to assess optimum levels of α and ß. However, the present work also highlighted the need to consider cost structure using realistic input parameters relevant to the question of interest.

Trajectories of depression severity in clinical trials of duloxetine: insights into antidepressant and placebo responses.

CONTEXT: The high percentage of failed clinical trials in depression may be due to high placebo response rates and the failure of standard statistical approaches to capture heterogeneity in treatment response. OBJECTIVE: To assess whether growth mixture modeling can provide insights into antidepressant and placebo responses in clinical trials of patients with major depression. DESIGN: We reanalyzed clinical trials of duloxetine to identify distinct trajectories of Hamilton Scale for Depression (HAM-D) scores during treatment. We analyzed the trajectories in the entire sample and then separately in all active arms and in all placebo arms. Effects of duloxetine hydrochloride, selective serotonin reuptake inhibitor (SSRI), and covariates on the probability of following a particular trajectory were assessed. Outcomes in different trajectories were compared using mixed-effects models. SETTING: Seven randomized double-blind clinical trials of duloxetine vs placebo and comparator SSRI. Patients A total of 2515 patients with major depression. INTERVENTIONS: Duloxetine and comparator SSRI. Main Outcome Measure Total score on the HAM-D. RESULTS: In the entire sample and in the antidepressant-treated subsample, we identified trajectories of responders (76.3% of the sample) and nonresponders (23.7% of the sample). However, placebo-treated patients were characterized by a single response trajectory. Duloxetine and SSRI did not differ in efficacy, and compared with placebo they significantly decreased the odds of following the nonresponder trajectory. Antidepressant responders had significantly better HAM-D scores over time than placebo-treated patients, but antidepressant nonresponders had significantly worse HAM-D scores over time than the placebo-treated patients. CONCLUSIONS: Most patients treated with serotonergic antidepressants showed a clinical trajectory over time that is superior to that of placebo-treated patients. However, some patients receiving these medications did more poorly than patients receiving placebo. These data highlight the importance of ongoing monitoring of medication risks and benefits during serotonergic antidepressant treatment. They should further stimulate the search for biomarkers or other predictors of responder status in guiding antidepressant treatment.

Recent developments in improving signal detection and reducing placebo response in psychiatric clinical trials.

Recent (2007-2010) empirical and theoretical literature on associations of trial design features with signal detection and placebo response were investigated, along with data and analytic considerations. Trials with greater percentages of patients randomized to placebo had larger average drug-placebo differences in two comprehensive meta-analyses (MDD and Schizophrenia). Excluding patients with large responses during double-blind placebo lead-ins resulted in small increases in drug-placebo differences. Core factor subscales of the HAMD yielded larger drug-placebo differences than the HAMD total score. Direct likelihood-based (MMRM) and similar analyses provided better control of false positive and false negative results than LOCF and BOCF. Theoretical considerations suggested that the number of sites and number of countries can influence power, depending on the correlation structure in the data and on how sites and countries are chosen. Use of centralized ratings reduced placebo response and improved drug-placebo differences. However, the number of comparisons was too small to draw conclusions. Use of patient ratings and reducing the number of study visits reduced placebo response, but their effects on signal detection were unclear. Practical experience with novel designs such as the sequential parallel approach hold promise for improvements in signal detection. Given the complexities of signal detection and placebo response, no single strategy is likely to fully solve the problem and combinations of approaches may be most useful. Utilizing appropriate analytic techniques and randomizing an adequate fraction of patients to placebo are perhaps the most broadly applicable approaches.

Signal detection and placebo response in schizophrenia: parallels with depression.

OBJECTIVE: Placebo response and the rate of failed clinical trials are increasing in schizophrenia, resembling previous experience with antidepressant clinical trials. In depression, the percent of patients randomized to placebo was shown to be strongly associated with drug-placebo differences (signal detection).We hypothesized that this factor would also be important in recent schizophrenia clinical trials. To test this hypothesis a database of acute schizophrenia placebo-controlled studies conducted between 1997 and 2008 was constructed. The database contained 27 studies, with 79 active treatment arms. As percentage of patients randomized to placebo increased, mean placebo improvement decreased (p = 0.047) and mean drug-placebo differences tended to increase (p = 0.166). The frequency of significant contrasts from studies with ≥ 25% randomized to placebo was 83.3%, compared with 58.3% in studies with <25% randomized to placebo. Caveats to these findings include limited data and confounding of potentially influential factors. These limitations prevent definitive conclusions. However, results are consistent with previous findings in depression where having a higher percent of patients randomized to placebo increased drug-placebo differences.