Using principal stratification in analysis of clinical trials.

The ICH E9(R1) addendum (2019) proposed principal stratification (PS) as one of five strategies for dealing with intercurrent events. Therefore, understanding the strengths, limitations, and assumptions of PS is important for the broad community of clinical trialists. Many approaches have been developed under the general framework of PS in different areas of research, including experimental and observational studies. These diverse applications have utilized a diverse set of tools and assumptions. Thus, need exists to present these approaches in a unifying manner. The goal of this tutorial is threefold. First, we provide a coherent and unifying description of PS. Second, we emphasize that estimation of effects within PS relies on strong assumptions and we thoroughly examine the consequences of these assumptions to understand in which situations certain assumptions are reasonable. Finally, we provide an overview of a variety of key methods for PS analysis and use a real clinical trial example to illustrate them. Examples of code for implementation of some of these approaches are given in Supplemental Materials.

An evaluation of the trimmed mean approach in clinical trials with dropout.

The trimmed mean is a method of dealing with patient dropout in clinical trials that considers early discontinuation of treatment a bad outcome rather than leading to missing data. The present investigation is the first comprehensive assessment of the approach across a broad set of simulated clinical trial scenarios. In the trimmed mean approach, all patients who discontinue treatment prior to the primary endpoint are excluded from analysis by trimming an equal percentage of bad outcomes from each treatment arm. The untrimmed values are used to calculated means or mean changes. An explicit intent of trimming is to favor the group with lower dropout because having more completers is a beneficial effect of the drug, or conversely, higher dropout is a bad effect. In the simulation study, difference between treatments estimated from trimmed means was greater than the corresponding effects estimated from untrimmed means when dropout favored the experimental group, and vice versa. The trimmed mean estimates a unique estimand. Therefore, comparisons with other methods are difficult to interpret and the utility of the trimmed mean hinges on the reasonableness of its assumptions: dropout is an equally bad outcome in all patients, and adherence decisions in the trial are sufficiently similar to clinical practice in order to generalize the results. Trimming might be applicable to other inter-current events such as switching to or adding rescue medicine. Given the well-known biases in some methods that estimate effectiveness, such as baseline observation carried forward and non-responder imputation, the trimmed mean may be a useful alternative when its assumptions are justifiable.

Points to consider for analyzing efficacy outcomes in long-term extension clinical trials.

This article focuses on 2 objectives in the analysis of efficacy in long-term extension studies of chronic diseases: (1) defining and discussing estimands of interest in such studies and (2) evaluating the performance of several multiple imputation methods that may be useful in estimating some of these estimands. Specifically, 4 estimands are defined and their clinical utility and inferential ramifications discussed. The performance of several multiple imputation methods and approaches were evaluated using simulated data. Results suggested that when interest is in a binary outcome derived from an underlying continuous measurement, it is preferable to impute the underlying continuous value that is subsequently dichotomized rather than to directly impute the binary outcome. Results also demonstrated that multivariate Gaussian models with Markov chain Monte Carlo imputation and sequential regression have minimal bias and the anticipated confidence interval coverage, even in settings with ordinal data where departures from normality are a concern. These approaches are further illustrated using a long-term extension study in psoriasis.

Choosing estimands in clinical trials with missing data.

Recent research has fostered new guidance on preventing and treating missing data. Consensus exists that clear objectives should be defined along with the causal estimands; trial design and conduct should maximize adherence to the protocol specified interventions; and a sensible primary analysis should be used along with plausible sensitivity analyses. Two general categories of estimands are effects of the drug as actually taken (de facto, effectiveness) and effects of the drug if taken as directed (de jure, efficacy). Motivated by examples, we argue that no single estimand is likely to meet the needs of all stakeholders and that each estimand has strengths and limitations. Therefore, stakeholder input should be part of an iterative study development process that includes choosing estimands that are consistent with trial objectives. To this end, an example is used to illustrate the benefit from assessing multiple estimands in the same study. A second example illustrates that maximizing adherence reduces sensitivity to missing data assumptions for de jure estimands but may reduce generalizability of results for de facto estimands if efforts to maximize adherence in the trial are not feasible in clinical practice. A third example illustrates that whether or not data after initiation of rescue medication should be included in the primary analysis depends on the estimand to be tested and the clinical setting. We further discuss the sample size and total exposure to placebo implications of including post-rescue data in the primary analysis. Copyright © 2016 John Wiley & Sons, Ltd.

Nonparametric covariate adjustment in estimating hazard ratios.

In randomized clinical trials with time-to-event outcomes, the hazard ratio is commonly used to quantify the treatment effect relative to a control. The Cox regression model is commonly used to adjust for relevant covariates to obtain more accurate estimates of the hazard ratio between treatment groups. However, it is well known that the treatment hazard ratio based on a covariate-adjusted Cox regression model is conditional on the specific covariates and differs from the unconditional hazard ratio that is an average across the population. Therefore, covariate-adjusted Cox models cannot be used when the unconditional inference is desired. In addition, the covariate-adjusted Cox model requires the relatively strong assumption of proportional hazards for each covariate. To overcome these challenges, a nonparametric randomization-based analysis of covariance method was proposed to estimate the covariate-adjusted hazard ratios for multivariate time-to-event outcomes. However, empirical evaluations of the performance (power and type I error rate) of the method have not been studied. Although the method is derived for multivariate situations, for most registration trials, the primary endpoint is a univariate outcome. Therefore, this approach is applied to univariate outcomes, and performance is evaluated through a simulation study in this paper. Stratified analysis is also investigated. As an illustration of the method, we also apply the covariate-adjusted and unadjusted analyses to an oncology trial.

To adjust or not to adjust for baseline when analyzing repeated binary responses? The case of complete data when treatment comparison at study end is of interest.

The benefits of adjusting for baseline covariates are not as straightforward with repeated binary responses as with continuous response variables. Therefore, in this study, we compared different methods for analyzing repeated binary data through simulations when the outcome at the study endpoint is of interest. Methods compared included chi-square, Fisher's exact test, covariate adjusted/unadjusted logistic regression (Adj.logit/Unadj.logit), covariate adjusted/unadjusted generalized estimating equations (Adj.GEE/Unadj.GEE), covariate adjusted/unadjusted generalized linear mixed model (Adj.GLMM/Unadj.GLMM). All these methods preserved the type I error close to the nominal level. Covariate adjusted methods improved power compared with the unadjusted methods because of the increased treatment effect estimates, especially when the correlation between the baseline and outcome was strong, even though there was an apparent increase in standard errors. Results of the Chi-squared test were identical to those for the unadjusted logistic regression. Fisher's exact test was the most conservative test regarding the type I error rate and also with the lowest power. Without missing data, there was no gain in using a repeated measures approach over a simple logistic regression at the final time point. Analysis of results from five phase III diabetes trials of the same compound was consistent with the simulation findings. Therefore, covariate adjusted analysis is recommended for repeated binary data when the study endpoint is of interest.

A phase 2, placebo-controlled study of the opioid receptor antagonist LY2196044 for the treatment of alcohol dependence.

BACKGROUND: Endogenous opioid-mediated reward pathways may play a role in the development and maintenance of alcohol dependence. This study tested whether LY2196044, an opioid receptor antagonist, in combination with medical management would reduce drinking in alcohol-dependent patients. METHODS: This was a multicenter, outpatient, randomized, double-blind, parallel, and placebo-controlled trial with a 16-week treatment period. Patients (N = 375) were alcohol-dependent, treatment-seeking adults. Patients were randomly assigned to once-daily LY2196044 (final doses of 125 or 250 mg/d) or placebo. DNA samples were collected at baseline. At each visit, patients underwent safety assessments, laboratory testing, efficacy measures, and medical management. Blood samples were also obtained for pharmacokinetic testing. The primary measure was the change from baseline in the percent heavy drinking days (HDD). Secondary efficacy measures were percent days abstinent per month and number of drinks per day. RESULTS: The treatment difference in change from baseline in % HDD between LY2196044 and placebo was not statistically significant (-43.02 vs. -38.72%, respectively; p = 0.12). There was a trend toward greater change from baseline in the percent days abstinent per month for the LY2196044 group compared with the placebo group (33.49 vs. 28.12%, respectively; p = 0.051). The decrease from baseline for mean number of drinks per day was statistically significantly greater in the LY2196044 group compared with the placebo group (-5.37 vs. -4.66 drinks per day, respectively; p = 0.013). LY2196044-treated patients who were dopamine receptor type 4-variable number tandem repeat L carriers had greater reductions in % HDD (p = 0.0565), increased percent days abstinent (p = 0.0496), and reduced drinks per day (p = 0.0069) than placebo-treated L carriers. The safety profile for LY2196044 appeared similar to that of other opioid antagonists. CONCLUSIONS: The results from this proof-of-concept clinical trial warrant further evaluation of LY2196044 for the treatment of alcohol dependence.

A multiple-imputation-based approach to sensitivity analyses and effectiveness assessments in longitudinal clinical trials.

It is important to understand the effects of a drug as actually taken (effectiveness) and when taken as directed (efficacy). The primary objective of this investigation was to assess the statistical performance of a method referred to as placebo multiple imputation (pMI) as an estimator of effectiveness and as a worst reasonable case sensitivity analysis in assessing efficacy. The pMI method assumes the statistical behavior of placebo- and drug-treated patients after dropout is the statistical behavior of placebo-treated patients. Thus, in the effectiveness context, pMI assumes no pharmacological benefit of the drug after dropout. In the efficacy context, pMI is a specific form of a missing not at random analysis expected to yield a conservative estimate of efficacy. In a simulation study with 18 scenarios, the pMI approach generally provided unbiased estimates of effectiveness and conservative estimates of efficacy. However, the confidence interval coverage was consistently greater than the nominal coverage rate. In contrast, last and baseline observation carried forward (LOCF and BOCF) were conservative in some scenarios and anti-conservative in others with respect to efficacy and effectiveness. As expected, direct likelihood (DL) and standard multiple imputation (MI) yielded unbiased estimates of efficacy and tended to overestimate effectiveness in those scenarios where a drug effect existed. However, in scenarios with no drug effect, and therefore where the true values for both efficacy and effectiveness were zero, DL and MI yielded unbiased estimates of efficacy and effectiveness.

Comparison of several multiple imputation strategies for repeated measures analysis of clinical scales: to truncate or not to?

We evaluated via a simulation study several strategies for imputing missing ordinal outcomes in a longitudinal clinical trial, contrasting methods that involve truncation of imputed values outside plausible ranges with those that do not. Our aim was to identify a preferred imputation strategy for estimating treatment difference at study endpoint. Plausible data were simulated via resampling of existing placebo data sets and adding treatment effect; then different imputation strategies were evaluated under missingness at random (MAR) and varying dropout rates. Our conclusion is that imputation methods based on rounding and truncation lead to larger bias than strategies based on simple methods based on (nontruncated) multivariate normal distribution.

A portfolio-based approach to optimize proof-of-concept clinical trials.

Improving proof-of-concept (PoC) studies is a primary lever for improving drug development. Since drug development is often done by institutions that work on multiple drugs simultaneously, the present work focused on optimum choices for rates of false positive (α) and false negative (ß) results across a portfolio of PoC studies. Simple examples and a newly derived equation provided conceptual understanding of basic principles regarding optimum choices of α and ß in PoC trials. In examples that incorporated realistic development costs and constraints, the levels of α and ß that maximized the number of approved drugs and portfolio value varied by scenario. Optimum choices were sensitive to the probability the drug was effective and to the proportion of total investment cost prior to establishing PoC. Results of the present investigation agree with previous research in that it is important to assess optimum levels of α and ß. However, the present work also highlighted the need to consider cost structure using realistic input parameters relevant to the question of interest.

The challenges of evaluating dose response in flexible-dose trials using marginal structural models.

Assessing dose response from flexible-dose clinical trials is problematic. The true dose effect may be obscured and even reversed in observed data because dose is related to both previous and subsequent outcomes. To remove selection bias, we propose marginal structural models, inverse probability of treatment-weighting (IPTW) methodology. Potential clinical outcomes are compared across dose groups using a marginal structural model (MSM) based on a weighted pooled repeated measures analysis (generalized estimating equations with robust estimates of standard errors), with dose effect represented by current dose and recent dose history, and weights estimated from the data (via logistic regression) and determined as products of (i) inverse probability of receiving dose assignments that were actually received and (ii) inverse probability of remaining on treatment by this time. In simulations, this method led to almost unbiased estimates of true dose effect under various scenarios. Results were compared with those obtained by unweighted analyses and by weighted analyses under various model specifications. The simulation showed that the IPTW MSM methodology is highly sensitive to model misspecification even when weights are known. Practitioners applying MSM should be cautious about the challenges of implementing MSM with real clinical data. Clinical trial data are used to illustrate the methodology.

Choosing Estimands in Clinical Trials: Putting the ICH E9(R1) Into Practice.

The National Research Council (NRC) Expert Panel Report on Prevention and Treatment of Missing Data in Clinical Trials highlighted the need for clearly defining objectives and estimands. That report sparked considerable discussion and literature on estimands and how to choose them. Importantly, consideration moved beyond missing data to include all postrandomization events that have implications for estimating quantities of interest (intercurrent events, aka ICEs). The ICH E9(R1) draft addendum builds on that research to outline key principles in choosing estimands for clinical trials, primarily with focus on confirmatory trials. This paper provides additional insights, perspectives, details, and examples to help put ICH E9(R1) into practice. Specific areas of focus include how the perspectives of different stakeholders influence the choice of estimands; the role of randomization and the intention-to-treat principle; defining the causal effects of a clearly defined treatment regimen, along with the implications this has for trial design and the generalizability of conclusions; detailed discussion of strategies for handling ICEs along with their implications and assumptions; estimands for safety objectives, time-to-event endpoints, early-phase and one-arm trials, and quality of life endpoints; and realistic examples of the thought process involved in defining estimands in specific clinical contexts.

Defining Efficacy Estimands in Clinical Trials: Examples Illustrating ICH E9(R1) Guidelines.

This paper provides examples of defining estimands in real-world scenarios following ICH E9(R1) guidelines. Detailed discussions on choosing the estimands and estimators can be found in our companion papers. Three scenarios of increasing complexity are illustrated. The first example is a proof-of-concept trial in major depressive disorder where the estimand is chosen to support the sponsor decision on whether to continue development. The second and third examples are confirmatory trials in severe asthma and rheumatoid arthritis respectively. We discuss the intercurrent events expected during each trial and how they can be handled so as to be consistent with the study objectives. The estimands discussed in these examples are not the only acceptable choices for their respective scenarios. The intent is to illustrate the key concepts rather than focus on specific choices. Emphasis is placed on following a study development process where estimands link the study objectives with data collection and analysis in a coherent manner, thereby avoiding disconnect between objectives, estimands, and analyses.

Duloxetine for the treatment of recurrent major depressive disorder in elderly patients: treatment outcomes in patients with comorbid arthritis.

BACKGROUND: Evaluation and treatment of major depression (MDD) in elderly patients is frequently complicated by the presence of comorbid medical conditions, which can reduce the effect of depression treatment, leading to lower rates of depressive-symptom improvement and higher rates of relapse. OBJECTIVE: The authors investigated results of antidepressant concurrent with arthritis pain treatment in elderly patients. METHOD: Patients age 65 and over with recurrent MDD were stratified by arthritis status and randomized to duloxetine (a dual reuptake-inhibitor of serotonin and norepinephrine) or placebo treatment for 8 weeks (duloxetine, N=117; placebo, N=55). RESULTS: Duloxetine significantly reduced MDD symptom severity in elderly patients with and without arthritis, and produced significant reduction in several pain measures in those patients with comorbid arthritis. DISCUSSION: The magnitude and time-course of depressive symptom improvement did not differ significantly between patients with and without arthritis. Some studies have suggested that the severity of pain in arthritis patients may be linked to depression severity.

Evaluating dose response from flexible dose clinical trials.

BACKGROUND: The true dose effect in flexible-dose clinical trials may be obscured and even reversed because dose and outcome are related. METHODS: To evaluate dose effect in response on primary efficacy scales from 2 randomized, double-blind, flexible-dose trials of patients with bipolar mania who received olanzapine (N = 234, 5-20 mg/day), or patients with schizophrenia who received olanzapine (N = 172, 10-20 mg/day), we used marginal structural models, inverse probability of treatment weighting (MSM, IPTW) methodology. Dose profiles for mean changes from baseline were evaluated using weighted MSM with a repeated measures model. To adjust for selection bias due to non-random dose assignment and dropouts, patient-specific time-dependent weights were determined as products of (i) stable weights based on inverse probability of receiving the sequence of dose assignments that was actually received by a patient up to given time multiplied by (ii) stable weights based on inverse probability of patient remaining on treatment by that time. Results were compared with those by unweighted analyses. RESULTS: While the observed difference in efficacy scores for dose groups for the unweighted analysis strongly favored lower doses, the weighted analyses showed no strong dose effects and, in some cases, reversed the apparent "negative dose effect." CONCLUSION: While naïve comparison of groups by last or modal dose in a flexible-dose trial may result in severely biased efficacy analyses, the MSM with IPTW estimators approach may be a valuable method of removing these biases and evaluating potential dose effect, which may prove useful for planning confirmatory trials.

The significance of treating somatic symptoms on functional outcome improvement in patients with major depressive disorder: a post hoc analysis of 2 trials.

BACKGROUND: Functional impairment is associated with major depressive disorder (MDD), and patients with MDD often present with somatic symptoms. OBJECTIVE: To examine the relationships between improved global functioning and core depressive symptoms as well as painful and nonpainful somatic symptoms in patients with MDD. METHOD: This post hoc analysis of 2 identical trials compared the efficacy of duloxetine with that of paroxetine or placebo as treatment of MDD. In the trials, patients with DSM-IV-defined MDD received duloxetine 80 mg/day (N = 188), duloxetine 120 mg/day (N = 196), paroxetine 20 mg/day (N = 183), or placebo (N = 192) for 8 weeks. The Sheehan Disability Scale (SDS), Maier subscale of the 17-item Hamilton Rating Scale for Depression, 21-item Somatic Symptom Inventory, and Visual Analog Scale for overall pain were used to measure functional impairment, core symptoms of depression, and nonpainful and painful somatic symptoms, respectively. Baseline-to-endpoint mean changes in SDS total and subdomains were measured using analysis of variance with last-observation-carried-forward Pearson partial correlations, and path analysis was used to assess the significance of associations and relative contributions of improvement in global functional impairment, depression, and somatic symptoms. The trials were conducted from November 2000 to July 2002. RESULTS: The difference between antidepressant treatment and placebo in SDS total and subdomains was significant (p < .001). At baseline and in change from baseline to endpoint, associations between global functional impairment and core depressive and somatic symptoms were all significant (p < .05). Path analysis demonstrated improvement of functional impairment attributed to treatment effect as 37.0% (core depressive symptoms), 13.0% (nonpainful somatic symptoms), and 11.0% (painful somatic symptoms). CONCLUSION: In patients with MDD, over a third of functional improvement associated with antidepressant therapy was mediated through improvement in core depressive symptoms. In addition, a significant proportion of functional improvement, although to a lesser degree, was associated with the treatment of both nonpainful and painful somatic symptoms.

The impact of missing data and how it is handled on the rate of false-positive results in drug development.

In drug development, a common choice for the primary analysis is to assess mean changes via analysis of (co)variance with missing data imputed by carrying the last or baseline observations forward (LOCF, BOCF). These approaches assume that data are missing completely at random (MCAR). Multiple imputation (MI) and likelihood-based repeated measures (MMRM) are less restrictive as they assume data are missing at random (MAR). Nevertheless, LOCF and BOCF remain popular, perhaps because it is thought that the bias in these methods lead to protection against falsely concluding that a drug is more effective than the control. We conducted a simulation study that compared the rate of false positive results or regulatory risk error (RRE) from BOCF, LOCF, MI, and MMRM in 32 scenarios that were generated from a 2(5) full factorial arrangement with data missing due to a missing not at random (MNAR) mechanism. Both BOCF and LOCF inflated RRE were compared to MI and MMRM. In 12 of the 32 scenarios, BOCF yielded inflated RRE compared with eight scenarios for LOCF, three scenarios for MI and four scenarios for MMRM. In no situation did BOCF or LOCF provide adequate control of RRE when MI and MMRM did not. Both MI and MMRM are better choices than either BOCF or LOCF for the primary analysis.

Improvements in psoriasis within different body regions vary over time following treatment with ixekizumab.

BACKGROUND: Ixekizumab is a high-affinity monoclonal antibody that selectively targets interleukin-17 A. OBJECTIVE: Examine the efficacy of ixekizumab in clearing psoriasis within different body regions. METHODS: Data from 3 placebo- (PBO) or PBO- and etanercept (ETN)-controlled trials were integrated. Patients with moderate-to-severe psoriasis were randomized to 12 weeks of PBO (UNCOVER-1, -2, -3, N = 792; UNCOVER-2, -3, N = 361), 50 mg ETN twice weekly (N = 740), or 80 mg ixekizumab every 2 (IXE Q2W; N = 1169; N = 736) or 4 weeks (IXE Q4W; N = 1165; N = 733) after a 160-mg starting dose. RESULTS: Mean percent improvements in regional Psoriasis Area and Severity Index (PASI) were noted at Week 1 and increased through Week 12 in the IXE Q2W (approved dosing regimen) group for each body region. Week 12 improvements were 91.4% (head/neck); 92.8% (trunk); 89.9% (arms); and 88.7% (legs) (all regions p < .001 vs. PBO). Mean regional PASI improvements at Week 12 were ≥84.2% for ixekizumab versus ≤70.9% for ETN in all regions (p < .001). Scaling and thickness reduced faster than erythema. CONCLUSIONS: Within 12 weeks of ixekizumab treatment, all signs of psoriasis across all body regions reached clinically meaningful improvements, with the head/neck and trunk responding quicker than psoriasis of the arms and legs, especially with reduced scaling and thickness.

Evaluation of sPGA × BSA as an Outcome Measure and Treatment Target for Clinical Practice.

Clinical outcome measures are becoming more important in psoriasis treatment. Reliable and standardized measures of severity feasible for clinical practice are needed. Our objective was to investigate body surface area (BSA) and the product of BSA and static Physician Global Assessment (sPGA) (ie, BSA × sPGA) as potential proxy measures for PASI scores. Data were pooled from three multicenter, randomized, double-blind, placebo-controlled, phase 3 trials of ixekizumab in patients with moderate to severe psoriasis (UNCOVER-1, -2, -3; N = 3,866). Assessments included the Psoriasis Area and Severity Index (PASI), BSA, and BSA × sPGA. Rank correlations between BSA × sPGA and PASI were stronger than between BSA and PASI (baseline, r = 0.759 vs. r = 0.707; week 12, r = 0.959 vs. r = 0.924). Week 12 concordance rates with PASI responses were as follows: for 75% reduction in PASI: BSA, 86.2%; BSA × sPGA, 93.8%; for 90% reduction in PASI: BSA, 86.9%; BSA × sPGA, 88.2%. The 75% reduction in PASI positive and negative predictive values were higher for BSA × sPGA versus BSA; for 90% reduction in PASI, positive predictive value was lower and negative predictive value was higher for BSA × sPGA versus BSA. Receiver operating characteristic curve analyses identified the most accurate percentage changes in BSA and BSA × sPGA as 66% and 83% for a 75% reduction in PASI cutoff and 84% and 94% for a 90% reduction in PASI, respectively. These results suggest that BSA and BSA × sPGA are viable tools for use as a PASI proxy by real-world practitioners and may be appropriate measurements for use in clinical practice for treat-to-target strategies.