Estimation of Treatment Policy Estimands for Continuous Outcomes Using Off-Treatment Sequential Multiple Imputation.

The estimands framework outlined in ICH E9 (R1) describes the components needed to precisely define the effects to be estimated in clinical trials, which includes how post-baseline 'intercurrent' events (IEs) are to be handled. In late-stage clinical trials, it is common to handle IEs like 'treatment discontinuation' using the treatment policy strategy and target the treatment effect on outcomes regardless of treatment discontinuation. For continuous repeated measures, this type of effect is often estimated using all observed data before and after discontinuation using either a mixed model for repeated measures (MMRM) or multiple imputation (MI) to handle any missing data. In basic form, both these estimation methods ignore treatment discontinuation in the analysis and therefore may be biased if there are differences in patient outcomes after treatment discontinuation compared with patients still assigned to treatment, and missing data being more common for patients who have discontinued treatment. We therefore propose and evaluate a set of MI models that can accommodate differences between outcomes before and after treatment discontinuation. The models are evaluated in the context of planning a Phase 3 trial for a respiratory disease. We show that analyses ignoring treatment discontinuation can introduce substantial bias and can sometimes underestimate variability. We also show that some of the MI models proposed can successfully correct the bias, but inevitably lead to increases in variance. We conclude that some of the proposed MI models are preferable to the traditional analysis ignoring treatment discontinuation, but the precise choice of MI model will likely depend on the trial design, disease of interest and amount of observed and missing data following treatment discontinuation.

Selection Bias in Health Research: Quantifying, Eliminating, or Exacerbating Health Disparities?

Purpose of review: To summarize recent literature on selection bias in disparities research addressing either descriptive or causal questions, with examples from dementia research. Recent findings: Defining a clear estimand, including the target population, is essential to assess whether generalizability bias or collider-stratification bias are threats to inferences. Selection bias in disparities research can result from sampling strategies, differential inclusion pipelines, loss to follow-up, and competing events. If competing events occur, several potentially relevant estimands can be estimated under different assumptions, with different interpretations. The apparent magnitude of a disparity can differ substantially based on the chosen estimand. Both randomized and observational studies may misrepresent health disparities or heterogeneity in treatment effects if they are not based on a known sampling scheme. Conclusion: Researchers have recently made substantial progress in conceptualization and methods related to selection bias. This progress will improve the relevance of both descriptive and causal health disparities research.

Covariate adjustment in randomized controlled trials: General concepts and practical considerations.

There has been a growing interest in covariate adjustment in the analysis of randomized controlled trials in past years. For instance, the US Food and Drug Administration recently issued guidance that emphasizes the importance of distinguishing between conditional and marginal treatment effects. Although these effects may sometimes coincide in the context of linear models, this is not typically the case in other settings, and this distinction is often overlooked in clinical trial practice. Considering these developments, this article provides a review of when and how to use covariate adjustment to enhance precision in randomized controlled trials. We describe the differences between conditional and marginal estimands and stress the necessity of aligning statistical analysis methods with the chosen estimand. In addition, we highlight the potential misalignment of commonly used methods in estimating marginal treatment effects. We hereby advocate for the use of the standardization approach, as it can improve efficiency by leveraging the information contained in baseline covariates while remaining robust to model misspecification. Finally, we present practical considerations that have arisen in our respective consultations to further clarify the advantages and limitations of covariate adjustment.

Multiply robust estimation of principal causal effects with noncompliance and survival outcomes.

Treatment noncompliance and censoring are two common complications in clinical trials. Motivated by the ADAPTABLE pragmatic clinical trial, we develop methods for assessing treatment effects in the presence of treatment noncompliance with a right-censored survival outcome. We classify the participants into principal strata, defined by their joint potential compliance status under treatment and control. We propose a multiply robust estimator for the causal effects on the survival probability scale within each principal stratum. This estimator is consistent even if one, sometimes two, of the four working models-on the treatment assignment, the principal strata, censoring, and the outcome-is misspecified. A sensitivity analysis strategy is developed to address violations of key identification assumptions, the principal ignorability and monotonicity. We apply the proposed approach to the ADAPTABLE trial to study the causal effect of taking low- versus high-dosage aspirin on all-cause mortality and hospitalization from cardiovascular diseases.

Comments on 'standard and reference-based conditional mean imputation': Regulators and trial statisticians be aware!

Accurate frequentist performance of a method is desirable in confirmatory clinical trials, but is not sufficient on its own to justify the use of a missing data method. Reference-based conditional mean imputation, with variance estimation justified solely by its frequentist performance, has the surprising and undesirable property that the estimated variance becomes smaller the greater the number of missing observations; as explained under jump-to-reference it effectively forces the true treatment effect to be exactly zero for patients with missing data.

[Special issue PRO] A demonstration of estimands and sensitivity analyses for time-to-deterioration of patient reported outcomes.

In oncology trials, health-related quality of life (HRQoL), specifically patient-reported symptom burden and functional status, can support the interpretation of survival endpoints, such as progression-free survival. However, applying time-to-event endpoints to patient-reported outcomes (PRO) data is challenging. For example, in time-to-deterioration analyses clinical events such as disease progression are common in many settings and are often handled through censoring the patient at the time of occurrence; however, disease progression and HRQoL are often related leading to informative censoring. Special consideration to the definition of events and intercurrent events (ICEs) is necessary. In this work, we demonstrate time-to-deterioration of PRO estimands and sensitivity analyses to answer research questions using composite, hypothetical, and treatment policy strategies applied to a single endpoint of disease-related symptoms. Multiple imputation methods under both the missing-at-random and missing-not-at-random assumptions are used as sensitivity analyses of primary estimands. Hazard ratios ranged from 0.52 to 0.66 over all the estimands and sensitivity analyses modeling a robust treatment effect favoring the treatment in time to disease symptom deterioration or death. Differences in the estimands include how people who experience disease progression or discontinue the randomized treatment due to AEs are accounted for in the analysis. We use the estimand framework to define interpretable and principled approaches for different time-to-deterioration research questions and provide practical recommendations. Reporting the proportions of patient events and patient censoring by reason helps understand the mechanisms that drive the results, allowing for optimal interpretation.

Estimands for clinical endpoints in tuberculosis treatment randomized controlled trials: a retrospective application in a completed trial.

BACKGROUND: Randomized trials for the treatment of tuberculosis (TB) rely on a composite primary outcome to capture unfavorable treatment responses. However, variability between trials in the outcome definition and estimation methods complicates across-trial comparisons and hinders the advancement of treatment guidelines. The International Council for Harmonization (ICH) provides international regulatory standards for clinical trials. The estimand framework outlined in the recent ICH E9(R1) addendum offers a timely opportunity for randomized trials of TB treatment to adopt broadly standardized outcome definitions and analytic approaches. We previously proposed and defined four estimands for use in this context. Our objective was to evaluate how the use of these estimands and choice of estimation method impacts results and interpretation of a large phase III TB trial. METHODS: We reanalyzed participant-level data from the REMoxTB trial. We applied four estimands and various methods of estimation to assess non-inferiority of both novel 4-month treatment regimens against standard of care. RESULTS: With each of the four estimands, we reached the same conclusion as the original trial analysis that the novel regimens were not non-inferior to standard of care. Each estimand and method of estimation gave similar estimates of the treatment effect with fluctuations in variance and differences driven by the methods applied for handling intercurrent events. CONCLUSIONS: Our application of estimands defined by the ICH E9 (R1) addendum offers a formalized framework for addressing the primary TB treatment trial objective and can promote uniformity in future trials by limiting heterogeneity in trial outcome definitions. We demonstrated the utility of our proposal using data from the REMoxTB randomized trial. We outlined methods for estimating each estimand and found consistent conclusions across estimands. We recommend future late-phase TB treatment trials to implement some or all of our estimands to promote rigorous outcome definitions and reduce variability between trials. TRIAL REGISTRATION: ClinicalTrials.gov NCT00864383. Registered on March 2009.

Time-to-event estimands and loss to follow-up in oncology in light of the estimands guidance.

Time-to-event estimands are central to many oncology clinical trials. The estimands framework (addendum to the ICH E9 guideline) calls for precisely defining the treatment effect of interest to align with the clinical question of interest and requires predefining the handling of intercurrent events (ICEs) that occur after treatment initiation and "affect either the interpretation or the existence of the measurements associated with the clinical question of interest." We discuss a practical problem in clinical trial design and execution, that is, in some clinical contexts it is not feasible to systematically follow patients to an event of interest. Loss to follow-up in the presence of intercurrent events can affect the meaning and interpretation of the study results. We provide recommendations for trial design, stressing the need for close alignment of the clinical question of interest and study design, impact on data collection, and other practical implications. When patients cannot be systematically followed, compromise may be necessary to select the best available estimand that can be feasibly estimated under the circumstances. We discuss the use of sensitivity and supplementary analyses to examine assumptions of interest.

CITIES: Clinical trials with intercurrent events simulator.

Although clinical trials are often designed with randomization and well-controlled protocols, complications will inevitably arise in the presence of intercurrent events (ICEs) such as treatment discontinuation. These can lead to missing outcome data and possibly confounding causal inference when the missingness is a function of a latent stratification of patients defined by intermediate outcomes. The pharmaceutical industry has been focused on developing new methods that can yield pertinent causal inferences in trials with ICEs. However, it is difficult to compare the properties of different methods developed in this endeavor as real-life clinical trial data cannot be easily shared to provide benchmark data sets. Furthermore, different methods consider distinct assumptions for the underlying data-generating mechanisms, and simulation studies often are customized to specific situations or methods. We develop a novel, general simulation model and corresponding Shiny application in R for clinical trials with ICEs, aptly named the Clinical Trials with Intercurrent Events Simulator (CITIES). It is formulated under the Rubin Causal Model where the considered treatment effects account for ICEs in clinical trials with repeated measures. CITIES facilitates the effective generation of data that resemble real-life clinical trials with respect to their reported summary statistics, without requiring the use of the original trial data. We illustrate the utility of CITIES via two case studies involving real-life clinical trials that demonstrate how CITIES provides a comprehensive tool for practitioners in the pharmaceutical industry to compare methods for the analysis of clinical trials with ICEs on identical, benchmark settings that resemble real-life trials.

Unresolved issues with noninferiority pragmatic trials: Results of a literature survey.

BACKGROUND: Issues with specification of margins, adherence, and analytic population can potentially bias results toward the alternative in randomized noninferiority pragmatic trials. To investigate this potential for bias, we conducted a targeted search of the medical literature to examine how noninferiority pragmatic trials address these issues. METHODS: An Ovid MEDLINE database search was performed identifying publications in New England Journal of Medicine, Journal of the American Medical Association, Lancet, or British Medical Journal published between 2015 and 2021 that included the words "pragmatic" or "comparative effectiveness" and "noninferiority" or "non-inferiority." Our search identified 14 potential trials, 12 meeting our inclusion criteria (11 individually randomized, 1 cluster-randomized). RESULTS: Eleven trials had results that met the criteria established for noninferiority. Noninferiority margins were prespecified for all trials; all but two trials provided justification of the margin. Most trials did some monitoring of treatment adherence. All trials conducted intent-to-treat or modified intent-to-treat analyses along with per-protocol analyses and these analyses reached similar conclusions. Only two trials included all randomized participants in the primary analysis, one used multiple imputation for missing data. The percentage excluded from primary analyses ranged from ∼2% to 30%. Reasons for exclusion included randomization in error, nonadherence, not receiving assigned treatment, death, withdrawal, lost to follow-up, and incomplete data. CONCLUSION: Specification of margins, adherence, and analytic population require careful consideration to prevent bias toward the alternative in noninferiority pragmatic trials. Although separate guidance has been developed for noninferiority and pragmatic trials, it is not compatible with conducting a noninferiority pragmatic trial. Hence, these trials should probably not be done in their current format without developing new guidelines.

Dissecting the restricted mean time in favor of treatment.

The restricted mean time in favor (RMT-IF) summarizes the treatment effect on a hierarchical composite endpoint with mortality at the top. Its crude decomposition into "stage-wise effects," i.e., the net average time gained by the treatment prior to each component event, does not reveal the patient state in which the extra time is spent. To obtain this information, we break each stage-wise effect into subcomponents according to the specific state to which the reference condition is improved. After re-expressing the subcomponents as functionals of the marginal survival functions of outcome events, we estimate them conveniently by plugging in the Kaplan -- Meier estimators. Their robust variance matrices allow us to construct joint tests on the decomposed units, which are particularly powerful against component-wise differential treatment effects. By reanalyzing a cancer trial and a cardiovascular trial, we acquire new insights into the quality and composition of the extra survival times, as well as the extra time with fewer hospitalizations, gained by the treatment in question. The proposed methods are implemented in the rmt package freely available on the Comprehensive R Archive Network (CRAN).

Reflections on estimands for patient-reported outcomes in cancer clinical trials.

It is common and important to include the patient's perspective of the impact of treatment on health-related quality of life (HRQoL) outcomes. In this commentary, we focus on applying the new addendum to ICH E9 guideline E9 (R1) relating to the estimand framework to Patient Reported Outcomes (PROs) collected in cancer clinical trials, from a statistician's viewpoint. Currently, common practice for statistical analysis of PRO endpoints of published cancer clinical trials demonstrates ambiguity, leaving critical questions unspecified, hindering conclusions about the effect of treatment on PRO endpoints as well as comparability between clinical trials. To avoid this scenario, we advocate the systematic use of the estimand framework which requires the prospective definition of clear PRO research questions. Among the five attributes of the estimands framework, the definition of the endpoint (what is the right PRO measure and timeframe to target and why?), the intercurrent event identification and management (what happens with PRO data post-disease progression, what is the impact of death?) and the population-level summary (what is an acceptable statistical summary for PRO data?) require the most attention for PRO estimands. We identify good practice and highlight discussion points including the challenges of statistical analysis in the presence of missing and/or unobservable data and in relation to death. Through this discussion we highlight that there is no "statistical magic", but that the estimand framework will help you find out what you really want to know when quantifying the benefit of treatments from the patients' perspective.

Estimands for clinical endpoints in Tuberculosis treatment randomized controlled trials: a retrospective application in a completed trial.

Background: Randomized trials for the treatment of tuberculosis (TB) rely on a composite primary outcome to capture unfavorable treatment responses. However, variability between trials in the outcome definition and estimation methods complicates across-trial comparisons and hinders the advancement of treatment guidelines. The International Council for Harmonization (ICH) provides international regulatory standards for clinical trials. The estimand framework outlined in the recent ICH E9(R1) addendum offers a timely opportunity for randomized trials of TB treatment to adopt broadly standardized outcome definitions and analytic approaches. We previously proposed and defined four estimands for use in this context. Our objective was to evaluate how the use of these estimands and choice of estimation method impacts results and interpretation of a large phase III TB trial. Methods: We reanalyzed participant level data from the REMoxTB trial. We applied four estimands and various methods of estimation to assess non-inferiority of both novel 4-month treatment regimens against standard of care. Results: With each of the four estimands we reached the same conclusion as the original trial analysis; that the novel regimens were not non-inferior to standard of care. Each estimand and method of estimation gave similar estimates of the treatment effect with fluctuations in variance and differences driven by the methods applied for handling intercurrent events. Conclusions: Our application of estimands defined by the ICH E9(R1) addendum offers a formalized framework for addressing the primary TB treatment trial objective and can promote uniformity in future trials by limiting heterogeneity in trial outcome definitions. We demonstrated the utility of our proposal using data from the REMoxTB randomized trial. We outlined methods for estimating each estimand and found consistent conclusions across estimands. We recommend future late-phase TB treatment trials to implement some or all of our estimands to promote rigorous outcome definitions and reduce variability between trials.Trial registration: NCT00864383.

A causal roadmap for generating high-quality real-world evidence.

Increasing emphasis on the use of real-world evidence (RWE) to support clinical policy and regulatory decision-making has led to a proliferation of guidance, advice, and frameworks from regulatory agencies, academia, professional societies, and industry. A broad spectrum of studies use real-world data (RWD) to produce RWE, ranging from randomized trials with outcomes assessed using RWD to fully observational studies. Yet, many proposals for generating RWE lack sufficient detail, and many analyses of RWD suffer from implausible assumptions, other methodological flaws, or inappropriate interpretations. The Causal Roadmap is an explicit, itemized, iterative process that guides investigators to prespecify study design and analysis plans; it addresses a wide range of guidance within a single framework. By supporting the transparent evaluation of causal assumptions and facilitating objective comparisons of design and analysis choices based on prespecified criteria, the Roadmap can help investigators to evaluate the quality of evidence that a given study is likely to produce, specify a study to generate high-quality RWE, and communicate effectively with regulatory agencies and other stakeholders. This paper aims to disseminate and extend the Causal Roadmap framework for use by clinical and translational researchers; three companion papers demonstrate applications of the Causal Roadmap for specific use cases.

The survival-incorporated median vs the median in the survivors or in the always-survivors: What are we measuring? and Why?

Many clinical studies evaluate the benefit of a treatment based on both survival and other continuous/ordinal clinical outcomes, such as quality of life scores. In these studies, when subjects die before the follow-up assessment, the clinical outcomes become undefined and are truncated by death. Treating outcomes as "missing" or "censored" due to death can be misleading for treatment effect evaluation. We show that if we use the median in the survivors or in the always-survivors as estimands to summarize clinical outcomes, we may conclude that a trade-off exists between the probability of survival and good clinical outcomes, even in settings where both the probability of survival and the probability of any good clinical outcome are better for one treatment. Therefore, we advocate not always treating death as a mechanism through which clinical outcomes are missing, but rather as part of the outcome measure. To account for the survival status, we describe the survival-incorporated median as an alternative summary measure for outcomes in the presence of death. The survival-incorporated median is the threshold such that 50% of the population is alive with an outcome above that threshold. Through conceptual examples and an application to a prostate cancer treatment study, we show that the survival-incorporated median provides a simple and useful summary measure to inform clinical practice.

Power and Sample Size Calculations for the Restricted Mean Time Analysis of Prioritized Composite Endpoints.

As a new way of reporting treatment effect, the restricted mean time in favor (RMT-IF) of treatment measures the net average time the treated have had a less serious outcome than the untreated over a specified time window. With multiple outcomes of differing severity, this offers a more interpretable and data-efficient alternative to the prototypical restricted mean (event-free) survival time. To facilitate its adoption in actual trials, we develop simple approaches to power and sample size calculations and implement them in user-friendly R programs. In doing so we model the bivariate outcomes of death and a nonfatal event using a Gumbel-Hougaard copula with component-wise proportional hazards structures, under which the RMT-IF estimand is derived in closed form. In a standard set-up for censoring, the variance of the nonparametric effect-size estimator is simplified and computed via a hybrid of numerical and Monte Carlo integrations, allowing us to compute the power and sample size as functions of component-wise hazard ratios. Simulation studies show that these formulas provide accurate approximations in realistic settings. To illustrate our methods, we consider designing a new trial to evaluate treatment effect on the composite outcomes of death and cancer relapse in lymph node-positive breast cancer patients, with baseline parameters calculated from a previous study.

Adherence, per-protocol effects, and the estimands framework.

In the statistical literature, treatment effects in clinical trials are frequently described as either ITT or per-protocol effects. The estimand given for the per-protocol effect is the effect in adherers, where adherers are typically defined as adhering to the intervention as specified in the trial protocol. This dichotomy of treatment effects is unhelpful when there are in reality multiple treatment effects that can be of clinical interest and relevance. The terms "per-protocol" and "adherence" are confusing to non-statisticians. Protocols always allow for discontinuation of randomized treatment so participants discontinuing have actually followed the protocol. When rescue or additional medication is available, the effect in adherers could mean the effect regardless of use of these medications or the effect in a counterfactual world where the participant did not take the medication. Adherence can mean continuing to be prescribed a treatment or some arbitrary level of compliance with a medication that has been prescribed. The ICH E9 (R1) estimands framework provides an improved alternative for the description of treatment effects in clinical trials. Identification of important intercurrent events and the strategy used to handle these events is key to determining the treatment effect. When designing a trial, estimands should be properly defined according to this framework. It is time the statistical literature abandoned describing treatment effects as the effect in adherers or the per-protocol effect.

Handling intercurrent events and missing data in non-inferiority trials using the estimand framework: A tuberculosis case study.

INTRODUCTION: The ICH E9 addendum outlining the estimand framework for clinical trials was published in 2019 but provides limited guidance around how to handle intercurrent events for non-inferiority studies. Once an estimand is defined, it is also unclear how to deal with missing values using principled analyses for non-inferiority studies. METHODS: Using a tuberculosis clinical trial as a case study, we propose a primary estimand, and an additional estimand suitable for non-inferiority studies. For estimation, multiple imputation methods that align with the estimands for both primary and sensitivity analysis are proposed. We demonstrate estimation methods using the twofold fully conditional specification multiple imputation algorithm and then extend and use reference-based multiple imputation for a binary outcome to target the relevant estimands, proposing sensitivity analyses under each. We compare the results from using these multiple imputation methods with those from the original study. RESULTS: Consistent with the ICH E9 addendum, estimands can be constructed for a non-inferiority trial which improves on the per-protocol/intention-to-treat-type analysis population previously advocated, involving respectively a hypothetical or treatment policy strategy to handle relevant intercurrent events. Results from using the 'twofold' multiple imputation approach to estimate the primary hypothetical estimand, and using reference-based methods for an additional treatment policy estimand, including sensitivity analyses to handle the missing data, were consistent with the original study's reported per-protocol and intention-to-treat analysis in failing to demonstrate non-inferiority. CONCLUSIONS: Using carefully constructed estimands and appropriate primary and sensitivity estimators, using all the information available, results in a more principled and statistically rigorous approach to analysis. Doing so provides an accurate interpretation of the estimand.

Single-world intervention graphs for defining, identifying, and communicating estimands in clinical trials.

Confusion often arises when attempting to articulate target estimand(s) of a clinical trial in plain language. We aim to rectify this confusion by using a type of causal graph called the Single-World Intervention Graph (SWIG) to provide a visual representation of the estimand that can be effectively communicated to interdisciplinary stakeholders. These graphs not only display estimands, but also illustrate the assumptions under which a causal estimand is identifiable by presenting the graphical relationships between the treatment, intercurrent events, and clinical outcomes. To demonstrate its usefulness in pharmaceutical research, we present examples of SWIGs for various intercurrent event strategies specified in the ICH E9(R1) addendum, as well as an example from a real-world clinical trial for chronic pain. code to generate all the SWIGs shown is this paper is made available. We advocate clinical trialists adopt the use of SWIGs in their estimand discussions during the planning stages of their studies.

A comparison of different population-level summary measures for randomised trials with time-to-event outcomes, with a focus on non-inferiority trials.

BACKGROUND: The population-level summary measure is a key component of the estimand for clinical trials with time-to-event outcomes. This is particularly the case for non-inferiority trials, because different summary measures imply different null hypotheses. Most trials are designed using the hazard ratio as summary measure, but recent studies suggested that the difference in restricted mean survival time might be more powerful, at least in certain situations. In a recent letter, we conjectured that differences between summary measures can be explained using the concept of the non-inferiority frontier and that for a fair simulation comparison of summary measures, the same analysis methods, making the same assumptions, should be used to estimate different summary measures. The aim of this article is to make such a comparison between three commonly used summary measures: hazard ratio, difference in restricted mean survival time and difference in survival at a fixed time point. In addition, we aim to investigate the impact of using an analysis method that assumes proportional hazards on the operating characteristics of a trial designed with any of the three summary measures. METHODS: We conduct a simulation study in the proportional hazards setting. We estimate difference in restricted mean survival time and difference in survival non-parametrically, without assuming proportional hazards. We also estimate all three measures parametrically, using flexible survival regression, under the proportional hazards assumption. RESULTS: Comparing the hazard ratio assuming proportional hazards with the other summary measures not assuming proportional hazards, relative performance varies substantially depending on the specific scenario. Fixing the summary measure, assuming proportional hazards always leads to substantial power gains compared to using non-parametric methods. Fixing the modelling approach to flexible parametric regression assuming proportional hazards, difference in restricted mean survival time is most often the most powerful summary measure among those considered. CONCLUSION: When the hazards are likely to be approximately proportional, reflecting this in the analysis can lead to large gains in power for difference in restricted mean survival time and difference in survival. The choice of summary measure for a non-inferiority trial with time-to-event outcomes should be made on clinical grounds; when any of the three summary measures discussed here is equally justifiable, difference in restricted mean survival time is most often associated with the most powerful test, on the condition that it is estimated under proportional hazards.