A note on "Simple graphical rules to assess selection bias in general-population and selected-sample treatment effects" by M. B. Mathur and I. Shpitser.

This short note is a commentary on the paper by Mathur and Shpitser (2024), with the aim to enlarge the class of graphs for which the conditional Average Treatment Effect is nonparametrically identified, by allowing the outcome to be on the pathway between the treatment and the selection indicator. A first straightforward generalization is possible when (i) the outcome Y is binary, and (ii) the population prevalence of Y is known a priori, or can be made the object of a sensitivity analysis. Furthermore, identification of the effect is possible also for Y having any nature, provided that a selection bias breaking node  V exists and the population prevalence of V is known.

A Causal Mediation Approach to Account for Interaction of Treatment and Intercurrent Events: Using Hypothetical Strategy.

Hypothetical strategy is a common strategy for handling intercurrent events (IEs). No current guideline or study considers treatment-IE interaction to target the estimand in any one IE-handling strategy. Based on the hypothetical strategy, we aimed to (1) assess the performance of three estimators with different considerations for the treatment-IE interaction in a simulation and (2) compare the estimation of these estimators in a real trial. Simulation data were generalized based on realistic clinical trials of Alzheimer's disease. The estimand of interest was the effect of treatment with no IE occurring under the hypothetical strategy. Three estimators, namely, G-estimation with and without interaction and IE-ignored estimation, were compared in scenarios where the treatment-IE interaction effect was set as -50% to 50% of the main effect. Bias was the key performance measure. The real case was derived from a randomized trial of methadone maintenance treatment. Only G-estimation with interaction exhibited unbiased estimations regardless of the existence, direction or magnitude of the treatment-IE interaction in those scenarios. Neglecting the interaction and ignoring the IE would introduce a bias as large as 0.093 and 0.241 (true value, -1.561) if the interaction effect existed. In the real case, compared with G-estimation with interaction, G-estimation without interaction and IE-ignored estimation increased the estimand of interest by 33.55% and 34.36%, respectively. This study highlights the importance of considering treatment-IE interaction in the estimand framework. In practice, it would be better to include the interaction in the estimator by default.

Estimands in clinical trials of complex disease processes.

Clinical trials with random assignment of treatment provide evidence about causal effects of an experimental treatment compared to standard care. However, when disease processes involve multiple types of possibly semi-competing events, specification of target estimands and causal inferences can be challenging. Intercurrent events such as study withdrawal, the introduction of rescue medication, and death further complicate matters. There has been much discussion about these issues in recent years, but guidance remains ambiguous. Some recommended approaches are formulated in terms of hypothetical settings that have little bearing in the real world. We discuss issues in formulating estimands, beginning with intercurrent events in the context of a linear model and then move on to more complex disease history processes amenable to multistate modeling. We elucidate the meaning of estimands implicit in some recommended approaches for dealing with intercurrent events and highlight the disconnect between estimands formulated in terms of potential outcomes and the real world.

Estimation of Treatment Policy Estimands for Continuous Outcomes Using Off-Treatment Sequential Multiple Imputation.

The estimands framework outlined in ICH E9 (R1) describes the components needed to precisely define the effects to be estimated in clinical trials, which includes how post-baseline 'intercurrent' events (IEs) are to be handled. In late-stage clinical trials, it is common to handle IEs like 'treatment discontinuation' using the treatment policy strategy and target the treatment effect on outcomes regardless of treatment discontinuation. For continuous repeated measures, this type of effect is often estimated using all observed data before and after discontinuation using either a mixed model for repeated measures (MMRM) or multiple imputation (MI) to handle any missing data. In basic form, both these estimation methods ignore treatment discontinuation in the analysis and therefore may be biased if there are differences in patient outcomes after treatment discontinuation compared with patients still assigned to treatment, and missing data being more common for patients who have discontinued treatment. We therefore propose and evaluate a set of MI models that can accommodate differences between outcomes before and after treatment discontinuation. The models are evaluated in the context of planning a Phase 3 trial for a respiratory disease. We show that analyses ignoring treatment discontinuation can introduce substantial bias and can sometimes underestimate variability. We also show that some of the MI models proposed can successfully correct the bias, but inevitably lead to increases in variance. We conclude that some of the proposed MI models are preferable to the traditional analysis ignoring treatment discontinuation, but the precise choice of MI model will likely depend on the trial design, disease of interest and amount of observed and missing data following treatment discontinuation.

Principles for Defining Estimands in Clinical Trials-A Proposal.

The ICH E9(R1) guideline outlines the estimand framework, which aligns planning, design, conduct, analysis, and interpretation of a clinical trial. The benefits and value of using this framework in clinical trials have been outlined in the literature, and guidance has been provided on how to choose the estimand and define the estimand attributes. Although progress has been made in the implementation of estimands in clinical trials, to the best of our knowledge, there is no published discussion on the basic principles that estimands in clinical trials should fulfill to be well defined and consistent with the ideas presented in the ICH E9(R1) guideline. Therefore, in this Viewpoint article, we propose four key principles for defining an estimand. These principles form a basis for well-defined treatment effects that reflect the estimand thinking process. We hope that this Viewpoint will complement ICH E9(R1) and stimulate a discussion on which fundamental properties an estimand in a clinical trial should have and that such discussions will eventually lead to an improved clarity and precision for defining estimands in clinical trials.

Broad versus narrow research questions in evidence synthesis: A parallel to (and plea for) estimands.

There has been a transition from broad to more specific research questions in the practice of network meta-analysis (NMA). Such convergence is also taking place in the context of individual registrational trials, following the recent introduction of the estimand framework, which is impacting the design, data collection strategy, analysis and interpretation of clinical trials. The language of estimands has much to offer to NMA, particularly given the "narrow" perspective of treatments and target populations taken in health technology assessment.

Selection Bias in Health Research: Quantifying, Eliminating, or Exacerbating Health Disparities?

Purpose of review: To summarize recent literature on selection bias in disparities research addressing either descriptive or causal questions, with examples from dementia research. Recent findings: Defining a clear estimand, including the target population, is essential to assess whether generalizability bias or collider-stratification bias are threats to inferences. Selection bias in disparities research can result from sampling strategies, differential inclusion pipelines, loss to follow-up, and competing events. If competing events occur, several potentially relevant estimands can be estimated under different assumptions, with different interpretations. The apparent magnitude of a disparity can differ substantially based on the chosen estimand. Both randomized and observational studies may misrepresent health disparities or heterogeneity in treatment effects if they are not based on a known sampling scheme. Conclusion: Researchers have recently made substantial progress in conceptualization and methods related to selection bias. This progress will improve the relevance of both descriptive and causal health disparities research.

Covariate adjustment in randomized controlled trials: General concepts and practical considerations.

There has been a growing interest in covariate adjustment in the analysis of randomized controlled trials in past years. For instance, the US Food and Drug Administration recently issued guidance that emphasizes the importance of distinguishing between conditional and marginal treatment effects. Although these effects may sometimes coincide in the context of linear models, this is not typically the case in other settings, and this distinction is often overlooked in clinical trial practice. Considering these developments, this article provides a review of when and how to use covariate adjustment to enhance precision in randomized controlled trials. We describe the differences between conditional and marginal estimands and stress the necessity of aligning statistical analysis methods with the chosen estimand. In addition, we highlight the potential misalignment of commonly used methods in estimating marginal treatment effects. We hereby advocate for the use of the standardization approach, as it can improve efficiency by leveraging the information contained in baseline covariates while remaining robust to model misspecification. Finally, we present practical considerations that have arisen in our respective consultations to further clarify the advantages and limitations of covariate adjustment.

Multiply robust estimation of principal causal effects with noncompliance and survival outcomes.

Treatment noncompliance and censoring are two common complications in clinical trials. Motivated by the ADAPTABLE pragmatic clinical trial, we develop methods for assessing treatment effects in the presence of treatment noncompliance with a right-censored survival outcome. We classify the participants into principal strata, defined by their joint potential compliance status under treatment and control. We propose a multiply robust estimator for the causal effects on the survival probability scale within each principal stratum. This estimator is consistent even if one, sometimes two, of the four working models-on the treatment assignment, the principal strata, censoring, and the outcome-is misspecified. A sensitivity analysis strategy is developed to address violations of key identification assumptions, the principal ignorability and monotonicity. We apply the proposed approach to the ADAPTABLE trial to study the causal effect of taking low- versus high-dosage aspirin on all-cause mortality and hospitalization from cardiovascular diseases.

Comments on 'standard and reference-based conditional mean imputation': Regulators and trial statisticians be aware!

Accurate frequentist performance of a method is desirable in confirmatory clinical trials, but is not sufficient on its own to justify the use of a missing data method. Reference-based conditional mean imputation, with variance estimation justified solely by its frequentist performance, has the surprising and undesirable property that the estimated variance becomes smaller the greater the number of missing observations; as explained under jump-to-reference it effectively forces the true treatment effect to be exactly zero for patients with missing data.

[Special issue PRO] A demonstration of estimands and sensitivity analyses for time-to-deterioration of patient reported outcomes.

In oncology trials, health-related quality of life (HRQoL), specifically patient-reported symptom burden and functional status, can support the interpretation of survival endpoints, such as progression-free survival. However, applying time-to-event endpoints to patient-reported outcomes (PRO) data is challenging. For example, in time-to-deterioration analyses clinical events such as disease progression are common in many settings and are often handled through censoring the patient at the time of occurrence; however, disease progression and HRQoL are often related leading to informative censoring. Special consideration to the definition of events and intercurrent events (ICEs) is necessary. In this work, we demonstrate time-to-deterioration of PRO estimands and sensitivity analyses to answer research questions using composite, hypothetical, and treatment policy strategies applied to a single endpoint of disease-related symptoms. Multiple imputation methods under both the missing-at-random and missing-not-at-random assumptions are used as sensitivity analyses of primary estimands. Hazard ratios ranged from 0.52 to 0.66 over all the estimands and sensitivity analyses modeling a robust treatment effect favoring the treatment in time to disease symptom deterioration or death. Differences in the estimands include how people who experience disease progression or discontinue the randomized treatment due to AEs are accounted for in the analysis. We use the estimand framework to define interpretable and principled approaches for different time-to-deterioration research questions and provide practical recommendations. Reporting the proportions of patient events and patient censoring by reason helps understand the mechanisms that drive the results, allowing for optimal interpretation.

Estimands for clinical endpoints in tuberculosis treatment randomized controlled trials: a retrospective application in a completed trial.

BACKGROUND: Randomized trials for the treatment of tuberculosis (TB) rely on a composite primary outcome to capture unfavorable treatment responses. However, variability between trials in the outcome definition and estimation methods complicates across-trial comparisons and hinders the advancement of treatment guidelines. The International Council for Harmonization (ICH) provides international regulatory standards for clinical trials. The estimand framework outlined in the recent ICH E9(R1) addendum offers a timely opportunity for randomized trials of TB treatment to adopt broadly standardized outcome definitions and analytic approaches. We previously proposed and defined four estimands for use in this context. Our objective was to evaluate how the use of these estimands and choice of estimation method impacts results and interpretation of a large phase III TB trial. METHODS: We reanalyzed participant-level data from the REMoxTB trial. We applied four estimands and various methods of estimation to assess non-inferiority of both novel 4-month treatment regimens against standard of care. RESULTS: With each of the four estimands, we reached the same conclusion as the original trial analysis that the novel regimens were not non-inferior to standard of care. Each estimand and method of estimation gave similar estimates of the treatment effect with fluctuations in variance and differences driven by the methods applied for handling intercurrent events. CONCLUSIONS: Our application of estimands defined by the ICH E9 (R1) addendum offers a formalized framework for addressing the primary TB treatment trial objective and can promote uniformity in future trials by limiting heterogeneity in trial outcome definitions. We demonstrated the utility of our proposal using data from the REMoxTB randomized trial. We outlined methods for estimating each estimand and found consistent conclusions across estimands. We recommend future late-phase TB treatment trials to implement some or all of our estimands to promote rigorous outcome definitions and reduce variability between trials. TRIAL REGISTRATION: ClinicalTrials.gov NCT00864383. Registered on March 2009.

Time-to-event estimands and loss to follow-up in oncology in light of the estimands guidance.

Time-to-event estimands are central to many oncology clinical trials. The estimands framework (addendum to the ICH E9 guideline) calls for precisely defining the treatment effect of interest to align with the clinical question of interest and requires predefining the handling of intercurrent events (ICEs) that occur after treatment initiation and "affect either the interpretation or the existence of the measurements associated with the clinical question of interest." We discuss a practical problem in clinical trial design and execution, that is, in some clinical contexts it is not feasible to systematically follow patients to an event of interest. Loss to follow-up in the presence of intercurrent events can affect the meaning and interpretation of the study results. We provide recommendations for trial design, stressing the need for close alignment of the clinical question of interest and study design, impact on data collection, and other practical implications. When patients cannot be systematically followed, compromise may be necessary to select the best available estimand that can be feasibly estimated under the circumstances. We discuss the use of sensitivity and supplementary analyses to examine assumptions of interest.

CITIES: Clinical trials with intercurrent events simulator.

Although clinical trials are often designed with randomization and well-controlled protocols, complications will inevitably arise in the presence of intercurrent events (ICEs) such as treatment discontinuation. These can lead to missing outcome data and possibly confounding causal inference when the missingness is a function of a latent stratification of patients defined by intermediate outcomes. The pharmaceutical industry has been focused on developing new methods that can yield pertinent causal inferences in trials with ICEs. However, it is difficult to compare the properties of different methods developed in this endeavor as real-life clinical trial data cannot be easily shared to provide benchmark data sets. Furthermore, different methods consider distinct assumptions for the underlying data-generating mechanisms, and simulation studies often are customized to specific situations or methods. We develop a novel, general simulation model and corresponding Shiny application in R for clinical trials with ICEs, aptly named the Clinical Trials with Intercurrent Events Simulator (CITIES). It is formulated under the Rubin Causal Model where the considered treatment effects account for ICEs in clinical trials with repeated measures. CITIES facilitates the effective generation of data that resemble real-life clinical trials with respect to their reported summary statistics, without requiring the use of the original trial data. We illustrate the utility of CITIES via two case studies involving real-life clinical trials that demonstrate how CITIES provides a comprehensive tool for practitioners in the pharmaceutical industry to compare methods for the analysis of clinical trials with ICEs on identical, benchmark settings that resemble real-life trials.

Unresolved issues with noninferiority pragmatic trials: Results of a literature survey.

BACKGROUND: Issues with specification of margins, adherence, and analytic population can potentially bias results toward the alternative in randomized noninferiority pragmatic trials. To investigate this potential for bias, we conducted a targeted search of the medical literature to examine how noninferiority pragmatic trials address these issues. METHODS: An Ovid MEDLINE database search was performed identifying publications in New England Journal of Medicine, Journal of the American Medical Association, Lancet, or British Medical Journal published between 2015 and 2021 that included the words "pragmatic" or "comparative effectiveness" and "noninferiority" or "non-inferiority." Our search identified 14 potential trials, 12 meeting our inclusion criteria (11 individually randomized, 1 cluster-randomized). RESULTS: Eleven trials had results that met the criteria established for noninferiority. Noninferiority margins were prespecified for all trials; all but two trials provided justification of the margin. Most trials did some monitoring of treatment adherence. All trials conducted intent-to-treat or modified intent-to-treat analyses along with per-protocol analyses and these analyses reached similar conclusions. Only two trials included all randomized participants in the primary analysis, one used multiple imputation for missing data. The percentage excluded from primary analyses ranged from ∼2% to 30%. Reasons for exclusion included randomization in error, nonadherence, not receiving assigned treatment, death, withdrawal, lost to follow-up, and incomplete data. CONCLUSION: Specification of margins, adherence, and analytic population require careful consideration to prevent bias toward the alternative in noninferiority pragmatic trials. Although separate guidance has been developed for noninferiority and pragmatic trials, it is not compatible with conducting a noninferiority pragmatic trial. Hence, these trials should probably not be done in their current format without developing new guidelines.

Dissecting the restricted mean time in favor of treatment.

The restricted mean time in favor (RMT-IF) summarizes the treatment effect on a hierarchical composite endpoint with mortality at the top. Its crude decomposition into "stage-wise effects," i.e., the net average time gained by the treatment prior to each component event, does not reveal the patient state in which the extra time is spent. To obtain this information, we break each stage-wise effect into subcomponents according to the specific state to which the reference condition is improved. After re-expressing the subcomponents as functionals of the marginal survival functions of outcome events, we estimate them conveniently by plugging in the Kaplan -- Meier estimators. Their robust variance matrices allow us to construct joint tests on the decomposed units, which are particularly powerful against component-wise differential treatment effects. By reanalyzing a cancer trial and a cardiovascular trial, we acquire new insights into the quality and composition of the extra survival times, as well as the extra time with fewer hospitalizations, gained by the treatment in question. The proposed methods are implemented in the rmt package freely available on the Comprehensive R Archive Network (CRAN).

Estimands for clinical endpoints in Tuberculosis treatment randomized controlled trials: a retrospective application in a completed trial.

Background: Randomized trials for the treatment of tuberculosis (TB) rely on a composite primary outcome to capture unfavorable treatment responses. However, variability between trials in the outcome definition and estimation methods complicates across-trial comparisons and hinders the advancement of treatment guidelines. The International Council for Harmonization (ICH) provides international regulatory standards for clinical trials. The estimand framework outlined in the recent ICH E9(R1) addendum offers a timely opportunity for randomized trials of TB treatment to adopt broadly standardized outcome definitions and analytic approaches. We previously proposed and defined four estimands for use in this context. Our objective was to evaluate how the use of these estimands and choice of estimation method impacts results and interpretation of a large phase III TB trial. Methods: We reanalyzed participant level data from the REMoxTB trial. We applied four estimands and various methods of estimation to assess non-inferiority of both novel 4-month treatment regimens against standard of care. Results: With each of the four estimands we reached the same conclusion as the original trial analysis; that the novel regimens were not non-inferior to standard of care. Each estimand and method of estimation gave similar estimates of the treatment effect with fluctuations in variance and differences driven by the methods applied for handling intercurrent events. Conclusions: Our application of estimands defined by the ICH E9(R1) addendum offers a formalized framework for addressing the primary TB treatment trial objective and can promote uniformity in future trials by limiting heterogeneity in trial outcome definitions. We demonstrated the utility of our proposal using data from the REMoxTB randomized trial. We outlined methods for estimating each estimand and found consistent conclusions across estimands. We recommend future late-phase TB treatment trials to implement some or all of our estimands to promote rigorous outcome definitions and reduce variability between trials.Trial registration: NCT00864383.

Reflections on estimands for patient-reported outcomes in cancer clinical trials.

It is common and important to include the patient's perspective of the impact of treatment on health-related quality of life (HRQoL) outcomes. In this commentary, we focus on applying the new addendum to ICH E9 guideline E9 (R1) relating to the estimand framework to Patient Reported Outcomes (PROs) collected in cancer clinical trials, from a statistician's viewpoint. Currently, common practice for statistical analysis of PRO endpoints of published cancer clinical trials demonstrates ambiguity, leaving critical questions unspecified, hindering conclusions about the effect of treatment on PRO endpoints as well as comparability between clinical trials. To avoid this scenario, we advocate the systematic use of the estimand framework which requires the prospective definition of clear PRO research questions. Among the five attributes of the estimands framework, the definition of the endpoint (what is the right PRO measure and timeframe to target and why?), the intercurrent event identification and management (what happens with PRO data post-disease progression, what is the impact of death?) and the population-level summary (what is an acceptable statistical summary for PRO data?) require the most attention for PRO estimands. We identify good practice and highlight discussion points including the challenges of statistical analysis in the presence of missing and/or unobservable data and in relation to death. Through this discussion we highlight that there is no "statistical magic", but that the estimand framework will help you find out what you really want to know when quantifying the benefit of treatments from the patients' perspective.

A causal roadmap for generating high-quality real-world evidence.

Increasing emphasis on the use of real-world evidence (RWE) to support clinical policy and regulatory decision-making has led to a proliferation of guidance, advice, and frameworks from regulatory agencies, academia, professional societies, and industry. A broad spectrum of studies use real-world data (RWD) to produce RWE, ranging from randomized trials with outcomes assessed using RWD to fully observational studies. Yet, many proposals for generating RWE lack sufficient detail, and many analyses of RWD suffer from implausible assumptions, other methodological flaws, or inappropriate interpretations. The Causal Roadmap is an explicit, itemized, iterative process that guides investigators to prespecify study design and analysis plans; it addresses a wide range of guidance within a single framework. By supporting the transparent evaluation of causal assumptions and facilitating objective comparisons of design and analysis choices based on prespecified criteria, the Roadmap can help investigators to evaluate the quality of evidence that a given study is likely to produce, specify a study to generate high-quality RWE, and communicate effectively with regulatory agencies and other stakeholders. This paper aims to disseminate and extend the Causal Roadmap framework for use by clinical and translational researchers; three companion papers demonstrate applications of the Causal Roadmap for specific use cases.

The survival-incorporated median vs the median in the survivors or in the always-survivors: What are we measuring? and Why?

Many clinical studies evaluate the benefit of a treatment based on both survival and other continuous/ordinal clinical outcomes, such as quality of life scores. In these studies, when subjects die before the follow-up assessment, the clinical outcomes become undefined and are truncated by death. Treating outcomes as "missing" or "censored" due to death can be misleading for treatment effect evaluation. We show that if we use the median in the survivors or in the always-survivors as estimands to summarize clinical outcomes, we may conclude that a trade-off exists between the probability of survival and good clinical outcomes, even in settings where both the probability of survival and the probability of any good clinical outcome are better for one treatment. Therefore, we advocate not always treating death as a mechanism through which clinical outcomes are missing, but rather as part of the outcome measure. To account for the survival status, we describe the survival-incorporated median as an alternative summary measure for outcomes in the presence of death. The survival-incorporated median is the threshold such that 50% of the population is alive with an outcome above that threshold. Through conceptual examples and an application to a prostate cancer treatment study, we show that the survival-incorporated median provides a simple and useful summary measure to inform clinical practice.