A safety estimand for late phase clinical trials where the analysis period varies over the subjects.

BACKGROUND/AIMS: Evaluating safety is as important as evaluating efficacy in a clinical trial, yet the tradition for safety analysis is rudimentary. This article explores more complex methodologies for safety evaluation, with the aim of improving the interpretability, as well as generalizability, of the results. METHODS: For studies where the analysis periods vary over the subjects, using the International Council for Harmonisation estimand framework, we construct a formal estimand that could be used in the setting of safety surveillance that answers the clinical question of 'What is the magnitude of the increase in risk of experiencing an adverse event if the treatment is taken, as prescribed, for a specific period of time?'. Estimation methodologies for this estimand are also discussed. RESULTS: The proposed estimand is similar to that found in the efficacy analyses of time to event data (e.g. in outcome studies), with the key difference of utilization of hypothetical intercurrent event strategy for the intercurrent event of treatment discontinuation. This is motivated by what we perceive to be a key difference for the safety objective compared to efficacy objectives, namely a desire for sensitivity (i.e. greater possibility of detecting a negative impact of the drug, if such exists) as opposed to the need to prove a positive effect of the drug in a conservative manner. CONCLUSION: It is valuable, and possible, to use the International Council for Harmonisation estimand framework not only for efficacy but also for safety evaluation, with the estimand driven by an interpretable, and relevant, clinical question.

Current developments of the estimand concept.

Since the introduction of the estimand in therapeutical studies, several adaptions have been developed. This short article highlights the important aspects of the estimand concept. A literature research was conducted to identify different extensions to this framework. Different modified strategies for intercurrent events are presented, as well as examples of methods to implement the estimand in clinical studies. The article reflects that the estimand is an ongoing research field with further exploration.

Application of hypothetical strategies in acute pain.

Since the publication of ICH E9 (R1), "Addendum to statistical principles for clinical trials: on choosing appropriate estimands and defining sensitivity analyses in clinical trials," there has been a lot of debate about the hypothetical strategy for handling intercurrent events. Arguments against the hypothetical strategy are twofold: (1) the clinical question has limited clinical/regulatory interest; (2) the estimation may need strong statistical assumptions. In this article, we provide an example of a hypothetical strategy handling use of rescue medications in the acute pain setting. We argue that the treatment effect of a drug that is attributable to the treatment alone is the clinical question of interest and is important to regulators. The hypothetical strategy is important when developing non-opioid treatment as it estimates the treatment effect due to treatment during the pre-specified evaluation period whereas the treatment policy strategy does not. Two widely acceptable and non-controversial clinical inputs are required to construct a reasonable estimator. More importantly, this estimator does not rely on additional strong statistical assumptions and is considered reasonable for regulatory decision making. In this article, we point out examples where estimators for a hypothetical strategy can be constructed without any strong additional statistical assumptions besides acceptable clinical inputs. We also showcase a new way to obtain estimation based on disease specific clinical knowledge instead of strong statistical assumptions. In the example presented, we clearly demonstrate the advantages of the hypothetical strategy compared to alternative strategies including the treatment policy strategy and a composite variable strategy.

Why estimands are needed to define treatment effects in clinical trials.

BACKGROUND: The estimand for a clinical trial is a precise definition of the treatment effect to be estimated. Traditionally, estimates of treatment effects are based on either an ITT analysis or a per-protocol analysis. However, there are important clinical questions which are not addressed by either of these analyses. For example, consider a trial where patients take a rescue medication. The ITT analysis includes data after use of rescue, while the per-protocol analysis excludes these patients altogether. Neither of these analyses addresses the important question of what the treatment effect would have been if patients did not take rescue medication. MAIN TEXT: Trial estimands provide a broader perspective compared to the limitations of ITT and per-protocol analysis. Trial treatment effects depend on how events occurring after treatment initiation such as use of alternative medication or discontinuation of the intervention are included in the definition. These events can be accounted for in different ways, depending on the clinical question of interest. CONCLUSION: The estimand framework is an important step forward in improving the clarity and transparency of clinical trials. The centrality of estimands to clinical trials is currently not reflected in methods recommended by the Cochrane group or the CONSORT statement, the current standard for reporting clinical trials in medical journals. We encourage revisions to these guidelines.

G-estimation of structural nested mean models for interval-censored data using pseudo-observations.

Two large-scale randomized clinical trials compared fenofibrate and placebo in diabetic patients with pre-existing retinopathy (FIELD study) or risk factors (ACCORD trial) on an intention-to-treat basis and reported a significant reduction in the progression of diabetic retinopathy in the fenofibrate arms. However, their analyses involved complications due to intercurrent events, that is, treatment-switching and interval-censoring. This article addresses these problems involved in estimation of causal effects of long-term use of fibrates in a cohort study that followed patients with type 2 diabetes for 8 years. We propose structural nested mean models (SNMMs) of time-varying treatment effects and pseudo-observation estimators for interval-censored data. The first estimator for SNMMs uses a nonparametric maximum likelihood estimator (MLE) as a pseudo-observation, while the second estimator is based on MLE under a parametric piecewise exponential distribution. Through numerical studies with real and simulated datasets, the pseudo-observations estimators of causal effects using the nonparametric Wellner-Zhan estimator perform well even under dependent interval-censoring. Its application to the diabetes study revealed that the use of fibrates in the first 4 years reduced the risk of diabetic retinopathy but did not support its efficacy beyond 4 years.

Using modified intention-to-treat as a principal stratum estimator for failure to initiate treatment.

BACKGROUND: A common intercurrent event affecting many trials is when some participants do not begin their assigned treatment. For example, in a double-blind drug trial, some participants may not receive any dose of study medication. Many trials use a 'modified intention-to-treat' approach, whereby participants who do not initiate treatment are excluded from the analysis. However, it is not clear (a) the estimand being targeted by such an approach and (b) the assumptions necessary for such an approach to be unbiased. METHODS: Using potential outcome notation, we demonstrate that a modified intention-to-treat analysis which excludes participants who do not begin treatment is estimating a principal stratum estimand (i.e. the treatment effect in the subpopulation of participants who would begin treatment, regardless of which arm they were assigned to). The modified intention-to-treat estimator is unbiased for the principal stratum estimand under the assumption that the intercurrent event is not affected by the assigned treatment arm, that is, participants who initiate treatment in one arm would also do so in the other arm (i.e. if someone began the intervention, they would also have begun the control, and vice versa). RESULTS: We identify two key criteria in determining whether the modified intention-to-treat estimator is likely to be unbiased: first, we must be able to measure the participants in each treatment arm who experience the intercurrent event, and second, the assumption that treatment allocation will not affect whether the participant begins treatment must be reasonable. Most double-blind trials will satisfy these criteria, as the decision to start treatment cannot be influenced by the allocation, and we provide an example of an open-label trial where these criteria are likely to be satisfied as well, implying that a modified intention-to-treat analysis which excludes participants who do not begin treatment is an unbiased estimator for the principal stratum effect in these settings. We also give two examples where these criteria will not be satisfied (one comparing an active intervention vs usual care, where we cannot identify which usual care participants would have initiated the active intervention, and another comparing two active interventions in an unblinded manner, where knowledge of the assigned treatment arm may affect the participant's choice to begin or not), implying that a modified intention-to-treat estimator will be biased in these settings. CONCLUSION: A modified intention-to-treat analysis which excludes participants who do not begin treatment can be an unbiased estimator for the principal stratum estimand. Our framework can help identify when the assumptions for unbiasedness are likely to hold, and thus whether modified intention-to-treat is appropriate or not.

Estimands, Handling of Missing Data and Impact on Assumed Effect Size and Power in Pivotal COVID-19 Treatment Trials.

Estimands play an important role for aligning study objectives, study design and analyses through a precise definition of the quantity of interest. For COVID-19 studies, apart from intercurrent events, high volume of missing data has been observed. We explore their impact on several estimands through a synthetic COVID-19 data generated from a discrete-time multi-state model. We compare estimators of these estimands based on their ability to closely match the true response rates and retain assumed power. The final choice of the estimand then needs to be aligned with clinically meaningful quantities of interest to patients, clinicians, regulators and payers.

Estimation of treatment effects in short-term depression studies. An evaluation based on the ICH E9(R1) estimands framework.

Estimands aim to incorporate intercurrent events in design, data collection and estimation of treatment effects in clinical trials. Our aim was to understand what estimands may correspond to efficacy analyses commonly employed in clinical trials conducted before publication of ICH E9(R1). We re-analysed six clinical trials evaluating a new anti-depression treatment. We selected the following analysis methods-ANCOVA on complete cases, following last observation carried forward (LOCF) imputation and following multiple imputation; mixed-models for repeated measurements without imputation (MMRM), MMRM following LOCF imputation and following jump-to-reference imputation; and pattern-mixture mixed models. We included a principal stratum analysis based on the predicted subset of the study population who would not discontinue due to adverse events or lack of efficacy. We translated each analysis into the implicitly targeted estimand, and formulated corresponding clinical questions. We could map six estimands to analysis methods. The same analysis method could be mapped to more than one estimand. The major difference between estimands was the strategy for intercurrent events, with other attributes mostly the same across mapped estimands. The quantitative differences in MADRS10 population-level summaries between the estimands were 4-8 points. Not all six estimands had a clinically meaningful interpretation. Only a few analyses would target the same estimand, hence only few could be used as sensitivity analyses. The fact that an analysis could estimate different estimands emphasises the importance of prospectively defining the estimands targeting the primary objective of a trial. The fact that an estimand can be targeted by different analyses emphasises the importance of prespecifying precisely the estimator for the targeted estimand.

Principal stratum strategy: Potential role in drug development.

A randomized trial allows estimation of the causal effect of an intervention compared to a control in the overall population and in subpopulations defined by baseline characteristics. Often, however, clinical questions also arise regarding the treatment effect in subpopulations of patients, which would experience clinical or disease related events post-randomization. Events that occur after treatment initiation and potentially affect the interpretation or the existence of the measurements are called intercurrent events in the ICH E9(R1) guideline. If the intercurrent event is a consequence of treatment, randomization alone is no longer sufficient to meaningfully estimate the treatment effect. Analyses comparing the subgroups of patients without the intercurrent events for intervention and control will not estimate a causal effect. This is well known, but post-hoc analyses of this kind are commonly performed in drug development. An alternative approach is the principal stratum strategy, which classifies subjects according to their potential occurrence of an intercurrent event on both study arms. We illustrate with examples that questions formulated through principal strata occur naturally in drug development and argue that approaching these questions with the ICH E9(R1) estimand framework has the potential to lead to more transparent assumptions as well as more adequate analyses and conclusions. In addition, we provide an overview of assumptions required for estimation of effects in principal strata. Most of these assumptions are unverifiable and should hence be based on solid scientific understanding. Sensitivity analyses are needed to assess robustness of conclusions.

A note on the draft International Council for Harmonisation guidance on estimands and sensitivity analysis.

In the second half of 2014, the Steering Committee of the International Council for Harmonisation endorsed the formation of an expert working group to develop an addendum to the International Council for Harmonisation E9 guideline (Statistical Principles for Clinical Trials). The addendum was to focus on two clinical trial topics: estimands and sensitivity analysis. A draft of the addendum, referred to as E9/R1, was developed by the expert working group and made available for public comments across the International Council for Harmonisation regions in the second half of 2017. A structured framework for clinical trial design and analysis proposed in the draft addendum are briefly described, including four key inputs for developing objective-driven estimands and strategies for tackling one of the inputs ('intercurrent events'). The proposed framework aligns each clinical trial objective with the corresponding statistical target of estimation (estimand), trial design and data to be collected, main method of estimation/inference, and sensitivity analysis to pressure test key analytic assumption(s) in the main analysis. A case study from the diabetes therapeutic area illustrates how the framework can be implemented in practice. International Council for Harmonisation E9/R1 is expected to enable better planning, conduct, analysis, and interpretation of randomised clinical trials. This will facilitate improvements in new drug applications and strengthen understanding of decision making by regulatory authorities and advisory committees.

Estimands in CNS trials - A review of strategies for addressing intercurrent events.

Background: The estimands framework represents a significant innovation for the design, conduct, analysis, and interpretation of clinical trials. An aim of the framework is to increase precision and transparency on the handling of intercurrent events (IEs), defined as events occurring after treatment initiation and affecting the endpoint. While the experience in constructing and reporting estimands in the published literature is limited, developers performing confirmatory studies are already making use of the new paradigm, allowing to survey the strategies proposed by applicants and endorsed by regulators. Methods: To identify strategies for handling IEs in confirmatory central nervous system (CNS) trials, we searched scientific advice letters issued by the European Medicines Agency (EMA) between 2017 and 2022. We developed a categorisation of the IEs and classified, according to the strategies defined in the framework, the strategies proposed by the Applicants and recommended by the agency. Strategies proposed and recommended were summarised by category of IEs, and the rationale for the choices was analysed qualitatively. Results: In total, 170 IEs were identified in 52 confirmatory trials. A clear preference for the treatment policy strategy for treatment discontinuation and for the hypothetical strategy for pandemic-related disruptions was identified. For other categories of IEs, there are more mixed patterns. Discussion: This study highlights the multidimensional nature of choosing a strategy for an IE. For different occurring IEs in confirmatory CNS trials different strategies are of regulatory interest, depending on the trial objective, underlying disease properties, rarity of disease, as well as frequency and timing of IEs and their relatedness to the disease.

Estimand for non-inferiority influenza vaccine immunogenicity trials.

According to recent regulatory guidance, clinical trial objectives should be translated into estimands, i.e., precise descriptions of that what is to be estimated. Hence, estimands are to be formulated for influenza vaccine immunogenicity (IVI) trials, notably for one of the most popular immunogenicity trials, the non-inferiority trial. In this paper an estimand for this trial design is proposed. An estimand should state how intercurrent events are handled. Intercurrent events are events that occur after the start of the trial and that affect the endpoint's measurement or interpretation or prevent its observation. In IVI trials the intercurrent events of interest are immunological intercurrent events (IIEs). Major IIEs are identified, i.e., protocol deviations occurring during the trial that affect immunogenicity endpoints, and the consequences for the trial data collection are discussed. In the statistical analysis endpoint values that are missing or excluded from the analysis due to IIEs are to be substituted by plausible values, by means of multiple imputing. Replacing values are based on predictors of the endpoint. A distinction is made between mandatory and non-mandatory predictors. Mandatory predictors are predictors necessary to prevent biased prediction. Non-mandatory predictors are predictors that limit the additional variance due to the imputing. The four steps of the multiple imputing are explained, and available software is listed.

A hybrid return to baseline imputation method to incorporate MAR and MNAR dropout missingness.

Missing data are inevitable in longitudinal clinical trials due to intercurrent events (ICEs) such as treatment interruption or premature discontinuation for different reasons. Missing at random (MAR) assumption is usually unverifiable and sensitivity analyses are often requested under missing not at random (MNAR) assumption. Return to baseline (RTB) imputation is a commonly used MNAR method. In practice, not all dropout missingness can be assumed MNAR. For example, missingness or dropouts due to COVID-19 can be reasonably assumed MAR. Therefore, traditional RTB is not applicable when there is both MAR and MNAR dropout missingness. Here we propose a hybrid strategy for RTB imputation which can handle missing data due to MAR and MNAR dropouts at the same time. Standard multiple imputation approach is proposed and an analytic likelihood based approach is derived to improve efficiency.

Handling Missing Data and Drop Out in Hospice/Palliative Care Trials Through the Estimand Framework.

CONTEXT: Missing data are common in hospice/palliative care randomized trials due to high drop-out because of the demographic of interest. It can introduce bias in the estimate of the treatment effect and its precision. OBJECTIVES: The International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) released updated guidance on statistical principles for clinical trials introducing the estimand framework to align trial objectives, trial conduct, statistical analysis and interpretation of results. Our objective is to present how the estimand framework can be used to guide the handling of missing data in palliative care trials. METHODS: We outline the estimand framework by highlighting the five elements of an estimand: treatment, population, variable, summary measure and intercurrent event handling. We list common intercurrent events in palliative care trials and present the five strategies for handling intercurrent events outlined in the ICH guidance. RESULTS: We describe common intercurrent events in palliative care trials and discuss and justify what analytic strategies could be followed with each. We provide an example using a palliative care trial comparing two opioids for pain relieve in participants with cancer pain. CONCLUSION: When planning a palliative care trial, the estimand should be explicitly stated, including how intercurrent events will be handled in the analysis. This should be informed by the scientific objectives of the trial. The estimand guides the handling of missing data during the conduct and analysis of the trial. Defining an estimand is not a statistical activity, but a multi-disciplinary process involving all stakeholders.

Modified reference based imputation and tipping point analysis in the presence of missing data due to COVID-19.

In longitudinal clinical trials, missing data are inevitable due to intercurrent events (ICEs) such as treatment interruption or premature discontinuation for different reasons. The COVID-19 pandemic has had substantial impact on clinical trials since early 2020 as it may result in missing data due to missed visits and premature discontinuations. The missing data due to COVID-19 can reasonably be assumed as missing at random (MAR). We propose a combined hypothetical strategy for sensitivity analyses to handle missing data due to both COVID-19 and non-COVID reasons. We modify the commonly used missing not at random (MNAR) methods, reference based imputation (RBI) and tipping point analysis, under this strategy. We propose the standard multiple imputation approach and derive an analytic likelihood based approach to implement the proposed methods to improve efficiency in applications. The proposed strategy and methods are applicable to a more general scenario when there are missing data due to both MAR and MNAR reasons.

Adjusted win ratio with stratification: Calculation methods and interpretation.

The win ratio is a general method of comparing locations of distributions of two independent, ordinal random variables, and it can be estimated without distributional assumptions. In this paper we provide a unified theory of win ratio estimation in the presence of stratification and adjustment by a numeric variable. Building step by step on the estimate of the crude win ratio we compare corresponding tests with well known non-parametric tests of group difference (Wilcoxon rank-sum test, Fligner-Policello test, van Elteren test, test based on the regression on ranks, and the rank analysis of covariance test). We show that the win ratio gives an interpretable treatment effect measure with corresponding test to detect treatment effect difference under minimal assumptions.

Challenges in Assessing the Impact of the COVID-19 Pandemic on the Integrity and Interpretability of Clinical Trials.

Abstract-The COVID-19 pandemic has a global impact on the conduct of clinical trials of medical products. This article discusses implications of the COVID-19 pandemic on clinical research methodology aspects and provides points to consider to assess and mitigate the risk of seriously compromising the integrity and interpretability of clinical trials. The information in this article will support discussions that need to occur cross-functionally on an ongoing basis to "integrate all available knowledge from the ethical, the medical, and the methodological perspective into decision making." This article aims at facilitating: (i) risk assessments of the impact of the pandemic on trial integrity and interpretability; (ii) identification of the relevant data and information related to the impact of the pandemic on the trial that needs to be collected; (iii) short-term decision making impacting ongoing trial operations; (iv) ongoing monitoring of the trial conduct until completion, including the possible involvement of data monitoring committees, and adequately documenting all measures taken to secure trial integrity throughout and after the pandemic, and (v) proper analysis and interpretation of the eventual interim or final trial data.

A narrative review of estimands in drug development and regulatory evaluation: old wine in new barrels?

BACKGROUND: An estimand defines the target of estimation for a clinical trial through specification of the treatment, target population, variable, population-level summary and of the strategies for intercurrent events. A carefully defined estimand aligns the clinical trial design and analysis with the scientific question of interest and adequately accounts for so-called intercurrent events. The ICH E9(R1) addendum suggests five estimand strategies. We evaluated to what extent current practice in drug development and regulatory assessment fits in the estimand framework. METHODS: We systematically evaluated what estimands, especially what strategies for intercurrent events are advised in European Medicines Agency disease guidelines, used in sponsors' trials and additionally requested by the European Medicines Agency in assessment dossiers. We selected four therapeutic areas: nervous system, oncology, cardiovascular diseases and respiratory diseases. For each, we evaluated all disease guidelines with approved drugs, the dossiers of the most recently approved drugs matching the guidelines and corresponding regulatory questions. RESULTS: Strategies for intercurrent events were present in 18 (53%) of 34 guidelines, in all 34 sponsor documentations and in 15 (44%) of 34 sets of regulatory questions. Treatment policy was advised in 13 (38%) guidelines and was applied in 9 corresponding sponsor documentations. Of these 9, it was the sole strategy in 4 cases and accompanied by another strategy in 5 cases. Hypothetical strategy was not advised in guidelines. However, it was the leading strategy applied in 25 (74%) sponsor documentations. Composite strategy was advised in 3 (9%) guidelines and applied accompanied by another strategy in 2 corresponding sponsor documentations. While on treatment strategy was not advised in guidelines, but was applied in 2 sponsor documentations. Principal stratum strategy was advised in 2 guidelines but not applied in corresponding sponsor documentations. Of the regulatory questions, treatment policy was present in 2 cases (6%), hypothetical in 6 cases (18%), composite in 6 cases (18%) and while on treatment in 1 case (3%). CONCLUSIONS: Estimand attributes are present in guidelines, sponsor documentations and regulatory questions, but not described as estimands. Treatment policy was most often advised in guidelines, but hypothetical was the leading strategy applied in sponsor documentations. Thus, results indicate not a full concordance between the regulatory target of estimation and what is actually estimated. The lack of concordance was mostly due to limitations in collection of intercurrent events data to enable a treatment policy strategy. There is, therefore, a need to better define estimands at the design stage and throughout the applications dossiers and assessment reports.