Why estimands are needed to define treatment effects in clinical trials.

BACKGROUND: The estimand for a clinical trial is a precise definition of the treatment effect to be estimated. Traditionally, estimates of treatment effects are based on either an ITT analysis or a per-protocol analysis. However, there are important clinical questions which are not addressed by either of these analyses. For example, consider a trial where patients take a rescue medication. The ITT analysis includes data after use of rescue, while the per-protocol analysis excludes these patients altogether. Neither of these analyses addresses the important question of what the treatment effect would have been if patients did not take rescue medication. MAIN TEXT: Trial estimands provide a broader perspective compared to the limitations of ITT and per-protocol analysis. Trial treatment effects depend on how events occurring after treatment initiation such as use of alternative medication or discontinuation of the intervention are included in the definition. These events can be accounted for in different ways, depending on the clinical question of interest. CONCLUSION: The estimand framework is an important step forward in improving the clarity and transparency of clinical trials. The centrality of estimands to clinical trials is currently not reflected in methods recommended by the Cochrane group or the CONSORT statement, the current standard for reporting clinical trials in medical journals. We encourage revisions to these guidelines.

From DREAM to REALITI-A and beyond: Mepolizumab for the treatment of eosinophil-driven diseases.

Effective treatment of inflammatory diseases is often challenging owing to their heterogeneous pathophysiology. Understanding of the underlying disease mechanisms is improving and it is now clear that eosinophils play a complex pathophysiological role in a broad range of type 2 inflammatory diseases. Standard of care for these conditions often still includes oral corticosteroids (OCS) and/or cytotoxic immune therapies, which are associated with debilitating side effects. Selective, biological eosinophil-reducing agents provide treatment options that improve clinical symptoms associated with eosinophilic inflammation and reduce OCS use. Mepolizumab is a humanized monoclonal antibody that binds to and neutralizes interleukin-5, the major cytokine involved in eosinophil proliferation, activation, and survival. Mepolizumab is approved for the treatment of severe eosinophilic asthma, eosinophilic granulomatosis with polyangiitis and hypereosinophilic syndrome. Additionally, the efficacy of add-on mepolizumab has been observed in patients with severe chronic rhinosinusitis with nasal polyposis and chronic obstructive pulmonary disease with an eosinophilic phenotype. Here, we review the development, approval, and real-world effectiveness of mepolizumab for the treatment of patients with severe eosinophilic asthma, from the DREAM to REALITI-A studies, and describe how knowledge from this journey extended to the use of mepolizumab and other biologics across a broad spectrum of eosinophilic diseases.

Estimation of a treatment policy estimand for time to event data using data collected post discontinuation of randomised treatment.

Discontinuation from randomised treatment is a common intercurrent event in clinical trials. When the target estimand uses a treatment policy strategy to deal with this intercurrent event, data after cessation of treatment is relevant to estimate the estimand and all efforts should be made to collect such data. Missing data may nevertheless occur due to participants withdrawing from the study and assumptions regarding the values for data that are missing are required for estimation. A missing-at-random assumption is commonly made in this setting, but it may not always be viewed as appropriate. Another potential approach is to assume missing values are similar to data collected after treatment discontinuation. This idea has been previously proposed in the context of recurrent event data. Here we extend this approach to time-to-event outcomes using the hazard function. We propose imputation models that allow for different hazard rates before and after treatment discontinuation and use the posttreatment discontinuation hazard to impute events for participants with missing follow-up periods due to study withdrawal. The imputation models are fitted as Andersen-Gill models. We illustrate the proposed methods with an example of a clinical trial in patients with chronic obstructive pulmonary disease.

Assessing efficacy in important subgroups in confirmatory trials: An example using Bayesian dynamic borrowing.

Assessment of efficacy in important subgroups - such as those defined by sex, age, race and region - in confirmatory trials is typically performed using separate analysis of the specific subgroup. This ignores relevant information from the complementary subgroup. Bayesian dynamic borrowing uses an informative prior based on analysis of the complementary subgroup and a weak prior distribution centred on a mean of zero to construct a robust mixture prior. This combination of priors allows for dynamic borrowing of prior information; the analysis learns how much of the complementary subgroup prior information to borrow based on the consistency between the subgroup of interest and the complementary subgroup. A tipping point analysis can be carried out to identify how much prior weight needs to be placed on the complementary subgroup component of the robust mixture prior to establish efficacy in the subgroup of interest. An attractive feature of the tipping point analysis is that it enables the evidence from the source subgroup, the evidence from the target subgroup, and the combined evidence to be displayed alongside each other. This method is illustrated with an example trial in severe asthma where efficacy in the adolescent subgroup was assessed using a mixture prior combining an informative prior from the adult data in the same trial with a non-informative prior.

What matters most? Different stakeholder perspectives on estimands for an invented case study in COPD.

In drug development, we ask ourselves which population, endpoint and treatment comparison should be investigated. In this context, we also debate what matters most to the different stakeholders that are involved in clinical drug development, for example, patients, physicians, regulators and payers. With the publication of draft ICH E9 addendum on estimands in 2017, we now have a common framework and language to discuss such questions in an informed and transparent way. This has led to the estimand discussion being a key element in study development, including design, analysis and interpretation of a treatment effect. At an invited session at the 2018 PSI annual conference, PSI hosted a role-play debate where the aim of the session was to mimic a regulatory and payer scientific advice discussion for a COPD drug. Including role-play views from an industry sponsor, a patient, a regulator and a payer. This paper presents the invented COPD case-study design and considerations relating to appropriate estimands are discussed by each of the stakeholders from their differing viewpoints with the additional inclusion of a technical (academic) perspective. The rationale for each perspective on approaches for handling intercurrent events is presented, with a key emphasis on the application of while-on-treatment and treatment policy estimands in this context. It is increasingly recognised that the treatment effect estimated by the treatment policy approach may not always be of primary clinical interest and may not appropriately communicate to patients the efficacy they can expect if they take the treatment as directed.

Strategies for composite estimands in confirmatory clinical trials: Examples from trials in nasal polyps and steroid reduction.

The draft addendum to the ICH E9 regulatory guideline asks for explicit definition of the treatment effect to be estimated in clinical trials. The draft guideline also introduces the concept of intercurrent events to describe events that occur post-randomisation that may affect efficacy assessment. Composite estimands allow incorporation of intercurrent events in the definition of the endpoint. A common example of an intercurrent event is discontinuation of randomised treatment and use of a composite strategy would assess treatment effect based on a variable that combines the outcome variable of interest with discontinuation of randomised treatment. Use of a composite estimand may avoid the need for imputation which would be required by a treatment policy estimand. The draft guideline gives the example of a binary approach for specifying a composite estimand. When the variable is measured on a non-binary scale, other options are available where the intercurrent event is given an extreme unfavourable value, for example comparison of median values or analysis based on categories of response. This paper reviews approaches to deriving a composite estimand and contrasts the use of this estimand to the treatment policy estimand. The benefits of using each strategy are discussed and examples of the use of the different approaches are given for a clinical trial in nasal polyposis and a steroid reduction trial in severe asthma.

Treatment policy estimands for recurrent event data using data collected after cessation of randomised treatment.

In the past, many clinical trials have withdrawn subjects from the study when they prematurely stopped their randomised treatment and have therefore only collected 'on-treatment' data. Thus, analyses addressing a treatment policy estimand have been restricted to imputing missing data under assumptions drawn from these data only. Many confirmatory trials are now continuing to collect data from subjects in a study even after they have prematurely discontinued study treatment as this event is irrelevant for the purposes of a treatment policy estimand. However, despite efforts to keep subjects in a trial, some will still choose to withdraw. Recent publications for sensitivity analyses of recurrent event data have focused on the reference-based imputation methods commonly applied to continuous outcomes, where imputation for the missing data for one treatment arm is based on the observed outcomes in another arm. However, the existence of data from subjects who have prematurely discontinued treatment but remained in the study has now raised the opportunity to use this 'off-treatment' data to impute the missing data for subjects who withdraw, potentially allowing more plausible assumptions for the missing post-study-withdrawal data than reference-based approaches. In this paper, we introduce a new imputation method for recurrent event data in which the missing post-study-withdrawal event rate for a particular subject is assumed to reflect that observed from subjects during the off-treatment period. The method is illustrated in a trial in chronic obstructive pulmonary disease (COPD) where the primary endpoint was the rate of exacerbations, analysed using a negative binomial model.

Biomarkers for severe eosinophilic asthma.

The last decade has seen the approval of several new biologics for the treatment of severe asthma-targeting specific endotypes and phenotypes. This review will examine how evidence generated from the mepolizumab clinical development program showed that blood eosinophil counts, rather than sputum or tissue eosinophil counts, evolved as a pharmacodynamic and predictive biomarker for the efficacy of treatment with mepolizumab in patients with severe eosinophilic asthma. Based on the available evidence and combined with clinical judgement, a baseline blood eosinophil threshold of 150 cells/µL or greater or a historical blood eosinophil threshold of 300 cells/µL or greater will allow selection of patients with severe eosinophilic asthma who are most likely to achieve clinically significant reductions in the rate of exacerbations with mepolizumab treatment.

Meta-analysis of asthma-related hospitalization in mepolizumab studies of severe eosinophilic asthma.

BACKGROUND: Studies show that mepolizumab can reduce the frequency of clinically significant exacerbations in patients with severe eosinophilic asthma, compared with placebo. However, important events such as hospitalizations and emergency room visits are rare and difficult to characterize in single studies. OBJECTIVE: We sought to compare hospitalization or hospitalization and/or emergency room visit rates in patients with severe eosinophilic asthma treated with mepolizumab or placebo in addition to standard of care for at least 24 weeks. METHODS: This study was conducted and reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement. PubMed and the GSK Clinical Study Register were searched for suitable studies. The primary end points were the rate of exacerbations requiring hospitalization and the rate of exacerbations requiring hospitalization/emergency room visit. The proportion of patients with 1 or more event was also assessed. All mepolizumab doses were combined and individual patient-level data were analyzed. RESULTS: Four studies (n = 1388) were eligible for inclusion. Mepolizumab significantly reduced the rate of exacerbations requiring hospitalization (relative rate, 0.49; 95% CI, 0.30-0.80; P = .004) and hospitalization/emergency room visit (relative rate, 0.49; 95% CI, 0.33-0.73; P < .001) versus placebo. Significant reductions of 45% and 38% were also observed for the proportion of patients experiencing 1 or more hospitalization and hospitalization and/or emergency room visit, respectively. CONCLUSIONS: Mepolizumab approximately halved exacerbations requiring hospitalization and/or emergency room visits compared with placebo in patients with severe eosinophilic asthma. This treatment addresses a key outcome in a patient population with a high unmet need (GSK Study 204664).

Oral glucocorticoid-sparing effect of mepolizumab in eosinophilic asthma.

BACKGROUND: Many patients with severe asthma require regular treatment with oral glucocorticoids despite the use of high-dose inhaled therapy. However, the regular use of systemic glucocorticoids can result in serious and often irreversible adverse effects. Mepolizumab, a humanized monoclonal antibody that binds to and inactivates interleukin-5, has been shown to reduce asthma exacerbations in patients with severe eosinophilic asthma. METHODS: In a randomized, double-blind trial involving 135 patients with severe eosinophilic asthma, we compared the glucocorticoid-sparing effect of mepolizumab (at a dose of 100 mg) with that of placebo administered subcutaneously every 4 weeks for 20 weeks. The primary outcome was the degree of reduction in the glucocorticoid dose (90 to 100% reduction, 75 to less than 90% reduction, 50 to less than 75% reduction, more than 0 to less than 50% reduction, or no decrease in oral glucocorticoid dose, a lack of asthma control during weeks 20 to 24, or withdrawal from treatment). Other outcomes included the rate of asthma exacerbations, asthma control, and safety. RESULTS: The likelihood of a reduction in the glucocorticoid-dose stratum was 2.39 times greater in the mepolizumab group than in the placebo group (95% confidence interval, 1.25 to 4.56; P=0.008). The median percentage reduction from baseline in the glucocorticoid dose was 50% in the mepolizumab group, as compared with no reduction in the placebo group (P=0.007). Despite receiving a reduced glucocorticoid dose, patients in the mepolizumab group, as compared with those in the placebo group, had a relative reduction of 32% in the annualized rate of exacerbations (1.44 vs. 2.12, P=0.04) and a reduction of 0.52 points with respect to asthma symptoms (P=0.004), as measured on the Asthma Control Questionnaire 5 (in which the minimal clinically important difference is 0.5 points). The safety profile of mepolizumab was similar to that of placebo. CONCLUSIONS: In patients requiring daily oral glucocorticoid therapy to maintain asthma control, mepolizumab had a significant glucocorticoid-sparing effect, reduced exacerbations, and improved control of asthma symptoms. (Funded by GlaxoSmithKline; SIRIUS ClinicalTrials.gov number, NCT01691508.).

Mepolizumab treatment in patients with severe eosinophilic asthma.

BACKGROUND: Some patients with severe asthma have frequent exacerbations associated with persistent eosinophilic inflammation despite continuous treatment with high-dose inhaled glucocorticoids with or without oral glucocorticoids. METHODS: In this randomized, double-blind, double-dummy study, we assigned 576 patients with recurrent asthma exacerbations and evidence of eosinophilic inflammation despite high doses of inhaled glucocorticoids to one of three study groups. Patients were assigned to receive mepolizumab, a humanized monoclonal antibody against interleukin-5, which was administered as either a 75-mg intravenous dose or a 100-mg subcutaneous dose, or placebo every 4 weeks for 32 weeks. The primary outcome was the rate of exacerbations. Other outcomes included the forced expiratory volume in 1 second (FEV1) and scores on the St. George's Respiratory Questionnaire (SGRQ) and the 5-item Asthma Control Questionnaire (ACQ-5). Safety was also assessed. RESULTS: The rate of exacerbations was reduced by 47% (95% confidence interval [CI], 29 to 61) among patients receiving intravenous mepolizumab and by 53% (95% CI, 37 to 65) among those receiving subcutaneous mepolizumab, as compared with those receiving placebo (P<0.001 for both comparisons). Exacerbations necessitating an emergency department visit or hospitalization were reduced by 32% in the group receiving intravenous mepolizumab and by 61% in the group receiving subcutaneous mepolizumab. At week 32, the mean increase from baseline in FEV1 was 100 ml greater in patients receiving intravenous mepolizumab than in those receiving placebo (P=0.02) and 98 ml greater in patients receiving subcutaneous mepolizumab than in those receiving placebo (P=0.03). The improvement from baseline in the SGRQ score was 6.4 points and 7.0 points greater in the intravenous and subcutaneous mepolizumab groups, respectively, than in the placebo group (minimal clinically important change, 4 points), and the improvement in the ACQ-5 score was 0.42 points and 0.44 points greater in the two mepolizumab groups, respectively, than in the placebo group (minimal clinically important change, 0.5 points) (P<0.001 for all comparisons). The safety profile of mepolizumab was similar to that of placebo. CONCLUSIONS: Mepolizumab administered either intravenously or subcutaneously significantly reduced asthma exacerbations and was associated with improvements in markers of asthma control. (Funded by GlaxoSmithKline; MENSA ClinicalTrials.gov number, NCT01691521.).

Subgroups: time to go back to basic statistical principles?

Exploratory subgroup analyses are an increasing source of controversy as part of the interpretation of the results of clinical trials. In this article, we review the major challenges of multiplicity, statistical methods available to assess consistency of effect, and the part appropriate design plays in mitigating the risk of false conclusions from subgroup analyses. We discuss the problems associated with using definitions of consistency based on effect sizes in specific subgroups. We argue that what is required is a return to basic statistical principles, including more use of modeling techniques.

Missing data sensitivity analysis for recurrent event data using controlled imputation.

Statistical analyses of recurrent event data have typically been based on the missing at random assumption. One implication of this is that, if data are collected only when patients are on their randomized treatment, the resulting de jure estimator of treatment effect corresponds to the situation in which the patients adhere to this regime throughout the study. For confirmatory analysis of clinical trials, sensitivity analyses are required to investigate alternative de facto estimands that depart from this assumption. Recent publications have described the use of multiple imputation methods based on pattern mixture models for continuous outcomes, where imputation for the missing data for one treatment arm (e.g. the active arm) is based on the statistical behaviour of outcomes in another arm (e.g. the placebo arm). This has been referred to as controlled imputation or reference-based imputation. In this paper, we use the negative multinomial distribution to apply this approach to analyses of recurrent events and other similar outcomes. The methods are illustrated by a trial in severe asthma where the primary endpoint was rate of exacerbations and the primary analysis was based on the negative binomial model.

Mepolizumab for severe eosinophilic asthma (DREAM): a multicentre, double-blind, placebo-controlled trial.

BACKGROUND: Some patients with severe asthma have recurrent asthma exacerbations associated with eosinophilic airway inflammation. Early studies suggest that inhibition of eosinophilic airway inflammation with mepolizumab-a monoclonal antibody against interleukin 5-is associated with a reduced risk of exacerbations. We aimed to establish efficacy, safety, and patient characteristics associated with the response to mepolizumab. METHODS: We undertook a multicentre, double-blind, placebo-controlled trial at 81 centres in 13 countries between Nov 9, 2009, and Dec 5, 2011. Eligible patients were aged 12-74 years, had a history of recurrent severe asthma exacerbations, and had signs of eosinophilic inflammation. They were randomly assigned (in a 1:1:1:1 ratio) to receive one of three doses of intravenous mepolizumab (75 mg, 250 mg, or 750 mg) or matched placebo (100 mL 0·9% NaCl) with a central telephone-based system and computer-generated randomly permuted block schedule stratified by whether treatment with oral corticosteroids was required. Patients received 13 infusions at 4-week intervals. The primary outcome was the rate of clinically significant asthma exacerbations, which were defined as validated episodes of acute asthma requiring treatment with oral corticosteroids, admission, or a visit to an emergency department. Patients, clinicians, and data analysts were masked to treatment assignment. Analyses were by intention to treat. This trial is registered with ClinicalTrials.gov, number NCT01000506. FINDINGS: 621 patients were randomised: 159 were assigned to placebo, 154 to 75 mg mepolizumab, 152 to 250 mg mepolizumab, and 156 to 750 mg mepolizumab. 776 exacerbations were deemed to be clinically significant. The rate of clinically significant exacerbations was 2·40 per patient per year in the placebo group, 1·24 in the 75 mg mepolizumab group (48% reduction, 95% CI 31-61%; p<0·0001), 1·46 in the 250 mg mepolizumab group (39% reduction, 19-54%; p=0·0005), and 1·15 in the 750 mg mepolizumab group (52% reduction, 36-64%; p<0·0001). Three patients died during the study, but the deaths were not deemed to be related to treatment. INTERPRETATION: Mepolizumab is an effective and well tolerated treatment that reduces the risk of asthma exacerbations in patients with severe eosinophilic asthma. FUNDING: GlaxoSmithKline.

Intent-to-treat analysis in the presence of off-treatment or missing data.

Intent-to-treat (ITT) analysis is viewed as the analysis of a clinical trial that provides the least bias, but difficult issues can arise. Common analysis methods such as mixed-effects and proportional hazards models are usually labeled as ITT analysis, but in practice they can often be inconsistent with a strict interpretation of the ITT principle. In trials where effective medications are available to patients withdrawing from treatment, ITT analysis can mask important therapeutic effects of the intervention studied in the trial. Analysis of on-treatment data may be subject to bias, but can address efficacy objectives when combined with careful review of the pattern of withdrawals across treatments particularly for those patients withdrawing due to lack of efficacy and adverse events.

Statistical resource needs to be increased in the European regulatory agencies.

The following viewpoint from PSI and EFSPI regarding the current level of statistical resource in the European regulatory agencies was first presented as a position paper to a meeting of the EU Heads of Agencies in July 2009, and was endorsed by EFPIA.

Why ITT analysis is not always the answer for estimating treatment effects in clinical trials.

For many years there has been a consensus among the Clinical Research community that ITT analysis represents the correct approach for the vast majority of trials. Recent worldwide regulatory guidance for pharmaceutical industry trials has allowed discussion of alternatives to the ITT approach to analysis; different treatment effects can be considered which may be more clinically meaningful and more relevant to patients and prescribers. The key concept is of a trial "estimand", a precise description of the estimated treatment effect. The strategy chosen to account for patients who discontinue treatment or take alternative medications which are not part of the randomised treatment regimen are important determinants of this treatment effect. One strategy to account for these events is treatment policy, which corresponds to an ITT approach. Alternative equally valid strategies address what the treatment effect is if the patient actually takes the treatment or does not use specific alternative medication. There is no single right answer to which strategy is most appropriate, the solution depends on the key clinical question of interest. The estimands framework discussed in the new guidance has been particularly useful in the context of the current COVID-19 pandemic and has clarified what choices are available to account for the impact of COVID-19 on clinical trials. Specifically, an ITT approach addresses a treatment effect that may not be generalisable beyond the current pandemic.

Clinical Development of Mepolizumab for the Treatment of Severe Eosinophilic Asthma: On the Path to Personalized Medicine.

The development of mepolizumab, an anti-IL-5 monoclonal antibody for the treatment of severe eosinophilic asthma, is an example of a clinical development program that evolved over time based on sound, basic scientific principles. Initial clinical data on the effects of mepolizumab on lung function in a general asthmatic population were disappointing. However, it became clear that mepolizumab may be more effective against other clinical endpoints, particularly asthma exacerbations, in patients with more severe disease. Furthermore, a developing understanding of asthma disease pathobiology led to the identification of an appropriate target population and predictive biomarker for mepolizumab treatment: patients with severe eosinophilic asthma and blood eosinophil count. Mepolizumab use provides clinically meaningful benefits in this target population, fulfilling an unmet need. This Clinical Commentary Review describes the clinical development of mepolizumab and details of how this program informed the development of other biologic therapies in patients with severe asthma. This account highlights how a personalized approach toward treatment of patients with severe eosinophilic asthma, supported by a large body of scientific evidence, ultimately led to new and effective treatments and improved patient outcomes.

Disease burden and efficacy of mepolizumab in patients with severe asthma and blood eosinophil counts of ≥150-300 cells/µL.

BACKGROUND: In patients with severe eosinophilic asthma, treatment decisions can be determined by blood eosinophil counts; however, a specific blood eosinophil threshold has not been defined for starting mepolizumab treatment. METHODS: We summarized the disease burden and efficacy of mepolizumab in patients with severe eosinophilic asthma and baseline blood eosinophil counts of ≥150-300 cells/µL and ≥300 cells/µL using data from the mepolizumab clinical development program (DREAM [NCT01000506], MENSA [NCT01691521], SIRIUS [NCT01691508] and MUSCA [NCT02281318]). RESULTS: The morbidity of asthma in patients with baseline blood eosinophil counts ≥150-300 cells/µL was similar to that in patients with blood eosinophil counts ≥300 cells/µL, with similar rates of exacerbations (2.8-3.5 events/year versus 2.8-3.8 events/year, respectively), asthma related emergency room visits, intubations and near fatal events. Use of maintenance oral corticosteroids (OCS) was similar across blood eosinophil count subgroups. Reductions in the rates of clinically significant exacerbations with mepolizumab versus placebo ranged from 27 to 49% in patients with blood eosinophil counts of ≥150-300 cells/µL for DREAM, MENSA and MUSCA. The odds of achieving a reduction in OCS in SIRIUS was 2.03 (95% CI: 0.53, 7.75) versus 1.79 (95% CI: 0.71, 4.52) in patients with blood eosinophil counts ≥150-300 cells/µL and ≥300 cells/µL, respectively. CONCLUSION: There is a high unmet clinical need in patients with blood eosinophil counts ≥150-300 cells/µL, and a clinically meaningful benefit is seen with mepolizumab in this subgroup. Mepolizumab is an efficacious treatment option for patients with blood eosinophil counts ≥150 cells/µL.