Visualizing the target estimand in comparative effectiveness studies with multiple treatments.

Aim: Comparative effectiveness research using real-world data often involves pairwise propensity score matching to adjust for confounding bias. We show that corresponding treatment effect estimates may have limited external validity, and propose two visualization tools to clarify the target estimand. Materials & methods: We conduct a simulation study to demonstrate, with bivariate ellipses and joy plots, that differences in covariate distributions across treatment groups may affect the external validity of treatment effect estimates. We showcase how these visualization tools can facilitate the interpretation of target estimands in a case study comparing the effectiveness of teriflunomide (TERI), dimethyl fumarate (DMF) and natalizumab (NAT) on manual dexterity in patients with multiple sclerosis. Results: In the simulation study, estimates of the treatment effect greatly differed depending on the target population. For example, when comparing treatment B with C, the estimated treatment effect (and respective standard error) varied from -0.27 (0.03) to -0.37 (0.04) in the type of patients initially receiving treatment B and C, respectively. Visualization of the matched samples revealed that covariate distributions vary for each comparison and cannot be used to target one common treatment effect for the three treatment comparisons. In the case study, the bivariate distribution of age and disease duration varied across the population of patients receiving TERI, DMF or NAT. Although results suggest that DMF and NAT improve manual dexterity at 1 year compared with TERI, the effectiveness of DMF versus NAT differs depending on which target estimand is used. Conclusion: Visualization tools may help to clarify the target population in comparative effectiveness studies and resolve ambiguity about the interpretation of estimated treatment effects.

Comparing tocilizumab biosimilar BAT1806/BIIB800 with reference tocilizumab in patients with moderate-to-severe rheumatoid arthritis with an inadequate response to methotrexate: a phase 3, randomised, multicentre, double-blind, active-controlled clinical trial.

BACKGROUND: Biosimilars provide an opportunity to address unmet medical need by expanding access to biological treatments. This study aimed to show equivalent efficacy, and comparable safety, immunogenicity, and pharmacokinetic profiles of a proposed tocilizumab biosimilar BAT1806/BIIB800, to reference tocilizumab, in participants with rheumatoid arthritis with an inadequate response to methotrexate. METHODS: This phase 3, multicentre, randomised, double-blind, active-controlled, equivalence study comprised a 24-week initial treatment period (results reported here) and a 24-week secondary treatment period. Participants were recruited at 54 centres across five countries (China, Ukraine, Poland, Georgia, and Bulgaria). Patients with active rheumatoid arthritis with an inadequate response to methotrexate were randomly assigned (1:1:2) to receive reference tocilizumab up to week 48, or reference tocilizumab up to week 24 followed by BAT1806/BIIB800 up to week 48 (the two reference tocilizumab groups were analysed as a single group in this analysis), or BAT1806/BIIB800 up to week 48 (the BAT1806/BIIB800 group), administered by intravenous infusion once every 4 weeks at a starting dose of 8 mg/kg. The primary endpoint was the proportion of participants who had a 20% improvement in American College of Rheumatology criteria (ACR20) at week 12 (for the European Medicines Agency [EMA]) or week 24 (for the US Food and Drug Administration [FDA] and China National Medical Products Administration [NMPA]) using prespecified equivalence margins (95% CI -14·5 to +14·5 [EMA], 90% CI -12·0 to +15·0 [FDA], and 95% CI -13·6 to +13·6 [NMPA]). The International Council for Harmonisation E9(R1) estimand framework, with strategies for addressing intercurrent events, was implemented for the efficacy evaluations with expected differences as per the predefined equivalence margins. This trial is registered at ClinicalTrials.gov (NCT03830203) and EudraCT (2018-002202-31), and is closed to new participants. FINDINGS: Between Dec 19, 2018, and Jan 5, 2021, we randomly assigned 621 participants: 309 to the reference tocilizumab group and 312 to the BAT1806/BIIB800 group. The mean age was 50·5 years (SD 12·0), 534 (86%) were women, 87 (14%) were men, and 368 (59%) were White. For the primary estimands, estimated ACR20 response rates were 64·8% in the reference tocilizumab group and 69·0% in the BAT1806/BIIB800 group (treatment difference 4·1% [95% CI -3·6 to 11·9]) at week 12, and 67·9% in the reference tocilizumab group and 69·9% in the BAT1806/BIIB800 group (treatment difference 1·9% [90% CI -4·0 to 7·9; 95% CI -5·2 to 9·1]) at week 24. All confidence intervals were contained within the predefined equivalence margins. Comparable pharmacokinetic and immunogenicity profiles were observed for the reference tocilizumab and BAT1806/BIIB800 groups. Adverse events were reported by 201 (65%) participants in the reference tocilizumab group and 206 (66%) in the BAT1806/BIIB800 group; 196 (63%) participants in the reference tocilizumab group and 201 (64%) participants in the BAT1806/BIIB800 group reported a treatment-emergent adverse event. Five participants had a fatal event (reference tocilizumab n=1; BAT1806/BIIB800 n=4). INTERPRETATION: BAT1806/BIIB800 showed equivalent efficacy, and comparable safety, immunogenicity, and pharmacokinetic profiles as reference tocilizumab. FUNDING: Bio-Thera Solutions and Biogen.

Estimation of treatment effects in short-term depression studies. An evaluation based on the ICH E9(R1) estimands framework.

Estimands aim to incorporate intercurrent events in design, data collection and estimation of treatment effects in clinical trials. Our aim was to understand what estimands may correspond to efficacy analyses commonly employed in clinical trials conducted before publication of ICH E9(R1). We re-analysed six clinical trials evaluating a new anti-depression treatment. We selected the following analysis methods-ANCOVA on complete cases, following last observation carried forward (LOCF) imputation and following multiple imputation; mixed-models for repeated measurements without imputation (MMRM), MMRM following LOCF imputation and following jump-to-reference imputation; and pattern-mixture mixed models. We included a principal stratum analysis based on the predicted subset of the study population who would not discontinue due to adverse events or lack of efficacy. We translated each analysis into the implicitly targeted estimand, and formulated corresponding clinical questions. We could map six estimands to analysis methods. The same analysis method could be mapped to more than one estimand. The major difference between estimands was the strategy for intercurrent events, with other attributes mostly the same across mapped estimands. The quantitative differences in MADRS10 population-level summaries between the estimands were 4-8 points. Not all six estimands had a clinically meaningful interpretation. Only a few analyses would target the same estimand, hence only few could be used as sensitivity analyses. The fact that an analysis could estimate different estimands emphasises the importance of prospectively defining the estimands targeting the primary objective of a trial. The fact that an estimand can be targeted by different analyses emphasises the importance of prespecifying precisely the estimator for the targeted estimand.

Applicability and added value of novel methods to improve drug development in rare diseases.

BACKGROUND: The ASTERIX project developed a number of novel methods suited to study small populations. The objective of this exercise was to evaluate the applicability and added value of novel methods to improve drug development in small populations, using real world drug development programmes as reported in European Public Assessment Reports. METHODS: The applicability and added value of thirteen novel methods developed within ASTERIX were evaluated using data from 26 European Public Assessment Reports (EPARs) for orphan medicinal products, representative of rare medical conditions as predefined through six clusters. The novel methods included were 'innovative trial designs' (six methods), 'level of evidence' (one method), 'study endpoints and statistical analysis' (four methods), and 'meta-analysis' (two methods) and they were selected from the methods developed within ASTERIX based on their novelty; methods that discussed already available and applied strategies were not included for the purpose of this validation exercise. Pre-requisites for application in a study were systematized for each method, and for each main study in the selected EPARs it was assessed if all pre-requisites were met. This direct applicability using the actual study design was firstly assessed. Secondary, applicability and added value were explored allowing changes to study objectives and design, but without deviating from the context of the drug development plan. We evaluated whether differences in applicability and added value could be observed between the six predefined condition clusters. RESULTS AND DISCUSSION: Direct applicability of novel methods appeared to be limited to specific selected cases. The applicability and added value of novel methods increased substantially when changes to the study setting within the context of drug development were allowed. In this setting, novel methods for extrapolation, sample size re-assessment, multi-armed trials, optimal sequential design for small sample sizes, Bayesian sample size re-estimation, dynamic borrowing through power priors and fall-back tests for co-primary endpoints showed most promise - applicable in more than 40% of evaluated EPARs in all clusters. Most of the novel methods were applicable to conditions in the cluster of chronic and progressive conditions, involving multiple systems/organs. Relatively fewer methods were applicable to acute conditions with single episodes. For the chronic clusters, Goal Attainment Scaling was found to be particularly applicable as opposed to other (non-chronic) clusters. CONCLUSION: Novel methods as developed in ASTERIX can improve drug development programs. Achieving optimal added value of these novel methods often requires consideration of the entire drug development program, rather than reconsideration of methods for a specific trial. The novel methods tested were mostly applicable in chronic conditions, and acute conditions with recurrent episodes.

Measurement error is often neglected in medical literature: a systematic review.

OBJECTIVES: In medical research, covariates (e.g., exposure and confounder variables) are often measured with error. While it is well accepted that this introduces bias and imprecision in exposure-outcome relations, it is unclear to what extent such issues are currently considered in research practice. The objective was to study common practices regarding covariate measurement error via a systematic review of general medicine and epidemiology literature. STUDY DESIGN AND SETTING: Original research published in 2016 in 12 high impact journals was full-text searched for phrases relating to measurement error. Reporting of measurement error and methods to investigate or correct for it were quantified and characterized. RESULTS: Two hundred and forty-seven (44%) of the 565 original research publications reported on the presence of measurement error. 83% of these 247 did so with respect to the exposure and/or confounder variables. Only 18 publications (7% of 247) used methods to investigate or correct for measurement error. CONCLUSIONS: Consequently, it is difficult for readers to judge the robustness of presented results to the existence of measurement error in the majority of publications in high impact journals. Our systematic review highlights the need for increased awareness about the possible impact of covariate measurement error. Additionally, guidance on the use of measurement error correction methods is necessary.