Sensitivity analysis for principal ignorability violation in estimating complier and noncomplier average causal effects.

An important strategy for identifying principal causal effects (popular estimands in settings with noncompliance) is to invoke the principal ignorability (PI) assumption. As PI is untestable, it is important to gauge how sensitive effect estimates are to its violation. We focus on this task for the common one-sided noncompliance setting where there are two principal strata, compliers and noncompliers. Under PI, compliers and noncompliers share the same outcome-mean-given-covariates function under the control condition. For sensitivity analysis, we allow this function to differ between compliers and noncompliers in several ways, indexed by an odds ratio, a generalized odds ratio, a mean ratio, or a standardized mean difference sensitivity parameter. We tailor sensitivity analysis techniques (with any sensitivity parameter choice) to several types of PI-based main analysis methods, including outcome regression, influence function (IF) based and weighting methods. We discuss range selection for the sensitivity parameter. We illustrate the sensitivity analyses with several outcome types from the JOBS II study. This application estimates nuisance functions parametrically - for simplicity and accessibility. In addition, we establish rate conditions on nonparametric nuisance estimation for IF-based estimators to be asymptotically normal - with a view to inform nonparametric inference.

Estimating the Complier Average Causal Effect in a Meta-Analysis of Randomized Clinical Trials With Binary Outcomes Accounting for Noncompliance: A Generalized Linear Latent and Mixed Model Approach.

Noncompliance, a common problem in randomized clinical trials (RCTs), can bias estimation of the effect of treatment receipt using a standard intention-to-treat analysis. The complier average causal effect (CACE) measures the effect of an intervention in the latent subpopulation that would comply with their assigned treatment. Although several methods have been developed to estimate the CACE in analyzing a single RCT, methods for estimating the CACE in a meta-analysis of RCTs with noncompliance await further development. This article reviews the assumptions needed to estimate the CACE in a single RCT and proposes a frequentist alternative for estimating the CACE in a meta-analysis, using a generalized linear latent and mixed model with SAS software (SAS Institute, Inc.). The method accounts for between-study heterogeneity using random effects. We implement the methods and describe an illustrative example of a meta-analysis of 10 RCTs evaluating the effect of receiving epidural analgesia in labor on cesarean delivery, where noncompliance varies dramatically between studies. Simulation studies are used to evaluate the performance of the proposed method.

Bayesian phase II clinical trial design with noncompliance.

Noncompliance issue is common in early phase clinical trials; and may lead to biased estimation of the intent-to-treat effect and incorrect conclusions for the clinical trial. In this work, we propose a Bayesian approach for sequentially monitoring the phase II randomized clinical trials that takes account for the noncompliance information. We adopt the principal stratification framework and propose to use Bayesian additive regression trees for selecting useful baseline covariates and estimating the complier average causal effect (CACE) for both efficacy and toxicity outcomes. The decision of early termination or not is then made adaptively based on the estimated CACE from the accumulated data. Simulation studies have confirmed the excellent performance of the proposed design in the presence of noncompliance.

Weighted estimators of the complier average causal effect on restricted mean survival time with observed instrument-outcome confounders.

A major concern in any observational study is unmeasured confounding of the relationship between a treatment and outcome of interest. Instrumental variable (IV) analysis methods are able to control for unmeasured confounding. However, IV analysis methods developed for censored time-to-event data tend to rely on assumptions that may not be reasonable in many practical applications, making them unsuitable for use in observational studies. In this report, we develop weighted estimators of the complier average causal effect (CACE) on the restricted mean survival time in the overall population as well as in an evenly matchable population (CACE-m). Our method is able to accommodate instrument-outcome confounding and adjust for covariate-dependent censoring, making it particularly suited for causal inference from observational studies. We establish the asymptotic properties and derive easily implementable asymptotic variance estimators for the proposed estimators. Through simulation studies, we show that the proposed estimators tend to be more efficient than instrument propensity score matching-based estimators or IPIW estimators. We apply our method to compare dialytic modality-specific survival for end stage renal disease patients using data from the U.S. Renal Data System.

Latent class instrumental variables and the monotonicity assumption.

A key aspect of the article by Lousdal on instrumental variables was a discussion of the monotonicity assumption. However, there was no mention of the history of the development of this assumption. The purpose of this letter is to note that Baker and Lindeman and Imbens and Angrist independently introduced the monotonicity assumption into the analysis of instrumental variables. The letter also places the monotonicity assumption in the context of the method of latent class instrumental variables.

Game On-Complier Average Causal Effect Estimation Reveals Sleeper Effects on Academic Attainment in a Randomized Trial of the Good Behavior Game.

To examine the efficacy of the Good Behavior Game (GBG) in improving children's reading attainment, and the extent to which this varies as a function of cumulative intervention intensity (dosage) and timing of outcome measurement. A 2-year cluster-randomized controlled trial was conducted. Seventy-seven primary schools from three regions in England were randomly assigned to intervention and control groups. Children (N = 3084) aged 67 at baseline were the target cohort. The GBG is an interdependent group-contingency behavior management strategy used by teachers in elementary schools. Reading attainment was assessed via national teacher assessment scores at baseline, and the Hodder Group Reading Test at post-test and 1-year post-intervention follow-up. Dosage was assessed using a bespoke online GBG scoreboard system. Multi-level intent-to-treat (ITT) and complier average causal effect (CACE) estimation were utilized. At post-test, no effects of the GBG on children's reading attainment were found in either the ITT or CACE models. At 1-year follow-up, results remained null in the ITT model, but a significant intervention effect was found among moderate compliers (Δ = 0.10) in the CACE model. The GBG can produce measurable improvements in children's academic attainment, but these effects may take time to become apparent and are contingent upon implementation dosage falling within an optimal range. The project was supported by funding from the Education Endowment Foundation and the National Institute for Health Research. ISRCTN: 64152096.

Simple Estimation of Patient-Oriented Effects From Randomized Trials: An Open and Shut CACE.

In randomized controlled trials, the intention-to-treat estimator provides an unbiased estimate of the causal effect of treatment assignment on the outcome. However, patients often want to know what the effect would be if they were to take the treatment as prescribed (the patient-oriented effect), and several researchers have suggested that the more relevant causal effect for this question is the complier average causal effect (CACE), also referred to as the local average treatment effect. Sophisticated approaches to estimating the CACE include Bayesian and frequentist methods for principal stratification, inverse-probability-of-treatment-weighted estimators, and instrumental-variable (IV) analysis. All of these approaches exploit information about adherence to assigned treatment to improve upon the intention-to-treat estimator, but they are rarely used in practice, probably because of their complexity. The IV principal stratification estimator is simple to implement but has had limited use in practice, possibly due to lack of familiarity. Here, we show that the IV principal stratification estimator is a modified per-protocol estimator that should be obtainable from any randomized controlled trial, and we provide a closed form for its robust variance (and its uncertainty). Finally, we illustrate sensitivity analyses we conducted to assess inference in light of potential violations of the exclusion restriction assumption.

A causal model for longitudinal randomised trials with time-dependent non-compliance.

In the presence of non-compliance, conventional analysis by intention-to-treat provides an unbiased comparison of treatment policies but typically under-estimates treatment efficacy. With all-or-nothing compliance, efficacy may be specified as the complier-average causal effect (CACE), where compliers are those who receive intervention if and only if randomised to it. We extend the CACE approach to model longitudinal data with time-dependent non-compliance, focusing on the situation in which those randomised to control may receive treatment and allowing treatment effects to vary arbitrarily over time. Defining compliance type to be the time of surgical intervention if randomised to control, so that compliers are patients who would not have received treatment at all if they had been randomised to control, we construct a causal model for the multivariate outcome conditional on compliance type and randomised arm. This model is applied to the trial of alternative regimens for glue ear treatment evaluating surgical interventions in childhood ear disease, where outcomes are measured over five time points, and receipt of surgical intervention in the control arm may occur at any time. We fit the models using Markov chain Monte Carlo methods to obtain estimates of the CACE at successive times after receiving the intervention. In this trial, over a half of those randomised to control eventually receive intervention. We find that surgery is more beneficial than control at 6months, with a small but non-significant beneficial effect at 12months.

Comments on "sensitivity of estimands in clinical trials with imperfect compliance" by Chen and Heitjan.

Chen and Heitjan (Sensitivity of estimands in clinical trials with imperfect compliance. Int J Biostat. 2023) used linear extrapolation to estimate the population average causal effect (PACE) from the complier average causal effect (CACE) in multiple randomized trials with all-or-none compliance. For extrapolating from CACE to PACE in this setting and in the paired availability design involving different availabilities of treatment among before-and-after studies, we recommend the sensitivity analysis in Baker and Lindeman (J Causal Inference, 2013) because it is not restricted to a linear model, as it involves various random effect and trend models.

Sensitivity of estimands in clinical trials with imperfect compliance.

In clinical trials that are subject to noncompliance, the commonly used intention-to-treat estimand is valid as a causal effect of treatment assignment but is sensitive to the level of compliance. An alternative estimand, the complier average causal effect (CACE), measures the average effect of treatment received in the latent subset of subjects who would comply with either assigned treatment. Because the principal stratum of compliers can vary with the circumstances of the trial, CACE too depends on the compliance fraction. We propose a model in which an underlying latent proto-compliance interacts with trial characteristics to determine a subject's compliance behavior. When the latent compliance is independent of the individual treatment effect, the average causal effect is constant across compliance classes, and CACE is robust across trials and equal to the population average causal effect. We demonstrate the potential degree of sensitivity of CACE in a simulation study, an analysis of data from a trial of vitamin A supplementation in children, and a meta-analysis of trials of epidural analgesia in labor.

Azithromycin distribution and childhood mortality in compliance-related subgroups in Niger: complier average causal effect and spillovers in a cluster-randomized, placebo-controlled trial.

BACKGROUND: Biannual azithromycin distribution to children 1-59 months old reduced all-cause mortality by 18% [incidence rate ratio (IRR) 0.82, 95% confidence interval (CI): 0.74, 0.90] in an intention-to-treat analysis of a randomized controlled trial in Niger. Estimation of the effect in compliance-related subgroups can support decision making around implementation of this intervention in programmatic settings. METHODS: The cluster-randomized, placebo-controlled design of the original trial enabled unbiased estimation of the effect of azithromycin on mortality rates in two subgroups: (i) treated children (complier average causal effect analysis); and (ii) untreated children (spillover effect analysis), using negative binomial regression. RESULTS: In Niger, 594 eligible communities were randomized to biannual azithromycin or placebo distribution and were followed from December 2014 to August 2017, with a mean treatment coverage of 90% [standard deviation (SD) 10%] in both arms. Subgroup analyses included 2581 deaths among treated children and 245 deaths among untreated children. Among treated children, the incidence rate ratio comparing mortality in azithromycin communities to placebo communities was 0.80 (95% CI: 0.72, 0.88), with mortality rates (deaths per 1000 person-years at risk) of 16.6 in azithromycin communities and 20.9 in placebo communities. Among untreated children, the incidence rate ratio was 0.91 (95% CI: 0.69, 1.21), with rates of 33.6 in azithromycin communities and 34.4 in placebo communities. CONCLUSIONS: As expected, this analysis suggested similar efficacy among treated children compared with the intention-to-treat analysis. Though the results were consistent with a small spillover benefit to untreated children, this trial was underpowered to detect spillovers.

Breast cancer overdiagnosis in stop-screen trials: More uncertainty than previously reported.

OBJECTIVE: According to the Independent UK Panel on Breast Cancer Screening, the most reliable estimates of overdiagnosis for breast cancer screening come from stop-screen trials Canada 1, Canada 2, and Malmo. The screen-interval overdiagnosis fraction is the fraction of cancers in a screening program that are overdiagnosed. We used the cumulative incidence method to estimate screen-interval overdiagnosis fraction. Our goal was to derive confidence intervals for estimated screen-interval overdiagnosis fraction and adjust for refusers in these trials. METHODS: We first show that the UK Panel's use of a 95% binomial confidence interval for estimated screen-interval overdiagnosis fraction was incorrect. We then derive a correct 95% binomial-Poisson confidence interval. We also use the method of latent-class instrumental variables to adjust for refusers. RESULTS: For the Canada 1 trial, the estimated screen-interval overdiagnosis fraction was 0.23 with a 95% binomial confidence interval of (0.18, 0.27) and a 95% binomial-Poisson confidence interval of (0.04, 0.41). For the Canada 2 trial, the estimated screen-interval overdiagnosis fraction was 0.16 with a 95% binomial confidence interval of (0.12, 0.19) and a 95% binomial-Poisson confidence interval of (-0.01, 0.32). For the Malmo trial, the estimated screen-interval overdiagnosis fraction was 0.19 with a 95% binomial confidence interval of (0.15, 0.22). Adjusting for refusers, the estimated screen-interval overdiagnosis fraction was 0.26 with a 95% binomial-Poisson confidence interval of (0.03, 0.50). CONCLUSION: The correct 95% binomial-Poisson confidence interval s for the estimated screen-interval overdiagnosis fraction based on the Canada 1, Canada 2, and Malmo stop-screen trials are much wider than the previously reported incorrect 95% binomial confidence intervals. The 95% binomial-Poisson confidence intervals widen as follow-up time increases, an unappreciated downside of longer follow-up in stop-screen trials.

Per-Protocol analyses produced larger treatment effect sizes than intention to treat: a meta-epidemiological study.

OBJECTIVE: To undertake meta-analysis and compare treatment effects estimated by the intention-to-treat (ITT) method and per-protocol (PP) method in randomized controlled trials (RCTs). PP excludes trial participants who are non-adherent to trial protocol in terms of eligibility, interventions, or outcome assessment. STUDY DESIGN AND SETTING: Five high impact journals were searched for all RCTs published between July 2017 to June 2019. Primary outcome was a pooled estimate that quantified the difference between the treatment effects estimated by the two methods. Results are presented as ratio of odds ratios (ROR). Meta-regression was used to explore the association between level of trial protocol non-adherence and treatment effect. Sensitivity analyses compared results with varying within-study correlations and across various study characteristics. RESULTS: Random-effects meta-analysis (N = 156) showed that PP estimates were on average 2% greater compared to the ITT estimates (ROR: 1.02, 95% CI: 1.00-1.04, P = 0.03). The divergence further increased with higher degree of protocol non-adherence. Sensitivity analyses reassured consistent results with various within-study correlations and across various study characteristics. CONCLUSION: There was evidence of larger treatment effect with PP compared to ITT analysis. PP analysis should not be used to assess the impact of protocol non-adherence in RCTs. Instead, in addition to ITT, investigators should consider randomization based casual method such as Complier Average Causal Effect (CACE).

Estimating causal effects of internet interventions in the context of nonadherence.

A substantial proportion of participants who are offered internet-based psychological treatments in randomized trials do not adhere and may therefore not receive treatment. Despite the availability of justified statistical methods for causal inference in such situations, researchers often rely on analytical strategies that either ignore adherence altogether or fail to provide causal estimands. The objective of this paper is to provide a gentle nontechnical introduction to complier average causal effect (CACE) analysis, which, under clear assumptions, can provide a causal estimate of the effect of treatment for a subsample of compliers. The article begins with a brief review of the potential outcome model for causal inference. After clarifying assumptions and model specifications for CACE in the latent variable framework, data from a previously published trial of an internet-based psychological treatment for irritable bowel syndrome are used to demonstrate CACE-analysis. Several model extensions are then briefly reviewed. The paper offers practical recommendations on how to analyze randomized trials of internet interventions in the context of nonadherence. It is argued that CACE-analysis, whenever it is considered appropriate, should be carried out as a complement to the standard intention-to-treat analysis and that the format of internet-based treatments is particularly well suited to such an analytical approach.

E-Freeze - a randomised controlled trial evaluating the clinical and cost effectiveness of a policy of freezing embryos followed by thawed frozen embryo transfer compared with a policy of fresh embryo transfer, in women undergoing in vitro fertilisation: a statistical analysis plan.

BACKGROUND: The E-Freeze trial is a multi-centre randomised controlled trial of fresh versus frozen embryo transfer for women undergoing in vitro fertilisation. This paper describes the statistical analysis plan for the E-Freeze trial. METHODS AND DESIGN: E-Freeze is a two-arm parallel-group, multi-centre, individually randomised controlled trial to determine if a policy of freezing embryos, followed by thawed frozen embryo transfer, results in a higher healthy baby rate when compared with the current policy of transferring fresh embryos. Couples undergoing their first, second or third cycle of in vitro fertilisation at fertility centres in the UK were randomised to either fresh or frozen embryo transfer. The primary outcome is a healthy baby, defined as a live singleton baby born at term with an appropriate weight for gestation. This paper describes the statistical analysis plan for the trial, including the analysis principles, definitions of outcomes, methods for primary analysis, pre-specified subgroup analysis and sensitivity analysis. This plan was finalised prior to completion of recruitment to the trial. TRIAL REGISTRATION: ISRCTN registry: ISRCTN61225414 . Registered on 29 December 2015.

Efficacy of the Fun For Wellness Online Intervention to Promote Well-Being Actions: A Secondary Data Analysis.

OBJECTIVE: Fun For Wellness (FFW) is a new online intervention designed to promote growth in well-being by providing capability-enhancing learning opportunities (e.g., play an interactive game) to participants. The purpose of this study was to provide an initial evaluation of the efficacy of the FFW intervention to increase well-being actions. MATERIALS AND METHODS: The study design was a secondary data analysis of a large-scale prospective, double-blind, parallel-group randomized controlled trial. Data were collected at baseline and 30 and 60 days postbaseline. A total of 479 adult employees at a major university in the southeast of the United States of America were enrolled. Participants who were randomly assigned to the FFW group were provided with 30 days of 24-hour access to the intervention. A two-class linear regression model with complier average causal effect estimation was fitted to well-being actions scores at 30 and 60 days. RESULTS: Intent-to-treat analysis provided evidence that the effect of being assigned to the FFW intervention, without considering actual participation in the FFW intervention, had a null effect on each dimension of well-being actions at 30 and 60 days. Participants who complied with the FFW intervention, however, had significantly higher well-being actions scores, compared to potential compliers in the Usual Care group, in the interpersonal dimension at 60 days, and the physical dimension at 30 days. CONCLUSIONS: Results from this secondary data analysis provide some supportive evidence for both the efficacy of and possible revisions to the FFW intervention in regard to promoting well-being actions.

Beyond intent to treat (ITT): A complier average causal effect (CACE) estimation primer.

Randomized control trials (RCTs) have long been the gold standard for allowing causal inferences to be made regarding the efficacy of a treatment under investigation, but traditional RCT data analysis perspectives do not take into account a common reality: imperfect participant compliance to treatment. Recent advances in both maximum likelihood parameter estimation and mixture modeling methodology have enabled treatment effects to be estimated, in the presence of less than ideal levels of participant compliance, via a Complier Average Causal Effect (CACE) structural equation mixture model. CACE is described in contrast to "intent to treat" (ITT), "per protocol", and "as treated" RCT data analysis perspectives. CACE model assumptions, specification, estimation, and interpretation will all be demonstrated with simulated data generated from a randomized controlled trial of cognitive-behavioral therapy for Juvenile Fibromyalgia. CACE analysis model figures, linear model equations, and Mplus estimation syntax examples are all provided. Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.

Statistical Considerations of Food Allergy Prevention Studies.

Clinical studies to prevent the development of food allergy have recently helped reshape public policy recommendations on the early introduction of allergenic foods. These trials are also prompting new research, and it is therefore important to address the unique design and analysis challenges of prevention trials. We highlight statistical concepts and give recommendations that clinical researchers may wish to adopt when designing future study protocols and analysis plans for prevention studies. Topics include selecting a study sample, addressing internal and external validity, improving statistical power, choosing alpha and beta, analysis innovations to address dilution effects, and analysis methods to deal with poor compliance, dropout, and missing data.

Assessing the sensitivity of methods for estimating principal causal effects.

The framework of principal stratification provides a way to think about treatment effects conditional on post-randomization variables, such as level of compliance. In particular, the complier average causal effect (CACE) - the effect of the treatment for those individuals who would comply with their treatment assignment under either treatment condition - is often of substantive interest. However, estimation of the CACE is not always straightforward, with a variety of estimation procedures and underlying assumptions, but little advice to help researchers select between methods. In this article, we discuss and examine two methods that rely on very different assumptions to estimate the CACE: a maximum likelihood ('joint') method that assumes the 'exclusion restriction,' (ER) and a propensity score-based method that relies on 'principal ignorability.' We detail the assumptions underlying each approach, and assess each methods' sensitivity to both its own assumptions and those of the other method using both simulated data and a motivating example. We find that the ER-based joint approach appears somewhat less sensitive to its assumptions, and that the performance of both methods is significantly improved when there are strong predictors of compliance. Interestingly, we also find that each method performs particularly well when the assumptions of the other approach are violated. These results highlight the importance of carefully selecting an estimation procedure whose assumptions are likely to be satisfied in practice and of having strong predictors of principal stratum membership.

Semiparametric transformation models for causal inference in time to event studies with all-or-nothing compliance.

We consider causal inference in randomized survival studies with right censored outcomes and all-or-nothing compliance, using semiparametric transformation models to estimate the distribution of survival times in treatment and control groups, conditional on covariates and latent compliance type. Estimands depending on these distributions, for example, the complier average causal effect (CACE), the complier effect on survival beyond time t, and the complier quantile effect are then considered. Maximum likelihood is used to estimate the parameters of the transformation models, using a specially designed expectation-maximization (EM) algorithm to overcome the computational difficulties created by the mixture structure of the problem and the infinite dimensional parameter in the transformation models. The estimators are shown to be consistent, asymptotically normal, and semiparametrically efficient. Inferential procedures for the causal parameters are developed. A simulation study is conducted to evaluate the finite sample performance of the estimated causal parameters. We also apply our methodology to a randomized study conducted by the Health Insurance Plan of Greater New York to assess the reduction in breast cancer mortality due to screening.