Inference without randomization or ignorability: A stability-controlled quasi-experiment on the prevention of tuberculosis.

ABSTRACT

The stability-controlled quasi-experiment (SCQE) is an approach to study the effects of nonrandomized, newly adopted treatments. While covariate adjustment techniques rely on a "no unobserved confounding" assumption, SCQE imposes an assumption on the change in the average nontreatment outcome between successive cohorts (the "baseline trend"). We provide inferential tools for SCQE and its first application, examining whether isoniazid preventive therapy (IPT) reduced tuberculosis (TB) incidence among 26 715 HIV patients in Tanzania. After IPT became available, 16% of untreated patients developed TB within a year, compared with only 0.5% of patients under treatment. Thus, a simple difference in means suggests a 15.5 percentage point (pp) lower risk (p ≪ .001). Adjusting for covariates using numerous techniques leaves this effectively unchanged. Yet, due to confounding biases, such estimates can be misleading regardless of their statistical strength. By contrast, SCQE reveals valid causal effect estimates for any chosen assumption on the baseline trend. For example, assuming a baseline trend near 0 (no change in TB incidence over time, absent this treatment) implies a small and insignificant effect. To argue IPT was beneficial requires arguing that the nontreatment incidence would have risen by at least 0.7 pp per year, which is plausible but far from certain. SCQE may produce narrow estimates when the plausible range of baseline trends can be sufficiently constrained, while in every case it tells us what baseline trends must be believed in order to sustain a given conclusion, protecting against inferences that rely upon infeasible assumptions.