Empirical use of causal inference methods to evaluate survival differences in a real-world registry vs those found in randomized clinical trials.

ABSTRACT

With heighted interest in causal inference based on real-world evidence, this empirical study sought to understand differences between the results of observational analyses and long-term randomized clinical trials. We hypothesized that patients deemed "eligible" for clinical trials would follow a different survival trajectory from those deemed "ineligible" and that this factor could partially explain results. In a large observational registry dataset, we estimated separate survival trajectories for hypothetically trial-eligible vs ineligible patients under both coronary artery bypass surgery (CABG) and percutaneous coronary intervention (PCI). We also explored whether results would depend on the causal inference method (inverse probability of treatment weighting vs optimal full propensity matching) or the approach to combine propensity scores from multiple imputations (the "across" vs "within" approaches). We found that, in this registry population of PCI/CABG multivessel patients, 32.5% would have been eligible for contemporaneous RCTs, suggesting that RCTs enroll selected populations. Additionally, we found treatment selection bias with different distributions of propensity scores between PCI and CABG patients. The different methodological approaches did not result in different conclusions. Overall, trial-eligible patients appeared to demonstrate at least marginally better survival than ineligible patients. Treatment comparisons by eligibility depended on disease severity. Among trial-eligible three-vessel diseased and trial-ineligible two-vessel diseased patients, CABG appeared to have at least a slight advantage with no treatment difference otherwise. In conclusion, our analyses suggest that RCTs enroll highly selected populations, and our findings are generally consistent with RCTs but less pronounced than major registry findings.