Challenges of estimating treatment effects after a positive interim analysis.

ABSTRACT

BACKGROUND: This research investigates why a beneficial treatment effect reported at the first interim analysis (IA) may diminish at a subsequent analysis (SA). We examined three challenges in interpreting treatment effects from randomized clinical trials (RCTs) after the first positive IA overestimation bias; non-proportional hazards; and heterogeneity in recruitment. We investigate how a penalized estimation method can address overestimation bias, and discuss additional factors to consider when interpreting positive IA results. METHODS: We identified oncology RCTs reporting positive results at the initial IA and a SA for event-free (EFS) and overall survival (OS). We modeled (1) the hazard ratio at IA (HRIA) versus its timing as measured by the information fraction (IF; i.e., events at IA versus total events sought); and (2), the ratio of HRIA to HRSA (rHR) versus the IF. This was repeated for HRIA adjusted for overestimation bias. Examples of the other two challenges were sought. RESULTS: Amongst 71 RCTs, HRIA were positively associated with the IF (slope EFS 0.83, 95 % CI 0.44-1.22; OS 0.25, 95 % CI 0.10-0.41). HRIA tended to exaggerate HRSA, and more so the lower the IF (slope rHR versus IF EFS 0.10, 95 % CI - 0.22 to 0.42; OS 0.26, 95 % CI 0.07-0.46). Adjusted HRIA did not exaggerate HRSA (slope rHR versus IF EFS - 0.14, 95 % CI - 0.67 to 0.39; OS 0.02, 95 % CI - 0.26 to 0.30). Examples of two other challenges are shown. CONCLUSION: Overestimation bias, non-proportional hazards, and heterogeneity in recruitment and other important treatments should be considered when communicating estimates of treatment effects from positive IAs.