Proportional Hazards Violations in Phase 3 Cancer Clinical Trials: A Potential Source of Trial Misinterpretation.

BACKGROUND: Survival analyses of novel agents with long-term responders often exhibit differential hazard rates over time. Such proportional hazards violations (PHVs) may reduce the power of the log-rank test and lead to misinterpretation of trial results. We aimed to characterize the incidence and study attributes associated with PHVs in phase 3 oncology trials and assess the utility of restricted mean survival time (RMST) and MaxCombo as additional analyses. METHODS: Clinicaltrials.gov and PubMed were searched to identify 2-arm, randomized, phase 3 superiority-design cancer trials with time-to-event primary endpoints and published results through 2020. Patient-level data were reconstructed from published Kaplan-Meier curves. PHVs were assessed using Schoenfeld residuals. RESULTS: Three hundred fifty-seven Kaplan-Meier comparisons across 341 trials were analyzed, encompassing 292,831 enrolled patients. PHVs were identified in 85/357 (23.8%; 95%CI 19.7%, 28.5%) comparisons. In multivariable analysis, non-OS endpoints (odds ratio [OR] 2.16 [95%CI 1.21, 3.87]; P=.009) were associated with higher odds of PHVs, and immunotherapy comparisons (OR 1.94 [95%CI 0.98, 3.86]; P=.058) were weakly suggestive of higher odds of PHVs. Few trials with PHVs (25/85, 29.4%) pre-specified a statistical plan to account for PHVs. Fourteen trials with PHVs exhibited discordant statistical signals with RMST or MaxCombo, of which ten (71%) reported negative results. CONCLUSION: PHVs are common across therapy types, and attempts to account for PHVs in statistical design are lacking despite the potential for results exhibiting non-proportional hazards to be misinterpreted.

Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials.

Most oncology trials define superiority of an experimental therapy compared to a control therapy according to frequentist significance thresholds, which are widely misinterpreted. Posterior probability distributions computed by Bayesian inference may be more intuitive measures of uncertainty, particularly for measures of clinical benefit such as the minimum clinically important difference (MCID). Here, we manually reconstructed 194,129 individual patient-level outcomes across 230 phase III, superiority-design, oncology trials. Posteriors were calculated by Markov Chain Monte Carlo sampling using standard priors. All trials interpreted as positive had probabilities > 90% for marginal benefits (HR < 1). However, 38% of positive trials had ≤ 90% probabilities of achieving the MCID (HR < 0.8), even under an enthusiastic prior. A subgroup analysis of 82 trials that led to regulatory approval showed 30% had ≤ 90% probability for meeting the MCID under an enthusiastic prior. Conversely, 24% of negative trials had > 90% probability of achieving marginal benefits, even under a skeptical prior, including 12 trials with a primary endpoint of overall survival. Lastly, a phase III oncology-specific prior from a previous work, which uses published summary statistics rather than reconstructed data to compute posteriors, validated the individual patient-level data findings. Taken together, these results suggest that Bayesian models add considerable unique interpretative value to phase III oncology trials and provide a robust solution for overcoming the discrepancies between refuting the null hypothesis and obtaining a MCID.

Bayesian Interim Analysis and Efficiency of Phase III Randomized Trials.

IMPORTANCE: Improving the efficiency of interim assessments in phase III trials should reduce trial costs, hasten the approval of efficacious therapies, and mitigate patient exposure to disadvantageous randomizations. OBJECTIVE: We hypothesized that in silico Bayesian early stopping rules improve the efficiency of phase III trials compared with the original frequentist analysis without compromising overall interpretation. DESIGN: Cross-sectional analysis. SETTING: 230 randomized phase III oncology trials enrolling 184,752 participants. PARTICIPANTS: Individual patient-level data were manually reconstructed from primary endpoint Kaplan-Meier curves. INTERVENTIONS: Trial accruals were simulated 100 times per trial and leveraged published patient outcomes such that only the accrual dynamics, and not the patient outcomes, were randomly varied. MAIN OUTCOMES AND MEASURES: Early stopping was triggered per simulation if interim analysis demonstrated ≥ 85% probability of minimum clinically important difference/3 for efficacy or futility. Trial-level early closure was defined by stopping frequencies ≥ 0.75. RESULTS: A total of 12,451 simulations (54%) met early stopping criteria. Trial-level early stopping frequency was highly predictive of the published outcome (OR, 7.24; posterior probability of association, >99.99%; AUC, 0.91; P < 0.0001). Trial-level early closure was recommended for 82 trials (36%), including 62 trials (76%) which had performed frequentist interim analysis. Bayesian early stopping rules were 96% sensitive (95% CI, 91% to 98%) for detecting trials with a primary endpoint difference, and there was a high level of agreement in overall trial interpretation (Bayesian Cohen's κ, 0.95; 95% CrI, 0.92 to 0.99). However, Bayesian interim analysis was associated with >99.99% posterior probability of reducing patient enrollment requirements ( P < 0.0001), with an estimated cumulative enrollment reduction of 20,543 patients (11%; 89 patients averaged equally over all studied trials) and an estimated cumulative cost savings of 851 million USD (3.7 million USD averaged equally over all studied trials). CONCLUSIONS AND RELEVANCE: Bayesian interim analyses may improve randomized trial efficiency by reducing enrollment requirements without compromising trial interpretation. Increased utilization of Bayesian interim analysis has the potential to reduce costs of late-phase trials, reduce patient exposures to ineffective therapies, and accelerate approvals of effective therapies. KEY POINTS: Question: What are the effects of Bayesian early stopping rules on the efficiency of phase III randomized oncology trials?Findings: Individual-patient level outcomes were reconstructed for 184,752 patients from 230 trials. Compared with the original interim analysis strategy, in silico Bayesian interim analysis reduced patient enrollment requirements and preserved the original trial interpretation. Meaning: Bayesian interim analysis may improve the efficiency of conducting randomized trials, leading to reduced costs, reduced exposure of patients to disadvantageous treatments, and accelerated approval of efficacious therapies.

Association of differential censoring with survival and suboptimal control arms among oncology clinical trials.

Differential censoring, which refers to censoring imbalance between treatment arms, may bias the interpretation of survival outcomes in clinical trials. In 146 phase III oncology trials with statistically significant time-to-event surrogate primary endpoints, we evaluated the association between differential censoring in the surrogate primary endpoints, control arm adequacy, and the subsequent statistical significance of overall survival results. Twenty-four (16%) trials exhibited differential censoring that favored the control arm, whereas 15 (10%) exhibited differential censoring that favored the experimental arm. Positive overall survival was more common in control arm differential censoring trials (63%) than in trials without differential censoring (37%) or with experimental arm differential censoring (47%; odds ratio = 2.64, 95% confidence interval = 1.10 to 7.20; P = .04). Control arm differential censoring trials more frequently used suboptimal control arms at 46% compared with 20% without differential censoring and 13% with experimental arm differential censoring (odds ratio = 3.60, 95% confidence interval = 1.29 to 10.0; P = .007). The presence of control arm differential censoring in trials with surrogate primary endpoints, especially in those with overall survival conversion, may indicate an inadequate control arm and should be examined and explained.

Factors Associated With Open Access Publishing Costs in Oncology Journals: Cross-sectional Observational Study.

BACKGROUND: Open access (OA) publishing represents an exciting opportunity to facilitate the dissemination of scientific information to global audiences. However, OA publishing is often associated with significant article processing charges (APCs) for authors, which may thus serve as a barrier to publication. OBJECTIVE: In this observational cohort study, we aimed to characterize the landscape of OA publishing in oncology and, further, identify characteristics of oncology journals that are predictive of APCs. METHODS: We identified oncology journals using the SCImago Journal & Country Rank database. All journals with an OA publication option and APC data openly available were included. We searched journal websites and tabulated journal characteristics, including APC amount (in US dollars), OA model (hybrid vs full), 2-year impact factor (IF), H-index, number of citable documents, modality/treatment specific (if applicable), and continent of origin. All APCs were converted to US-dollar equivalents for final analyses. Selecting variables with significant associations in the univariable analysis, we generated a multiple regression model to identify journal characteristics independently associated with OA APC amount. An audit of a random 10% sample of the data was independently performed by 2 authors to ensure data accuracy, precision, and reproducibility. RESULTS: Of 367 oncology journals screened, 251 met the final inclusion criteria. The median APC was US $2957 (IQR 1958-3450). The majority of journals (n=156, 62%) adopted the hybrid OA publication model and were based in Europe (n=119, 47%) or North America (n=87, 35%). The median (IQR) APC for all journals was US $2957 (1958-3540). Twenty-five (10%) journals had APCs greater than US $4000. There were 10 (4%) journals that offered OA publication with no publication charge. Univariable testing showed that journals with a greater number of citable documents (P<.001), higher 2-year IF (P<.001), higher H-index (P<.001), and those using the hybrid OA model (P<.001), or originating in Europe or North America (P<.001) tended to have higher APCs. In our multivariable model, the number of citable documents (ß=US $367, SD US $133; P=.006), 2-year IF (US $1144, SD US $177; P<.001), hybrid OA publishing model (US $991, SD US $189; P<.001), and North American origin (US $838, SD US $186; P<.001) persisted as significant predictors of processing charges. CONCLUSIONS: OA publication costs are greater in oncology journals that publish more citable articles, use the hybrid OA model, have a higher IF, and are based in North America or Europe. These findings may inform targeted action to help the oncology community fully appreciate the benefits of open science.