Off-Protocol Radiation Therapy in Phase 3 Metastatic Solid Tumor Trials.

PURPOSE: Increasing data suggest that radiation therapy, particularly ablative radiation therapy, alters the natural history of metastatic disease. For patients with metastatic disease enrolled in prospective trials testing systemic therapy, the use of off-protocol radiation therapy to improve clinical symptoms or extend the duration of study systemic therapy may influence study endpoints. We sought to evaluate how often off-protocol radiation therapy was permitted among systemic therapy phase 3 trials, how often off-protocol radiation therapy is used, and whether off-protocol radiation therapy correlated with study outcomes. METHODS AND MATERIALS: Two-arm, superiority-design, phase 3 randomized trials testing systemic therapy were screened from ClinicalTrials.gov. Protocol availability was required to assess the trial approach to off-protocol radiation therapy if not described in the manuscript. Adjusted odds ratios with 95% CI were calculated by logistic regression. RESULTS: A total of 112 trials enrolling 80,134 patients were included, with publication dates between 2010 and 2019. Of these, off-protocol radiation therapy was allowed, not discussed, or prohibited during study systemic therapy in 52% (N =58), 25% (N = 28), and 23% (N = 26) of trials, respectively. However, only 2% (2 of 112) of trials reported off-protocol radiation therapy utilization rates, although no data were reported on the use of ablative off-protocol radiation therapy. No trials evaluated or adjusted for the potential influence of off-protocol radiation therapy on study endpoints. Among the subset of open-label studies, trials permissive toward off-protocol radiation therapy were more likely to meet their primary endpoint (adjusted odds ratio, 4.50; 95% CI, 1.23-20.23; P = .04). CONCLUSIONS: Although most trials allowed off-protocol radiation therapy during the receipt of the study systemic therapy, the influence of off-protocol radiation therapy, especially ablative radiation therapy, on study outcomes is underevaluated among phase 3 systemic therapy trials.

Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials.

Most oncology trials define superiority of an experimental therapy compared to a control therapy according to frequentist significance thresholds, which are widely misinterpreted. Posterior probability distributions computed by Bayesian inference may be more intuitive measures of uncertainty, particularly for measures of clinical benefit such as the minimum clinically important difference (MCID). Here, we manually reconstructed 194,129 individual patient-level outcomes across 230 phase III, superiority-design, oncology trials. Posteriors were calculated by Markov Chain Monte Carlo sampling using standard priors. All trials interpreted as positive had probabilities > 90% for marginal benefits (HR < 1). However, 38% of positive trials had ≤ 90% probabilities of achieving the MCID (HR < 0.8), even under an enthusiastic prior. A subgroup analysis of 82 trials that led to regulatory approval showed 30% had ≤ 90% probability for meeting the MCID under an enthusiastic prior. Conversely, 24% of negative trials had > 90% probability of achieving marginal benefits, even under a skeptical prior, including 12 trials with a primary endpoint of overall survival. Lastly, a phase III oncology-specific prior from a previous work, which uses published summary statistics rather than reconstructed data to compute posteriors, validated the individual patient-level data findings. Taken together, these results suggest that Bayesian models add considerable unique interpretative value to phase III oncology trials and provide a robust solution for overcoming the discrepancies between refuting the null hypothesis and obtaining a MCID.

Secondary Endpoint Utilization and Publication Rate among Phase III Oncology Trials.

Secondary endpoints (SEP) provide crucial information in the interpretation of clinical trials, but their features are not yet well understood. Thus, we sought to empirically characterize the scope and publication rate of SEPs among late-phase oncology trials. We assessed SEPs for each randomized, published phase III oncology trial across all publications and ClinicalTrials.gov, performing logistic regressions to evaluate associations between trial characteristics and SEP publication rates. After screening, a total of 280 trials enrolling 244,576 patients and containing 2,562 SEPs met the inclusion criteria. Only 22% of trials (62/280) listed all SEPs consistently between ClinicalTrials.gov and the trial protocol. The absolute number of SEPs per trial increased over time, and trials sponsored by industry had a greater number of SEPs (median 9 vs. 5 SEPs per trial; P < 0.0001). In total, 69% of SEPs (1,770/2,562) were published. The publication rate significantly varied by SEP category [X2 (5, N = 2,562) = 245.86; P < 0.001]. SEPs that place the most burden on patients, such as patient-reported outcomes and translational correlatives, were published at 63% (246/393) and 44% (39/88), respectively. Trials with more SEPs were associated with lower overall SEP publication rates. Overall, our findings are that SEP publication rates in late-phase oncology trials are highly variable based on the type of SEP. To avoid undue burden on patients and promote transparency of findings, trialists should weigh the biological and clinical relevance of each SEP together with its feasibility at the time of trial design. SIGNIFICANCE: In this investigation, we characterized the utilization and publication rates of SEPs among late-phase oncology trials. Our results draw attention to the proliferation of SEPs in recent years. Although overall publication rates were high, underpublication was detected among endpoints that may increase patient burden (such as translational correlatives and patient-reported outcomes).

Improving the clinical meaning of surrogate endpoints: An empirical assessment of clinical progression in phase III oncology trials.

Disease progression in clinical trials is commonly defined by radiologic measures. However, clinical progression may be more meaningful to patients, may occur even when radiologic criteria for progression are not met, and often requires a change in therapy in clinical practice. The objective of this study was to determine the utilization of clinical progression criteria within progression-based trial endpoints among phase III trials testing systemic therapies for metastatic solid tumors. The primary manuscripts and protocols of phase III trials were reviewed for whether clinical events, such as refractory pain, tumor bleeding, or neurologic compromise, could constitute a progression event. Univariable logistic regression computed odds ratios (OR) and 95% CI for associations between trial-level covariates and clinical progression. A total of 216 trials enrolling 148,190 patients were included, with publication dates from 2006 through 2020. A major change in clinical status was included in the progression criteria of 13% of trials (n = 27), most commonly as a secondary endpoint (n = 22). Only 59% of trials (n = 16) reported distinct clinical progression outcomes that constituted the composite surrogate endpoint. Compared with other disease sites, genitourinary trials were more likely to include clinical progression definitions (16/33 [48%] vs. 11/183 [6%]; OR, 14.72; 95% CI, 5.99 to 37.84; p < .0001). While major tumor-related clinical events were seldom considered as disease progression events, increased attention to clinical progression may improve the meaningfulness and clinical applicability of surrogate endpoints for patients with metastatic solid tumors.

Increasing Power in Phase III Oncology Trials With Multivariable Regression: An Empirical Assessment of 535 Primary End Point Analyses.

PURPOSE: A previous study demonstrated that power against the (unobserved) true effect for the primary end point (PEP) of most phase III oncology trials is low, suggesting an increased risk of false-negative findings in the field of late-phase oncology. Fitting models with prognostic covariates is a potential solution to improve power; however, the extent to which trials leverage this approach, and its impact on trial interpretation at scale, is unknown. To that end, we hypothesized that phase III trials using multivariable PEP analyses are more likely to demonstrate superiority versus trials with univariable analyses. METHODS: PEP analyses were reviewed from trials registered on ClinicalTrials.gov. Adjusted odds ratios (aORs) were calculated by logistic regressions. RESULTS: Of the 535 trials enrolling 454,824 patients, 69% (n = 368) used a multivariable PEP analysis. Trials with multivariable PEP analyses were more likely to demonstrate PEP superiority (57% [209 of 368] v 42% [70 of 167]; aOR, 1.78 [95% CI, 1.18 to 2.72]; P = .007). Among trials with a multivariable PEP model, 16 conditioned on covariates and 352 stratified by covariates. However, 108 (35%) of 312 trials with stratified analyses lost power by categorizing a continuous variable, which was especially common among immunotherapy trials (aOR, 2.45 [95% CI, 1.23 to 4.92]; P = .01). CONCLUSION: Trials increasing power by fitting multivariable models were more likely to demonstrate PEP superiority than trials with unadjusted analysis. Underutilization of conditioning models and empirical power loss associated with covariate categorization required by stratification were identified as barriers to power gains. These findings underscore the opportunity to increase power in phase III trials with conventional methodology and improve patient access to effective novel therapies.

Bayesian Interim Analysis and Efficiency of Phase III Randomized Trials.

IMPORTANCE: Improving the efficiency of interim assessments in phase III trials should reduce trial costs, hasten the approval of efficacious therapies, and mitigate patient exposure to disadvantageous randomizations. OBJECTIVE: We hypothesized that in silico Bayesian early stopping rules improve the efficiency of phase III trials compared with the original frequentist analysis without compromising overall interpretation. DESIGN: Cross-sectional analysis. SETTING: 230 randomized phase III oncology trials enrolling 184,752 participants. PARTICIPANTS: Individual patient-level data were manually reconstructed from primary endpoint Kaplan-Meier curves. INTERVENTIONS: Trial accruals were simulated 100 times per trial and leveraged published patient outcomes such that only the accrual dynamics, and not the patient outcomes, were randomly varied. MAIN OUTCOMES AND MEASURES: Early stopping was triggered per simulation if interim analysis demonstrated ≥ 85% probability of minimum clinically important difference/3 for efficacy or futility. Trial-level early closure was defined by stopping frequencies ≥ 0.75. RESULTS: A total of 12,451 simulations (54%) met early stopping criteria. Trial-level early stopping frequency was highly predictive of the published outcome (OR, 7.24; posterior probability of association, >99.99%; AUC, 0.91; P < 0.0001). Trial-level early closure was recommended for 82 trials (36%), including 62 trials (76%) which had performed frequentist interim analysis. Bayesian early stopping rules were 96% sensitive (95% CI, 91% to 98%) for detecting trials with a primary endpoint difference, and there was a high level of agreement in overall trial interpretation (Bayesian Cohen's κ, 0.95; 95% CrI, 0.92 to 0.99). However, Bayesian interim analysis was associated with >99.99% posterior probability of reducing patient enrollment requirements ( P < 0.0001), with an estimated cumulative enrollment reduction of 20,543 patients (11%; 89 patients averaged equally over all studied trials) and an estimated cumulative cost savings of 851 million USD (3.7 million USD averaged equally over all studied trials). CONCLUSIONS AND RELEVANCE: Bayesian interim analyses may improve randomized trial efficiency by reducing enrollment requirements without compromising trial interpretation. Increased utilization of Bayesian interim analysis has the potential to reduce costs of late-phase trials, reduce patient exposures to ineffective therapies, and accelerate approvals of effective therapies. KEY POINTS: Question: What are the effects of Bayesian early stopping rules on the efficiency of phase III randomized oncology trials?Findings: Individual-patient level outcomes were reconstructed for 184,752 patients from 230 trials. Compared with the original interim analysis strategy, in silico Bayesian interim analysis reduced patient enrollment requirements and preserved the original trial interpretation. Meaning: Bayesian interim analysis may improve the efficiency of conducting randomized trials, leading to reduced costs, reduced exposure of patients to disadvantageous treatments, and accelerated approval of efficacious therapies.