Prevalence and Impact of Bypassing or Overriding Phase 2 Trials in Neurologic Drug Development.

BACKGROUND AND OBJECTIVES: Pivotal trials for neurologic drugs in clinical development are often initiated without a phase 2 trial ("bypass") or despite a negative phase 2 efficacy result ("override"). Such practices may degrade the risk/benefit ratio of phase 3 trials. The aim of this study is to estimate the proportion of phase 3 trials for 10 neurologic diseases started without a positive phase 2 trial, to identify factors associated with this practice, and to investigate any association with unfavorable phase 3 trial outcomes. METHODS: We searched ClinicalTrials.gov for phase 3 trials completed during 2011-2021, with at least 1 research site in the United States, Canada, the European Union, the United Kingdom, or Australia, and investigating drugs or biologics for treatment of 10 neurologic conditions. Our primary objective was to assess the prevalence of phase 2 bypass/override by searching for preceding phase 2 trials. We used Fisher exact tests to determine whether phase 3 trial characteristics and trial results were associated with phase 2 bypass/override. RESULTS: Of the 1,188 phase 3 trials captured in our search, 113 met eligibility for inclusion. Of these, 46% were not preceded by a phase 2 trial that was positive on an efficacy endpoint (31% bypassed and 15% overrode phase 2 trial). Phase 2 bypass/override was not associated with industry funding (77% vs 89%, 95% CI 0.75-7.55, p = 0.13) or testing already approved interventions (23% vs 15%, 95% CI 0.60-5.14, p = 0.33). Overall, phase 3 trials based on phase 2 bypassed/override were statistically significantly less likely to be positive on their primary outcome (31% vs 57%, respectively, 95% CI 1.21-6.92, p = 0.01). This effect disappeared when indications characterized by nearly universal positive or negative results were excluded. Trials that bypassed/overrode phase 2 trials were not statistically significantly more likely to be terminated early because of safety or futility (29% vs 15%, respectively, 95% CI 0.15-1.18, p = 0.11) and did not show increased risk of adverse events in experimental arms (RR = 1.46, 95% CI 1.19-1.79, vs RR = 1.36, 95% CI 1.10-1.69, respectively, p = 0.65). DISCUSSION: Almost half of the neurologic disease phase 3 trials were initiated without the support of a positive phase 2 trial. Although our analysis does not establish harm with bypass/override, its prevalence and the scientific rationale for phase 2 trial testing favor development of criteria defining when phase 2 bypass/override is justified.

Survival Benefit Associated With Participation in Clinical Trials of Anticancer Drugs: A Systematic Review and Meta-Analysis.

Importance: Many cancer clinical investigators view clinical trials as offering better care for patients than routine clinical care. However, definitive evidence of clinical benefit from trial participation (hereafter referred to as the participation effect) has yet to emerge. Objective: To conduct a systematic review and meta-analysis of the evidence examining whether patient participation in cancer trials was associated with greater survival benefit compared with routine care. Data Sources: Studies were found through PubMed and Embase (January 1, 2000, until August 31, 2022), as well as backward and forward citation searching. Study Selection: Studies were included that compared overall survival of trial participants and routine care patients. Data Extraction and Synthesis: Data extraction and methodological quality assessment were completed by 2 independent coders using Covidence software. Data were pooled using a random-effects model and analyzed based on the quality of the comparison between trial participants and routine care patients (ie, extent to which studies controlled for bias and confounders). Main Outcomes and Measures: The hazard ratio (HR) for overall survival of trial participants vs routine care patients. Results: Thirty-nine publications were included, comprising 85 comparisons of trial participants and routine care patients. The meta-analysis revealed a statistically significant overall survival benefit for trial participants (HR, 0.76 [95% CI, 0.69-0.82]) when all studies were pooled, regardless of design or quality. However, survival benefits diminished in study subsets that matched trial participants and routine care patients for eligibility criteria (HR, 0.85 [95% CI, 0.75-0.97]) and disappeared when only high-quality studies were pooled (HR, 0.91 [95% CI, 0.80-1.05]). They also disappeared when estimates were adjusted for potential publication bias (HR, 0.94 [95% CI, 0.86-1.03]). Conclusions and Relevance: Many studies suggest a survival benefit for cancer trial participants. However, these benefits were not detected in studies using designs addressing important sources of bias and confounding. Pooled results of high-quality studies are not consistent with a beneficial effect of trial participation on its own.

The Benefits and Risks of Receiving Investigational Solid Tumor Drugs in Randomized Trials : A Systematic Review and Meta-analysis.

BACKGROUND: Many patients participate in cancer trials to access new therapies. The extent to which new treatments produce clinical benefit for trial participants is unclear. PURPOSE: To estimate the progression-free survival (PFS) and overall survival (OS) advantage of assignment to experimental groups in randomized trials for 6 solid tumors. DATA SOURCES: ClinicalTrials.gov was searched for trials of investigational drugs with results posted between 2017 and 2021. STUDY SELECTION: Investigational drugs were defined as those not yet having full approval from the U.S. Food and Drug Administration for the study indication. Trials were included if they were randomized and tested drugs or biologics. DATA EXTRACTION: Data extraction was completed by 2 independent reviewers. Data were pooled using a random-effects model. DATA SYNTHESIS: The sample included 128 trials comprising 141 comparisons of a new drug and a comparator. These comparisons included 47 050 patients. The pooled hazard ratio for PFS was 0.80 (95% CI, 0.75 to 0.85), indicating statistically significant benefit for patients in experimental groups. This corresponded to a median PFS advantage of 1.25 months (CI, 0.80 to 1.68 months). The pooled hazard ratio for OS was 0.92 (CI, 0.88 to 0.95), corresponding to a survival gain of 1.18 months (CI, 0.72 to 1.71 months). The absolute risk for a serious adverse event for comparator group patients was 29.56% (CI, 26.64% to 32.65%), with an increase in risk of 7.40% (CI, 5.66% to 9.14%) for patients in experimental groups. LIMITATIONS: Trials in this sample were heterogeneous. Comparator group interventions were assumed to reflect standard of care. CONCLUSION: Assignment to experimental groups produces statistically significant survival gains. However, the absolute survival gain is small, and toxicity is statistically significantly greater. The findings of this review provide reassuring evidence that patients are not meaningfully disadvantaged by assignment to comparator groups. PRIMARY FUNDING SOURCE: Canadian Institutes of Health Research.

Bypassing phase 2 in cancer drug development erodes the risk/benefit balance in phase 3 trials.

OBJECTIVES: Drug developers sometimes launch phase 3 (P3) trials without supporting evidence from phase 2 (P2) trials. We call this practice "P2 bypass." The aims of this study were to estimate the prevalence of P2 bypass and to compare the safety and efficacy results for P3 trials that bypassed with those that did not. STUDY DESIGN AND SETTING: We created a sample of P3 solid tumor trials registered on ClinicalTrials.gov with primary completion dates between 2013 and 2019. We then attempted to match each with a supporting P2 trial using strict and broad criteria. P3 outcomes were meta-analyzed using a random effects model with subgroup contrast between trials that bypassed and those that did not. RESULTS: 129 P3 trial arms met eligibility and nearly half involved P2 bypass. P3 trials involving P2 bypass produced significantly and nonsignificantly worse pooled efficacy estimates using broad and strict matching criteria, respectively. We did not observe significant differences in safety outcomes between P3 trials that bypassed P2 and those that did not. CONCLUSION: The risk/benefit balance of P3 trials that bypassed P2 is less favourable than for trials supported by P2.

The proportion of randomized controlled trials that inform clinical practice.

Prior studies suggest that clinical trials are often hampered by problems in design, conduct, and reporting that limit their uptake in clinical practice. We have described 'informativeness' as the ability of a trial to guide clinical, policy, or research decisions. Little is known about the proportion of initiated trials that inform clinical practice. We created a cohort of randomized interventional clinical trials in three disease areas (ischemic heart disease, diabetes mellitus, and lung cancer) that were initiated between January 1, 2009 and December 31, 2010 using ClinicalTrials.gov. We restricted inclusion to trials aimed at answering a clinical question related to the treatment or prevention of disease. Our primary outcome was the proportion of clinical trials fulfilling four conditions of informativeness: importance of the clinical question, trial design, feasibility, and reporting of results. Our study included 125 clinical trials. The proportion meeting four conditions for informativeness was 26.4% (95% CI 18.9-35.0). Sixty-seven percent of participants were enrolled in informative trials. The proportion of informative trials did not differ significantly between our three disease areas. Our results suggest that the majority of randomized interventional trials designed to guide clinical practice possess features that may compromise their ability to do so. This highlights opportunities to improve the scientific vetting of clinical research.

Relationship between lay and expert perceptions of COVID-19 vaccine development timelines in Canada and USA.

OBJECTIVES: Compare lay expectations of medical development to those of experts in the context of SARS-CoV-2 vaccine development. METHODS: A short online survey of experts and lay people measuring when participants believe important vaccine milestones would occur and how likely potential setbacks were. Samples of US and Canadian lay people recruited through Qualtrics. The expert sample was created through a contact network in vaccine development and supplemented with corresponding authors of recent scholarly review articles on vaccine development. RESULTS: In aggregate, lay people gave responses that were within 3 months of experts, tending to be later than experts for early milestones and earlier for later milestones. Median lay best estimates for when a vaccine would be available to the public were 08/2021 and 09/2021 for the US and Canadian samples, compared with 09-10/2021 for the experts. However, many individual lay responses showed more substantial disagreement with expert opinions, with 54% of lay best estimates of when a vaccine would be available to the public being before the median expert soonest estimate or after the median expert latest estimate. Lay people were much more pessimistic about vaccine development encountering setbacks than experts (median probability 59% of boxed warning compared with only 30% for experts). Misalignment between layperson and expert expectations was not explained by any demographic variables collected in our survey. CONCLUSION: Median lay expectations were generally similar to experts. At the individual level, however, lay people showed substantial variation with many believing milestones would occur much sooner than experts. Lay people were in general much more pessimistic about the prospect of setbacks than were experts.