The Benefits and Risks of Receiving Investigational Solid Tumor Drugs in Randomized Trials : A Systematic Review and Meta-analysis.

BACKGROUND: Many patients participate in cancer trials to access new therapies. The extent to which new treatments produce clinical benefit for trial participants is unclear. PURPOSE: To estimate the progression-free survival (PFS) and overall survival (OS) advantage of assignment to experimental groups in randomized trials for 6 solid tumors. DATA SOURCES: ClinicalTrials.gov was searched for trials of investigational drugs with results posted between 2017 and 2021. STUDY SELECTION: Investigational drugs were defined as those not yet having full approval from the U.S. Food and Drug Administration for the study indication. Trials were included if they were randomized and tested drugs or biologics. DATA EXTRACTION: Data extraction was completed by 2 independent reviewers. Data were pooled using a random-effects model. DATA SYNTHESIS: The sample included 128 trials comprising 141 comparisons of a new drug and a comparator. These comparisons included 47 050 patients. The pooled hazard ratio for PFS was 0.80 (95% CI, 0.75 to 0.85), indicating statistically significant benefit for patients in experimental groups. This corresponded to a median PFS advantage of 1.25 months (CI, 0.80 to 1.68 months). The pooled hazard ratio for OS was 0.92 (CI, 0.88 to 0.95), corresponding to a survival gain of 1.18 months (CI, 0.72 to 1.71 months). The absolute risk for a serious adverse event for comparator group patients was 29.56% (CI, 26.64% to 32.65%), with an increase in risk of 7.40% (CI, 5.66% to 9.14%) for patients in experimental groups. LIMITATIONS: Trials in this sample were heterogeneous. Comparator group interventions were assumed to reflect standard of care. CONCLUSION: Assignment to experimental groups produces statistically significant survival gains. However, the absolute survival gain is small, and toxicity is statistically significantly greater. The findings of this review provide reassuring evidence that patients are not meaningfully disadvantaged by assignment to comparator groups. PRIMARY FUNDING SOURCE: Canadian Institutes of Health Research.

Research encouraging off-label use of quetiapine: A systematic meta-epidemiological analysis.

BACKGROUND: Researchers often conduct small studies on testing a drug's efficacy in off-label indications. If positive results from these exploratory studies are not followed up by larger, randomized, double-blinded trials, physicians cannot be sure of a drug's clinical value. This may lead to off-label prescriptions of ineffective treatments. We aim to describe the way clinical studies fostered off-label prescription of the antipsychotic drug quetiapine (Seroquel). METHODS: In this systematic meta-epidemiological analysis, we searched EMBASE, MEDLINE, Cochrane CENTRAL and PsycINFO databases and included clinical studies testing quetiapine for unapproved indications between May 1995 and May 2022. We then assessed the frequency with which publications providing low-level evidence suggesting efficacy of quetiapine for off-label indications was not followed up by large, randomized and double-blinded trials within 5 years. RESULTS: In total, 176 published studies were identified that reported potential efficacy of quetiapine in at least 26 indications. Between 2000 and 2007, publication of exploratory studies suggesting promise for off-label indications rapidly outpaced publication of confirmatory trials. In the 24 indications with a minimum of 5 years of follow-up from the first positive exploratory study, 19 (79%) were not followed up with large confirmatory trials within 5 years. At least nine clinical practice guidelines recommend the use of quetiapine for seven off-label indications in which published confirmatory evidence is lacking. CONCLUSION: Many small, post-approval studies suggested the promise of quetiapine for numerous off-label indications. These findings generally went unconfirmed in large, blinded, randomized trials years after first being published. The imbalance of exploratory and confirmatory studies likely encourages ineffective off-label treatment.

The impact of feedback training on prediction of cancer clinical trial results.

INTRODUCTION: Funders must make difficult decisions about which squared treatments to prioritize for randomized trials. Earlier research suggests that experts have no ability to predict which treatments will vindicate their promise. We tested whether a brief training module could improve experts' trial predictions. METHODS: We randomized a sample of breast cancer and hematology-oncology experts to the presence or absence of a feedback training module where experts predicted outcomes for five recently completed randomized controlled trials and received feedback on accuracy. Experts then predicted primary outcome attainment for a sample of ongoing randomized controlled trials. Prediction skill was assessed by Brier scores, which measure the average deviation between their predictions and actual outcomes. Secondary outcomes were discrimination (ability to distinguish between positive and non-positive trials) and calibration (higher predictions reflecting higher probability of trials being positive). RESULTS: A total of 148 experts (46 for breast cancer, 54 for leukemia, and 48 for lymphoma) were randomized between May and December 2017 and included in the analysis (1217 forecasts for 25 trials). Feedback did not improve prediction skill (mean Brier score for control: 0.22, 95% confidence interval = 0.20-0.24 vs feedback arm: 0.21, 95% confidence interval = 0.20-0.23; p = 0.51). Control and feedback arms showed similar discrimination (area under the curve = 0.70 vs 0.73, p = 0.24) and calibration (calibration index = 0.01 vs 0.01, p = 0.81). However, experts in both arms offered predictions that were significantly more accurate than uninformative forecasts of 50% (Brier score = 0.25). DISCUSSION: A short training module did not improve predictions for cancer trial results. However, expert communities showed unexpected ability to anticipate positive trials.Pre-registration record: https://aspredicted.org/4ka6r.pdf.

Survival Benefit Associated With Participation in Clinical Trials of Anticancer Drugs: A Systematic Review and Meta-Analysis.

Importance: Many cancer clinical investigators view clinical trials as offering better care for patients than routine clinical care. However, definitive evidence of clinical benefit from trial participation (hereafter referred to as the participation effect) has yet to emerge. Objective: To conduct a systematic review and meta-analysis of the evidence examining whether patient participation in cancer trials was associated with greater survival benefit compared with routine care. Data Sources: Studies were found through PubMed and Embase (January 1, 2000, until August 31, 2022), as well as backward and forward citation searching. Study Selection: Studies were included that compared overall survival of trial participants and routine care patients. Data Extraction and Synthesis: Data extraction and methodological quality assessment were completed by 2 independent coders using Covidence software. Data were pooled using a random-effects model and analyzed based on the quality of the comparison between trial participants and routine care patients (ie, extent to which studies controlled for bias and confounders). Main Outcomes and Measures: The hazard ratio (HR) for overall survival of trial participants vs routine care patients. Results: Thirty-nine publications were included, comprising 85 comparisons of trial participants and routine care patients. The meta-analysis revealed a statistically significant overall survival benefit for trial participants (HR, 0.76 [95% CI, 0.69-0.82]) when all studies were pooled, regardless of design or quality. However, survival benefits diminished in study subsets that matched trial participants and routine care patients for eligibility criteria (HR, 0.85 [95% CI, 0.75-0.97]) and disappeared when only high-quality studies were pooled (HR, 0.91 [95% CI, 0.80-1.05]). They also disappeared when estimates were adjusted for potential publication bias (HR, 0.94 [95% CI, 0.86-1.03]). Conclusions and Relevance: Many studies suggest a survival benefit for cancer trial participants. However, these benefits were not detected in studies using designs addressing important sources of bias and confounding. Pooled results of high-quality studies are not consistent with a beneficial effect of trial participation on its own.

Timing for First-in-Minor Clinical Trials of New Cancer Drugs.

OBJECTIVES: To describe the delay for first-in-minor cancer clinical trials and its relationship with the Food and Drug Administration (FDA) approval. STUDY DESIGN: We used ClinicalTrials.gov to create a sample of pediatric-relevant cancer drugs starting efficacy testing in adults from 2006 through 2011. We characterized the delay between first-in-adult efficacy trials and first-in-minor trials. We also assessed the proportion of drugs evaluated in minors that failed to gain approval, the proportions that were not evaluated in minors before receiving the FDA approval, and whether shorter delay was associated with larger effect sizes or greater probability of regulatory approval. RESULTS: Thirty-four percent of the 185 drugs in our cohort were evaluated in minors; the median delay to clinical trials was 4.16 years. Of all drugs, 17% received the FDA approval, 41% of which were never tested in minors before licensing. Of the 153 drugs not attaining approval, 78% were not evaluated in minors. Earlier testing did not significantly predict greater response rates (P = .13). Drugs not attaining regulatory approval were evaluated significantly earlier (3.0 for drugs not approved vs 5.4 years delayed testing for approved drugs, P = .019). CONCLUSIONS: New cancer drugs were typically evaluated in minors years after adult efficacy evaluation. This delay likely eliminated some drugs lacking desirable pharmacology before pediatric testing. However, some drugs that were eliminated may have had activity in pediatric indications. Approaches for prioritizing drugs for pediatric testing warrants further consideration.

Predicting Clinical Trial Results: A Synthesis of Five Empirical Studies and Their Implications.

Expectations about future events underlie practically every decision we make, including those in medical research. This paper reviews five studies undertaken to assess how well medical experts could predict the outcomes of clinical trials. It explains why expert trial forecasting was the focus of study and argues that forecasting skill affords insights into the quality of expert judgment and might be harnessed to improve decision-making in care, policy, and research. The paper also addresses potential criticisms of the research agenda and summarizes key findings from the five studies of trial forecasting. Together, the studies suggest that trials frequently deliver surprising results to expert communities and that individual experts are often uninformative when it comes to forecasting trial outcome and recruitment. However, the findings also suggest that expert forecasts often contain a "signal" about whether a trial will be positive, especially when forecasts are aggregated. The paper concludes with needs for further research and tentative policy recommendations.

Participant Recruitment From Low- and Middle-Income Countries for Pivotal Trials of Drugs Approved by the U.S. Food and Drug Administration : A Cross-Sectional Analysis.

BACKGROUND: Many participants in clinical trials supporting U.S. Food and Drug Administration (FDA) drug approvals are recruited from outside the United States, including from low- and middle-income countries (LMICs). Where participants are recruited for pivotal trials has implications for ethical research conduct and generalizability. OBJECTIVE: To describe LMIC recruitment for pivotal trials of newly approved drugs for cancer, neurologic disease, and cardiovascular disease. DESIGN: Cross-sectional analysis. SETTING: Pivotal trials of new cancer, cardiovascular, and neurologic drugs approved from 2012 to 2019 matched to ClinicalTrials.gov, FDA records, and publications. MEASUREMENTS: Host countries and available per country enrollments were extracted. The primary end point was the proportion of pivotal trials enrolling participants in LMICs. The secondary end point was the proportion of pivotal trial participants contributed by LMICs for each indication area. RESULTS: Data were obtained from 66 new drugs and 144 pivotal clinical trials. All cardiovascular approvals (12 drugs, 29 trials) and neurologic approvals (26 drugs, 54 trials) were analyzed, as well as a random sample of cancer approvals (28 of 85 drugs [33%]) matched to their pivotal trials (61 of 210 trials [29%]). Among the trials, 56% in cancer, 79% in cardiovascular disease, and 56% in neurology recruited from an LMIC. For multicountry trials, country-level enrollment figures were not available for 71 trials (55%). For those reporting per country enrollment, the percentage of participants recruited from LMICs was 8% for cancer trials, 36% for cardiovascular trials, and 17% for neurology trials. LIMITATIONS: The study was limited to FDA-approved drugs in 3 areas, including a sample of cancer drugs. Pivotal trials of nonapproved drugs or drugs for other indications were not captured. CONCLUSION: Most pivotal trials for FDA-approved drugs recruit from LMICs. Publications and FDA documents generally do not provide country-level data on recruitment. PRIMARY FUNDING SOURCE: None.

Success rates for US and Canadian anticancer drug development efforts in pediatric oncology.

BACKGROUND: Approximately 5% of adult cancer interventions put into clinical testing attain regulatory approval. Little is known about corresponding rates for pediatric cancer. METHODS: Our primary outcomes were the proportion of interventions graduating to the next trial phase, randomized trials, and/or clinical practice. We created a saturation sample of clinical trials by searching ClinicalTrials.gov for all pediatric anticancer trials in the United States or Canada. Trial characteristics were extracted automatically from ClinicalTrials.gov records, and cancer indication/drug class categorization, biomarker enrichment, and Food and Drug Administration (FDA) approval status at time of recruitment were double-extracted from each record. Regulatory approval status and labeling modifications for each intervention were determined by searching Drugs@FDA and the New Pediatric Labeling Information Database. RESULTS: Five hundred eighty-nine pediatric trials launched between 1987 and 2019 were captured. The overall probability of graduation was 17.0%; 18.9% of interventions graduated from phase 1 to phase 2 trials, and 1.6% of interventions graduated from phase 2 to phase 3 trials. The proportion of interventions advancing from phase 1 to FDA approval was 3.6%, and 1.9% of interventions tested in phase 1 advanced to a randomized phase 2 trial. Only biomarker enrichment was significantly predictive of graduation from phase 1 to phase 2 trials (p = .011). CONCLUSION: The proportion of interventions advancing from phase 1 testing to FDA approval was similar to estimates for adult oncology. Our findings highlight the challenges for current paradigms of pediatric anticancer drug development.

Practical steps to identifying the research risk of pragmatic trials.

BACKGROUND: Pragmatic randomized clinical trials that compare two or more purportedly "within the standard of care" interventions attempt to provide real-world evidence for policy and practice decisions. There is considerable debate regarding their research risk status, which in turn could lead to debates about appropriate consent requirements. Yet no practical guidance for identifying the research risks of pragmatic randomized clinical trials is available. METHODS: We developed a practical, four-step process for identifying and evaluating the research risk of pragmatic trials that can be applied to those pragmatic randomized clinical trials that compare two or more "standard of care" or "accepted" interventions. RESULTS: Using a variety of examples of standard of care pragmatic randomized clinical trials (ranging from trials comparing: insurance coverage conditions, patient reminders for health screens, intensive care unit procedures, post-stroke interventions, and drugs for life-threatening conditions), we illustrate in a four-step process how any pragmatic randomized clinical trial purportedly comparing standard interventions can be evaluated for their research risks. CONCLUSION: Although determining the risk status of a standard of care pragmatic randomized clinical trial is only one necessary element in the ethical oversight of such pragmatic randomized clinical trials, it is a central element. Our four-step process of pragmatic randomized clinical trial risk determination provides a practical, transparent, and systematic approach with likely low risk of bias.

Probability of Success and Timelines for the Development of Vaccines for Emerging and Reemerged Viral Infectious Diseases.

BACKGROUND: Anticipated success rates and timelines for COVID-19 vaccine development vary. Recent experience with developing and testing viral vaccine candidates can inform expectations regarding the development of safe and effective vaccines. OBJECTIVE: To estimate timelines and probabilities of success for recent vaccine candidates. DESIGN: ClinicalTrials.gov was searched to identify trials testing viral vaccines that had not advanced to phase 2 before 2005, and the progress of each vaccine from phase 1 through to U.S. Food and Drug Administration (FDA) licensure was tracked. Trial characteristics were double-coded. (Registration: Open Science Framework [https://osf.io/dmuzx/]). SETTING: Trials launched between January 2005 and March 2020. PARTICIPANTS: Preventive viral vaccine candidates for 23 emerging or reemerged viral infectious diseases. MEASUREMENTS: The primary end point was the probability of vaccines advancing from launch of phase 2 to FDA licensure within 10 years. RESULTS: In total, 606 clinical trials forming 220 distinct development trajectories (267 343 enrolled participants) were identified. The probability of vaccines progressing from phase 2 to licensure within 10 years was 10.0% (95% CI, 2.6% to 16.9%), with most approvals representing H1N1 or H5N1 vaccines. The average timeline from phase 2 to approval was 4.4 years (range, 6.4 weeks to 13.9 years). The probabilities of advancing from phase 1 to 2, phase 2 to 3, and phase 3 to licensure within the total available follow-up time were 38.2% (CI, 30.7% to 45.0%), 38.3% (CI, 23.1% to 50.5%), and 61.1% (CI, 3.7% to 84.3%), respectively. LIMITATIONS: The study did not account for preclinical development and relied primarily on ClinicalTrials.gov and FDA resources. Success probabilities do not capture the varied reasons why vaccines fail to advance to regulatory approval. CONCLUSION: Success probabilities and timelines varied widely across different vaccine types and diseases. If a SARS-CoV-2 vaccine is licensed within 18 months of the start of the pandemic, it will mark an unprecedented achievement for noninfluenza viral vaccine development. PRIMARY FUNDING SOURCE: McGill Interdisciplinary Initiative in Infection and Immunity (MI4) Emergency COVID-19 Research Funding program.

Can Oncologists Predict the Efficacy of Treatments in Randomized Trials?

BACKGROUND: Decisions about trial funding, ethical approval, or clinical practice guideline recommendations require expert judgments about the potential efficacy of new treatments. We tested whether individual and aggregated expert opinion of oncologists could predict reliably the efficacy of cancer treatments tested in randomized controlled trials. MATERIALS AND METHODS: An international sample of 137 oncologists specializing in genitourinary, lung, and colorectal cancer provided forecasts on primary outcome attainment for five active randomized cancer trials within their subspecialty; skill was assessed using Brier scores (BS), which measure the average squared deviation between forecasts and outcomes. RESULTS: A total of 40% of trials in our sample reported positive primary outcomes. Experts generally anticipated this overall frequency (mean forecast, 34%). Individual experts on average outperformed random predictions (mean BS = 0.29 [95% confidence interval (CI), 0.28-0.33] vs. 0.33) but underperformed prediction algorithms that always guessed 50% (BS = 0.25) or that were trained on base rates (BS = 0.19). Aggregating forecasts improved accuracy (BS = 0.25; 95% CI, 0.16-0.36]). Neither individual experts nor aggregated predictions showed appreciable discrimination between positive and nonpositive trials (area under the curve of a receiver operating characteristic curve, 0.52 and 0.43, respectively). CONCLUSION: These findings are based on a limited sample of trials. However, they reinforce the importance of basing research and policy decisions on the results of randomized trials rather than expert opinion or low-level evidence. IMPLICATIONS FOR PRACTICE: Predictions of oncologists, either individually or in the aggregate, did not anticipate reliably outcomes for randomized trials in cancer. These findings suggest that pooled expert opinion about treatment efficacy is no substitute for randomized trials. They also underscore the challenges of using expert opinion to prioritize interventions for clinical trials or to make recommendations in clinical practice guidelines.

The proportion of North American cancer trials that evaluate novel targets.

Major advances in cancer care often emerge from the development of novel targets. We randomly sampled 10% of cancer trials on clinicaltrials.gov with start dates 2013-2016 to determine the proportion of trials and research subjects directed at evaluating novel targets. We found that 87 of 378 trials (23.0%) enrolling 9225 of 44,525 patients (20.7%) tested interventions that are directed towards novel targets. 146 of 378 trials (38.6%) enrolling 19,132 of 44,525 patients (43.0%) investigated treatments that were not FDA approved but utilized a previously studied target for treating cancer. Combined, 233 of 378 trials (61.6%) enrolling 28,357 of 44,525 patients (63.9%) investigated treatments that were not FDA approved. Furthermore, 36 of 378 trials (9.5%) enrolling 6592 of 44,525 patients (14.8%) investigated FDA approved anticancer drugs in their approved indication and combination while 109 of 378 trials (28.8%) enrolling 9576 of 44,525 patients (21.5%) investigated FDA approved anticancer drugs outside of their approved indication or combination. Logistic regression found that phase 1 trials were significantly more likely to test novel target interventions than phase 2 and 3 trials (p value = 0.00197 and 0.00130 respectively). Industry sponsored trials were also significantly more likely to involve novel target interventions than non-industry trials (p value <0.001). In conclusion, most cancer trials involve unapproved treatments, but a majority of these treatments are well-characterized or involve a previously studied target to treat cancer.

Comparison of Patient and Expert Perceptions of the Attainment of Research Milestones in Parkinson's Disease.

BACKGROUND: Commentators suggest that patients have unrealistic expectations about the pace of research advances and that such expectations interfere with patient decision-making. OBJECTIVE: The objective of this study was to compare expert expectations about the timing of research milestone attainment with those of patients who follow Parkinson's disease (PD) research. METHODS: Patients with PD and experts were asked to provide forecasts about 11 milestones in PD research in an online survey. PD experts were identified from a Michael J. Fox Foundation database, highly ranked neurology centers in the United States and Canada, and corresponding authors of articles on PD in top medical journals. Patients with PD were recruited through the Michael J. Fox Foundation. We tested whether patient forecasts differed on average from expert forecasts. We also tested whether differences between patient forecasts and the average expert forecasts were associated with any demographic factors. RESULTS: A total of 256 patients and 249 PD experts completed the survey. For 9 of the 11 milestones, patients' forecasts were on average higher than those of experts. Only exercise therapy met our 10% difference threshold for practical significance. Education was the only demographic that predicted patient deviations from expert forecasts on milestone forecasts. Patients offered significantly higher forecasts than experts that the clinical trials used in milestone queries would report positive primary outcomes. CONCLUSIONS: Differences between patient and expert expectations about research milestones were generally minor, suggesting that there is little cause for concern that patients who follow PD research are unduly swayed by inaccurate representations of research advancement in the media or elsewhere. © 2020 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Preclinical efficacy studies in investigator brochures: Do they enable risk-benefit assessment?

Human protection policies require favorable risk-benefit judgments prior to launch of clinical trials. For phase I and II trials, evidence for such judgment often stems from preclinical efficacy studies (PCESs). We undertook a systematic investigation of application materials (investigator brochures [IBs]) presented for ethics review for phase I and II trials to assess the content and properties of PCESs contained in them. Using a sample of 109 IBs most recently approved at 3 institutional review boards based at German Medical Faculties between the years 2010-2016, we identified 708 unique PCESs. We then rated all identified PCESs for their reporting on study elements that help to address validity threats, whether they referenced published reports, and the direction of their results. Altogether, the 109 IBs reported on 708 PCESs. Less than 5% of all PCESs described elements essential for reducing validity threats such as randomization, sample size calculation, and blinded outcome assessment. For most PCESs (89%), no reference to a published report was provided. Only 6% of all PCESs reported an outcome demonstrating no effect. For the majority of IBs (82%), all PCESs were described as reporting positive findings. Our results show that most IBs for phase I/II studies did not allow evaluators to systematically appraise the strength of the supporting preclinical findings. The very rare reporting of PCESs that demonstrated no effect raises concerns about potential design or reporting biases. Poor PCES design and reporting thwart risk-benefit evaluation during ethical review of phase I/II studies.

The bench is closer to the bedside than we think: Uncovering the ethical ties between preclinical researchers in translational neuroscience and patients in clinical trials.

Millions of people worldwide currently suffer from serious neurological diseases and injuries for which there are few, and often no, effective treatments. The paucity of effective interventions is, no doubt, due in large part to the complexity of the disorders, as well as our currently limited understanding of their pathophysiology. The bleak picture for patients, however, is also attributable to avoidable impediments stemming from quality concerns in preclinical research that often escape detection by research regulation efforts. In our essay, we connect the dots between these concerns about the quality of preclinical research and their potential ethical impact on the patients who volunteer for early trials of interventions informed by it. We do so in hopes that a greater appreciation among preclinical researchers of these serious ethical consequences can lead to a greater commitment within the research community to adopt widely available tools and measures that can help to improve the quality of research.

The Effect of Overdose Education and Naloxone Distribution: An Umbrella Review of Systematic Reviews.

Background. Opioids contribute to more than 60 000 deaths annually in North America. While the expansion of overdose education and naloxone distribution (OEND) programs has been recommended in response to the opioid crisis, their effectiveness remains unclear. Objectives. To conduct an umbrella review of systematic reviews to provide a broad-based conceptual scheme of the effect and feasibility of OEND and to identify areas for possible optimization. Search Methods. We conducted the umbrella review of systematic reviews by searching PubMed, Embase, PsycINFO, Epistemonikos, the Cochrane Database of Systematic Reviews, and the reference lists of relevant articles. Briefly, an academic librarian used a 2-concept search, which included opioid subject headings and relevant keywords with a modified PubMed systematic review filter. Selection Criteria. Eligible systematic reviews described comprehensive search strategies and inclusion and exclusion criteria, evaluated the quality or risk of bias of included studies, were published in English or French, and reported data relevant to either the safety or effectiveness of OEND programs, or optimal strategies for the management of opioid overdose with naloxone in out-of-hospital settings. Data Collection and Analysis. Two reviewers independently extracted study characteristics and the quality of included reviews was assessed in duplicate with AMSTAR-2, a critical appraisal tool for systematic reviews. Review quality was rated critically low, low, moderate, or high based on 7 domains: protocol registration, literature search adequacy, exclusion criteria, risk of bias assessment, meta-analytical methods, result interpretation, and presence of publication bias. Summary tables were constructed, and confidence ratings were provided for each outcome by using a previously modified version of the Royal College of General Practitioners' clinical guidelines. Main Results. Six systematic reviews containing 87 unique studies were included. We found that OEND programs produce long-term knowledge improvement regarding opioid overdose, improve participants' attitudes toward naloxone, provide sufficient training for participants to safely and effectively manage overdoses, and effectively reduce opioid-related mortality. High-concentration intranasal naloxone (> 2 mg/mL) was as effective as intramuscular naloxone at the same dose, whereas lower-concentration intranasal naloxone was less effective. Evidence was limited for other naloxone formulations, as well as the need for hospital transport after overdose reversal. The preponderance of evidence pertained persons who use heroin. Author's Conclusions. Evidence suggests that OEND programs are effective for reducing opioid-related mortality; however, additional high-quality research is required to optimize program delivery. Public Health Implications. Community-based OEND programs should be implemented widely in high-risk populations.

The Effect of Overdose Education and Naloxone Distribution: An Umbrella Review of Systematic Reviews.

Background. Opioids contribute to more than 60 000 deaths annually in North America. While the expansion of overdose education and naloxone distribution (OEND) programs has been recommended in response to the opioid crisis, their effectiveness remains unclear. Objectives. To conduct an umbrella review of systematic reviews to provide a broad-based conceptual scheme of the effect and feasibility of OEND and to identify areas for possible optimization. Search Methods. We conducted the umbrella review of systematic reviews by searching PubMed, Embase, PsycINFO, Epistemonikos, the Cochrane Database of Systematic Reviews, and the reference lists of relevant articles. Briefly, an academic librarian used a 2-concept search, which included opioid subject headings and relevant keywords with a modified PubMed systematic review filter. Selection Criteria. Eligible systematic reviews described comprehensive search strategies and inclusion and exclusion criteria, evaluated the quality or risk of bias of included studies, were published in English or French, and reported data relevant to either the safety or effectiveness of OEND programs, or optimal strategies for the management of opioid overdose with naloxone in out-of-hospital settings. Data Collection and Analysis. Two reviewers independently extracted study characteristics and the quality of included reviews was assessed in duplicate with AMSTAR-2, a critical appraisal tool for systematic reviews. Review quality was rated critically low, low, moderate, or high based on 7 domains: protocol registration, literature search adequacy, exclusion criteria, risk of bias assessment, meta-analytical methods, result interpretation, and presence of publication bias. Summary tables were constructed, and confidence ratings were provided for each outcome by using a previously modified version of the Royal College of General Practitioners' clinical guidelines. Main Results. Six systematic reviews containing 87 unique studies were included. We found that OEND programs produce long-term knowledge improvement regarding opioid overdose, improve participants' attitudes toward naloxone, provide sufficient training for participants to safely and effectively manage overdoses, and effectively reduce opioid-related mortality. High-concentration intranasal naloxone (> 2 mg/mL) was as effective as intramuscular naloxone at the same dose, whereas lower-concentration intranasal naloxone was less effective. Evidence was limited for other naloxone formulations, as well as the need for hospital transport after overdose reversal. The preponderance of evidence pertained persons who use heroin. Author's Conclusions. Evidence suggests that OEND programs are effective for reducing opioid-related mortality; however, additional high-quality research is required to optimize program delivery. Public Health Implications. Community-based OEND programs should be implemented widely in high-risk populations.

Can cancer researchers accurately judge whether preclinical reports will reproduce?

There is vigorous debate about the reproducibility of research findings in cancer biology. Whether scientists can accurately assess which experiments will reproduce original findings is important to determining the pace at which science self-corrects. We collected forecasts from basic and preclinical cancer researchers on the first 6 replication studies conducted by the Reproducibility Project: Cancer Biology (RP:CB) to assess the accuracy of expert judgments on specific replication outcomes. On average, researchers forecasted a 75% probability of replicating the statistical significance and a 50% probability of replicating the effect size, yet none of these studies successfully replicated on either criterion (for the 5 studies with results reported). Accuracy was related to expertise: experts with higher h-indices were more accurate, whereas experts with more topic-specific expertise were less accurate. Our findings suggest that experts, especially those with specialized knowledge, were overconfident about the RP:CB replicating individual experiments within published reports; researcher optimism likely reflects a combination of overestimating the validity of original studies and underestimating the difficulties of repeating their methodologies.

Inefficiencies and Patient Burdens in the Development of the Targeted Cancer Drug Sorafenib: A Systematic Review.

Failure in cancer drug development exacts heavy burdens on patients and research systems. To investigate inefficiencies and burdens in targeted drug development in cancer, we conducted a systematic review of all prelicensure trials for the anticancer drug, sorafenib (Bayer/Onyx Pharmaceuticals). We searched Embase and MEDLINE databases on October 14, 2014, for prelicensure clinical trials testing sorafenib against cancers. We measured risk by serious adverse event rates, benefit by objective response rates and survival, and trial success by prespecified primary endpoint attainment with acceptable toxicity. The first two clinically useful applications of sorafenib were discovered in the first 2 efficacy trials, after five drug-related deaths (4.6% of 108 total) and 93 total patient-years of involvement (2.4% of 3,928 total). Thereafter, sorafenib was tested in 26 indications and 67 drug combinations, leading to one additional licensure. Drug developers tested 5 indications in over 5 trials each, comprising 56 drug-related deaths (51.8% of 108 total) and 1,155 patient-years (29.4% of 3,928 total) of burden in unsuccessful attempts to discover utility against these malignancies. Overall, 32 Phase II trials (26% of Phase II activity) were duplicative, lacked appropriate follow-up, or were uninformative because of accrual failure, constituting 1,773 patients (15.6% of 11,355 total) participating in prelicensure sorafenib trials. The clinical utility of sorafenib was established early in development, with low burden on patients and resources. However, these early successes were followed by rapid and exhaustive testing against various malignancies and combination regimens, leading to excess patient burden. Our evaluation of sorafenib development suggests many opportunities for reducing costs and unnecessary patient burden in cancer drug development.

Increasing efficiency of preclinical research by group sequential designs.

Despite the potential benefits of sequential designs, studies evaluating treatments or experimental manipulations in preclinical experimental biomedicine almost exclusively use classical block designs. Our aim with this article is to bring the existing methodology of group sequential designs to the attention of researchers in the preclinical field and to clearly illustrate its potential utility. Group sequential designs can offer higher efficiency than traditional methods and are increasingly used in clinical trials. Using simulation of data, we demonstrate that group sequential designs have the potential to improve the efficiency of experimental studies, even when sample sizes are very small, as is currently prevalent in preclinical experimental biomedicine. When simulating data with a large effect size of d = 1 and a sample size of n = 18 per group, sequential frequentist analysis consumes in the long run only around 80% of the planned number of experimental units. In larger trials (n = 36 per group), additional stopping rules for futility lead to the saving of resources of up to 30% compared to block designs. We argue that these savings should be invested to increase sample sizes and hence power, since the currently underpowered experiments in preclinical biomedicine are a major threat to the value and predictiveness in this research domain.