The impact of feedback training on prediction of cancer clinical trial results.

INTRODUCTION: Funders must make difficult decisions about which squared treatments to prioritize for randomized trials. Earlier research suggests that experts have no ability to predict which treatments will vindicate their promise. We tested whether a brief training module could improve experts' trial predictions. METHODS: We randomized a sample of breast cancer and hematology-oncology experts to the presence or absence of a feedback training module where experts predicted outcomes for five recently completed randomized controlled trials and received feedback on accuracy. Experts then predicted primary outcome attainment for a sample of ongoing randomized controlled trials. Prediction skill was assessed by Brier scores, which measure the average deviation between their predictions and actual outcomes. Secondary outcomes were discrimination (ability to distinguish between positive and non-positive trials) and calibration (higher predictions reflecting higher probability of trials being positive). RESULTS: A total of 148 experts (46 for breast cancer, 54 for leukemia, and 48 for lymphoma) were randomized between May and December 2017 and included in the analysis (1217 forecasts for 25 trials). Feedback did not improve prediction skill (mean Brier score for control: 0.22, 95% confidence interval = 0.20-0.24 vs feedback arm: 0.21, 95% confidence interval = 0.20-0.23; p = 0.51). Control and feedback arms showed similar discrimination (area under the curve = 0.70 vs 0.73, p = 0.24) and calibration (calibration index = 0.01 vs 0.01, p = 0.81). However, experts in both arms offered predictions that were significantly more accurate than uninformative forecasts of 50% (Brier score = 0.25). DISCUSSION: A short training module did not improve predictions for cancer trial results. However, expert communities showed unexpected ability to anticipate positive trials.Pre-registration record: https://aspredicted.org/4ka6r.pdf.

Design of placebo-controlled randomized trials of anticancer agents: Ethical considerations based on a review of published trials.

BACKGROUND: Limited information exists about the design of placebo-controlled cancer trials. Through a systematic review of trials published in 2013, we describe placebo use in randomized trials testing anticancer agents and analyze strategies that increase exposure to the experimental regimen. METHODS: Trials were classified as add-on (placebo in combination with standard treatment) or placebo-only. Strategies to allow more than half of the participants to receive the experimental regimen were reviewed. The risk-benefit ratio of receiving the experimental agent was considered favorable if the difference in primary outcome was significant (p ≤ 0.05), neutral if there was no significant difference in the primary outcome and the experimental agent did not add substantial toxicity, and unfavorable otherwise. RESULTS: Eighty trials were included (32,694 participants). Most trials were add-on (69%). The risk-benefit outcome was favorable, neutral, and unfavorable to the experimental agent in 52%, 32%, and 16% of placebo-only trials and 25%, 53%, and 22%, respectively, of add-on trials. Four strategies increased exposure to the experimental regimen: one-way crossover (23%), uneven randomization (21%), three-arms (13%), and randomized discontinuation design (4%); these strategies were used more often in placebo-only trials. CONCLUSION: A minority of participants received placebo alone and strategies to increase experimental exposure were used commonly. Fewer than half of the studies had favorable outcomes, thus defending the use of placebo controls, when there is no established treatment. Strategies that increase patient exposure to experimental agents rather than placebo may expose them to non-beneficial, sometimes toxic, experimental agents.

Benefit, burden, and impact for a cohort of post-approval cancer combination trials.

BACKGROUND: After approval, drug developers often pursue trials aimed at extending the uses of a new drug by combining it with other drugs. Little is known about the risk and benefits associated with such research. METHODS: To establish a historic benchmark of risk and benefit, we searched Medline and Embase for clinical trials testing anti-cancer drugs in combination within 5 years of approval by the Food and Drug Administration of 12 anti-cancer "index" drugs first licensed 2005-2007 inclusive. Risk was assessed based on grade 3 or above drug-related adverse events; benefit was assessed based on efficacy outcomes and advancement of combinations into clinical practice guidelines or approval by the Food and Drug Administration. RESULTS: We captured 323 published post-approval trials exploring combinations, including 266 unique combination-indication pairings and enrolling 29,835 patients. The pooled risk ratios for treatment-related grade 3-4 severe adverse events and deaths attributed to the study drugs for trials randomized between a combination arm and a comparator were 1.54 (1.33-1.79) and 1.51 (1.16-1.97), respectively. The pooled hazard ratios for overall survival and progression-free survival were 0.99 (0.92-1.05) and 0.85 (0.79-0.93), respectively. None of the combination-indication pairings launched after initial drug approval received approval by the Food and Drug Administration, and 13 pairings (4.9%) were recommended by the National Comprehensive Cancer Network within 5 years of the first trial within that pairing. The proportion of patients in our sample who participated in trials leading to an approval by the Food and Drug Administration or a National Comprehensive Cancer Network guideline recommendation was 12.7% with 5 years of follow-up, and 22.3% among pairings for which there were 8 years of follow-up. CONCLUSION: Patients were just as likely to benefit in the treatment arm as the control arm in terms of overall survival, but they were more likely to experience a treatment-related severe adverse event in post-approval trials of combination therapy.

Comparison between protocols and publications for prognostic and predictive cancer biomarker studies.

BACKGROUND: Method prespecification in study protocols is important for controlling bias in reports. The primary goal of this study was to assess potential for discordance between study protocols and publications reporting predictive or prognostic cancer biomarker research. Secondary objectives included comparing characteristics of publications with accessible protocols compared to those without. METHODS: Publications reporting predictive or prognostic cancer biomarker research were identified from 15 major journals, 2012-2015. Protocols were sought online or through repeated queries of corresponding authors. The following four items were extracted: (1) biomarkers, (2) biospecimen/assays, (3) sample size, (4) endpoints. We defined "explicit discordance" as the presence of major inconsistencies on these items. RESULTS: Of 149 eligible publications, we obtained 19 eligible protocols online (13%). Out of a random sample of 103 publications where protocols were not available online, 12 protocols (12%) were furnished by corresponding authors; 8 (8% of authors) explicitly stated the absence of a protocol. Among 24 retrospective cohort studies, no protocol could be accessed. We found explicit discordance between publications and protocols for 18 studies (58%), in particular choice of biomarkers (36%), biospecimen/assays (6%), or endpoints (29%). CONCLUSION: Protocols are generally not accessible or not used for cancer biomarker studies. Publications were often explicitly discordant with protocols, particularly regarding biomarkers and endpoints. Our findings point to common unaddressed risk of bias in publications of major journals reporting the relationship between cancer biomarkers and clinical endpoints.

Organ preservation for rectal cancer (GRECCAR 2): a prospective, randomised, open-label, multicentre, phase 3 trial.

BACKGROUND: Organ preservation is a concept proposed for patients with rectal cancer after a good clinical response to neoadjuvant chemotherapy, to potentially avoid morbidity and side-effects of rectal excision. The objective of this study was to compare local excision and total mesorectal excision in patients with a good response after chemoradiotherapy for lower rectal cancer. METHODS: We did a prospective, randomised, open-label, multicentre, phase 3 trial at 15 tertiary centres in France that were experts in the treatment of rectal cancer. Patients aged 18 years and older with stage T2T3 lower rectal carcinoma, of maximum size 4 cm, who had a good clinical response to neoadjuvant chemoradiotherapy (residual tumour ≤2 cm) were centrally randomly assigned by the surgeon before surgery to either local excision or total mesorectal excision surgery. Randomisation, which was done via the internet, was not stratified and used permuted blocks of size eight. In the local excision group, a completion total mesorectal excision was required if tumour stage was ypT2-3. The primary endpoint was a composite outcome of death, recurrence, morbidity, and side-effects at 2 years after surgery, to show superiority of local excision over total mesorectal excision in the modified intention-to-treat (ITT) population (expected proportions of patients having at least one event were 25% vs 60% for superiority). This trial was registered with ClinicalTrials.gov, number NCT00427375. FINDINGS: From March 1, 2007, to Sept 24, 2012, 186 patients received chemoradiotherapy and were enrolled in the study. 148 good clinical responders were randomly assigned to treatment, three were excluded (because they had metastatic disease, tumour >8 cm from anal verge, and withdrew consent), and 145 were analysed: 74 in the local excision group and 71 in the total mesorectal excision group. In the local excision group, 26 patients had a completion total mesorectal excision. At 2 years in the modified ITT population, one or more events from the composite primary outcome occurred in 41 (56%) of 73 patients in the local excision group and 33 (48%) of 69 in the total mesorectal excision group (odds ratio 1·33, 95% CI 0·62-2·86; p=0·43). In the modified ITT analysis, there was no difference between the groups in all components of the composite outcome, and superiority was not shown for local excision over total mesorectal excision. INTERPRETATION: We failed to show superiority of local excision over total mesorectal excision, because many patients in the local excision group received a completion total mesorectal excision that probably increased morbidity and side-effects, and compromised the potential advantages of local excision. Better patient selection to avoid unnecessary completion total mesorectal excision could improve the strategy. FUNDING: National Cancer Institute of France, Sanofi, Roche Pharma.

Deciphering assumptions about stepped wedge designs: the case of Ebola vaccine research.

Ethical concerns about randomising persons to a no-treatment arm in the context of Ebola epidemic led to consideration of alternative designs. The stepped wedge (SW) design, in which participants or clusters are randomised to receive an intervention at different time points, gained popularity. Common arguments in favour of using this design are (1) when an intervention is likely to do more good than harm, (2) all participants should receive the experimental intervention at some time point during the study and (3) the design might be preferable for practical reasons. We examine these assumptions when considering Ebola vaccine research. First, based on the claim that a stepped wedge design is indicated when it is likely that the intervention will do more good than harm, we reviewed published and ongoing SW trials to explore previous use of this design to test experimental drugs or vaccines, and found that SW design has never been used for trials of experimental drugs or vaccines. Given that Ebola vaccines were all experimental with no prior efficacy data, the use of a stepped wedge design would have been unprecedented. Second, we show that it is rarely true that all participants receive the intervention in SW studies, but rather, depending on certain design features, all clusters receive the intervention. Third, we explore whether the SW design is appealing for feasibility reasons and point out that there is significant complexity. In the setting of the Ebola epidemic, spatiotemporal variation may have posed problematic challenges to a stepped wedge design for vaccine research. Finally, we propose a set of points to consider for scientific reviewers and ethics committees regarding proposals for SW designs.

Dose finding with longitudinal data: simpler models, richer outcomes.

Phase I oncology clinical trials are designed to identify the optimal dose that will be recommended for phase II trials. This dose is typically defined as the dose associated with a certain probability of severe toxicity at cycle 1, although toxicity is repeatedly measured over cycles on an ordinal scale. Recently, a proportional odds mixed-effect model for ordinal outcomes has been proposed to (i) identify the optimal dose accounting for repeated events and (ii) to provide some framework to explore time trend. We compare this approach to a method based on repeated binary variables and to a method based on an under-parameterized model of the dose-time toxicity relationship. We show that repeated binary and ordinal outcomes both improve the accuracy of dose-finding trials in the same proportion; ordinal outcomes are, however, superior to detect time trend even in the presence of nonproportional odds models. Moreover, less parameterized models led to the best operating characteristics. These approaches are illustrated on two dose-finding phase I trials. Integration of repeated measurements is appealing in phase I dose-finding trials.

Fresh-frozen plasma transfusion did not reduce 30-day mortality in patients undergoing cardiopulmonary bypass cardiac surgery with excessive bleeding: the PLASMACARD multicenter cohort study.

BACKGROUND: During on-pump cardiac surgery, hemorrhagic complications occur frequently. Fresh-frozen plasma (FFP) is widely transfused to provide coagulation factors. Yet, no randomized clinical trial has demonstrated its benefits on mortality. We assessed the relationship between therapeutic transfusion of FFP and 30-day mortality in cardiac surgery patients suffering from excessive bleeding in a prospective cohort study. STUDY DESIGN AND METHODS: Adult patients who underwent on-pump cardiac surgery and experienced excessive bleeding during the 48-hour perioperative period were recruited from 15 French centers between February 2004 and January 2006. Patients who received a preventive FFP transfusion were excluded. The association between FFP transfusion and all cause 30-day mortality was estimated using a Cox proportional hazards model, adjusted for confounding. A propensity score (PS) sensitivity analysis was also performed. RESULTS: Among 967 patients included in this study, 58.1% received FFP. The median dose was 11.3 mL/kg (interquartile range, 7.6-19.5). The cumulative 30-day mortality rate was 11.3% (95% confidence interval [CI], 9.5-13.5). FFP transfusion was associated with a higher 30-day mortality (hazard ratio [HR], 3.2; 95% CI, 1.7-6.1) in univariate analysis; however, after adjusting for prognostic factors, there was no longer any association (HR, 1.5; 95% CI, 0.8-3.0, p = 0.20). The results of the PS analysis were consistent with the adjusted analysis. CONCLUSION: Among on-pump cardiac surgery patients experiencing excessive perioperative bleeding, there is no evidence of a beneficial impact of FFP transfusion on mortality.

Dose-finding design using mixed-effect proportional odds model for longitudinal graded toxicity data in phase I oncology clinical trials.

Phase I oncology clinical trials are designed to identify the optimal dose that will be recommended for phase II trials. This dose is typically defined as the dose associated with a certain probability of severe toxicity during the first cycle of treatment, although toxicity is repeatedly measured over cycles on an ordinal scale. We propose a new adaptive dose-finding design using longitudinal measurements of ordinal toxic adverse events, with proportional odds mixed-effect models. Likelihood-based inference is implemented. The optimal dose is then the dose producing a target rate of severe toxicity per cycle. This model can also be used to identify cumulative or late toxicities. The performances of this approach were compared with those of the continual reassessment method in a simulation study. Operating characteristics were evaluated in terms of correct identification of the target dose, distribution of the doses allocated and power to detect trends in the risk of toxicities over time. This approach was also used to reanalyse data from a phase I oncology trial. Use of a proportional odds mixed-effect model appears to be feasible in phase I dose-finding trials, increases the ability of selecting the correct dose and provides a tool to detect cumulative effects.

Pseudomonas aeruginosa acquisition on an intensive care unit: relationship between antibiotic selective pressure and patients' environment.

INTRODUCTION: The purpose of this study was to investigate the relationship among Pseudomonas aeruginosa acquisition on the intensive care unit (ICU), environmental contamination and antibiotic selective pressure against P. aeruginosa. METHODS: An open, prospective cohort study was carried out in a 16-bed medical ICU where P. aeruginosa was endemic. Over a six-month period, all patients without P. aeruginosa on admission and with a length of stay >72 h were included. Throat, nasal, rectal, sputum and urine samples were taken on admission and at weekly intervals and screened for P. aeruginosa. All antibiotic treatments were recorded daily. Environmental analysis included weekly tap water specimen culture and the presence of other patients colonized with P. aeruginosa. RESULTS: A total of 126 patients were included, comprising 1,345 patient-days. Antibiotics were given to 106 patients (antibiotic selective pressure for P. aeruginosa in 39). P. aeruginosa was acquired by 20 patients (16%) and was isolated from 164/536 environmental samples (31%). Two conditions were independently associated with P. aeruginosa acquisition by multivariate analysis: (i) patients receiving ≥3 days of antibiotic selective pressure together with at least one colonized patient on the same ward on the previous day (odds ratio (OR) = 10.3 ((% confidence interval (CI): 1.8 to 57.4); P = 0.01); and (ii) presence of an invasive device (OR = 7.7 (95% CI: 2.3 to 25.7); P = 0.001). CONCLUSIONS: Specific interaction between both patient colonization pressure and selective antibiotic pressure is the most relevant factor for P. aeruginosa acquisition on an ICU. This suggests that combined efforts are needed against both factors to decrease colonization with P. aeruginosa.

Patient Participation in Clinical Trials of Oncology Drugs and Biologics Preceding Approval by the US Food and Drug Administration.

Importance: Several studies have estimated the financial inputs for successful drug development. Such analyses do not capture the large investment that patient study participants commit to drug development. Objective: To estimate the volume of patients required to achieve a first US Food and Drug Administration (FDA) approval for a new anticancer drug or biologic therapy. Design, Setting, and Participants: This cohort study included a random sample of prelicense oncology drugs and biologics with a trial site in the United States that were launched into clinical efficacy testing between January 1, 2006, and December 31, 2010. Drugs and biologics were identified using ClinicalTrials.gov registration records. Total patient enrollment was captured over an 8-year span, and each intervention was classified based on whether it received FDA approval and was deemed as having intermediate or substantial value according to the American Society of Clinical Oncology Value Framework (ASCO-VF) score. Secondarily, the association between patient numbers and intervention characteristics was tested. Data were analyzed in February 2020. Main Outcomes and Measure: The prespecified primary outcome was the number of patients enrolled in prelicense trials per FDA approval. Results: A total of 120 drugs and biologics were included in our study, with 84 (70.0%) targeted agents, 20 (16.7%) immunotherapies, and 71 (59.2%) novel agents. A total of 13 drugs and biologics (10.8%; 95% CI, 5.3%-16.8%) in our sample gained FDA approval within 8 years, of which 1 (7.7%) was deemed of intermediate value and 3 (23.1%) were deemed of substantial value using ASCO-VF scoring. Overall, 158 810 patients were enrolled in 1335 trials testing these drugs and biologics, 47 913 (30.2%) in trials that led to FDA approval and 110 897 (69.8%) in trials that did not. An estimated 12 217 (95% CI, 7970-22 215) patient study participants contributed to prelicense trials per FDA approval. The estimated number of patients needed to produce a single FDA-approved drug or biologic of intermediate or substantial ASCO-VF clinical value was 39 703 (95% CI, 19 391-177 991). Conclusions and Relevance: The results of this cohort study make visible the substantial patient investment required for prelicense oncology drug development. Such analyses can be used to devise policies that maximize the clinical impact of research on a per-patient basis.

Patient burden and clinical advances associated with postapproval monotherapy cancer drug trials: a retrospective cohort study.

OBJECTIVES: After regulatory approval, drug companies, public funding agencies and academic researchers often pursue trials aimed at extending the uses of a new drug by testing it in new non-approved indications. Patient burden and clinical impact of such research are not well understood. DESIGN AND SETTING: We conducted a retrospective cohort study of postapproval clinical trials launched within 5 years after the drug's first approval, testing anticancer drugs in monotherapy in indications that were first pursued after a drug's first Food and Drug Administration (FDA) license, for all 12 anticancer drugs approved between 2005 and 2007. FDA, Medline and Embase search date 2019 February 12. PRIMARY AND SECONDARY OUTCOME MEASURES: Our primary objective was to measure burden and clinical impact for patients enrolling in these trials. Each trial was sorted into a 'trajectory' defined by the drug and cancer indication. The risk was operationalised by proportions of grade 3-4 severe adverse events and deaths. The clinical impact was measured by estimating the proportion of patients participating in trajectories that resulted in FDA approval, uptake into National Comprehensive Cancer Network (NCCN) clinical practice guidelines or advancement to randomised controlled trials within 8 years. RESULTS: Our search captured 104 published trials exploring monotherapy, including 69 unique trajectories. In total, trials in our sample enrolled 4699 patients. Grade 3-4 adverse events were experienced by 19.6% of patients; grade 5 events were experienced by 2.8% of patients. None of the trajectories launched after initial drug approval received FDA approval. Five trajectories were recommended by the NCCN within 8 years of the first trial within that trajectory. Eleven trajectories were advanced to randomised controlled testing. CONCLUSIONS: The challenges associated with unlocking new applications for drugs that first received approval from 2005 to 2007 were similar to those for developing new drugs altogether. Our findings can help inform priority setting in research and provide a basis for calibrating expectations when considering enrolment in label-extending trials.

Assessment of Pregabalin Postapproval Trials and the Suggestion of Efficacy for New Indications: A Systematic Review.

Importance: After a drug receives regulatory approval, researchers often pursue small, underpowered trials, called exploratory trials, aimed at testing additional indications. If favorable early findings from exploratory trials are not promptly followed by confirmatory trials, then physicians, patients, and payers can be left uncertain about a drug's clinical value (clinical agnosticism). Such findings may encourage the off-label use of ineffective drugs. Objective: To characterize the relationship between exploratory and confirmatory postapproval trials for the blockbuster drug, pregabalin (Lyrica). Evidence Review: Ovid MEDLINE and Embase databases were used to identify clinical trials published prior to January 2018 and that tested the efficacy of pregabalin for nonapproved indications. Indications, trial outcomes, publication dates, and trial design elements were recorded. Time elapsed was calculated between the generation of clinical agnosticism about pregabalin (ie, publications reporting positive or inconclusive evidence of efficacy on a primary endpoint) and it being addressed (publication of at least 1 confirmatory trial in the same indication, regardless of outcome). Findings: There were 238 trials identified that tested the efficacy of pregabalin in at least 33 indications; 5 indications eventually received European Medicines Agency and/or US Food and Drug Administration marketing approval. Sixty-seven percent (22 of 33) of first publications for new indications may have generated clinical agnosticism. Of those indications with at least 5 years of follow-up, 63% (17 of 27) may have generated agnosticism that was not addressed in confirmatory trials within 5 years. As pregabalin development expanded from indications that received regulatory approval to other indications, the linkage of exploratory to confirmatory trial publication diminished. Conclusions and Relevance: After initial approval, exploratory evidence suggesting the value of pregabalin for new indications often went unconfirmed for extended periods of time. Poor coordination between exploratory and confirmatory testing may represent an important vehicle through which off-label prescription is recommended in clinical practice guidelines and encouraged in the absence of confirmatory trial evidence.

Prediction of cytomegalovirus (CMV) plasma load from evaluation of CMV whole-blood load in samples from renal transplant recipients.

In a prospective cohort of 82 renal transplant recipients, we evaluated the capacity of the cytomegalovirus (CMV) load in whole blood (WB) to predict the plasma CMV load, aiming to identify active CMV infections by using WB samples only and to deduce a WB threshold. Using quantitative real-time PCR, a total of 1,474 WB samples were assayed, of which 279 were positive for CMV, and 140 out of the 276 paired plasma samples tested positive. Thirty (36.6%) patients presented with at least one positive plasma PCR result, and 21 infection episodes (19 patients) required curative treatment (median follow-up time, 12 months). When the plasma CMV load was >500 copies/ml (n = 70), more than 94% (95% confidence interval, 86.0%, 98.4%) of WB samples had >500 copies/ml. Two prediction models were built: log(10) plasma viral load (VL) was calculated as -0.3777 + 0.9342 x log(10) WB VL and as -0.3777 + 0.8563 x log(10) WB VL for patients with and without treatment, respectively. In the validation sample (578 routine samples), 77.2% of the observed and expected plasma viral loads were concordant (95% confidence intervals, 73.5 and 80.5%). According to the model, the plasma viral load was >500 copies/ml when the WB load was >3,170 or >4,000 copies/ml in patients with or without treatment, respectively. WB seems to be an appropriate candidate for routine CMV monitoring of transplant recipients by using a single assay.

Phase I study of axitinib and everolimus in metastatic solid tumours and extension to metastatic renal cell carcinoma: Results of EVAX study.

PURPOSE: Anti-angiogenic and mammalian target of rapamycin inhibitors have shown efficacy in solid tumours. Reported combination of both drugs was deemed to be too toxic. Due to a potential favourable safety profile of axitinib (AX), a phase I study combining everolimus (EV) and AX for solid tumours was explored. EXPERIMENTAL DESIGN: Patients (pts) with advanced cancers were enrolled in an escalation phase I study to investigate the safety of the combination. Pharmacokinetic profile and functional vascular imaging were performed. An extension to pts with naive metastatic renal cell carcinoma (MRCC) was explored. RESULTS: 15 pts were included over three different dose levels (DLs); DL 0: AX 3 mg BID (twice daily)/EV 5 mg OD (once daily); DL 1: AX 5 mg BID/EV 5 mg OD and DL 2: AX 5 mg BID/EV 10 mg OD for 28 d. One dose-limiting toxicity (DLT) was reported at DL 0: grade (Gr) III diarrhoea and one DLT at DL 2: Gr III asthenia. Three severe adverse events (AEs) in two pts were unexpected: jaw osteonecrosis, recurrent renal failure and cardiomyopathy. Maximum tolerated dose (MTD) was level 2. After 1st cycle, Gr III or Gr II AEs of interest were mainly asthenia, diarrhoea and anorexia. All pts but one showed tumour shrinkage. Partial responses (PRs) were seen in one pt with bladder carcinoma and in one pt in 1st line MRCC in the escalating phase. In the extension phase in naive MRCC treated at MTD, five pts had a PR and one pt had a prolonged stable disease. CONCLUSION: The recommended dose for phase II is AX 5 mg BID/EV 10 mg OD.