New treatments compared to established treatments in randomized trials.

BACKGROUND: The proportion of proposed new treatments that are 'successful' is of ethical, scientific, and public importance. We investigated how often new, experimental treatments evaluated in randomized controlled trials (RCTs) are superior to established treatments. OBJECTIVES: Our main question was: "On average how often are new treatments more effective, equally effective or less effective than established treatments?" Additionally, we wanted to explain the observed results, i.e. whether the observed distribution of outcomes is consistent with the 'uncertainty requirement' for enrollment in RCTs. We also investigated the effect of choice of comparator (active versus no treatment/placebo) on the observed results. SEARCH METHODS: We searched the Cochrane Methodology Register (CMR) 2010, Issue 1 in The Cochrane Library (searched 31 March 2010); MEDLINE Ovid 1950 to March Week 2 2010 (searched 24 March 2010); and EMBASE Ovid 1980 to 2010 Week 11 (searched 24 March 2010). SELECTION CRITERIA: Cohorts of studies were eligible for the analysis if they met all of the following criteria: (i) consecutive series of RCTs, (ii) registered at or before study onset, and (iii) compared new against established treatments in humans. DATA COLLECTION AND ANALYSIS: RCTs from four cohorts of RCTs met all inclusion criteria and provided data from 743 RCTs involving 297,744 patients. All four cohorts consisted of publicly funded trials. Two cohorts involved evaluations of new treatments in cancer, one in neurological disorders, and one for mixed types of diseases. We employed kernel density estimation, meta-analysis and meta-regression to assess the probability of new treatments being superior to established treatments in their effect on primary outcomes and overall survival. MAIN RESULTS: The distribution of effects seen was generally symmetrical in the size of difference between new versus established treatments. Meta-analytic pooling indicated that, on average, new treatments were slightly more favorable both in terms of their effect on reducing the primary outcomes (hazard ratio (HR)/odds ratio (OR) 0.91, 99% confidence interval (CI) 0.88 to 0.95) and improving overall survival (HR 0.95, 99% CI 0.92 to 0.98). No heterogeneity was observed in the analysis based on primary outcomes or overall survival (I(2) = 0%). Kernel density analysis was consistent with the meta-analysis, but showed a fairly symmetrical distribution of new versus established treatments indicating unpredictability in the results. This was consistent with the interpretation that new treatments are only slightly superior to established treatments when tested in RCTs. Additionally, meta-regression demonstrated that results have remained stable over time and that the success rate of new treatments has not changed over the last half century of clinical trials. The results were not significantly affected by the choice of comparator (active versus placebo/no therapy). AUTHORS' CONCLUSIONS: Society can expect that slightly more than half of new experimental treatments will prove to be better than established treatments when tested in RCTs, but few will be substantially better. This is an important finding for patients (as they contemplate participation in RCTs), researchers (as they plan design of the new trials), and funders (as they assess the 'return on investment'). Although we provide the current best evidence on the question of expected 'success rate' of new versus established treatments consistent with a priori theoretical predictions reflective of 'uncertainty or equipoise hypothesis', it should be noted that our sample represents less than 1% of all available randomized trials; therefore, one should exercise the appropriate caution in interpretation of our findings. In addition, our conclusion applies to publicly funded trials only, as we did not include studies funded by commercial sponsors in our analysis.

Registration of noncommercial randomised clinical trials: the feasibility of using trial registries to monitor the number of trials.

BACKGROUND: A 2003 survey suggested the number of noncommercial trials in the UK was declining. Formation of the NIHR in 2006 and increased research spending by the Department of Health may have increased the number of noncommercial trials but no data are available. METHODS: Available data on UK noncommercial trials (were obtained from the two relevant registries: ISRCTN register for the UK, and US ClinicalTrials.gov. Data on each trial were sorted by start year, and compared with the: 2003 survey, and UKCRN portfolio database from 2007. RESULTS: The number of UK noncommercial trials registered rose from 25 in 1990 to 188 in 1999, peaked at 533 in 2003, and fell back to 334 in 2009. Total trials registered was similar to but slightly above those in the 2003 survey up to 1998, then rose sharply to 2002 before falling to 2007. From 2007 to 2009 the number registered to start each year was similar to but slightly above the UKCRN database. Less than 10% of UK noncommercial trials registered with ClinGov for most years before 2005, but this rose to 35% by 2009. CONCLUSIONS: For the periods of overlap, trial registration data provide fairly similar totals to other sources on the number of noncommercial trials starting each year. The rise and fall in the number of trials registered between 1999 and 2007 was due to those registered in the ISRCTN database as funded by NHS Trusts. After 2007, the number of trials registered as funded by NHS Trusts has fallen in the ISRCTN register but these trials may have migrated to the US ClinGov register. The total number of noncommercial trial starts, excluding those funded by NHS Trusts, has been upward since around 2002. By 2009 the two main funders were NIHR and charities. Feasibility of using registration data to monitor the number of noncommercial trials has been demonstrated but is complicated by the use of two registers and difficulties in accessing the data. We recommend an annual report on the number of noncommercial trials registering each year.

Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme.

BACKGROUND: Previous research reviewed treatment success and whether the collective uncertainty principle is met in RCTs in the US National Cancer Institute portfolio. This paper classifies clinical trials funded by the UK HTA programme by results using the method applied to the US Cancer Institute trials, and compares the two portfolios. METHODS: Data on all completed randomised controlled trials funded by the HTA programme 1993-2008 were extracted. Each trial's primary results was classified into six categories; 1) statistically significant in favour of the new treatment, 2) statistically significant in favour of the control treatment 3) true negative, 4) truly inconclusive, 5) inconclusive in favour of new treatment or 6) inconclusive in favour of control treatment. Trials were classified by comparing the 95% confidence interval for the difference in primary outcome to the difference specified in the sample size calculation. The results were compared with Djulbegovic's analysis of NCI trials. RESULTS: Data from 51 superiority trials were included, involving over 48,000 participants and a range of diseases and interventions. 85 primary comparisons were available because some trials had more than two randomised arms or had several primary outcomes. The new treatment had superior results (whether significant or not) in 61% of the comparisons (52/85 95% CI 49.9% to 71.6%). The results were conclusive in 46% of the comparisons (19% statistically significant in favour of the new treatment, 5% statistically significant in favour of the control and 22% true negative). The results were classified as truly inconclusive (i.e. failed to answer the question asked) for 24% of comparisons (20/85). HTA trials included fewer truly inconclusive and statistically significant results and more results rated as true negative than NCI trials. CONCLUSIONS: The pattern of results in HTA trials is similar to that of the National Cancer Institute portfolio. Differences that existed were plausible given the differences in the types of trials -HTA trials are more pragmatic. The results indicate HTA trials are compatible with equipoise. This classification usefully summarises the results from clinical trials and enables comparisons of different portfolios of trials.

"Flogging dead horses": evaluating when have clinical trials achieved sufficiency and stability? A case study in cardiac rehabilitation.

BACKGROUND: Most systematic reviews conclude that another clinical trial is needed. Measures of sufficiency and stability may indicate whether this is true. OBJECTIVES: To show how evidence accumulated on centre-based versus home-based cardiac rehabilitation, including estimates of sufficiency and stability METHODS: Systematic reviews of clinical trials of home versus centre-based cardiac rehabilitation were used to develop a cumulative meta-analysis over time. We calculated the standardised mean difference (SMD) in effect, confidence intervals and indicators of sufficiency and stability. Sufficiency refers to whether the meta-analytic database adequately demonstrates that an intervention works - is statistically superior to another. It does this by assessing the number of studies with null results that would be required to make the meta-analytic effect non-statistically significant. Stability refers to whether the direction and size of the effect is stable as new studies are added to the meta-analysis. RESULTS: The standardised mean effect difference reduced over fourteen comparisons from a non-significant difference favouring home-based cardiac rehabilitation to a very small difference favouring hospital (SMD -0.10, 95% CI -0.32 to 0.13). This difference did not reach the sufficiency threshold (failsafe ratio 0.039<1) but did achieve the criteria for stability (cumulative slope 0.003<0.005). CONCLUSIONS: The evidence points to a relatively small effect difference which was stable but not sufficient in terms of the suggested thresholds. Sufficiency should arguably be based on substantive significance and decided by patients. Research on patient preferences should be the priority. Sufficiency and stability measures are useful tools that need to be tested in further case studies.

A systematic review of models to predict recruitment to multicentre clinical trials.

BACKGROUND: Less than one third of publicly funded trials managed to recruit according to their original plan often resulting in request for additional funding and/or time extensions. The aim was to identify models which might be useful to a major public funder of randomised controlled trials when estimating likely time requirements for recruiting trial participants. The requirements of a useful model were identified as usability, based on experience, able to reflect time trends, accounting for centre recruitment and contribution to a commissioning decision. METHODS: A systematic review of English language articles using MEDLINE and EMBASE. Search terms included: randomised controlled trial, patient, accrual, predict, enroll, models, statistical; Bayes Theorem; Decision Theory; Monte Carlo Method and Poisson. Only studies discussing prediction of recruitment to trials using a modelling approach were included. Information was extracted from articles by one author, and checked by a second, using a pre-defined form. RESULTS: Out of 326 identified abstracts, only 8 met all the inclusion criteria. Of these 8 studies examined, there are five major classes of model discussed: the unconditional model, the conditional model, the Poisson model, Bayesian models and Monte Carlo simulation of Markov models. None of these meet all the pre-identified needs of the funder. CONCLUSIONS: To meet the needs of a number of research programmes, a new model is required as a matter of importance. Any model chosen should be validated against both retrospective and prospective data, to ensure the predictions it gives are superior to those currently used.

Carbon cost of pragmatic randomised controlled trials: retrospective analysis of sample of trials.

OBJECTIVE: To calculate the global warming potential, in carbon dioxide (CO(2)) equivalent emissions, from a sample of pragmatic randomised controlled trials. DESIGN: Retrospective analysis. Data source Internal data held by NIHR Evaluation, Trials and Studies Coordinating Centre. Studies included All eligible pragmatic randomised controlled trials funded by the NIHR Health Technology Assessment programme during 2002 and 2003. MAIN OUTCOME MEASURE: CO(2) equivalents for trial activities calculated with standard conversion factors. RESULTS: 12 pragmatic randomised controlled trials involving more than 4800 participants and a wide range of technologies were included. The average CO(2) emission generated by the trials was 78.4 (range 42.1-112.7) tonnes. This is equivalent to that produced in one year by approximately nine people in the United Kingdom. Commuting to work by the trial team generated the most emissions (average 21 (11.5-35.0) tonnes per trial), followed by study centres' fuel use (18 (9.3-32.2) tonnes per trial), trial team related travel (15 (2.0-29.0) tonnes per trial), and participant related travel (13 (0-46.7) tonnes per trial). CONCLUSIONS: CO(2) emissions from pragmatic randomised controlled trials are generated in areas where steps could be taken to reduce them. A large proportion of the CO(2) emissions come from travel related to various aspects of a trial. The results of this research are likely to underestimate the total CO(2) emissions associated with the trials studied, because of the sources of information available. Further research is needed to explore the additional CO(2) emissions generated by clinical trials, over and above those generated by routine care. The results from this project will feed into NIHR guidelines that will advise researchers on how to reduce CO(2) emissions.

Experiences of hospital care reported by bereaved relatives of patients after a stroke: a retrospective survey using the VOICES questionnaire.

AIM: This paper is a report of a study conducted to explore the determinants of satisfaction with health and social care services in the last 3 months and 3 days of life as reported by bereaved relatives of those who died from a stroke in an institutional setting. BACKGROUND: There is limited research about how best to meet the needs of those who die from stroke. A thorough understanding of the determinants of satisfaction with end of life care is crucial for effective service provision to increase awareness of the needs of dying patients. METHODS: During a six-month period in 2003, a population-based survey of bereaved relatives of patients after stroke was conducted using a stroke-specific version of the Views of Informal Carers Evaluation of Services postal questionnaire (183 informants, response rate 37%). The sub-sample included those informants who reported that the deceased person had died in an institutional setting (91%, n = 165). The analysis was divided into two phases: univariate (Pearson chi-square test) and multivariate phase (logistic regression). RESULTS: Logistic regressions showed that discussing any worries about the treatment of the deceased person and feeling that the doctors and nurses knew enough about their condition were predictors of satisfaction with doctors and nurses in the last 3 months of life. Meeting the personal care needs of the deceased person, being involved in decisions and feeling that the deceased person died in the right place were predictors of satisfaction with care in the last 3 days of life. CONCLUSION: End of life care needs to address the individual needs of patients who die from stroke and those close to them. This study shows that individualised end of life care increases satisfaction and, although the data reported in this paper reflect care in 2003, there is no more recent evidence that contradicts this important overall finding.