A systematic survey of randomised trials that stopped early for reasons of futility.

BACKGROUND: Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice. METHODS: We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size. RESULTS: We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis. CONCLUSIONS: Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

Systematic review and simulation study of ignoring clustered data in surgical trials.

BACKGROUND: Multiple surgical procedures in a single patient are relatively common and lead to dependent (clustered) data. This dependency needs to be accounted for in study design and data analysis. A systematic review was performed to assess how clustered data were handled in inguinal hernia trials. The impact of ignoring clustered data was estimated using simulations. METHODS: PubMed, Embase and the Cochrane Library were reviewed systematically for RCTs published between 2004 and 2013, including patients undergoing unilateral or bilateral inguinal hernia repair. Study characteristics determining the appropriateness of handling clustered data were extracted. Using simulations, various statistical methods accounting for clustered data were compared with an analysis ignoring clustering by assuming 100 hernias, with a varying percentage of patients having bilateral hernias. RESULTS: Of the 50 eligible trials including patients with bilateral hernias, 20 (40 per cent) did not provide information on how they dealt with clustered data and 18 (36 per cent) avoided clustering by assessing the outcome by patient and not by hernia. None of the remaining 12 trials (24 per cent) considered clustering in the design or analysis. In the simulations, ignoring clustering led to an increased type I error rate of up to 12 per cent and to a loss in power of up to 15 per cent, depending on whether the patient or the hernia was the randomization unit. CONCLUSION: Clustering was rarely considered in inguinal hernia trials. The simulations underline the importance of considering clustering as part of the statistical analysis to avoid false-positive and false-negative results, and hence inappropriate study conclusions.

First-line treatment and outcome of elderly patients with primary central nervous system lymphoma (PCNSL)--a systematic review and individual patient data meta-analysis.

BACKGROUND: To investigate prognosis and effects of first-line therapy in elderly primary central nervous system lymphoma (PCNSL) patients. PATIENTS AND METHODS: A systematic review of studies about first-line therapy in immunocompetent patients ≥60 years with PCNSL until 2014 and a meta-analysis of individual patient data from eligible studies and international collaborators were carried out. RESULTS: We identified 20 eligible studies; from 13 studies, we obtained individual data of 405 patients, which were pooled with data of 378 additional patients (N = 783). Median age and Karnofsky Performance Score (KPS) was 68 years (range: 60-90 years) and 60% (range: 10%-100%), respectively. Treatments varied greatly, 573 (73%) patients received high-dose methotrexate (HD-MTX)-based therapy. A total of 276 patients received whole-brain radiotherapy (median 36 Gy, range 28.5-70 Gy). KPS ≥ 70% was the strongest prognostic factor for mortality [hazard ratio (HR) 0.50, 95% confidence interval (CI) 0.41-0.62]. After a median follow-up of 40 months, HD-MTX-based therapy was associated with improved survival (HR 0.70, 95% CI 0.53-0.93). There was no difference between HD-MTX plus oral chemotherapy and more aggressive HD-MTX-based therapies (HR 1.39, 95% CI 0.90-2.15). Radiotherapy was associated with an improved survival, but correlated with an increased risk for neurological side-effects (odds ratio 5.23, 95% CI 2.33-11.74). CONCLUSIONS: Elderly PCNSL patients benefit from HD-MTX-based therapy, especially if combined with oral alkylating agents. More aggressive HD-MTX protocols do not seem to improve outcome. WBRT may improve outcome, but is associated with increased risk for neurological side-effects. Prospective trials for elderly PCNSL patients are warranted.

Contrast-enhanced ultrasound using SonoVue® (sulphur hexafluoride microbubbles) compared with contrast-enhanced computed tomography and contrast-enhanced magnetic resonance imaging for the characterisation of focal liver lesions and detection of liver metastases: a systematic review and cost-effectiveness analysis.

BACKGROUND: Medical imaging techniques are important in the management of many patients with liver disease. Unenhanced ultrasound examinations sometimes identify focal abnormalities in the liver that may require further investigation, primarily to distinguish liver cancers from benign abnormalities. One important factor in selecting an imaging test is the ability to provide a rapid diagnosis. Options for additional imaging investigations include computed tomography (CT) and/or magnetic resonance imaging (MRI) and biopsy when the diagnosis remains uncertain. CT and MRI usually require referral with associated waiting time and are sometimes contraindicated. The use of contrast agents may improve the ability of ultrasound to distinguish between liver cancer and benign abnormalities and, because it can be performed at the same appointment as unenhanced ultrasound, more rapid diagnoses may be possible. OBJECTIVE: To compare the clinical effectiveness and cost-effectiveness of contrast-enhanced ultrasound (CEUS) using SonoVue(®) with that of contrast-enhanced computed tomography (CECT) and contrast-enhanced magnetic resonance imaging (CEMRI) for the assessment of adults with focal liver lesions (FLLs) in whom previous liver imaging is inconclusive. DATA SOURCES: Eight bibliographic databases including MEDLINE, EMBASE, Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effects were searched from 2000 to September/October 2011. Research registers and conference proceedings were also searched. REVIEW METHODS: Systematic review methods followed published guidance. Risk of bias was assessed using a modified version of the QUADAS-2 tool. Results were stratified by clinical indication for imaging (characterisation of FLLs detected on ultrasound surveillance of cirrhosis patients, detection of liver metastases, characterisation of incidentally detected FLLs, assessment of treatment response). For incidental FLLs, pooled estimates of sensitivity and specificity, with 95% CIs, were calculated using a random-effects model. For other clinical indications a narrative summary was used. The cost-effectiveness of CEUS was modelled separately for the three main clinical applications considered [characterisation of FLLs detected on ultrasound surveillance of cirrhosis patients, detection of liver metastases in patients with colorectal cancer (CRC), characterisation of incidentally detected FLLs]. RESULTS: Of the 854 references identified, 19 (describing 18 studies) were included in the review. Hand searching of conference proceedings identified a further three studies. Twenty of the 21 studies included in the systematic review were diagnostic test accuracy studies. Studies in cirrhosis patients reported varying estimates of test performance. There was no consistent evidence of a significant difference in performance between imaging modalities. It was unclear whether or not CEUS alone is adequate to rule out hepatocellular carcinoma (HCC) for FLLs of < 30 mm; one study indicated that CEUS may be better at ruling out HCC for FLLs of 11-30 mm [very small FLLs (< 10 mm) excluded]. There was no consistent evidence of a difference in test performance between imaging modalities for the detection of metastases; CEUS alone may be adequate to rule out liver metastases in colorectal cancer. In patients with incidentally detected FLLs, the pooled estimates of sensitivity for any malignancy using CEUS and CECT were 95.1% and 94.6%, respectively, and the corresponding specificity estimates were 93.8% and 93.1% respectively. One study comparing CEUS with CEMRI reported similar sensitivity and lower specificity for both modalities. In the surveillance of cirrhosis, CEUS was as effective as but £379 less costly than CECT. CEMRI was £1063 more costly than CEUS and gained 0.022 QALYs. In the detection of liver metastases from CRC, CEUS cost £1 more than CECT, and at a lifetime time horizon they yielded equal QALYs. CEMRI was dominated by CECT. In the characterisation of incidentally detected FLLs, CEUS was slightly more effective than CECT and CEMRI (by 0.0002 QALYs and 0.0026 QALYs respectively) and less costly (by £52 and £131 respectively). LIMITATIONS: There were a number of methodological issues specific to the studies included in this review. The main indication for liver imaging in the populations considered is likely to be to rule out primary liver cancer or metastases. Therefore, patient-level analyses of test performance are of particular interest. Some of the studies included in this review reported per-patient analyses; however, no study clearly stated how results were defined (e.g. was the presence of any positive lesion regarded as a positive test for the whole patient). In addition, a number of studies reported data for one lesion per patient (treated as per-patient data in this assessment). These studies generally selected the largest lesion or the lesion 'most suspicious for malignancy' for inclusion in analyses, with the consequence that estimates of test performance may have been exaggerated. The applicability of studies included in this review may be limited, as the majority of imaging studies were interpreted by multiple, experienced operators and the prevalence of malignancy in included studies appeared higher than might be expected in clinical practice. The cost-effectiveness analyses did not take into account the potential benefits of reduced anxiety that may arise from potentially shorter waiting times associated with SonoVue CEUS. CONCLUSIONS: SonoVue CEUS could provide similar diagnostic performance to other imaging modalities (CECT and CEMRI) for the assessment of FLLs. Economic analyses indicated that CEUS was a cost-effective replacement for CEMRI. The use of CEUS instead of CECT was considered cost-effective in the surveillance of cirrhosis and the characterisation of incidentally detected FLLs, with similar costs and effects for the detection of liver metastases from CRC. Further research is needed to compare the effects of different imaging modalities (SonoVue CEUS, CECT, CEMRI) on therapeutic planning, treatment and clinical outcomes. Future test accuracy studies should provide standardised definitions of a positive imaging test, and compare all three imaging modalities in the same patient group. STUDY REGISTRATION: PROSPERO: CRD42011001694. FUNDING: The National Institute for Health Research Health Technology Assessment programme.