A systematic survey of randomised trials that stopped early for reasons of futility.

BACKGROUND: Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice. METHODS: We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size. RESULTS: We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis. CONCLUSIONS: Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

Randomised trials with provision for early stopping for benefit (or harm): The impact on the estimated treatment effect.

Stopping rules for clinical trials are primarily intended to control Type I error rates if interim analyses are planned, but less is known about the impact that potential stopping has on estimating treatment benefit. In this paper, we derive analytic expressions for (1) the over-estimation of benefit in studies that stop early, (2) the under-estimation of benefit in completed studies, and (3) the overall bias in studies with a stopping rule. We also examine the probability of stopping early and the situation in meta-analyses. Numerical evaluations show that the greatest concern is with over-estimation of benefit in stopped studies, especially if the probability of stopping early is small. The overall bias is usually less than 10% of the true benefit, and under-estimation in completed studies is also typically small. The probability of stopping depends on the true treatment effect and sample size. The magnitude of these effects depends on the particular rule adopted, but we show that the maximum overall bias is the same for all stopping rules. We also show that an essentially unbiased meta-analysis estimate of benefit can be recovered, even if some component studies have stopping rules. We illustrate these methods using data from three clinical trials. The results confirm our earlier empirical work on clinical trials. Investigators may consult our numerical results for guidance on potential mis-estimation and bias in the treatment effect if a stopping rule is adopted. Particular concern is warranted in studies that actually stop early, where interim results may be quite misleading.

Quantifying the bias in the estimated treatment effect in randomized trials having interim analyses and a rule for early stopping for futility.

In this paper, we consider the potential bias in the estimated treatment effect obtained from clinical trials, the protocols of which include the possibility of interim analyses and an early termination of the study for reasons of futility. In particular, by considering the conditional power at an interim analysis, we derive analytic expressions for various parameters of interest: (i) the underestimation or overestimation of the treatment effect in studies that stop for futility; (ii) the impact of the interim analyses on the estimation of treatment effect in studies that are completed, i.e. that do not stop for futility; (iii) the overall estimation bias in the estimated treatment effect in a single study with such a stopping rule; and (iv) the probability of stopping at an interim analysis. We evaluate these general expressions numerically for typical trial scenarios. Results show that the parameters of interest depend on a number of factors, including the true underlying treatment effect, the difference that the trial is designed to detect, the study power, the number of planned interim analyses and what assumption is made about future data to be observed after an interim analysis. Because the probability of stopping early is small for many practical situations, the overall bias is often small, but a more serious issue is the potential for substantial underestimation of the treatment effect in studies that actually stop for futility. We also consider these ideas using data from an illustrative trial that did stop for futility at an interim analysis. Copyright © 2017 John Wiley & Sons, Ltd.

Meta-analysis of antibiotics versus appendicectomy for non-perforated acute appendicitis.

BACKGROUND: For more than a century, appendicectomy has been the treatment of choice for appendicitis. Recent trials have challenged this view. This study assessed the benefits and harms of antibiotic therapy compared with appendicectomy in patients with non-perforated appendicitis. METHODS: A comprehensive search was conducted for randomized trials comparing antibiotic therapy with appendicectomy in patients with non-perforated appendicitis. Key outcomes were analysed using random-effects meta-analysis, and the quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. RESULTS: Five studies including 1116 patients reported major complications in 25 (4·9 per cent) of 510 patients in the antibiotic and 41 (8·4 per cent) of 489 in the appendicectomy group: risk difference -2·6 (95 per cent c.i. -6·3 to 1·1) per cent (low-quality evidence). Minor complications occurred in 11 (2·2 per cent) of 510 and 61 (12·5 per cent) of 489 patients respectively: risk difference -7·2 (-18·1 to 3·8) per cent (very low-quality evidence). Of 550 patients in the antibiotic group, 47 underwent appendicectomy within 1 month: pooled estimate 8·2 (95 per cent c.i. 5·2 to 11·8) per cent (high-quality evidence). Within 1 year, appendicitis recurred in 114 of 510 patients in the antibiotic group: pooled estimate 22·6 (15·6 to 30·4) per cent (high-quality evidence). For every 100 patients with non-perforated appendicitis, initial antibiotic therapy compared with prompt appendicectomy may result in 92 fewer patients receiving surgery within the first month, and 23 more experiencing recurrent appendicitis within the first year. CONCLUSION: The choice of medical versus surgical management in patients with clearly uncomplicated appendicitis is value- and preference-dependent, suggesting a change in practice towards shared decision-making is necessary.

Grading quality of evidence and strength of recommendations in clinical practice guidelines part 3 of 3. The GRADE approach to developing recommendations.

This is the third and last article in the series about the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the quality of evidence and the strength of recommendations in clinical practice guidelines and its application in the field of allergy. We describe the factors that influence the strength of recommendations about the use of diagnostic, preventive and therapeutic interventions: the balance of desirable and undesirable consequences, the quality of a body of evidence related to a decision, patients' values and preferences, and considerations of resource use. We provide examples from two recently developed guidelines in the field of allergy that applied the GRADE approach. The main advantages of this approach are the focus on patient important outcomes, explicit consideration of patients' values and preferences, the systematic approach to collecting the evidence, the clear separation of the concepts of quality of evidence and strength of recommendations, and transparent reporting of the decision process. The focus on transparency facilitates understanding and implementation and should empower patients, clinicians and other health care professionals to make informed choices.

Exploratory factor analysis of the Brazilian OHIP for edentulous subjects.

The use of seven domains for the Oral Health Impact Profile (OHIP)-EDENT was not supported for its Brazilian version, making data interpretation in clinical settings difficult. Thus, the aim of this study was to assess patients' responses for the translated OHIP-EDENT in a group of edentulous subjects and to develop factor scales for application in future studies. Data from 103 conventional and implant-retained complete denture wearers (36 men, mean age of 69.1 +/- 10.3 years) were assessed using the Brazilian version of the OHIP-EDENT. Oral health-related quality of life domains were identified by factor analysis using principal component analysis as the extraction method, followed by varimax rotation. Factor analysis identified four factors that accounted for 63% of the 19 items total variance, named masticatory discomfort and disability (four items), psychological discomfort and disability (five items), social disability (five items) and oral pain and discomfort (five items). Four factors/domains of the Brazilian OHIP-EDENT version represent patient-important aspects of oral health-related quality of life.

Immunotherapy improves health-related quality of life of adult patients with dermal reactions following yellow jacket stings.

BACKGROUND: In many European centers insect venom allergic patients with a reaction confined to the skin are only offered an epinephrine auto-injector and not venom immunotherapy (VIT). Previously we showed that VIT improves health-related quality of life (HRQL) of yellow jacket allergic patients with more than dermal reactions. OBJECTIVE: To examine whether HRQL of dermal reactors is impaired and to examine the influence of VIT on HRQL in comparison with the EpiPen. METHODS: Patients with solely dermal reactions were asked if they were willing to be randomized either to VIT or EpiPen, after receiving patient information. Before and 1 year after enrollment, patients completed the Vespid allergy Quality of Life Questionnaire (VQLQ), Burden of Treatment and Expectation of Outcome. RESULTS: Of 55 patients eligible for the study, 29 consented to randomization: 15 to VIT, 14 to EpiPen. The remaining 26 patients preferred to choose their treatment: 11 VIT and 15 EpiPen. The VQLQ score of patients randomized to VIT improved (mean change 0.83 (SD 0.87), in contrast to patients randomized to the EpiPen whose scores deteriorated (mean change -0.42 (SD 0.64), P < 0.0001), resulting in an overall difference of 1.25 [95% confidence interval (CI): 0.63-1.87]. With a minimal important difference of 0.5 indicating a clinically significant improvement, VIT generated an number needed to treat (NNT) of 1.7. Dermal reactors did not consider VIT burdensome and rated this treatment as being superior to the EpiPen. CONCLUSION: VIT results in a clinically significant improvement of HRQL in most patients with reactions limited to the skin following yellow jacket stings. Prescription of an EpiPen in patients not choosing this treatment is associated with deterioration in HRQL and should therefore be avoided as definitive treatment in these patients.

Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions.

The GRADE (Grades of Recommendation, Assessment, Development, and Evaluation) approach provides guidance to grading the quality of underlying evidence and the strength of recommendations in health care. The GRADE system's conceptual underpinnings allow for a detailed stepwise process that defines what role the quality of the available evidence plays in the development of health care recommendations. The merit of GRADE is not that it eliminates judgments or disagreements about evidence and recommendations, but rather that it makes them transparent. This first article in a three-part series describes the GRADE framework in relation to grading the quality of evidence about interventions based on examples from the field of allergy and asthma. In the GRADE system, the quality of evidence reflects the extent to which a guideline panel's confidence in an estimate of the effect is adequate to support a particular recommendation. The system classifies quality of evidence as high, moderate, low, or very low according to factors that include the study methodology, consistency and precision of the results, and directness of the evidence.

Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies.

The GRADE approach to grading the quality of evidence and strength of recommendations provides a comprehensive and transparent approach for developing clinical recommendations about using diagnostic tests or diagnostic strategies. Although grading the quality of evidence and strength of recommendations about using tests shares the logic of grading recommendations for treatment, it presents unique challenges. Guideline panels and clinicians should be alert to these special challenges when using the evidence about the accuracy of tests as the basis for clinical decisions. In the GRADE system, valid diagnostic accuracy studies can provide high quality evidence of test accuracy. However, such studies often provide only low quality evidence for the development of recommendations about diagnostic testing, as test accuracy is a surrogate for patient-important outcomes at best. Inferring from data on accuracy that using a test improves outcomes that are important to patients requires availability of an effective treatment, improved patients' wellbeing through prognostic information, or - by excluding an ominous diagnosis - reduction of anxiety and the opportunity for earlier search for an alternative diagnosis for which beneficial treatment can be available. Assessing the directness of evidence supporting the use of a diagnostic test requires judgments about the relationship between test results and patient-important consequences. Well-designed and conducted studies of allergy tests in parallel with efforts to evaluate allergy treatments critically will encourage improved guideline development for allergic diseases.

Interpretation of treatment changes in 6-minute walk distance in patients with COPD.

There is uncertainty about the interpretation of changes in the 6-min walk distance (6MWD) in chronic obstructive pulmonary disease (COPD) patients and whether the minimal important difference (MID) for this useful outcome measure exists. Data were used from nine trials enrolling a wide spectrum of COPD patients with 6MWD at baseline and follow-up and used to determine threshold values for important changes in 6MWD using three distribution-based methods. Anchor-based methods to determine a MID were also evaluated. Data were included of 460 COPD patients with a mean+/-sd forced expiratory volume in one second (FEV(1)) of 39.2+/-14.1% predicted and 6MWD of 361+/-112 m at baseline. Threshold values for important effects in 6MWD were between 29 and 42 m, respectively, using the empirical rule effect size and the standardised response mean. The threshold value was 35 m (95% confidence interval 30-42 m) based on the standard error of measurement. Correlations of 6MWD with patient-reported anchors were too low to provide meaningful MID estimates. 6-min walk distance should change by approximately 35 m for patients with moderate to severe chronic obstructive pulmonary disease in order to represent an important effect. This corresponds to a 10% change of baseline 6-min walk distance. The low correlations of 6-min walk distance with patient-reported anchors question whether a minimal important difference exists for the 6-min walk distance.

Evaluating the evidence for assessing BNP and NT-proBNP levels.

OBJECTIVES: To provide an overview of 5 systematic reviews in this issue of Clinical Biochemistry addressing the diagnostic and prognostic power of natriuretic peptides. DESIGN AND METHODS: Editorial. RESULTS: Limited quality of the primary studies and the large variability in their results limit the inferences that can be drawn from reviews in this series. CONCLUSIONS: The current data justify neither enthusiastic use, nor confident rejection, of BNP and NT-proBNP levels to inform the prognosis or diagnosis of heart failure.

Oxygen therapy during exercise training in chronic obstructive pulmonary disease.

BACKGROUND: Exercise training within the context of pulmonary rehabilitation improves outcomes of exercise capacity, dyspnea and health-related quality of life in individuals with chronic obstructive pulmonary disease (COPD). Supplemental oxygen in comparison to placebo increases exercise capacity in patients performing single-assessment exercise tests. The addition of supplemental oxygen during exercise training may enable individuals with COPD to tolerate higher levels of activity with less exertional symptoms, ultimately improving quality of life. OBJECTIVES: To determine how supplemental oxygen in comparison to control (compressed air or room air) during the exercise-training component of a pulmonary rehabilitation program affects exercise capacity, dyspnea and health-related quality of life in individuals with COPD. SEARCH STRATEGY: All records in the Cochrane Airways Group Specialized Register of trials coded as 'COPD' were searched using the following terms: (oxygen* or O2*) AND (exercis* or train* or rehabilitat* or fitness* or physical* or activ* or endur* or exert* or walk* or cycle*). Searching the Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library), MEDLINE, EMBASE and CINAHL databases identified studies. The last search was carried out in June 2006. SELECTION CRITERIA: Only randomized controlled trials (RCTs) comparing oxygen-supplemented exercise training to non-supplemented exercise training (control group) were considered for inclusion. Participants were 18 years or older, diagnosed with COPD and did not meet criteria for long-term oxygen therapy. No studies with mixed populations (pulmonary fibrosis, cystic fibrosis, etc) were included. Exercise training was greater than or equal to three weeks in duration and included a minimum of two sessions a week. DATA COLLECTION AND ANALYSIS: Two review authors independently selected trials for inclusion in the review and extracted data. Weighted mean differences (WMD) with 95% confidence intervals (CI) were calculated using a random-effects model. Missing data were requested from authors of primary studies. MAIN RESULTS: Five RCTs met the inclusion criteria. The maximum number of studies compared in the meta-analysis was three (31 on oxygen versus 32 control participants), because all included studies did not measure the same outcomes. When two studies were pooled, statistically significant improvements of oxygen-supplemented exercise training were found in constant power exercise time, WMD 2.68 minutes (95% CI 0.07 to 5.28 minutes). Supplemental oxygen increased the average exercise time from 6 to 14 minutes; the control intervention increased average exercise time from 6 to 12 minutes. Constant power exercise end-of-test Borg score (on a scale from 1 to 10) also showed statistically significant improvements with oxygen-supplemented exercise training, WMD -1.22 units (95% CI -2.39 to -0.06). One study showed a significant improvement in the change of Borg score after the shuttle walk test, by -1.46 units (95% CI -2.72 to -0.19). There were no significant differences in maximal exercise outcomes, functional exercise outcomes (six-minute walk test), shuttle walk distance, health-related quality of life or oxygenation status. According to the GRADE system most outcomes were rated as low quality because they were limited by study quality. AUTHORS' CONCLUSIONS: This review provides little support for oxygen supplementation during exercise training for individuals with COPD, but the evidence is very limited. Studies with larger number of participants and strong design are required to permit strong conclusions, especially for functional outcomes such as symptom alleviation, health-related quality of life and ambulation.

Low-intensity pulsed ultrasound for treatment of tibial fractures: an economic evaluation of the TRUST study.

AIMS: This 501-patient, multi-centre, randomised controlled trial sought to establish the effect of low-intensity, pulsed, ultrasound (LIPUS) on tibial shaft fractures managed with intramedullary nailing. We conducted an economic evaluation as part of this trial. PATIENTS AND METHODS: Data for patients' use of post-operative healthcare resources and time taken to return to work were collected and costed using publicly available sources. Health-related quality of life, assessed using the Health Utilities Index Mark-3 (HUI-3), was used to derive quality-adjusted life years (QALYs). Costs and QALYs were compared between LIPUS and control (a placebo device) from a payer and societal perspective using non-parametric bootstrapping. All costs are reported in 2015 Canadian dollars unless otherwise stated. RESULTS: With a cost per device of $3,995, the mean cost was significantly higher for patients treated with LIPUS versus placebo from a payer (mean increase = $3647, 95% confidence interval (CI) $3244 to $4070; p < 0.001) or a societal perspective (mean increase = $3425, 95% CI $1568 to $5283; p < 0.001). LIPUS did not provide a significant benefit in terms of QALYs gained (mean difference = 0.023 QALYs, 95% CI -0.035 to 0.069; p = 0.474). Incremental cost-effectiveness ratios of LIPUS compared with placebo were $155 433/QALY from a payer perspective and $146 006/QALY from a societal perspective. CONCLUSION: At the current price, LIPUS is not cost-effective for fresh tibial fractures managed with intramedullary nailing. Cite this article: Bone Joint J 2017;99-B:1526-32.

Validation of a symptom diary for patients with gastro-oesophageal reflux disease.

BACKGROUND: Symptom diaries are potentially attractive but, because of concerns about patient compliance, they have had limited use in clinical trials. We assessed the validity and responsiveness of a symptom diary for patients with gastro-oesophageal reflux disease. METHODS: We included 215 patients with gastro-oesophageal reflux disease after starting treatment for 4 weeks with 40 mg esomeprazole once daily. Patients recorded whether they experienced night-time heartburn (yes/no), the severity of daytime heartburn on a scale from 1 (no heartburn) to 4 (severe heartburn) and their antacid use. Patients also completed a number of disease-specific and preference-based Health-related Quality of Life questionnaires at baseline and follow-up. RESULTS: Consistent with a priori predictions, daytime heartburn showed moderate to strong correlations with the Quality of Life in Reflux and Dyspepsia questionnaire (0.36-0.67) and four scales of symptom severity (0.36-0.70) for baseline, follow-up and change scores, but low correlations with the Standard Gamble. Responsiveness of the daytime heartburn item was excellent with a mean change from baseline to follow-up of -1.3 (95% CI -1.4 to -1.1) and a standardized response mean of 1.33 while responsiveness of the daily antacid use item was moderate (mean change scores -1.8 tablets taken, 95% CI -2.3 to -1.3 and standardized response mean of 0.64). CONCLUSIONS: The excellent psychometric properties of this simple gastro-oesophageal reflux disease diary make it an attractive measure for future trials.

Quality of life in stage II breast cancer: an instrument for clinical trials.

A questionnaire has been developed for use as an outcome measure in clinical trials of adjuvant chemotherapy in women with stage II breast cancer. The selection of items for this Breast Cancer Chemotherapy Questionnaire (BCQ) was based on the problems and experiences felt to be most important by women undergoing adjuvant chemotherapy. The BCQ consists of 30 questions that focus on loss of attractiveness, fatigue, physical symptoms, inconvenience, emotional distress, and feelings of hope and support from others. The BCQ, other instruments that evaluate quality-of-life (Spitzer, Karnofsky, and Rand), and patient and physician global assessments were administered serially to 418 patients taking part in a randomized trial comparing a 12-week regimen and a 36-week regimen of adjuvant chemotherapy. The validity of the BCQ is supported by its correlation with the Rand Emotional (r = .58), Rand Physical (r = .60), and Spitzer (r = .62) questionnaires. The BCQ correlated more strongly with global ratings of both physical and emotional function by the patients and their physicians than the other instruments. A comparison of the quality-of-life outcomes of patients in the two treatment groups in the period beyond 3 months after initiation of treatment, when one group had completed the treatment course and the other was still on treatment, revealed that the BCQ and Karnofsky were the only instruments able to demonstrate differences between the groups (P less than .0001). Hence, the BCQ is a valid and responsive method of assessing treatment-related morbidity in patients receiving adjuvant chemotherapy for stage II breast cancer.

Measurement of health-related quality of life in heart failure.

Measurement of health-related quality of life is becoming increasingly relevant to controlled clinical trials. Two basic approaches to health-related quality of life measurement are available: generic instruments that attempt to provide a summary of health-related quality of life, and specific instruments that focus on problems associated with individual disease states, patient groups or areas of function. Generic instruments include health profiles and instruments that generate utility measures of health-related quality of life. The approaches are not mutually exclusive. Each approach has its strengths and weaknesses and may be suitable under different circumstances. A number of disease-specific measures for heart failure have been developed. For each measure, there is some evidence regarding key measurement properties, responsiveness and validity. An instrument is responsive if it can detect important changes, even if those changes are small. An instrument is valid if it is really measuring what it is supposed to measure. Only one disease-specific instrument that addresses a wide range of health-related quality of life impairment, the Minnesota Living with Heart Failure Questionnaire, has shown responsiveness in the context of double-blind, multicenter, pharmaceutical clinical trials. Both generic and specific measures should find increasing use in clinical trials in heart failure. Careful attention to rigorous administration is necessary for useful results. Trials should be constructed to obtain additional data regarding both validity and changes in instrument score that correspond to small, medium and large changes in health-related quality of life. If these guidelines are followed, useful information about the impact of interventions on heart failure will become increasingly available.

Acute response to bronchodilator. An imperfect guide for bronchodilator therapy in chronic airflow limitation.

We conducted a four-period cross-over randomized trial in which we found that patients with chronic airflow limitation demonstrated symptomatic improvement with both inhaled albuterol and oral theophylline. The response, however, was not uniform. We therefore tested the ability of acute change in forced expired volume in one second (FEV1) following inhaled beta agonist to predict long-term symptomatic response to albuterol and theophylline. We found that the reproducibility of acute change in FEV1 over three repetitions was poor (intraclass correlation 0.17). Furthermore, the mean improvement FEV1 following inhaled albuterol across the three repetitions did not relate closely to symptomatic response to either albuterol or theophylline. We conclude that acute response to inhaled beta agonist is not useful for identifying patients with chronic airflow limitation who are likely to benefit from bronchodilator treatment.