A systematic survey of randomised trials that stopped early for reasons of futility.

BACKGROUND: Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice. METHODS: We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size. RESULTS: We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis. CONCLUSIONS: Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

Randomised trials with provision for early stopping for benefit (or harm): The impact on the estimated treatment effect.

Stopping rules for clinical trials are primarily intended to control Type I error rates if interim analyses are planned, but less is known about the impact that potential stopping has on estimating treatment benefit. In this paper, we derive analytic expressions for (1) the over-estimation of benefit in studies that stop early, (2) the under-estimation of benefit in completed studies, and (3) the overall bias in studies with a stopping rule. We also examine the probability of stopping early and the situation in meta-analyses. Numerical evaluations show that the greatest concern is with over-estimation of benefit in stopped studies, especially if the probability of stopping early is small. The overall bias is usually less than 10% of the true benefit, and under-estimation in completed studies is also typically small. The probability of stopping depends on the true treatment effect and sample size. The magnitude of these effects depends on the particular rule adopted, but we show that the maximum overall bias is the same for all stopping rules. We also show that an essentially unbiased meta-analysis estimate of benefit can be recovered, even if some component studies have stopping rules. We illustrate these methods using data from three clinical trials. The results confirm our earlier empirical work on clinical trials. Investigators may consult our numerical results for guidance on potential mis-estimation and bias in the treatment effect if a stopping rule is adopted. Particular concern is warranted in studies that actually stop early, where interim results may be quite misleading.

Oxygen therapy for acutely ill medical patients: a clinical practice guideline

What is the best way to use oxygen therapy for patients with an acute medical illness? A systematic review published in the Lancet in April 2018 found that supplemental oxygen in inpatients with normal oxygen saturation increases mortality.1 Its authors concluded that oxygen should be administered conservatively, but they did not make specific recommendations on how to do it. An international expert panel used that review to inform this guideline. It aims to promptly and transparently translate potentially practice-changing evidence to usable recommendations for clinicians and patients.2 The panel used the GRADE framework and following standards for trustworthy guidelines.3

Low-intensity pulsed ultrasound for treatment of tibial fractures: an economic evaluation of the TRUST study.

AIMS: This 501-patient, multi-centre, randomised controlled trial sought to establish the effect of low-intensity, pulsed, ultrasound (LIPUS) on tibial shaft fractures managed with intramedullary nailing. We conducted an economic evaluation as part of this trial. PATIENTS AND METHODS: Data for patients' use of post-operative healthcare resources and time taken to return to work were collected and costed using publicly available sources. Health-related quality of life, assessed using the Health Utilities Index Mark-3 (HUI-3), was used to derive quality-adjusted life years (QALYs). Costs and QALYs were compared between LIPUS and control (a placebo device) from a payer and societal perspective using non-parametric bootstrapping. All costs are reported in 2015 Canadian dollars unless otherwise stated. RESULTS: With a cost per device of $3,995, the mean cost was significantly higher for patients treated with LIPUS versus placebo from a payer (mean increase = $3647, 95% confidence interval (CI) $3244 to $4070; p < 0.001) or a societal perspective (mean increase = $3425, 95% CI $1568 to $5283; p < 0.001). LIPUS did not provide a significant benefit in terms of QALYs gained (mean difference = 0.023 QALYs, 95% CI -0.035 to 0.069; p = 0.474). Incremental cost-effectiveness ratios of LIPUS compared with placebo were $155 433/QALY from a payer perspective and $146 006/QALY from a societal perspective. CONCLUSION: At the current price, LIPUS is not cost-effective for fresh tibial fractures managed with intramedullary nailing. Cite this article: Bone Joint J 2017;99-B:1526-32.

Quantifying the bias in the estimated treatment effect in randomized trials having interim analyses and a rule for early stopping for futility.

In this paper, we consider the potential bias in the estimated treatment effect obtained from clinical trials, the protocols of which include the possibility of interim analyses and an early termination of the study for reasons of futility. In particular, by considering the conditional power at an interim analysis, we derive analytic expressions for various parameters of interest: (i) the underestimation or overestimation of the treatment effect in studies that stop for futility; (ii) the impact of the interim analyses on the estimation of treatment effect in studies that are completed, i.e. that do not stop for futility; (iii) the overall estimation bias in the estimated treatment effect in a single study with such a stopping rule; and (iv) the probability of stopping at an interim analysis. We evaluate these general expressions numerically for typical trial scenarios. Results show that the parameters of interest depend on a number of factors, including the true underlying treatment effect, the difference that the trial is designed to detect, the study power, the number of planned interim analyses and what assumption is made about future data to be observed after an interim analysis. Because the probability of stopping early is small for many practical situations, the overall bias is often small, but a more serious issue is the potential for substantial underestimation of the treatment effect in studies that actually stop for futility. We also consider these ideas using data from an illustrative trial that did stop for futility at an interim analysis. Copyright © 2017 John Wiley & Sons, Ltd.

Guideline for opioid therapy and chronic noncancer pain

Guideline for opioid therapy and chronic noncancer pain: the objective is to inform the prescribing of opioids for adults with chronic noncancer pain.

Meta-analysis of antibiotics versus appendicectomy for non-perforated acute appendicitis.

BACKGROUND: For more than a century, appendicectomy has been the treatment of choice for appendicitis. Recent trials have challenged this view. This study assessed the benefits and harms of antibiotic therapy compared with appendicectomy in patients with non-perforated appendicitis. METHODS: A comprehensive search was conducted for randomized trials comparing antibiotic therapy with appendicectomy in patients with non-perforated appendicitis. Key outcomes were analysed using random-effects meta-analysis, and the quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. RESULTS: Five studies including 1116 patients reported major complications in 25 (4·9 per cent) of 510 patients in the antibiotic and 41 (8·4 per cent) of 489 in the appendicectomy group: risk difference -2·6 (95 per cent c.i. -6·3 to 1·1) per cent (low-quality evidence). Minor complications occurred in 11 (2·2 per cent) of 510 and 61 (12·5 per cent) of 489 patients respectively: risk difference -7·2 (-18·1 to 3·8) per cent (very low-quality evidence). Of 550 patients in the antibiotic group, 47 underwent appendicectomy within 1 month: pooled estimate 8·2 (95 per cent c.i. 5·2 to 11·8) per cent (high-quality evidence). Within 1 year, appendicitis recurred in 114 of 510 patients in the antibiotic group: pooled estimate 22·6 (15·6 to 30·4) per cent (high-quality evidence). For every 100 patients with non-perforated appendicitis, initial antibiotic therapy compared with prompt appendicectomy may result in 92 fewer patients receiving surgery within the first month, and 23 more experiencing recurrent appendicitis within the first year. CONCLUSION: The choice of medical versus surgical management in patients with clearly uncomplicated appendicitis is value- and preference-dependent, suggesting a change in practice towards shared decision-making is necessary.

Fluid type and the use of renal replacement therapy in sepsis: a systematic review and network meta-analysis.

Fluid resuscitation, along with the early administration of antibiotics, is the cornerstone of treatment for patients with sepsis. However, whether differences in resuscitation fluids impact on the requirements for renal replacement therapy (RRT) remains unclear. To examine this issue, we performed a network meta-analysis (NMA), including direct and indirect comparisons, that addressed the effect of different resuscitation fluids on the use of RRT in patients with sepsis. The data sources MEDLINE, EMBASE, ACPJC, CINAHL and Cochrane Central Register were searched up to March 2014. Eligible studies included randomized trials reported in any language that enrolled adult patients with sepsis or septic shock and addressed the use of RRT associated with alternative resuscitation fluids. The risk of bias for individual studies and the overall certainty of the evidence were assessed. Ten studies (6664 patients) that included a total of nine direct comparisons were assessed. NMA at the four-node level showed that an increased risk of receiving RRT was associated with fluid resuscitation with starch versus crystalloid [odds ratio (OR) 1.39, 95% credibility interval (CrI) 1.17-1.66, high certainty]. The data suggested no difference between fluid resuscitation with albumin and crystalloid (OR 1.04, 95% CrI 0.78-1.38, moderate certainty) or starch (OR 0.74, 95% CrI 0.53-1.04, low certainty). NMA at the six-node level showed a decreased risk of receiving RRT with balanced crystalloid compared to heavy starch (OR 0.50, 95% CrI 0.34-0.74, moderate certainty) or light starch (OR 0.70, 95% CrI 0.49-0.99, high certainty). There was no significant difference between balanced crystalloid and saline (OR 0.85, 95% CrI 0.56-1.30, low certainty) or albumin (OR 0.82, 95% CrI 0.49-1.37, low certainty). Of note, these trials vary in terms of case mix, fluids evaluated, duration of fluid exposure and risk of bias. Imprecise estimates contributed to low confidence in most estimates of effect. Among the patients with sepsis, fluid resuscitation with crystalloids compared to starch resulted in reduced use of RRT; the same may be true for albumin versus starch.

Diagnosis of DVT: antithrombotic therapy and prevention of thrombosis: American College of Chest Physicians evidence-based clinical practice guidelines

BACKGROUND: Objective testing for DVT is crucial because clinical assessment alone is unreliable and the consequences of misdiagnosis are serious. This guideline focuses on the identification of optimal strategies for the diagnosis of DVT in ambulatory adults. METHODS: The methods of this guideline follow those described in Methodology for the Development of Antithrombotic Therapy and Prevention of Thrombosis Guidelines: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. RESULTS: We suggest that clinical assessment of pretest probability of DVT, rather than performing the same tests in all patients, should guide the diagnostic process for a first lower extremity DVT (Grade 2B). In patients with a low pretest probability of first lower extremity DVT, we recommend initial testing with D-dimer or ultrasound (US) of the proximal veins over no diagnostic testing (Grade 1B), venography (Grade 1B), or whole-leg US (Grade 2B). In patients with moderate pretest probability, we recommend initial testing with a highly sensitive D-dimer, proximal compression US, or whole-leg US rather than no testing (Grade 1B) or venography (Grade 1B). In patients with a high pretest probability, we recommend proximal compression or whole-leg US over no testing (Grade 1B) or venography (Grade 1B). CONCLUSIONS: Favored strategies for diagnosis of first DVT combine use of pretest probability assessment, D-dimer, and US. There is lower-quality evidence available to guide diagnosis of recurrent DVT, upper extremity DVT, and DVT during pregnancy.

Methodology for the development of antithrombotic therapy and prevention of thrombosis guidelines: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines

BACKGROUND: To develop the Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: ACCP Evidence-Based Clinical Practice Guidelines (AT9), the American College of Chest Physicians (ACCP) assembled a panel of clinical experts, information scientists, decision scientists, and systematic review and guideline methodologists. METHODS: Clinical areas were designated as articles, and a methodologist without important intellectual or financial conflicts of interest led a panel for each article. Only panel members without significant conflicts of interest participated in making recommendations. Panelists specified the population, intervention and alternative, and outcomes for each clinical question and defined criteria for eligible studies. Panelists and an independent evidence-based practice center executed systematic searches for relevant studies and evaluated the evidence, and where resources and evidence permitted, they created standardized tables that present the quality of the evidence and key results in a transparent fashion. RESULTS: One or more recommendations relate to each specific clinical question, and each recommendation is clearly linked to the underlying body of evidence. Judgments regarding the quality of evidence and strength of recommendations were based on approaches developed by the Grades of Recommendations, Assessment, Development, and Evaluation Working Group. Panel members constructed scenarios describing relevant health states and rated the disutility associated with these states based on an additional systematic review of evidence regarding patient values and preferences for antithrombotic therapy. These ratings guided value and preference decisions underlying the recommendations. Each topic panel identified questions in which resource allocation issues were particularly important and, for these issues, experts in economic analysis provided additional searches and guidance. CONCLUSIONS: AT9 methodology reflects the current science of evidence-based clinical practice guideline development, with reliance on high-quality systematic reviews, a standardized process for quality assessment of individual studies and the body of evidence, an explicit process for translating the evidence into recommendations, disclosure of financial as well as intellectual conflicts of interest followed by management of disclosed conflicts, and extensive peer review.(AU)

Grading quality of evidence and strength of recommendations in clinical practice guidelines part 3 of 3. The GRADE approach to developing recommendations.

This is the third and last article in the series about the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the quality of evidence and the strength of recommendations in clinical practice guidelines and its application in the field of allergy. We describe the factors that influence the strength of recommendations about the use of diagnostic, preventive and therapeutic interventions: the balance of desirable and undesirable consequences, the quality of a body of evidence related to a decision, patients' values and preferences, and considerations of resource use. We provide examples from two recently developed guidelines in the field of allergy that applied the GRADE approach. The main advantages of this approach are the focus on patient important outcomes, explicit consideration of patients' values and preferences, the systematic approach to collecting the evidence, the clear separation of the concepts of quality of evidence and strength of recommendations, and transparent reporting of the decision process. The focus on transparency facilitates understanding and implementation and should empower patients, clinicians and other health care professionals to make informed choices.

Reamed versus nonreamed intramedullary nailing of lower extremity long bone fractures: a systematic overview and meta-analysis

Structured abstract from an article published on the Journal of Orthopaedic Trauma which aims to determine the effect of of reamed versus nonreamed intramedullary (IM) nailing of lower extremity long bone fractures, on the rates of nonunion, implant failure, malunion, compartment syndrome, pulmonary embolus and infection.

Exploratory factor analysis of the Brazilian OHIP for edentulous subjects.

The use of seven domains for the Oral Health Impact Profile (OHIP)-EDENT was not supported for its Brazilian version, making data interpretation in clinical settings difficult. Thus, the aim of this study was to assess patients' responses for the translated OHIP-EDENT in a group of edentulous subjects and to develop factor scales for application in future studies. Data from 103 conventional and implant-retained complete denture wearers (36 men, mean age of 69.1 +/- 10.3 years) were assessed using the Brazilian version of the OHIP-EDENT. Oral health-related quality of life domains were identified by factor analysis using principal component analysis as the extraction method, followed by varimax rotation. Factor analysis identified four factors that accounted for 63% of the 19 items total variance, named masticatory discomfort and disability (four items), psychological discomfort and disability (five items), social disability (five items) and oral pain and discomfort (five items). Four factors/domains of the Brazilian OHIP-EDENT version represent patient-important aspects of oral health-related quality of life.

Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies.

The GRADE approach to grading the quality of evidence and strength of recommendations provides a comprehensive and transparent approach for developing clinical recommendations about using diagnostic tests or diagnostic strategies. Although grading the quality of evidence and strength of recommendations about using tests shares the logic of grading recommendations for treatment, it presents unique challenges. Guideline panels and clinicians should be alert to these special challenges when using the evidence about the accuracy of tests as the basis for clinical decisions. In the GRADE system, valid diagnostic accuracy studies can provide high quality evidence of test accuracy. However, such studies often provide only low quality evidence for the development of recommendations about diagnostic testing, as test accuracy is a surrogate for patient-important outcomes at best. Inferring from data on accuracy that using a test improves outcomes that are important to patients requires availability of an effective treatment, improved patients' wellbeing through prognostic information, or - by excluding an ominous diagnosis - reduction of anxiety and the opportunity for earlier search for an alternative diagnosis for which beneficial treatment can be available. Assessing the directness of evidence supporting the use of a diagnostic test requires judgments about the relationship between test results and patient-important consequences. Well-designed and conducted studies of allergy tests in parallel with efforts to evaluate allergy treatments critically will encourage improved guideline development for allergic diseases.

Immunotherapy improves health-related quality of life of adult patients with dermal reactions following yellow jacket stings.

BACKGROUND: In many European centers insect venom allergic patients with a reaction confined to the skin are only offered an epinephrine auto-injector and not venom immunotherapy (VIT). Previously we showed that VIT improves health-related quality of life (HRQL) of yellow jacket allergic patients with more than dermal reactions. OBJECTIVE: To examine whether HRQL of dermal reactors is impaired and to examine the influence of VIT on HRQL in comparison with the EpiPen. METHODS: Patients with solely dermal reactions were asked if they were willing to be randomized either to VIT or EpiPen, after receiving patient information. Before and 1 year after enrollment, patients completed the Vespid allergy Quality of Life Questionnaire (VQLQ), Burden of Treatment and Expectation of Outcome. RESULTS: Of 55 patients eligible for the study, 29 consented to randomization: 15 to VIT, 14 to EpiPen. The remaining 26 patients preferred to choose their treatment: 11 VIT and 15 EpiPen. The VQLQ score of patients randomized to VIT improved (mean change 0.83 (SD 0.87), in contrast to patients randomized to the EpiPen whose scores deteriorated (mean change -0.42 (SD 0.64), P < 0.0001), resulting in an overall difference of 1.25 [95% confidence interval (CI): 0.63-1.87]. With a minimal important difference of 0.5 indicating a clinically significant improvement, VIT generated an number needed to treat (NNT) of 1.7. Dermal reactors did not consider VIT burdensome and rated this treatment as being superior to the EpiPen. CONCLUSION: VIT results in a clinically significant improvement of HRQL in most patients with reactions limited to the skin following yellow jacket stings. Prescription of an EpiPen in patients not choosing this treatment is associated with deterioration in HRQL and should therefore be avoided as definitive treatment in these patients.