Are surgical trials with negative results being interpreted correctly?

BACKGROUND: Many published accounts of clinical trials report no differences between the treatment arms, while being underpowered to find differences. This study determined how the authors of these reports interpreted their findings. STUDY DESIGN: We examined 54 reports of surgical trials chosen randomly from a database of 110 influential trials conducted in 2008. Seven that reported having adequate statistical power (ß ≥ 0.9) were excluded from further analysis, as were the 32 that reported significant differences between the treatment arms. We examined the remaining 15 to see whether the authors interpreted their negative findings appropriately. Appropriate interpretations discussed the lack of power and/or called for larger studies. RESULTS: Three of the 7 trials that did not report an a priori power calculation offered inappropriate interpretations, as did 3 of the 8 trials that reported an a priori power < 0.90. However, we examined only a modest number of trial reports from 1 year. CONCLUSIONS: Negative findings in underpowered trials were often interpreted as showing the equivalence of the treatment arms with no discussion of the issue of being underpowered. This may lead clinicians to accept new treatments that have not been validated.

A controlled trial of arthroscopic surgery for osteoarthritis of the knee.

BACKGROUND: Many patients report symptomatic relief after undergoing arthroscopy of the knee for osteoarthritis, but it is unclear how the procedure achieves this result. We conducted a randomized, placebo-controlled trial to evaluate the efficacy of arthroscopy for osteoarthritis of the knee. METHODS: A total of 180 patients with osteoarthritis of the knee were randomly assigned to receive arthroscopic débridement, arthroscopic lavage, or placebo surgery. Patients in the placebo group received skin incisions and underwent a simulated débridement without insertion of the arthroscope. Patients and assessors of outcome were blinded to the treatment-group assignment. Outcomes were assessed at multiple points over a 24-month period with the use of five self-reported scores--three on scales for pain and two on scales for function--and one objective test of walking and stair climbing. A total of 165 patients completed the trial. RESULTS: At no point did either of the intervention groups report less pain or better function than the placebo group. For example, mean (+/-SD) scores on the Knee-Specific Pain Scale (range, 0 to 100, with higher scores indicating more severe pain) were similar in the placebo, lavage, and débridement groups: 48.9+/-21.9, 54.8+/-19.8, and 51.7+/-22.4, respectively, at one year (P=0.14 for the comparison between placebo and lavage; P=0.51 for the comparison between placebo and débridement) and 51.6+/-23.7, 53.7+/-23.7, and 51.4+/-23.2, respectively, at two years (P=0.64 and P=0.96, respectively). Furthermore, the 95 percent confidence intervals for the differences between the placebo group and the intervention groups exclude any clinically meaningful difference. CONCLUSIONS: In this controlled trial involving patients with osteoarthritis of the knee, the outcomes after arthroscopic lavage or arthroscopic débridement were no better than those after a placebo procedure.