Bias resulting from the use of 'assay sensitivity' as an inclusion criterion for meta-analysis.

Assay sensitivity has been proposed as a criterion for including psychiatric clinical outcome studies in meta-analyses. The authors assess the performance of assay sensitivity as a method for determining study appropriateness for meta-analysis by calculating expected standard drug vs placebo effect sizes for various combinations of high quality and flawed studies. In the absence of flawed studies, expected effect sizes are close to unbiased only when sample sizes are very large. In the presence of flawed studies, expected effect sizes tend to be substantially biased except under simultaneous conditions of high power, a large proportion of flawed studies, and a population standard vs placebo effect size of flawed studies considerably lower than that of high quality studies. The authors conclude that this method is not robust and can lead to serious bias. Unless it can be shown that specific conditions hold, assay sensitivity should not be used to make quality judgments of studies.

Reducing exposure of clinical research subjects to placebo treatments.

The ethics of pill placebo and placebo psychotherapy conditions in clinical research are controversial. Even when not life threatening, mental disorders dramatically diminish the quality of life. Pill-placebo conditions in drug treatment research have been justified on the grounds that a placebo versus standard drug comparison is necessary to test the quality of the study, viz., the assay sensitivity method. The assay sensitivity method of judging study quality, however, results in misclassification of the quality of some studies, leading to bias in effect size estimation in the context of meta-analyses. This bias is of particular concern in relation to studies comparing psychotherapies to psychotropic drugs, which are conducted outside of the Food and Drug Administration (FDA) context. In cases in which control conditions may be justified on grounds other than as essential elements of an assay sensitivity test, statistical methods to reduce the number of study participants exposed to placebo should be strongly considered. Of the methods available, group sequential methods are the most widely used. Group sequential methods involve successive looks at accumulating data, with rules for terminating a trial (or an arm of a trial) early if results are strong enough.