Application of the Composite Quality Score (CQS-2B) versus Cochrane's Risk of Bias tool (Version 2) in systematic reviews of clinical trials - an exploratory study.

ABSTRACT

Objectives: To explore whether systematic review conclusions generated from Cochrane's second version of its Risk of Bias tool (RoB 2) for trial appraisal differ when the Composite Quality Score, Version 2.B (CQS-2B) is used instead and to develop a testable hypothesis based on these findings. Methods: PubMed was searched for one single systematic review. From the review's accepted trials, data concerning effect estimates and overall bias risk according to the RoB 2 tool were extracted. All trial reports were appraised again using the CQS-2B. Datasets were stratified according to overall bias risk (RoB 2) or corroboration (C-) level (CQS-2B). The effect estimates from trials with 'low bias risk' (RoB 2) and with highest C-level (CQS-2B) were pooled separately. These pooled effect estimates were statistically and all clinical conclusions qualitatively compared. Results: The pooled effect estimates for trials with 'low bias risk' (RoB 2) were -0.07, 95% CI -0.10 to -0.04 (I2 = 0.0%) and for the highest C-levels (CQS-2B) 0.08, 95% CI -0.12 to -0.04 (I2 = 57.0%). The difference was statistically not significant (p = 0.70). Contrary to the RoB 2 tool, no clinical conclusions in line with the CQS-2B were made, because the effect estimates were judged to be erroneously overestimated, due to high risk of bias. Conclusion: A testable hypothesis was generated suggesting that trial appraisal using the CQS-2B may provide more conservative conclusions based on similar data than with the RoB 2 tool.