Methodological review showed correct absolute effect size estimates for time-to-event outcomes in less than one-third of cancer-related systematic reviews.

OBJECTIVES: To evaluate in how many cancer-related Cochrane reviews hazard ratio (HR)-based absolute effects in summary of findings (SoF) tables have been correctly calculated and reported. STUDY DESIGN AND SETTING: We identified all Cochrane cancer intervention reviews that reported an HR for at least one outcome and provided a SoF table, published between January 2011 and December 2017 in the Cochrane Database of Systematic Reviews. RESULTS: In 28 reviews (29%) of 96 included Cochrane reviews, absolute effects in the SoF tables were calculated in a correct manner. In 23 reviews (24%), absolute effects had been correctly calculated, but there was no explanation given why authors calculated event-free survival (e.g., overall survival) throughout the review but reported number of events in SoF tables (e.g., death). Twelve reviews (13%) provided incorrect absolute effects. For seven reviews (7%), it was unclear if absolute effects were correctly calculated. In 26 (27%) reviews, no absolute effects based on the given HR were calculated. CONCLUSIONS: In less than one-third of cancer-related Cochrane reviews, absolute effect size estimates were correctly calculated and reported. There is a need for guidance on how to calculate and report absolute effect estimates based on HR data.

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer.

OBJECTIVE: To compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates. DESIGN: Methodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database. DATA SOURCES: Cancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded. RESULTS: We included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95% CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95% CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95% CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95% CI 0.02 to 0.09), use individual patient data (RR 0.03; 95% CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95% CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0-143); high-impact journal SRs: 74.45 (0-652)). CONCLUSIONS: When comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.