Restricting evidence syntheses of interventions to English-language publications is a viable methodological shortcut for most medical topics: a systematic review.

OBJECTIVES: To assess the impact of restricting systematic reviews of conventional or alternative medical treatments or diagnostic tests to English-language publications. STUDY DESIGN AND SETTING: We systematically searched MEDLINE (Ovid), the Science Citation Index Expanded (Web of Science), and Current Contents Connect (Web of Science) up to April 24, 2020. Eligible methods studies assessed the impact of restricting systematic reviews to English-language publications on effect estimates and conclusions. Two reviewers independently screened the literature; one investigator performed the data extraction, a second investigator checked for completeness and accuracy. We synthesized the findings narratively. RESULTS: Eight methods studies (10 publications) met the inclusion criteria; none addressed language restrictions in diagnostic test accuracy reviews. The included studies analyzed nine to 147 meta-analyses and/or systematic reviews. The proportions of non-English-language publications ranged from 2% to 100%. Based on five methods studies, restricting literature searches or inclusion criteria to English-language publications led to a change in statistical significance in 23/259 meta-analyses (9%). Most commonly, the statistical significance was lost, but had no impact on the conclusions of systematic reviews. CONCLUSION: Restricting systematic reviews to English-language publications appears to have little impact on the effect estimates and conclusions of systematic reviews.

Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study.

OBJECTIVES: We aimed to assess whether limiting the inclusion criteria solely to English-language publications affected the overall conclusions of evidence syntheses. STUDY DESIGN AND SETTING: Our analyses used a dataset of a previous methods study that included 59 randomly selected Cochrane intervention reviews with no language restrictions. First, we ascertained the publication language of all 2,026 included publications. Next, we excluded studies based on the following criteria: (1) publication solely in non-English language, or (2) main publication (in case of multiple publications of the same study) in non-English language. We then re-calculated meta-analyses for outcomes that were presented in the main summary of findings tables of the Cochrane reports. If the direction of the effect estimate or the statistical significance changed, authors of the respective Cochrane reviews were consulted to assess whether the new evidence base would have changed their conclusions. The primary outcome of our analyses examined the proportion of conclusions that would change with the exclusion of non-English publications. We set the threshold for the approach as noninferior if the upper limit of the 95% confidence interval of the proportion of changed conclusions did not cross a margin of 10%. RESULTS: Across all 59 Cochrane reviews, 29 (49%) included 80 non-English publications. For 16 (27%) of these Cochrane reviews, the exclusion of non-English publications resulted in the exclusion of at least one study. In the remaining 13 Cochrane reviews, the non-English publications were not the only or main publication of the study or they did not contribute to the main summary of the findings table, so their exclusion did not result in an exclusion of the study. Overall, the exclusion of non-English publications led to the exclusion of 31 studies contributing to 40 outcomes. For 38 of the 40 outcomes, the exclusion of non-English studies did not markedly alter the size or direction of effect estimates or statistical significance. In two outcomes, the statistical significance changed, but authors would have still drawn the same conclusion, albeit with less certainty. Thus, the proportion of changed conclusions in our sample was 0.0% (95% CI 0.0-0.6), which indicated the noninferiority of the approach. However, the majority of excluded studies were small. CONCLUSION: Exclusion of non-English publications from systematic reviews on clinical interventions had a minimal effect on overall conclusions and could be a viable methodological shortcut, especially for rapid reviews.