Determinants of Citation in Epidemiological Studies on Phthalates: A Citation Analysis.

Citing of previous publications is an important factor in knowledge development. Because of the great amount of publications available, only a selection of studies gets cited, for varying reasons. If the selection of citations is associated with study outcome this is called citation bias. We will study determinants of citation in a broader sense, including e.g. study design, journal impact factor or the funding source of the publication. As a case study we assess which factors drive citation in the human literature on phthalates, specifically the metabolite mono(2-ethylhexyl) phthalate (MEHP). A systematic literature search identified all relevant publications on human health effect of MEHP. Data on potential determinants of citation were extracted in duplo. Specialized software was used to create a citation network, including all potential citation pathways. Random effect logistic regression was used to assess whether these determinants influence the likelihood of citation. 112 Publications on MEHP were identified, with 5684 potential citation pathways of which 551 were actual citations. Reporting of a harmful point estimate, journal impact factor, authority of the author, a male corresponding author, research performed in North America and self-citation were positively associated with the likelihood of being cited. In the literature on MEHP, citation is mostly driven by a number of factors that are not related to study outcome. Although the identified determinants do not necessarily give strong indications of bias, it shows selective use of published literature for a variety of reasons.

Citation bias and other determinants of citation in biomedical research: findings from six citation networks.

OBJECTIVES: When the probability of being cited depends on the outcome of that study, this is called citation bias. The aim of this study is to assess the determinants of citation and how these compare across six different biomedical research fields. STUDY DESIGN AND SETTING: Citation network analyses were performed for six biomedical research questions. After identifying all relevant publications, all potential citations were mapped together with the actually performed citations in each network. As determinants of citation we assessed the following: study outcome, study design, sample size, journal impact factor, gender, affiliation, authority and continent of the corresponding author, funding source, title of the publication, number of references, and self-citation. Random effect logistic regression analysis was used to assess these factors. RESULTS: Four out of six networks showed evidence for citation bias. Self-citation, authority of the author, and journal impact factor were also positively associated with the probability of citation in all networks. CONCLUSION: The probability of being cited seems associated with positive study outcomes, the authority of its authors, and the journal in which that article is published. In addition, each network showed specific characteristics that impact the citation dynamics and that need to be considered when performing and interpreting citation analyses.

Determinants of citation in the literature on diesel exhaust exposure and lung cancer: a citation analysis.

OBJECTIVES: Epidemiological research on the association between diesel exhaust exposure and lung cancer risk has some methodological challenges that give rise to different conclusions and intense debates. This raises the question about the role of selective citation and of citation bias in particular. Our aim was to investigate the occurrence and prevalence of selective citation in this field. DESIGN: Citation analysis. SETTING: Web of Science Core Collection. PARTICIPANTS: We identified 96 publications in this network, with 4317 potential citations. For each publication, we extracted characteristics such as study conclusion and funding source. Some of these characteristics are related to the study content: study design, sample size, method of diesel exposure assessment, type of diesel technology under investigation, and whether smoking had been adjusted for. PRIMARY AND SECONDARY OUTCOME MEASURES: Whether a citation occurs or not, measured and analysed according to the preregistered protocol. Exploratively we analysed the association between funding source and study conclusion. RESULTS: Methodological content of a study was clearly related to citation, studies using more sophisticated methods were more likely to be cited. There was some evidence for citation bias: supportive publications had a higher chance of being cited than non-supportive ones, but after adjustment for study quality, this effect decreased substantially (adjusted OR 1.3, 95% CI 1.0 to 1.7). Explorative analyses indicated that three quarters of non-profit funded publications had a supportive study conclusion against only one quarter of the industry-funded publications. CONCLUSIONS: There is evidence for selective citation within this field, but the evidence for citation bias was weak. It seems that factors related to the methodology had more impact on citation than the conclusion of a study. Interestingly, publications that were funded by industry were more skeptical about a causal relationship between diesel exhaust and lung cancer compared to non-profit-funded publications.

Selective citation in the literature on the hygiene hypothesis: a citation analysis on the association between infections and rhinitis.

OBJECTIVE: Our objective was to assess the occurrence and determinants of selective citation in scientific publications on Strachan's original hygiene hypothesis. His hypothesis states that lack of exposure to infections in early childhood increases the risk of rhinitis. SETTING: Web of Science Core Collection. PARTICIPANTS: We identified 110 publications in this network, consisting of 5551 potential citations. PRIMARY AND SECONDARY OUTCOME MEASURES: Whether a citation occurs or not, measured and analysed according to the preregistered protocol. RESULTS: We found evidence for citation bias in this field: publications supportive of the hypothesis were cited more often than non-supportive publications (OR adjusted for study design [adjOR] 2.2, 95% CI 1.6 to 3.1), and the same was the case for publications with mixed findings (adjOR 3.1, 95% CI 2.2 to 4.5). Other relevant determinants for citation were type of exposure, specificity, journal impact factor, authority and self-citation. Surprisingly, prospective cohort studies were cited less often than other empirical studies. CONCLUSIONS: There is clear evidence for selective citation in this research field, and particularly for citation bias.

The strong focus on positive results in abstracts may cause bias in systematic reviews: a case study on abstract reporting bias.

BACKGROUND: Research articles tend to focus on positive findings in their abstract, especially if multiple outcomes have been studied. At the same time, search queries in databases are generally limited to the abstract, title and keywords fields of an article. Negative findings are therefore less likely to be detected by systematic searches and to appear in systematic reviews. We aim to assess the occurrence of this 'abstract reporting bias' and quantify its impact in the literature on the association between diesel exhaust exposure (DEE) and bladder cancer. METHODS: We set up a broad search query related to DEE and cancer in general. Full-texts of the articles identified in the search output were manually scanned. Articles were included if they reported, anywhere in the full-text, the association between DEE and bladder cancer. We assume that the use of a broad search query and manual full-text scanning allowed us to catch all the relevant articles, including those in which bladder cancer was not mentioned in the abstract, title or keywords. RESULTS: We identified 28 articles. Only 12 of these (43%) had mentioned bladder in their abstract, title or keywords. A meta-analysis based on these 12 detectable articles yielded a pooled risk estimate of 1.10 (95% confidence interval [CI] 0.97-1.25), whereas the meta-analysis based on all 28 articles yielded a pooled estimate of 1.03 (95% CI 0.96-1.11). CONCLUSIONS: This case study on abstract reporting bias shows that (a) more than half of all relevant articles were missed by a conventional search query and (b) this led to an overestimation of the pooled effect. Detection of articles will be improved if all studied exposure and outcome variables are reported in the keywords. The restriction on the maximum number of keywords should be lifted.

Citation bias in the literature on dietary trans fatty acids and serum cholesterol.

OBJECTIVE: Balanced citations are a necessary condition for a sound development of scientific knowledge, whereas selective citations may bias scientific consensus. In this study, we assess which determinants influenced the likelihood of being cited in the literature on trans fatty acids and cholesterol. STUDY DESIGN AND SETTING: We conducted a citation network analysis of the literature concerning trans fats and low density cholesterol and high density cholesterol. Each publication was scored on various potential determinants of citation, such as study outcome, study design, sample size, journal impact factor, and funding source. We applied random effect logistic regression to identify determinants of citation. RESULTS: A network of 108 publications was identified, containing 5,041 potential citation paths and 669 utilized citation paths. Reporting statistically significant results was found to be a strong predictor of citation, together with sample size, journal impact factor, and the authority of the authors. CONCLUSION: Within the literature on trans fat intake and cholesterol, selective citations are based on several grounds. Especially the effect of reporting significant results on citation requires special attention because disproportionate attention is paid to publications suggesting a harmful effect of trans fat on cholesterol.

Scientific citations favor positive results: a systematic review and meta-analysis.

OBJECTIVES: Citation bias concerns the selective citation of scientific articles based on their results. We brought together all available evidence on citation bias across scientific disciplines and quantified its impact. STUDY DESIGN AND SETTING: An extensive search strategy was applied to the Web of Science Core Collection and Medline, yielding 52 studies in total. We classified these studies on scientific discipline, selection method, and other variables. We also performed random-effects meta-analyses to pool the effect of positive vs. negative results on subsequent citations. Finally, we checked for other determinants of citation as reported in the citation bias literature. RESULTS: Evidence for the occurrence of citation bias was most prominent in the biomedical sciences and least in the natural sciences. Articles with statistically significant results were cited 1.6 (95% confidence interval [CI] 1.3-1.8) times more often than articles with nonsignificant results. Articles in which the authors explicitly conclude to have found support for their hypothesis were cited 2.7 (CI 2.0-3.7) times as often. Article results and journal impact factor were associated with citation more often than any other reported determinant. CONCLUSION: Similar to what we already know on publication bias, also citation bias can lead to an overrepresentation of positive results and unfounded beliefs.

Selective citation in the literature on swimming in chlorinated water and childhood asthma: a network analysis.

BACKGROUND: Knowledge development depends on an unbiased representation of the available evidence. Selective citation may distort this representation. Recently, some controversy emerged regarding the possible impact of swimming on childhood asthma, raising the question about the role of selective citation in this field. Our objective was to assess the occurrence and determinants of selective citation in scientific publications on the relationship between swimming in chlorinated pools and childhood asthma. METHODS: We identified scientific journal articles on this relationship via a systematic literature search. The following factors were taken into account: study outcome (authors' conclusion, data-based conclusion), other content-related article characteristics (article type, sample size, research quality, specificity), content-unrelated article characteristics (language, publication title, funding source, number of authors, number of affiliations, number of references, journal impact factor), author characteristics (gender, country, affiliation), and citation characteristics (time to citation, authority, self-citation). To assess the impact of these factors on citation, we performed a series of univariate and adjusted random-effects logistic regressions, with potential citation path as unit of analysis. RESULTS: Thirty-six articles were identified in this network, consisting of 570 potential citation paths of which 191 (34%) were realized. There was strong evidence that articles with at least one author in common, cited each other more often than articles that had no common authors (odds ratio (OR) 5.2, 95% confidence interval (CI) 3.1-8.8). Similarly, the chance of being cited was higher for articles that were empirical rather than narrative (OR 4.2, CI 2.6-6.7), that reported a large sample size (OR 5.8, CI 2.9-11.6), and that were written by authors with a high authority within the network (OR 4.1, CI 2.1-8.0). Further, there was some evidence for citation bias: articles that confirmed the relation between swimming and asthma were cited more often (OR 1.8, CI 1.1-2.9), but this finding was not robust. CONCLUSIONS: There is clear evidence of selective citation in this research field, but the evidence for citation bias is not very strong.