Citation bias and other determinants of citation in biomedical research: findings from six citation networks.

OBJECTIVES: When the probability of being cited depends on the outcome of that study, this is called citation bias. The aim of this study is to assess the determinants of citation and how these compare across six different biomedical research fields. STUDY DESIGN AND SETTING: Citation network analyses were performed for six biomedical research questions. After identifying all relevant publications, all potential citations were mapped together with the actually performed citations in each network. As determinants of citation we assessed the following: study outcome, study design, sample size, journal impact factor, gender, affiliation, authority and continent of the corresponding author, funding source, title of the publication, number of references, and self-citation. Random effect logistic regression analysis was used to assess these factors. RESULTS: Four out of six networks showed evidence for citation bias. Self-citation, authority of the author, and journal impact factor were also positively associated with the probability of citation in all networks. CONCLUSION: The probability of being cited seems associated with positive study outcomes, the authority of its authors, and the journal in which that article is published. In addition, each network showed specific characteristics that impact the citation dynamics and that need to be considered when performing and interpreting citation analyses.

Determinants of citation in the literature on diesel exhaust exposure and lung cancer: a citation analysis.

OBJECTIVES: Epidemiological research on the association between diesel exhaust exposure and lung cancer risk has some methodological challenges that give rise to different conclusions and intense debates. This raises the question about the role of selective citation and of citation bias in particular. Our aim was to investigate the occurrence and prevalence of selective citation in this field. DESIGN: Citation analysis. SETTING: Web of Science Core Collection. PARTICIPANTS: We identified 96 publications in this network, with 4317 potential citations. For each publication, we extracted characteristics such as study conclusion and funding source. Some of these characteristics are related to the study content: study design, sample size, method of diesel exposure assessment, type of diesel technology under investigation, and whether smoking had been adjusted for. PRIMARY AND SECONDARY OUTCOME MEASURES: Whether a citation occurs or not, measured and analysed according to the preregistered protocol. Exploratively we analysed the association between funding source and study conclusion. RESULTS: Methodological content of a study was clearly related to citation, studies using more sophisticated methods were more likely to be cited. There was some evidence for citation bias: supportive publications had a higher chance of being cited than non-supportive ones, but after adjustment for study quality, this effect decreased substantially (adjusted OR 1.3, 95% CI 1.0 to 1.7). Explorative analyses indicated that three quarters of non-profit funded publications had a supportive study conclusion against only one quarter of the industry-funded publications. CONCLUSIONS: There is evidence for selective citation within this field, but the evidence for citation bias was weak. It seems that factors related to the methodology had more impact on citation than the conclusion of a study. Interestingly, publications that were funded by industry were more skeptical about a causal relationship between diesel exhaust and lung cancer compared to non-profit-funded publications.

Determinants of Citation in Epidemiological Studies on Phthalates: A Citation Analysis.

Citing of previous publications is an important factor in knowledge development. Because of the great amount of publications available, only a selection of studies gets cited, for varying reasons. If the selection of citations is associated with study outcome this is called citation bias. We will study determinants of citation in a broader sense, including e.g. study design, journal impact factor or the funding source of the publication. As a case study we assess which factors drive citation in the human literature on phthalates, specifically the metabolite mono(2-ethylhexyl) phthalate (MEHP). A systematic literature search identified all relevant publications on human health effect of MEHP. Data on potential determinants of citation were extracted in duplo. Specialized software was used to create a citation network, including all potential citation pathways. Random effect logistic regression was used to assess whether these determinants influence the likelihood of citation. 112 Publications on MEHP were identified, with 5684 potential citation pathways of which 551 were actual citations. Reporting of a harmful point estimate, journal impact factor, authority of the author, a male corresponding author, research performed in North America and self-citation were positively associated with the likelihood of being cited. In the literature on MEHP, citation is mostly driven by a number of factors that are not related to study outcome. Although the identified determinants do not necessarily give strong indications of bias, it shows selective use of published literature for a variety of reasons.

The strong focus on positive results in abstracts may cause bias in systematic reviews: a case study on abstract reporting bias.

BACKGROUND: Research articles tend to focus on positive findings in their abstract, especially if multiple outcomes have been studied. At the same time, search queries in databases are generally limited to the abstract, title and keywords fields of an article. Negative findings are therefore less likely to be detected by systematic searches and to appear in systematic reviews. We aim to assess the occurrence of this 'abstract reporting bias' and quantify its impact in the literature on the association between diesel exhaust exposure (DEE) and bladder cancer. METHODS: We set up a broad search query related to DEE and cancer in general. Full-texts of the articles identified in the search output were manually scanned. Articles were included if they reported, anywhere in the full-text, the association between DEE and bladder cancer. We assume that the use of a broad search query and manual full-text scanning allowed us to catch all the relevant articles, including those in which bladder cancer was not mentioned in the abstract, title or keywords. RESULTS: We identified 28 articles. Only 12 of these (43%) had mentioned bladder in their abstract, title or keywords. A meta-analysis based on these 12 detectable articles yielded a pooled risk estimate of 1.10 (95% confidence interval [CI] 0.97-1.25), whereas the meta-analysis based on all 28 articles yielded a pooled estimate of 1.03 (95% CI 0.96-1.11). CONCLUSIONS: This case study on abstract reporting bias shows that (a) more than half of all relevant articles were missed by a conventional search query and (b) this led to an overestimation of the pooled effect. Detection of articles will be improved if all studied exposure and outcome variables are reported in the keywords. The restriction on the maximum number of keywords should be lifted.

Selective citation in the literature on the hygiene hypothesis: a citation analysis on the association between infections and rhinitis.

OBJECTIVE: Our objective was to assess the occurrence and determinants of selective citation in scientific publications on Strachan's original hygiene hypothesis. His hypothesis states that lack of exposure to infections in early childhood increases the risk of rhinitis. SETTING: Web of Science Core Collection. PARTICIPANTS: We identified 110 publications in this network, consisting of 5551 potential citations. PRIMARY AND SECONDARY OUTCOME MEASURES: Whether a citation occurs or not, measured and analysed according to the preregistered protocol. RESULTS: We found evidence for citation bias in this field: publications supportive of the hypothesis were cited more often than non-supportive publications (OR adjusted for study design [adjOR] 2.2, 95% CI 1.6 to 3.1), and the same was the case for publications with mixed findings (adjOR 3.1, 95% CI 2.2 to 4.5). Other relevant determinants for citation were type of exposure, specificity, journal impact factor, authority and self-citation. Surprisingly, prospective cohort studies were cited less often than other empirical studies. CONCLUSIONS: There is clear evidence for selective citation in this research field, and particularly for citation bias.

Citation bias in the literature on dietary trans fatty acids and serum cholesterol.

OBJECTIVE: Balanced citations are a necessary condition for a sound development of scientific knowledge, whereas selective citations may bias scientific consensus. In this study, we assess which determinants influenced the likelihood of being cited in the literature on trans fatty acids and cholesterol. STUDY DESIGN AND SETTING: We conducted a citation network analysis of the literature concerning trans fats and low density cholesterol and high density cholesterol. Each publication was scored on various potential determinants of citation, such as study outcome, study design, sample size, journal impact factor, and funding source. We applied random effect logistic regression to identify determinants of citation. RESULTS: A network of 108 publications was identified, containing 5,041 potential citation paths and 669 utilized citation paths. Reporting statistically significant results was found to be a strong predictor of citation, together with sample size, journal impact factor, and the authority of the authors. CONCLUSION: Within the literature on trans fat intake and cholesterol, selective citations are based on several grounds. Especially the effect of reporting significant results on citation requires special attention because disproportionate attention is paid to publications suggesting a harmful effect of trans fat on cholesterol.

Impact of changes in human reproduction on the incidence of endocrine-related diseases.

The incidence rates of a wide range of diseases and conditions have increased over the last decades. There is controversy over the origin of these increases, whether they are caused by exposure to compounds thought to have an effect on the endocrine system, the "endocrine disruption theory", or whether some other factor is responsible. In this analysis, the authors take a closer look at the role that changes in reproductive factors have played in this respect. They apply the relative risks of age at first pregnancy and parity or family size to a set of Dutch demographic data from 1955 and 2015 and calculate the percentage of disease increase explained. The decrease in parity over the last decades explains an increase of 26% in testicular cancer. The combination of decrease in parity and increase in maternal age at first pregnancy explains an increase of 34% in hypospadias prevalence. This combination of decreased parity and increased maternal age at pregnancy explains an increase of 24% in childhood obesity prevalence. The authors further point to a perhaps even more profound effect of the trend toward smaller families. This trend has led to an estimated doubling of the proportion of children born from subfertile couples. Since children born from subfertile couples are more likely to be preterm or of low birth weight, the incidence of these conditions must have increased as well. Low birth weight and preterm delivery are risk factors for a wide range of diseases and conditions. The changes in human reproduction over the last decades have had a profound impact on the incidence of a range of diseases and conditions in the next generation and thus provide a sound explanation for a substantial portion of the reported increases. Key messages The incidence rates of a wide range of diseases and conditions have increased in the Western societies over the last decades. Many have argued that these increases are attributable to compounds thought to have effects on the human endocrine system: the endocrine disruption theory This analysis shows, however, that human reproductive factors such as maternal age at first pregnancy and parity explain substantial proportions of the reported increases.

Responsible Epidemiologic Research Practice: a guideline developed by a working group of the Netherlands Epidemiological Society.

OBJECTIVES: To develop a guideline on Responsible Epidemiologic Research Practice that will increase value and transparency, increase the accountability of the epidemiologists, and reduce research waste. SETTING: A working group of the Netherland Epidemiological Society was given the task of developing a guideline that would meet these objectives. Several publications about the need to prevent Detrimental Research Practices triggered this work. Among these were a series in the Lancet on research waste and a subsequent series on transparency in the Journal of Clinical Epidemiology. The reputation and trust in epidemiologic research is still high, and the Netherlands Epidemiological Society wishes to keep it that way. The guideline deals with how epidemiologic research should be conducted, archived, and disclosed. It does not deal with the more technical aspects, such as required sample size, choice of study design, and so forth. The guideline describes each step in the process of conducting an epidemiologic study, from the first idea to the ultimate publication and beyond. METHODS: The working group reviewed the literature on responsible research conduct, including the various existing codes of conduct. It applied the general principles from these codes to the elements of an epidemiologic study and formulated specific recommendations for each of these. Next step was to draft the guideline. Preceding the 2016 annual national epidemiology conference in Wageningen, a preconference was organized to discuss the draft guideline and to assess support. Support was clearly present, and the provided recommendations were incorporated into the draft guideline. In March 2017, a draft version of the guideline was sent to all 1,100 members of the society with the request to review and provide comments. All received responses were positive, and some minor additions were made. The Responsible Epidemiologic Research Practice guideline has now been approved by the board of the Netherlands Epidemiological Society. CONCLUSION: With the Responsible Epidemiologic Research Practice guideline, we hope to contribute to better research practices in epidemiology but perhaps also in adjacent disciplines.

Scientific citations favor positive results: a systematic review and meta-analysis.

OBJECTIVES: Citation bias concerns the selective citation of scientific articles based on their results. We brought together all available evidence on citation bias across scientific disciplines and quantified its impact. STUDY DESIGN AND SETTING: An extensive search strategy was applied to the Web of Science Core Collection and Medline, yielding 52 studies in total. We classified these studies on scientific discipline, selection method, and other variables. We also performed random-effects meta-analyses to pool the effect of positive vs. negative results on subsequent citations. Finally, we checked for other determinants of citation as reported in the citation bias literature. RESULTS: Evidence for the occurrence of citation bias was most prominent in the biomedical sciences and least in the natural sciences. Articles with statistically significant results were cited 1.6 (95% confidence interval [CI] 1.3-1.8) times more often than articles with nonsignificant results. Articles in which the authors explicitly conclude to have found support for their hypothesis were cited 2.7 (CI 2.0-3.7) times as often. Article results and journal impact factor were associated with citation more often than any other reported determinant. CONCLUSION: Similar to what we already know on publication bias, also citation bias can lead to an overrepresentation of positive results and unfounded beliefs.

Selective citation in the literature on swimming in chlorinated water and childhood asthma: a network analysis.

BACKGROUND: Knowledge development depends on an unbiased representation of the available evidence. Selective citation may distort this representation. Recently, some controversy emerged regarding the possible impact of swimming on childhood asthma, raising the question about the role of selective citation in this field. Our objective was to assess the occurrence and determinants of selective citation in scientific publications on the relationship between swimming in chlorinated pools and childhood asthma. METHODS: We identified scientific journal articles on this relationship via a systematic literature search. The following factors were taken into account: study outcome (authors' conclusion, data-based conclusion), other content-related article characteristics (article type, sample size, research quality, specificity), content-unrelated article characteristics (language, publication title, funding source, number of authors, number of affiliations, number of references, journal impact factor), author characteristics (gender, country, affiliation), and citation characteristics (time to citation, authority, self-citation). To assess the impact of these factors on citation, we performed a series of univariate and adjusted random-effects logistic regressions, with potential citation path as unit of analysis. RESULTS: Thirty-six articles were identified in this network, consisting of 570 potential citation paths of which 191 (34%) were realized. There was strong evidence that articles with at least one author in common, cited each other more often than articles that had no common authors (odds ratio (OR) 5.2, 95% confidence interval (CI) 3.1-8.8). Similarly, the chance of being cited was higher for articles that were empirical rather than narrative (OR 4.2, CI 2.6-6.7), that reported a large sample size (OR 5.8, CI 2.9-11.6), and that were written by authors with a high authority within the network (OR 4.1, CI 2.1-8.0). Further, there was some evidence for citation bias: articles that confirmed the relation between swimming and asthma were cited more often (OR 1.8, CI 1.1-2.9), but this finding was not robust. CONCLUSIONS: There is clear evidence of selective citation in this research field, but the evidence for citation bias is not very strong.

Lymphatic and hematopoietic cancers among benzene-exposed workers.

OBJECTIVE: High benzene exposure is related to acute nonlymphocytic leukemia. Recently, myelodysplastic syndrome has been observed at low benzene exposure levels. METHODS: We updated a mortality study of workers with benzene exposure examining acute nonlymphocytic leukemia and myelodysplastic syndrome. We calculated standardized mortality ratios with 95% confidence intervals and examined latency and trends for cumulative exposure levels. RESULTS: All leukemias (standardized mortality ratio = 1.21; 95% confidence interval = 0.74 to 1.97) and acute non-lymphocytic leukemia (standardized mortality ratio = 1.04; 95% confidence interval = 0.34 to 2.44) were at expected levels. We observed one death from myelodysplastic syndrome (standardized mortality ratio = 6.48; 95% confidence interval = 0.17 to 38.15). We observed no trend for cumulative exposure levels. CONCLUSIONS: Our results for all leukemias are consistent with a small increase in risk observed in the lower-exposed subgroups of the Pliofilm study; however, our results are also consistent with no increased risk especially for acute nonlymphocytic leukemia.

Review of 2,4-dichlorophenoxyacetic acid (2,4-D) biomonitoring and epidemiology.

A qualitative review of the epidemiological literature on the herbicide 2,4-dichlorophenoxyacetic acid (2,4-D) and health after 2001 is presented. In order to compare the exposure of the general population, bystanders and occupational groups, their urinary levels were also reviewed. In the general population, 2,4-D exposure is at or near the level of detection (LOD). Among individuals with indirect exposure, i.e. bystanders, the urinary 2,4-D levels were also very low except in individuals with opportunity for direct contact with the herbicide. Occupational exposure, where exposure was highest, was positively correlated with behaviors related to the mixing, loading and applying process and use of personal protection. Information from biomonitoring studies increases our understanding of the validity of the exposure estimates used in epidemiology studies. The 2,4-D epidemiology literature after 2001 is broad and includes studies of cancer, reproductive toxicity, genotoxicity, and neurotoxicity. In general, a few publications have reported statistically significant associations. However, most lack precision and the results are not replicated in other independent studies. In the context of biomonitoring, the epidemiology data give no convincing or consistent evidence for any chronic adverse effect of 2,4-D in humans.

Framework for integrating human and animal data in chemical risk assessment.

Although regulatory agencies formally encourage the integration of all available data in chemical risk assessment, consistent implementation of this practice has been constrained by the lack of a clear, systematic method for doing so. In this paper, we describe a methodology for evaluating, classifying and integrating human and animal data into the risk assessment process that incorporates: (1) a balanced appraisal of human and animal data, (2) relevance to different stages of the risk assessment process, and (3) accommodation for different data quality requirements. The proposed framework offers a flexible, step-wise approach for determining which set of available data best support the chemical risk assessment that involves the rating and relative ranking of human and animal data quality. The evaluation of human data incorporates seven data quality elements, nature and specificity of the lead effect; evaluation of animal data incorporates data quality and relevance to humans. Results of simulations with selected chemicals previously evaluated in a formal risk assessment generally agreed with existing regulatory guidance. Application of the proposed framework across a wider range of chemical agents will improve transparency of the risk assessment process and validity of results, while informing continuous refinements to this evolving methodology.

Strengthening the reliability and credibility of observational epidemiology studies by creating an Observational Studies Register.

OBJECTIVE: To evaluate the need for the creation of a system in which observational epidemiology studies are registered; an Observational Studies Register (OSR). STUDY DESIGN AND SETTING: The current scientific process for observational epidemiology studies is described. Next, a parallel is made with the clinical trials area, where the creation of clinical trial registers has greatly restored and improved their credibility and reliability. Next, the advantages and disadvantages of an OSR are compared. RESULTS: The advantages of an OSR outweigh its disadvantages. CONCLUSION: The creation of an OSR, similar to the existing Clinical Trials Registers, will improve the assessment of publication bias and will provide an opportunity to compare the original study protocol with the results reported in the publication. Reliability, credibility, and transparency of observational epidemiology studies are strengthened by the creation of an OSR. We propose a structured, collaborative, and coordinated approach for observational epidemiology studies that can provide solutions for existing weaknesses and will strengthen credibility and reliability, similar to the approach currently used in clinical trials, where Clinical Trials Registers have played a key role in strengthening their scientific value.

Low level occupational benzene exposure and hematological parameters.

At high and prolonged exposure levels (e.g. >30 ppm), benzene can cause hematological effects. However, there is conflicting evidence on potential hematological effects at lower concentrations. We conducted a study to examine hematological effects at low benzene exposure levels in an occupational setting. Extensive exposure data and data from routine hematology examinations were available for Dow employees at the Terneuzen site in the Netherlands. We compared 8532 blood samples of Dow employees with low benzene exposure to 12,173 samples of employees with no benzene exposure that were available for the period between 1981 and 2007. Based on 21,584 benzene air measurements, a Job Exposure Matrix (JEM) was constructed for all employees with exposure. The JEM was used to estimate benzene exposure in the year in which each blood sample was collected. The average lymphocyte counts for the exposed and non-exposed group were similar. By means of mixed model regression adjustments were made for smoking, age and month of blood sample. These adjustments did not change the results and there was no indication for an adverse effect on any of the hematological parameters under investigation. A further stratification of the exposed population into three subgroups (<0.5 ppm, 0.5-1 ppm and >1 ppm) showed no significant differences for any of the hematological parameters between the three exposure categories or compared with the non-exposed group. The analysis modeling the continuous exposure effect relationship showed similar findings. This study does not indicate that workers exposed to low benzene concentrations are at an increased risk for hematological effects.

Mortality study update of ethylene oxide workers in chemical manufacturing: a 15 year update.

OBJECTIVE: To investigate the long-term mortality patterns of workers with past exposure to ethylene oxide (EO). METHODS: We redefined and updated a cohort of male workers employed in industrial facilities where EO was produced or used. All 2063 men were employed between 1940 and the end of 1988 and were observed for mortality through 2003. Cause specific Standardized Mortality Ratios were calculated. Internal analyses were made by applying Cox proportional hazards models to the data. RESULTS: No indications were found for excess cancer risks from EO exposures, including the lymphohematopoietic malignancies. There were 11 leukemia deaths and 11.8 expected and 12 non-Hodgkin lymphoma deaths and 11.5 expected. Proportional hazards modeling for all cause, leukemia and lymphoid malignancies mortality revealed no trends or associations with cumulative exposure. CONCLUSION: Despite the relatively high EO exposures in the past and extensive mortality follow-up, the cause specific mortality rates are comparable with those of the general US population. The Standardized Mortality Ratio analyses and the proportional hazards modeling for all cause mortality, leukemia and lymphoid malignancies mortality do not indicate exposure related effects in this cohort.

Screening questionnaire Balansmeter proved successful in predicting future long-term sickness absence in office workers.

OBJECTIVE: To develop and validate a screening instrument to identify employees at high risk for future long-term sickness absence. STUDY DESIGN AND SETTING: The instrument was developed (n=5,601) and internally validated (n=3,383) through data analyses of the Maastricht Cohort Study, among a group of office workers not absent from work. External validation was performed in a cohort of 3,895 bank employees. RESULTS: The screening instrument, Balansmeter, captures 34 questions on demographics, work environment, private situation, (mental) health, and sickness absence history. The Balansmeter showed good predictive values for future sickness absence (>28 days) in men (internal validation relative risk [RR] 4.69 [95% confidence interval (CI): 2.74, 8.02]; external validation RR 3.90 [95% CI: 2.35, 6.45]) and women (internal validation RR 4.16 [95% CI: 2.05, 8.43]; external validation RR 2.62 [95% CI: 1.44, 4.77]). CONCLUSION: It is possible to predict future sickness absence. The Balansmeter can be considered a valuable screening instrument.