Do citations and impact factors relate to the real numbers in publications? A case study of citation rates, impact, and effect sizes in ecology and evolutionary biology.

Metrics of success or impact in academia may do more harm than good. To explore the value of citations, the reported efficacy of treatments in ecology and evolution from close to 1,500 publications was examined. If citation behavior is rationale, i.e. studies that successfully applied a treatment and detected greater biological effects are cited more frequently, then we predict that larger effect sizes increases study relative citation rates. This prediction was not supported. Citations are likely thus a poor proxy for the quantitative merit of a given treatment in ecology and evolutionary biology-unlike evidence-based medicine wherein the success of a drug or treatment on human health is one of the critical attributes. Impact factor of the journal is a broader metric, as one would expect, but it also unrelated to the mean effect sizes for the respective populations of publications. The interpretation by the authors of the treatment effects within each study differed depending on whether the hypothesis was supported or rejected. Significantly larger effect sizes were associated with rejection of a hypothesis. This suggests that only the most rigorous studies reporting negative results are published or that authors set a higher burden of proof in rejecting a hypothesis. The former is likely true to a major extent since only 29 % of the studies rejected the hypotheses tested. These findings indicate that the use of citations to identify important papers in this specific discipline-at least in terms of designing a new experiment or contrasting treatments-is of limited value.

With great power comes great responsibility: the importance of rejection, power, and editors in the practice of scientific publishing.

Peer review is an important element of scientific communication but deserves quantitative examination. We used data from the handling service manuscript Central for ten mid-tier ecology and evolution journals to test whether number of external reviews completed improved citation rates for all accepted manuscripts. Contrary to a previous study examining this issue using resubmission data as a proxy for reviews, we show that citation rates of manuscripts do not correlate with the number of individuals that provided reviews. Importantly, externally-reviewed papers do not outperform editor-only reviewed published papers in terms of visibility within a 5-year citation window. These findings suggest that in many instances editors can be all that is needed to review papers (or at least conduct the critical first review to assess general suitability) if the purpose of peer review is to primarily filter and that journals can consider reducing the number of referees associated with reviewing ecology and evolution papers.

Does publication in top-tier journals affect reviewer behavior?

We show that when ecologists act as reviewers their reported rejection rates recommended for manuscripts increases with their publication frequency in high impact factor journals. Rejection rate however does not relate to reviewer age. These results indicate that the likelihood of getting a paper accepted for publication may depend upon factors in addition to scientific merit. Multiple reviewer selection for a given manuscript therefore should consider not only appropriate expertise, but also reviewers that have variable publication experience with a range of different journals to ensure balanced treatment. Interestingly since age did not relate to rejection rates, more senior scientists are not necessarily more jaded in reviewing practices.

Systematic variation in reviewer practice according to country and gender in the field of ecology and evolution.

The characteristics of referees and the potential subsequent effects on the peer-review process are an important consideration for science since the integrity of the system depends on the appropriate evaluation of merit. In 2006, we conducted an online survey of 1334 ecologists and evolutionary biologists pertaining to the review process. Respondents were from Europe, North America and other regions of the world, with the majority from English first language countries. Women comprised a third of all respondents, consistent with their representation in the scientific academic community. Among respondents we found no correlation between the time typically taken over a review and the reported average rejection rate. On average, Europeans took longer over reviewing a manuscript than North Americans, and females took longer than males, but reviewed fewer manuscripts. Males recommended rejection of manuscripts more frequently than females, regardless of region. Hence, editors and potential authors should consider alternative sets of criteria, to what exists now, when selecting a panel of referees to potentially balance different tendencies by gender or region.

Double-blind review favours increased representation of female authors.

Double-blind peer review, in which neither author nor reviewer identity are revealed, is rarely practised in ecology or evolution journals. However, in 2001, double-blind review was introduced by the journal Behavioral Ecology. Following this policy change, there was a significant increase in female first-authored papers, a pattern not observed in a very similar journal that provides reviewers with author information. No negative effects could be identified, suggesting that double-blind review should be considered by other journals.

Egg mass in an asynchronously hatching parrot: does variation offset constraints imposed by laying order?

Life history theory predicts phenotypic trade-offs between the number and quality of offspring produced. Intraspecific variation in egg mass is common in birds and increased egg size can have positive effects on offspring fitness. However, evidence of a trade-off with clutch size is limited. We analyzed variation in mass of 5,743 Green-rumped parrotlet (Forpus passerinus) eggs laid over 15 years to evaluate the potential for facultative adjustment of egg mass and factors governing variation. Heavier eggs had an increased probability of both hatching and fledging but egg mass did not affect postfledging recruitment. Offspring egg mass differed between populations and the potential for seasonal adjustment to egg mass may be related to environmental factors such as seed density. Egg mass was moderately heritable (h2 = 0.42) which accounts for some of the individual variation detected, and these results are likely attributable to strong maternal effects. We found an effect of female age on egg mass, but no effects of previous reproductive experience. Finally, egg mass was strongly governed by position within the laying sequence, independent of clutch size, and such adjustment may facilitate brood reduction under some conditions in this highly asynchronous species.