Bibliometric analysis with reference publication year spectroscopy showed how key programmes drove developmental care in newborn infants.

AIM: We investigated the historical origins of developmental care for newborn infants using Reference Publication Year Spectroscopy (RPYS), an innovative method of bibliometric analysis. METHODS: A Web of Science search query that combined infant and intervention-related synonyms was performed on 2 February 2022. RPYS analysis was performed on this dataset to identify the most referenced historical publications for developmental care in newborn infants. Median deviation analysis identified the peak publication years, including the most cited historical references. Landmark publications were defined as those belonging to the top 10% of the most frequently referenced publications for at least 20 years. RESULTS: The RPYS peaks showed an early phase (1936-1986), during which infant development was studied and analysed, leading to a conceptualisation of developmental care for newborn infants. The following years (1987-2020), showed an explosion of interest in developmental care and highlighted two main programmes. The Neonatal Individualized Developmental Care Assessment Program (NIDCAP) and the Infant Health and Development Program (IHDP) inspired numerous publications during those years, which strove to demonstrate evidence of their clinical benefits. CONCLUSION: Developmental care has become increasingly important, thanks to the implementation of NIDCAP and IHDP.

Are quality assessments in science affected by anchoring effects? The proposal of a follow-up study.

We plan to empirically study the assessment of scientific papers within the framework of the anchoring-and-adjustment heuristic. This study is a follow-up study which is intended to answer open questions from the previous study with the same topic Bornmann (2021) and Bornmann (2023). The previous and follow-up studies address a central question in research evaluation: does bibliometrics create the social order in science it is designed to measure or does bibliometrics reflect the given social order (which is dependent on the intrinsic quality of research)? If bibliometrics creates the social order, it can be interpreted as an anchoring-and-adjustment heuristic. In the planned study, we shall undertake a survey of corresponding authors with an available email address in the Web of Science database. The authors are asked to assess the quality of articles that they cited in previous papers. The authors are randomly assigned to different experimental settings in which they receive (or not) citation information or a numerical access code to enter the survey. The control group will not receive any further numerical information. In the statistical analyses, we estimate how (strongly) the quality assessments of the cited papers are adjusted by the respondents to the anchor value (citation counts or access code). Thus, we are interested in whether possible adjustments in the assessments can not only be produced by quality-related information (citation counts), but also by numbers that are not related to quality, i.e. the access code. Strong effects of the anchors would mean that bibliometrics (or any other number) create the social order they are supposed to measure.

Anchoring effects in the assessment of papers: An empirical survey of citing authors.

In our study, we have empirically studied the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether the assessment of a paper can be influenced by numerical information that act as an anchor (e.g. citation impact). We have undertaken a survey of corresponding authors with an available email address in the Web of Science database. The authors were asked to assess the quality of papers that they cited in previous papers. Some authors were assigned to three treatment groups that receive further information alongside the cited paper: citation impact information, information on the publishing journal (journal impact factor) or a numerical access code to enter the survey. The control group did not receive any further numerical information. We are interested in whether possible adjustments in the assessments can not only be produced by quality-related information (citation impact or journal impact), but also by numbers that are not related to quality, i.e. the access code. Our results show that the quality assessments of papers seem to depend on the citation impact information of single papers. The other information (anchors) such as an arbitrary number (an access code) and journal impact information did not play a (important) role in the assessments of papers. The results point to a possible anchoring bias caused by insufficient adjustment: it seems that the respondents assessed cited papers in another way when they observed paper impact values in the survey. We conclude that initiatives aiming at reducing the use of journal impact information in research evaluation either were already successful or overestimated the influence of this information.

How relevant is climate change research for climate change policy? An empirical analysis based on Overton data.

Climate change is an ongoing topic in nearly all areas of society since many years. A discussion of climate change without referring to scientific results is not imaginable. This is especially the case for policies since action on the macro scale is required to avoid costly consequences for society. In this study, we deal with the question of how research on climate change and policy are connected. In 2019, the new Overton database of policy documents was released including links to research papers that are cited by policy documents. The use of results and recommendations from research on climate change might be reflected in citations of scientific papers in policy documents. Although we suspect a lot of uncertainty related to the coverage of policy documents in Overton, there seems to be an impact of international climate policy cycles on policy document publication. We observe local peaks in climate policy documents around major decisions in international climate diplomacy. Our results point out that IGOs and think tanks-with a focus on climate change-have published more climate change policy documents than expected. We found that climate change papers that are cited in climate change policy documents received significantly more citations on average than climate change papers that are not cited in these documents. Both areas of society (science and policy) focus on similar climate change research fields: biology, earth sciences, engineering, and disease sciences. Based on these and other empirical results in this study, we propose a simple model of policy impact considering a chain of different document types: The chain starts with scientific assessment reports (systematic reviews) that lead via science communication documents (policy briefs, policy reports or plain language summaries) and government reports to legislative documents.

Same-gender citations do not indicate a substantial gender homophily bias.

Can the male citation advantage (more citations for papers written by male than female scientists) be explained by gender homophily bias, i.e., the preference of scientists to cite other scientists of the same gender category? Previous studies report much evidence that this is the case. However, the observed gender homophily bias may be overestimated by overlooking structural aspects such as the gender composition of research topics in which scientists specialize. When controlling for research topics at a high level of granularity, there is only little evidence for a gender homophily bias in citation decisions. Our study points out the importance of controlling structural aspects such as gendered specialization in research topics when investigating gender bias in science.

Anchoring effects in the assessment of papers: The proposal for an empirical survey of citing authors.

In our planned study, we shall empirically study the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether citation decisions are (mainly) driven by the quality of cited references. The design of our study is oriented towards the study by Teplitskiy, Duede [10]. We shall undertake a survey of corresponding authors with an available email address in the Web of Science database. The authors are asked to assess the quality of papers that they cited in previous papers. Some authors will be assigned to three treatment groups that receive further information alongside the cited paper: citation information, information on the publishing journal (journal impact factor), or a numerical access code to enter the survey. The control group will not receive any further numerical information. In the statistical analyses, we estimate how (strongly) the quality assessments of the cited papers are adjusted by the respondents to the anchor value (citation, journal, or access code). Thus, we are interested in whether possible adjustments in the assessments can not only be produced by quality-related information (citation or journal), but also by numbers that are not related to quality, i.e. the access code. The results of the study may have important implications for quality assessments of papers by researchers and the role of numbers, citations, and journal metrics in assessment processes.

Heat waves: a hot topic in climate change research.

Research on heat waves (periods of excessively hot weather, which may be accompanied by high humidity) is a newly emerging research topic within the field of climate change research with high relevance for the whole of society. In this study, we analyzed the rapidly growing scientific literature dealing with heat waves. No summarizing overview has been published on this literature hitherto. We developed a suitable search query to retrieve the relevant literature covered by the Web of Science (WoS) as complete as possible and to exclude irrelevant literature (n = 8,011 papers). The time evolution of the publications shows that research dealing with heat waves is a highly dynamic research topic, doubling within about 5 years. An analysis of the thematic content reveals the most severe heat wave events within the recent decades (1995 and 2003), the cities and countries/regions affected (USA, Europe, and Australia), and the ecological and medical impacts (drought, urban heat islands, excess hospital admissions, and mortality). An alarming finding is that the limit for survivability may be reached at the end of the twenty-first century in many regions of the world due to the fatal combination of rising temperatures and humidity levels measured as "wet-bulb temperature" (WBT). Risk estimation and future strategies for adaptation to hot weather are major political issues. We identified 104 citation classics, which include fundamental early works of research on heat waves and more recent works (which are characterized by a relatively strong connection to climate change).

Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers.

Methodological mistakes, data errors, and scientific misconduct are considered prevalent problems in science that are often difficult to detect. In this study, we explore the potential of using data from Twitter for discovering problems with publications. In this case study, we analyzed tweet texts of three retracted publications about COVID-19 (Coronavirus disease 2019)/SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) and their retraction notices. We did not find early warning signs in tweet texts regarding one publication, but we did find tweets that casted doubt on the validity of the two other publications shortly after their publication date. An extension of our current work might lead to an early warning system that makes the scientific community aware of problems with certain publications. Other sources, such as blogs or post-publication peer-review sites, could be included in such an early warning system. The methodology proposed in this case study should be validated using larger publication sets that also include a control group, i.e., publications that were not retracted.

A call for governments to pause Twitter censorship: using Twitter data as social-spatial sensors of COVID-19/SARS-CoV-2 research diffusion.

In this study we determined whether Twitter data can be used as social-spatial sensors to show how research on COVID-19/SARS-CoV-2 diffuses through the population to reach the people that are affected by the disease. We performed a cross-sectional bibliometric analysis between 23rd March and 14th April 2020. Three sources of data were used: (1) deaths per number of population for COVID-19/SARS-CoV-2 retrieved from John Hopkins University and Worldometer, (2) publications related to COVID-19/SARS-CoV-2 retrieved from World Health Organisation COVID-19 database, and (3) tweets of these publications retrieved from Altmetric.com and Twitter. In the analysis, the number of publications used was 1761, and number of tweets used was 751,068. Mapping of worldwide data illustrated that high Twitter activity was related to high numbers of COVID-19/SARS-CoV-2 deaths, with tweets inversely weighted with number of publications. Regression models of worldwide data showed a positive correlation between the national deaths per number of population and tweets when holding number of publications constant (coefficient 0.0285, S.E. 0.0003, p < 0.001). Twitter can play a crucial role in the rapid research response during the COVID-19/SARS-CoV-2 pandemic, especially to spread research with prompt public scrutiny. Governments are urged to pause censorship of social media platforms to support the scientific community's fight against COVID-19/SARS-CoV-2.

Are papers addressing certain diseases perceived where these diseases are prevalent? The proposal to use Twitter data as social-spatial sensors.

We propose to use Twitter data as social-spatial sensors. This study deals with the question whether research papers on certain diseases are perceived by people in regions (worldwide) that are especially concerned by these diseases. Since (some) Twitter data contain location information, it is possible to spatially map the activity of Twitter users referring to certain papers (e.g., dealing with tuberculosis). The resulting maps reveal whether heavy activity on Twitter is correlated with large numbers of people having certain diseases. In this study, we focus on tuberculosis, human immunodeficiency virus (HIV), and malaria, since the World Health Organization ranks these diseases as the top three causes of death worldwide by a single infectious agent. The results of the social-spatial Twitter maps (and additionally performed regression models) reveal the usefulness of the proposed sensor approach. One receives an impression of how research papers on the diseases have been perceived by people in regions that are especially concerned by these diseases. Our study demonstrates a promising approach for using Twitter data for research evaluation purposes beyond simple counting of tweets.

Historical roots of pain management in infants: A bibliometric analysis using reference publication year spectroscopy.

Retrospective evaluations of the historical role of previously published research are often fraught with subjective bias and misrepresentation, which leads to contested scientific claims. This paper investigates the historical roots of infant pain management using novel quantitative methods to identify the published literature and evaluate its relative importance. A bibliometric analysis named "reference publication year spectroscopy" (RPYS), was performed using the program CitedReferencesExplorer (CRExplorer) to avoid the subjectivity associated with comparative evaluations of individual research studies. Web of Science (WoS) search queries on infant-related synonyms, pain-related synonyms, and analgesia or anesthesia-related synonyms were combined using the Boolean operator "AND," to identify all publications related to pain management in infants. The RPYS analyses were based on 8697 papers in our publication set containing the citations for 86268 references. Selected cited publications were associated with peak citation years in 1951, 1954, 1957, 1965, 1987, 1990, 1997, 1999, and 2000. Subsequent analyses suggested that research on infant pain management made rapid progress during 1982-1992. Landmark publications were defined as those belonging to the top 10% of the most frequently referenced publications for longer than 25 years. Through this analysis, we identified and ranked 24 landmark publications to illustrate the historical background and early research on infant pain management. From the first-ever application of RPYS (an objective, reproducible approach to study the early history of any scholarly activity) to pain research, infant pain management appears rooted in the scientific rationale for neonatal pain perception, randomized trials of opioid anesthesia/analgesia, and studies describing the facial expressions and crying activity following heel-lance procedures in newborns.

Influential cited references in FEMS Microbiology Letters: lessons from Reference Publication Year Spectroscopy (RPYS).

The journal FEMS Microbiology Letters covers all aspects of microbiology including virology. On which scientific shoulders do the papers published in this journal stand? Which are the classic papers used by the authors? We aim to answer these questions in this study by applying the Reference Publication Year Spectroscopy (RPYS) analysis to all papers published in this journal between 1977 and 2017. In total, 16 837 publications with 410 586 cited references are analyzed. Mainly, the studies published in the journal FEMS Microbiology Letters draw knowledge from methods developed to quantify or characterize biochemical substances such as proteins, nucleic acids, lipids, or carbohydrates and from improvements of techniques suitable for studies of bacterial genetics. The techniques frequently used for studying the genetic of microorganisms in FEMS Microbiology Letters' studies were developed using samples prepared from microorganisms. Methods required for the investigation of proteins, carbohydrates, or lipids were mostly transferred from other fields of life science to microbiology.

The graduation shift of German universities of applied sciences.

In research into higher education, the evaluation of completion and dropout rates has generated a steady stream of interest for decades. While most studies only calculate quotes using student and graduate numbers for both phenomena, we propose to additionally consider the budget available to universities. We transfer the idea of the excellence shift indicator [1] from the research to the teaching area, in particular to the completion rate of educational entities. The graduation shift shows the institutions' ability to produce graduates as measured against their basic academic teaching efficiency. It is an important advantage of the graduation shift that it avoids the well-known heterogeneity problem in efficiency measurements. Our study is based on German universities of applied science. Given their politically determined focus on education, this dataset is well-suited for introducing and evaluating the graduation shift. Using a comprehensive dataset covering the years 2008 to 2013, we show that the graduation shift produces results, which correlate closely with the results of the well-known graduation rate and standard Data Envelopment Analysis (DEA). Compared to the graduation rate, the graduation shift is preferable because it allows to take the budget of institutions into account. Compared to the DEA, the computation of the graduation shift is easy, the results are robust, and non-economists can understand them results. Thus, we recommend the graduation shift as an alternative method of efficiency measurement in the teaching area.

The Relative Influences of Government Funding and International Collaboration on Citation Impact.

A recent publication in Nature reports that public R&D funding is only weakly correlated with the citation impact of a nation's articles as measured by the field-weighted citation index (FWCI; defined by Scopus). On the basis of the supplementary data, we up-scaled the design using Web of Science data for the decade 2003-2013 and OECD funding data for the corresponding decade assuming a 2-year delay (2001-2011). Using negative binomial regression analysis, we found very small coefficients, but the effects of international collaboration are positive and statistically significant, whereas the effects of government funding are negative, an order of magnitude smaller, and statistically nonsignificant (in two of three analyses). In other words, international collaboration improves the impact of research articles, whereas more government funding tends to have a small adverse effect when comparing OECD countries.

Identifying "hot papers" and papers with "delayed recognition" in large-scale datasets by using dynamically normalized citation impact scores.

"Hot papers" (HPs) are papers which received a boost of citations shortly after publication. Papers with "delayed recognition" (DRs) received scarcely impact over a long time period, before a considerable citation boost started. DRs have attracted a lot of attention in scientometrics and beyond. Based on a comprehensive dataset with more than 5,000,000 papers published between 1980 and 1990, we identified HPs and DRs. In contrast to many other studies on DRs, which are based on raw citation counts, we calculated dynamically field-normalized impact scores for the search of HPs and DRs. This study is intended to investigate the differences between HPs (n = 323) and DRs (n = 315). The investigation of the journals which have published HPs and DRs revealed that some journals (e.g. Physical Review Letters and PNAS) were able to publish significantly more HPs than other journals. This pattern did not appear in DRs. Many HPs and DRs have been published by authors from the USA; however, in contrast to other countries, authors from the USA have published statistically significantly more HPs than DRs. Whereas "Biochemistry & Molecular Biology," "Immunology," and "Cell Biology" have published significantly more HPs than DRs, the opposite result arrived for "Surgery" and "Orthopedics." The results of the analysis of certain properties of HPs and DRs (e.g. number of pages) suggest that the emergence of DRs is an unpredictable process.

Field- and time-normalization of data with many zeros: an empirical analysis using citation and Twitter data.

Thelwall (J Informetr 11(1):128-151, 2017a. 10.1016/j.joi.2016.12.002; Web indicators for research evaluation: a practical guide. Morgan and Claypool, London, 2017b) proposed a new family of field- and time-normalized indicators, which is intended for sparse data. These indicators are based on units of analysis (e.g., institutions) rather than on the paper level. They compare the proportion of mentioned papers (e.g., on Twitter) of a unit with the proportion of mentioned papers in the corresponding fields and publication years. We propose a new indicator (Mantel-Haenszel quotient, MHq) for the indicator family. The MHq is rooted in the Mantel-Haenszel (MH) analysis. This analysis is an established method, which can be used to pool the data from several 2 × 2 cross tables based on different subgroups. We investigate using citations and assessments by peers whether the indicator family can distinguish between quality levels defined by the assessments of peers. Thus, we test the convergent validity. We find that the MHq is able to distinguish between quality levels in most cases while other indicators of the family are not. Since our study approves the MHq as a convergent valid indicator, we apply the MHq to four different Twitter groups as defined by the company Altmetric. Our results show that there is a weak relationship between the Twitter counts of all four Twitter groups and scientific quality, much weaker than between citations and scientific quality. Therefore, our results discourage the use of Twitter counts in research evaluation.

Measuring Individual Performance with Comprehensive Bibliometric Reports as an Alternative to h-Index Values.

The h-index is frequently used to measure the performance of single scientists in Korea (and beyond). No single indicator alone, however, is able to provide a stable and complete assessment of performance. The Stata command bibrep.ado is introduced which automatically produces bibliometric reports for single researchers (senior researchers working in the natural or life sciences). The user of the command receives a comprehensive bibliometric report which can be used in research evaluation instead of the h-index.