Monitoring Open Science as transformative change: Towards a systemic framework.

Following a flurry of policies for Open Science (OS), there is now a wave of initiatives to monitor its adoption. However, the great diversity of understandings and activities related to Open Science makes monitoring very challenging. There is a danger that by focusing on what can be readily observed (e.g. publications) many other OS activities are overlooked (e.g. participation), with a potential narrowing of OS scope, streetlight effects, and deviation from the values of OS. Since Open Science can be understood as a systemic transformation of the research system, we have borrowed concepts from Transformative Innovation Policies frameworks which aim at evaluating socio-technical transitions. In accordance with this view of OS as a systemic transformation, we propose that the new monitoring efforts should shift towards: (i) systemic perspectives which considers the various actions related to OS, including policies and outputs (e.g. datasets) but also processes (e.g. participatory events), outcomes (e.g. citizen interest in science) and expected impacts (e.g. better scientific contributions to addressing societal problems); (ii) implementation of monitoring as reflexive learning (rather than accountability or benchmarking); (iii) mapping the directionality of the activities and the values associated with the choices in directions. In summary, a monitoring framework for OS requires a profound change in conventional monitoring practices. The scope should broaden from current focus on outputs (such as publications) towards the processes of connection that make science 'open' (usage, co-creation and dialogue), as well as towards outcomes (changes in practices) and the longer-term impacts that reflect the values and normative commitments of OS.

Cross-disease spillover from research funding: Evidence from four diseases.

There is widespread appreciation for the role of research in addressing health problems. However, there is limited evidence on the extent to which research can be targeted to specific diseases. Analyses highlighting a concentration of research funding towards certain diseases have prompted growing scrutiny over the allocation of research funding. In this paper, we show that research funding targeted to a disease often results in publications relating to other diseases. Using data from the world's largest biomedical research funders, we estimated the frequency and direction of this cross-disease spillover by examining 337,573 grant-publication pairs for four diseases. We found the majority of our grant-publication pairs were cross-disease spillovers. We also found some variation between "rich" and "poor" diseases, in terms of the frequency and direction of cross-disease spillover. These differences are likely to be related to characteristics of the diseases themselves, as well as features of the research environment. One implication of frequent cross-disease spillover is that although more investment in areas of research such as neglected diseases is necessary, it may not be sufficient to improve the alignment between research funding and health needs.

Do synthesis centers synthesize? A semantic analysis of topical diversity in research.

Synthesis centers are a form of scientific organization that catalyzes and supports research that integrates diverse theories, methods and data across spatial or temporal scales to increase the generality, parsimony, applicability, or empirical soundness of scientific explanations. Synthesis working groups are a distinctive form of scientific collaboration that produce consequential, high-impact publications. But no one has asked if synthesis working groups synthesize: are their publications substantially more diverse than others, and if so, in what ways and with what effect? We investigate these questions by using Latent Dirichlet Analysis to compare the topical diversity of papers published by synthesis center collaborations with that of papers in a reference corpus. Topical diversity was operationalized and measured in several ways, both to reflect aggregate diversity and to emphasize particular aspects of diversity (such as variety, evenness, and balance). Synthesis center publications have greater topical variety and evenness, but less disparity, than do papers in the reference corpus. The influence of synthesis center origins on aspects of diversity is only partly mediated by the size and heterogeneity of collaborations: when taking into account the numbers of authors, distinct institutions, and references, synthesis center origins retain a significant direct effect on diversity measures. Controlling for the size and heterogeneity of collaborative groups, synthesis center origins and diversity measures significantly influence the visibility of publications, as indicated by citation measures. We conclude by suggesting social processes within collaborations that might account for the observed effects, by inviting further exploration of what this novel textual analysis approach might reveal about interdisciplinary research, and by offering some practical implications of our results.

Exploring why global health needs are unmet by research efforts: the potential influences of geography, industry and publication incentives.

BACKGROUND: It has been well established that research is not addressing health needs in a balanced way - much more research is conducted on diseases with more burden in high-income countries than on those with more burden in lower-income countries. In this study, we explore whether these imbalances persist and inquire about the possible influence of three factors, namely geography, industry and publication incentives. METHODS: We use WHO data on the Global Burden of Disease as a proxy measure of health needs and bibliometric information as a proxy for research efforts. Scientific publications on diseases were collected from MEDLINE using MeSH terms to identify relevant publications. We used Web of Science to collect author affiliations and citation data. We developed a correspondence table between WHO ICD-10 and MeSH descriptors to compare global health needs and research efforts. This correspondence table is available as supplementary material. RESULTS: Research output is heavily concentrated in high-income countries and is mainly focused on their health needs, resulting in a relative lack of attention to diseases in lower income countries. A new finding is that diseases with a similar burden in high- and middle-income countries are also under-researched, both globally and in relation to disease burden in high- and middle-income countries. Global industrial R&D is found to be very similar to the focus of public research. Diseases more prevalent in high-income countries generate ten-fold more research attention than those in low-income countries. We find no discernible preference towards diseases of high-income countries versus those of low-income countries in the top 25% most prestigious journals. However, in middle-income countries, citation rates are substantially lower for diseases most prevalent in low- and middle-income countries. CONCLUSIONS: From a global perspective, the imbalance between research needs and research efforts persists as most of the research effort concentrates on diseases affecting high-income countries. Both pharmaceutical companies and the public sector also tend to focus on diseases with more burden in high-income countries. Our findings indicate that researchers in middle-income countries receive more citations when researching diseases more prevalent in high-income countries, and this may divert the attention of researchers in these countries from diseases more prevalent in their contexts.

Mapping research in assisted reproduction worldwide.

RESEARCH QUESTION: What are the current research trends in human assisted reproduction around the world? DESIGN: An analysis of 26,000+ scientific publications (articles, letters and reviews) produced worldwide between 2005 and 2016. The corpus of publications indexed in PubMed was obtained by combining the Medical Subject Heading (MeSH) terms: 'Reproductive techniques', 'Reproductive medicine', 'Reproductive health', 'Fertility', 'Infertility' and 'Germ cells'. An analysis was then carried out using text mining algorithms to obtain the main topics of interest. RESULTS: A total of 44 main topics were identified, which were then further grouped into 11 categories: 'Laboratory techniques', 'Male factor', 'Quality of ART, ethics and law', 'Female factor', 'Public health and infectious diseases', 'Basic research and genetics', 'Pregnancy complications and risks', 'General - infertility & ART', 'Psychosocial aspects', 'Cancer' and 'Research methodology'. The USA was the leading country in terms of number of publications, followed by the UK, China and France. Research content in high-income countries is fairly homogeneous across categories and it is dominated by 'Laboratory techniques' in Western-Southern Europe, and by 'Quality of ART, ethics and law' in North America, Australia and New Zealand. 'Laboratory techniques' is also the most abundant category on a yearly basis. CONCLUSIONS: This study identifies the current hot topics on human assisted reproduction worldwide and their temporal trends for 2005-2016. This provides an innovative picture of the current research that could help explore the areas where further research is needed.

Does Interdisciplinary Research Lead to Higher Citation Impact? The Different Effect of Proximal and Distal Interdisciplinarity.

This article analyses the effect of degree of interdisciplinarity on the citation impact of individual publications for four different scientific fields. We operationalise interdisciplinarity as disciplinary diversity in the references of a publication, and rather than treating interdisciplinarity as a monodimensional property, we investigate the separate effect of different aspects of diversity on citation impact: i.e. variety, balance and disparity. We use a Tobit regression model to examine the effect of these properties of interdisciplinarity on citation impact, controlling for a range of variables associated with the characteristics of publications. We find that variety has a positive effect on impact, whereas balance and disparity have a negative effect. Our results further qualify the separate effect of these three aspects of diversity by pointing out that all three dimensions of interdisciplinarity display a curvilinear (inverted U-shape) relationship with citation impact. These findings can be interpreted in two different ways. On the one hand, they are consistent with the view that, while combining multiple fields has a positive effect in knowledge creation, successful research is better achieved through research efforts that draw on a relatively proximal range of fields, as distal interdisciplinary research might be too risky and more likely to fail. On the other hand, these results may be interpreted as suggesting that scientific audiences are reluctant to cite heterodox papers that mix highly disparate bodies of knowledge--thus giving less credit to publications that are too groundbreaking or challenging.

Global maps of science based on the new Web-of-Science categories.

In August 2011, Thomson Reuters launched version 5 of the Science and Social Science Citation Index in the Web of Science (WoS). Among other things, the 222 ISI Subject Categories (SCs) for these two databases in version 4 of WoS were renamed and extended to 225 WoS Categories (WCs). A new set of 151 Subject Areas was added, but at a higher level of aggregation. Perhaps confusingly, these Subject Areas are now abbreviated "SC" in the download, whereas "WC" is used for WoS Categories. Since we previously used the ISI SCs as the baseline for a global map in Pajek (Pajek is freely available at http://vlado.fmf.uni-lj.si/pub/networks/pajek/) (Rafols et al., Journal of the American Society for Information Science and Technology 61:1871-1887, 2010) and brought this facility online (at http://www.leydesdorff.net/overlaytoolkit), we recalibrated this map for the new WC categories using the Journal Citation Reports 2010. In the new installation, the base maps can also be made using VOSviewer (VOSviewer is freely available at http://www.VOSviewer.com/) (Van Eck and Waltman, Scientometrics 84:523-538, 2010).