Community consensus on core open science practices to monitor in biomedicine.

The state of open science needs to be monitored to track changes over time and identify areas to create interventions to drive improvements. In order to monitor open science practices, they first need to be well defined and operationalized. To reach consensus on what open science practices to monitor at biomedical research institutions, we conducted a modified 3-round Delphi study. Participants were research administrators, researchers, specialists in dedicated open science roles, and librarians. In rounds 1 and 2, participants completed an online survey evaluating a set of potential open science practices, and for round 3, we hosted two half-day virtual meetings to discuss and vote on items that had not reached consensus. Ultimately, participants reached consensus on 19 open science practices. This core set of open science practices will form the foundation for institutional dashboards and may also be of value for the development of policy, education, and interventions.

Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets.

In this article, we investigate the surge in use of COVID-19-related preprints by media outlets. Journalists are a main source of reliable public health information during crises and, until recently, journalists have been reluctant to cover preprints because of the associated scientific uncertainty. Yet, uploads of COVID-19 preprints and their uptake by online media have outstripped that of preprints about any other topic. Using an innovative approach combining altmetrics methods with content analysis, we identified a diversity of outlets covering COVID-19-related preprints during the early months of the pandemic, including specialist medical news outlets, traditional news media outlets, and aggregators. We found a ubiquity of hyperlinks as citations and a multiplicity of framing devices for highlighting the scientific uncertainty associated with COVID-19 preprints. These devices were rarely used consistently (e.g., mentioning that the study was a preprint, unreviewed, preliminary, and/or in need of verification). About half of the stories we analyzed contained framing devices emphasizing uncertainty. Outlets in our sample were much less likely to identify the research they mentioned as preprint research, compared to identifying it as simply "research." This work has significant implications for public health communication within the changing media landscape. While current best practices in public health risk communication promote identifying and promoting trustworthy sources of information, the uptake of preprint research by online media presents new challenges. At the same time, it provides new opportunities for fostering greater awareness of the scientific uncertainty associated with health research findings.

"The Best Home for This Paper": A Qualitative Study of How Authors Select Where to Submit Manuscripts.

Introduction: For authors, selecting a target journal to submit a manuscript is a critical decision with career implications. In the discipline of medical education, research conducted in 2016 found that authors were influenced by multiple factors such as a journal's prestige and its mission. However, since this research was conducted the publishing landscape has shifted to include a broader variety of journals, an increased threat of predatory journals, and new publishing models. This study updates and expands upon how medical education authors decide which journal to submit to with the aim of describing the motivational factors and journal characteristics that guide authors' decision making. Methods: The authors conducted five qualitative focus groups in which twenty-two medical education authors and editors participated. During the focus groups participants were engaged in a discussion about how they select a journal to submit their manuscripts. Audio from all focus groups was transcribed. Transcripts were analyzed using codebook thematic analysis. Results: Participants considered multiple factors when selecting a target journal. Factors included a journal's impact, the scope of a journal, journal quality, and technical factors (e.g., word limits). Participants also described how social factors influenced their process and that open access plays a role that could both encourage or deter submission. Discussion: The findings describe the motivational factors and influential signals that guide authors in their journal selection decision making. These findings confirm, extend, and update journal selection factors reported in medical education and other disciplines. Notably, these findings emphasize the role of social factors, relationships and personal experiences, which were absent from previous work. Additionally, we observed increased consideration of open acces and a shift away from an emphasis on journal prestige.

Making science public: a review of journalists' use of Open Access research.

Science journalists are uniquely positioned to increase the societal impact of open research outputs by contextualizing and communicating findings in ways that highlight their relevance and implications for non-specialist audiences. Yet, it is unclear to what degree journalists use open research outputs, such as open access publications or preprints, in their reporting; what factors motivate or constrain this use; and how the recent surge in openly available research seen during the COVID-19 pandemic has affected this. This article examines these questions through a review of relevant literature published from 2018 onwards-particularly literature relating to the COVID-19 pandemic-as well as seminal articles outside the search dates. We find that research that explicitly examines journalists' engagement with open access publications or preprints is scarce, with existing literature mostly addressing the topic tangentially or as a secondary concern, rather than a primary focus. Still, the limited body of evidence points to several factors that may hamper journalists' use of these outputs and thus warrant further exploration. These include an overreliance on traditional criteria for evaluating scientific quality; concerns about the trustworthiness of open research outputs; and challenges using and verifying the findings. We also find that, while the COVID-19 pandemic encouraged journalists to explore open research outputs such as preprints, the extent to which these explorations will become established journalistic practices remains unclear. Furthermore, we note that current research is overwhelmingly authored and focused on the Global North, and the United States specifically. We conclude with recommendations for future research that attend to issues of equity and diversity, and more explicitly examine the intersections of open access and science journalism.

Using social annotation to construct knowledge with others: A case study across undergraduate courses.

Background: Social annotation (SA) is a genre of learning technology that enables the addition of digital notes to shared texts and affords contextualized peer-to-peer online discussion. A small body of literature examines how SA, as asynchronous online discussion, can contribute to students' knowledge construction (KC)-or a process whereby learners collaborate through shared socio-cognitive practices. This case study analyzed how SA enabled student participation in seven KC activities, such as interpretation and elaboration. Methods: We analyzed 2,121 annotations written by 59 students in three undergraduate courses at a Canadian University in the first months of 2019. Using a method of open coding and constant comparison, we coded each annotation for evidence of KC activities. Results: Results showed a range of KC activities in students' SA. Across courses, interpretation was the most common KC activity (40%), followed by elaboration (20%). Annotations that were part of peer-to-peer discussion included all seven types of KC activities, but some activities, such as consensus building, support, and conflict, were almost exclusively found in replies to others. Conclusions: This study suggests that SA is a productive form of online learning through which undergraduate students in multiple disciplinary contexts can interact with peers, make sense of academic content, and construct knowledge by reading and writing together.

Identifying science in the news: An assessment of the precision and recall of Altmetric.com news mention data.

The company Altmetric is often used to collect mentions of research in online news stories, yet there have been concerns about the quality of this data. This study investigates these concerns. Using a manual content analysis of 400 news stories as a comparison method, we analyzed the precision and recall with which Altmetric identified mentions of research in 8 news outlets. We also used logistic regression to identify the characteristics of research mentions that influence their likelihood of being successfully identified. We find that, for a predefined set of outlets, Altmetric's news mention data were relatively accurate (F-score = 0.80), with very high precision (0.95) and acceptable recall (0.70), although recall is below 0.50 for some news outlets. Altmetric is more likely to successfully identify mentions of research that include a hyperlink to the research item, an author name, and/or the title of a publication venue. This data source appears to be less reliable for mentions of research that provide little or no bibliometric information, as well as for identifying mentions of scholarly monographs, conference presentations, dissertations, and non-English research articles. Our findings suggest that, with caveats, scholars can use Altmetric news mention data as a relatively reliable source to identify research mentions across a range of outlets with high precision and acceptable recall, offering scholars the potential to conserve resources during data collection. Our study does not, however, offer an assessment of completeness or accuracy of Altmetric news data overall. Supplementary Information: The online version contains supplementary material available at 10.1007/s11192-022-04510-7.

Science in motion: A qualitative analysis of journalists' use and perception of preprints.

This qualitative study explores how and why journalists use preprints-unreviewed research papers-in their reporting. Through thematic analysis of interviews conducted with 19 health and science journalists in the second year of the COVID-19 pandemic, it applies a theoretical framework that conceptualizes COVID-19 preprint research as a form of post-normal science, characterized by high scientific uncertainty and societal relevance, urgent need for political decision-making, and value-related policy considerations. Findings suggest that journalists approach the decision to cover preprints as a careful calculation, in which the potential public benefits and the ease of access preprints provided were weighed against risks of spreading misinformation. Journalists described viewing unreviewed studies with extra skepticism and relied on diverse strategies to find, vet, and report on them. Some of these strategies represent standard science journalism, while others, such as labeling unreviewed studies as preprints, mark a departure from the norm. However, journalists also reported barriers to covering preprints, as many felt they lacked the expertise or the time required to fully understand or vet the research. The findings suggest that coverage of preprints is likely to continue post-pandemic, with important implications for scientists, journalists, and the publics who read their work.

The role of collegiality in academic review, promotion, and tenure.

Review, promotion, and tenure (RPT) processes at universities typically assess candidates along three dimensions: research, teaching, and service. In recent years, some have argued for the inclusion of a controversial fourth criterion: collegiality. While collegiality plays a role in the morale and effectiveness of academic departments, it is amorphic and difficult to assess, and could be misused to stifle dissent or enforce homogeneity. Despite this, some institutions have opted to include this additional element in their RPT documents and processes, but it is unknown the extent of this practice and how it varies across institution type and disciplinary units. This study is based on two sets of data: survey data collected as part of a project that explored the publishing decisions of faculty and how these related to perceived importance in RPT processes, and 864 RPT documents collected from 129 universities from the United States and Canada. We analysed these RPT documents to determine the degree to which collegiality and related terms are mentioned, if they are defined, and if and how they may be assessed during the RPT process. Results show that when collegiality and related terms appear in these documents they are most often just briefly mentioned. It is less common for collegiality and related terms to be defined or assessed in RPT documents. Although the terms are mentioned across all types of institutions, there is a statistically significant difference in how prevalent they are at each. Collegiality is more commonly mentioned in the documents of doctoral research-focused universities (60%), than of master's universities and colleges (31%) or baccalaureate colleges (15%). Results from the accompanying survey of faculty also support this finding: individuals from R-Types were more likely to perceive collegiality to be a factor in their RPT processes. We conclude that collegiality likely plays an important role in RPT processes, whether it is explicitly acknowledged in policies and guidelines or not, and point to several strategies in how it might be best incorporated in the assessment of academic careers.

Analysis of single comments left for bioRxiv preprints till September 2019.

INTRODUCTION: While early commenting on studies is seen as one of the advantages of preprints, the type of such comments, and the people who post them, have not been systematically explored. MATERIALS AND METHODS: We analysed comments posted between 21 May 2015 and 9 September 2019 for 1983 bioRxiv preprints that received only one comment on the bioRxiv website. The comment types were classified by three coders independently, with all differences resolved by consensus. RESULTS: Our analysis showed that 69% of comments were posted by non-authors (N = 1366), and 31% by the preprints' authors themselves (N = 617). Twelve percent of non-author comments (N = 168) were full review reports traditionally found during journal review, while the rest most commonly contained praises (N = 577, 42%), suggestions (N = 399, 29%), or criticisms (N = 226, 17%). Authors' comments most commonly contained publication status updates (N = 354, 57%), additional study information (N = 158, 26%), or solicited feedback for the preprints (N = 65, 11%). CONCLUSIONS: Our results indicate that comments posted for bioRxiv preprints may have potential benefits for both the public and the scholarly community. Further research is needed to measure the direct impact of these comments on comments made by journal peer reviewers, subsequent preprint versions or journal publications.

How faculty define quality, prestige, and impact of academic journals.

Despite the calls for change, there is significant consensus that when it comes to evaluating publications, review, promotion, and tenure processes should aim to reward research that is of high "quality," is published in "prestigious" journals, and has an "impact." Nevertheless, such terms are highly subjective and present challenges to ascertain precisely what such research looks like. Accordingly, this article responds to the question: how do faculty from universities in the United States and Canada define the terms quality, prestige, and impact of academic journals? We address this question by surveying 338 faculty members from 55 different institutions in the U.S. and Canada. While relying on self-reported definitions that are not linked to their behavior, this study's findings highlight that faculty often describe these distinct terms in overlapping ways. Additionally, results show that marked variance in definitions across faculty does not correspond to demographic characteristics. This study's results highlight the subjectivity of common research terms and the importance of implementing evaluation regimes that do not rely on ill-defined concepts and may be context specific.

Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations.

Using an online survey of academics at 55 randomly selected institutions across the US and Canada, we explore priorities for publishing decisions and their perceived importance within review, promotion, and tenure (RPT). We find that respondents most value journal readership, while they believe their peers most value prestige and related metrics such as impact factor when submitting their work for publication. Respondents indicated that total number of publications, number of publications per year, and journal name recognition were the most valued factors in RPT. Older and tenured respondents (most likely to serve on RPT committees) were less likely to value journal prestige and metrics for publishing, while untenured respondents were more likely to value these factors. These results suggest disconnects between what academics value versus what they think their peers value, and between the importance of journal prestige and metrics for tenured versus untenured faculty in publishing and RPT perceptions.

Identifying diffusion patterns of research articles on Twitter: A case study of online engagement with open access articles.

The growing presence of research shared on social media, coupled with the increase in freely available research, invites us to ask whether scientific articles shared on platforms like Twitter diffuse beyond the academic community. We explore a new method for answering this question by identifying 11 articles from two open access biology journals that were shared on Twitter at least 50 times and by analyzing the follower network of users who tweeted each article. We find that diffusion patterns of scientific articles can take very different forms, even when the number of times they are tweeted is similar. Our small case study suggests that most articles are shared within single-connected communities with limited diffusion to the public. The proposed approach and indicators can serve those interested in the public understanding of science, science communication, or research evaluation to identify when research diffuses beyond insular communities.

Making headlines: an analysis of US government-funded cancer research mentioned in online media.

OBJECTIVE: To characterise how online media coverage of journal articles on cancer funded by the US government varies by cancer type and stage of the cancer control continuum and to compare the disease prevalence rates with the amount of funded research published for each cancer type and with the amount of media attention each receives. DESIGN: A cross-sectional study. SETTING: The United States. PARTICIPANTS: The subject of analysis was 11 436 journal articles on cancer funded by the US government published in 2016. These articles were identified via PubMed and characterised as receiving online media attention based on data provided by Altmetric. RESULTS: 16.8% (n=1925) of articles published on US government-funded research were covered in the media. Published journal articles addressed all common cancers. Frequency of journal articles differed substantially across the common cancers, with breast cancer (n=1284), lung cancer (n=630) and prostate cancer (n=586) being the subject of the most journal articles. Roughly one-fifth to one-fourth of journal articles within each cancer category received online media attention. Media mentions were disproportionate to actual burden of each cancer type (ie, incidence and mortality), with breast cancer articles receiving the most media mentions. Scientific articles also covered the stages of the cancer continuum to varying degrees. Across the 13 most common cancer types, 4.4% (n=206) of articles focused on prevention and control, 11.7% (n=550) on diagnosis and 10.7% (n=502) on therapy. CONCLUSIONS: Findings revealed a mismatch between prevalent cancers and cancers highlighted in online media. Further, journal articles on cancer control and prevention received less media attention than other cancer continuum stages. Media mentions were not proportional to actual public cancer burden nor volume of scientific publications in each cancer category. Results highlight a need for continued research on the role of media, especially online media, in research dissemination.

The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future.

Review, promotion, and tenure (RPT) processes significantly affect how faculty direct their own career and scholarly progression. Although RPT practices vary between and within institutions, and affect various disciplines, ranks, institution types, genders, and ethnicity in different ways, some consistent themes emerge when investigating what faculty would like to change about RPT. For instance, over the last few decades, RPT processes have generally increased the value placed on research, at the expense of teaching and service, which often results in an incongruity between how faculty actually spend their time vs. what is considered in their evaluation. Another issue relates to publication practices: most agree RPT requirements should encourage peer-reviewed works of high quality, but in practice, the value of publications is often assessed using shortcuts such as the prestige of the publication venue, rather than on the quality and rigor of peer review of each individual item. Open access and online publishing have made these issues even murkier due to misconceptions about peer review practices and concerns about predatory online publishers, which leaves traditional publishing formats the most desired despite their restricted circulation. And, efforts to replace journal-level measures such as the impact factor with more precise article-level metrics (e.g., citation counts and altmetrics) have been slow to integrate with the RPT process. Questions remain as to whether, or how, RPT practices should be changed to better reflect faculty work patterns and reduce pressure to publish in only the most prestigious traditional formats. To determine the most useful way to change RPT, we need to assess further the needs and perceptions of faculty and administrators, and gain a better understanding of the level of influence of written RPT guidelines and policy in an often vague process that is meant to allow for flexibility in assessing individuals.

Local chatter or international buzz? Language differences on posts about Zika research on Twitter and Facebook.

BACKGROUND: When the Zika virus outbreak became a global health emergency in early 2016, the scientific community responded with an increased output of Zika-related research. This upsurge in research naturally made its way into academic journals along with editorials, news, and reports. However, it is not yet known how or whether these scholarly communications were distributed to the populations most affected by Zika. METHODOLOGY/PRINCIPAL FINDINGS: To understand how scientific outputs about Zika reached global and local audiences, we collected Tweets and Facebook posts that linked to Zika-related research in the first six months of 2016. Using a language detection algorithm, we found that up to 90% of Twitter and 76% of Facebook posts are in English. However, when none of the authors of the scholarly article are from English-speaking countries, posts on both social media are less likely to be in English. The effect is most pronounced on Facebook, where the likelihood of posting in English is between 11 and 16% lower when none of the authors are from English-speaking countries, as compared to when some or all are. Similarly, posts about papers written with a Brazilian author are 13% more likely to be in Portuguese on Facebook than when made on Twitter. CONCLUSIONS/SIGNIFICANCE: Our main conclusion is that scholarly communication on Twitter and Facebook of Zika-related research is dominated by English, despite Brazil being the epicenter of the Zika epidemic. This result suggests that scholarly findings about the Zika virus are unlikely to be distributed directly to relevant populations through these popular online mediums. Nevertheless, there are differences between platforms. Compared to Twitter, scholarly communication on Facebook is more likely to be in the language of an author's country. The Zika outbreak provides a useful case-study for understanding how scientific outputs are communicated to relevant populations. Our results suggest that Facebook is a more effective channel than Twitter, if communication is desired to be in the native language of the affected country. Further research should explore how local media-such as governmental websites, newspapers and magazines, as well as television and radio-disseminate scholarly publication.

Authorial and institutional stratification in open access publishing: the case of global health research.

Using a database of recent articles published in the field of Global Health research, we examine institutional sources of stratification in publishing access outcomes. Traditionally, the focus on inequality in scientific publishing has focused on prestige hierarchies in established print journals. This project examines stratification in contemporary publishing with a particular focus on subscription vs. various Open Access (OA) publishing options. Findings show that authors working at lower-ranked universities are more likely to publish in closed/paywalled outlets, and less likely to choose outlets that involve some sort of Article Processing Charge (APCs; gold or hybrid OA). We also analyze institutional differences and stratification in the APC costs paid in various journals. Authors affiliated with higher-ranked institutions, as well as hospitals and non-profit organizations pay relatively higher APCs for gold and hybrid OA publications. Results suggest that authors affiliated with high-ranked universities and well-funded institutions tend to have more resources to choose pay options with publishing. Our research suggests new professional hierarchies developing in contemporary publishing, where various OA publishing options are becoming increasingly prominent. Just as there is stratification in institutional representation between different types of publishing access, there is also inequality within access types.

The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles.

Despite growing interest in Open Access (OA) to scholarly literature, there is an unmet need for large-scale, up-to-date, and reproducible studies assessing the prevalence and characteristics of OA. We address this need using oaDOI, an open online service that determines OA status for 67 million articles. We use three samples, each of 100,000 articles, to investigate OA in three populations: (1) all journal articles assigned a Crossref DOI, (2) recent journal articles indexed in Web of Science, and (3) articles viewed by users of Unpaywall, an open-source browser extension that lets users find OA articles using oaDOI. We estimate that at least 28% of the scholarly literature is OA (19M in total) and that this proportion is growing, driven particularly by growth in Gold and Hybrid. The most recent year analyzed (2015) also has the highest percentage of OA (45%). Because of this growth, and the fact that readers disproportionately access newer articles, we find that Unpaywall users encounter OA quite frequently: 47% of articles they view are OA. Notably, the most common mechanism for OA is not Gold, Green, or Hybrid OA, but rather an under-discussed category we dub Bronze: articles made free-to-read on the publisher website, without an explicit Open license. We also examine the citation impact of OA articles, corroborating the so-called open-access citation advantage: accounting for age and discipline, OA articles receive 18% more citations than average, an effect driven primarily by Green and Hybrid OA. We encourage further research using the free oaDOI service, as a way to inform OA policy and practice.

De la tecnocracia a la equidad y la sustentabilidad: por otra ciencia abierta / From technocracy to equity and sustainability: for another open science

Foi por meio dos estudos sobre softwares de periódicos científicos que Juan Pablo Alperin passou a desenvolver pesquisas sobre a comunicação científica, principalmente em relação à temática de acesso aberto e sua questão mais ampla que envolve a ciência aberta. Em entrevista à Reciis, o professor e pesquisador argumenta que é preciso pensar em como se abrir o acesso ao conhecimento e não apenas garanti-lo. Nesse sentido, a reflexão sobre como fazer ciência aberta promove discussões para tornar o sistema de pesquisa científica mais equânime e colaborativo. O professor comenta sobre os efeitos positivos e os desafios que a pandemia de covid-19 promoveu em relação à comunicação científica. Alperin destaca iniciativas na América Latina voltadas a uma ciência mais democrática e reafirma a necessidade de se defender uma ciência aberta que interrogue o seu modo de agir, sendo, com isso, menos tecnocrática. Juan Pablo Alperin é professor associado da Simon Fraser University.