Identifying barriers and enablers to rigorous conduct and reporting of preclinical laboratory studies.

Use of rigorous study design methods and transparent reporting in publications are 2 key strategies proposed to improve the reproducibility of preclinical research. Despite promotion of these practices by funders and journals, assessments suggest uptake is low in preclinical research. Thirty preclinical scientists were interviewed to better understand barriers and enablers to rigorous design and reporting. The interview guide was informed by the Theoretical Domains Framework, which is a framework used to understand determinants of current and desired behavior. Four global themes were identified; 2 reflecting enablers and 2 reflecting barriers. We found that basic scientists are highly motivated to apply the methods of rigorous design and reporting and perceive a number of benefits to their adoption (e.g., improved quality and reliability). However, there was varied awareness of the guidelines and in implementation of these practices. Researchers also noted that these guidelines can result in disadvantages, such as increased sample sizes, expenses, time, and can require several personnel to operationalize. Most researchers expressed additional resources such as personnel and education/training would better enable the application of some methods. Using existing guidance (Behaviour Change Wheel (BCW); Expert Recommendations for Implementing Change (ERIC) project implementation strategies), we mapped and coded our interview findings to identify potential interventions, policies, and implementation strategies to improve routine use of the guidelines by preclinical scientists. These findings will help inform specific strategies that may guide the development of programs and resources to improve experimental design and transparent reporting in preclinical research.

Community consensus on core open science practices to monitor in biomedicine.

The state of open science needs to be monitored to track changes over time and identify areas to create interventions to drive improvements. In order to monitor open science practices, they first need to be well defined and operationalized. To reach consensus on what open science practices to monitor at biomedical research institutions, we conducted a modified 3-round Delphi study. Participants were research administrators, researchers, specialists in dedicated open science roles, and librarians. In rounds 1 and 2, participants completed an online survey evaluating a set of potential open science practices, and for round 3, we hosted two half-day virtual meetings to discuss and vote on items that had not reached consensus. Ultimately, participants reached consensus on 19 open science practices. This core set of open science practices will form the foundation for institutional dashboards and may also be of value for the development of policy, education, and interventions.

Evaluating the Impact of Peer Review on the Completeness of Reporting in Imaging Diagnostic Test Accuracy Research.

BACKGROUND: Despite the nearly ubiquitous reported use of peer review among reputable medical journals, there is limited evidence to support the use of peer review to improve the quality of biomedical research and in particular, imaging diagnostic test accuracy (DTA) research. PURPOSE: To evaluate whether peer review of DTA studies published by imaging journals is associated with changes in completeness of reporting, transparency for risk of bias assessment, and spin. STUDY TYPE: Retrospective cross-sectional study. STUDY SAMPLE: Cross-sectional study of articles published in Journal of Magnetic Resonance Imaging (JMRI), Canadian Association of Radiologists Journal (CARJ), and European Radiology (EuRad) before March 31, 2020. ASSESSMENT: Initial submitted and final versions of manuscripts were evaluated for completeness of reporting using the Standards for Reporting Diagnostic Accuracy Studies (STARD) 2015 and STARD for Abstracts guidelines, transparency of reporting for risk of bias assessment based on Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2), and actual and potential spin using modified published criteria. STATISTICAL TESTS: Two-tailed paired t-tests and paired Wilcoxon signed-rank tests were used for comparisons. A P value <0.05 was considered to be statistically significant. RESULTS: We included 84 diagnostic accuracy studies accepted by three journals between 2014 and 2020 (JMRI = 30, CARJ = 23, and EuRad = 31) of the 692 which were screened. Completeness of reporting according to STARD 2015 increased significantly between initial submissions and final accepted versions (average reported items: 16.67 vs. 17.47, change of 0.80 [95% confidence interval 0.25-1.17]). No significant difference was found for the reporting of STARD for Abstracts (5.28 vs. 5.25, change of -0.03 [-0.15 to 0.11], P = 0.74), QUADAS-2 (6.08 vs. 6.11, change of 0.03 [-1.00 to 0.50], P = 0.92), actual "spin" (2.36 vs. 2.40, change of 0.04 [0.00 to 1.00], P = 0.39) or potential "spin" (2.93 vs. 2.81, change of -0.12 [-1.00 to 0.00], P = 0.23) practices. CONCLUSION: Peer review is associated with a marginal improvement in completeness of reporting in published imaging DTA studies, but not with improvement in transparency for risk of bias assessment or reduction in spin. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY STAGE: 1.

Stress testing journals: a quasi-experimental study of rejection rates of a previously published paper.

BACKGROUND: When a journal receives a duplicate publication, the ability to identify the submitted work as previously published, and reject it, is an assay to publication ethics best practices. The aim of this study was to evaluate how three different types of journals, namely open access (OA) journals, subscription-based journals, and presumed predatory journals, responded to receiving a previously published manuscript for review. METHODS: We performed a quasi-experimental study in which we submitted a previously published article to a random sample of 602 biomedical journals, roughly 200 journals from each journal type sampled: OA journals, subscription-based journals, and presumed predatory journals. Three hundred and three journals received a Word version in manuscript format, while 299 journals received the formatted publisher's PDF version of the published article. We then recorded responses to the submission received after approximately 1 month. Responses were reviewed, extracted, and coded in duplicate. Our primary outcome was the rate of rejection of the two types of submitted articles (PDF vs Word) within our three journal types. RESULTS: We received correspondence back from 308 (51.1%) journals within our study timeline (32 days); (N = 46 predatory journals, N = 127 OA journals, N = 135 subscription-based journals). Of the journals that responded, 153 received the Word version of the paper, while 155 received the PDF version. Four journals (1.3%) accepted our paper, 291 (94.5%) journals rejected the paper, and 13 (4.2%) requested a revision. A chi-square test looking at journal type, and submission type, was significant (χ2 (4) = 23.50, p < 0.001). All four responses to accept our article came from presumed predatory journals, 3 of which received the Word format and 1 that received the PDF format. Less than half of journals that rejected our submissions did so because they identified ethical issues such as plagiarism with the manuscript (133 (45.7%)). CONCLUSION: Few journals accepted our submitted paper. However, our findings suggest that all three types of journals may not have adequate safeguards in place to recognize and act on plagiarism or duplicate submissions.

Care plans for women pregnant using assisted reproductive technologies: a systematic review.

BACKGROUND: Between 1 and 5% of children in industrialized countries are conceived through Assisted Reproductive Technologies (ART). As infertility and the use of ART may be associated with adverse perinatal outcomes, care plans specific to these pregnancies are needed. We conducted a systematic review to examine the existing care plans specific to women pregnant following Assisted Reproductive Technologies (ART). METHODS: MEDLINE, Embase and the Cochrane Library were searched by a senior information specialist. The population of interest included women becoming pregnant with ART (e.g., Intra-Uterine Insemination, In Vitro Fertilization (IVF), Intracytoplasmic Sperm Injection (ICSI), and surrogacy). All proposed care plans were sought that pertained to any aspect of care during pregnancy and delivery. Only Clinical Practice Guidelines (CPGs) addressing the recommendations and plans for the care of ART pregnant women were included. The search was restricted to the publication dates 2007 to June 12, 2017 when the search was run. The search was not restricted by language, however only English and French language guidelines were considered for inclusion. RESULTS: After screening 2078 citations, a total of ten CPGs were included. The following key clinical messages were prevalent: (1) although there was no supporting evidence, antenatal care for ART pregnancies should be provided by specialist with knowledge in obstetrics; (2) high-order multiple pregnancies are the greatest risk of ART and selective reduction options should be discussed; (3) there is some evidence of increased risk of congenital abnormalities and prenatal genetic and anatomic screening is recommended, especially in IVF-ICSI pregnancies; (4) due to a lack of or conflicting evidence, treatment of venous thromboembolism, antithrombotic therapy, treatment for hypothyroidism, and women with positive thyroid antibodies is recommended to be the same as in spontaneous pregnancies; and lastly (5) since an increased level of distress is a recognized feature in these pregnancies, psychosocial care and counselling should be considered. CONCLUSIONS: There is a lack of CPGs specific to ART pregnancies. While we identified a small number of recommendations for ART pregnancies, specific interventions and models of care aiming at decreasing adverse maternal and perinatal outcomes following ART should be developed, implemented, and evaluated.

Predatory Invitations from Journals: More Than Just a Nuisance?

Physicians and academic researchers are frequently targeted with spam invitations to submit manuscripts to predatory journals. This study was conducted to understand the nature and characteristics of these invitations. All spam e-mails received by an academic medical oncologist over a 3-month period were collected and categorized. Presumed predatory journal invitations were analyzed and cross-checked against Beall's list of "potential, probable, or possible predatory" journals and publishers. Invitations to submit to predatory journals were the most common single type of spam received. The Oncologist 2017;22:236-240.

Core competencies for scientific editors of biomedical journals: consensus statement.

BACKGROUND: Scientific editors are responsible for deciding which articles to publish in their journals. However, we have not found documentation of their required knowledge, skills, and characteristics, or the existence of any formal core competencies for this role. METHODS: We describe the development of a minimum set of core competencies for scientific editors of biomedical journals. RESULTS: The 14 key core competencies are divided into three major areas, and each competency has a list of associated elements or descriptions of more specific knowledge, skills, and characteristics that contribute to its fulfillment. CONCLUSIONS: We believe that these core competencies are a baseline of the knowledge, skills, and characteristics needed to perform competently the duties of a scientific editor at a biomedical journal.

Hormonal Contraceptive Use During Relationship Formation and Sexual Desire During Pregnancy.

Women who are regularly cycling exhibit different partner preferences than those who use hormonal contraception. Preliminary evidence appears to suggest that during pregnancy women's partner preferences also diverge from those prevalent while regularly cycling. This is consistent with the general assertion that women's mate preferences are impacted by hormonal variation. During pregnancy, women's preferences are thought to closely resemble those displayed by women who are using hormonal contraception. Here, based on this literature, we compared levels of sexual desire among pregnant women who met their partner while using hormonal contraception and pregnant women who met their partner while regularly cycling. We predicted that women who met their partner while using hormonal contraception would experience higher levels of in-pair sexual desire during pregnancy since these women will have partner preferences that more closely match those prevalent at the time of their partner choice. Our results provided support for the idea that previous contraceptive use/non-use may impact subsequent sexual desire for the partner during pregnancy. Pregnant women who met their partner while using hormonal contraception (N = 37) were shown to have higher levels of in-pair sexual desire than those who met while regularly cycling (N = 47). In contrast, levels of extra-pair desire were not related to previous use/non-use of hormonal contraception. These findings were robust when controlling for a number of relevant individual difference variables known to impact sexual desire. Our results contribute to our understanding of factors affecting relationship functioning during pregnancy.

Partner choice, relationship satisfaction, and oral contraception: the congruency hypothesis.

Hormonal fluctuation across the menstrual cycle explains temporal variation in women's judgment of the attractiveness of members of the opposite sex. Use of hormonal contraceptives could therefore influence both initial partner choice and, if contraceptive use subsequently changes, intrapair dynamics. Associations between hormonal contraceptive use and relationship satisfaction may thus be best understood by considering whether current use is congruent with use when relationships formed, rather than by considering current use alone. In the study reported here, we tested this congruency hypothesis in a survey of 365 couples. Controlling for potential confounds (including relationship duration, age, parenthood, and income), we found that congruency in current and previous hormonal contraceptive use, but not current use alone, predicted women's sexual satisfaction with their partners. Congruency was not associated with women's nonsexual satisfaction or with the satisfaction of their male partners. Our results provide empirical support for the congruency hypothesis and suggest that women's sexual satisfaction is influenced by changes in partner preference associated with change in hormonal contraceptive use.

Key concepts in clinical epidemiology: research integrity definitions and challenges.

Research integrity is guided by a set of principles to ensure research reliability and rigor. It serves as a pillar to uphold society's trust in science and foster scientific progress. However, over the past 2 decades, a surge in research integrity concerns, including fraudulent research, reproducibility challenges, and questionable practices, has raised critical questions about the reliability of scientific outputs, particularly in biomedical research. In the biomedical sciences, any breaches in research integrity could potentially lead to a domino effect impacting patient care, medical interventions, and the broader implementation of healthcare policies. Addressing these breaches requires measures such as rigorous research methods, transparent reporting, and changing the research culture. Institutional support through clear guidelines, robust training, and mentorship is crucial to fostering a culture of research integrity. However, structural and institutional factors, including research incentives and recognition systems, play an important role in research behavior. Therefore, promoting research integrity demands a collective effort from all stakeholders to maintain public trust in the scientific community and ensure the reliability of science. Here we discuss some definitions and principles, the implications for biomedical sciences, and propose actionable steps to foster research integrity.

Complementary, alternative, and integrative medicine researchers' practices and perceived barriers related to open science: An international, cross-sectional survey.

INTRODUCTION AND OBJECTIVE: Open science (OS) aims to make the dissemination of knowledge and the research process transparent and accessible to everyone. With the increasing popularity of complementary, alternative, and integrative medicine (CAIM), our goal was to explore what are CAIM researchers' practices and perceived barriers related to OS. METHODS: We conducted an anonymous online survey of researchers who published in journals listed in Scopus containing the words "complementary", "alternative", or "integrative" medicine in their names. We emailed 6040 researchers our purpose-built electronic survey after extracting their email address from one of their publications in our sample of journals. We questioned their familiarity with different OS concepts, along with their experiences and challenges engaging in these practices over the last 12 months. RESULTS: The survey was completed by 392 researchers (6.5% response rate, 97.1% completion rate). Most respondents were CAIM researchers familiar with the overall concept of OS, indicated by those actively publishing open access (OA) (n = 244, 76.0%), registering a study protocol (n = 148, 48.0%), and using reporting guidelines (n = 181, 59.0%) in the past 12 months. Preprinting, sharing raw data, and sharing study materials were less popular. A lack of funding was reported as the greatest barrier to publishing OA by most respondents (n = 252, 79.0%), and that additional funding is the most significant incentive in applying more OS practices to their research (n = 229,72.2%). With respect to preprinting barriers, 36.3% (n = 110) participants believed there are potential harms in sharing non-peer-reviewed work and 37.0% (n = 112) feared preprinting would reduce the likelihood of their manuscript being accepted by a journal. Respondents were also concerned about intellectual property control regarding sharing data (n = 94, 31.7%) and research study materials (n = 80, 28.7%). CONCLUSIONS: Although many participants were familiar with and practiced aspects of OS, many reported facing barriers relating to lack of funding to enable OS and perceived risks of revealing research ideas and data prior to publication. Future research should monitor the adoption and implementation of OS interventions in CAIM.

Cardiology researchers' practices and perceived barriers to open science: an international survey.

OBJECTIVE: Open science is a movement and set of practices to conduct research more transparently. Implementing open science will significantly improve public access and supports equity. It also has the potential to foster innovation and reduce duplication through data and materials sharing. Here, we survey an international group of researchers publishing in cardiovascular journals regarding their perceptions and practices related to open science. METHODS: We identified the top 100 'Cardiology and Cardiovascular Medicine' subject category journals from the SCImago journal ranking platform. This is a publicly available portal that draws from Scopus. We then extracted the corresponding author's name and email from all articles published in these journals between 1 March 2021 and 1 March 2022. Participants were sent a purpose-built survey about open science. The survey contained primarily multiple choice and scale-based questions for which we report count data and percentages. For the few text-based responses we conducted thematic content analysis. RESULTS: 198 participants responded to our survey. Participants had a mean response of 6.8 (N=197, SD=1.8) on a 9-point scale with endpoints, not at all familiar (1) and extremely familiar (9), when indicating how familiar they were with open science. When asked about where they obtained open science training, most participants indicated this was done on the job self-initiated while conducting research (n=103, 52%), or that they had no formal training with respect to open science (n=72, 36%). More than half of the participants indicated they would benefit from practical support from their institution on how to perform open science practices (N=106, 54%). A diversity of barriers to each of the open science practices presented to participants were acknowledged. Participants indicated that funding was the most essential incentive to adopt open science. CONCLUSIONS: It is clear that policy alone will not lead to the effective implementation of open science. This survey serves as a baseline for the cardiovascular research community's open science performance and perception and can be used to inform future interventions and monitoring.

Knowledge and motivations of training in peer review: An international cross-sectional survey.

BACKGROUND: Despite having a crucial role in scholarly publishing, peer reviewers do not typically require any training. The purpose of this study was to conduct an international survey on the current perceptions and motivations of researchers regarding peer review training. METHODS: A cross-sectional online survey was conducted of biomedical researchers. A total of 2000 corresponding authors from 100 randomly selected medical journals were invited via email. Quantitative items were reported using frequencies and percentages or means and SE, as appropriate. A thematic content analysis was conducted for qualitative items in which two researchers independently assigned codes to the responses for each written-text question, and subsequently grouped the codes into themes. A descriptive definition of each category was then created and unique themes-as well as the number and frequency of codes within each theme-were reported. RESULTS: A total of 186 participants completed the survey of which 14 were excluded. The majority of participants indicated they were men (n = 97 of 170, 57.1%), independent researchers (n = 108 of 172, 62.8%), and primarily affiliated with an academic organization (n = 103 of 170, 62.8%). A total of 144 of 171 participants (84.2%) indicated they had never received formal training in peer review. Most participants (n = 128, 75.7%) agreed-of which 41 (32.0%) agreed strongly-that peer reviewers should receive formal training in peer review prior to acting as a peer reviewer. The most preferred training formats were online courses, online lectures, and online modules. Most respondents (n = 111 of 147, 75.5%) stated that difficulty finding and/or accessing training was a barrier to completing training in peer review. CONCLUSION: Despite being desired, most biomedical researchers have not received formal training in peer review and indicated that training was difficult to access or not available.

Limited online training opportunities exist for scholarly peer reviewers.

OBJECTIVES: To create a comprehensive list of all openly available online trainings in scholarly peer review and to analyze their characteristics. STUDY DESIGN AND SETTING: A systematic review of online training material in scholarly peer review openly accessible between 2012 and 2022. Training characteristics were presented in evidence tables and summarized narratively. A risk of bias tool was purpose-built for this study to evaluate the included training material as evidence-based. RESULTS: Fourty-two training opportunities in manuscript peer review were identified, of which only twenty were openly accessible. Most were online modules (n = 12, 60%) with an estimated completion time of less than 1 hour (n = 13, 65%). Using our ad hoc risk of bias tool, four sources (20%) met our criteria of evidence-based. CONCLUSION: Our comprehensive search of the literature identified 20 openly accessible online training materials in manuscript peer review. For such a crucial step in the dissemination of literature, a lack of training could potentially explain disparities in the quality of scholarly publishing.

Epidemiological characteristics and prevalence rates of research reproducibility across disciplines: A scoping review of articles published in 2018-2019.

Background: Reproducibility is a central tenant of research. We aimed to synthesize the literature on reproducibility and describe its epidemiological characteristics, including how reproducibility is defined and assessed. We also aimed to determine and compare estimates for reproducibility across different fields. Methods: We conducted a scoping review to identify English language replication studies published between 2018 and 2019 in economics, education, psychology, health sciences, and biomedicine. We searched Medline, Embase, PsycINFO, Cumulative Index of Nursing and Allied Health Literature - CINAHL, Education Source via EBSCOHost, ERIC, EconPapers, International Bibliography of the Social Sciences (IBSS), and EconLit. Documents retrieved were screened in duplicate against our inclusion criteria. We extracted year of publication, number of authors, country of affiliation of the corresponding author, and whether the study was funded. For the individual replication studies, we recorded whether a registered protocol for the replication study was used, whether there was contact between the reproducing team and the original authors, what study design was used, and what the primary outcome was. Finally, we recorded how reproducibilty was defined by the authors, and whether the assessed study(ies) successfully reproduced based on this definition. Extraction was done by a single reviewer and quality controlled by a second reviewer. Results: Our search identified 11,224 unique documents, of which 47 were included in this review. Most studies were related to either psychology (48.6%) or health sciences (23.7%). Among these 47 documents, 36 described a single reproducibility study while the remaining 11 reported at least two reproducibility studies in the same paper. Less than the half of the studies referred to a registered protocol. There was variability in the definitions of reproduciblity success. In total, across the 47 documents 177 studies were reported. Based on the definition used by the author of each study, 95 of 177 (53.7%) studies reproduced. Conclusions: This study gives an overview of research across five disciplines that explicitly set out to reproduce previous research. Such reproducibility studies are extremely scarce, the definition of a successfully reproduced study is ambiguous, and the reproducibility rate is overall modest. Funding: No external funding was received for this work.