Structured peer review: pilot results from 23 Elsevier journals.

Background: Reviewers rarely comment on the same aspects of a manuscript, making it difficult to properly assess manuscripts' quality and the quality of the peer review process. The goal of this pilot study was to evaluate structured peer review implementation by: 1) exploring whether and how reviewers answered structured peer review questions, 2) analysing reviewer agreement, 3) comparing that agreement to agreement before implementation of structured peer review, and 4) further enhancing the piloted set of structured peer review questions. Methods: Structured peer review consisting of nine questions was piloted in August 2022 in 220 Elsevier journals. We randomly selected 10% of these journals across all fields and IF quartiles and included manuscripts that received two review reports in the first 2 months of the pilot, leaving us with 107 manuscripts belonging to 23 journals. Eight questions had open-ended fields, while the ninth question (on language editing) had only a yes/no option. The reviews could also leave Comments-to-Author and Comments-to-Editor. Answers were independently analysed by two raters, using qualitative methods. Results: Almost all the reviewers (n = 196, 92%) provided answers to all questions even though these questions were not mandatory in the system. The longest answer (Md 27 words, IQR 11 to 68) was for reporting methods with sufficient details for replicability or reproducibility. The reviewers had the highest (partial) agreement (of 72%) for assessing the flow and structure of the manuscript, and the lowest (of 53%) for assessing whether interpretation of the results was supported by data, and for assessing whether the statistical analyses were appropriate and reported in sufficient detail (52%). Two thirds of the reviewers (n = 145, 68%) filled out the Comments-to-Author section, of which 105 (49%) resembled traditional peer review reports. These reports contained a Md of 4 (IQR 3 to 5) topics covered by the structured questions. Absolute agreement regarding final recommendations (exact match of recommendation choice) was 41%, which was higher than what those journals had in the period from 2019 to 2021 (31% agreement, P = 0.0275). Conclusions: Our preliminary results indicate that reviewers successfully adapted to the new review format, and that they covered more topics than in their traditional reports. Individual question analysis indicated the greatest disagreement regarding the interpretation of the results and the conducting and the reporting of statistical analyses. While structured peer review did lead to improvement in reviewer final recommendation agreements, this was not a randomized trial, and further studies should be performed to corroborate this. Further research is also needed to determine whether structured peer review leads to greater knowledge transfer or better improvement of manuscripts.

What is the state of the art on traditional medicine interventions for zoonotic diseases in the Indian subcontinent? A scoping review of the peer-reviewed evidence base.

BACKGROUND: Traditional medicine (TM) interventions are plausible therapeutic alternatives to conventional medical interventions against emerging and endemic zoonotic diseases, particularly in low-and middle-income countries that may lack resources and infrastructure. Despite the growing popularity in the usage of TM interventions, their clinical safety and effectiveness are still contested within conventional healthcare in many countries. METHODS: We conducted a scoping review of the peer-reviewed literature that synthesises and maps the evidence on TM interventions for the treatment and prevention of zoonoses on the Indian subcontinent. The region, a global hotspot of biodiversity and emerging infections, is characterised by high prevalence of TM use. Based on the scientific literature (mostly case study research, n=l06 studies), our review (1) maps the scope of the literature, (2) synthesises the evidence on the application of TM interventions for zoonoses, and (3) critically reflects on the state of TM and identifies areas for future research focus. RESULTS: The evidence synthesis confirmed widespread usage of TM interventions for zoonoses on the subcontinent, with the majority of research reported from India (n=99 studies, 93.4%), followed by Pakistan (n=3 studies, 2.8%), Bangladesh (n=2 studies, 1.9%), and Sri Lanka (n=1, 0.9%). Most of the reviewed studies reported on ethno-medicinal uses of plant species, primarily for treating dengue (n=20 studies), tuberculosis (n=18 studies), Escherichia coli infection (n=16 studies), lymphatic filariasis and cholera (n=9 apiece). However, the evidence on the safety and effectiveness of these reported TM interventions is limited, indicating that these data are rarely collected and/or shared within the peer-reviewed literature. CONCLUSION: This review thus highlights that, whilst TMs are already being used and could offer more widely accessible interventions against emerging and endemic zoonoses and ectoparasites, there is an urgent need for rigorous clinical testing and validation of the safety and effectiveness of these interventions.

Clinical Nurse Peer Review: A Process That Works for the Nurse and the Organization.

Clinical peer review is a strategy that engages nurses in elevating not only the safety of patients but also their influence on practice. There is little guidance in the literature about how to operationalize peer review in a way that promotes just culture. In a postpandemic era, where nurse engagement and retention are low, this article describes how to implement and measure the impact of clinical peer review on practice trends and empower nurses to influence system-wide change.

Assessing ChatGPT's ability to emulate human reviewers in scientific research: A descriptive and qualitative approach.

BACKGROUND: ChatGPT is an AI platform whose relevance in the peer review of scientific articles is steadily growing. Nonetheless, it has sparked debates over its potential biases and inaccuracies. This study aims to assess ChatGPT's ability to qualitatively emulate human reviewers in scientific research. METHODS: We included the first submitted version of the latest twenty original research articles published by the 3rd of July 2023, in a high-profile medical journal. Each article underwent evaluation by a minimum of three human reviewers during the initial review stage. Subsequently, three researchers with medical backgrounds and expertise in manuscript revision, independently and qualitatively assessed the agreement between the peer reviews generated by ChatGPT version GPT-4 and the comments provided by human reviewers for these articles. The level of agreement was categorized into complete, partial, none, or contradictory. RESULTS: 720 human reviewers' comments were assessed. There was a good agreement between the three assessors (Overall kappa >0.6). ChatGPT's comments demonstrated complete agreement in terms of quality and substance with 48 (6.7 %) human reviewers' comments, partially agreed with 92 (12.8 %), identifying issues necessitating further elaboration or recommending supplementary steps to address concerns, had no agreement with a significant 565 (78.5 %), and contradicted 15 (2.1 %). ChatGPT comments on methods had the lowest proportion of complete agreement (13 comments, 3.6 %), while general comments on the manuscript displayed the highest proportion of complete agreement (17 comments, 22.1 %). CONCLUSION: ChatGPT version GPT-4 has a limited ability to emulate human reviewers within the peer review process of scientific research.

How and why eLife selects papers for peer review.

When deciding which submissions should be peer reviewed, eLife editors consider whether they will be able to find high-quality reviewers, and whether the reviews will be valuable to the scientific community.

Unlocking Peer Review: Elevating Scholarly Writing and Research Competence in Urology Residency.

PURPOSE OF REVIEW: It is incumbent upon training programs to set the foundation for evidence-based practices and to create opportunities for trainees to develop into academic leaders. As dedicated resident research time and funding have declined in recent years, residency programs and the field at large will need to create new ways to incorporate scholarly activity into residency curricula. RECENT FINDINGS: Literature across specialties demonstrates barriers to resident involvement including lack of time, cost, and absent scholarly mentorship. Peer review stands as a ready-made solution that can be formalized into a collaborative relationship with journals. A formal relationship between professional societies, academic journals, and residencies can facilitate the use of peer review as a teaching tool for residency programs.

Where have all the reviewers gone?

In this editorial the editor considers the growing challenges journals are facing in securing peer reviewers, some of the approaches being tried to address this problem, and the prospects for sustaining communities of scholars with and without an ongoing commitment to peer review.

Scientific and Regulatory Policy Committee Points to Consider: Review of the United States Food and Drug Administration (FDA) Guidance on Pathology Peer Review in Nonclinical Toxicology Studies.

In December 2021, the United States Food and Drug Administration (FDA) issued the final guidance for industry titled Pathology Peer Review in Nonclinical Toxicology Studies: Questions and Answers. The stated purpose of the FDA guidance is to provide information to sponsors, applicants, and nonclinical laboratory personnel regarding the management and conduct of histopathology peer review as part of nonclinical toxicology studies conducted in compliance with good laboratory practice (GLP) regulations. On behalf of and in collaboration with global societies of toxicologic pathology and the Society of Quality Assurance, the Scientific and Regulatory Policy Committee (SRPC) of the Society of Toxicologic Pathology (STP) initiated a review of this FDA guidance. The STP has previously published multiple papers related to the scientific conduct of a pathology peer review of nonclinical toxicology studies and appropriate documentation practices. The objectives of this review are to provide an in-depth analysis and summary interpretation of the FDA recommendations and share considerations for the conduct of pathology peer review in nonclinical toxicology studies that claim compliance to GLP regulations. In general, this working group is in agreement with the recommendations from the FDA guidance that has added clear expectations for pathology peer review preparation, conduct, and documentation.