Structured peer review: pilot results from 23 Elsevier journals.

Background: Reviewers rarely comment on the same aspects of a manuscript, making it difficult to properly assess manuscripts' quality and the quality of the peer review process. The goal of this pilot study was to evaluate structured peer review implementation by: 1) exploring whether and how reviewers answered structured peer review questions, 2) analysing reviewer agreement, 3) comparing that agreement to agreement before implementation of structured peer review, and 4) further enhancing the piloted set of structured peer review questions. Methods: Structured peer review consisting of nine questions was piloted in August 2022 in 220 Elsevier journals. We randomly selected 10% of these journals across all fields and IF quartiles and included manuscripts that received two review reports in the first 2 months of the pilot, leaving us with 107 manuscripts belonging to 23 journals. Eight questions had open-ended fields, while the ninth question (on language editing) had only a yes/no option. The reviews could also leave Comments-to-Author and Comments-to-Editor. Answers were independently analysed by two raters, using qualitative methods. Results: Almost all the reviewers (n = 196, 92%) provided answers to all questions even though these questions were not mandatory in the system. The longest answer (Md 27 words, IQR 11 to 68) was for reporting methods with sufficient details for replicability or reproducibility. The reviewers had the highest (partial) agreement (of 72%) for assessing the flow and structure of the manuscript, and the lowest (of 53%) for assessing whether interpretation of the results was supported by data, and for assessing whether the statistical analyses were appropriate and reported in sufficient detail (52%). Two thirds of the reviewers (n = 145, 68%) filled out the Comments-to-Author section, of which 105 (49%) resembled traditional peer review reports. These reports contained a Md of 4 (IQR 3 to 5) topics covered by the structured questions. Absolute agreement regarding final recommendations (exact match of recommendation choice) was 41%, which was higher than what those journals had in the period from 2019 to 2021 (31% agreement, P = 0.0275). Conclusions: Our preliminary results indicate that reviewers successfully adapted to the new review format, and that they covered more topics than in their traditional reports. Individual question analysis indicated the greatest disagreement regarding the interpretation of the results and the conducting and the reporting of statistical analyses. While structured peer review did lead to improvement in reviewer final recommendation agreements, this was not a randomized trial, and further studies should be performed to corroborate this. Further research is also needed to determine whether structured peer review leads to greater knowledge transfer or better improvement of manuscripts.

What is the state of the art on traditional medicine interventions for zoonotic diseases in the Indian subcontinent? A scoping review of the peer-reviewed evidence base.

BACKGROUND: Traditional medicine (TM) interventions are plausible therapeutic alternatives to conventional medical interventions against emerging and endemic zoonotic diseases, particularly in low-and middle-income countries that may lack resources and infrastructure. Despite the growing popularity in the usage of TM interventions, their clinical safety and effectiveness are still contested within conventional healthcare in many countries. METHODS: We conducted a scoping review of the peer-reviewed literature that synthesises and maps the evidence on TM interventions for the treatment and prevention of zoonoses on the Indian subcontinent. The region, a global hotspot of biodiversity and emerging infections, is characterised by high prevalence of TM use. Based on the scientific literature (mostly case study research, n=l06 studies), our review (1) maps the scope of the literature, (2) synthesises the evidence on the application of TM interventions for zoonoses, and (3) critically reflects on the state of TM and identifies areas for future research focus. RESULTS: The evidence synthesis confirmed widespread usage of TM interventions for zoonoses on the subcontinent, with the majority of research reported from India (n=99 studies, 93.4%), followed by Pakistan (n=3 studies, 2.8%), Bangladesh (n=2 studies, 1.9%), and Sri Lanka (n=1, 0.9%). Most of the reviewed studies reported on ethno-medicinal uses of plant species, primarily for treating dengue (n=20 studies), tuberculosis (n=18 studies), Escherichia coli infection (n=16 studies), lymphatic filariasis and cholera (n=9 apiece). However, the evidence on the safety and effectiveness of these reported TM interventions is limited, indicating that these data are rarely collected and/or shared within the peer-reviewed literature. CONCLUSION: This review thus highlights that, whilst TMs are already being used and could offer more widely accessible interventions against emerging and endemic zoonoses and ectoparasites, there is an urgent need for rigorous clinical testing and validation of the safety and effectiveness of these interventions.

Where have all the reviewers gone?

In this editorial the editor considers the growing challenges journals are facing in securing peer reviewers, some of the approaches being tried to address this problem, and the prospects for sustaining communities of scholars with and without an ongoing commitment to peer review.

Unlocking Peer Review: Elevating Scholarly Writing and Research Competence in Urology Residency.

PURPOSE OF REVIEW: It is incumbent upon training programs to set the foundation for evidence-based practices and to create opportunities for trainees to develop into academic leaders. As dedicated resident research time and funding have declined in recent years, residency programs and the field at large will need to create new ways to incorporate scholarly activity into residency curricula. RECENT FINDINGS: Literature across specialties demonstrates barriers to resident involvement including lack of time, cost, and absent scholarly mentorship. Peer review stands as a ready-made solution that can be formalized into a collaborative relationship with journals. A formal relationship between professional societies, academic journals, and residencies can facilitate the use of peer review as a teaching tool for residency programs.

So .You Want to be a Peer Reviewer?

Peer review is essential to preserving the integrity of the scientific publication process. Peer reviewers must adhere to the norms of the peer review process, including confidentiality, avoiding actual or apparent conflicts of interest, timeliness, constructiveness, and thoroughness. This mini review will discuss some of the different formats in which peer review might occur, as well as advantages and disadvantages of each. The topics then shift to providing advice for prospective reviewers, as well as a suggested format for use in writing a review.

Development of a standardized peer review and oversight process in cases of suspected child physical abuse.

Peer review of medical opinions provided in cases of suspected child physical abuse is generally considered to be best practice for pediatricians engaged in this field. However, there are no published standardized guidelines on how pediatricians should undertake physical abuse peer review including case selection and process. Due to the high-stakes nature in the field of child abuse pediatrics, rigorous quality assurance practices and oversight mechanisms are essential to safeguard children, families, health care providers, and intersecting systems. The Suspected Child Abuse and Neglect program at The Hospital for Sick Children, Toronto, Canada developed a structured peer review process for cases of suspected physical abuse. Included in the process is an approach for the evaluation of institutional complaints received related to a child abuse pediatrician's medical opinion. This quality assurance process is presented so that other child abuse pediatricians and programs may replicate or adapt the protocol for their own local context.

[The peer-review process: critical issues and challenges from an online survey]. / Il processo della revisione tra pari: criticità e sfide emerse da una survey online.

BACKGROUND: the peer-review process, which is the foundation of modern scientific production, represents one of its essential elements. However, despite numerous benefits, it presents several critical issues. OBJECTIVES: to collect the opinions of a group of researchers from the epidemiological scientific community on peer-review processes. DESIGN: cross-sectional study using a questionnaire evaluation. SETTING AND PARTICIPANTS: a 29-question survey was administered to 516 healthcare professionals through the SurveyMonkey platform. The questions focused on the individual characteristics of the respondents and their perceived satisfaction with some characteristics of the review process as well as their propensity of changing some aspects of it. In addition, three open-ended questions were included, allowing respondents to provide comments on the role that reviewers and the review process should play. Descriptive statistics were produced in terms of absolute frequencies and percentages for the information collected through the questionnaire. Secondly, a multiple logistic regression analysis was conducted to assess the willingness to change certain aspects of peer review, adjusting for covariates such as age, sex, being the author of at least one scientific work, being a reviewer of at least one scientific work, and belonging to a specific discipline. The results are expressed as odds ratios (ORs) and their 95% confidence intervals (95%CI). Text analysis and representation using word cloud were also used for an open-ended question. MAIN OUTCOMES MEASURES: level of satisfaction regarding some characteristics of the peer-review process. RESULTS: a total of 516 participants completed the questionnaire. Specifically, 87.2% (N. 450) of the participants were the authors of at least one scientific publication, 78.7% were first authors at least once (N. 406), and 71.5% acted as reviewers within the peer-review process (N. 369). The results obtained from the multiple logistic regression models did not highlight any significant differences in terms of propensity to change for age and sex categories, except for a lower propensity of the under 35 age group towards unmasking, defined as the presence of reviewers and editorial boards names on the publish article (OR <35 years vs 45-54 years: 0.51; 95%CI 0.29-0.89) and a higher propensity for post-formatting proposals, defined as the possibility of formatting the article following journal guidelines after the acceptance, among those under 45 (OR <35 years vs 45-54 years: 1.73; 95%CI 0.90-3.31; OR 35-44 years vs 45-54 years: 2.02; 95%CI 1.10-3.72). Finally, approximately 50% of respondents found it appropriate to receive credits for the revision work performed, while approximately 30% found it appropriate to receive a discount on publication fees for the same journal in which they acted as reviewers. CONCLUSIONS: the peer-review process is considered essential, but imperfect, by the professionals who participated in the questionnaire, thus providing a clear picture of the value that peer-review adds rigorously to each scientific work and the need to continue constructive dialogue on this topic within the scientific community.

Exploring ChatGPT's abilities in medical article writing and peer review.

AIM: To evaluate the quality of ChatGPT-generated case reports and assess the ability of ChatGPT to peer review medical articles. METHODS: This study was conducted from February to April 2023. First, ChatGPT 3.0 was used to generate 15 case reports, which were then peer-reviewed by expert human reviewers. Second, ChatGPT 4.0 was employed to peer review 15 published short articles. RESULTS: ChatGPT was capable of generating case reports, but these reports exhibited inaccuracies, particularly when it came to referencing. The case reports received mixed ratings from peer reviewers, with 33.3% of professionals recommending rejection. The reports' overall merit score was 4.9±1.8 out of 10. The review capabilities of ChatGPT were weaker than its text generation abilities. The AI as a peer reviewer did not recognize major inconsistencies in articles that had undergone significant content changes. CONCLUSION: While ChatGPT demonstrated proficiency in generating case reports, there were limitations in terms of consistency and accuracy, especially in referencing.

Perspective on the Journal of Physician Assistant Education : From Association Newsletter to Academic Peer-Reviewed Journal Celebrating 25 Years.

ABSTRACT: For 25 years, the Journal of PA Education (JPAE) and its predecessor publications have been the pre-eminent venues for disseminating and promulgating information and research on the physician assistant (PA) profession. In this article, former and current editors in chief have compiled a detailed history of the journal, its development, and its trajectory into the future, outlining the journey taken by Association of PA Programs/PA Education Association to catalog faculty scholarship through a peer-reviewed journal. Allowing for the referencing of articles and thus adding to the body of knowledge on PAs and PA education, JPAE has not only endured but thrived. This article speaks to the collective effort and excellence of staff, and the many volunteer reviewers, feature editors, and editorial board members who have nurtured JPAE along the way through numerous changes, challenges, and triumphs.

The Purpose of Journal Peer Review Is to Make Articles Better.

Authors may have the misconception that the purpose of peer review is to serve as an arbiter or referee, or in other words, to make a binary, Accept After Revision versus Reject, decision whether an article will be published in our journal. In truth, although making that difficult decision is part of the process, it is only a part. The principal goal of peer review is to make articles better.

Development and Validation of a Scoring Rubric for Editorial Evaluation of Peer-review Quality: A Pilot Study.

Introduction: Despite the importance of peer review to publications, there is no generally accepted approach for editorial evaluation of a peer review's value to a journal editor's decision-making. The graduate medical education editors of the Western Journal of Emergency Medicine Special Issue in Educational Research & Practice (Special Issue) developed and studied the holistic editor's scoring rubric (HESR) with the objective of assessing the quality of a review and an emphasis on the degree to which it informs a holistic appreciation for the submission under consideration. Methods: Using peer-review guidelines from several journals, the Special Issue's editors formulated the rubric as descriptions of peer reviews of varying degree of quality from the ideal to the unacceptable. Once a review was assessed by each editor using the rubric, the score was submitted to a third party for blinding purposes. We compared the performance of the new rubric to a previously used semantic differential scale instrument. Kane's validity framework guided the evaluation of the new scoring rubric around three basic assumptions: improved distribution of scores; relative consistency rather than absolute inter-rater reliability across editors; and statistical evidence that editors valued peer reviews that contributed most to their decision-making. Results: Ninety peer reviews were the subject of this study, all were assessed by two editors. Compared to the highly skewed distribution of the prior rating scale, the distribution of the new scoring rubric was bell shaped and demonstrated full use of the rubric scale. Absolute agreement between editors was low to moderate, while relative consistency between editor's rubric ratings was high. Finally, we showed that recommendations of higher rated peer reviews were more likely to concur with the editor's formal decision. Conclusion: Early evidence regarding the HESR supports the use of this instrument in determining the quality of peer reviews as well as its relative importance in informing editorial decision-making.