The Peer Review Process: Past, Present, and Future.

The peer review process is a fundamental aspect of modern scientific paper publishing, underpinning essential quality control. First conceptualised in the 1700s, it is an iterative process that aims to elevate scientific literature to the highest standards whilst preventing publication of scientifically unsound, potentially misleading, and even plagiarised information. It is widely accepted that the peer review of scientific papers is an irreplaceable and fundamental aspect of the research process. However, the rapid growth of research and technology has led to a huge increase in the number of publications. This has led to increased pressure on the peer review system. There are several established peer review methodologies, ranging from single and double blind to open and transparent review, but their implementation across journals and research fields varies greatly. Some journals are testing entirely novel approaches (such as collaborative reviews), whilst others are piloting changes to established methods. Given the unprecedented growth in publication numbers, and the ensuing burden on journals, editors, and reviewers, it is imperative to improve the quality and efficiency of the peer review process. Herein we evaluate the peer review process, from its historical origins to current practice and future directions.

There is no easy fix to peer review but paying referees and regulating the number of submissions might help.

The exponential increase in the number of submissions, further accelerated by generative AI, and the decline in the availability of experts are burdening the peer review process. This has led to high unethical desk rejection rates, a growing appeal for the publication of unreviewed preprints, and a worrying proliferation of predatory journals. The idea of monetarily compensating peer reviewers has been around for many years; maybe, it is time to take it seriously as one way to save the peer review process. Here, I argue that paying reviewers, when done in a fair and transparent way, is a viable solution. Like the case of professional language editors, part-time or full-time professional reviewers, managed by universities or for-profit companies, can be an integral part of modern peer review. Being a professional reviewer could be financially attractive to retired senior researchers and to researchers who enjoy evaluating papers but are not motivated to do so for free. Moreover, not all produced research needs to go through peer review, and thus persuading researchers to limit submissions to their most novel and useful research could also help bring submission volumes to manageable levels. Overall, this paper reckons that the problem is not the peer review process per se but rather its function within an academic ecosystem dominated by an unhealthy culture of 'publish or perish'. Instead of reforming the peer review process, academia has to look for better science dissemination schemes that promote collaboration over competition, engagement over judgement, and research quality and sustainability over quantity.

Structured peer review: pilot results from 23 Elsevier journals.

Background: Reviewers rarely comment on the same aspects of a manuscript, making it difficult to properly assess manuscripts' quality and the quality of the peer review process. The goal of this pilot study was to evaluate structured peer review implementation by: 1) exploring whether and how reviewers answered structured peer review questions, 2) analysing reviewer agreement, 3) comparing that agreement to agreement before implementation of structured peer review, and 4) further enhancing the piloted set of structured peer review questions. Methods: Structured peer review consisting of nine questions was piloted in August 2022 in 220 Elsevier journals. We randomly selected 10% of these journals across all fields and IF quartiles and included manuscripts that received two review reports in the first 2 months of the pilot, leaving us with 107 manuscripts belonging to 23 journals. Eight questions had open-ended fields, while the ninth question (on language editing) had only a yes/no option. The reviews could also leave Comments-to-Author and Comments-to-Editor. Answers were independently analysed by two raters, using qualitative methods. Results: Almost all the reviewers (n = 196, 92%) provided answers to all questions even though these questions were not mandatory in the system. The longest answer (Md 27 words, IQR 11 to 68) was for reporting methods with sufficient details for replicability or reproducibility. The reviewers had the highest (partial) agreement (of 72%) for assessing the flow and structure of the manuscript, and the lowest (of 53%) for assessing whether interpretation of the results was supported by data, and for assessing whether the statistical analyses were appropriate and reported in sufficient detail (52%). Two thirds of the reviewers (n = 145, 68%) filled out the Comments-to-Author section, of which 105 (49%) resembled traditional peer review reports. These reports contained a Md of 4 (IQR 3 to 5) topics covered by the structured questions. Absolute agreement regarding final recommendations (exact match of recommendation choice) was 41%, which was higher than what those journals had in the period from 2019 to 2021 (31% agreement, P = 0.0275). Conclusions: Our preliminary results indicate that reviewers successfully adapted to the new review format, and that they covered more topics than in their traditional reports. Individual question analysis indicated the greatest disagreement regarding the interpretation of the results and the conducting and the reporting of statistical analyses. While structured peer review did lead to improvement in reviewer final recommendation agreements, this was not a randomized trial, and further studies should be performed to corroborate this. Further research is also needed to determine whether structured peer review leads to greater knowledge transfer or better improvement of manuscripts.

Where have all the reviewers gone?

In this editorial the editor considers the growing challenges journals are facing in securing peer reviewers, some of the approaches being tried to address this problem, and the prospects for sustaining communities of scholars with and without an ongoing commitment to peer review.

Unlocking Peer Review: Elevating Scholarly Writing and Research Competence in Urology Residency.

PURPOSE OF REVIEW: It is incumbent upon training programs to set the foundation for evidence-based practices and to create opportunities for trainees to develop into academic leaders. As dedicated resident research time and funding have declined in recent years, residency programs and the field at large will need to create new ways to incorporate scholarly activity into residency curricula. RECENT FINDINGS: Literature across specialties demonstrates barriers to resident involvement including lack of time, cost, and absent scholarly mentorship. Peer review stands as a ready-made solution that can be formalized into a collaborative relationship with journals. A formal relationship between professional societies, academic journals, and residencies can facilitate the use of peer review as a teaching tool for residency programs.

So .You Want to be a Peer Reviewer?

Peer review is essential to preserving the integrity of the scientific publication process. Peer reviewers must adhere to the norms of the peer review process, including confidentiality, avoiding actual or apparent conflicts of interest, timeliness, constructiveness, and thoroughness. This mini review will discuss some of the different formats in which peer review might occur, as well as advantages and disadvantages of each. The topics then shift to providing advice for prospective reviewers, as well as a suggested format for use in writing a review.

Exploring ChatGPT's abilities in medical article writing and peer review.

AIM: To evaluate the quality of ChatGPT-generated case reports and assess the ability of ChatGPT to peer review medical articles. METHODS: This study was conducted from February to April 2023. First, ChatGPT 3.0 was used to generate 15 case reports, which were then peer-reviewed by expert human reviewers. Second, ChatGPT 4.0 was employed to peer review 15 published short articles. RESULTS: ChatGPT was capable of generating case reports, but these reports exhibited inaccuracies, particularly when it came to referencing. The case reports received mixed ratings from peer reviewers, with 33.3% of professionals recommending rejection. The reports' overall merit score was 4.9±1.8 out of 10. The review capabilities of ChatGPT were weaker than its text generation abilities. The AI as a peer reviewer did not recognize major inconsistencies in articles that had undergone significant content changes. CONCLUSION: While ChatGPT demonstrated proficiency in generating case reports, there were limitations in terms of consistency and accuracy, especially in referencing.

[The peer-review process: critical issues and challenges from an online survey]. / Il processo della revisione tra pari: criticità e sfide emerse da una survey online.

BACKGROUND: the peer-review process, which is the foundation of modern scientific production, represents one of its essential elements. However, despite numerous benefits, it presents several critical issues. OBJECTIVES: to collect the opinions of a group of researchers from the epidemiological scientific community on peer-review processes. DESIGN: cross-sectional study using a questionnaire evaluation. SETTING AND PARTICIPANTS: a 29-question survey was administered to 516 healthcare professionals through the SurveyMonkey platform. The questions focused on the individual characteristics of the respondents and their perceived satisfaction with some characteristics of the review process as well as their propensity of changing some aspects of it. In addition, three open-ended questions were included, allowing respondents to provide comments on the role that reviewers and the review process should play. Descriptive statistics were produced in terms of absolute frequencies and percentages for the information collected through the questionnaire. Secondly, a multiple logistic regression analysis was conducted to assess the willingness to change certain aspects of peer review, adjusting for covariates such as age, sex, being the author of at least one scientific work, being a reviewer of at least one scientific work, and belonging to a specific discipline. The results are expressed as odds ratios (ORs) and their 95% confidence intervals (95%CI). Text analysis and representation using word cloud were also used for an open-ended question. MAIN OUTCOMES MEASURES: level of satisfaction regarding some characteristics of the peer-review process. RESULTS: a total of 516 participants completed the questionnaire. Specifically, 87.2% (N. 450) of the participants were the authors of at least one scientific publication, 78.7% were first authors at least once (N. 406), and 71.5% acted as reviewers within the peer-review process (N. 369). The results obtained from the multiple logistic regression models did not highlight any significant differences in terms of propensity to change for age and sex categories, except for a lower propensity of the under 35 age group towards unmasking, defined as the presence of reviewers and editorial boards names on the publish article (OR <35 years vs 45-54 years: 0.51; 95%CI 0.29-0.89) and a higher propensity for post-formatting proposals, defined as the possibility of formatting the article following journal guidelines after the acceptance, among those under 45 (OR <35 years vs 45-54 years: 1.73; 95%CI 0.90-3.31; OR 35-44 years vs 45-54 years: 2.02; 95%CI 1.10-3.72). Finally, approximately 50% of respondents found it appropriate to receive credits for the revision work performed, while approximately 30% found it appropriate to receive a discount on publication fees for the same journal in which they acted as reviewers. CONCLUSIONS: the peer-review process is considered essential, but imperfect, by the professionals who participated in the questionnaire, thus providing a clear picture of the value that peer-review adds rigorously to each scientific work and the need to continue constructive dialogue on this topic within the scientific community.