Cultural distance, gender and praise in peer review.

Background: Understanding review comments holds significant importance within the realm of scientific discourse. This study aims to conduct an empirical analysis of factors associated with praise in peer review.Methods: The study involved manual labeling of "praise" in 952 review comments drawn from 301 articles published in the British Medical Journal, followed by regression analysis.Results: The study reveals that authors tend to receive longer praise when they share a cultural proximity with the reviewers. Additionally, it is observed that female reviewers are more inclined to provide praiseConclusions: In summary, these discoveries contribute valuable insights for the development of a constructive peer review process and the establishment of a more inclusive research culture.

Experience of a peer prereview service including a junior reviewer program in a professional association.

DISCLAIMER: In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time. PURPOSE: Scientific journals have reportedly low acceptance rates. Peer review support services offering manuscript review before journal submission are uncommon but may increase success with the publication process. The purpose of this report is to describe the development and experience with a peer prereview (PPR) service offered by the Clinical Pharmacy and Pharmacology (CPP) section of the Society of Critical Care Medicine. SUMMARY: The CPP PPR service provides comprehensive review on research manuscripts before submission to a scientific journal. A junior reviewer program provides an opportunity for mentorship and professional development of reviewers. After the review is complete, authors and reviewers complete a survey regarding their experience with the service. Authors are also contacted biannually to determine their manuscript's status. From the program's start in 2011 through June 2022, 64% of the 58 manuscripts reviewed were accepted or published in journals with impact factors ranging from 0.915 to 9.296. Of the 31 (82%) authors responding to the survey, 94% said that the service met or exceeded their expectations and 84% would recommend the service to a colleague. Of the 29 (76%) reviewers responding to the survey, 100% said that the service met or exceeded their expectations and 92% would be willing to review again. CONCLUSION: The majority of manuscripts that went through the CPP PPR service were accepted for publication, and both authors and reviewers were highly satisfied with their experience. The CPP PPR service can be used as a framework for other institutions to implement.

Persistent ethnic disparities in authorship within top European and North American medical journals: a serial cross-sectional analysis.

OBJECTIVES: We examined the ethnic origin of authors who published research articles in leading medical journals over the past two decades. STUDY DESIGN AND SETTING: We carried out a serial cross-sectional analysis of first and last authors who published original research articles in the British Medical Journal, Lancet, Journal of American Medical Association, and New England Journal of Medicine in 2002, 2012, and 2022. The main outcome was the change in proportion of authors over time according to ethnic origin (Anglo, North/West European, South/West European, Asian, Arab and Middle Eastern, African), gender (male, female), and institutional affiliation in percentage points. RESULTS: Most authors were of Anglo descent (44%), although the proportion of non-European authors grew between 2002 and 2022. East Asian, South Asian, and Arab & Middle Eastern last authors accounted for a greater proportion of authors over time, gaining between 3 to 6 percentage points, while African authors made no gains. Gains were gender-specific, with non-European men gaining 8 points as first and last authors, but non-European women gaining 5 points as last authors only. Most non-European authors were affiliated with North American (42.9%) or European (22.4%) institutions, while non-European authors from other institutions did not make meaningful gains over time. CONCLUSION: Ethnic diversity of authors in leading medical journals has increased somewhat over time, but non-European men account for most of the gains. Non-European women have yet to make comparable advancement as authors.

Peer review of the pesticide risk assessment of the active substance pyrimethanil.

The conclusions of the European Food Safety Authority (EFSA) following the peer review of the initial risk assessments carried out by the competent authorities of the rapporteur Member State Czech Republic and co-rapporteur Member State Austria for the pesticide active substance pyrimethanil and of confirmatory data following the maximum residue limit (MRL) review under Article 12 of Regulation (EC) No 396/2005 are reported. The context of the peer review was that required by Commission Implementing Regulation (EU) No 844/2012, as amended by Commission Implementing Regulation (EU) No 2018/1659. The conclusions were reached on the basis of the evaluation of the representative uses of pyrimethanil as a fungicide on grapevine and pome fruit (field uses), strawberry and lettuce (field and greenhouse uses). The reliable end points, appropriate for use in regulatory risk assessment are presented. Missing information identified as being required by the regulatory framework is listed. Concerns are identified.

Peer review protection: Pish-Posh or pivotal policy?

The Healthcare Quality Improvement Act (HCQIA) of 1986 is a pivotal federal mandate designed to enhance medical care quality through effective professional peer review. Importantly, it offers legal immunity to reviewers under specified conditions and mandates the reporting of adverse actions to the National Practitioner Data Bank (NPDB). This article explores the implementation of peer review processes in hospitals and the potentially severe ramifications of failure to report, using the scenario of a diagnostic radiologist performing high-end vascular interventional procedures, whose performance came under scrutiny, highlighting the intersection of federal and state laws, accreditation standards, hospital policies, and physician professionalism standards and reporting duties.

An evaluation of the preprints produced at the beginning of the 2022 mpox public health emergency.

BACKGROUND: Preprints are scientific articles that have not undergone the peer-review process. They allow the latest evidence to be rapidly shared, however it is unclear whether they can be confidently used for decision-making during a public health emergency. This study aimed to compare the data and quality of preprints released during the first four months of the 2022 mpox outbreak to their published versions. METHODS: Eligible preprints (n = 76) posted between May to August 2022 were identified through an established mpox literature database and followed to July 2024 for changes in publication status. Quality of preprints and published studies was assessed by two independent reviewers to evaluate changes in quality, using validated tools that were available for the study design (n = 33). Tools included the Newcastle-Ottawa Scale; Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2); and JBI Critical Appraisal Checklists. The questions in each tool led to an overall quality assessment of high quality (no concerns with study design, conduct, and/or analysis), moderate quality (minor concerns) or low quality (several concerns). Changes in data (e.g. methods, outcomes, results) for preprint-published pairs (n = 60) were assessed by one reviewer and verified by a second. RESULTS: Preprints and published versions that could be evaluated for quality (n = 25 pairs) were mostly assessed as low quality. Minimal to no change in quality from preprint to published was identified: all observational studies (10/10), most case series (6/7) and all surveillance data analyses (3/3) had no change in overall quality, while some diagnostic test accuracy studies (3/5) improved or worsened their quality assessment scores. Among all pairs (n = 60), outcomes were often added in the published version (58%) and less commonly removed (18%). Numerical results changed from preprint to published in 53% of studies, however most of these studies (22/32) had changes that were minor and did not impact main conclusions of the study. CONCLUSIONS: This study suggests the minimal changes in quality, results and main conclusions from preprint to published versions supports the use of preprints, and the use of the same critical evaluation tools on preprints as applied to published studies, in decision-making during a public health emergency.

Effectiveness of a report writing training program using peer review: evidence from first- year medical students.

BACKGROUND: Report writing in class provides basic training for academic writing. However, report writing education in medical schools in Japan has rarely been reported and no teaching strategy has been established for it. METHODS: This study developed a report writing program using peer review for first-year medical students consisting of two 120-minute classes. The goal of being able to write reports appropriately was established and presented to students at the beginning of the program. In session 1, students decided on a topic, gathered information, and structured their report. In session 2, students' written reports were peer reviewed. The reports were improved based on the peer reviews. The responses of the pre- and post-program questionnaires were evaluated to determine the program's effectiveness. The other reports that were assigned one month after the program were used as comparison with the reports of students who did not participate. Furthermore, the long-term effects of the program were also evaluated by comparing the results of students from the previous year who did not participate in the program with those of program participants six months after the program. RESULTS: A total of 106 students completed the questionnaire evaluation. The program was rated as being acceptable for the students. Self-assessment of report writing significantly improved after the program. The report scores of program participants (n = 99) were significantly higher overall and in all domains than were those of non-participants (n = 99). The self-assessment of students who participated (n = 96) was significantly higher than that of non-participants (n = 109). No difference was observed for students' sense of burden in report writing. CONCLUSIONS: The report writing skills of medical students can be improved by clearly establishing the goals of report writing and practicing the basic skills of report writing step-by-step. Moreover, the use of peer review may enhance the effectiveness of learning opportunities for report writing.

Supporting open science: Advances in Clinical and Experimental Medicine and preprints.

This editorial outlines the issue of preprints in scholarly communication. It presents the policy regarding them in Advances in Clinical and Medical Problems and a summary of papers released as preprints and subsequently published in this journal or rejected until July 10, 2024. The introduction discusses the definition of preprint, and leading preprint servers are listed. Policies of 2 such services - Research Square and medRxiv - most frequently chosen by Adv Clin Exp Med authors are then described, followed by a broad outline of the advantages of preprints and controversies surrounding them, based on selected literature on this topic. The next section discusses the policies of most renowned medical journals and publishers regarding preprints. The preprint policy of Adv Clin Exp Med is then thoroughly explained, as well as its reasons. All papers previously released as preprints and published in this journal in 2021-2024 are presented, focusing on meaningful differences between them. Rejected papers previously released as preprints, submitted to Adv Clin Exp Med in 2022-2024, are also listed and discussed. The conclusion is that the basis for endorsing preprints in this journal is not that they benefit this journal but that they serve the scientific community as a whole and science in general by facilitating rapid dissemination of results and fostering immediate assessment of those results by other investigators and debate around them. The most justified line of action is educating authors about the benefits and problems related to preprints.

"If You're Talking, I Think You're Muted": Follow-up Analysis of Weekly Peer Review Discussion and Plan Changes After Transitioning From Virtual to In-Person Format.

AIMS: During the COVID-19 public health emergency, we previously identified decreased rates of radiotherapy (RT) peer review (PR) discussion and plan changes in virtual versus in-person PR conferences. To expand on these findings, we continued to prospectively collect data on all PR conferences from 2021 to 2023 and performed a follow-up analysis before and after the transition back to in-person PR. MATERIALS AND METHODS: A prospectively maintained database of weekly PR cases was queried for consecutive cases reviewed before and after the transition from virtual to in-person conferences. Rates of PR discussion and change recommendations were summarized and compared between the virtual and in-person groups. A survey was developed and administered to assess participants' perceived levels of engagement, opinions on optimal PR format, and preferences for future meetings before and 3 months after the transition back to in-person PR. RESULTS: In total, 2,103 RT plans were reviewed: 1,590 virtually and 513 after the transition back to in-person. There was no difference in faculty attendance between groups. The proportion of cases with PR discussion increased from virtual (9.8%) to in-person (25.5%) format (p < 0.001). In the virtual group, 8.1% of cases had 1 topic and 1.7% had 2+ topics discussed. This increased to 15.8% and 9.7% during in-person PR, respectively (p < 0.001). The rate of change recommendation also increased from 1.5% (virtual) to 3.3% (in-person, p = 0.016). Among cases with at least 1 topic discussed, there was no difference in changes. Survey-reported distraction significantly decreased from virtual to in-person PR (p < 0.001). CONCLUSION: Upon returning to in-person PR conferences, peer discussion and plan change recommendations significantly increased and returned to pre-pandemic levels, and participants' perceived levels of distraction were reduced. In an increasingly virtual world, additional efforts to develop best practices that maximize PR discussion and minimize distraction outside virtual conferences are warranted.

Peer review of the pesticide risk assessment of the active substance amidosulfuron.

The conclusions of the European Food Safety Authority (EFSA) following the peer review of the initial risk assessments carried out by the competent authorities of the rapporteur Member State, Finland, and co-rapporteur Member State, Croatia, for the pesticide active substance amidosulfuron and the assessment of confirmatory data following the Article 12 MRL review are reported. The context of the peer review was that required by Commission Implementing Regulation (EU) No 844/2012, as amended by Commission Implementing Regulation (EU) No 2018/1659. The conclusions were reached on the basis of the evaluation of the representative uses of amidosulfuron as a post-emergence herbicide on winter cereals, spring cereals, flax and grass/pasture (all field uses). The reliable end points, appropriate for use in regulatory risk assessment are presented. Missing information identified as being required by the regulatory framework is listed. Concerns are reported where identified.

Peer review of the pesticide risk assessment of the active substance Bacillus subtilis strain RTI477.

The conclusions of the European Food Safety Authority (EFSA) following the peer review of the initial risk assessments carried out by the competent authority of the rapporteur Member State the Netherlands for the pesticide active substance Bacillus subtilis strain RTI477 are reported. The context of the peer review was that required by Regulation (EC) No 1107/2009 of the European Parliament and of the Council. The conclusions were reached on the basis of the evaluation of the representative uses of B. subtilis strain RTI477 as a fungicide for the control of soil-borne diseases in greenhouses (permanent) and walk-in tunnels and field crops; by drip and drench irrigation on lettuce, cucurbit and solanaceous vegetables; field application by treating tubers whilst laying in furrow on potato; seed treatment application on maize, sunflower, sugar beet and winter oilseed rape. The reliable endpoints, appropriate for use in regulatory risk assessment, are presented. Missing information identified as being required by the regulatory framework is listed. Concerns are reported when identified.

Peer review of the pesticide risk assessment of the active substance Bacillus velezensis RTI301.

The conclusions of the European Food Safety Authority (EFSA) following the peer review of the initial risk assessments carried out by the competent authority of the rapporteur Member State The Netherlands for the pesticide active substance Bacillus velezensis strain RTI301 are reported. The context of the peer review was that required by Regulation (EC) No 1107/2009 of the European Parliament and of the Council. The conclusions were reached on the basis of the evaluation of the representative uses of B. velezensis strain RTI301 as a fungicide for the control of soil-borne diseases in greenhouses (permanent) and walk-in tunnels (non-permanent) and field crops; by drip and drench irrigation on lettuce, cucurbit and solanaceous vegetables; field application by treating tubers whilst laying in furrow on potato; seed treatment application on maize, sunflower, sugar beet and winter oilseed rape. The reliable endpoints, appropriate for use in regulatory risk assessment, are presented. Missing information identified as being required by the regulatory framework is listed. Concerns are reported when identified.

Examining uncertainty in journal peer reviewers' recommendations: a cross-sectional study.

The peer review process is used throughout science but has often been criticized for being inconsistent, with decisions dependent on the peers who did the reviewing. Much of the decision inconsistency arises from the differences between reviewers in terms of their expertise, training and experience. Another source of uncertainty is within reviewers as they must make a single recommendation (e.g. 'Accept'), when they may have wavered between two (e.g. 'Accept' or 'Reject'). We estimated the size of within-reviewer uncertainty using post-review surveys at three journals. We asked reviewers to think outside the recommendation they gave (e.g. 'Accept') and assign percentages to all other recommendations (e.g. 'Major revision'). Reviewers who were certain could assign 100% to one recommendation. Twenty-three per cent of reviewers reported no uncertainty (95% confidence interval 19-27%). Women were associated with more uncertainty at one journal, and protocol papers were associated with more uncertainty at one journal. Reviewers commonly experience some uncertainty when peer-reviewing journal articles. This uncertainty is part of the variability in peer reviewers' recommendation.

Peer review of the pesticide risk assessment of the active substance bensulfuron-methyl.

The conclusions of the European Food Safety Authority (EFSA) following the peer review of the initial risk assessments carried out by the competent authorities of the rapporteur Member State Italy and co-rapporteur Member State Spain for the pesticide active substance bensulfuron-methyl are reported. The context of the peer review was that required by Commission Implementing Regulation (EU) No 844/2012, as amended by Commission Implementing Regulation (EU) No 2018/1659. The conclusions were reached on the basis of the evaluation of the representative uses of bensulfuron-methyl as an herbicide on rice and spring cereals (spring wheat, spring barley, oat, rye, triticale). The reliable end points, appropriate for use in regulatory risk assessment are presented. Missing information identified as being required by the regulatory framework is listed. Concerns are identified.

Reviewer Experience Detecting and Judging Human Versus Artificial Intelligence Content: The Stroke Journal Essay Contest.

Artificial intelligence (AI) large language models (LLMs) now produce human-like general text and images. LLMs' ability to generate persuasive scientific essays that undergo evaluation under traditional peer review has not been systematically studied. To measure perceptions of quality and the nature of authorship, we conducted a competitive essay contest in 2024 with both human and AI participants. Human authors and 4 distinct LLMs generated essays on controversial topics in stroke care and outcomes research. A panel of Stroke Editorial Board members (mostly vascular neurologists), blinded to author identity and with varying levels of AI expertise, rated the essays for quality, persuasiveness, best in topic, and author type. Among 34 submissions (22 human and 12 LLM) scored by 38 reviewers, human and AI essays received mostly similar ratings, though AI essays were rated higher for composition quality. Author type was accurately identified only 50% of the time, with prior LLM experience associated with improved accuracy. In multivariable analyses adjusted for author attributes and essay quality, only persuasiveness was independently associated with odds of a reviewer assigning AI as author type (adjusted odds ratio, 1.53 [95% CI, 1.09-2.16]; P=0.01). In conclusion, a group of experienced editorial board members struggled to distinguish human versus AI authorship, with a bias against best in topic for essays judged to be AI generated. Scientific journals may benefit from educating reviewers on the types and uses of AI in scientific writing and developing thoughtful policies on the appropriate use of AI in authoring manuscripts.