Universal Principled Review: A Community-Driven Method to Improve Peer Review.

Despite being a staple of our science, the process of pre-publication peer review has few agreed-upon standards defining its goals or ideal execution. As a community of reviewers and authors, we assembled an evaluation format and associated specific standards for the process as we think it should be practiced. We propose that we apply, debate, and ultimately extend these to improve the transparency of our criticism and the speed with which quality data and ideas become public.

Navigating the journey as a peer reviewer: A roadmap to success.

Peer review is an essential step in scientific progress and clinical improvement, providing opportunity for research to be critically evaluated and improved by one's colleagues. Pharmacists from all job settings are called to serve as peer reviewers in the ever-growing publication landscape of the profession. Despite challenges to engagement such as time and compensation, peer review provides considerable professional development for both authors and reviewers alike. This article will serve as a practical guide for peer reviewers, discussing best practices as well as the handling of different situations that may arise during the process.

Strategies for Developing Journal Peer Reviewers: A Scoping Review.

AIM: This scoping review examined development strategies for preparing reviewers to critically appraise the content of manuscripts submitted to peer-reviewed journals. BACKGROUND: The journal peer review process is the crux of building the science of nursing education to inform teaching and learning. METHOD: Using the Joanna Briggs Institute procedure for scoping reviews, five databases were searched for articles published in English in peer-reviewed health sciences journals between 2012 and 2022 that included strategies for developing journal peer reviewers. RESULTS: Of the 44 articles included in the review, a majority were commentaries (52%) published by medicine (61%), followed by nursing (9%) and multidisciplinary journals (9%). Reviewer development strategies aligned with three themes: pedagogical approaches, resources, and personal practices. CONCLUSION: Although multiple disciplines addressed peer reviewer development, a comprehensive and effective approach was not reported in the reviewed literature. The findings can inform a multilevel reviewer development program led by academic nurse educators.

An overview of the peer review process in biomedical sciences.

OBJECTIVE: This paper aims to provide an introductory resource for beginner peer reviewers in psychiatry and the broader biomedical science field. It will provide a concise overview of the peer review process, alongside some reviewing tips and tricks. CONCLUSION: The peer review process is a fundamental aspect of biomedical science publishing. The model of peer review offered varies between journals and usually relies on a pool of volunteers with differing levels of expertise and scope. The aim of peer review is to collaboratively leverage reviewers' collective knowledge with the objective of increasing the quality and merit of published works. The limitations, methodology and need for transparency in the peer review process are often poorly understood. Although imperfect, the peer review process provides some degree of scientific rigour by emphasising the need for an ethical, comprehensive and systematic approach to reviewing articles. Contributions from junior reviewers can add significant value to manuscripts.

Estimating the deep replicability of scientific findings using human and artificial intelligence.

Replicability tests of scientific papers show that the majority of papers fail replication. Moreover, failed papers circulate through the literature as quickly as replicating papers. This dynamic weakens the literature, raises research costs, and demonstrates the need for new approaches for estimating a study's replicability. Here, we trained an artificial intelligence model to estimate a paper's replicability using ground truth data on studies that had passed or failed manual replication tests, and then tested the model's generalizability on an extensive set of out-of-sample studies. The model predicts replicability better than the base rate of reviewers and comparably as well as prediction markets, the best present-day method for predicting replicability. In out-of-sample tests on manually replicated papers from diverse disciplines and methods, the model had strong accuracy levels of 0.65 to 0.78. Exploring the reasons behind the model's predictions, we found no evidence for bias based on topics, journals, disciplines, base rates of failure, persuasion words, or novelty words like "remarkable" or "unexpected." We did find that the model's accuracy is higher when trained on a paper's text rather than its reported statistics and that n-grams, higher order word combinations that humans have difficulty processing, correlate with replication. We discuss how combining human and machine intelligence can raise confidence in research, provide research self-assessment techniques, and create methods that are scalable and efficient enough to review the ever-growing numbers of publications-a task that entails extensive human resources to accomplish with prediction markets and manual replication alone.

Gender Affirming Surgery: A Comprehensive, Systematic Review of All Peer-reviewed Literature and Methods of Assessing Patient-centered Outcomes (Part 1: Breast/Chest, Face, and Voice).

OBJECTIVE: To perform the first systematic review of all available gender-affirming surgery (GAS) publications across all procedures to assess both outcomes reported in the literature and the methods used for outcome assessment. SUMMARY OF BACKGROUND DATA: Rapidly increasing clinical volumes of gender-affirming surgeries have stimulated a growing need for high-quality clinical research. Although some procedures have been performed for decades, each individual procedure has limited data, necessitating synthesis of the entire literature to understand current knowledge and guide future research. METHODS: A systematic review was performed following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify all outcomes measures in GAS cohorts, including PCOs, complications, and functional outcomes. Outcome data were pooled to assess currently reported complication, satisfaction, and other outcome rates. RESULTS: Overall, 15,186 references were identified, 4162 papers advanced to abstract review, and 1826 underwent full-text review. After review, there were 406 GAS cohort publications. Of non-genitoplasty titles, 35 were mastectomy, 6 mammoplasty, 21 facial feminization, and 31 voice/cartilage. Although 59.1% of non-genitoplasty papers addressed PCOs in some form, only 4.3% used instruments partially-validated in transgender patients. Overall, data were reported heterogeneously and were biased towards high-volume centers. CONCLUSIONS: This study represents the most comprehensive review of GAS literature. By aggregating all previously utilized measurement instruments, this study offers a foundation for discussions about current methodologic limitations and what dimensions must be included in assessing surgical success. We have assembled a comprehensive list of outcome instruments; this offers an ideal starting basis for emerging discussions between patients and providers about deficiencies which new, better instruments and metrics must address. The lack of consistent use of the same outcome measures and validated GAS-specific instruments represent the 2 primary barriers to high-quality research where improvement efforts should be focused.

Optimizing peer review to minimize the risk of retracting COVID-19-related literature.

Retractions of COVID-19 literature in both preprints and the peer-reviewed literature serve as a reminder that there are still challenging issues underlying the integrity of the biomedical literature. The risks to academia become larger when such retractions take place in high-ranking biomedical journals. In some cases, retractions result from unreliable or nonexistent data, an issue that could easily be avoided by having open data policies, but there have also been retractions due to oversight in peer review and editorial verification. As COVID-19 continues to affect academics and societies around the world, failures in peer review might also constitute a public health risk. The effectiveness by which COVID-19 literature is corrected, including through retractions, depends on the stringency of measures in place to detect errors and to correct erroneous literature. It also relies on the stringent implementation of open data policies.

Integration of Peer Review in PACS Results in a Marked Increase in the Discrepancies Reported.

OBJECTIVE. The objective of this article is to assess the impact of integrating peer review in PACS on the reporting of discrepancies. Our hypothesis is that a PACS-integrated machine-randomized and semiblinded peer review tool leads to an increase in discrepancies reported. MATERIALS AND METHODS. A PACS tool was implemented to prompt radiologists to perform peer review of prior comparison studies in a randomized fashion. The reviewed radiologist's name was omitted from the prior report in PACS. Before this implementation, radiologists entered peer reviews directly on the RADPEER website. Three academic subspecialty sections comprising 24 radiologists adopted the tool (adopters group). Three sections comprising 14 radiologists did not adopt the tool (nonadopters group). Peer review submissions were analyzed for 4 months before and 4 months after the implementation. The mean rate of significant discrepancies (RADPEER score 2b or higher) reported per radiologist was calculated and the discrepancy rates of the periods before and after the implementation were compared. RESULTS. The mean significant discrepancy rate reported per radiologist in the adopters group increased from 0.19% ± 0.46% (SD) before the implementation to 0.93% ± 1.45% after implementation (p = 0.01). No significant discrepancies were reported by the nonadopters group in either period. CONCLUSION. In this single institutional retrospective analysis, integrating peer review in PACS resulted in a fivefold increase in reported significant discrepancies. These results suggest that peer review data are influenced by the design of the tool used including PACS integration, randomization, and blinding.

Opinion on Performing Pathology Peer-Review During the Global Pandemic: Challenges and Opportunities.

The coronavirus disease 2019 pandemic has affected business on numerous fronts in unprecedented and abrupt ways. From site closures and local "stay-at-home orders" to travel advisories and restrictions, the day-to-day practice of toxicologic pathology has been impacted dramatically and rapidly. A critical function of Toxicologic Pathologists is performing pathology peer review for nonclinical studies. Traditionally, corroborating the findings of histological assessment could be achieved through shipment of histopathological slides to the peer review pathologist, or by the peer review pathologist traveling to the location of the slides (eg, the test facility). Since early 2020, many pathologists have been unable to perform the latter due to local, regional, national, test facility, company, and/or personal restrictions. The disruption for some has been minimal, while others are working from home for the first time. We recommend that contingency plans for all peer review procedures and personnel should be in-place to accommodate sudden and unexpected workflow transitions. Now, more than ever, approaching peer reviews with enhanced adaptability will help ensure success.

Evaluation of intervention reporting in published emergency medicine clinical trials.

OBJECTIVE: The extent of intervention reporting in emergency medicine journals remains unclear. The primary objective is to assess overall completion of the Template for Intervention Description and Replication (TIDieR) checklist described in emergency medicine randomized clinical trials (RCTs). The secondary outcomes were to (1) compare reporting before and after TIDieR publication; (2) evaluate factors associated with intervention reporting. METHODS: Our cross-sectional study used Google Scholar's metrics to identify seven emergency medicine journals; of which, we randomly sampled 300 articles. Using two PubMed searches, we extracted 150 RCTs before and after publications of TIDieR. Two investigators independently extracted data. The primary analysis to measure overall completion included descriptive statistics for each checklist item. Our secondary analysis used an interrupted time series analysis and generalized estimating equations to determine the effect of TIDieR publication on intervention reporting. RESULTS: Our initial search yielded 635 articles; from which, we randomly sampled 300 articles. We excluded 67 articles, leaving 233 for analysis. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). Of the 233 trials, 42.9% provided information about materials, 67% provided intervention procedures, and 99.1% provided intervention delivery. The least reported items were intervention modifications (2.6%), intervention adherence assessment methods (3.4%), and intervention adherence assessment outcomes (2.2%). CONCLUSIONS: The completeness of intervention reporting is suboptimal in emergency medicine journals, necessitating improvement. The current state of adherence could be improved through the combined efforts of journal editors, major editorial organizations, and authors.

Bioethics as a Vocation.

This essay offers practical guidance for the activity of bioethics scholarship in the form of maxims or rules of thumb, explicated by the author's work in bioethics research, mentoring, peer review, and journal editing. It is organized into three sections: (1) education, (2) seeking ideas for projects, and (3) writing papers.

Discovering the Benefits of Group Peer Review of Submitted Manuscripts.

Problem: Traditionally, journal editors expect individuals to complete peer reviews of submitted manuscripts on their own. Recently, a number of editors of health sciences journals have begun to support, and even espouse, the practice of group peer review (GPR). With GPR, multiple individuals work together to complete the review with permission from the journal editor. Motivated by the idea that GPR could provide a meaningful service learning experience for participants in an interprofessional educational scholarship course, we conducted three such reviews and subsequently reflected on our experience and the lessons we learned. We frame our reflections using guiding principles from the domains of peer review, professional development, and educational scholarship. Intervention: The course director arranged for manuscripts to review with the editors of three health sciences journals. Each GPR occurred during a separate weekly session of the course. Each GPR was completed using a similar set of steps, which included (a) gaining familiarity with review criteria, (b) reading aloud and discussing the manuscript's abstract as a class, (c) reading and critiquing assigned sections as individuals and then small groups, (d) building consensus and sharing notes, (e) having the course director synthesize notes into a single review for submission to the journal. Context: The course on educational scholarship involved 15 faculty representing faculty from the University of Utah's School of Medicine, College of Nursing, College of Pharmacy, College of Health, and School of Dentistry. The course director led three GPR sessions mid-way through the yearlong course. Impact: Participants' reflections indicate that GPR (a) conformed to principles of effective peer review; (b) resulted in a meaningful service learning experience within a formal professional development program, deepening understanding of core concepts of educational scholarship; and (c) represented an authentic example of engaging in educational scholarship (i.e., designing and evaluating an intervention while drawing upon and contributing to a body of shared understanding within a community of practice). Lessons Learned: Our principles-based approach to completing GPR within a professional development course on educational scholarship can serve as a model for others to follow. A rigorous, meaningful group review can occur in 1 hour using a combination of group and individual activities focused on matching review criteria to the submitted manuscript. As a result, we continue to include GPR in future offerings of this interprofessional course on educational scholarship, and we continue to study ways to optimize its value as a service learning experience.

Peers as OSCE assessors for junior medical students - a review of routine use: a mixed methods study.

BACKGROUND: Peer-assisted learning is well established in medical education; however, peer tutors rarely act as assessors for the OSCE. In the compulsory, near-peer teaching programme covering basic medical skills at the University of Heidelberg, peer tutors serve as assessors on a formative OSCE. This study aimed to investigate the feasibility and acceptance of peer assessors and to survey the perceived advantages and disadvantages of their use. METHODS: In 2016 and 2017 all OSCE peer assessors (third to sixth-year medical students) and all of the peer-assessed students in 2017 (second-year-medical students) were invited to participate in a survey. Both groups were asked to complete a tablet-based questionnaire immediately after the OSCE. Peer assessors were asked to rate eight statements and the peer-assessed students to rate seven statements on a five-point Likert scale. Both were asked to comment on the advantages and disadvantages of peer-assessors. RESULTS: Overall, 74 of 76 peer assessors and 307 of 308 peer-assessed students participated in the study. 94% (67/74) of peer assessors and 90% (276/307) of the peer-assessed group thought that it is important to have peer tutors as assessors. Of the peer assessors, 92% (68/74) felt confident in giving structured feedback during the OSCE and 66% (49/74) felt they had improved their teaching skills. Of the peer-assessed students, 99% (306/307) were satisfied with their peers as OSCE assessors and 96% (292/307) considered the peer feedback during the OSCE as helpful. The participants mentioned structural benefits, such as lower costs, and suggested the quality of the OSCE was higher due to the use of peer assessors. The use of peer assessors was found to be beneficial for the learners in the form of high-quality feedback and an overall reduction in stress. Furthermore, the use of peer assessors was found to be beneficial for the peer assessors (improved teaching and clinical skills). CONCLUSION: From a learner's perspective, the use of peer assessors for a formative OSCE that is part of a near-peer teaching program aimed at junior medical students is favourable for all. A broad implementation of peer assessment in the formative OSCE should be encouraged to investigate effects on quality and stress-reduction.

Peer Review Tool for General Radiography Technologists Improves Image Quality.

PURPOSE: Quality improvement is vital to ensure health-care providers meet optimal patient care standards. Within our jurisdiction, accreditation requires image peer review as part of the quality assurance program. We propose a method to improve quality assurance in radiography by implementing a novel software-based peer review system for radiography technologists. METHODS: This is a retrospective study. A peer review tool was developed in Microsoft Excel and Visual Basic. The tool has 14 image quality criteria, which were selected based on national and international criteria, each containing standardized answers ensuring a common scoring regime. The tool provides data analysis and storage of all peer reviews performed. Radiography supervisors utilized the tool to evaluate image quality of various body parts at 28 hospitals. The tool enabled each Medical Imaging Department to objectively score images at their own hospital. Approximately 2% of all radiographs were randomly chosen for peer review. Additionally, the tool allowed for regional analysis based on hospital, body part, and quality criterion. RESULTS: Initial findings exposed equipment-related issues such as worn imaging plates, artifacts, and poor exposures, which prompted increased preventative maintenance. Other documented issues included foreign objects, inadequate collimation and centering, and inconsistent usage of lead markers. After identifying quality assurance-related issues, hospitals implemented education, resulting in improved overall image quality scores in subsequent audits. CONCLUSION: The peer review tool helped identify and correct various issues affecting image quality and ensures our program meets required accreditation standards. Furthermore, staff found utilizing the tool to identify areas for improvement improved collaboration, ongoing education, and support between staff.

A systematic review of psychodynamic theories in community psychology: Discovering the unconscious in community work.

The aim of this systematic review is to provide a narrative synthesis of the peer-reviewed literature regarding the role of psychodynamics in community psychology. The authors screened 301 records on the topic, found in major citation databases (Scopus and Web of Science) without time or language restrictions. Ten articles addressing the review question were identified, showing the contributions of interpersonal psychoanalysis, Adlerian psychology, the Tavistock psychodynamic model, and Lacanian psychoanalysis. Several points of synergy between community psychology and psychodynamics were outlined mainly concerning empowerment theory, preventative and ecological perspective, power, and social order. Besides, the view on the community life, the role of emotion, and the conceptualization of the unconscious domain are discussed. Implications for community interventions are highlighted, regarding clients' demands, the role of community practitioners, and the use of transference/countertransference in consultative work. Limitations and future directions are also considered.

Implementation of Technology to Enhance Nursing Practice Peer Review Processes.

Nursing Peer Review is a foundational and essential element of professional nursing practice. It is a systematic methodology to improve nurse and patient outcomes. The process can be labor-intensive and cumbersome in managing data from diverse data sources, especially if the process is manual. Directors of Professional Practice in a health care system partnered with an external vendor to create an interactive software platform where technology was leveraged to streamline the review process including review of aggregate data and trend analyses and generate reports using an electronic database. This resulted in a 75% reduction in the number of steps and subsequently the time required to complete the review process from initial screening to referral and closure. The generation of actionable data facilitated active engagement of clinical nurses in addressing identified clinical issues using process improvement and evidence-based practice methods. A critical feature of the software platform is that it provides actionable data that can be used to improve patient safety and fosters accountability for clinical nurses to promote self-regulation of nursing practice.

A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals.

BACKGROUND: Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals. METHODS: Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes. RESULTS: After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers' tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items). CONCLUSIONS: Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.

Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study.

BACKGROUND: The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process. METHODS: We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9). RESULTS: The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal's usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88-1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82-89] versus 20% [16-24] and in identifying a switch in primary outcome 61% [44-77] versus 11% [3-26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57-65] versus 77% [74-81] and to identify a switch in primary outcome 77% [67-86] versus 98% [92-100]. CONCLUSIONS: Trained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting. TRIAL REGISTRATION: Clinical.Trials.gov NCT03119376 (Registered April, 18, 2017).

Peer Review Practices for Evaluating Biomedical Research Grants: A Scientific Statement From the American Heart Association.

The biomedical research enterprise depends on the fair and objective peer review of research grants, leading to the distribution of resources through efficient and robust competitive methods. In the United States, federal funding agencies and foundations collectively distribute billions of dollars annually to support biomedical research. For the American Heart Association, a Peer Review Subcommittee is charged with establishing the highest standards for peer review. This scientific statement reviews the current literature on peer review practices, describes the current American Heart Association peer review process and those of other agencies, analyzes the strengths and weaknesses of American Heart Association peer review practices, and recommends best practices for the future.