Coronary revascularisation outcome questionnaire: validation study of the Serbian version.

PURPOSE: To translate, culturally adapt and validate the Coronary Revascularisation Outcome Questionnaire (CROQ), a disease-specific tool for measuring health-related quality of life (HRQoL) in patients with ischaemic heart disease (IHD), into Serbian language (CROQ-S). METHODS: Validation study was performed at the Clinic for Cardiac Surgery and Clinic for Cardiology, University Clinical Centre of Serbia. We included a convenience sample of 600 patients with IHD divided into four groups. Acceptability, reliability and validity of the CROQ-S were assessed. RESULTS: CROQ-S was acceptable to patients as demonstrated by less than 1% of missing data for each single item. Cronbach's Alpha was higher than the criterion of 0.70 for all scales in each version except the Cognitive Functioning scale which only met this criterion in the CABG pre-revascularisation version. Mean values of item-total correlations were greater than 0.30 for all scales except the Cognitive Functioning scale in both the pre-revascularisation groups. Compared to the original version, exploratory factor analysis in our study showed more factors; however, the majority of items had a factor loading greater than 0.3 on the right scale. Correlations of CROQ-S scales with the 36-Item Short Form Health Survey and Seattle Angina Questionnaire showed the expected pattern whereby scales measuring similar constructs were most highly correlated. CONCLUSION: CROQ-S is an acceptable, reliable and valid disease-specific instrument for measuring HRQoL in this sample of Serbian speaking patients with IHD both before and after coronary revascularisation. However, the Cognitive Functioning scale did not meet all the psychometric criteria and further validation of its responsiveness is required.

Health journal coverage of climate change and health: a bibliometric study.

OBJECTIVES: To find what proportion of a broad set of health journals have published on climate change and health, how many articles they have published, and when they first published on the subject. DESIGN: Bibliometric study. SETTING AND PARTICIPANTS: We conducted electronic searches in Ovid MEDLINE ALL for articles about climate change and human health published from 1860 to 31 December 2022 in 330 health journals. There were no limits by language or publication type. Results were independently screened by two raters for article eligibility. RESULTS: After screening there were 2932 eligible articles published across 253 of the 330 journals between 1947 and 2022; most (2795/2932; 95%) were published in English. A few journals published articles in the early 90s, but there has been a rapid increase since about 2006. We were unable to categorise the types of publication but estimate that fewer than half are research papers. While articles were published in journals in 39 countries, two-thirds (1929/2932; 66%) were published in a journal published in the UK or the USA. Almost a quarter (77/330; 23%) of the journals published no eligible articles, and almost three-quarters (241/330; 73%) published five articles or fewer. The publication of joint editorials in over 200 journals in 2021 and 2022 boosted the number of journals publishing something on climate change and health. A third of the (112/330; 34%) journals in our sample published at least one of the joint editorials, and almost a third of those (32/112; 29%) were publishing on climate change and health for the first time. CONCLUSIONS: Health journals are rapidly increasing the amount they publish on climate change and health, but despite climate change being the major threat to global health many journals had until recently published little or nothing. A joint editorial published in multiple journals increased coverage, and for many journals it was the first thing they published on climate change and health.

Systematic review search strategies are poorly reported and not reproducible: a cross-sectional metaresearch study.

OBJECTIVES: To determine the reproducibility of biomedical systematic review search strategies. STUDY DESIGN AND SETTING: A cross-sectional reproducibility study was conducted on a random sample of 100 systematic reviews indexed in MEDLINE in November 2021. The primary outcome measure is the percentage of systematic reviews for which all database searches can be reproduced, operationalized as fulfilling six key Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension (PRISMA-S) reporting guideline items and having all database searches reproduced within 10% of the number of original results. Key reporting guideline items included database name, multi-database searching, full search strategies, limits and restrictions, date(s) of searches, and total records. RESULTS: The 100 systematic review articles contained 453 database searches. Only 22 (4.9%) database searches reported all six PRISMA-S items. Forty-seven (10.4%) database searches could be reproduced within 10% of the number of results from the original search; six searches differed by more than 1,000% between the originally reported number of results and the reproduction. Only one systematic review article provided the necessary search details to be fully reproducible. CONCLUSION: Systematic review search reporting is poor. To correct this will require a multifaceted response from authors, peer reviewers, journal editors, and database providers.

Identification of retracted publications and completeness of retraction notices in public health.

OBJECTIVES: Retraction is intended to be a mechanism to correct the published body of knowledge when necessary due to fraudulent, fatally flawed, or ethically unacceptable publications. However, the success of this mechanism requires that retracted publications be consistently identified as such and that retraction notices contain sufficient information to understand what is being retracted and why. Our study investigated how clearly and consistently retracted publications in public health are being presented to researchers. STUDY DESIGN AND SETTING: This is a cross-sectional study, using 441 retracted research publications in the field of public health. Records were retrieved for each of these publications from 11 resources, while retraction notices were retrieved from publisher websites and full-text aggregators. The identification of the retracted status of the publication was assessed using criteria from the Committee on Publication Ethics and the National Library of Medicine. The completeness of the associated retraction notices was assessed using criteria from Committee on Publication Ethics and Retraction Watch. RESULTS: Two thousand eight hundred forty-one records for retracted publications were retrieved, of which less than half indicated that the article had been retracted. Less than 5% of publications were identified as retracted through all resources through which they were available. Within single resources, if and how retracted publications were identified varied. Retraction notices were frequently incomplete, with no notices meeting all the criteria. CONCLUSIONS: The observed inconsistencies and incomplete notices pose a threat to the integrity of scientific publishing and highlight the need to better align with existing best practices to ensure more effective and transparent dissemination of information on retractions.

Prognosis Research Strategy (PROGRESS) 2: prognostic factor research.

Prognostic factor research aims to identify factors associated with subsequent clinical outcome in people with a particular disease or health condition. In this article, the second in the PROGRESS series, the authors discuss the role of prognostic factors in current clinical practice, randomised trials, and developing new interventions, and explain why and how prognostic factor research should be improved.

Prognosis Research Strategy (PROGRESS) 3: prognostic model research.

Prognostic models are abundant in the medical literature yet their use in practice seems limited. In this article, the third in the PROGRESS series, the authors review how such models are developed and validated, and then address how prognostic models are assessed for their impact on practice and patient outcomes, illustrating these ideas with examples.

Analysis of peer reviewers' response to invitations by gender and geographical region: cohort study of manuscripts reviewed at 21 biomedical journals before and during covid-19 pandemic.

OBJECTIVES: To describe gender and geographical inequalities in invitations to review and the response to these invitations and to assess whether inequalities increased during the covid-19 pandemic. DESIGN: Retrospective cohort study. SETTING: 19 specialist medical journals and two large general medical journals from BMJ Publishing Group. POPULATION: Reviewers invited to review manuscripts submitted between 1 January 2018 and 31 May 2021. The cohort was followed up to 28 February 2022. MAIN OUTCOME MEASURES: Reviewer's agreement to review. RESULTS: A total of 257 025 reviewers were invited (38.6% (88 454/228 869) women), and 90 467 (35.2%) agreed to review. Invited reviewers were mainly (217 682; 84.7%) affiliated with high income countries: Europe (122 414; 47.6%), North America (66 931; 26.0%), Africa (25 735; 10.0%), Asia (22 693; 8.8%), Oceania (16 175; 6.3%), and South America (3076; 1.2%). Independent factors associated with agreement to review were gender (odds ratio 0.89, 95% confidence interval 0.87 to 0.92, for women compared with men), geographical affiliation (2.89, 2.73 to 3.06, for Asia; 3.32, 2.94 to 3.75 for South America; 1.35, 1.27 to 1.43, for Oceania; and 0.35, 0.33 to 0.37, for Africa compared with Europe), and country income (0.47, 0.45 to 0.49, for upper middle income; 5.12, 4.67 to 5.61, for lower middle income; and 4.66, 3.79 to 5.73, for low income compared with high income country). Agreement was also independently associated with editor's gender (0.96, 0.93 to 0.99, for women compared with men), last author's geographical affiliation (0.80, 0.78 to 0.83, for Asia; 0.89, 0.85 to 0.94, for Oceania compared with Europe), impact factor (1.78, 1.27 to 2.50, for >10 compared with <5), and type of peer review process (0.52, 0.35 to 0.77, for open compared with anonymised). During the first and second phases of the pandemic, agreement was lower than in the pre-pandemic period (P<0.001). The interaction between time periods and covid-19 related topic and reviewer's gender was non-significant. However, significant interaction was found between time periods and covid-19 related topic and reviewer's geographical affiliation. CONCLUSIONS: To reduce bias and improve diversity, editors need to identify and implement effective strategies and continually evaluate progress against these to ensure that more women and researchers from upper middle income and low income countries are involved in review.

Reminding Peer Reviewers of Reporting Guideline Items to Improve Completeness in Published Articles: Primary Results of 2 Randomized Trials.

Importance: Numerous studies have shown that adherence to reporting guidelines is suboptimal. Objective: To evaluate whether asking peer reviewers to check if specific reporting guideline items were adequately reported would improve adherence to reporting guidelines in published articles. Design, Setting, and Participants: Two parallel-group, superiority randomized trials were performed using manuscripts submitted to 7 biomedical journals (5 from the BMJ Publishing Group and 2 from the Public Library of Science) as the unit of randomization, with peer reviewers allocated to the intervention or control group. Interventions: The first trial (CONSORT-PR) focused on manuscripts that presented randomized clinical trial (RCT) results and reported following the Consolidated Standards of Reporting Trials (CONSORT) guideline, and the second trial (SPIRIT-PR) focused on manuscripts that presented RCT protocols and reported following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. The CONSORT-PR trial included manuscripts that described RCT primary results (submitted July 2019 to July 2021). The SPIRIT-PR trial included manuscripts that contained RCT protocols (submitted June 2020 to May 2021). Manuscripts in both trials were randomized (1:1) to the intervention or control group; the control group received usual journal practice. In the intervention group of both trials, peer reviewers received an email from the journal that asked them to check whether the 10 most important and poorly reported CONSORT (for CONSORT-PR) or SPIRIT (for SPIRIT-PR) items were adequately reported in the manuscript. Peer reviewers and authors were not informed of the purpose of the study, and outcome assessors were blinded. Main Outcomes and Measures: The difference in the mean proportion of adequately reported 10 CONSORT or SPIRIT items between the intervention and control groups in published articles. Results: In the CONSORT-PR trial, 510 manuscripts were randomized. Of those, 243 were published (122 in the intervention group and 121 in the control group). A mean proportion of 69.3% (95% CI, 66.0%-72.7%) of the 10 CONSORT items were adequately reported in the intervention group and 66.6% (95% CI, 62.5%-70.7%) in the control group (mean difference, 2.7%; 95% CI, -2.6% to 8.0%). In the SPIRIT-PR trial, of the 244 randomized manuscripts, 178 were published (90 in the intervention group and 88 in the control group). A mean proportion of 46.1% (95% CI, 41.8%-50.4%) of the 10 SPIRIT items were adequately reported in the intervention group and 45.6% (95% CI, 41.7% to 49.4%) in the control group (mean difference, 0.5%; 95% CI, -5.2% to 6.3%). Conclusions and Relevance: These 2 randomized trials found that it was not useful to implement the tested intervention to increase reporting completeness in published articles. Other interventions should be assessed and considered in the future. Trial Registration: ClinicalTrials.gov Identifiers: NCT05820971 (CONSORT-PR) and NCT05820984 (SPIRIT-PR).

Evaluation of editors' abilities to predict the citation potential of research manuscripts submitted to The BMJ: a cohort study.

OBJECTIVE: To evaluate the ability of The BMJ editors to predict the number of times submitted research manuscripts will be cited. DESIGN: Cohort study. SETTING: Manuscripts submitted to The BMJ, reviewed, and subsequently scheduled for discussion at a prepublication meeting between 27 August 2015 and 29 December 2016. PARTICIPANTS: 10 BMJ research team editors. MAIN OUTCOME MEASURES: Reviewed manuscripts were rated independently by attending editors for citation potential in the year of first publication plus the next year: no citations, below average (<10 citations), average (10-17 citations), or high (>17 citations). Predicted citations were subsequently compared with actual citations extracted from Web of Science (WOS). RESULTS: Of 534 manuscripts reviewed, 505 were published as full length articles (219 in The BMJ) by end of 2019 and indexed in WOS, 22 were unpublished, and one abstract was withdrawn. Among the 505 manuscripts, the median (IQR [range]) number of citations in the year of publication plus the following year was 9 (4-17 [0-150]); 277 (55%) manuscripts were cited <10 times, 105 (21%) were cited 10-17 times, and 123 (24%) cited >17 times. Manuscripts accepted by The BMJ were cited more highly (median 12 (IQR 7-24) citations) than those rejected (median 7 (3-12) citations). For all 10 editors, predicted ratings tended to increase in line with actual citations, but with considerable variation within categories; nine failed to identify the correct citation category for >50% (range 31%-52%) of manuscripts, and κ ranged between 0.01 to 0.19 for agreement between predicted and actual categories. Editors more often rated papers that achieved high actual citation counts as having low citation potential than the reverse. Collectively, the mean percentage of editors predicting the correct citation category was 43%, and for 160 (32%) manuscripts at least 50% of editors predicted the right category. CONCLUSIONS: Editors weren't good at estimating the citation potential of manuscripts individually or as a group; there is no wisdom of the crowd when it comes to BMJ editors.

Peer Reviewed Evaluation of Registered End-Points of Randomised Trials (the PRE-REPORT study): a stepped wedge, cluster-randomised trial.

OBJECTIVE: To test whether providing relevant clinical trial registry information to peer reviewers evaluating trial manuscripts decreases discrepancies between registered and published trial outcomes. DESIGN: Stepped wedge, cluster-randomised trial, with clusters comprised of eligible manuscripts submitted to each participating journal between 1 November 2018 and 31 October 2019. SETTING: Thirteen medical journals. PARTICIPANTS: Manuscripts were eligible for inclusion if they were submitted to a participating journal during the study period, presented results from the primary analysis of a clinical trial, and were peer reviewed. INTERVENTIONS: During the control phase, there were no changes to pre-existing peer review practices. After journals crossed over into the intervention phase, peer reviewers received a data sheet describing whether trials were registered, the initial registration and enrolment dates, and the registered primary outcome(s) when enrolment began. MAIN OUTCOME MEASURE: The presence of a clearly defined, prospectively registered primary outcome consistent with the primary outcome in the published trial manuscript, as determined by two independent outcome assessors. RESULTS: We included 419 manuscripts (243 control and 176 intervention). Participating journals published 43% of control-phase manuscripts and 39% of intervention-phase manuscripts (model-estimated percentage difference between intervention and control trials = -10%, 95% CI -25% to 4%). Among the 173 accepted trials, published primary outcomes were consistent with clearly defined, prospectively registered primary outcomes in 40 of 105 (38%) control-phase trials and 27 of 68 (40%) intervention-phase trials. A linear mixed model did not show evidence of a statistically significant primary outcome effect from the intervention (estimated difference between intervention and control=-6% (90% CI -27% to 15%); one-sided p value=0.68). CONCLUSIONS: These results do not support use of the tested intervention as implemented here to increase agreement between prospectively registered and published trial outcomes. Other approaches are needed to improve the quality of outcome reporting of clinical trials. TRIAL REGISTRATION NUMBER: ISRCTN41225307.

International challenges without borders: a descriptive study of family physicians' educational needs in the field of diabetes.

BACKGROUND: The optimal care of persons with diabetes by general practitioners and family physicians (GP/FP) is complex and requires multiple competencies. This is a fairly unrecognized key challenge in the healthcare systems. In some cases, local and national Continuous Professional Development (CPD) initiatives target these challenges; however there have been few international initiatives, possibly because challenges emerging from different studies have not been linked across national boundaries. In this context, the authors have compiled data about gaps and/or barriers inherent to GP/FP care of persons with type 2 diabetes from Austria, Canada, Germany and the United Kingdom. METHODS: Secondary analyzes of pre-existing studies were conducted to identify challenges in the care of patients with type 2 diabetes as faced by GPs/FPs. Two sources of data were reviewed: unpublished research data from collaborating organizations and articles from a literature search (in English and German). Articles retrieved were scanned by the research team for relevance to the study objectives and to extract existing gaps and barriers. The identified challenges were then categorized along three major axes: (1) phase of the continuum of care {from screening to management}; (2) learning domain {knowledge, skills, attitudes, behavior, context}; and (3) by country/region. Compilation and categorization were performed by qualitative researchers and discrepancies were resolved through discussion until concordance was achieved. RESULTS AND DISCUSSION: Thirteen challenges faced by GPs/FPs in the care for patients with type 2 diabetes were common in at least 3 of the 4 targeted countries/regions. These issues were found across the entire continuum of care and included: pathophysiology of diabetes, diagnostic criteria, treatment targets assessment, drugs' modes of action, decision-making in therapies, treatment guidelines, insulin therapy, adherence, management of complications, lifestyle changes, team integration, bureaucracy and third-party payers. The issues reported were not restricted to the physicians' knowledge, but also related to their skills, attitudes, behaviours and context. CONCLUSIONS: This study revealed challenges faced by GPs/FPs when caring for patients with diabetes, which were similar across international and health system borders. Common issues might be addressed more efficiently through international educational designs, adapted to each country's healthcare system, helping develop and maintain physicians' competencies.

Evaluation of an online interactive Diabetes Needs Assessment Tool (DNAT) versus online self-directed learning: a randomised controlled trial.

BACKGROUND: Methods for the dissemination, understanding and implementation of clinical guidelines need to be examined for their effectiveness to help doctors integrate guidelines into practice. The objective of this randomised controlled trial was to evaluate the effectiveness of an interactive online Diabetes Needs Assessment Tool (DNAT) (which constructs an e-learning curriculum based on individually identified knowledge gaps), compared with self-directed e-learning of diabetes guidelines. METHODS: Health professionals were randomised to a 4-month learning period and either given access to diabetes learning modules alone (control group) or DNAT plus learning modules (intervention group). Participants completed knowledge tests before and after learning (primary outcome), and surveys to assess the acceptability of the learning and changes to clinical practice (secondary outcomes). RESULTS: Sixty four percent (677/1054) of participants completed both knowledge tests. The proportion of nurses (5.4%) was too small for meaningful analysis so they were excluded. For the 650 doctors completing both tests, mean (SD) knowledge scores increased from 47.4% (12.6) to 66.8% (11.5) [intervention group (n = 321, 64%)] and 47.3% (12.9) to 67.8% (10.8) [control group (n = 329, 66%)], (ANCOVA p = 0.186). Both groups were satisfied with the usability and usefulness of the learning materials. Seventy seven percent (218/284) of the intervention group reported combining the DNAT with the recommended reading materials was "very useful"/"useful". The majority in both groups (184/287, 64.1% intervention group and 206/299, 68.9% control group) [95% CI for the difference (-2.8 to 12.4)] reported integrating the learning into their clinical practice. CONCLUSIONS: Both groups experienced a similar and significant improvement in knowledge. The learning materials were acceptable and participants incorporated the acquired knowledge into practice. TRIAL REGISTRATION: ISRCTN: ISRCTN67215088.

Female authorship of covid-19 research in manuscripts submitted to 11 biomedical journals: cross sectional study.

OBJECTIVE: To describe prominent authorship positions held by women and the overall percentage of women co-authoring manuscripts submitted during the covid-19 pandemic compared with the previous two years. DESIGN: Cross sectional study. SETTING: Nine specialist and two large general medical journals. POPULATION: Authors of research manuscripts submitted between 1 January 2018 and 31 May 2021. MAIN OUTCOME MEASURES: Primary outcome: first author's gender. SECONDARY OUTCOMES: last and corresponding authors' gender; number (percentage) of women on authorship byline in "pre-pandemic" period (1 January 2018 to 31 December 2019) and in "covid-19" and "non-covid-19" manuscripts during pandemic. RESULTS: A total of 63 259 manuscripts were included. The number of female first, last, and corresponding authors respectively were 1313 (37.1%), 996 (27.9%), and 1119 (31.1%) for covid-19 manuscripts (lowest values in Jan-May 2020: 230 (29.4%), 165 (21.1%), and 185 (22.9%)), compared with 8583 (44.9%), 6118 (31.2%), and 7273 (37.3%) for pandemic non-covid-19 manuscripts and 12 724 (46.0%), 8923 (31.4%), and 10 981 (38.9%) for pre-pandemic manuscripts. The adjusted odds ratio of having a female first author in covid-19 manuscripts was <1.00 in all groups (P<0.001) compared with pre-pandemic (lowest in Jan-May 2020: 0.55, 98.75% confidence interval 0.43 to 0.70). The adjusted odds ratio of having a woman as last or corresponding author was significantly lower for covid-19 manuscripts in all time periods (except for the two most recent periods for last author) compared with pre-pandemic (lowest values in Jan-May 2020: 0.74 (0.57 to 0.97) for last and 0.61 (0.49 to 0.77) for corresponding author). The odds ratios for pandemic non-covid-19 manuscripts were not significantly different compared with pre-pandemic manuscripts. The median percentage of female authors on the byline was lower for covid-19 manuscripts (28.6% in Jan-May 2020) compared with pre-pandemic (36.4%) and non-covid-19 pandemic manuscripts (33.3% in Jan-May 2020). Gender disparities in all prominent authorship positions and the proportion of women authors on the byline narrowed in the most recent period (Feb-May 2021) compared with the early pandemic period (Jan-May 2020) and were very similar to values observed for pre-pandemic manuscripts. CONCLUSIONS: Women have been underrepresented as co-authors and in prominent authorship positions in covid-19 research, and this gender disparity needs to be corrected by those involved in academic promotion and awarding of research grants. Women attained some prominent authorship positions equally or more frequently than before the pandemic on non-covid-19 related manuscripts submitted at some time points during the pandemic.

Consistency of causal claims in observational studies: a review of papers published in a general medical journal.

OBJECTIVE: To evaluate the consistency of causal statements in observational studies published in The BMJ. DESIGN: Review of observational studies published in a general medical journal. DATA SOURCE: Cohort and other longitudinal studies describing an exposure-outcome relationship published in The BMJ in 2018. We also had access to the submitted papers and reviewer reports. MAIN OUTCOME MEASURES: Proportion of published research papers with 'inconsistent' use of causal language. Papers where language was consistently causal or non-causal were classified as 'consistently causal' or 'consistently not causal', respectively. For the 'inconsistent' papers, we then compared the published and submitted version. RESULTS: Of 151 published research papers, 60 described eligible studies. Of these 60, we classified the causal language used as 'consistently causal' (48%), 'inconsistent' (20%) and 'consistently not causal'(32%). Eleven out of 12 (92%) of the 'inconsistent' papers were already inconsistent on submission. The inconsistencies found in both submitted and published versions were mainly due to mismatches between objectives and conclusions. One section might be carefully phrased in terms of association while the other presented causal language. When identifying only an association, some authors jumped to recommending acting on the findings as if motivated by the evidence presented. CONCLUSION: Further guidance is necessary for authors on what constitutes a causal statement and how to justify or discuss assumptions involved. Based on screening these papers, we provide a list of expressions beyond the obvious 'cause' word which may inspire a useful more comprehensive compendium on causal language.

A randomized trial of an editorial intervention to reduce spin in the abstract's conclusion of manuscripts showed no significant effect.

OBJECTIVE: To estimate the effect of an intervention compared to the usual peer-review process on reducing spin in the abstract's conclusion of biomedical study reports. STUDY DESIGN AND SETTING: We conducted a two-arm, parallel-group RCT in a sample of primary research manuscripts submitted to BMJ Open. The authors received short instructions alongside the peer reviewers' comments in the intervention group. We assessed the presence of spin (primary outcome), types of spin, and wording change in the revised abstract's conclusion. Outcome assessors were blinded to the intervention assignment. RESULTS: Of the 184 manuscripts randomized, 108 (54 intervention, 54 control) were selected for revision and could be evaluated for the presence of spin. The proportion of manuscripts with spin was 6% lower (95% CI: 24% lower to 13% higher) in the intervention group (57%, 31/54) than in the control group (63%, 34/54). The wording of the revised abstract's conclusion was changed in 34/54 (63%) manuscripts in the intervention group and 26/54 (48%) in the control group. The four prespecified types of spin involved (i) selective reporting (12 in the intervention group vs. 8 in the control group), (ii) including information not supported by evidence (9 vs. 9), and (iii) interpretation not consistent with the study results (14 vs. 18), and (iv) unjustified recommendations for practice (5 vs. 11). CONCLUSION: These short instructions to authors did not have a statistically significant effect on reducing spin in revised abstract conclusions, and based on the confidence interval, the existence of a large effect can be excluded. Other interventions to reduce spin in reports of original research should be evaluated. STUDY REGISTRATION: osf.io/xnuyt.

Improving peer review of systematic reviews by involving librarians and information specialists: protocol for a randomized controlled trial.

BACKGROUND: Problems continue to exist with the reporting quality and risk of bias in search methods and strategies in systematic reviews and related review types. Peer reviewers who are not familiar with what is required to transparently and fully report a search may not be prepared to review the search components of systematic reviews, nor may they know what is likely to introduce bias into a search. Librarians and information specialists, who have expertise in searching, may offer specialized knowledge that would help improve systematic review search reporting and lessen risk of bias, but they are underutilized as methodological peer reviewers. METHODS: This study will evaluate the effect of adding librarians and information specialists as methodological peer reviewers on the quality of search reporting and risk of bias in systematic review searches. The study will be a pragmatic randomized controlled trial using 150 systematic review manuscripts submitted to BMJ and BMJ Open as the unit of randomization. Manuscripts that report on completed systematic reviews and related review types and have been sent for peer review are eligible. For each manuscript randomized to the intervention, a librarian/information specialist will be invited as an additional peer reviewer using standard practices for each journal. First revision manuscripts will be assessed in duplicate for reporting quality and risk of bias, using adherence to 4 items from PRISMA-S and assessors' judgements on 4 signaling questions from ROBIS Domain 2, respectively. Identifying information from the manuscripts will be removed prior to assessment. DISCUSSION: The primary outcomes for this study are quality of reporting as indicated by differences in the proportion of adequately reported searches in first revision manuscripts between intervention and control groups and risk of bias as indicated by differences in the proportions of first revision manuscripts with high, low, and unclear bias. If the intervention demonstrates an effect on search reporting or bias, this may indicate a need for journal editors to work with librarians and information specialists as methodological peer reviewers. TRIAL REGISTRATION: Open Science Framework. Registered on June 17, 2021, at https://doi.org/10.17605/OSF.IO/W4CK2 .

Surveys of current status in biomedical science grant review: funding organisations' and grant reviewers' perspectives.

BACKGROUND: The objectives of this research were (a) to describe the current status of grant review for biomedical projects and programmes from the perspectives of international funding organisations and grant reviewers, and (b) to explore funders' interest in developing uniform requirements for grant review aimed at making the processes and practices of grant review more consistent, transparent, and user friendly. METHODS: A survey to a convenience sample of 57 international public and private organisations that give grants for biomedical research was conducted. Nine participating organisations then emailed a random sample of their external reviewers an invitation to participate in a second electronic survey. RESULTS: A total of 28 of 57 (49%) organisations in 19 countries responded. Organisations reported these problems as frequent or very frequent: declined review requests (16), late reports (10), administrative burden (7), difficulty finding new reviewers (4), and reviewers not following guidelines (4). The administrative burden of the process was reported to have increased over the past 5 years. In all, 17 organisations supported the idea of uniform requirements for conducting grant review and for formatting grant proposals. A total of 258/418 (62%) reviewers responded from 22 countries. Of those, 48% (123/258) said their institutions encouraged grant review, yet only 7% (17/258) were given protected time and 74% (192/258) received no academic recognition for this. Reviewers rated these factors as extremely or very important in deciding to review proposals: 51% (131/258) desire to support external fairness, 47% (120/258) professional duty, 46% (118/258) relevance of the proposal's topic, 43% (110/258) wanting to keep up to date, 40% (104/258) desire to avoid suppression of innovation. Only 16% (42/258) reported that guidance from funders was very clear. In all, 85% (220/258) had not been trained in grant review and 64% (166/258) wanted this. CONCLUSIONS: Funders reported a growing workload of biomedical proposals that is getting harder to peer review. Just under half of grant reviewers take part for the good of science and professional development, but many report lack of academic and practical support and clear guidance. Around two-thirds of funders supported the development of uniform requirements for the format and peer review of proposals to help ease the current situation.

Awareness, usage and perceptions of authorship guidelines: an international survey of biomedical authors.

OBJECTIVES: To investigate authors' awareness and use of authorship guidelines, and to assess their perceptions of the fairness of authorship decisions. DESIGN: A cross-sectional online survey. SETTING AND PARTICIPANTS: Corresponding authors of research papers submitted in 2014 to 18 BMJ journals. RESULTS: 3859/12 646 (31%) researchers responded. They worked in 93 countries and varied in research experience. Of these, 1326 (34%) reported their institution had an authorship policy providing criteria for authorship; 2871 (74%) were 'very familiar' with the International Committee of Medical Journal Editors' authorship criteria and 3358 (87%) reported that guidelines were beneficial when preparing manuscripts. Furthermore, 2609 (68%) reported that their use was 'sometimes' or 'frequently' encouraged in their research setting. However, 2859 respondents (74%) reported that they had been involved in a study at least once where someone was added as an author who had not contributed substantially (honorary authorship), and 1305 (34%) where someone was not listed as an author but had contributed substantially (ghost authorship). Only 740 (19%) reported that they had never experienced either honorary or ghost authorship; 1115 (29%) reported that they had experienced both at least once. There was no clear pattern in experience of authorship misappropriation by continent. For their last coauthored article, 2187 (57%) reported that explicit authorship criteria had been used to determine eligibility, and 3088 (80%) felt that the decision made was fair. When institutions frequently encouraged use of authorship guidelines, authorship eligibility was more likely to be discussed early (817 of 1410, 58%) and perceived as fairer (1273 of 1410, 90%) compared with infrequent encouragement (974 of 2449, 40%, and 1891 of 2449, 74%). CONCLUSIONS: Despite a high level of awareness of authorship guidelines and criteria, these are not so widely used; more explicit encouragement of their use by institutions may result in more favourable use of guidelines by authors.

Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial.

OBJECTIVE: To evaluate the impact of an editorial intervention to improve completeness of reporting of reports of randomised trials. DESIGN: Randomised controlled trial (RCT). SETTING: BMJ Open's quality improvement programme. PARTICIPANTS: 24 manuscripts describing RCTs. INTERVENTIONS: We used an R Shiny application to randomise manuscripts (1:1 allocation ratio, blocks of 4) to the intervention (n=12) or control (n=12) group. The intervention was performed by a researcher with expertise in the content of the Consolidated Standards of Reporting Trials (CONSORT) and consisted of an evaluation of completeness of reporting of eight core CONSORT items using the submitted checklist to locate information, and the production of a report containing specific requests for authors based on the reporting issues found, provided alongside the peer review reports. The control group underwent the usual peer review. OUTCOMES: The primary outcome is the number of adequately reported items (0-8 scale) in the revised manuscript after the first round of peer review. The main analysis was intention-to-treat (n=24), and we imputed the scores of lost to follow-up manuscripts (rejected after peer review and not resubmitted). The secondary outcome is the proportion of manuscripts where each item was adequately reported. Two blinded reviewers assessed the outcomes independently and in duplicate and solved disagreements by consensus. We also recorded the amount of time to perform the intervention. RESULTS: Manuscripts in the intervention group (mean: 7.01; SD: 1.47) were more completely reported than those in the control group (mean: 5.68; SD: 1.43) (mean difference 1.43, 95% CI 0.31 to 2.58). We observed the main differences in items 6a (outcomes), 9 (allocation concealment mechanism), 11a (blinding) and 17a (outcomes and estimation). The mean time to perform the intervention was 87 (SD 42) min. CONCLUSIONS: We demonstrated the benefit of involving a reporting guideline expert in the editorial process. Improving the completeness of RCTs is essential to enhance their usability. TRIAL REGISTRATION NUMBER: NCT03751878.