The impact of mandatory conflict of interest disclosures on editors' manuscript acceptance decisions: A cross-sectional observational study.

Objective: Although current ethical standards mandate conflict of interest (COI) disclosure by authors of peer-reviewed publications, it is unknown whether disclosure affects a manuscript's fate. Our objective was to identify associations between author COI disclosure and editorial decision to publish. Methods: We performed a cross-sectional observational study of editorial decisions for original research and brief research report manuscripts submitted to Annals of Emergency Medicine between June 2014 and January 2018 using data from the journal's editorial decision software and data from a prior study that characterized author COI for the same manuscripts. Outcomes of interest included final editor decision to publish (primary), initial editor decision, and number of revisions. We compared outcomes for manuscripts with COI versus those without and by type of COI (commercial/government/other). Results: Out of 1312 manuscripts in the sample, 65.1% had no COI declarations, and 34.9% had one or more. Overall likelihood of editorial decision to publish was 13.5% (115/854) for articles without COI and 26.9% (123/458) for those with COI. Overall likelihood of editorial decision to publish was 19.8% (19/96) for articles with commercial COI only versus 33.3% (35/105) for those with government COI only. Conclusions: Articles with author-reported COI were more likely to be published than those without such a declaration. Additionally, results suggest that reports of government COI are associated with improved chance of publication. Authorities should consider relaxing COI requirements temporarily to allow investigators to perform larger scale, randomized controlled studies of the impact of mandated COI disclosure.

Effect of revealing authors' conflicts of interests in peer review: randomized controlled trial.

OBJECTIVE: To assess the effect of disclosing authors' conflict of interest declarations to peer reviewers at a medical journal. DESIGN: Randomized controlled trial. SETTING: Manuscript review process at the Annals of Emergency Medicine. PARTICIPANTS: Reviewers (n=838) who reviewed manuscripts submitted between 2 June 2014 and 23 January 2018 inclusive (n=1480 manuscripts). INTERVENTION: Reviewers were randomized to either receive (treatment) or not receive (control) authors' full International Committee of Medical Journal Editors format conflict of interest disclosures before reviewing manuscripts. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain "counterfactual scores"-that is, the scores they believed they would have given had they been assigned to the opposite arm-as well as attitudes toward conflicts of interest. MAIN OUTCOME MEASURE: Overall quality score that reviewers assigned to the manuscript on submitting their review (1 to 5 scale). Secondary outcomes were scores the reviewers submitted for the seven more specific quality ratings and counterfactual scores elicited in the follow-up survey. RESULTS: Providing authors' conflict of interest disclosures did not affect reviewers' mean ratings of manuscript quality (Mcontrol=2.70 (SD 1.11) out of 5; Mtreatment=2.74 (1.13) out of 5; mean difference 0.04, 95% confidence interval -0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol= 2.85 (1.12) out of 5; Mtreatment=2.96 (1.16) out of 5; mean difference 0.11, -0.05 to 0.26). Similarly, no effect of the treatment was seen on any of the other seven quality ratings that the reviewers assigned. Reviewers acknowledged conflicts of interest as an important matter and believed that they could correct for them when they were disclosed. However, their counterfactual scores did not differ from actual scores (Mactual=2.69; Mcounterfactual=2.67; difference in means 0.02, 0.01 to 0.02). When conflicts were reported, a comparison of different source types (for example, government, for-profit corporation) found no difference in effect. CONCLUSIONS: Current ethical standards require disclosure of conflicts of interest for all scientific reports. As currently implemented, this practice had no effect on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.

Industry payments to physician journal editors.

BACKGROUND: Open Payments is a United States federal program mandating reporting of medical industry payments to physicians, increasing transparency of physician conflicts of interest (COI). Study objectives were to assess industry payments to physician-editors, and to compare their financial COI rate to all physicians within the specialty. METHODS AND FINDINGS: We performed a retrospective analysis of prospectively collected data, reviewing Open Payments from August 1, 2013 to December 31, 2016. We reviewed general payments ("… not made in connection with a research agreement") and research funding to "top tier" physician-editors of highly-cited medical journals. We compared payments to physician-editors and physicians-by-specialty. In 35 journals, 333 (74.5%) of 447 "top tier" US-based editors met inclusion criteria. Of these, 212 (63.7%) received industry-associated payments in the study period. In an average year, 141 (42.3%) of physician-editors received any direct payments to themselves including general payments and research payments, 66 (19.8%) received direct payments >$5,000 (National Institutes of Health threshold for a Significant Financial Interest) and 51 (15.3%) received >$10,000. Mean annual general payments to physician-editors was $55,157 (median 3,512, standard deviation 561,885, range 10-10,981,153). Median general payments to physician-editors were mostly higher compared to all physicians within their specialty. Mean annual direct research payment to the physician-editor was $14,558 (median 4,000, range 15-174,440). Mean annual indirect research funding to the physician-editor's institution (highly valued by academic leaders such as departmental chairs and deans) was $175,282 (median 49,107, range 0.18-5,000,000). The main study limitation was difficulty identifying physician-editors primarily responsible for making manuscript decisions. CONCLUSIONS: A substantial minority of physician-editors receive payments from industry within any given year, sometimes quite large. Most editors received payment of some kind during the four-year study period. Given the extent of editors' influences on the medical literature, more robust and accessible editor financial COI declarations are recommended.

Medical journal editors lacked familiarity with scientific publication issues despite training and regular exposure.

OBJECTIVE: To characterize medical editors by determining their demographics, training, potential sources of conflict of interest (COI), and familiarity with ethical standards. STUDY DESIGN AND SETTING: We selected editors of clinical medical journals with the highest annual citation rates. One hundred eighty-three editors were electronically surveyed (response rate, 52%) on demographics and experiences with editorial training, publication ethics, industry, and scientific publication organizations. RESULTS: Editors reported formal (76%) and informal (89%) training in medical editing topics. Most editors saw publication ethics issues (e.g., authorship, COIs) at least once a year. When presented with four questions about editorial issues discussed in commonly cited authoritative policy sources, performance was poor on topics of authorship (30% answered correctly), COI (15%), peer review (16%), and plagiarism (17%). Despite this, confidence level in editorial skills on a Likert scale from the beginning to the end of the survey dropped only slightly from 4.2 to 3.9 (P<0.0001). CONCLUSION: Our study presents a current look at editors of major clinical medical journals. Most editors reported training in medical editing topics, saw ethical issues regularly, and were aware of scientific publication organizations, but their knowledge of four common and well-disseminated publication ethics topics appears poor.

Perceptions of conflict of interest disclosures among peer reviewers.

BACKGROUND: Disclosure of financial conflicts of interest (COI) is intended to help reviewers assess the impact of potential bias on the validity of research results; however, there have been no empiric assessments of how reviewers understand and use disclosures in article evaluation. We investigate reviewers' perceptions of potential bias introduced by particular author disclosures, and whether reviewer characteristics are associated with a greater likelihood of perceiving bias. METHODOLOGY/PRINCIPAL FINDINGS: Of the 911 active reviewers from the Annals of Emergency Medicine, 410 were randomly selected and invited to complete our web-based, 3-part survey. We completed descriptive analysis of all survey responses and compared those responses across reviewer characteristics using 2 × 2 analyses and the Fisher exact test. We had a response rate of 54%. The majority of reviewers surveyed reported a high level of skepticism regarding financial relationships between authors and industry without a clear or consistent translation of that skepticism into the self-reported actions that characterize manuscript assessment. Only 13% of respondents believed physician consultants authoring articles based on company data are likely to have unlimited data access. 54% believed that bias most likely exists with any honorarium, regardless of monetary amount. Between 46% and 64%, depending on the type of financial relationship disclosed, reported that their recommendation for publication remains unchanged. Respondents reporting personal financial ties to industry were less likely to perceive bias in industry relationships and less likely to believe that bias exists with any monetary amount of honoraria. CONCLUSIONS: We recommend that the monetary amount of all financial relationships be reported with manuscript submissions, lead authors certify that they have unrestricted access to data, and reviewers disclose any financial ties to industry whether or not they are related to the manuscript under review. Further research is required to better understand reviewers' perceptions of financial relationships between authors and industry in order to develop clear and consistent guidelines for incorporating the perception of potential bias into manuscript assessments.

Implementation of a journal peer reviewer stratification system based on quality and reliability.

STUDY OBJECTIVE: Before starting this study, Annals of Emergency Medicine had a large and unwieldy reviewer pool that demonstrated substantial variability in quality and reliability. We hypothesize that a tiered reviewer stratification system might enable our journal editors to target the bulk of their review invitations to our better reviewers and thus improve our efficiency. METHODS: In 2003, we instituted a 3-tiered hierarchic classification for our reviewers and stratified them within these categories according to predefined criteria for reviewer quality and reliability. Our approximately 50 editors then targeted the bulk of their review invitations to the top performance tier. RESULTS: Comparing 2009 data with 2002 (the year before the system), we found fewer late reviews (13% versus 32%) and fewer reviewers not used in a given year (28% versus 59%). More top-tier reviewer invitations led to an on-time review (48% versus 37%) in 2009 compared with 2002. Editors have found the system to be simple and easy to use. No serious problems have been identified. CONCLUSION: Implementation of a tiered system stratifying journal peer reviewers by quality and reliability was readily accomplished by Annals and has appeared to improve the efficiency of our peer review.