Peer review comments on drug trials submitted to medical journals differ depending on sponsorship, results and acceptance: a retrospective cohort study.

OBJECTIVE: During peer review, submitted manuscripts are scrutinised by independent experts to assist journal editors in their decision-making and to help improve the quality of articles. In this retrospective cohort study, peer review comments for drug trials submitted to medical journals were analysed to investigate whether there is a relation between the content of these comments and sponsorship, direction of results and decision about acceptance. DESIGN/SETTING: Descriptive content analysis of reviewer comments made on manuscripts on drug trials submitted to eight medical journals (January 2010-April 2012). For each manuscript, the number of reviewers, decision about acceptance, sponsorship and direction of results were extracted. Reviewer comments were classified using a predefined checklist. RESULTS: Reviewer reports for 246 manuscripts were assessed. Industry-sponsored trials were more likely to receive comments about lack of novelty (8.9%) than industry-supported (2.5%) and non-industry trials (6.1%, overall p=0.038). Non-industry trials more often received comments about poor experimental design (69.7%) than industry-supported (58.8%) and industry-sponsored trials (52.9%, overall p=0.019). Non-industry trials were also more likely to receive comments regarding inappropriate statistical analyses (28.4%) than industry-supported (23.5%) and industry-sponsored trials (15.1%, overall p=0.006). Manuscripts with negative results were more likely to receive comments about inappropriate conclusions (29.3%) than those with positive results (18.9%, p=0.010). Rejected manuscripts had more often received comments on the research question not being clinically relevant (7.8%) than accepted manuscripts (1.6%, p=0.002), and also on lack of novelty (8.3% vs 2.6%, p=0.008) and poor experimental design (68.6% vs 50.5%, p<0.001). CONCLUSIONS: Reviewers identified fewer shortcomings regarding design and statistical analyses in industry-related trials, but commented more often on a lack of novelty in industry-sponsored trials. Negative trial results did not significantly influence the nature of comments other than appropriateness of the conclusion. Manuscript acceptance was primarily related to the research question and methodological robustness of studies.

Does Insert Type Affect Clinical and Functional Outcome in Total Knee Arthroplasty? A Randomised Controlled Clinical Trial With 5-Year Follow-Up.

New insert types have been developed to improve clinical and functional outcome in mobile bearing (MB-TKA) and fixed bearing total knee arthroplasty (FB-TKA). A prospective single blinded randomised controlled clinical trial was performed to evaluate 2 types of MB-TKA inserts and 2 types of FB-TKA inserts of the Genesis II prosthesis (Smith & Nephew) in 146 patients with 5-years follow-up. A significant difference (P=.042) between the MB-TKA inserts was found in KSS function scores, but clinical significance is expected to be limited. Goniometry, temporal gait parameters and QoL were similar in all groups. Survival was significantly better (P=.047) for FB-TKA. The comparable outcome and higher revision rate in MB-TKA indicate that FB-TKA may be preferential for the Genesis II implant system.

Differences between information in registries and articles did not influence publication acceptance.

OBJECTIVES: To assess whether journals are more likely to reject manuscripts with differences between information in registries and articles. We compared differences by sponsorship and assessed whether selective reporting favored publication of significant outcomes. STUDY DESIGN AND SETTING: Drug trials submitted to eight journals (January 2010-April 2012) were included. Publication status, primary outcomes, enrollment, and sponsorship were extracted. Primary outcomes and enrollment in registries and registration timing were reviewed. Prospective registration included registration before study start. Consistency between registered and reported information was evaluated. RESULTS: For 226 submitted manuscripts, primary outcomes were specified in both article and registry. Sixty six of 226 (29.2%) had primary outcome differences; 14 of 66 manuscripts with differences (21.2%) and 46 of 160 without differences (28.8%) were accepted. Fifty manuscripts (22.4%) had sample size differences; 10 of 50 with differences (20.0%) and 49 of 173 without differences (28.3%) were accepted. Industry-sponsored trials had less differences and were more often prospectively registered. After adjustment for sponsorship, differences and/or retrospective registration were not associated with decreased chance of acceptance (odds ratio 0.56; 95% confidence interval: 0.27, 1.13). Primary outcome differences favored significant outcomes in 49% of manuscripts. CONCLUSION: Differences between registered and reported information are not decisive for rejection. Editors should assess consistency between registries and articles to address selective reporting.

Shortcomings of protocols of drug trials in relation to sponsorship as identified by Research Ethics Committees: analysis of comments raised during ethical review.

BACKGROUND: Submission of study protocols to research ethics committees (RECs) constitutes one of the earliest stages at which planned trials are documented in detail. Previous studies have investigated the amendments requested from researchers by RECs, but the type of issues raised during REC review have not been compared by sponsor type. The objective of this study was to identify recurring shortcomings in protocols of drug trials based on REC comments and to assess whether these were more common among industry-sponsored or non-industry trials. METHODS: Retrospective analysis of 226 protocols of drug trials approved in 2010-2011 by three RECs affiliated to academic medical centres in The Netherlands. For each protocol, information on sponsorship, number of participating centres, participating countries, study phase, registration status of the study drug, and type and number of subjects was retrieved. REC comments were extracted from decision letters sent to investigators after review and were classified using a predefined checklist that was based on legislation and guidelines on clinical drug research and previous literature. RESULTS: Most protocols received comments regarding participant information and consent forms (n = 182, 80.5%), methodology and statistical analyses (n = 160, 70.8%), and supporting documentation, including trial agreements and certificates of insurance (n = 154, 68.1%). Of the submitted protocols, 122 (54.0%) were non-industry and 104 (46.0%) were industry-sponsored trials. Non-industry trials more often received comments on subject selection (n = 44, 36.1%) than industry-sponsored trials (n = 18, 17.3%; RR, 1.58; 95% CI, 1.01 to 2.47), and on methodology and statistical analyses (n = 95, 77.9% versus n = 65, 62.5%, respectively; RR, 1.18; 95% CI, 1.01 to 1.37). Non-industry trials less often received comments on supporting documentation (n = 72, 59.0%) than industry-sponsored trials (n = 82, 78.8%; RR, 0.83; 95% CI, 0.72 to 0.95). CONCLUSIONS: RECs identified important ethical and methodological shortcomings in protocols of both industry-sponsored and non-industry drug trials. Investigators, especially of non-industry trials, should better prepare their research protocols in order to facilitate the ethical review process.

Role of editorial and peer review processes in publication bias: analysis of drug trials submitted to eight medical journals.

BACKGROUND: Publication bias is generally ascribed to authors and sponsors failing to submit studies with negative results, but may also occur after submission. We evaluated whether submitted manuscripts on randomized controlled trials (RCTs) with drugs are more likely to be accepted if they report positive results. METHODS: Manuscripts submitted from January 2010 through April 2012 to one general medical journal (BMJ) and seven specialty journals (Annals of the Rheumatic Diseases, British Journal of Ophthalmology, Gut, Heart, Thorax, Diabetologia, and Journal of Hepatology) were included, if at least one study arm assessed the efficacy or safety of a drug and a statistical test was used to evaluate treatment effects. Publication status was retrospectively retrieved from submission systems or provided by journals. Sponsorship and trial results were extracted from manuscripts and classified according to predefined criteria. Main outcome measure was acceptance for publication. RESULTS: Of 15,972 manuscripts submitted, 472 (3.0%) were drug RCTs, of which 98 (20.8%) were published. Among submitted drug RCTs, 287 (60.8%) had positive and 185 (39.2%) negative results. Of these, 60 (20.9%) and 38 (20.5%), respectively, were published. Manuscripts on non-industry trials (n = 213) reported positive results in 138 (64.8%) manuscripts, compared to 71 (47.7%) on industry-supported trials (n = 149), and 78 (70.9%) on industry-sponsored trials (n = 110). Twenty-seven (12.7%) non-industry trials were published, compared to 27 (18.1%) industry-supported and 44 (40.0%) industry-sponsored trials. After adjustment for other trial characteristics, manuscripts reporting positive results were not more likely to be published (OR, 1.00; 95% CI, 0.61 to 1.66). Submission to specialty journals, sample size, multicentre status, journal impact factor, and corresponding authors from Europe or US were significantly associated with publication. CONCLUSIONS: For the selected journals, there was no tendency to preferably publish manuscripts on drug RCTs that reported positive results, suggesting that publication bias may occur mainly prior to submission.

Recommendations for a uniform assessment of publication bias related to funding source.

BACKGROUND: Numerous studies on publication bias in clinical drug research have been undertaken, particularly on the association between sponsorship and favourable outcomes. However, no standardized methodology for the classification of outcomes and sponsorship has been described. Dissimilarities and ambiguities in this assessment impede the ability to compare and summarize results of studies on publication bias. To guide authors undertaking such studies, this paper provides recommendations for a uniform assessment of publication bias related to funding source. METHODS AND RESULTS: As part of ongoing research into publication bias, 472 manuscripts on randomised controlled trials (RCTs) with drugs, submitted to eight medical journals from January 2010 through April 2012, were reviewed. Information on trial results and sponsorship was extracted from manuscripts. During the start of this evaluation, several problems related to the classification of outcomes, inclusion of post-hoc analyses and follow-up studies of RCTs in the study sample, and assessment of the role of the funding source were encountered. A comprehensive list of recommendations addressing these problems was composed. To assess internal validity, reliability and usability of these recommendations were tested through evaluation of manuscripts submitted to journals included in our study. CONCLUSIONS: The proposed recommendations represent a first step towards a uniform method of classifying trial outcomes and sponsorship. This is essential to draw valid conclusions on the role of the funding source in publication bias and will ensure consistency across future studies.