Vibration of effects resulting from treatment selection in mixed-treatment comparisons: a multiverse analysis on network meta-analyses of antidepressants in major depressive disorder.

OBJECTIVE: It is frequent to find overlapping network meta-analyses (NMAs) on the same topic with differences in terms of both treatments included and effect estimates. We aimed to evaluate the impact on effect estimates of selecting different treatment combinations (ie, network geometries) for inclusion in NMAs. DESIGN: Multiverse analysis, covering all possible NMAs on different combinations of treatments. SETTING: Data from a previously published NMA exploring the comparative effectiveness of 22 treatments (21 antidepressants and a placebo) for the treatment of acute major depressive disorder. PARTICIPANTS: Cipriani et al explored a dataset of 116 477 patients included in 522 randomised controlled trials. MAIN OUTCOME MEASURES: For each possible treatment selection, we performed an NMA to estimate comparative effectiveness on treatment response and treatment discontinuation for the treatments included (231 between-treatment comparisons). The distribution of effect estimates of between-treatment comparisons across NMAs was computed, and the direction, magnitude and statistical significance of the 1st and 99th percentiles were compared. RESULTS: 4 116 254 different NMAs concerned treatment response. Among possible network geometries, 172/231 (74%) pairwise comparisons exhibited opposite effects between the 1st and 99th percentiles, 57/231 (25%) comparisons exhibited statistically significant results in opposite directions, 118 of 231 (51%) comparisons derived results that were both significant and non-significant at 5% risk and 56/231 (24%) treatment pairs obtained consistent results with only significant differences (or only non-significant differences) at 5% risk. Comparisons based on indirect evidence only were associated with greater variability in effect estimates. Comparisons with small absolute values observed in the complete NMA more frequently obtained statistically significant results in opposite directions. Similar results were observed for treatment discontinuation. CONCLUSION: In this multiverse analysis, we observed that the selection of treatments to be included in an NMA could have considerable consequences on treatment effect estimations. TRIAL REGISTRATION: https://osf.io/mb5dy.

Reporting of interventional clinical trial results in an academic center: a survey of completed studies.

BACKGROUND: The dissemination of clinical trial results is an important scientific and ethical endeavour. This survey of completed interventional studies in a French academic center describes their reporting status. METHODS: We explored all interventional studies sponsored by Rennes University Hospital identified on the French Open Science Monitor which tracks trials registered on EUCTR or clinicaltrials.gov, and provides an automatic assessment of the reporting of results. For each study, we ascertained the actual reporting of results using systematic searches on the hospital internal database, bibliographic databases (Google Scholar, PubMed), and by contacting all principal investigators (PIs). We describe several features (including total budget and numbers of trial participants) of the studies that did not report any results. RESULTS: The French Open Science Monitor identified 93 interventional studies, among which 10 (11%) reported results. In contrast, our survey identified 36 studies (39%) reporting primary analysis results and an additional 18 (19%) reporting results for secondary analyses (without results for their primary analysis). The overall budget for studies that did not report any results was estimated to be €5,051,253 for a total of 6,735 trial participants. The most frequent reasons for the absence of results reported by PIs were lack of time for 18 (42%), and logistic difficulties (e.g. delay in obtaining results or another blocking factor) for 12 (28%). An association was found between non-publication and negative results (adjusted Odds Ratio = 4.70, 95% Confidence Interval [1.67;14.11]). CONCLUSIONS: Even allowing for the fact that automatic searches underestimate the number of studies with published results, the level of reporting was disappointingly low. This amounts to a waste of trial participants' implication and money. Corrective actions are needed. TRIAL REGISTRATION: https://osf.io/q5hcs.

Assessing the magnitude of changes from protocol to publication-a survey on Cochrane and non-Cochrane Systematic Reviews.

Objective: To explore differences between published reviews and their respective protocols in a sample of 97 non-Cochrane Systematic Reviews (non-CSRs) and 97 Cochrane Systematic Reviews (CSRs) in terms of PICOS (Patients/Population, Intervention, Comparison/Control, Outcome, Study type) elements and the extent to which they were reported. Study Design and Setting: We searched PubMed and Cochrane databases to identify non-CSRs and CSRs that were published in 2018. We then searched for their corresponding Cochrane or PROSPERO protocols. The published reviews were compared to their protocols. The primary outcome was changes from protocol to review in terms of PICOS elements. Results: We identified a total of 227 changes from protocol to review in PICOS elements, 1.11 (Standard Deviation (SD), 1.22) changes per review for CSRs and 1.23 (SD, 1.12) for non-CSRs per review. More than half of each sub-sample (54.6% of CSRs and 67.0% of non-CSRs) (Absolute Risk Reduction (ARR) 12.4% [-1.3%; 26.0%]) had changes in PICOS elements. For both subsamples, approximately a third of all changes corresponded to changes related to primary outcomes. Marked differences were found between the sub-samples for the reporting of changes. 95.8% of the changes in PICOS items were not reported in the non-CSRs compared to 42.6% in the CSRs (ARR 53.2% [43.2%; 63.2%]). Conclusion: CSRs showed better results than non-CSRs in terms of the reporting of changes. Reporting of changes from protocol needs to be promoted and requires general improvement. The limitations of this study lie in its observational design. Registration: https://osf.io/6j8gd/.

Are large prospective trials on antidepressants in mental disorders seeding trials? A descriptive study of trials registered on ClinicalTrials.gov.

OBJECTIVES: This descriptive study of registered trials aimed to identify large clinical trials on antidepressants for mental disorders: (1) to assess what proportion could be labelled as 'seeding trials' (trials for marketing purposes) and (2) to describe their methodological characteristics and outcomes. DESIGN: A search was conducted across all trials registered on ClinicalTrials.gov by drug name in March 2017. SETTING: All trials registered in the database of ClinicalTrials.gov were screened. Large registered studies were received and studies focusing prospectively on the effects of antidepressants in mental health disorders. Specific data items were extracted automatically, and subsequently inspected, corrected and completed by hand. PARTICIPANTS: Prospective studies were selected focusing on the effects of antidepressants in any mental health disorder with 800 participants or more planned for inclusion. MAIN OUTCOME MEASURES: Three members from the study team independently assessed the following 'seeding trial' characteristics in each registered study: a high level of involvement of the product manufacturer in the study design, in the data analysis and reporting of the study, an abnormally low ratio of patient numbers to study site, spin and/or omissions of clinically relevant findings in the abstracts, and conclusions that focused on secondary endpoints and surrogate markers. Secondary outcomes were the exploration of a functional outcome and suicidality. RESULTS: 31 trials were identified from clinical trials database. 18/31 were published (58%). 8 of these 18 (44%) studies were identified as possible seeding trials. 13/31 (42%) large trials planned to explore functioning and 5/31 (16%) suicidality. CONCLUSIONS: Large trials are rare in the field of antidepressant research. Some could be 'seeding trials'. Few explored suicidality. Identifying seeding trials from incomplete data entries in registries, especially when almost half of the studies were still unpublished, posed considerable challenges. The delay between our research and publication limits the strength of our conclusions. PROSPERO REGISTRATION NUMBER: CRD42017065591.

Published registered reports are rare, limited to one journal group, and inadequate for randomized controlled trials in the clinical field.

OBJECTIVE: Registered reports (RR) are a publication format implying a peer-review of the protocol before the start of the study, followed by an in-principle acceptance (IPA) by the journal before the study starts. We aimed to describe randomized controlled trials (RCTs) in the clinical field published as RR. STUDY DESIGN AND SETTING: This cross-sectional study included RR results for RCTs, identified on PubMed/Medline and on a list compiled by the Center for Open Science. It explored the proportion of reports that received IPA (and/or published a protocol before inclusion of the first patient) and changes in the primary outcome. RESULTS: A total of 93 RCTs publications identified as RR were included. All but one were published in the same journal group. The date of the IPA was never documented. For most of these reports (79/93, 84.9%), a protocol was published after the date of inclusion of the first patient. A change in the primary outcome was noted in 40/93 (44%) of them. Thirteen of the 40 (33%) mentioned this change. CONCLUSIONS: RCTs in the clinical field identified as RR were rare, originated from a single journal group, and did not comply with the basic features of this format.