Funding and conclusions of network meta-analyses on targeted therapies in inflammatory diseases: an overview.

OBJECTIVES: To explore the association between industry funding and network meta-analyses' (NMAs) conclusion, and the use in Clinical Practice Guidelines (CPGs) of NMAs. STUDY DESIGN AND SETTING: This was an overview of NMAs and CPGs. We searched PubMed/MEDLINE, Epistemonikos, and several guideline databases up to February 18th 2023. We included CPGs from the last 5 years and NMAs of randomized controlled trials that evaluated targeted therapies in immune-mediated inflammatory diseases. Data extraction and outcome assessments were done in duplicate by independent authors. RESULTS: We included 216 NMAs and 99 CPGs. 31% (67/216) were industry-funded. The proportion of industry-funded NMAs that cited one treatment as being best was 44% (25/57) compared to 26% (30/116) for nonindustry-funded (OR = 2.24 [1.15-4.39]; aOR = 1.76 [0.81-3.81]). The abstract's conclusion of 39/67 (58%) industry-funded and 69/149 (46%) nonindustry-funded NMAs were considered unsupported by the results (OR = 1.61 [0.90-2.89]; aOR = 1.40 [0.71-2.78]). All industry-funded NMAs that cited one treatment as best cited their own sponsored drug. 59/99 (60%) CPGs included at least one NMA, with 23/59 (39%) of them citing industry-funded NMAs. CONCLUSIONS: We did not find evidence that industry-funded NMAs were more likely to have unsupported conclusions or to cite only one treatment as being best in their conclusions compared to non-industry-funded NMAs. However, almost all industry-funded NMAs favored their own treatments. Even though 40% of the CPGs did not rely on NMA, over a third of those who did used industry-funded NMAs. Limitations include the possible misclassification due to undisclosed funding and potential confounders that have not been accounted for.

Panel stacking is a threat to consensus statement validity.

Consensus statements can be very influential in medicine and public health. Some of these statements use systematic evidence synthesis but others fail on this front. Many consensus statements use panels of experts to deduce perceived consensus through Delphi processes. We argue that stacking of panel members toward one particular position or narrative is a major threat, especially in absence of systematic evidence review. Stacking may involve financial conflicts of interest, but nonfinancial conflicts of strong advocacy can also cause major bias. Given their emerging importance, we describe here how such consensus statements may be misleading, by analyzing in depth a recent high-impact Delphi consensus statement on COVID-19 recommendations as a case example. We demonstrate that many of the selected panel members and at least 35% of the core panel members had advocated toward COVID-19 elimination (Zero-COVID) during the pandemic and were leading members of aggressive advocacy groups. These advocacy conflicts were not declared in the Delphi consensus publication, with rare exceptions. Therefore, we propose that consensus statements should always require rigorous evidence synthesis and maximal transparency on potential biases toward advocacy or lobbyist groups to be valid. While advocacy can have many important functions, its biased impact on consensus panels should be carefully avoided.

A novel framework to assess haematology and oncology registration trials: The THEOREMM project.

BACKGROUND: Methodological limitations affect a significant number of oncology and haematology trials, raising concerns about the applicability of their results. For example, a suboptimal control arm or limited access to best care upon progression may skew the trial results toward a benefit in the experimental arm. Beyond the fact that such limitations do not prevent drugs reaching the market, other assessment tools, such as those developed by professional societies-ESMO-MCBS and ASCO Value Framework-do not integrate these important shortcomings. METHODS: We propose creating a novel framework with the scope of assessing registration cancer clinical trials in haematology and oncology (randomized or single arm)-that is trials leading to a marketing authorization. The main steps of the methods are (1) assembling a scientific board; (2) defining the scope, goal and methods through pre-specified, pre-registered and protocolized methodology; (3) preregistration of the protocol; (4) conducting a scoping review of limitations and biases affecting oncology trials and assessing existing scores or methods; (5) developing a list of features to be included and assessed within the framework; (6) assessing each feature through a questionnaire sent to highly cited haematologists and oncologists involved in clinical trials; and (7) finalizing the first version of framework. RESULTS: Not applicable. CONCLUSIONS: Our proposal emerged in response to the lack of consideration for key limitations in current trial assessments. The goal is to create a framework specifically designed to assess single trials leading to marketing authorization in the field of oncology and haematogy.

Vibration of effects resulting from treatment selection in mixed-treatment comparisons: a multiverse analysis on network meta-analyses of antidepressants in major depressive disorder.

OBJECTIVE: It is frequent to find overlapping network meta-analyses (NMAs) on the same topic with differences in terms of both treatments included and effect estimates. We aimed to evaluate the impact on effect estimates of selecting different treatment combinations (ie, network geometries) for inclusion in NMAs. DESIGN: Multiverse analysis, covering all possible NMAs on different combinations of treatments. SETTING: Data from a previously published NMA exploring the comparative effectiveness of 22 treatments (21 antidepressants and a placebo) for the treatment of acute major depressive disorder. PARTICIPANTS: Cipriani et al explored a dataset of 116 477 patients included in 522 randomised controlled trials. MAIN OUTCOME MEASURES: For each possible treatment selection, we performed an NMA to estimate comparative effectiveness on treatment response and treatment discontinuation for the treatments included (231 between-treatment comparisons). The distribution of effect estimates of between-treatment comparisons across NMAs was computed, and the direction, magnitude and statistical significance of the 1st and 99th percentiles were compared. RESULTS: 4 116 254 different NMAs concerned treatment response. Among possible network geometries, 172/231 (74%) pairwise comparisons exhibited opposite effects between the 1st and 99th percentiles, 57/231 (25%) comparisons exhibited statistically significant results in opposite directions, 118 of 231 (51%) comparisons derived results that were both significant and non-significant at 5% risk and 56/231 (24%) treatment pairs obtained consistent results with only significant differences (or only non-significant differences) at 5% risk. Comparisons based on indirect evidence only were associated with greater variability in effect estimates. Comparisons with small absolute values observed in the complete NMA more frequently obtained statistically significant results in opposite directions. Similar results were observed for treatment discontinuation. CONCLUSION: In this multiverse analysis, we observed that the selection of treatments to be included in an NMA could have considerable consequences on treatment effect estimations. TRIAL REGISTRATION: https://osf.io/mb5dy.

Reporting of interventional clinical trial results in an academic center: a survey of completed studies.

BACKGROUND: The dissemination of clinical trial results is an important scientific and ethical endeavour. This survey of completed interventional studies in a French academic center describes their reporting status. METHODS: We explored all interventional studies sponsored by Rennes University Hospital identified on the French Open Science Monitor which tracks trials registered on EUCTR or clinicaltrials.gov, and provides an automatic assessment of the reporting of results. For each study, we ascertained the actual reporting of results using systematic searches on the hospital internal database, bibliographic databases (Google Scholar, PubMed), and by contacting all principal investigators (PIs). We describe several features (including total budget and numbers of trial participants) of the studies that did not report any results. RESULTS: The French Open Science Monitor identified 93 interventional studies, among which 10 (11%) reported results. In contrast, our survey identified 36 studies (39%) reporting primary analysis results and an additional 18 (19%) reporting results for secondary analyses (without results for their primary analysis). The overall budget for studies that did not report any results was estimated to be €5,051,253 for a total of 6,735 trial participants. The most frequent reasons for the absence of results reported by PIs were lack of time for 18 (42%), and logistic difficulties (e.g. delay in obtaining results or another blocking factor) for 12 (28%). An association was found between non-publication and negative results (adjusted Odds Ratio = 4.70, 95% Confidence Interval [1.67;14.11]). CONCLUSIONS: Even allowing for the fact that automatic searches underestimate the number of studies with published results, the level of reporting was disappointingly low. This amounts to a waste of trial participants' implication and money. Corrective actions are needed. TRIAL REGISTRATION: https://osf.io/q5hcs.

Assessing the magnitude of changes from protocol to publication-a survey on Cochrane and non-Cochrane Systematic Reviews.

Objective: To explore differences between published reviews and their respective protocols in a sample of 97 non-Cochrane Systematic Reviews (non-CSRs) and 97 Cochrane Systematic Reviews (CSRs) in terms of PICOS (Patients/Population, Intervention, Comparison/Control, Outcome, Study type) elements and the extent to which they were reported. Study Design and Setting: We searched PubMed and Cochrane databases to identify non-CSRs and CSRs that were published in 2018. We then searched for their corresponding Cochrane or PROSPERO protocols. The published reviews were compared to their protocols. The primary outcome was changes from protocol to review in terms of PICOS elements. Results: We identified a total of 227 changes from protocol to review in PICOS elements, 1.11 (Standard Deviation (SD), 1.22) changes per review for CSRs and 1.23 (SD, 1.12) for non-CSRs per review. More than half of each sub-sample (54.6% of CSRs and 67.0% of non-CSRs) (Absolute Risk Reduction (ARR) 12.4% [-1.3%; 26.0%]) had changes in PICOS elements. For both subsamples, approximately a third of all changes corresponded to changes related to primary outcomes. Marked differences were found between the sub-samples for the reporting of changes. 95.8% of the changes in PICOS items were not reported in the non-CSRs compared to 42.6% in the CSRs (ARR 53.2% [43.2%; 63.2%]). Conclusion: CSRs showed better results than non-CSRs in terms of the reporting of changes. Reporting of changes from protocol needs to be promoted and requires general improvement. The limitations of this study lie in its observational design. Registration: https://osf.io/6j8gd/.

Are large prospective trials on antidepressants in mental disorders seeding trials? A descriptive study of trials registered on ClinicalTrials.gov.

OBJECTIVES: This descriptive study of registered trials aimed to identify large clinical trials on antidepressants for mental disorders: (1) to assess what proportion could be labelled as 'seeding trials' (trials for marketing purposes) and (2) to describe their methodological characteristics and outcomes. DESIGN: A search was conducted across all trials registered on ClinicalTrials.gov by drug name in March 2017. SETTING: All trials registered in the database of ClinicalTrials.gov were screened. Large registered studies were received and studies focusing prospectively on the effects of antidepressants in mental health disorders. Specific data items were extracted automatically, and subsequently inspected, corrected and completed by hand. PARTICIPANTS: Prospective studies were selected focusing on the effects of antidepressants in any mental health disorder with 800 participants or more planned for inclusion. MAIN OUTCOME MEASURES: Three members from the study team independently assessed the following 'seeding trial' characteristics in each registered study: a high level of involvement of the product manufacturer in the study design, in the data analysis and reporting of the study, an abnormally low ratio of patient numbers to study site, spin and/or omissions of clinically relevant findings in the abstracts, and conclusions that focused on secondary endpoints and surrogate markers. Secondary outcomes were the exploration of a functional outcome and suicidality. RESULTS: 31 trials were identified from clinical trials database. 18/31 were published (58%). 8 of these 18 (44%) studies were identified as possible seeding trials. 13/31 (42%) large trials planned to explore functioning and 5/31 (16%) suicidality. CONCLUSIONS: Large trials are rare in the field of antidepressant research. Some could be 'seeding trials'. Few explored suicidality. Identifying seeding trials from incomplete data entries in registries, especially when almost half of the studies were still unpublished, posed considerable challenges. The delay between our research and publication limits the strength of our conclusions. PROSPERO REGISTRATION NUMBER: CRD42017065591.