Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis.

BACKGROUND: We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs. METHODS AND FINDINGS: We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations. CONCLUSIONS: We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

Reporting Quality of Journal Abstracts for Surgical Randomized Controlled Trials Before and After the Implementation of the CONSORT Extension for Abstracts.

BACKGROUND: Adequate reporting is crucial in full-text publications but even more so in abstracts because they are the most frequently read part of a publication. In 2008, an extension for abstracts of the Consolidated Standards of Reporting Trials (CONSORT-A) statement was published, defining which items should be reported in abstracts of randomized controlled trials (RCTs). Therefore, we compared the adherence of RCT abstracts to CONSORT-A before and after the publication of CONSORT-A. METHODS: RCTs published in the five surgical journals with the highest impact factor were identified through PubMed for 2005-2007 and 2014-2016. Adherence to 15 CONSORT-A items and two additional items for abstracts of non-pharmacological trials was assessed in duplicate. We compared the overall adherence to CONSORT-A between the two time periods using an unpaired t test and explored adherence to specific items. RESULTS: A total of 192 and 164 surgical RCT abstracts were assessed (2005-2007 and 2014-2016, respectively). In the pre-CONSORT-A phase, the mean score of adequately reported items was 6.14 (95% confidence interval [CI] 5.90-6.38) and 8.11 in the post-CONSORT-A phase (95% CI 7.83-8.39; mean difference 1.97, 95% CI 1.60-2.34; p < 0.0001). The comparison of individual items indicated a significant improvement in 9 of the 15 items. The three least reported items in the post-CONSORT-A phase were randomization (2.4%), blinding (13.4%), and funding (0.0%). Specific items for non-pharmacological trials were rarely reported (approximately 10%). CONCLUSION: The reporting in abstracts of surgical RCTs has improved after the implementation of CONSORT-A. More importantly, there is still ample room for improvement.

A longitudinal assessment of trial protocols approved by research ethics committees: The Adherance to SPIrit REcommendations in the UK (ASPIRE-UK) study.

BACKGROUND: To assess the quality of reporting of RCT protocols approved by UK research ethics committees before and after the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. METHODS: We had access to RCT study protocols that received ethical approval in the UK in 2012 (n=103) and 2016 (n=108). From those, we assessed the adherence to the 33 SPIRIT items (i.e. a total of 64 components of the 33 SPIRIT items). We descriptively analysed the adherence to SPIRIT guidelines as proportion of adequately reported items (median and interquartile range [IQR]) and stratified the results by year of approval and sponsor. RESULTS: The proportion of reported SPIRIT items increased from a median of 64.9% (IQR, 57.6-69.2%) in 2012 to a median of 72.5% (IQR, 65.3-78.3%) in 2016. Industry-sponsored RCTs reported more SPIRIT items in 2012 (median 67.4%; IQR, 64.1-69.4%) compared to non-industry-sponsored trials (median 59.8%; IQR, 46.5-67.7%). This gap between industry- and non-industry-sponsored trials increased in 2016 (industry-sponsored: median 75.6%; IQR, 71.2-79.0% vs non-industry-sponsored: median 65.3%; IQR, 51.6-76.3%). CONCLUSIONS: The adherence to SPIRIT guidelines has improved in the UK from 2012 to 2016 but remains on a modest level, especially for non-industry-sponsored RCTs.

A meta-research study of randomized controlled trials found infrequent and delayed availability of protocols.

OBJECTIVES: Availability of randomized controlled trial (RCT) protocols is essential for the interpretation of trial results and research transparency. STUDY DESIGN AND SETTING: In this study, we determined the availability of RCT protocols approved in Switzerland, Canada, Germany, and the United Kingdom in 2012. For these RCTs, we searched PubMed, Google Scholar, Scopus, and trial registries for publicly available protocols and corresponding full-text publications of results. We determined the proportion of RCTs with (1) publicly available protocols, (2) publications citing the protocol, and (3) registries providing a link to the protocol. A multivariable logistic regression model explored factors associated with protocol availability. RESULTS: Three hundred twenty-six RCTs were included, of which 118 (36.2%) made their protocol publicly available; 56 (47.6% 56 of 118) provided as a peer-reviewed publication and 48 (40.7%, 48 of 118) provided as supplementary material. A total of 90.9% (100 of 110) of the protocols were cited in the main publication, and 55.9% (66 of 118) were linked in the clinical trial registry. Larger sample size (>500; odds ratio [OR] = 5.90, 95% confidence interval [CI], 2.75-13.31) and investigator sponsorship (OR = 1.99, 95% CI, 1.11-3.59) were associated with increased protocol availability. Most protocols were made available shortly before the publication of the main results. CONCLUSION: RCT protocols should be made available at an early stage of the trial.

Reporting quality of clinical trial protocols: a repeated cross-sectional study about the Adherence to SPIrit Recommendations in Switzerland, CAnada and GErmany (ASPIRE-SCAGE).

OBJECTIVES: Comprehensive protocols are key for the planning and conduct of randomised clinical trials (RCTs). Evidence of low reporting quality of RCT protocols led to the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist in 2013. We aimed to examine the quality of reporting of RCT protocols from three countries before and after the publication of the SPIRIT checklist. DESIGN: Repeated cross sectional study. SETTING: Swiss, German and Canadian research ethics committees (RECs). PARTICIPANTS: RCT protocols approved by RECs in 2012 (n=257) and 2016 (n=292). PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcomes were the proportion of reported SPIRIT items per protocol and the proportion of trial protocols reporting individual SPIRIT items. We compared these outcomes in protocols approved in 2012 and 2016, and built regression models to explore factors associated with adherence to SPIRIT. For each protocol, we also extracted information on general trial characteristics and assessed whether individual SPIRIT items were reported RESULTS: The median proportion of reported SPIRIT items among RCT protocols showed a non-significant increase from 72% (IQR, 63%-79%) in 2012 to 77% (IQR, 68%-82%) in 2016. However, in a preplanned subgroup analysis, we detected a significant improvement in investigator-sponsored protocols: the median proportion increased from 64% (IQR, 55%-72%) in 2012 to 76% (IQR, 64%-83%) in 2016, while for industry-sponsored protocols median adherence was 77% (IQR 72%-80%) for both years. The following trial characteristics were independently associated with lower adherence to SPIRIT: single-centre trial, no support from a clinical trials unit or contract research organisation, and investigator-sponsorship. CONCLUSIONS: In 2012, industry-sponsored RCT protocols were reported more comprehensively than investigator-sponsored protocols. After publication of the SPIRIT checklist, investigator-sponsored protocols improved to the level of industry-sponsored protocols, which did not improve.

Reliability of Trial Information Across Registries for Trials With Multiple Registrations: A Systematic Review.

Importance: Clinical trial registries are important for gaining an overview of ongoing research efforts and for deterring and identifying publication bias and selective outcome reporting. The reliability of the information in trial registries is uncertain. Objective: To assess the reliability of information across registries for trials with multiple registrations. Evidence Review: For this systematic review, 360 protocols of randomized clinical trials (RCTs) approved by research ethics committees in Switzerland, the UK, Canada, and Germany in 2012 were evaluated. Clinical trial registries were searched from March to September 2019 for corresponding registrations of these RCTs. For RCTS that were recorded in more than 1 clinical trial registry, key trial characteristics that should be identical among all trial registries (ie, sponsor, funding source, primary outcome, target sample size, trial status, date of first patient enrollment, results available, and main publication indexed) were extracted in duplicate. Agreement between the different trial registries for these key characteristics was analyzed descriptively. Data analyses were conducted from May 1 to November 30, 2020. Representatives from clinical trial registries were interviewed to discuss the study findings between February 1 and March 31, 2021. Findings: The analysis included 197 RCTs registered in more than 1 trial registry (151 in 2 registries and 46 in 3 registries), with 188 trials in ClinicalTrials.gov, 185 in the European Union Drug Regulating Authorities Clinical Trials Database (EudraCT), 20 in ISRCTN, and 47 in other registries. The agreement of key information across all registries was as follows: 178 of 197 RCTs (90%; 95% CI, 85%-94%) for sponsor, 18 of 20 (90%; 95% CI, 68%-99%) for funding source (funding was not reported on ClinicalTrials.gov), 154 of 197 (78%; 95% CI, 72%-84%) for primary outcome, 90 of 197 (46%; 95% CI, 39%-53%) for trial status, 122 of 194 (63%; 95% CI, 56%-70%) for target sample size, and 43 of 57 (75%; 95% CI, 62%-86%) for the date of first patient enrollment when the comparison time was increased to 30 days (date of first patient enrollment was not reported on EudraCT). For results availability in trial registries, agreement was 122 of 197 RCTs (62%; 95% CI, 55%-69%) for summary data reported in the registry and 91 of 197 (46%; 95% CI, 39%-53%) for whether a published article with the main results was indexed. Different legal requirements were stated as the main reason for inconsistencies by representatives of clinical trial registries. Conclusions and Relevance: In this systematic review, for a substantial proportion of registered RCTs, information about key trial characteristics was inconsistent across trial registries, raising concerns about the reliability of the information provided in these registries. Further harmonization across clinical trial registries may be necessary to increase their usefulness.

Reporting quality of trial protocols improved for non-regulated interventions but not regulated interventions: A repeated cross-sectional study.

OBJECTIVES: To investigate the adherence of randomised controlled trial (RCT) protocols evaluating non-regulated interventions (including dietary interventions, surgical procedures, behavioural and lifestyle interventions, and exercise programmes) in comparison with regulated interventions to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 Statement. METHODS: We conducted a repeated cross-sectional investigation in a random sample of RCT protocols approved in 2012 (n = 257) or 2016 (n = 292) by research ethics committees in Switzerland, Germany, or Canada. We investigated the proportion of accurately reported SPIRIT checklist items in protocols of trials with non-regulated as compared to regulated interventions. RESULTS: Overall, 131 (24%) of trial protocols tested non-regulated interventions. In 2012, the median proportion of SPIRIT items reported in these protocols (59%, interquartile range [IQR], 53%-69%) was lower than in protocols with regulated interventions (median, 74%, IQR, 66%-80%). In 2016, the reporting quality of protocols with non-regulated interventions (median, 75%, IQR, 62%-83%) improved to the level of regulated intervention protocols, which had not changed on average. CONCLUSIONS: Reporting of RCT protocols evaluating non-regulated interventions improved between 2012 and 2016, although remained suboptimal. SPIRIT recommendations need to be further endorsed by researchers, ethics committees, funding agencies, and journals to optimize reporting of RCT protocols.

Design, analysis and reporting of multi-arm trials and strategies to address multiple testing.

BACKGROUND: It is unclear how multiple treatment comparisons are managed in the analysis of multi-arm trials, particularly related to reducing type I (false positive) and type II errors (false negative). METHODS: We conducted a cohort study of clinical-trial protocols that were approved by research ethics committees in the UK, Switzerland, Germany and Canada in 2012. We examined the use of multiple-testing procedures to control the overall type I error rate. We created a decision tool to determine the need for multiple-testing procedures. We compared the result of the decision tool to the analysis plan in the protocol. We also compared the pre-specified analysis plans in trial protocols to their publications. RESULTS: Sixty-four protocols for multi-arm trials were identified, of which 50 involved multiple testing. Nine of 50 trials (18%) used a single-step multiple-testing procedures such as a Bonferroni correction and 17 (38%) used an ordered sequence of primary comparisons to control the overall type I error. Based on our decision tool, 45 of 50 protocols (90%) required use of a multiple-testing procedure but only 28 of the 45 (62%) accounted for multiplicity in their analysis or provided a rationale if no multiple-testing procedure was used. We identified 32 protocol-publication pairs, of which 8 planned a global-comparison test and 20 planned a multiple-testing procedure in their trial protocol. However, four of these eight trials (50%) did not use the global-comparison test. Likewise, 3 of the 20 trials (15%) did not perform the multiple-testing procedure in the publication. The sample size of our study was small and we did not have access to statistical-analysis plans for the included trials in our study. CONCLUSIONS: Strategies to reduce type I and type II errors are inconsistently employed in multi-arm trials. Important analytical differences exist between planned analyses in clinical-trial protocols and subsequent publications, which may suggest selective reporting of analyses.

Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to SPIrit REcommendations (ASPIRE) study and associated projects.

BACKGROUND: Clearly structured and comprehensive protocols are an essential component to ensure safety of participants, data validity, successful conduct, and credibility of results of randomized clinical trials (RCTs). Funding agencies, research ethics committees (RECs), regulatory agencies, medical journals, systematic reviewers, and other stakeholders rely on protocols to appraise the conduct and reporting of RCTs. In response to evidence of poor protocol quality, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline was published in 2013 to improve the accuracy and completeness of clinical trial protocols. The impact of these recommendations on protocol completeness and associations between protocol completeness and successful RCT conduct and publication remain uncertain. OBJECTIVES AND METHODS: Aims of the Adherence to SPIrit REcommendations (ASPIRE) study are to investigate adherence to SPIRIT checklist items of RCT protocols approved by RECs in the UK, Switzerland, Germany, and Canada before (2012) and after (2016) the publication of the SPIRIT guidelines; determine protocol features associated with non-adherence to SPIRIT checklist items; and assess potential differences in adherence across countries. We assembled an international cohort of RCTs based on 450 protocols approved in 2012 and 402 protocols approved in 2016 by RECs in Switzerland, the UK, Germany, and Canada. We will extract data on RCT characteristics and adherence to SPIRIT for all included protocols. We will use multivariable regression models to investigate temporal changes in SPIRIT adherence, differences across countries, and associations between SPIRIT adherence of protocols with RCT registration, completion, and publication of results. We plan substudies to examine the registration, premature discontinuation, and non-publication of RCTs; the use of patient-reported outcomes in RCT protocols; SPIRIT adherence of RCT protocols with non-regulated interventions; the planning of RCT subgroup analyses; and the use of routinely collected data for RCTs. DISCUSSION: The ASPIRE study and associated substudies will provide important information on the impact of measures to improve the reporting of RCT protocols and on multiple aspects of RCT design, trial registration, premature discontinuation, and non-publication of RCTs observing potential changes over time.

Resource use, costs, and approval times for planning and preparing a randomized clinical trial before and after the implementation of the new Swiss human research legislation.

BACKGROUND: The preparation of a randomized controlled trial (RCT) requires substantial resources and the administrative processes can be burdensome. To facilitate the conduct of RCTs it is important to better understand cost drivers. In January 2014 the enactment of the new Swiss Legislation on Human Research (LHR) considerably changed the regulatory framework in Switzerland. We assess if the new LHR was associated with change in (i) resource use and costs to prepare an RCT, and (ii) approval times with research ethics committees (RECs) and the regulatory authority Swissmedic. METHODS: We surveyed investigators of RCTs which were approved by RECs in 2012 or in 2016 and asked for RCT preparation costs using a pre-specified item list. Additionally, we collected approval times from RECs and Swissmedic. RESULTS: The response rates of the investigator survey were 8.3% (19/228) for 2012 and 16.5% (47/285) in 2016. The median preparation cost of an RCT was USD 72,400 (interquartile range [IQR]: USD 59,500-87,700; n = 18) in 2012 and USD 72,600 (IQR: USD 42,800-169,600; n = 35) in 2016. For single centre RCTs a median REC approval time of 82 (IQR: 49-107; n = 38) days in 2012 and 92 (IQR: 65-131; n = 63) days in 2016 was observed. The median Swissmedic approval time for any clinical trial was 27 (IQR: 19-51; n = 213) days in 2012 and 49 (IQR: 36-67; n = 179) days in 2016. The total duration for achieving RCT approval from both authorities (REC and Swissmedic) in the parallel submission procedure applied in 2016 could not be assessed. CONCLUSION: Based on limited data the costs to plan and prepare RCTs in Switzerland were approximately USD 72,000 in 2012 and 2016. For effective and valid research on costs and approval times of RCTs a greater willingness to share cost information among investigators and more collaboration between stakeholders with data linkage is necessary.