Early stopping in clinical PET studies: How to reduce expense and exposure.

Clinical positron emission tomography (PET) research is costly and entails exposing participants to radioactivity. Researchers should therefore aim to include just the number of subjects needed to fulfill the purpose of the study. In this tutorial we show how to apply sequential Bayes Factor testing in order to stop the recruitment of subjects in a clinical PET study as soon as enough data have been collected to make a conclusion. By using simulations, we demonstrate that it is possible to stop a study early, while keeping the number of erroneous conclusions low. We then apply sequential Bayes Factor testing to a real PET data set and show that it is possible to obtain support in favor of an effect while simultaneously reducing the sample size with 30%. Using this procedure allows researchers to reduce expense and radioactivity exposure for a range of effect sizes relevant for PET research.

COVID-19 hits a trial: Arguments against hastily deviating from the plan.

The COVID-19 pandemic has substantially impacted the conduct of clinical trials. While initially preparing for a period of time, where it would likely be impossible to supervise trials in the usual way and precautionary measures had to be implemented to care for medication supply and general safety of study participants it is now important to consider, how the impact of the pandemic on trial outcome can be assessed, which measures are needed to decide, how to proceed with the trial and what is needed to compensate to irregularity introduced by the pandemic situation. Obviously not all trials will suffer to the same degree: some trials may be close to finalizing recruitment, others may not yet have started. Similarly not all clinical trials investigate vulnerable patient populations, but some will and may in addition have recruited to an extent that beneficial effects achieved in the initial phase of the trial may be outweighed by an increase e.g. in mortality that impacts both treatment groups. The situation is further complicated by the fact that the pandemic reached different countries in the world and even cities in one country at different points in time with different severity. Our example is a randomized and double-blind clinical trial comparing digitoxin and placebo in patients with advanced chronic heart failure. This trial has recruited roughly 1/3 of the overall 2200 patients when the disease outbreak reached Germany. We discuss how simulations and theoretical considerations can be used to address questions about the need to increase the overall sample-size to be recruited to compensate for a potential shrinkage of the treatment effect caused by the COVID-19 pandemic and what role the degree of consistency could play when comparing pre-, during- and post- COVID-19 periods of trial conduct regarding the question, whether the treatment effect can be considered consistent and with this generalizable. This is dependent on the size of the treatment effect and the impact of the pandemic. We argue, that in case of doubt, it may be wise to proceed with the original study plan.

Rates of Discontinuation and Nonpublication of Head and Neck Cancer Randomized Clinical Trials.

Importance: Randomized clinical trials (RCTs) play an important role in clinical decision-making, and discontinuation or nonpublication of these trials are causes of great concern. The extent of discontinued or unpublished RCTs about head and neck cancer remains unclear. Objective: To assess the rate of discontinuation or nonpublication of RCTs involving patients with head and neck cancer. This objective was measured by observing 3 domains: discontinuation of trial, nonpublication of trial data, and feasibility of contacting trial investigators of aforementioned trials. Evidence Review: For this study, the sample was derived using the ClinicalTrials.gov advanced search feature on March 18, 2019, to locate completed and discontinued RCTs pertaining to head and neck cancer registered before this date. Trials were analyzed to identify reasons for trial discontinuation and publication status of each trial. If publication status or reason for trial discontinuation was not allocated through the systematic search of ClinicalTrials.gov, the corresponding author was emailed to determine publication status. Findings: After exclusions, 130 RCTs were included. Of these trials, 92 (70.8%) were completed and 38 (29.2%) were discontinued for various reasons. The most common reason for discontinuation of trials was committee recommendations. Of the 130 analyzed trials, 67 (51.5%) were published in a peer-reviewed journal and 63 (48.5%) were unpublished trials. Of the 92 completed trials, 55 (59.8%) were published and 37 (40.2%) remained unpublished 3 or more years after trial completion. Trials funded by other sources (private, nonprofit, or the National Institutes of Health) were more likely to reach publication than industry-funded RCTs (unadjusted odds ratio, 4.3 [95% CI, 1.3-14.0]; adjusted odds ratio, 4.1 [95% CI, 1.2-14.3]). Conclusions and Relevance: Of RCTs in head and neck cancer, 29.2% were discontinued and 40.2% completed trials never reached publication. The findings suggest that needs exist for RCT guidance of head and neck cancer. The reporting of reasons for trial discontinuation appears to be lacking, and trial publication rates were low. This study is relevant to many physicians and researchers because it identifies potential sources of decreased research productivity and ethics.

Discontinuation of surgical versus nonsurgical clinical trials: an analysis of 88,498 trials.

BACKGROUND: It has been previously reported that over 20% of surgical trials will be discontinued prematurely raising ethical and financial concerns. Previous studies have been limited in scope owing to the need for manual review of selected trials. To date, there has been no broad analysis comparing surgical and nonsurgical registered clinical trials. MATERIALS AND METHODS: ClinicalTrials.gov was queried October 7, 2017 for all US trials from 2005 to 2017. Trials were assigned to surgical or nonsurgical groups by automated sorting. The sorting algorithm was validated by comparison with manual assignments made by blinded investigators. Comparisons were made between trial status, funding sources, and trial design. The reasons for discontinuation were examined and tabulated. RESULTS: The database search yielded 82,719 nonsurgical and 5779 surgical trials after automatic assignment. The algorithm for assignments had an overall accuracy of 87.99% and a positive likelihood ratio of 6.09 and negative likelihood ratio of 0.093. Significant differences existed in trial status (nonsurgical versus surgical: completed: 55.51% versus 39.49%, P < 0.001 and discontinued: 11.07% versus 15.97%, P < 0.001). Discontinuation due to poor recruitment was more commonly cited by surgical trials (44.65% versus 34.74% P < 0.001). Industry funding predicted discontinuation for all trials (odds ratio 1.63 P < 0.001) and surgical trials independently (OR 1.25 P = 0.041). Patient enrollment, reporting results, and NIH funding were all protective against discontinuation. CONCLUSIONS: Surgical trials are more likely to prematurely discontinue than nonsurgical trials. Industry funding independently predicts trial discontinuation. Poor recruitment is a major cause of early trial discontinuation for all trials and is more pronounced in surgical trials.

Robust inference for group sequential trials.

For ethical reasons, group sequential trials were introduced to allow trials to stop early in the event of extreme results. Endpoints in such trials are usually mortality or irreversible morbidity. For a given endpoint, the norm is to use a single test statistic and to use that same statistic for each analysis. This approach is risky because the test statistic has to be specified before the study is unblinded, and there is loss in power if the assumptions that ensure optimality for each analysis are not met. To minimize the risk of moderate to substantial loss in power due to a suboptimal choice of a statistic, a robust method was developed for nonsequential trials. The concept is analogous to diversification of financial investments to minimize risk. The method is based on combining P values from multiple test statistics for formal inference while controlling the type I error rate at its designated value.This article evaluates the performance of 2 P value combining methods for group sequential trials. The emphasis is on time to event trials although results from less complex trials are also included. The gain or loss in power with the combination method relative to a single statistic is asymmetric in its favor. Depending on the power of each individual test, the combination method can give more power than any single test or give power that is closer to the test with the most power. The versatility of the method is that it can combine P values from different test statistics for analysis at different times. The robustness of results suggests that inference from group sequential trials can be strengthened with the use of combined tests.

Initiation and continuation of randomized trials after the publication of a trial stopped early for benefit asking the same study question: STOPIT-3 study design.

BACKGROUND: Randomized control trials (RCTs) stopped early for benefit (truncated RCTs) are increasingly common and, on average, overestimate the relative magnitude of benefit by approximately 30%. Investigators stop trials early when they consider it is no longer ethical to enroll patients in a control group. The goal of this systematic review is to determine how investigators of ongoing or planned RCTs respond to the publication of a truncated RCT addressing a similar question. METHODS/DESIGN: We will conduct systematic reviews to update the searches of 210 truncated RCTs to identify similar trials ongoing at the time of publication, or started subsequently, to the truncated trials ('subsequent RCTs'). Reviewers will determine in duplicate the similarity between the truncated and subsequent trials. We will analyze the epidemiology, distribution, and predictors of subsequent RCTs. We will also contact authors of subsequent trials to determine reasons for beginning, continuing, or prematurely discontinuing their own trials, and the extent to which they rely on the estimates from truncated trials. DISCUSSION: To the extent that investigators begin or continue subsequent trials they implicitly disagree with the decision to stop the truncated RCT because of an ethical mandate to administer the experimental treatment. The results of this study will help guide future decisions about when to stop RCTs early for benefit.

The dangers of stopping a trial too early.

To ensure that participants in randomized controlled trials are protected from harm, interim analyses and review of results by an independent data monitoring committee have become standard practice. If an analysis of accumulating data partway through a trial reveals an unanticipated degree of benefit or toxicity, or differences in outcomes between the intervention and control groups are so unimpressive that any prospect of a positive result with the planned sample size is extremely unlikely, investigators may stop the trial earlier than originally scheduled. The practice of stopping randomized controlled trials early is, however, problematic, especially if the trial is stopped for apparent benefit. Concerns in trials stopped early for apparent benefit include appropriate interpretation of results and ethical problems concerning trial participants, clinicians, and society as a whole. In this article, we review the epidemiology of trials stopped early and illustrate some of the problems and controversies associated with stopping randomized controlled trials early for apparent benefit. Finally, we offer guidance for clinicians, those running clinical trials, and authors of systematic reviews.

Clinical research involving children: registration, completeness, and publication.

BACKGROUND AND OBJECTIVE: Effective health care for children must be based on thorough analyses of the best research evidence. The objective of this study was to examine registration, completeness, and publication of studies involving children. METHODS: We searched the ClinicalTrials.gov registry to identify all closed studies involving children and examined them for completeness and availability of results. We examined publication in peer-reviewed journals for 160 randomly selected National Institutes of Health (NIH)-funded studies from 2000 through 2010 and for 758 randomly selected completed studies. RESULTS: Of 3428 closed studies involving children identified in ClinicalTrials.gov, 2385 (70%) were completed, 28 (0.8%) suspended, 152 (4.4%) terminated, and 38 (1.1%) withdrawn. The proportion of non-completed studies (terminated and suspended) increased linearly by 186% between 2001 and 2009, from 1.9% to 8.4%. Of the 152 terminated studies, 48 did not report reasons for termination, 21 cited safety concerns, and 83 cited poor recruitment or other administrative reasons. Only 29% of completed studies were published. Publication that did occur was an average of 2 years after study completion. Completed interventional studies were published more often than observational studies. Completed industry-funded studies were published less often than studies funded by the NIH. Registered NIH-funded trials were published more often than unregistered. CONCLUSIONS: Results are unavailable for more than half of the studies involving children, revealing a substantial publication bias. Registration and posting of results on ClinicalTrials.gov should be mandatory for all studies involving children.

Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Part 1: Ethical considerations.

The protection of patient safety is the principal responsibility of clinical trial investigators, and must be assured even if that were to prevent successful completion of a trial. Yet, the decision to prematurely stop a blinded, randomized controlled clinical trial can be extremely complicated, involving a tangle of ethical, statistical, and practical issues. Questions are quickly answered when conclusive evidence of harm has been established for trial participants, or when the potential for harm exceeds an acceptable limit of comfort for an oversight body. Less readily addressed are those situations in which early alarms warn of possible harm, but the data are too preliminary or incomplete to reach a satisfactory decision as to whether or not to stop the study. Early study termination without sufficient evidence disallows the study question from being answered and may allow an inferior treatment to remain in use, or prevent a superior one from being discovered. Even without early stopping, as a study proceeds, worrisome trends may lead to overzealous (or overly cautious) looks at study data which could jeopardize the integrity of the findings. Trial investigators and safety monitoring groups, aided by objective statistical rules and thoughtful deliberations, share responsibility for patient welfare. Statistical guidelines must not frustrate ethical concerns, but, rather, should be designed to promote the highest ethical and scientific outcomes possible, safeguarding both trial participants and the public - the ultimate beneficiaries of clinical trials.