COVID-19 hits a trial: Arguments against hastily deviating from the plan.

The COVID-19 pandemic has substantially impacted the conduct of clinical trials. While initially preparing for a period of time, where it would likely be impossible to supervise trials in the usual way and precautionary measures had to be implemented to care for medication supply and general safety of study participants it is now important to consider, how the impact of the pandemic on trial outcome can be assessed, which measures are needed to decide, how to proceed with the trial and what is needed to compensate to irregularity introduced by the pandemic situation. Obviously not all trials will suffer to the same degree: some trials may be close to finalizing recruitment, others may not yet have started. Similarly not all clinical trials investigate vulnerable patient populations, but some will and may in addition have recruited to an extent that beneficial effects achieved in the initial phase of the trial may be outweighed by an increase e.g. in mortality that impacts both treatment groups. The situation is further complicated by the fact that the pandemic reached different countries in the world and even cities in one country at different points in time with different severity. Our example is a randomized and double-blind clinical trial comparing digitoxin and placebo in patients with advanced chronic heart failure. This trial has recruited roughly 1/3 of the overall 2200 patients when the disease outbreak reached Germany. We discuss how simulations and theoretical considerations can be used to address questions about the need to increase the overall sample-size to be recruited to compensate for a potential shrinkage of the treatment effect caused by the COVID-19 pandemic and what role the degree of consistency could play when comparing pre-, during- and post- COVID-19 periods of trial conduct regarding the question, whether the treatment effect can be considered consistent and with this generalizable. This is dependent on the size of the treatment effect and the impact of the pandemic. We argue, that in case of doubt, it may be wise to proceed with the original study plan.

Reporting on data monitoring committees in neonatal randomised controlled trials is inconsistent.

AIM: To evaluate the reported use of data monitoring committees (DMCs), the frequency of interim analysis, prespecified stopping rules and early trial termination in neonatal randomised controlled trials (RCTs). METHODS: We reviewed neonatal RCTs published in four high-impact general medical journals, specifically looking at safety issues including documented involvement of a DMC, stated interim analysis, stopping rules and early trial termination. We searched all journal issues over an 11-year period (2003-2013) and recorded predefined parameters on each item for RCTs meeting inclusion criteria. RESULTS: Seventy neonatal trials were identified in four general medical journals: Lancet, New England Journal of Medicine (NEJM), British Medical Journal and Journal of American Medical Association. A total of 43 (61.4%) studies reported the presence of a DMC, 36 (51.4%) explicitly mentioned interim analysis, stopping rules were reported in 15 (21.4%) RCTs and seven (10%) trials were terminated early. The NEJM most frequently reported these parameters compared to the other three journals reviewed. CONCLUSION: While the majority of neonatal RCTs report on DMC involvement and interim analysis, there is still scope for improvement. Clear documentation of safety-related issues should be a central component of reporting in neonatal trials involving newborn infants.

Stopping guidelines for an effectiveness trial: what should the protocol specify?

BACKGROUND: Despite long-standing problems in decisions to stop clinical trials, stopping guidelines are often vague or unspecified in the trial protocol. Clear, well-conceived guidelines are especially important to assist the data monitoring committees for effectiveness trials. MAIN TEXT: To specify better stopping guidelines in the protocol for such trials, the clinical investigators and trial statistician should carefully consider the following kinds of questions: 1. How should the relative importance of the treatment benefits and hazards be assessed? 2. For decisions to stop a trial for benefit: (a) What would be the minimum clinically important difference for the study population? (b) How should the probability that the benefit exceeds that difference be assessed? (c) When should the interim analyses include data from other trials? (d) Would the evidence meet state-of-the-art standards for treatment recommendations and practice guidelines? 3. Should less evidence be required to stop the trial for harm than for benefit? 4. When should conventional stopping guidelines for futility be used for comparative effectiveness trials? CONCLUSION: Both clinical and statistical expertise are required to address such challenging questions for effectiveness trials. Their joint consideration by clinical investigators and statisticians is needed to define better stopping guidelines before starting the trial.

An analysis of protocols and publications suggested that most discontinuations of clinical trials were not based on preplanned interim analyses or stopping rules.

OBJECTIVES: To investigate the frequency of interim analyses, stopping rules, and data safety and monitoring boards (DSMBs) in protocols of randomized controlled trials (RCTs); to examine these features across different reasons for trial discontinuation; and to identify discrepancies in reporting between protocols and publications. STUDY DESIGN AND SETTING: We used data from a cohort of RCT protocols approved between 2000 and 2003 by six research ethics committees in Switzerland, Germany, and Canada. RESULTS: Of 894 RCT protocols, 289 prespecified interim analyses (32.3%), 153 stopping rules (17.1%), and 257 DSMBs (28.7%). Overall, 249 of 894 RCTs (27.9%) were prematurely discontinued; mostly due to reasons such as poor recruitment, administrative reasons, or unexpected harm. Forty-six of 249 RCTs (18.4%) were discontinued due to early benefit or futility; of those, 37 (80.4%) were stopped outside a formal interim analysis or stopping rule. Of 515 published RCTs, there were discrepancies between protocols and publications for interim analyses (21.1%), stopping rules (14.4%), and DSMBs (19.6%). CONCLUSION: Two-thirds of RCT protocols did not consider interim analyses, stopping rules, or DSMBs. Most RCTs discontinued for early benefit or futility were stopped without a prespecified mechanism. When assessing trial manuscripts, journals should require access to the protocol.

A practical design for a dual-agent dose-escalation trial that incorporates pharmacokinetic data.

Traditionally, model-based dose-escalation trial designs recommend a dose for escalation based on an assumed dose-toxicity relationship. Pharmacokinetic data are often available but are currently only utilised by clinical teams in a subjective manner to aid decision making if the dose-toxicity model recommendation is felt to be too high. Formal incorporation of pharmacokinetic data in dose-escalation could therefore make the decision process more efficient and lead to an increase in the precision of the resulting recommended dose, as well as decreasing the subjectivity of its use. Such an approach is investigated in the dual-agent setting using a Bayesian design, where historical single-agent data are available to advise the use of pharmacokinetic data in the dual-agent setting. The dose-toxicity and dose-exposure relationships are modelled independently and the outputs combined in the escalation rules. Implementation of stopping rules highlight the practicality of the design. This is demonstrated through an example which is evaluated using simulation.

Bayesian sequential monitoring design for two-arm randomized clinical trials with noncompliance.

In early-phase clinical trials, interim monitoring is commonly conducted based on the estimated intent-to-treat effect, which is subject to bias in the presence of noncompliance. To address this issue, we propose a Bayesian sequential monitoring trial design based on the estimation of the causal effect using a principal stratification approach. The proposed design simultaneously considers efficacy and toxicity outcomes and utilizes covariates to predict a patient's potential compliance behavior and identify the causal effects. Based on accumulating data, we continuously update the posterior estimates of the causal treatment effects and adaptively make the go/no-go decision for the trial. Numerical results show that the proposed method has desirable operating characteristics and addresses the issue of noncompliance.

[GRADE guidelines: 4. Rating the quality of evidence - limitations of clinical trials (risk of bias)]. / GRADE-Leitlinien: 4. Bewertung der Qualität der Evidenz - Studienlimitationen (Risiko für Bias).

In the GRADE approach, randomised trials start as high-quality evidence and observational studies as low-quality evidence, but both can be rated down if most of the relevant evidence comes from studies that suffer from a high risk of bias. Well-established limitations of randomised trials include failure to conceal allocation, failure to blind, loss to follow-up, and failure to appropriately consider the intention-to-treat principle. More recently, recognised limitations include stopping early for apparent benefit and selective reporting of outcomes according to the results. Key limitations of observational studies include use of inappropriate controls and failure to adequately adjust for prognostic imbalance. Risk of bias may vary across outcomes (e.g., loss to follow-up may be far less for all-cause mortality than for quality of life), a consideration that many systematic reviews ignore. In deciding whether to rate down for risk of bias - whether for randomised trials or observational studies-authors should not take an approach that averages across studies. Rather, for any individual outcome, when there are some studies with a high risk, and some with a low risk of bias, they should consider including only the studies with a lower risk of bias.

Early stopping rules in oncology: considerations for clinicians.

The number of cancer-related clinical trials has been rapidly increasing over the past decade. Along with this increase, oncology studies stopped early for benefit or harm have also been more common. Clinicians treating cancer patients often are faced with the challenge of having to decide whether or not to incorporate information from these new studies into their daily clinical practice. This review article explains the role of the Data and Safety Monitoring Committee in stopping trials early; provides examples of oncology trials stopped early; and reviews some of the controversies and statistical concepts associated with early stopping rules. In addition, a simple and practical approach to interpreting the findings of trials that are stopped early is provided to assist clinicians in deciding how to incorporate information from these studies into their daily practice.

Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Part 2: Statistical considerations.

If the primary objective of a trial is to learn about the ability of a new treatment to help future patients without sacrificing the safe and effective treatment of the current patients, then a Bayesian design with frequent assessments of the accumulating data should be considered. Unfortunately, Bayesian analyses typically do not have standard approaches, and because of the subjectivity of prior probabilities and the possibility for introducing bias, statisticians have developed other methods for statistical inference that only depend on deductive probabilities. However, these frequentist probabilities are just theories about how certain relative frequencies will develop over time. They have no real meaning in a single experiment. Designed to work well in the long run, p-values become hard to explain for individual experiments. Fortunately, the controversy surrounding Bayes' theorem comes, not from the representation of evidence, but from the use of probabilities to measure belief. A prior distribution is not necessary. The likelihood function contains all of the information in a trial relevant for making inferences about the parameters. Monitoring clinical trials is a dynamic process which requires flexibility to respond to unforeseen developments. Likelihood ratios allow the data to speak for themselves, without regard for the probability of observing weak or misleading evidence, and decisions to stop, or continue, a trial can be made at any time, with all of the available information. A likelihood based method is needed.

Data monitoring committees, interim analysis and early termination in paediatric trials.

AIM: To evaluate whether paediatric randomized clinical trials (RCTs) adopt recent guidance on Data Monitoring Committees (DMCs), interim analysis and early termination. METHODS: We reviewed paediatric RCTs that reported on DMCs, interim analysis or early termination, published in eight general medical and paediatric journals (2005-2007). We searched full-text databases for eligible trials and recorded predefined parameters on each item. Reported activities were compared with current scientific guidance. RESULTS: A total of 110 of 648 paediatric trials (17%) reported on DMC, interim analysis or early stopping. Various approaches for convening a DMC were identified; information on DMC composition and independence was limited. Strict predefined statistical stopping 'rules' were reported in 10 of 23 trials, and interim analyses were more frequently performed on efficacy than on safety outcomes (39/45 vs 27/45). No adjustment for repeated testing was reported in 11 of 33 trials reporting monitoring methods and in 7 of 17 early terminated trials. Validity of results from early stopped trials was threatened by small sample sizes. Incomplete reporting hampered a full analysis. CONCLUSION: Few paediatric trials report on DMCs' roles, interim analysis or early stopping. Heterogeneous practices and apparent shortcomings jeopardize the validity of trial results. Easily accessible guidelines for the design, conduct and reporting of paediatric DMCs are needed.