Methods for efficacy evaluations of antibacterial treatments in multiple body-site infection trials.

Unmet medical need exists for serious bacterial diseases caused by multidrug-resistant infections, necessitating an urgent need for newer therapies with greater treatment benefits to patients. For meeting this need, the usual approach has been to conduct separate clinical trials, each trial targeting infection at a single body-site, e.g., for respiratory tract, intra-abdominal site, urinary tract, or blood. However, for the unmet medical need situations, this approach seems inefficient for developing antibacterial drugs with activity against single species or against multiple species of bacteria for a broader indication. Instead, a streamlined clinical development program for such situations can benefit by considering multiple body-site infection trials. Such trials would enroll patients with infections at different body-sites, but with similar severity and comorbidity for avoiding potential treatment effect heterogeneity. Such trials, when properly designed and conducted, can be informative and can save time and resources in drug development. Goals for such trials would be to first demonstrate that there is evidence of an overall treatment effect, and then to show that the treatment effects at individual body-sites reveal consistency in contributing to the overall treatment effect, or to identify a subset of body-sites for which greater treatment effect can be supported by a specified statistical decision criterion. For this, we propose here an information-based procedure for the demonstration of treatment effect overall across all body-sites, or for a subset of body-sites, on considering two types of error rates of falsely concluding treatment effect.

Consistency-ensured parametric tests for critical events of composite endpoints.

Composite endpoints (CEs) are commonly used in clinical trials when clinically important events are rare or when the disease is multifaceted. However, components of a CE often differ markedly in their clinical importance. The overall treatment effect on the composite can be driven by less-important, yet more frequently occurring, components, with no effects on some clinically important components. These situations create difficulties in interpreting the results of the CE. The literature has proposed several approaches for handling these conditions, for example, by setting requirements on the results of the clinically important components. However, for a rare event, it can be difficult to draw an appropriate conclusion about its contribution to the overall result of the composite. Here, we propose combining clinically important components to jointly increase their power and to require that their findings meet a prespecified level of evidence, called the consistency criterion. With the increase in power, the study can then be designed with the objectives of establishing efficacy for the composite and/or for the subset of clinically critical components. In this regard, we introduce multiple testing strategies, which account for the consistency requirement and for the correlation between these two endpoints. We illustrate the methodology using the PROactive trial.

Tutorial on statistical considerations on subgroup analysis in confirmatory clinical trials.

Clinical trials target patients who are expected to benefit from a new treatment under investigation. However, the magnitude of the treatment benefit, if it exists, often depends on the patient baseline characteristics. It is therefore important to investigate the consistency of the treatment effect across subgroups to ensure a proper interpretation of positive study findings in the overall population. Such assessments can provide guidance on how the treatment should be used. However, great care has to be taken when interpreting consistency results. An observed heterogeneity in treatment effect across subgroups can arise because of chance alone, whereas true heterogeneity may be difficult to detect by standard statistical tests because of their low power. This tutorial considers issues related to subgroup analyses and their impact on the interpretation of findings of completed trials that met their main objectives. In addition, we provide guidance on the design and analysis of clinical trials that account for the expected heterogeneity of treatment effects across subgroups by establishing treatment benefit in a pre-defined targeted subgroup and/or the overall population. Copyright © 2016 John Wiley & Sons, Ltd.

Validity of the Hochberg procedure revisited for clinical trial applications.

There is much interest in using the Hochberg procedure (HP) for statistical tests on primary endpoints of confirmatory clinical trials. The procedure is simple to use and enjoys more power than the Bonferroni and the Holm procedures. However, the HP is not assumption free like the other two procedures. It controls the familywise type I error rate when test statistics (used for statistical tests) are independent or if dependent satisfy a conditionally independent formulation. Otherwise, its properties for dependent tests at present are not fully understood. Consequently, its use for confirmatory trials, especially for their primary endpoints, remains worrisome. Confirmatory trials are typically designed with 1-2 primary endpoints. Therefore, a question was raised at the Food and Drug Administration as to whether the HP is a valid test for the simple case of performing treatment-to-control comparisons on two primary endpoints when their test statistics are not independent. Confirmatory trials for statistical tests normally use simple test statistics, such as the normal Z, student's t, and chi-square. The literature does include some work on the HP for dependent cases covering these test statistics, but concerns remain regarding its use for confirmatory trials for which endpoint tests are mostly of the dependent kind. The purpose of this paper is therefore to revisit this procedure and provide sufficient details for better understanding of its performance for dependent cases related to the aforementioned question. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

Statistical Perspectives on Subgroup Analysis: Testing for Heterogeneity and Evaluating Error Rate for the Complementary Subgroup.

Substantial heterogeneity in treatment effects across subgroups can cause significant findings in the overall population to be driven predominantly by those of a certain subgroup, thus raising concern on whether the treatment should be prescribed for the least benefitted subgroup. Because of its low power, a nonsignificant interaction test can lead to incorrectly prescribing treatment for the overall population. This article investigates the power of the interaction test and its implications. Also, it investigates the probability of prescribing the treatment to a nonbenefitted subgroup on the basis of a nonsignificant interaction test and other recently proposed criteria.

Hierarchical nested trial design (HNTD) for demonstrating treatment efficacy of new antibacterial drugs in patient populations with emerging bacterial resistance.

In the last decade or so, pharmaceutical drug development activities in the area of new antibacterial drugs for treating serious bacterial diseases have declined, and at the same time, there are worries that the increased prevalence of antibiotic-resistant bacterial infections, especially the increase in drug-resistant Gram-negative infections, limits available treatment options . A recent CDC report, 'Antibiotic Resistance Threats in the United States', indicates that antimicrobial resistance is one of our most serious health threats. However, recently, new ideas have been proposed to change this situation. An idea proposed in this regard is to conduct randomized clinical trials in which some patients, on the basis of a diagnostic test, may show presence of bacterial pathogens that are resistant to the control treatment, whereas remaining patients would show pathogens that are susceptible to the control. The control treatment in such trials can be the standard of care or the best available therapy approved for the disease. Patients in the control arm with resistant pathogens can have the option for rescue therapies if their clinical signs and symptoms worsen. A statistical proposal for such patient populations is to use a hierarchical noninferiority-superiority nested trial design that is informative and allows for treatment-to-control comparisons for the two subpopulations without any statistical penalty. This design can achieve in the same trial dual objectives: (i) to show that the new drug is effective for patients with susceptible pathogens on the basis of a noninferiority test and (ii) to show that it is superior to the control in patients with resistant pathogens. This paper addresses statistical considerations and methods for achieving these two objectives for this design. Published 2014. This article is a U.S. Government work and is in the public domain in the USA.

Key multiplicity issues in clinical drug development.

Much progress has been made over the past decade with the development of novel methods for addressing increasingly more complex multiplicity problems arising in confirmatory Phase III clinical trials. This includes traditional problems with a single source of multiplicity, for example, analysis of multiple endpoints or dose-placebo contrasts. In addition, more advanced problems with several sources of multiplicity have attracted attention in clinical drug development. These problems include two or more families of objectives such as multiple endpoints evaluated at multiple dose levels or in multiple patient populations. This paper provides a review of concepts that play a central role in defining and solving multiplicity problems (error rate definitions) and introduces main classes of multiple testing procedures widely used in clinical trials (nonparametric, semiparametric, and parametric procedures). The paper also presents recent advances in multiplicity research, including gatekeeping procedures for clinical trials with multiple sets of objectives. The concepts and methods introduced in the paper are illustrated using several case studies on the basis of real clinical trials. Software implementation of commonly used multiple testing and gatekeeping procedures is discussed.

Multiplicity considerations for subgroup analysis subject to consistency constraint.

A significant heterogeneity in response across subgroups of a clinical trial implies that the average response from the overall population might not characterize the treatment effect; and as noted by different regulatory guidances, can cause concerns in interpreting study findings and might lead to restricting treatment labeling. However, along with the challenges raised by the heterogeneity, recently there has been growing interest in taking advantage of the expected variability in response across subgroups to increase the chance of success of a trial by designing the trial with objectives of establishing efficacy claims for the total population and a targeted subgroup. For such trials, there have been several approaches to address the multiplicity issue with the two paths of success. This manuscript advocates the utility of setting a threshold on the treatment effect for the subgroups at the design stage to guide determination of the population labeling when significant findings for the total population have been established. Specifically, it proposes that licensing treatment for the total population requires, in addition to significant findings for this population, that the treatment effect in the least benefited (complementary) subgroup meets the treatment effect threshold at a minimum; otherwise, the treatment would be restricted to the targeted subgroup only. Setting such a threshold can be based on clinical considerations, including toxicity and adverse events, in addition to treatment effect in the subgroup. This manuscript expands some of the multiplicity approaches to account for the threshold requirement and investigates the impact of the threshold requirement on study power.

A consistency-adjusted strategy for accommodating an underpowered primary endpoint.

A clinical trial might involve more than one clinically important endpoint, each of which can characterize the treatment effect of the experimental drug under investigation. Underlying the concept of using such endpoints interchangeably to establish an efficacy claim, or pooling different endpoints to constitute a composite endpoint, is the assumption that findings from such endpoints are consistent with each other. While such an assumption about consistency of efficacy findings appears to be intuitive, it is seldom considered in the design and analysis literature of clinical trials with multiple endpoints. Failure to account for consistency of efficacy findings of two candidate endpoints to establish efficacy, at the design stage, has led to difficulties in interpreting study findings. This article presents a flexible testing strategy for accommodating findings of an alternative to the designated primary endpoint (or a subgroup) to support an efficacy claim. The method is built on the following two premises: (i) Efficacy findings of the designated primary endpoint, although nonsignificant, need to be supportive of those of the alternative endpoint, and (ii) the significance level allocated for testing the second endpoint is determined adaptively based on the magnitude of the p-value for the designated primary endpoint. The method takes into account the hierarchical ordering of the hypotheses tested and the correlation between the test statistics for the two endpoints to increase the chance of a positive trial. We discuss control of the type I error rate for the proposed test strategy and compare its power with that of other methods. In addition, we consider its application to two clinical trials.

Addressing multiplicity issues of a composite endpoint and its components in clinical trials.

Randomized controlled clinical trials often use a composite endpoint as a primary endpoint especially when treatment effects or frequency of individual components of the composite are likely to be small and combining them makes clinical sense for the disease under study. An advantage of the composite endpoint is that, as it combines multiple endpoints to a single endpoint, it reduces or eliminates the multiplicity problem of testing multiple endpoints. In addition, accumulating evidence from individual endpoints into the composite endpoint can lead to better study power and reduce the study size and the duration of the trial. However, composite endpoints can also lead to ambiguous findings and consequently cause difficulty in interpreting study results, for example, when individual component endpoints of a composite show treatment effects in different directions. Also, multiplicity issues will arise if a study sponsor seeks efficacy claims for specific components of the composite or for a targeted subgroup of patients. This paper visits some of these issues and presents some solutions through applications of multiple testing strategies.

A consistency-adjusted alpha-adaptive strategy for sequential testing.

In a clinical trial with two clinically important endpoints, each of which can fully characterize a treatment benefit to support an efficacy claim by itself, a minimum degree of consistency in the findings is expected; otherwise interpretation of study findings can be problematic. Clinical trial literature contains examples where lack of consistency in the findings of clinically relevant endpoints led to difficulties in interpreting study results. The aim of this paper is to introduce this consistency concept at the study design stage and investigate the consequences of its implementation in the statistical analysis plan. The proposed methodology allows testing of hierarchically ordered endpoints to proceed as long as a pre-specified consistency criterion is met. In addition, while an initial allocation of the alpha level is specified for the ordered endpoints at the design stage, the methodology allows the alpha level allocated to the second endpoint to be adaptive to the findings of the first endpoint. In addition, the methodology takes into account the correlation between the endpoints in calculating the significance level and the power of the test for the next endpoint. The proposed Consistency-Adjusted Alpha-Adaptive Strategy (CAAAS) is very general. Several of the well-known multiplicity adjustment approaches arise as special cases of this strategy by appropriate selection of the consistency level and the form of alpha-adaptation function. We discuss control of the Type I error rate as well as power of the proposed methodology and consider its application to clinical trial data.

Method of balanced adjustment in testing co-primary endpoints.

In a clinical trial, if there are three or more co-primary endpoints, the type II error could increase depending on the correlation among the endpoints and their treatment effect sizes. To keep the type II error under control one may have to consider larger sample sizes. However, in cases where treatment effect size of at least one of the endpoints is likely to be small, the required sample size estimates can exceed reasonable bounds. Patel (1991) proposed an approach that adjusts the significance level for testing each primary endpoint based on the idea of restricting the null space. In Chuang-Stein et al. (2007), the upward adjustment to the significance levels is based on controlling an average type I error rate. In the scenario that statistical significance of each individual hypothesis is not required, we introduce a compromise testing approach in which the significance level for a co-primary endpoint is adjusted upward only if the treatment shows high significance in one (or more than one) of the remaining co-primary endpoints. The adjustment depends on the correlation among the endpoints: larger adjustment is needed for cases of smaller correlation. The method is applicable for the scenario where the null space is restricted. Our testing approach controls maximum joint false positive rate over the restricted null space.

A flexible strategy for testing subgroups and overall population.

Subgroup analyses in addition to the total study population analysis are common in clinical trials. However, it is well recognized that findings from subgroup analyses do not provide confirmatory evidence for subgroup treatment effects without placing a priori criteria for ensuring that their findings are scientifically sound. In this paper we address some of the common pitfalls of subgroup analyses. Subgroups analyses inherently have low power for detecting treatment effects. We investigate the power interplay for a subgroup analysis and that for the total study population and list factors that impact the power of a subgroup analysis. Then we introduce a flexible statistical strategy for testing a pre-specified sequence of hypotheses for both the overall and a subgroup. The proposed method strongly controls the familywise Type I error rate and enjoys higher power than other traditional methods. This testing strategy allows testing for a subgroup once a pre-specified degree of consistency in the efficacy findings between the subgroup and the overall study population is met. In addition, it accounts for the dependency between test statistics for the subgroup and the overall study population. We discuss the power performance of this new method and provide significance levels for subgroup analysis. Finally, we illustrate its application through retrospective analysis of data from three published clinical trials.

A decision rule for evaluating several independent clinical trials collectively.

Usually, in applying for market approval of a new drug, more than one similarly designed clinical trial is conducted to support efficacy claims. How to evaluate these trials collectively and assess the overall type I error of a decision rule can be a challenging statistical issue. In this paper, we propose a decision rule to collectively evaluate p-values from several similarly designed and independently conducted trials. A concept of overall hypotheses, which is essentially union or intersection combinations of individual trials' hypotheses, is used so that the overall type I error can be controlled at desired levels. We also discuss some important properties of the approach, including the selection of the overall type I error rates and power. Examples are presented.

Efficacy endpoint selection and multiplicity adjustment methods in clinical trials with inherent multiple endpoint issues.

The ideal approach for the design and analysis of clinical trials is to select a single primary endpoint that provides a complete characterization of the disease under study and permits an efficient evaluation of the effect of a test drug. However, this is often not possible for a number of diseases or clinical trials. This paper examines some practical clinical decision-making scenarios for the selection and analysis of efficacy outcome measures in clinical trials with inherent multiplicity components.