Methods for efficacy evaluations of antibacterial treatments in multiple body-site infection trials.

Unmet medical need exists for serious bacterial diseases caused by multidrug-resistant infections, necessitating an urgent need for newer therapies with greater treatment benefits to patients. For meeting this need, the usual approach has been to conduct separate clinical trials, each trial targeting infection at a single body-site, e.g., for respiratory tract, intra-abdominal site, urinary tract, or blood. However, for the unmet medical need situations, this approach seems inefficient for developing antibacterial drugs with activity against single species or against multiple species of bacteria for a broader indication. Instead, a streamlined clinical development program for such situations can benefit by considering multiple body-site infection trials. Such trials would enroll patients with infections at different body-sites, but with similar severity and comorbidity for avoiding potential treatment effect heterogeneity. Such trials, when properly designed and conducted, can be informative and can save time and resources in drug development. Goals for such trials would be to first demonstrate that there is evidence of an overall treatment effect, and then to show that the treatment effects at individual body-sites reveal consistency in contributing to the overall treatment effect, or to identify a subset of body-sites for which greater treatment effect can be supported by a specified statistical decision criterion. For this, we propose here an information-based procedure for the demonstration of treatment effect overall across all body-sites, or for a subset of body-sites, on considering two types of error rates of falsely concluding treatment effect.

Consistency ensured test strategies for supportive evidence of treatment efficacy in noninferiority clinical trials.

Noninferiority (NI) clinical trials are designed to demonstrate that a new treatment is not unacceptably worse than an active control on a clinically meaningful endpoint. While such an endpoint can be of any type, the focus of this manuscript is on the binary-type endpoint. Examples of this endpoint can be clinical cure endpoint for patients with bacterial diseases or based on a pre-specified virological threshold for viral diseases. However, in addition to assessing such a binary endpoint for the NI comparison, the trial may also evaluate a second clinically relevant endpoint for providing additional support to the evidence of the designated primary endpoint. Specifically, if the trial is successful in demonstrating statistical significance on the first endpoint, then observing at least a positive trend in efficacy on the second endpoint may provide additional supportive evidence of efficacy. The second endpoint can be a time-to-event type endpoint, such as time-to-symptom resolution (TSR) or time to all-cause mortality for infectious disease trials, time-to-wound closure for wound healing trials, or other endpoints. We propose two consistency ensured test strategies for the two hypotheses of a trial, one associated with the binary endpoint and the other with the second endpoint, both with the objective of drawing inference regarding the efficacy of the new treatment based on findings from testing the two hypotheses. A key feature of these test strategies is that basically it does not require multiplicity adjustment of the significance levels. We conclude with general discussion of the testing methods and possible applications to unmet medical need trials.

Consistency-ensured parametric tests for critical events of composite endpoints.

Composite endpoints (CEs) are commonly used in clinical trials when clinically important events are rare or when the disease is multifaceted. However, components of a CE often differ markedly in their clinical importance. The overall treatment effect on the composite can be driven by less-important, yet more frequently occurring, components, with no effects on some clinically important components. These situations create difficulties in interpreting the results of the CE. The literature has proposed several approaches for handling these conditions, for example, by setting requirements on the results of the clinically important components. However, for a rare event, it can be difficult to draw an appropriate conclusion about its contribution to the overall result of the composite. Here, we propose combining clinically important components to jointly increase their power and to require that their findings meet a prespecified level of evidence, called the consistency criterion. With the increase in power, the study can then be designed with the objectives of establishing efficacy for the composite and/or for the subset of clinically critical components. In this regard, we introduce multiple testing strategies, which account for the consistency requirement and for the correlation between these two endpoints. We illustrate the methodology using the PROactive trial.

Tutorial on statistical considerations on subgroup analysis in confirmatory clinical trials.

Clinical trials target patients who are expected to benefit from a new treatment under investigation. However, the magnitude of the treatment benefit, if it exists, often depends on the patient baseline characteristics. It is therefore important to investigate the consistency of the treatment effect across subgroups to ensure a proper interpretation of positive study findings in the overall population. Such assessments can provide guidance on how the treatment should be used. However, great care has to be taken when interpreting consistency results. An observed heterogeneity in treatment effect across subgroups can arise because of chance alone, whereas true heterogeneity may be difficult to detect by standard statistical tests because of their low power. This tutorial considers issues related to subgroup analyses and their impact on the interpretation of findings of completed trials that met their main objectives. In addition, we provide guidance on the design and analysis of clinical trials that account for the expected heterogeneity of treatment effects across subgroups by establishing treatment benefit in a pre-defined targeted subgroup and/or the overall population. Copyright © 2016 John Wiley & Sons, Ltd.

Statistical Perspectives on Subgroup Analysis: Testing for Heterogeneity and Evaluating Error Rate for the Complementary Subgroup.

Substantial heterogeneity in treatment effects across subgroups can cause significant findings in the overall population to be driven predominantly by those of a certain subgroup, thus raising concern on whether the treatment should be prescribed for the least benefitted subgroup. Because of its low power, a nonsignificant interaction test can lead to incorrectly prescribing treatment for the overall population. This article investigates the power of the interaction test and its implications. Also, it investigates the probability of prescribing the treatment to a nonbenefitted subgroup on the basis of a nonsignificant interaction test and other recently proposed criteria.

Advanced multiplicity adjustment methods in clinical trials.

During the last decade, many novel approaches for addressing multiplicity problems arising in clinical trials have been introduced in the literature. These approaches provide great flexibility in addressing given clinical trial objectives and yet maintain strong control of the familywise error rate. In this tutorial article, we review multiple testing strategies that are related to the following: (a) recycling local significance levels to test hierarchically ordered hypotheses; (b) adapting the significance level for testing a hypothesis to the findings of testing previous hypotheses within a given test sequence, also in view of certain consistency requirements; (c) grouping hypotheses into hierarchical families of hypotheses along with recycling the significance level between those families; and (d) graphical methods that permit repeated recycling of the significance level. These four different methodologies are related to each other, and we point out some connections as we describe and illustrate them. By contrasting the main features of these approaches, our objective is to help practicing statisticians to select an appropriate method for their applications. In this regard, we discuss how to apply some of these strategies to clinical trial settings and provide algorithms to calculate critical values and adjusted p-values for their use in practice. The methods are illustrated with several numerical examples.

Multiplicity considerations for subgroup analysis subject to consistency constraint.

A significant heterogeneity in response across subgroups of a clinical trial implies that the average response from the overall population might not characterize the treatment effect; and as noted by different regulatory guidances, can cause concerns in interpreting study findings and might lead to restricting treatment labeling. However, along with the challenges raised by the heterogeneity, recently there has been growing interest in taking advantage of the expected variability in response across subgroups to increase the chance of success of a trial by designing the trial with objectives of establishing efficacy claims for the total population and a targeted subgroup. For such trials, there have been several approaches to address the multiplicity issue with the two paths of success. This manuscript advocates the utility of setting a threshold on the treatment effect for the subgroups at the design stage to guide determination of the population labeling when significant findings for the total population have been established. Specifically, it proposes that licensing treatment for the total population requires, in addition to significant findings for this population, that the treatment effect in the least benefited (complementary) subgroup meets the treatment effect threshold at a minimum; otherwise, the treatment would be restricted to the targeted subgroup only. Setting such a threshold can be based on clinical considerations, including toxicity and adverse events, in addition to treatment effect in the subgroup. This manuscript expands some of the multiplicity approaches to account for the threshold requirement and investigates the impact of the threshold requirement on study power.

A consistency-adjusted strategy for accommodating an underpowered primary endpoint.

A clinical trial might involve more than one clinically important endpoint, each of which can characterize the treatment effect of the experimental drug under investigation. Underlying the concept of using such endpoints interchangeably to establish an efficacy claim, or pooling different endpoints to constitute a composite endpoint, is the assumption that findings from such endpoints are consistent with each other. While such an assumption about consistency of efficacy findings appears to be intuitive, it is seldom considered in the design and analysis literature of clinical trials with multiple endpoints. Failure to account for consistency of efficacy findings of two candidate endpoints to establish efficacy, at the design stage, has led to difficulties in interpreting study findings. This article presents a flexible testing strategy for accommodating findings of an alternative to the designated primary endpoint (or a subgroup) to support an efficacy claim. The method is built on the following two premises: (i) Efficacy findings of the designated primary endpoint, although nonsignificant, need to be supportive of those of the alternative endpoint, and (ii) the significance level allocated for testing the second endpoint is determined adaptively based on the magnitude of the p-value for the designated primary endpoint. The method takes into account the hierarchical ordering of the hypotheses tested and the correlation between the test statistics for the two endpoints to increase the chance of a positive trial. We discuss control of the type I error rate for the proposed test strategy and compare its power with that of other methods. In addition, we consider its application to two clinical trials.

Addressing multiplicity issues of a composite endpoint and its components in clinical trials.

Randomized controlled clinical trials often use a composite endpoint as a primary endpoint especially when treatment effects or frequency of individual components of the composite are likely to be small and combining them makes clinical sense for the disease under study. An advantage of the composite endpoint is that, as it combines multiple endpoints to a single endpoint, it reduces or eliminates the multiplicity problem of testing multiple endpoints. In addition, accumulating evidence from individual endpoints into the composite endpoint can lead to better study power and reduce the study size and the duration of the trial. However, composite endpoints can also lead to ambiguous findings and consequently cause difficulty in interpreting study results, for example, when individual component endpoints of a composite show treatment effects in different directions. Also, multiplicity issues will arise if a study sponsor seeks efficacy claims for specific components of the composite or for a targeted subgroup of patients. This paper visits some of these issues and presents some solutions through applications of multiple testing strategies.

A consistency-adjusted alpha-adaptive strategy for sequential testing.

In a clinical trial with two clinically important endpoints, each of which can fully characterize a treatment benefit to support an efficacy claim by itself, a minimum degree of consistency in the findings is expected; otherwise interpretation of study findings can be problematic. Clinical trial literature contains examples where lack of consistency in the findings of clinically relevant endpoints led to difficulties in interpreting study results. The aim of this paper is to introduce this consistency concept at the study design stage and investigate the consequences of its implementation in the statistical analysis plan. The proposed methodology allows testing of hierarchically ordered endpoints to proceed as long as a pre-specified consistency criterion is met. In addition, while an initial allocation of the alpha level is specified for the ordered endpoints at the design stage, the methodology allows the alpha level allocated to the second endpoint to be adaptive to the findings of the first endpoint. In addition, the methodology takes into account the correlation between the endpoints in calculating the significance level and the power of the test for the next endpoint. The proposed Consistency-Adjusted Alpha-Adaptive Strategy (CAAAS) is very general. Several of the well-known multiplicity adjustment approaches arise as special cases of this strategy by appropriate selection of the consistency level and the form of alpha-adaptation function. We discuss control of the Type I error rate as well as power of the proposed methodology and consider its application to clinical trial data.

Modeling longitudinal count data with dropouts.

This paper explores the utility of different approaches for modeling longitudinal count data with dropouts arising from a clinical study for the treatment of actinic keratosis lesions on the face and balding scalp. A feature of these data is that as the disease for subjects on the active arm improves their data show larger dispersion compared with those on the vehicle, exhibiting an over-dispersion relative to the Poisson distribution. After fitting the marginal (or population averaged) model using the generalized estimating equation (GEE), we note that inferences from such a model might be biased as dropouts are treatment related. Then, we consider using a weighted GEE (WGEE) where each subject's contribution to the analysis is weighted inversely by the subject's probability of dropout. Based on the model findings, we argue that the WGEE might not address the concerns about the impact of dropouts on the efficacy findings when dropouts are treatment related. As an alternative, we consider likelihood-based inference where random effects are added to the model to allow for heterogeneity across subjects. Finally, we consider a transition model where, unlike the previous approaches that model the log-link function of the mean response, we model the subject's actual lesion counts. This model is an extension of the Poisson autoregressive model of order 1, where the autoregressive parameter is taken to be a function of treatment as well as other covariates to induce different dispersions and correlations for the two treatment arms. We conclude with a discussion about model selection.

Clinical trials and statistical analyses: what should dermatologists look for in a report?

Clinicians need to evaluate the quality of individual clinical studies and synthesize the information from multiple clinical studies to provide insights in selecting appropriate therapies for patients. Understanding the key statistical principles that underlie a clinical trial and how they may be implemented can help clinicians properly interpret the efficacy and safety findings of clinical trials. Several factors should be considered when evaluating clinical studies reported in the literature, as important differences might exist among reported studies, thereby impacting the reliability of their findings. Studies vary in terms of study design, conduct, analysis, and presentation of findings. The key features to consider when evaluating clinical trials are inferential intent (exploratory versus confirmatory), choice of control group, randomization, extent of blinding, prespecification of analyses, appropriate handling of missing data, and multiple end points. Making comparisons across studies is extremely difficult and rarely statistically justified. However, this article will point out issues to keep in mind when evaluating multiple studies, such as variations in design and study populations.

A flexible strategy for testing subgroups and overall population.

Subgroup analyses in addition to the total study population analysis are common in clinical trials. However, it is well recognized that findings from subgroup analyses do not provide confirmatory evidence for subgroup treatment effects without placing a priori criteria for ensuring that their findings are scientifically sound. In this paper we address some of the common pitfalls of subgroup analyses. Subgroups analyses inherently have low power for detecting treatment effects. We investigate the power interplay for a subgroup analysis and that for the total study population and list factors that impact the power of a subgroup analysis. Then we introduce a flexible statistical strategy for testing a pre-specified sequence of hypotheses for both the overall and a subgroup. The proposed method strongly controls the familywise Type I error rate and enjoys higher power than other traditional methods. This testing strategy allows testing for a subgroup once a pre-specified degree of consistency in the efficacy findings between the subgroup and the overall study population is met. In addition, it accounts for the dependency between test statistics for the subgroup and the overall study population. We discuss the power performance of this new method and provide significance levels for subgroup analysis. Finally, we illustrate its application through retrospective analysis of data from three published clinical trials.

The impact of missing data in a generalized integer-valued autoregression model for count data.

The impact of the missing data mechanism on estimates of model parameters for continuous data has been extensively investigated in the literature. In comparison, minimal research has been carried out for the impact of missing count data. The focus of this article is to investigate the impact of missing data on a transition model, termed the generalized autoregressive model of order 1 for longitudinal count data. The model has several features, including modeling dependence and accounting for overdispersion in the data, that make it appealing for the clinical trial setting. Furthermore, the model can be viewed as a natural extension of the commonly used log-linear model. Following introduction of the model and discussion of its estimation we investigate the impact of different missing data mechanisms on estimates of the model parameters through a simulation experiment. The findings of the simulation experiment show that, as in the case of normally distributed data, estimates under the missing completely at random (MCAR) and missing at random (MAR) mechanisms are close to their analogue for the full dataset and that the missing not at random (MNAR) mechanism has the greatest bias. Furthermore, estimates based on imputing the last observed value carried forward (LOCF) for missing data under the MAR assumption are similar to those of the MAR. This latter finding might be attributed to the Markov property underlying the model and to the high level of dependence among successive observations used in the simulation experiment. Finally, we consider an application of the generalized autoregressive model to a longitudinal epilepsy dataset analyzed in the literature.