Scaled average bioequivalence methods for highly variable drugs: Leveling-off soft limits and the EMA's 2010 guideline (some ways to improve its type I error control).

The regulatory EMA's reference scaled average bioequivalence (RSABE) approach for highly variable drugs suffers from some type I error control problems at the neighborhood of the 30% coefficient of variation (CV), where the bioequivalence (BE) limits change from constant to linearly scaled. This paper analyses BE inference methods based on the "Leveling-off" (LO) soft sigmoid expanding BE limits that were proposed as an appealing surrogate for the EMA's limits and compares both approaches, on the replicated and partially replicated crossover designs. The initially proposed version of the LO method also has type I error inflation problems, albeit attenuated. But given its more mathematically regular character, it is more suitable for analytical corrections. Here we introduce two improvements over LO, one based on the application of Howe's method and the other based on correcting the estimation error. They further reduce the type I error inflation, although it does not disappear completely. Finally, the effect of heteroscedasticity on the above results is studied. It leads to inflation or deflation of the type I error, depending on the design and the type of heteroscedasticity (variability of the test product greater than that of the reference product or the opposite). The replicated design is much more stable against these effects than the partially replicated design and maintains these improvements much better.

An equivalence test between features lists, based on the Sorensen-Dice index and the joint frequencies of GO term enrichment.

BACKGROUND: In integrative bioinformatic analyses, it is of great interest to stablish the equivalence between gene or (more in general) feature lists, up to a given level and in terms of their annotations in the Gene Ontology. The aim of this article is to present an equivalence test based on the proportion of GO terms which are declared as enriched in both lists simultaneously. RESULTS: On the basis of these data, the dissimilarity between gene lists is measured by means of the Sorensen-Dice index. We present two flavours of the same test: One of them based on the asymptotic normality of the test statistic and the other based on the bootstrap method. CONCLUSIONS: The accuracy of these tests is studied by means of simulation and their possible interest is illustrated by using them over two real datasets: A collection of gene lists related to cancer and a collection of gene lists related to kidney rejection after transplantation.

An iterative method to protect the type I error rate in bioequivalence studies under two-stage adaptive 2×2 crossover designs.

Bioequivalence studies are the pivotal clinical trials submitted to regulatory agencies to support the marketing applications of generic drug products. Average bioequivalence (ABE) is used to determine whether the mean values for the pharmacokinetic measures determined after administration of the test and reference products are comparable. Two-stage 2×2 crossover adaptive designs (TSDs) are becoming increasingly popular because they allow making assumptions on the clinically meaningful treatment effect and a reliable guess for the unknown within-subject variability. At an interim look, if ABE is not declared with an initial sample size, they allow to increase it depending on the estimated variability and to enroll additional subjects at a second stage, or to stop for futility in case of poor likelihood of bioequivalence. This is crucial because both parameters must clearly be prespecified in protocols, and the strategy agreed with regulatory agencies in advance with emphasis on controlling the overall type I error. We present an iterative method to adjust the significance levels at each stage which preserves the overall type I error for a wide set of scenarios which should include the true unknown variability value. Simulations showed adjusted significance levels higher than 0.0300 in most cases with type I error always below 5%, and with a power of at least 80%. TSDs work particularly well for coefficients of variation below 0.3 which are especially useful due to the balance between the power and the percentage of studies proceeding to stage 2. Our approach might support discussions with regulatory agencies.

An equivalence approach to the integrative analysis of feature lists.

BACKGROUND: Although a few comparison methods based on the biological meaning of gene lists have been developed, the goProfiles approach is one of the few that are being used for that purpose. It consists of projecting lists of genes into predefined levels of the Gene Ontology, in such a way that a multinomial model can be used for estimation and testing. Of particular interest is the fact that it may be used for proving equivalence (in the sense of "enough similarity") between two lists, instead of proving differences between them, which seems conceptually better suited to the end goal of establishing similarity among gene lists. An equivalence method has been derived that uses a distance-based approach and the confidence interval inclusion principle. Equivalence is declared if the upper limit of a one-sided confidence interval for the distance between two profiles is below a pre-established equivalence limit. RESULTS: In this work, this method is extended to establish the equivalence of any number of gene lists. Additionally, an algorithm to obtain the smallest equivalence limit that would allow equivalence between two or more lists to be declared is presented. This algorithm is at the base of an iterative method of graphic visualization to represent the most to least equivalent gene lists. These methods deal adequately with the problem of adjusting for multiple testing. The applicability of these techniques is illustrated in two typical situations: (i) a collection of cancer-related gene lists, suggesting which of them are more reasonable to combine -as claimed by the authors- and (ii) a collection of pathogenesis-based transcript sets, showing which of these are more closely related. The methods developed are available in the goProfiles Bioconductor package. CONCLUSIONS: The method provides a simple yet powerful and statistically well-grounded way to classify a set of genes or other feature lists by establishing their equivalence at a given equivalence threshold. The classification results can be viewed using standard visualization methods. This may be applied to a variety of problems, from deciding whether a series of datasets generating the lists can be combined to the simplification of groups of lists.

Controlling type I error in the reference-scaled bioequivalence evaluation of highly variable drugs.

Reference-scaled average bioequivalence (RSABE) approaches for highly variable drugs are based on linearly scaling the bioequivalence limits according to the reference formulation within-subject variability. RSABE methods have type I error control problems around the value where the limits change from constant to scaled. In all these methods, the probability of type I error has only one absolute maximum at this switching variability value. This allows adjusting the significance level to obtain statistically correct procedures (that is, those in which the probability of type I error remains below the nominal significance level), at the expense of some potential power loss. In this paper, we explore adjustments to the EMA and FDA regulatory RSABE approaches, and to a possible improvement of the original EMA method, designated as HoweEMA. The resulting adjusted methods are completely correct with respect to type I error probability. The power loss is generally small and tends to become irrelevant for moderately large (affordable in real studies) sample sizes.

Two-stage designs versus European scaled average designs in bioequivalence studies for highly variable drugs: Which to choose?

The usual approach to determine bioequivalence for highly variable drugs is scaled average bioequivalence, which is based on expanding the limits as a function of the within-subject variability in the reference formulation. This requires separately estimating this variability and thus using replicated or semireplicated crossover designs. On the other hand, regulations also allow using common 2 × 2 crossover designs based on two-stage adaptive approaches with sample size reestimation at an interim analysis. The choice between scaled or two-stage designs is crucial and must be fully described in the protocol. Using Monte Carlo simulations, we show that both methodologies achieve comparable statistical power, though the scaled method usually requires less sample size, but at the expense of each subject being exposed more times to the treatments. With an adequate initial sample size (not too low, eg, 24 subjects), two-stage methods are a flexible and efficient option to consider: They have enough power (eg, 80%) at the first stage for non-highly variable drugs, and, if otherwise, they provide the opportunity to step up to a second stage that includes additional subjects.

Consumer's risk in the EMA and FDA regulatory approaches for bioequivalence in highly variable drugs.

The 2010 US Food and Drug Administration and European Medicines Agency regulatory approaches to establish bioequivalence in highly variable drugs are both based on linearly scaling the bioequivalence limits, both take a 'scaled average bioequivalence' approach. The present paper corroborates previous work suggesting that none of them adequately controls type I error or consumer's risk, so they result in invalid test procedures in the neighbourhood of a within-subject coefficient of variation osf 30% for the reference (R) formulation. The problem is particularly serious in the US Food and Drug Administration regulation, but it is also appreciable in the European Medicines Agency one. For the partially replicated TRR/RTR/RRT and the replicated TRTR/RTRT crossover designs, we quantify these type I error problems by means of a simulation study, discuss their possible causes and propose straightforward improvements on both regulatory procedures that improve their type I error control while maintaining an adequate power. Copyright © 2015 John Wiley & Sons, Ltd.

Carryover negligibility and relevance in bioequivalence studies.

The carryover effect is a recurring issue in the pharmaceutical field. It may strongly influence the final outcome of an average bioequivalence study. Testing a null hypothesis of zero carryover is useless: not rejecting it does not guarantee the non-existence of carryover, and rejecting it is not informative of the true degree of carryover and its influence on the validity of the final outcome of the bioequivalence study. We propose a more consistent approach: even if some carryover is present, is it enough to seriously distort the study conclusions or is it negligible? This is the central aim of this paper, which focuses on average bioequivalence studies based on 2 × 2 crossover designs and on the main problem associated with carryover: type I error inflation. We propose an equivalence testing approach to these questions and suggest reasonable negligibility or relevance limits for carryover. Finally, we illustrate this approach on some real datasets.

Group-by-Treatment Interaction Effects in Comparative Bioavailability Studies.

Comparative bioavailability studies often involve multiple groups of subjects for a variety of reasons, such as clinical capacity limitations. This raises questions about the validity of pooling data from these groups in the statistical analysis and whether a group-by-treatment interaction should be evaluated. We investigated the presence or absence of group-by-treatment interactions through both simulation techniques and a meta-study of well-controlled trials. Our findings reveal that the test falsely detects an interaction when no true group-by-treatment interaction exists. Conversely, when a true group-by-treatment interaction does exist, it often goes undetected. In our meta-study, the detected group-by-treatment interactions were observed at approximately the level of the test and, thus, can be considered false positives. Testing for a group-by-treatment interaction is both misleading and uninformative. It often falsely identifies an interaction when none exists and fails to detect a real one. This occurs because the test is performed between subjects in crossover designs, and studies are powered to compare treatments within subjects. This work demonstrates a lack of utility for including a group-by-treatment interaction in the model when assessing single-site comparative bioavailability studies, and the clinical trial study structure is divided into groups.

Empirical study of correlated survival times for recurrent events with proportional hazards margins and the effect of correlation and censoring.

BACKGROUND: In longitudinal studies where subjects experience recurrent incidents over a period of time, such as respiratory infections, fever or diarrhea, statistical methods are required to take into account the within-subject correlation. METHODS: For repeated events data with censored failure, the independent increment (AG), marginal (WLW) and conditional (PWP) models are three multiple failure models that generalize Cox's proportional hazard model. In this paper, we revise the efficiency, accuracy and robustness of all three models under simulated scenarios with varying degrees of within-subject correlation, censoring levels, maximum number of possible recurrences and sample size. We also study the methods performance on a real dataset from a cohort study with bronchial obstruction. RESULTS: We find substantial differences between methods and there is not an optimal method. AG and PWP seem to be preferable to WLW for low correlation levels but the situation reverts for high correlations. CONCLUSIONS: All methods are stable in front of censoring, worsen with increasing recurrence levels and share a bias problem which, among other consequences, makes asymptotic normal confidence intervals not fully reliable, although they are well developed theoretically.

Comparison of lists of genes based on functional profiles.

BACKGROUND: How to compare studies on the basis of their biological significance is a problem of central importance in high-throughput genomics. Many methods for performing such comparisons are based on the information in databases of functional annotation, such as those that form the Gene Ontology (GO). Typically, they consist of analyzing gene annotation frequencies in some pre-specified GO classes, in a class-by-class way, followed by p-value adjustment for multiple testing. Enrichment analysis, where a list of genes is compared against a wider universe of genes, is the most common example. RESULTS: A new global testing procedure and a method incorporating it are presented. Instead of testing separately for each GO class, a single global test for all classes under consideration is performed. The test is based on the distance between the functional profiles, defined as the joint frequencies of annotation in a given set of GO classes. These classes may be chosen at one or more GO levels. The new global test is more powerful and accurate with respect to type I errors than the usual class-by-class approach. When applied to some real datasets, the results suggest that the method may also provide useful information that complements the tests performed using a class-by-class approach if gene counts are sparse in some classes. An R library, goProfiles, implements these methods and is available from Bioconductor, http://bioconductor.org/packages/release/bioc/html/goProfiles.html. CONCLUSIONS: The method provides an inferential basis for deciding whether two lists are functionally different. For global comparisons it is preferable to the global chi-square test of homogeneity. Furthermore, it may provide additional information if used in conjunction with class-by-class methods.

The effect of variability and carryover on average bioequivalence assessment: a simulation study.

The purpose of this study was to evaluate the effect of residual variability and carryover on average bioequivalence (ABE) studies performed under a 22 crossover design. ABE is usually assessed by means of the confidence interval inclusion principle. Here, the interval under consideration was the standard 'shortest' interval, which is the mainstream approach in practice. The evaluation was performed by means of a simulation study under different combinations of carryover and residual variability besides of formulation effect and sample size. The evaluation was made in terms of percentage of ABE declaration, coverage and interval precision. As is well known, high levels of variability distort the ABE procedures, particularly its type II error control (i.e. high variabilities make difficult to declare bioequivalence when it holds). The effect of carryover is modulated by variability and is especially disturbing for the type I error control. In the presence of carryover, the risk of erroneously declaring bioequivalence may become high, especially for low variabilities and large sample sizes. We end up with some hints concerning the controversy about pretesting for carryover before performing ABE analysis.

Statistical Methods for Quality Equivalence of Topical Products. 0.5 mg/g Betamethasone Ointment as a Case-Study.

This study examines the statistical implications, and their possible implementation, of the "Draft guideline on quality and equivalence of topical products" issued by the European Medicines Agency in 2018, with particular focus on the section devoted to quality equivalence of physical properties. A new confidence interval to conduct the quality equivalence test and a way to cope with the multiplicity of quality parameters are presented and discussed. As an example, the results and the statistical analysis of a study on betamethasone 0.5 mg/g ointment are presented. It is suggested that the equivalence limits proposed in the draft guideline are overly strict: It is as difficult to declare quality equivalence between two packaging formats of the same reference product as to declare quality equivalence between the reference and the test product.