An Overview of Current Statistical Methods for Implementing Quality Tolerance Limits.

BACKGROUND: In 2016, the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use updated its efficacy guideline for good clinical practice and introduced predefined quality tolerance limits (QTLs) as a quality control in clinical trials. QTLs are complementary to Quality by Design (QbD) principles (ICH-E8) and are one of the components of the risk-based clinical trial quality management system. METHODS: Currently the framework for QTLs process is well established, extensively describing the operational aspects of Defining, Monitoring and Reporting, but a single source of commonly used methods to establish QTLs and secondary limits is lacking. This paper will primarily focus on closing this gap and include applications of statistical process control and Bayesian methods on commonly used study level quality parameters such as premature treatment discontinuation, study discontinuation and significant protocol deviations as examples. CONCLUSIONS: Application of quality tolerance limits to parameters that correspond to critical to quality factors help identify systematic errors. Some situations pose special challenges to implementing QTLs and not all methods are optimal in every scenario. Early warning signals, in addition to QTL, are necessary to trigger actions to further minimize the possibility of an end-of-study excursion.

A Cochran-Armitage-type and a score-free global test for multivariate ordinal data.

We propose a Cochran-Armitage-type and a score-free global test that can be used to assess the presence of an association between a set of ordinally scaled covariates and an outcome variable within the range of generalized linear models. Both tests are developed within the framework of the well-established 'global test' methodology and as such are feasible in high-dimensional data situations under any correlation and enable adjustment for covariates. The Cochran-Armitage-type test, for which an intimate connection with the traditional score-based Cochran-Armitage test is shown, rests upon explicit assumptions on the distances between the covariates' ordered categories. The score-free test, in contrast, parametrizes these distances and thus keeps them flexible, rendering it ideally suited for covariates measured on an ordinal scale. As confirmed by means of simulations, the Cochran-Armitage-type test focuses its power on set-outcome relationships where the distances between the covariates' categories are equal or close to those assumed, whereas the score-free test spreads its power over a wide range of possible set-outcome relationships, putting more emphasis on monotonic than on non-monotonic ones. Based on the tests' power properties, it is discussed when to favour one or the other, and the practical merits of both of them are illustrated by an application in the field of rehabilitation medicine. Our proposed tests are implemented in the R package globaltest. Copyright © 2016 John Wiley & Sons, Ltd.

Over-optimism in bioinformatics: an illustration.

MOTIVATION: In statistical bioinformatics research, different optimization mechanisms potentially lead to 'over-optimism' in published papers. So far, however, a systematic critical study concerning the various sources underlying this over-optimism is lacking. RESULTS: We present an empirical study on over-optimism using high-dimensional classification as example. Specifically, we consider a 'promising' new classification algorithm, namely linear discriminant analysis incorporating prior knowledge on gene functional groups through an appropriate shrinkage of the within-group covariance matrix. While this approach yields poor results in terms of error rate, we quantitatively demonstrate that it can artificially seem superior to existing approaches if we 'fish for significance'. The investigated sources of over-optimism include the optimization of datasets, of settings, of competing methods and, most importantly, of the method's characteristics. We conclude that, if the improvement of a quantitative criterion such as the error rate is the main contribution of a paper, the superiority of new algorithms should always be demonstrated on independent validation data. AVAILABILITY: The R codes and relevant data can be downloaded from http://www.ibe.med.uni-muenchen.de/organisation/mitarbeiter/020_professuren/boulesteix/overoptimism/, such that the study is completely reproducible.