Studentized permutation method for comparing two restricted mean survival times with small sample from randomized trials.

Recent observations, especially in cancer immunotherapy clinical trials with time-to-event outcomes, show that the commonly used proportional hazard assumption is often not justifiable, hampering an appropriate analysis of the data by hazard ratios. An attractive alternative advocated is given by the restricted mean survival time (RMST), which does not rely on any model assumption and can always be interpreted intuitively. Since methods for the RMST based on asymptotic theory suffer from inflated type-I error under small sample sizes, a permutation test was proposed recently leading to more convincing results in simulations. However, classical permutation strategies require an exchangeable data setup between comparison groups which may be limiting in practice. Besides, it is not possible to invert related testing procedures to obtain valid confidence intervals, which can provide more in-depth information. In this paper, we address these limitations by proposing a studentized permutation test as well as respective permutation-based confidence intervals. In an extensive simulation study, we demonstrate the advantage of our new method, especially in situations with relatively small sample sizes and unbalanced groups. Finally, we illustrate the application of the proposed method by re-analyzing data from a recent lung cancer clinical trial.

Stability selection for mixed effect models with large numbers of predictor variables: A simulation study.

Covariate selection when the number of available variables is large relative to the number of observations is problematic in epidemiology and remains the focus of continued research. Whilst a variety of statistical methods have been developed to attempt to overcome this issue, at present very few methods are available for wide data that include a clustered outcome. The purpose of this research was to make an empirical evaluation of a new method for covariate selection in wide data settings when the dependent variable is clustered. We used 3300 simulated datasets with a variety of defined structures and known sets of true predictor variables to conduct an empirical evaluation of a mixed model stability selection procedure. Comparison was made with an alternative method based on regularisation using the least absolute shrinkage and selection operator (Lasso) penalty. Model performance was assessed using several metrics including the true positive rate (proportion of true covariates selected in a final model) and false discovery rate (proportion of variables selected in a final model that were non-true (false) variables). For stability selection, the false discovery rate was consistently low, generally remaining ≤ 0.02 indicating that on average fewer than 1 in 50 of the variables selected in a final model were false variables. This was in contrast to the Lasso-based method in which the false discovery rate was between 0.59 and 0.72, indicating that generally more than 60% of variables selected in a final model were false variables. In contrast however, the Lasso method attained higher true positive rates than stability selection, although both methods achieved good results. For the Lasso method, true positive rates remained ≥ 0.93 whereas for stability selection the true positive rate was 0.73-0.97. Our results suggest both methods may be of value for covariate selection with high dimensional data with a clustered outcome. When high specificity is needed for identification of true covariates, stability selection appeared to offer the better solution, although with a slight loss of sensitivity. Conversely when high sensitivity is needed, the Lasso approach may be useful, even if accompanied by a substantial loss of specificity. Overall, the results indicated the loss of sensitivity when employing stability selection is relatively small compared to the loss of specificity when using the Lasso and therefore stability selection may provide the better option for the analyst when evaluating data of this type.

A national-scale vegetation multimetric index (VMMI) as an indicator of wetland condition across the conterminous United States.

In 2011, the US Environmental Protection Agency and its partners conducted the first National Wetland Condition Assessment at the continental-scale of the conterminous United States. A probability design for site selection was used to allow an unbiased assessment of wetland condition. We developed a vegetation multimetric index (VMMI) as a parsimonious biological indicator of ecological condition applicable to diverse wetland types at national and regional scales. Vegetation data (species presence and cover) were collected from 1138 sites that represented seven broad estuarine intertidal and inland wetland types. Using field collected data and plant species trait information, we developed 405 candidate metrics with potential for distinguishing least disturbed (reference) from most disturbed sites. Thirty-five of the metrics passed range, repeatability, and responsiveness screens and were considered as potential component metrics for the VMMI. A permutation approach was used to calculate thousands of randomly constructed potential national-scale VMMIs with 4, 6, 8, or 10 metrics. The best performing VMMI was identified based on limited redundancy among constituent metrics, sensitivity, repeatability, and precision. This final VMMI had four broadly applicable metrics (floristic quality index, relative importance of native species, richness of disturbance-tolerant species, and relative cover of native monocots). VMMI values and weights from the survey design for probability sites (n = 967) were used to estimate wetland area in good, fair, and poor condition, nationally and for each of 10 ecoregion by wetland type reporting groups. Strengths and limitations of the national VMMI for describing ecological condition are highlighted.

Semi-parametric analysis of overdispersed count and metric data with varying follow-up times: Asymptotic theory and small sample approximations.

Count data are common endpoints in clinical trials, for example magnetic resonance imaging lesion counts in multiple sclerosis. They often exhibit high levels of overdispersion, that is variances are larger than the means. Inference is regularly based on negative binomial regression along with maximum-likelihood estimators. Although this approach can account for heterogeneity it postulates a common overdispersion parameter across groups. Such parametric assumptions are usually difficult to verify, especially in small trials. Therefore, novel procedures that are based on asymptotic results for newly developed rate and variance estimators are proposed in a general framework. Moreover, in case of small samples the procedures are carried out using permutation techniques. Here, the usual assumption of exchangeability under the null hypothesis is not met due to varying follow-up times and unequal overdispersion parameters. This problem is solved by the use of studentized permutations leading to valid inference methods for situations with (i) varying follow-up times, (ii) different overdispersion parameters, and (iii) small sample sizes.

Explorations in statistics: the assumption of normality.

Learning about statistics is a lot like learning about science: the learning is more meaningful if you can actively explore. This twelfth installment of Explorations in Statistics explores the assumption of normality, an assumption essential to the meaningful interpretation of a t test. Although the data themselves can be consistent with a normal distribution, they need not be. Instead, it is the theoretical distribution of the sample mean or the theoretical distribution of the difference between sample means that must be roughly normal. The most versatile approach to assess normality is to bootstrap the sample mean, the difference between sample means, or t itself. We can then assess whether the distributions of these bootstrap statistics are consistent with a normal distribution by studying their normal quantile plots. If we suspect that an inference we make from a t test may not be justified-if we suspect that the theoretical distribution of the sample mean or the theoretical distribution of the difference between sample means is not normal-then we can use a permutation method to analyze our data.

Rank-based permutation approaches for non-parametric factorial designs.

Inference methods for null hypotheses formulated in terms of distribution functions in general non-parametric factorial designs are studied. The methods can be applied to continuous, ordinal or even ordered categorical data in a unified way, and are based only on ranks. In this set-up Wald-type statistics and ANOVA-type statistics are the current state of the art. The first method is asymptotically exact but a rather liberal statistical testing procedure for small to moderate sample size, while the latter is only an approximation which does not possess the correct asymptotic α level under the null. To bridge these gaps, a novel permutation approach is proposed which can be seen as a flexible generalization of the Kruskal-Wallis test to all kinds of factorial designs with independent observations. It is proven that the permutation principle is asymptotically correct while keeping its finite exactness property when data are exchangeable. The results of extensive simulation studies foster these theoretical findings. A real data set exemplifies its applicability.

Detecting treatment-covariate interactions using permutation methods.

The primary objective of a Randomized Clinical Trial usually is to investigate whether one treatment is better than its alternatives on average. However, treatment effects may vary across different patient subpopulations. In contrast to demonstrating one treatment is superior to another on the average sense, one is often more concerned with the question that, for a particular patient, or a group of patients with similar characteristics, which treatment strategy is most appropriate to achieve a desired outcome. Various interaction tests have been proposed to detect treatment effect heterogeneity; however, they typically examine covariates one at a time, do not offer an integrated approach that incorporates all available information, and can greatly increase the chance of a false positive finding when the number of covariates is large. We propose a new permutation test for the null hypothesis of no interaction effects for any covariate. The proposed test allows us to consider the interaction effects of many covariates simultaneously without having to group subjects into subsets based on pre-specified criteria and applies generally to randomized clinical trials of multiple treatments. The test provides an attractive alternative to the standard likelihood ratio test, especially when the number of covariates is large. We illustrate the proposed methods using a dataset from the Treatment of Adolescents with Depression Study.