Error, power, and cluster separation rates of pairwise multiple testing procedures.

In comparing multiple treatments, 2 error rates that have been studied extensively are the familywise and false discovery rates. Different methods are used to control each of these rates. Yet, it is rare to find studies that compare the same methods on both of these rates, and also on the per-family error rate, the expected number of false rejections. Although the per-family error rate and the familywise error rate are similar in most applications when the latter is controlled at a conventional low level (e.g., .05), the 2 measures can diverge considerably with methods that control the false discovery rate at that same level. Furthermore, we shall consider both rejections of true hypotheses (Type I errors) and rejections of false hypotheses where the observed outcomes are in the incorrect direction (Type III errors). We point out that power estimates based on the number of correct rejections do not consider the pattern of those rejections, which is important in interpreting the total outcome. The present study introduces measures of interpretability based on the pattern of separation of treatments into nonoverlapping sets and compares methods on these measures. In general, range-based (configural) methods are more likely to obtain interpretable patterns based on treatment separation than individual p-value-based measures. Recommendations for practice based on these results are given in the article. Although the article is complex, these recommendations can be understood without the necessity for detailed perusal of the supporting material.

Association between serotonin transporter polymorphisms and problem behavior in adult males with intellectual disabilities.

Research on aggression over the past two decades has focused on gene-environment interaction models to explain the relative contribution of each to this behavioral phenotype in various clinical populations. Recent investigations suggest a link between aggression in people with intellectual disabilities the functionality of the serotonin transporter. The aims in this study were to examine the possible association of the STin2 and/or the 5-HTTLPR serotonin transporter polymorphisms in adult males with and without intellectual disabilities, and to examine the association of these polymorphisms with aggression in people with intellectual disabilities. DNA samples and behavioral records were obtained from adult males with intellectual disabilities, distinguished only by the presence or absence of aggression. No association was found between either transporter polymorphism for aggression. However, the long 5-HTTLPR allele, and not the short allele or the heterozygous state, was associated with the severity of aggression. The association with aggression appears to be genetically complex, suggesting there may be other genes, interactions between genes, and/or environmental relations occasioning aggression in people with intellectual disabilities.

Simulating multivariate g-and-h distributions.

The Tukey family of g-and-h distributions is often used to model univariate real-world data. There is a paucity of research demonstrating appropriate multivariate data generation using the g-and-h family of distributions with specified correlations. Therefore, the methodology and algorithms are presented to extend the g-and-h family from univariate to multivariate data generation. An example is provided along with a Monte Carlo simulation demonstrating the methodology. In addition, algorithms written in Mathematica 7.0 are available from the authors for implementing the procedure.

Applying clinical significance methodology to alcoholism treatment trials: determining recovery outcome status with individual- and population-based measures.

BACKGROUND: The current analysis applies clinical significance methodology to alcoholism treatment outcome research using data available from Project MATCH. Because of its high internal validity and its inclusion of multiple measures assessing multiple outcome dimensions, MATCH was considered an ideal study to explore the utility of this methodology. METHODS: Data reported here are from a total of 1,726 participants enrolled in either aftercare (n = 774) or outpatient (n = 952) arms of the study. First, a cutoff score was determined differentiating functional versus dysfunctional status on three outcome measures: percent days abstinent (PDA), mean drinks per drinking day (DDD) and negative consequences of alcohol use. Second, the reliable change in pre- to post-treatment scores on these three measures was calculated. RESULTS: The results reported herein support the importance of distinguishing between statistical and clinical significance of outcomes. During three months post-treatment, approximately one-half of the treated patients were "recovered" (i.e., both functional and reliably changed) with respect to both PDA (i.e., 51%) and negative consequences of drinking (i.e., 47%); however, only about one-third of individuals remained recovered throughout the full one-year follow-up period (i.e., 33% on PDA and 35% on negative consequences). These individual-based change outcomes compared similarly to a population-based indicator of heavy drinking. Alternatively, only about one-quarter of participants were recovered using two distinct criteria for mean DDD (i.e., 23-29%), and even fewer participants remained recovered on mean DDD over the full one-year follow-up period (i.e., about 14-18%). CONCLUSIONS: Based on study limitations, more work is required to make clinical significance methodology practically useful to alcoholism treatment trials including more precise definitions of functional status and relative change as well as better interpretation of the inter-relationship between multiple measures assessing multiple outcome domains.

Repeated Measures Interaction Test with Aligned Ranks.

The Welch-James (WJ) and the Huynh Improved General Approximation (IGA) tests for interaction were examined with respect to Type I error in a between- by within-subjects repeated measures design when data were non-normal, non-spherical and heterogeneous, particularly when group sizes were unequal. The tests were computed with aligned ranks and compared to the use of least squares and robust estimators (i.e., trimmed means and Winsorized variances/covariances). Critical values were either obtained theoretically or through a bootstrapping method. The IGA and WJ procedures based on aligned ranks always provided a valid test of a repeated measures interaction effect when group sizes were equal and covariance matrices across groups were homogeneous. On the other hand, the use of aligned ranks did not provide a valid test for a repeated measures interaction when covariance matrices were non-spherical with unequal variances across the levels of the repeated measures factor combined with unequal covariance matrices across the grouping factor. The IGA and WJ procedures based on robust estimators provided a valid test of the interaction across investigated conditions, however under a heavy-tailed distribution, the IGA and WJ procedures based on least squares estimators showed better Type I error control than when based on robust estimators.

A Comparison of Data Analysis Strategies for Testing Omnibus Effects in Higher-Order Repeated Measures Designs.

Methods for analyzing repeated measures data, in addition to the conventional and corrected degrees of freedom univariate and multivariate solutions, are presented in this review. These "newer" methods offer researchers either improved control over Type I errors and/or greater power to detect treatment effects when (a) certain assumptions are violated, and/or (b) missing data exists. In particular, Huynh's (1978) Improved General Approximate method, a multivariate Welch (1951)/James (1951)-type test, the mixedmodel approach (Littell, Milliken, Stroup, & Wolfinger, 1996) and Boik's (1997) empirical Bayes method are discussed. We review the literature regarding these procedures with respect to their robustness, ability to handle missing data, and availability of software to obtain numerical results.