Rejection odds and rejection ratios: A proposal for statistical practice in testing hypotheses.

Much of science is (rightly or wrongly) driven by hypothesis testing. Even in situations where the hypothesis testing paradigm is correct, the common practice of basing inferences solely on p-values has been under intense criticism for over 50 years. We propose, as an alternative, the use of the odds of a correct rejection of the null hypothesis to incorrect rejection. Both pre-experimental versions (involving the power and Type I error) and post-experimental versions (depending on the actual data) are considered. Implementations are provided that range from depending only on the p-value to consideration of full Bayesian analysis. A surprise is that all implementations - even the full Bayesian analysis - have complete frequentist justification. Versions of our proposal can be implemented that require only minor modifications to existing practices yet overcome some of their most severe shortcomings.

A Bayesian approach to subgroup identification.

This article discusses subgroup identification, the goal of which is to determine the heterogeneity of treatment effects across subpopulations. Searching for differences among subgroups is challenging because it is inherently a multiple testing problem with the complication that test statistics for subgroups are typically highly dependent, making simple multiplicity corrections such as the Bonferroni correction too conservative. In this article, a Bayesian approach to identify subgroup effects is proposed, with a scheme for assigning prior probabilities to possible subgroup effects that accounts for multiplicity and yet allows for (preexperimental) preference to specific subgroups. The analysis utilizes a new Bayesian model selection methodology and, as a by-product, produces individual probabilities of treatment effect that could be of use in personalized medicine. The analysis is illustrated on an example involving subgroup analysis of biomarker effects on treatments.

Statistical interpretation of the RV144 HIV vaccine efficacy trial in Thailand: a case study for statistical issues in efficacy trials.

Recently, the RV144 randomized, double-blind, efficacy trial in Thailand reported that a prime-boost human immunodeficiency virus (HIV) vaccine regimen conferred ∼30% protection against HIV acquisition. However, different analyses seemed to give conflicting results, and a heated debate ensued as scientists and the broader public struggled with their interpretation. The lack of accounting for statistical principles helped flame the debate, and we leverage these principles to provide a more scientific interpretation. We first address interpretation of frequentist results, including interpretation of P values, synthesis of results from multiple analyses (ie, intention-to-treat versus per-protocol/fully immunized), and accounting for external efficacy trials. Second, we address how Bayesian statistics, which provide clearly interpretable statements about probabilities that the vaccine efficacy takes certain values, provide more information for weighing the evidence about efficacy than do frequentist statistics alone. Third, we evaluate RV144 for completeness of end point ascertainment and integrity of blinding, necessary tasks for establishing robustly interpretable results.