Simultaneous Inference Using Multiple Marginal Models.

This tutorial describes single-step low-dimensional simultaneous inference with a focus on the availability of adjusted p values and compatible confidence intervals for more than just the usual mean value comparisons. The basic idea is, first, to use the influence of correlation on the quantile of the multivariate t-distribution: the higher the less conservative. In addition, second, the estimability of the correlation matrix using the multiple marginal models approach (mmm) using multiple models in the class of linear up to generalized linear mixed models. The underlying maxT-test using mmm is discussed by means of several real data scenarios using selected R packages. Surprisingly, different features are highlighted, among them: (i) analyzing different-scaled, correlated, multiple endpoints, (ii) analyzing multiple correlated binary endpoints, (iii) modeling dose as qualitative factor and/or quantitative covariate, (iv) joint consideration of several tuning parameters within the poly-k trend test, (v) joint testing of dose and time, (vi) considering several effect sizes, (vii) joint testing of subgroups and overall population in multiarm randomized clinical trials with correlated primary endpoints, (viii) multiple linear mixed effect models, (ix) generalized estimating equations, and (x) nonlinear regression models.

The Tukey trend test: Multiplicity adjustment using multiple marginal models.

In dose-response analysis, it is a challenge to choose appropriate linear or curvilinear shapes when considering multiple, differently scaled endpoints. It has been proposed to fit several marginal regression models that try sets of different transformations of the dose levels as explanatory variables for each endpoint. However, the multiple testing problem underlying this approach, involving correlated parameter estimates for the dose effect between and within endpoints, could only be adjusted heuristically. An asymptotic correction for multiple testing can be derived from the score functions of the marginal regression models. Based on a multivariate t-distribution, the correction provides a one-step adjustment of p-values that accounts for the correlation between estimates from different marginal models. The advantages of the proposed methodology are demonstrated through three example datasets, involving generalized linear models with differently scaled endpoints, differing covariates, and a mixed effect model and through simulation results. The methodology is implemented in an R package.

Using historical control data in bioassays for regulatory toxicology.

Historical control data (HCD) consist of pooled control group responses from bioassays. These data must be collected and are often used or reported in regulatory toxicology studies for multiple purposes: as quality assurance for the test system, to help identify toxicological effects and their effect-size relevance and to address the statistical multiple comparison problem. The current manuscript reviews the various classical and potential new approaches for using HCD. Issues in current practice are identified and recommendations for improved use and discussion are provided. Furthermore, stakeholders are invited to discuss whether it is necessary to consider uncertainty when using HCD formally and statistically in toxicological discussions and whether binary inclusion/exclusion criteria for HCD should be revised to a tiered information contribution to assessments. Overall, the critical value of HCD in toxicological bioassays is highlighted when used in a weight-of-evidence assessment.

Mechanistic basis for the activation of plant membrane receptor kinases by SERK-family coreceptors.

Plant-unique membrane receptor kinases with leucine-rich repeat ectodomains (LRR-RKs) can sense small molecule, peptide, and protein ligands. Many LRR-RKs require SERK-family coreceptor kinases for high-affinity ligand binding and receptor activation. How one coreceptor can contribute to the specific binding of distinct ligands and activation of different LRR-RKs is poorly understood. Here we quantitatively analyze the contribution of SERK3 to ligand binding and activation of the brassinosteroid receptor BRI1 and the peptide hormone receptor HAESA. We show that while the isolated receptors sense their respective ligands with drastically different binding affinities, the SERK3 ectodomain binds the ligand-associated receptors with very similar binding kinetics. We identify residues in the SERK3 N-terminal capping domain, which allow for selective steroid and peptide hormone recognition. In contrast, residues in the SERK3 LRR core form a second, constitutive receptor-coreceptor interface. Genetic analyses of protein chimera between BRI1 and SERK3 define that signaling-competent complexes are formed by receptor-coreceptor heteromerization in planta. A functional BRI1-HAESA chimera suggests that the receptor activation mechanism is conserved among different LRR-RKs, and that their signaling specificity is encoded in the kinase domain of the receptor. Our work pinpoints the relative contributions of receptor, ligand, and coreceptor to the formation and activation of SERK-dependent LRR-RK signaling complexes regulating plant growth and development.

Alternatives to statistical decision trees in regulatory (eco-)toxicological bioassays.

The goal of (eco-) toxicological testing is to experimentally establish a dose or concentration-response and to identify a threshold with a biologically relevant and probably non-random deviation from "normal". Statistical tests aid this process. Most statistical tests have distributional assumptions that need to be satisfied for reliable performance. Therefore, most statistical analyses used in (eco-)toxicological bioassays use subsequent pre- or assumption-tests to identify the most appropriate main test, so-called statistical decision trees. There are however several deficiencies with the approach, based on study design, type of tests used and subsequent statistical testing in general. When multiple comparisons are used to identify a non-random change against negative control, we propose to use robust testing, which can be generically applied without the need of decision trees. Visualization techniques and reference ranges also offer advantages over the current pre-testing approaches. We aim to promulgate the concepts in the (eco-) toxicological community and initiate a discussion for regulatory acceptance.

Use compatibility intervals in regulatory toxicology.

Recently it was recommended to avoid significance tests, in particular dichotomization into significant/non-significant on the basis of a p-value and a fixed 5% significance level (i.e. false positive rate). As an alternative, the interpretation of a suitable effect size and its compatibility interval is recommended, i.e. confidence intervals whose compatibility with the data, the assumptions, and the models is shown. This concept is used for the evaluation of assays in regulatory toxicology with special emphasis on the proof of hazard and proof of safety. Three case studies for multiple endpoints, multiple models and the consideration of historical controls illustrate the applicability of this concept. The corresponding software code for the open-source R project for statistical computing (www.r-project.org) is provided.

Multi-arm trials with multiple primary endpoints and missing values.

We present an extension of multiple contrast tests for multiple endpoints to the case of missing values. The endpoints are assumed to be normally distributed and correlated and to have equal covariance matrices for the different treatments. Different multivariate t distributions will be applied, differing in endpoint-specific degrees of freedom. In contrast to competing methods, the familywise error type I is maintained in the strong sense in an admissible range, and the problem of different marginal errors type I is avoided. The information of all observations is exploited, thereby enabling a gain in power compared with a complete case analysis.

Simultaneous small-sample comparisons in longitudinal or multi-endpoint trials using multiple marginal models.

Simultaneous inference in longitudinal, repeated-measures, and multi-endpoint designs can be onerous, especially when trying to find a reasonable joint model from which the interesting effects and covariances are estimated. A novel statistical approach known as multiple marginal models greatly simplifies the modelling process: the core idea is to "marginalise" the problem and fit multiple small models to different portions of the data, and then estimate the overall covariance matrix in a subsequent, separate step. Using these estimates guarantees strong control of the family-wise error rate, however only asymptotically. In this paper, we show how to make the approach also applicable to small-sample data problems. Specifically, we discuss the computation of adjusted P values and simultaneous confidence bounds for comparisons of randomised treatment groups as well as for levels of a nonrandomised factor such as multiple endpoints, repeated measures, or a series of points in time or space. We illustrate the practical use of the method with a data example.

Identification of the minimum effective dose for normally distributed data using a Bayesian variable selection approach.

The identification of the minimum effective dose is of high importance in the drug development process. In early stage screening experiments, establishing the minimum effective dose can be translated into a model selection based on information criteria. The presented alternative, Bayesian variable selection approach, allows for selection of the minimum effective dose, while taking into account model uncertainty. The performance of Bayesian variable selection is compared with the generalized order restricted information criterion on two dose-response experiments and through the simulations study. Which method has performed better depends on the complexity of the underlying model and the effect size relative to noise.

A General Framework for the Evaluation of Genetic Association Studies Using Multiple Marginal Models.

OBJECTIVE: In this study, we present a simultaneous inference procedure as a unified analysis framework for genetic association studies. METHODS: The method is based on the formulation of multiple marginal models that reflect different modes of inheritance. The basic advantage of this methodology is that no explicit formulation of the correlation between the test statistics is required. Moreover, the genotype scores are considered as a quantitative explanatory variable, i.e., regression models are used. RESULTS: The proposed approach covers a wide variety of endpoints (binary, count, quantitative, and time-to-event data). In addition, multiple endpoints of different types can be assessed simultaneously. This allows the detection of pleiotropic effects while taking the mode of inheritance into account. Moreover, multiple loci can be assessed simultaneously. CONCLUSION: The flexibility of the proposed approach is demonstrated while analyzing a variety of data examples.

Boxplots for grouped and clustered data in toxicology.

The vast majority of toxicological papers summarize experimental data as bar charts of means with error bars. While these graphics are easy to generate, they often obscure essential features of the data, such as outliers or subgroups of individuals reacting differently to a treatment. In particular, raw values are of prime importance in toxicology; therefore, we argue they should not be hidden in messy supplementary tables but rather unveiled in neat graphics in the results section. We propose jittered boxplots as a very compact yet comprehensive and intuitively accessible way of visualizing grouped and clustered data from toxicological studies together with individual raw values and indications of statistical significance. A web application to create these plots is available online.

Testing for qualitative interaction using ratios of treatment differences.

Testing for or against a qualitative interaction is relevant in randomized clinical trials that use a common primary factor treatment and have a secondary factor, such as the centre, region, subgroup, gender or biomarker. Interaction contrasts are formulated for ratios of differences between the levels of the primary treatment factor. Simultaneous confidence intervals allow for interpreting the magnitude and the relevance of the qualitative interaction. The proposed method is demonstrated by means of a multi-centre clinical trial, using the R package mratios.

Statistical methods and software for validation studies on new in vitro toxicity assays.

When a new in vitro assay method is introduced, it should be validated against the best available knowledge or a reference standard assay. For assays resulting in a simple binary outcome, the data can be displayed as a 2×2 table. Based on the estimated sensitivity and specificity, and the assumed prevalence of true positives in the population of interest, the positive and negative predictive values of the new assay can be calculated. We briefly discuss the experimental design of validation experiments and previously published methods for computing confidence intervals for predictive values. The application of the methods is illustrated for two toxicological examples, by using tools available in the free software, namely, R: confidence intervals for predictive values are computed for a validation study of an in vitro test battery, and sample size calculation is illustrated for an acute toxicity assay. The R code necessary to reproduce the results is given.

Statistical analysis of the hen's egg test for micronucleus induction (HET-MN assay).

The HET-MN assay (hen's egg test for micronucleus induction) is different from other in vitro genotoxicity assays in that it includes toxicologically important features such as absorption, distribution, metabolic activation, and excretion of the test compound. As a promising follow-up to complement existing in vitro test batteries for genotoxicity, the HET-MN is currently undergoing a formal validation. To optimize the validation, the present study describes a critical analysis of previously obtained HET-MN data to check the experimental design and to identify the most appropriate statistical procedure to evaluate treatment effects. Six statistical challenges (I-VI) of general relevance were identified, and remedies were provided which can be transferred to similarly designed test methods: a Williams-type trend test is proposed for overdispersed counts (II) by means of a square-root transformation which is robust for small sample sizes (I), variance heterogeneity (III), and possible downturn effects at high doses (IV). Due to near-to-zero or even zero-count data occurring in the negative control (V), a conditional comparison of the treatment groups against the mean of the historical controls (VI) instead of the concurrent control was proposed, which is in accordance with US-FDA recommendations. For the modified Williams-type tests, the power can be estimated depending on the magnitude and shape of the trend, the number of dose groups, and the magnitude of the MN counts in the negative control. The experimental design used previously (i.e. six eggs per dose group, scoring of 1000 cells per egg) was confirmed. The proposed approaches are easily available in the statistical computing environment R, and the corresponding R-codes are provided.

Statistical evaluation of toxicological assays: Dunnett or Williams test-take both.

The US National Toxicology Program recommends the use of the parametric multiple comparison procedures of Dunnett and Williams for the evaluation of repeated toxicity studies. For endpoints where either increasing or decreasing effects are of toxicological relevance, we recommend the use of the two-sided Dunnett test exclusively. For the many other endpoints, where a priori only one direction is of toxicological relevance, however, we recommend the combination of Dunnett and Williams test. In particular, we recommend the so-called Umbrella-protected Williams test which offers insights for all interesting monotone and non-monotone alternatives while only suffering a marginal loss in power compared to the Dunnett test. We illustrate the power difference analytically and compare the approach for different endpoint types using three real data examples to alternative tests available. Nonparametric tests, which are suitable for the evaluation of skewed distributed or scores data, are also considered. Particular attention is given to the different interpretations of the findings revealed by the different test. R programs used for the analyses are provided.

Human embryonic stem cell-derived test systems for developmental neurotoxicity: a transcriptomics approach.

Developmental neurotoxicity (DNT) and many forms of reproductive toxicity (RT) often manifest themselves in functional deficits that are not necessarily based on cell death, but rather on minor changes relating to cell differentiation or communication. The fields of DNT/RT would greatly benefit from in vitro tests that allow the identification of toxicant-induced changes of the cellular proteostasis, or of its underlying transcriptome network. Therefore, the 'human embryonic stem cell (hESC)-derived novel alternative test systems (ESNATS)' European commission research project established RT tests based on defined differentiation protocols of hESC and their progeny. Valproic acid (VPA) and methylmercury (MeHg) were used as positive control compounds to address the following fundamental questions: (1) Does transcriptome analysis allow discrimination of the two compounds? (2) How does analysis of enriched transcription factor binding sites (TFBS) and of individual probe sets (PS) distinguish between test systems? (3) Can batch effects be controlled? (4) How many DNA microarrays are needed? (5) Is the highest non-cytotoxic concentration optimal and relevant for the study of transcriptome changes? VPA triggered vast transcriptional changes, whereas MeHg altered fewer transcripts. To attenuate batch effects, analysis has been focused on the 500 PS with highest variability. The test systems differed significantly in their responses (<20 % overlap). Moreover, within one test system, little overlap between the PS changed by the two compounds has been observed. However, using TFBS enrichment, a relatively large 'common response' to VPA and MeHg could be distinguished from 'compound-specific' responses. In conclusion, the ESNATS assay battery allows classification of human DNT/RT toxicants on the basis of their transcriptome profiles.

Intralymphatic immunotherapy for cat allergy induces tolerance after only 3 injections.

BACKGROUND: Subcutaneous allergen-specific immunotherapy frequently causes allergic side effects and requires 30 to 80 injections over 3 to 5 years. OBJECTIVE: We sought to improve immunotherapy by using intralymphatic allergen administration (intralymphatic immunotherapy [ILIT]) and by targeting allergen to the MHC class II pathway. METHODS: Recombinant major cat dander allergen Fel d 1 was fused to a translocation sequence (TAT) and to part of the human invariant chain, generating a modular antigen transporter (MAT) vaccine (MAT-Fel d 1). In a randomized double-blind trial ILIT with MAT-Fel d 1 in alum was compared with ILIT with placebo (saline in alum) in allergic patients (ClinicalTrials.govNCT00718679). RESULTS: ILIT with MAT-Fel d 1 elicited no adverse events. After 3 placebo injections within 2 months, nasal tolerance increased less than 3-fold, whereas 3 intralymphatic injections with MAT-Fel d 1 increased nasal tolerance 74-fold (P < .001 vs placebo). ILIT with MAT-Fel d 1 stimulated regulatory T-cell responses (P = .026 vs placebo) and increased cat dander-specific IgG(4) levels by 5.66-fold (P = .003). The IgG(4) response positively correlated with IL-10 production (P < .001). CONCLUSION: In a first-in-human clinical study ILIT with MAT-Fel d 1 was safe and induced allergen tolerance after 3 injections.

Model-specific tests on variance heterogeneity for detection of potentially interacting genetic loci.

BACKGROUND: Trait variances among genotype groups at a locus are expected to differ in the presence of an interaction between this locus and another locus or environment. A simple maximum test on variance heterogeneity can thus be used to identify potentially interacting single nucleotide polymorphisms (SNPs). RESULTS: We propose a multiple contrast test for variance heterogeneity that compares the mean of Levene residuals for each genotype group with their average as an alternative to a global Levene test. We applied this test to a Bogalusa Heart Study dataset to screen for potentially interacting SNPs across the whole genome that influence a number of quantitative traits. A user-friendly implementation of this method is available in the R statistical software package multcomp. CONCLUSIONS: We show that the proposed multiple contrast test of model-specific variance heterogeneity can be used to test for potential interactions between SNPs and unknown alleles, loci or covariates and provide valuable additional information compared with traditional tests. Although the test is statistically valid for severely unbalanced designs, care is needed in interpreting the results at loci with low allele frequencies.

Two new approaches to improve the analysis of BALB/c 3T3 cell transformation assay data.

Validation activities of the BALB/c 3T3 cell transformation assay (CTA) - a test method used for the assessment of the carcinogenic potential of compounds - have revealed the need for statistical analysis tailored to specific features of BALB/c 3T3 CTA data. Whereas a standard statistical approach for the Syrian hamster embryo (SHE) CTA was considered sufficient, an international expert group was gathered by the European Centre for the Validation of Alternative Methods (ECVAM) to review commonly applied statistical approaches for BALB/c 3T3 CTA. As it was concluded that none of the commonly applied approaches is entirely appropriate, two novel statistical approaches were found to be recommended for the evaluation of BALB/c 3T3 CTA data accounting for possible non-monotone concentration-response relationship and variance heterogeneity: a negative binomial generalised linear model with William's-type downturn-protected trend tests and a normalisation of the data by a specific transformation allowing for application of a general linear model that estimates effects assuming a normal distribution with William's-type protected tests. Both approaches are described in this article and their performance and the quality of the results they generate is demonstrated using exemplary data. Our work confirmed that both approaches are suitable for the statistical analysis of BALB/c 3T3 CTA data and that each of them is superior to commonly used methods. Furthermore, a procedure dichotomising data into negatives and positives is proposed which allows re-testing in cases where inconclusive data are encountered. The scripts of the statistical evaluation programs written in R - a freely available statistical software - are appended including exemplary outputs (Appendix A).

Statistical evaluation of mortality in long-term carcinogenicity bioassays using a Williams-type procedure.

Several doses and a control group can be compared under order restriction using the Williams procedure for normally distributed endpoints assuming variance homogeneity. Comparison of the survival functions represents a secondary endpoint in long-term in vivo bioassays of carcinogenicity. Therefore, a Williams-type procedure for the comparison of survival functions is proposed for the assumption of the Cox proportional hazards model or the general frailty Cox model to allow a joint analysis over sex and strains. Interpretation according to both statistical significance and biological relevance is possible with simultaneous confidence intervals for hazard ratios. Related survival data can be analyzed using the R packages survival, coxme, and multcomp. Together with the R packages MCPAN and nparcomp, Dunnett- or Williams-type procedures are now available for the statistical analysis of the following endpoint types in toxicology: (i) normally distributed, (ii) non-normally distributed, (iii) score (ordered categorical) data, (iv) crude proportions, (v) survival functions, and (vi) time-to-tumor data with and without cause-of-death information.