A multi-arm multi-stage platform design that allows preplanned addition of arms while still controlling the family-wise error.

There is growing interest in platform trials that allow for adding of new treatment arms as the trial progresses as well as being able to stop treatments part way through the trial for either lack of benefit/futility or for superiority. In some situations, platform trials need to guarantee that error rates are controlled. This paper presents a multi-stage design, that allows additional arms to be added in a platform trial in a preplanned fashion, while still controlling the family-wise error rate, under the assumption of known number and timing of treatments to be added, and no time trends. A method is given to compute the sample size required to achieve a desired level of power and we show how the distribution of the sample size and the expected sample size can be found. We focus on power under the least favorable configuration which is the power of finding the treatment with a clinically relevant effect out of a set of treatments while the rest have an uninteresting treatment effect. A motivating trial is presented which focuses on two settings, with the first being a set number of stages per active treatment arm and the second being a set total number of stages, with treatments that are added later getting fewer stages. Compared to Bonferroni, the savings in the total maximum sample size are modest in a trial with three arms, <1% of the total sample size. However, the savings are more substantial in trials with more arms.

Treatment selection in multi-arm multi-stage designs: With application to a postpartum haemorrhage trial.

BACKGROUND: Multi-arm multi-stage trials are an efficient, adaptive approach for testing many treatments simultaneously within one protocol. In settings where numbers of patients available to be entered into trials and resources might be limited, such as primary postpartum haemorrhage, it may be necessary to select a pre-specified subset of arms at interim stages even if they are all showing some promise against the control arm. This will put a limit on the maximum number of patients required and reduce the associated costs. Motivated by the World Health Organization Refractory HaEmorrhage Devices trial in postpartum haemorrhage, we explored the properties of such a selection design in a randomised phase III setting and compared it with other alternatives. The objectives are: (1) to investigate how the timing of treatment selection affects the operating characteristics; (2) to explore the use of an information-rich (continuous) intermediate outcome to select the best-performing arm, out of four treatment arms, compared with using the primary (binary) outcome for selection at the interim stage; and (3) to identify factors that can affect the efficiency of the design. METHODS: We conducted simulations based on the refractory haemorrhage devices multi-arm multi-stage selection trial to investigate the impact of the timing of treatment selection and applying an adaptive allocation ratio on the probability of correct selection, overall power and familywise type I error rate. Simulations were also conducted to explore how other design parameters will affect both the maximum sample size and trial timelines. RESULTS: The results indicate that the overall power of the trial is bounded by the probability of 'correct' selection at the selection stage. The results showed that good operating characteristics are achieved if the treatment selection is conducted at around 17% of information time. Our results also showed that although randomising more patients to research arms before selection will increase the probability of selecting correctly, this will not increase the overall efficiency of the (selection) design compared with the fixed allocation ratio of 1:1 to all arms throughout. CONCLUSIONS: Multi-arm multi-stage selection designs are efficient and flexible with desirable operating characteristics. We give guidance on many aspects of these designs including selecting the intermediate outcome measure, the timing of treatment selection, and choosing the operating characteristics.

Multiple-testing correction in metabolome-wide association studies.

BACKGROUND: The search for statistically significant relationships between molecular markers and outcomes is challenging when dealing with high-dimensional, noisy and collinear multivariate omics data, such as metabolomic profiles. Permutation procedures allow for the estimation of adjusted significance levels without assuming independence among metabolomic variables. Nevertheless, the complex non-normal structure of metabolic profiles and outcomes may bias the permutation results leading to overly conservative threshold estimates i.e. lower than those from a Bonferroni or Sidak correction. METHODS: Within a univariate permutation procedure we employ parametric simulation methods based on the multivariate (log-)Normal distribution to obtain adjusted significance levels which are consistent across different outcomes while effectively controlling the type I error rate. Next, we derive an alternative closed-form expression for the estimation of the number of non-redundant metabolic variates based on the spectral decomposition of their correlation matrix. The performance of the method is tested for different model parametrizations and across a wide range of correlation levels of the variates using synthetic and real data sets. RESULTS: Both the permutation-based formulation and the more practical closed form expression are found to give an effective indication of the number of independent metabolic effects exhibited by the system, while guaranteeing that the derived adjusted threshold is stable across outcome measures with diverse properties.

Multi-arm multi-stage clinical trials for time-to-event outcomes.

This paper investigates the use of a general multi-arm multi-stage (MAMS) approach for time-to-event outcomes that would streamline simultaneous comparison of a large number of promising therapies in clinical trials, thus significantly reducing the time and the number of patients needed to evaluate the treatment. Controlling type I error in this setting is different than regular clinical trials as this approach incorporates both multiple comparison between arms and multiple stages. Historically, pairwise (PWER) and familywise (FWER) type I error rates have been primarily used to regulate the type I error in such designs. This paper will focus on constructing the efficacy and futility boundaries for a MAMS clinical trial in two different scenarios. In the first, it is assumed that the same outcome is used throughout the clinical trial for both intermediate and final assessments. In this scenario, we propose using the generalized Dunnett procedure that controls FWER. In the latter scenario, where intermediate and final outcomes are different in nature, we propose modifications to the existing method that originally concentrated on controlling PWER and extend the method to include FWER in the design. We also explore the performance of the proposed MAMS design in a setting where the proportional hazard assumption is violated in the presence of a delayed treatment effect and demonstrate the loss of power because of that. An alternative test statistic that can help circumvent this problem to maintain the desired power is also suggested.

A primer on strong vs weak control of familywise error rate.

Multiple comparison adjustments have a long history, yet confusion remains about which procedures control type 1 error rate in a strong sense and how to show this. Part of the confusion stems from a powerful technique called the closed testing principle, whose statement is deceptively simple, but is sometimes misinterpreted. This primer presents a straightforward way to think about multiplicity adjustment.

Adaptive multiarm multistage clinical trials.

Two methods for designing adaptive multiarm multistage (MAMS) clinical trials, originating from conceptually different group sequential frameworks are presented, and their operating characteristics are compared. In both methods pairwise comparisons are made, stage-by-stage, between each treatment arm and a common control arm with the goal of identifying active treatments and dropping inactive ones. At any stage one may alter the future course of the trial through adaptive changes to the prespecified decision rules for treatment selection and sample size reestimation, and notwithstanding such changes, both methods guarantee strong control of the family-wise error rate. The stage-wise MAMS approach was historically the first to be developed and remains the standard method for designing inferentially seamless phase 2-3 clinical trials. In this approach, at each stage, the data from each treatment comparison are summarized by a single multiplicity adjusted P-value. These stage-wise P-values are combined by a prespecified combination function and the resultant test statistic is monitored with respect to the classical two-arm group sequential efficacy boundaries. The cumulative MAMS approach is a more recent development in which a separate test statistic is constructed for each treatment comparison from the cumulative data at each stage. These statistics are then monitored with respect to multiplicity adjusted group sequential efficacy boundaries. We compared the powers of the two methods for designs with two and three active treatment arms, under commonly utilized decision rules for treatment selection, sample size reestimation and early stopping. In our investigations, which were carried out over a reasonably exhaustive exploration of the parameter space, the cumulative MAMS designs were more powerful than the stage-wise MAMS designs, except for the homogeneous case of equal treatment effects, where a small power advantage was discernable for the stage-wise MAMS designs.

Estimation of a significance threshold for epigenome-wide association studies.

Epigenome-wide association studies (EWAS) are designed to characterise population-level epigenetic differences across the genome and link them to disease. Most commonly, they assess DNA-methylation status at cytosine-guanine dinucleotide (CpG) sites, using platforms such as the Illumina 450k array that profile a subset of CpGs genome wide. An important challenge in the context of EWAS is determining a significance threshold for declaring a CpG site as differentially methylated, taking multiple testing into account. We used a permutation method to estimate a significance threshold specifically for the 450k array and a simulation extrapolation approach to estimate a genome-wide threshold. These methods were applied to five different EWAS datasets derived from a variety of populations and tissue types. We obtained an estimate of α=2.4×10-7 for the 450k array, and a genome-wide estimate of α=3.6×10-8. We further demonstrate the importance of these results by showing that previously recommended sample sizes for EWAS should be adjusted upwards, requiring samples between ∼10% and ∼20% larger in order to maintain type-1 errors at the desired level.

Robust spatial extent inference with a semiparametric bootstrap joint inference procedure.

Spatial extent inference (SEI) is widely used across neuroimaging modalities to adjust for multiple comparisons when studying brain-phenotype associations that inform our understanding of disease. Recent studies have shown that Gaussian random field (GRF)-based tools can have inflated family-wise error rates (FWERs). This has led to substantial controversy as to which processing choices are necessary to control the FWER using GRF-based SEI. The failure of GRF-based methods is due to unrealistic assumptions about the spatial covariance function of the imaging data. A permutation procedure is the most robust SEI tool because it estimates the spatial covariance function from the imaging data. However, the permutation procedure can fail because its assumption of exchangeability is violated in many imaging modalities. Here, we propose the (semi-) parametric bootstrap joint (PBJ; sPBJ) testing procedures that are designed for SEI of multilevel imaging data. The sPBJ procedure uses a robust estimate of the spatial covariance function, which yields consistent estimates of standard errors, even if the covariance model is misspecified. We use the methods to study the association between performance and executive functioning in a working memory functional magnetic resonance imaging study. The sPBJ has similar or greater power to the PBJ and permutation procedures while maintaining the nominal type 1 error rate in reasonable sample sizes. We provide an R package to perform inference using the PBJ and sPBJ procedures.

Improved statistical evaluation of group differences in connectomes by screening-filtering strategy with application to study maturation of brain connections between childhood and adolescence.

Detecting local differences between groups of connectomes is a great challenge in neuroimaging, because the large number of tests that have to be performed and the impact on multiplicity correction. Any available information should be exploited to increase the power of detecting true between-group effects. We present an adaptive strategy that exploits the data structure and the prior information concerning positive dependence between nodes and connections, without relying on strong assumptions. As a first step, we decompose the brain network, i.e., the connectome, into subnetworks and we apply a screening at the subnetwork level. The subnetworks are defined either according to prior knowledge or by applying a data driven algorithm. Given the results of the screening step, a filtering is performed to seek real differences at the node/connection level. The proposed strategy could be used to strongly control either the family-wise error rate or the false discovery rate. We show by means of different simulations the benefit of the proposed strategy, and we present a real application of comparing connectomes of preschool children and adolescents.

Multiplicity adjustments in testing for bioequivalence.

Bioequivalence of two drugs is usually demonstrated by rejecting two one-sided null hypotheses using the two one-sided tests for pharmacokinetic parameters: area under the concentration-time curve (AUC) and maximum concentration (Cmax). By virtue of the intersection-union test, there is no need for multiplicity adjustment in testing the two one-sided null hypotheses within each parameter. However, the decision rule for bioequivalence often requires equivalence to be achieved simultaneously on both parameters that contain four one-sided null hypotheses together; without adjusting for multiplicity, the family wise error rate (FWER) could fail to be controlled at the nominal type-I error rate α. The multiplicity issue for bioequivalence in this regard is scarcely discussed in the literature. To address this issue, we propose two approaches including a closed test procedure that controls FWER for the simultaneous AUC and Cmax bioequivalence and requires no adjustment of the type-I error, and an alpha-adaptive sequential testing (AAST) that controls FWER by pre-specifying the significance level on AUC (α1) and obtaining it for Cmax (α2) adaptively after testing of AUC. While both methods control FWER, the closed test requires testing of eight intersection null hypotheses each at α, and AAST is at times accomplished through a slight deduction in α1 and no deduction in α2 relative to α. The latter considers equivalence reached in AUC a higher importance than that in Cmax. Illustrated with published data, the two approaches, although operate differently, can lead to the same substantive conclusion and are better than a traditional method like Bonferroni adjustment.

A multiple testing method for hypotheses structured in a directed acyclic graph.

We present a novel multiple testing method for testing null hypotheses that are structured in a directed acyclic graph (DAG). The method is a top-down method that strongly controls the familywise error rate and can be seen as a generalization of Meinshausen's procedure for tree-structured hypotheses. Just as Meinshausen's procedure, our proposed method can be used to test for variable importance, only the corresponding variable clusters can be chosen more freely, because the method allows for multiple parent nodes and partially overlapping hypotheses. An important application of our method is in gene set analysis, in which one often wants to test multiple gene sets as well as individual genes for their association with a clinical outcome. By considering the genes and gene sets as nodes in a DAG, our method enables us to test both for significant gene sets as well as for significant individual genes within the same multiple testing procedure. The method will be illustrated by testing Gene Ontology terms for evidence of differential expression in a survival setting and is implemented in the R package cherry.

Quasi-periodic patterns (QPP): large-scale dynamics in resting state fMRI that correlate with local infraslow electrical activity.

Functional connectivity measurements from resting state blood-oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) are proving a powerful tool to probe both normal brain function and neuropsychiatric disorders. However, the neural mechanisms that coordinate these large networks are poorly understood, particularly in the context of the growing interest in network dynamics. Recent work in anesthetized rats has shown that the spontaneous BOLD fluctuations are tightly linked to infraslow local field potentials (LFPs) that are seldom recorded but comparable in frequency to the slow BOLD fluctuations. These findings support the hypothesis that long-range coordination involves low frequency neural oscillations and establishes infraslow LFPs as an excellent candidate for probing the neural underpinnings of the BOLD spatiotemporal patterns observed in both rats and humans. To further examine the link between large-scale network dynamics and infraslow LFPs, simultaneous fMRI and microelectrode recording were performed in anesthetized rats. Using an optimized filter to isolate shared components of the signals, we found that time-lagged correlation between infraslow LFPs and BOLD is comparable in spatial extent and timing to a quasi-periodic pattern (QPP) found from BOLD alone, suggesting that fMRI-measured QPPs and the infraslow LFPs share a common mechanism. As fMRI allows spatial resolution and whole brain coverage not available with electroencephalography, QPPs can be used to better understand the role of infraslow oscillations in normal brain function and neurological or psychiatric disorders.

Comparing connectomes across subjects and populations at different scales.

Brain connectivity can be represented by a network that enables the comparison of the different patterns of structural and functional connectivity among individuals. In the literature, two levels of statistical analysis have been considered in comparing brain connectivity across groups and subjects: 1) the global comparison where a single measure that summarizes the information of each brain is used in a statistical test; 2) the local analysis where a single test is performed either for each node/connection which implies a multiplicity correction, or for each group of nodes/connections where each subset is summarized by one single test in order to reduce the number of tests to avoid a penalizing multiplicity correction. We comment on the different levels of analysis and present some methods that have been proposed at each scale. We highlight as well the possible factors that could influence the statistical results and the questions that have to be addressed in such an analysis.

Neural correlates of time-varying functional connectivity in the rat.

Functional connectivity between brain regions, measured with resting state functional magnetic resonance imaging, holds great potential for understanding the basis of behavior and neuropsychiatric diseases. Recently it has become clear that correlations between the blood oxygenation level dependent (BOLD) signals from different areas vary over the course of a typical scan (6-10 min in length), though the changes are obscured by standard methods of analysis that assume the relationships are stationary. Unfortunately, because similar variability is observed in signals that share no temporal information, it is unclear which dynamic changes are related to underlying neural events. To examine this question, BOLD data were recorded simultaneously with local field potentials (LFP) from interhemispheric primary somatosensory cortex (SI) in anesthetized rats. LFP signals were converted into band-limited power (BLP) signals including delta, theta, alpha, beta and gamma. Correlation between signals from interhemispheric SI was performed in sliding windows to produce signals of correlation over time for BOLD and each BLP band. Both BOLD and BLP signals showed large changes in correlation over time and the changes in BOLD were significantly correlated to the changes in BLP. The strongest relationship was seen when using the theta, beta and gamma bands. Interestingly, while steady-state BOLD and BLP correlate with the global fMRI signal, dynamic BOLD becomes more like dynamic BLP after the global signal is regressed. As BOLD sliding window connectivity is partially reflecting underlying LFP changes, the present study suggests it may be a valuable method of studying dynamic changes in brain states.

Multiplicity adjustments in parallel-group multi-arm trials sharing a control group: Clear guidance is needed.

Multi-arm, parallel-group clinical trials are an efficient way of testing several new treatments, treatment regimens or doses. However, guidance on the requirement for statistical adjustment to control for multiple comparisons (type I error) using a shared control group is unclear. We argue, based on current evidence, that adjustment is not always necessary in such situations. We propose that adjustment should not be a requirement in multi-arm, parallel-group trials testing distinct treatments and sharing a control group, and we call for clearer guidance from stakeholders, such as regulators and scientific journals, on the appropriate settings for adjustment of multiplicity.

Family-wise type I error rate control for an extended 2-in-1 design with graphical approach in oncology drug development.

Nowadays, in oncology drug development, when an experimental treatment shows a promising anti-tumor effect in Phase I efficacy expansion, a Phase III pivotal trial may be launched directly. To mitigate the risk of skipping the traditional randomized Phase II proof of concept (POC) study, the 2-in-1 design was proposed by Chen et al. (2018). This design has gained great research and application interest since its publication and been extended in many ways. The original 2-in-1 design controls family-wise type I error rate (FWER) for one hypothesis in Phase II part and one hypothesis in Phase III part. However, in practice, for a stand-alone Phase III study usually there are multiple hypotheses with group sequential interim analyses and the multiplicity is controlled by the graphical approach. It is desirable that these features of the Phase III design are retained when 2-in-1 design is considered. The multiplicity control for a 2-in-1 design with multiple hypotheses in Phase III has been addressed mainly by the Bonferroni approach in the literature. For the more powerful graphical approach, while Jin and Zhang (2021) discussed the FWER control for a special 2-in-1 design, in which Phase II and Phase III have exactly the same hypotheses, the FWER control for a more common 2-in-1 design (i.e., one hypothesis in Phase II and multiple hypotheses in Phase III) is yet investigated. This paper provides the analytical conditions under which FWER is controlled with the graphical approach in such a 2-in-1 design. It also provides the numeric explorations of FWER control for such design with group sequential interim analyses in Phase III, as a direct Phase III design normally would have. As a result, our work helps lower the hurdle of the application of the 2-in-1 design and pave the way for its wider application.

POWER-ENHANCED MULTIPLE DECISION FUNCTIONS CONTROLLING FAMILY-WISE ERROR AND FALSE DISCOVERY RATES.

Improved procedures, in terms of smaller missed discovery rates (MDR), for performing multiple hypotheses testing with weak and strong control of the family-wise error rate (FWER) or the false discovery rate (FDR) are developed and studied. The improvement over existing procedures such as the Sidák procedure for FWER control and the Benjamini-Hochberg (BH) procedure for FDR control is achieved by exploiting possible differences in the powers of the individual tests. Results signal the need to take into account the powers of the individual tests and to have multiple hypotheses decision functions which are not limited to simply using the individual p-values, as is the case, for example, with the Sidák, Bonferroni, or BH procedures. They also enhance understanding of the role of the powers of individual tests, or more precisely the receiver operating characteristic (ROC) functions of decision processes, in the search for better multiple hypotheses testing procedures. A decision-theoretic framework is utilized, and through auxiliary randomizers the procedures could be used with discrete or mixed-type data or with rank-based nonparametric tests. This is in contrast to existing p-value based procedures whose theoretical validity is contingent on each of these p-value statistics being stochastically equal to or greater than a standard uniform variable under the null hypothesis. Proposed procedures are relevant in the analysis of high-dimensional "large M, small n" data sets arising in the natural, physical, medical, economic and social sciences, whose generation and creation is accelerated by advances in high-throughput technology, notably, but not limited to, microarray technology.

Online control of the familywise error rate.

Biological research often involves testing a growing number of null hypotheses as new data are accumulated over time. We study the problem of online control of the familywise error rate, that is testing an a priori unbounded sequence of hypotheses (p-values) one by one over time without knowing the future, such that with high probability there are no false discoveries in the entire sequence. This paper unifies algorithmic concepts developed for offline (single batch) familywise error rate control and online false discovery rate control to develop novel online familywise error rate control methods. Though many offline familywise error rate methods (e.g., Bonferroni, fallback procedures and Sidak's method) can trivially be extended to the online setting, our main contribution is the design of new, powerful, adaptive online algorithms that control the familywise error rate when the p-values are independent or locally dependent in time. Our numerical experiments demonstrate substantial gains in power, that are also formally proved in an idealized Gaussian sequence model. A promising application to the International Mouse Phenotyping Consortium is described.

Amygdala size varies with stress perception.

Stress is inevitably linked to life. It has many and complex facets. Notably, perception of stressful stimuli is an important factor when mounting stress responses and measuring its impact. Indeed, moved by the increasing number of stress-triggered pathologies, several groups drew on advanced neuroimaging techniques to explore stress effects on the brain. From that, several regions and circuits have been linked to stress, and a comprehensive integration of the distinct findings applied to common individuals is being pursued, but with conflicting results. Herein, we performed a volumetric regression analysis using participants' perceived stress as a variable of interest. Data shows that increased levels of perceived stress positively associate with the right amygdala and anterior hippocampal volumes.

Dose-dependent volume loss in subcortical deep grey matter structures after cranial radiotherapy.

BACKGROUND AND PURPOSE: The relation between radiotherapy (RT) dose to the brain and morphological changes in healthy tissue has seen recent increased interest. There already is evidence for changes in the cerebral cortex and white matter, as well as selected subcortical grey matter (GM) structures. We studied this relation in all deep GM structures, to help understand the aetiology of post-RT neurocognitive symptoms. MATERIALS AND METHODS: We selected 31 patients treated with RT for grade II-IV glioma. Pre-RT and 1 year post-RT 3D T1-weighted MRIs were automatically segmented, and the changes in volume of the following structures were assessed: amygdala, nucleus accumbens, caudate nucleus, hippocampus, globus pallidus, putamen, and thalamus. The volumetric changes were related to the mean RT dose received by each structure. Hippocampal volumes were entered into a population-based nomogram to estimate hippocampal age. RESULTS: A significant relation between RT dose and volume loss was seen in all examined structures, except the caudate nucleus. The volume loss rates ranged from 0.16 to 1.37%/Gy, corresponding to 4.9-41.2% per 30 Gy. Hippocampal age, as derived from the nomogram, was seen to increase by a median of 11 years. CONCLUSION: Almost all subcortical GM structures are susceptible to radiation-induced volume loss, with higher volume loss being observed with increasing dose. Volume loss of these structures is associated with neurological deterioration, including cognitive decline, in neurodegenerative diseases. To support a causal relationship between radiation-induced deep GM loss and neurocognitive functioning in glioma patients, future studies are needed that directly correlate volumetrics to clinical outcomes.