The empirical replicability of task-based fMRI as a function of sample size.

Replicating results (i.e. obtaining consistent results using a new independent dataset) is an essential part of good science. As replicability has consequences for theories derived from empirical studies, it is of utmost importance to better understand the underlying mechanisms influencing it. A popular tool for non-invasive neuroimaging studies is functional magnetic resonance imaging (fMRI). While the effect of underpowered studies is well documented, the empirical assessment of the interplay between sample size and replicability of results for task-based fMRI studies remains limited. In this work, we extend existing work on this assessment in two ways. Firstly, we use a large database of 1400 subjects performing four types of tasks from the IMAGEN project to subsample a series of independent samples of increasing size. Secondly, replicability is evaluated using a multi-dimensional framework consisting of 3 different measures: (un)conditional test-retest reliability, coherence and stability. We demonstrate not only a positive effect of sample size, but also a trade-off between spatial resolution and replicability. When replicability is assessed voxelwise or when observing small areas of activation, a larger sample size than typically used in fMRI is required to replicate results. On the other hand, when focussing on clusters of voxels, we observe a higher replicability. In addition, we observe variability in the size of clusters of activation between experimental paradigms or contrasts of parameter estimates within these.

Desmopressin (melt) therapy in children with monosymptomatic nocturnal enuresis and nocturnal polyuria results in improved neuropsychological functioning and sleep.

BACKGROUND: There is a high comorbidity between nocturnal enuresis, sleep disorders and psychological problems. The aim of this study was to investigate whether a decrease in nocturnal diuresis volume not only improves enuresis but also ameliorates disrupted sleep and (neuro)psychological dysfunction, the major comorbidities of this disorder. METHODS: In this open-label, prospective phase IV study, 30 children with monosymptomatic nocturnal enuresis (MNE) underwent standardized video-polysomnographic testing and multi-informant (neuro)psychological testing at baseline and 6 months after the start of desmopressin treatment in the University Hospital Ghent, Belgium. Primary endpoints were the effect on sleep and (neuro)psychological functioning. The secondary endpoint was the change in the first undisturbed sleep period or the time to the first void. RESULTS: Thirty children aged between 6 and 16 (mean 10.43, standard deviation 3.08) years completed the study. The results demonstrated a significant decrease in periodic limb movements during sleep (PLMS) and a prolonged first undisturbed sleep period. Additionally, (neuro)psychological functioning was improved on several domains. CONCLUSIONS: The study demonstrates that the degree of comorbidity symptoms is at least aggravated by enuresis (and/or high nocturnal diuresis rate) since sleep and (neuro)psychological functioning were significantly ameliorated by treatment of enuresis. These results indicate that enuresis is not such a benign condition as has previously been assumed.

Evaluation of Second-Level Inference in fMRI Analysis.

We investigate the impact of decisions in the second-level (i.e., over subjects) inferential process in functional magnetic resonance imaging on (1) the balance between false positives and false negatives and on (2) the data-analytical stability, both proxies for the reproducibility of results. Second-level analysis based on a mass univariate approach typically consists of 3 phases. First, one proceeds via a general linear model for a test image that consists of pooled information from different subjects. We evaluate models that take into account first-level (within-subjects) variability and models that do not take into account this variability. Second, one proceeds via inference based on parametrical assumptions or via permutation-based inference. Third, we evaluate 3 commonly used procedures to address the multiple testing problem: familywise error rate correction, False Discovery Rate (FDR) correction, and a two-step procedure with minimal cluster size. Based on a simulation study and real data we find that the two-step procedure with minimal cluster size results in most stable results, followed by the familywise error rate correction. The FDR results in most variable results, for both permutation-based inference and parametrical inference. Modeling the subject-specific variability yields a better balance between false positives and false negatives when using parametric inference.

Microbial diversity and metabolite composition of Belgian red-brown acidic ales.

Belgian red-brown acidic ales are sour and alcoholic fermented beers, which are produced by mixed-culture fermentation and blending. The brews are aged in oak barrels for about two years, after which mature beer is blended with young, non-aged beer to obtain the end-products. The present study evaluated the microbial community diversity of Belgian red-brown acidic ales at the end of the maturation phase of three subsequent brews of three different breweries. The microbial diversity was compared with the metabolite composition of the brews at the end of the maturation phase. Therefore, mature brew samples were subjected to 454 pyrosequencing of the 16S rRNA gene (bacteria) and the internal transcribed spacer region (yeasts) and a broad range of metabolites was quantified. The most important microbial species present in the Belgian red-brown acidic ales investigated were Pediococcus damnosus, Dekkera bruxellensis, and Acetobacter pasteurianus. In addition, this culture-independent analysis revealed operational taxonomic units that were assigned to an unclassified fungal community member, Candida, and Lactobacillus. The main metabolites present in the brew samples were L-lactic acid, D-lactic acid, and ethanol, whereas acetic acid was produced in lower quantities. The most prevailing aroma compounds were ethyl acetate, isoamyl acetate, ethyl hexanoate, and ethyl octanoate, which might be of impact on the aroma of the end-products.

Bootstrapping fMRI Data: Dealing with Misspecification.

The validity of inference based on the General Linear Model (GLM) for the analysis of functional magnetic resonance imaging (fMRI) time series has recently been questioned. Bootstrap procedures that partially avoid modeling assumptions may offer a welcome solution. We empirically compare two voxelwise GLM-based bootstrap approaches: a semi-parametric approach, relying solely on a model for the expected signal; and a fully parametric bootstrap approach, requiring an additional parameterization of the temporal structure. While the fully parametric approach assumes independent whitened residuals, the semi-parametric approach relies on independent blocks of residuals. The evaluation is based on inferential properties and the potential to reproduce important data characteristics. Different noise structures and data-generating mechanisms for the signal are simulated. When the model for the noise and expected signal is correct, we find that the fully parametric approach works well, with respect to both inference and reproduction of data characteristics. However, in the presence of misspecification, the fully parametric approach can be improved with additional blocking. The semi-parametric approach performs worse than the (fully) parametric approach with respect to inference but achieves comparable results as the parametric approach with additional blocking with respect to image reproducibility. We demonstrate that when the expected signal is incorrect GLM-based bootstrapping can overcome the poor performance of classical (non-bootstrap) parametric inference. We illustrate both approaches on a study exploring the neural representation of object representation in the visual pathway.

Irrelevant location information influences accuracy in bowling.

Most cognitive control effects, although numerously reported in computer task studies, have rarely been tested outside the laboratory. The purpose of this study was twofold. First, we aimed to improve the ecological validity of a well-studied congruency effect. The Simon effect (Simon, 1969) is the observation that an irrelevant stimulus location can facilitate or impede task performance when it is congruent or incongruent with the response location. Secondly, we wanted to investigate the role of action experience on the Simon effect. In this study, experienced bowlers were asked to hit either the left- or rightmost pin, depending on the pitch of a tone. Irrelevant to the task, this tone could be presented in the congruent or incongruent ear. Our results demonstrate that the Simon effect can be observed outside the laboratory and that weekly training at bowling may help in shielding against irrelevant location stimuli.

Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability.

Functional Magnetic Resonance Imaging is a widespread technique in cognitive psychology that allows visualizing brain activation. The data analysis encompasses an enormous number of simultaneous statistical tests. Procedures that either control the familywise error rate or the false discovery rate have been applied to these data. These methods are mostly validated in terms of average sensitivity and specificity. However, procedures are not comparable if requirements on their error rates differ. Moreover, less attention has been given to the instability or variability of results. In a simulation study in the context of imaging, we first compare the Bonferroni and Benjamini-Hochberg procedures. Considering Bonferroni as a way to control the expected number of type I errors enables more lenient thresholding compared to familywise error rate control and a direct comparison between both procedures. We point out that while the same balance is obtained between average sensitivity and specificity, the Benjamini-Hochberg procedure appears less stable. Secondly, we have implemented the procedure of Gordon et al. () (originally proposed for gene selection) that includes stability, measured through bootstrapping, in the decision criterion. Simulations indicate that the method attains the same balance between sensitivity and specificity. It improves the stability of Benjamini-Hochberg but does not outperform Bonferroni, making this computationally heavy bootstrap procedure less appealing. Third, we show how stability of thresholding procedures can be assessed using real data. In a dataset on face recognition, we again find that Bonferroni renders more stable results.