Improving the Eligibility of Task-Based fMRI Studies for Meta-Analysis: A Review and Reporting Recommendations.

Decisions made during the analysis or reporting of an fMRI study influence the eligibility of that study to be entered into a meta-analysis. In a meta-analysis, results of different studies on the same topic are combined. To combine the results, it is necessary that all studies provide equivalent pieces of information. However, in task-based fMRI studies we see a large variety in reporting styles. Several specific meta-analysis methods have been developed to deal with the reporting practices occurring in task-based fMRI studies, therefore each requiring a specific type of input. In this manuscript we provide an overview of the meta-analysis methods and the specific input they require. Subsequently we discuss how decisions made during the study influence the eligibility of a study for a meta-analysis and finally we formulate some recommendations about how to report an fMRI study so that it complies with as many meta-analysis methods as possible.

Review Paper: Reporting Practices for Task fMRI Studies.

What are the standards for the reporting methods and results of fMRI studies, and how have they evolved over the years? To answer this question we reviewed 160 papers published between 2004 and 2019. Reporting styles for methods and results of fMRI studies can differ greatly between published studies. However, adequate reporting is essential for the comprehension, replication and reuse of the study (for instance in a meta-analysis). To aid authors in reporting the methods and results of their task-based fMRI study the COBIDAS report was published in 2016, which provides researchers with clear guidelines on how to report the design, acquisition, preprocessing, statistical analysis and results (including data sharing) of fMRI studies (Nichols et al. in Best Practices in Data Analysis and Sharing in Neuroimaging using fMRI, 2016). In the past reviews have been published that evaluate how fMRI methods are reported based on the 2008 guidelines, but they did not focus on how task based fMRI results are reported. This review updates reporting practices of fMRI methods, and adds an extra focus on how fMRI results are reported. We discuss reporting practices about the design stage, specific participant characteristics, scanner characteristics, data processing methods, data analysis methods and reported results.

Nonlinear mediation analysis with high-dimensional mediators whose causal structure is unknown.

With multiple possible mediators on the causal pathway from a treatment to an outcome, we consider the problem of decomposing the effects along multiple possible causal path(s) through each distinct mediator. Under a path-specific effects framework, such fine-grained decompositions necessitate stringent assumptions, such as correctly specifying the causal structure among the mediators, and no unobserved confounding among the mediators. In contrast, interventional direct and indirect effects for multiple mediators can be identified under much weaker conditions, while providing scientifically relevant causal interpretations. Nonetheless, current estimation approaches require (correctly) specifying a model for the joint mediator distribution, which can be difficult when there is a high-dimensional set of possibly continuous and noncontinuous mediators. In this article, we avoid the need to model this distribution, by developing a definition of interventional effects previously suggested for longitudinal mediation. We propose a novel estimation strategy that uses nonparametric estimates of the (counterfactual) mediator distributions. Noncontinuous outcomes can be accommodated using nonlinear outcome models. Estimation proceeds via Monte Carlo integration. The procedure is illustrated using publicly available genomic data to assess the causal effect of a microRNA expression on the 3-month mortality of brain cancer patients that is potentially mediated by expression values of multiple genes.

Disentangling indirect effects through multiple mediators without assuming any causal structure among the mediators.

hen multiple mediators exist on the causal pathway from treatment to outcome, path analysis prevails for disentangling indirect effects along paths linking possibly several mediators. However, separately evaluating each indirect effect along different posited paths demands stringent assumptions, such as correctly specifying the mediators' causal structure, and no unobserved confounding among the mediators. These assumptions may be unfalsifiable in practice and, when they fail to hold, can result in misleading conclusions about the mediators. Nevertheless, these assumptions are avoidable when substantive interest is in inference about the indirect effects specific to each distinct mediator. In this article, we introduce a new definition of indirect effects called interventional indirect effects from the causal inference and epidemiology literature. Interventional indirect effects can be unbiasedly estimated without the assumptions above while retaining scientifically meaningful interpretations. We show that under a typical class of linear and additive mean models, estimators of interventional indirect effects adopt the same analytical form as prevalent product-of-coefficient estimators assuming a parallel mediator model. Prevalent estimators are therefore unbiased when estimating interventional indirect effects-even when there are unknown causal effects among the mediators-but require a different causal interpretation. When other mediators moderate the effect of each mediator on the outcome, and the mediators' covariance is affected by treatment, such an indirect effect due to the mediators' mutual dependence (on one another) cannot be attributed to any mediator alone. We exploit the proposed definitions of interventional indirect effects to develop novel estimators under such settings. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Discussion on "distributional independent component analysis for diverse neuroimaging modalities" by Ben Wu, Subhadip Pal, Jian Kang, and Ying Guo.

We are grateful for the opportunity to provide a discussion on this paper. We will first focus on the general context. Next, we will emphasize the novel key ideas proposed by the authors before formulating some open questions.

The empirical replicability of task-based fMRI as a function of sample size.

Replicating results (i.e. obtaining consistent results using a new independent dataset) is an essential part of good science. As replicability has consequences for theories derived from empirical studies, it is of utmost importance to better understand the underlying mechanisms influencing it. A popular tool for non-invasive neuroimaging studies is functional magnetic resonance imaging (fMRI). While the effect of underpowered studies is well documented, the empirical assessment of the interplay between sample size and replicability of results for task-based fMRI studies remains limited. In this work, we extend existing work on this assessment in two ways. Firstly, we use a large database of 1400 subjects performing four types of tasks from the IMAGEN project to subsample a series of independent samples of increasing size. Secondly, replicability is evaluated using a multi-dimensional framework consisting of 3 different measures: (un)conditional test-retest reliability, coherence and stability. We demonstrate not only a positive effect of sample size, but also a trade-off between spatial resolution and replicability. When replicability is assessed voxelwise or when observing small areas of activation, a larger sample size than typically used in fMRI is required to replicate results. On the other hand, when focussing on clusters of voxels, we observe a higher replicability. In addition, we observe variability in the size of clusters of activation between experimental paradigms or contrasts of parameter estimates within these.

A likelihood ratio approach for functional localization in fMRI.

BACKGROUND: To increase power when analyzing fMRI data, researchers often define functional regions of interest (fROIs). It is crucial that this fROI is defined with an optimal balance between both false positives and false negatives to ensure maximal spatial accuracy and to avoid potentially biased results in the main fMRI experiment. Additionally, since the fROI is defined in each subject separately, the used method should attune to the general level of activation of the individual. NEW METHOD: We investigate the benefits of the maximized likelihood ratio (mLR) method. This method is based on the likelihood paradigm where likelihood ratios are used to reflect relative statistical evidence in favor of an a priori defined practically relevant alternative hypothesis as compared to the null hypothesis of no activation. RESULTS: Through both simulations and real data, we show that the mLR method provides cumulative evidence for voxels that are active with an effect size that is larger than the one a priori defined in the alternative. Furthermore, an optimal balance between Type I and Type II errors is achieved when the alternative is an underestimation of the true effect size. COMPARISON WITH EXISTING METHODS: The mLR method is compared with false discovery rate corrected null hypothesis significance testing and regular likelihood ratio testing. It performs as good as or outperformed both methods in both detection of practically relevant voxels and the trade- off between false positives and false negatives. CONCLUSIONS: The mLR method provides fROIs that are both spatially accurate and practically relevant.

Estimation of Controlled Direct Effects in Longitudinal Mediation Analyses with Latent Variables in Randomized Studies.

In a randomized study with longitudinal data on a mediator and outcome, estimating the direct effect of treatment on the outcome at a particular time requires adjusting for confounding of the association between the outcome and all preceding instances of the mediator. When the confounders are themselves affected by treatment, standard regression adjustment is prone to severe bias. In contrast, G-estimation requires less stringent assumptions than path analysis using SEM to unbiasedly estimate the direct effect even in linear settings. In this article, we propose a G-estimation method to estimate the controlled direct effect of treatment on the outcome, by adapting existing G-estimation methods for time-varying treatments without mediators. The proposed method can accommodate continuous and noncontinuous mediators, and requires no models for the confounders. Unbiased estimation only requires correctly specifying a mean model for either the mediator or the outcome. The method is further extended to settings where the mediator or outcome, or both, are latent, and generalizes existing methods for single measurement occasions of the mediator and outcome to longitudinal data on the mediator and outcome. The methods are utilized to assess the effects of an intervention on physical activity that is possibly mediated by motivation to exercise in a randomized study.

A bootstrap version of the Hausman test to assess the impact of cluster-level endogeneity beyond the random intercept model.

In the random intercept model for clustered data, the random effect is typically assumed to be independent of predictors. Violation of this assumption due to unmeasured cluster-level confounding (endogeneity) induces bias in the estimates of effects of within-cluster predictors. Treating cluster-specific intercepts as fixed rather than random avoids this bias. The Hausman test contrasts the fixed effect estimator with the traditional random effect estimator in the random intercept model to test for the presence of cluster-level endogeneity and has a known asymptotic χ2 -distribution under correct model specification. Unmeasured cluster-level heterogeneity may, however, interact with predictors as well, necessitating random slope models. Relying on either cluster or residual resampling in a bootstrap procedure, we propose two extensions of the Hausman test that can easily be used beyond the random intercept model. We compare the original Hausman test and its robust version to the newly proposed bootstrap tests in terms of empirical type I error rate and power. Under additive unmeasured heterogeneity, all methods perform equally well, whereas the original and robust Hausman tests are too liberal or too conservative under additional slope heterogeneity, both bootstrap Hausman tests maintain appropriate performance. Moreover, both bootstrap tests show robustness against misspecification in the presence of unit-level heteroscedasticity and temporal correlation.

Assessing robustness against potential publication bias in Activation Likelihood Estimation (ALE) meta-analyses for fMRI.

The importance of integrating research findings is incontrovertible and procedures for coordinate-based meta-analysis (CBMA) such as Activation Likelihood Estimation (ALE) have become a popular approach to combine results of fMRI studies when only peaks of activation are reported. As meta-analytical findings help building cumulative knowledge and guide future research, not only the quality of such analyses but also the way conclusions are drawn is extremely important. Like classical meta-analyses, coordinate-based meta-analyses can be subject to different forms of publication bias which may impact results and invalidate findings. The file drawer problem refers to the problem where studies fail to get published because they do not obtain anticipated results (e.g. due to lack of statistical significance). To enable assessing the stability of meta-analytical results and determine their robustness against the potential presence of the file drawer problem, we present an algorithm to determine the number of noise studies that can be added to an existing ALE fMRI meta-analysis before spatial convergence of reported activation peaks over studies in specific regions is no longer statistically significant. While methods to gain insight into the validity and limitations of results exist for other coordinate-based meta-analysis toolboxes, such as Galbraith plots for Multilevel Kernel Density Analysis (MKDA) and funnel plots and egger tests for seed-based d mapping, this procedure is the first to assess robustness against potential publication bias for the ALE algorithm. The method assists in interpreting meta-analytical results with the appropriate caution by looking how stable results remain in the presence of unreported information that may differ systematically from the information that is included. At the same time, the procedure provides further insight into the number of studies that drive the meta-analytical results. We illustrate the procedure through an example and test the effect of several parameters through extensive simulations. Code to generate noise studies is made freely available which enables users to easily use the algorithm when interpreting their results.

Flexible Mediation Analysis With Multiple Mediators.

The advent of counterfactual-based mediation analysis has triggered enormous progress on how, and under what assumptions, one may disentangle path-specific effects upon combining arbitrary (possibly nonlinear) models for mediator and outcome. However, current developments have largely focused on single mediators because required identification assumptions prohibit simple extensions to settings with multiple mediators that may depend on one another. In this article, we propose a procedure for obtaining fine-grained decompositions that may still be recovered from observed data in such complex settings. We first show that existing analytical approaches target specific instances of a more general set of decompositions and may therefore fail to provide a comprehensive assessment of the processes that underpin cause-effect relationships between exposure and outcome. We then outline conditions for obtaining the remaining set of decompositions. Because the number of targeted decompositions increases rapidly with the number of mediators, we introduce natural effects models along with estimation methods that allow for flexible and parsimonious modeling. Our procedure can easily be implemented using off-the-shelf software and is illustrated using a reanalysis of the World Health Organization's Large Analysis and Review of European Housing and Health Status (WHO-LARES) study on the effect of mold exposure on mental health (2002-2003).

Introducing Alternative-Based Thresholding for Defining Functional Regions of Interest in fMRI.

In fMRI research, one often aims to examine activation in specific functional regions of interest (fROIs). Current statistical methods tend to localize fROIs inconsistently, focusing on avoiding detection of false activation. Not missing true activation is however equally important in this context. In this study, we explored the potential of an alternative-based thresholding (ABT) procedure, where evidence against the null hypothesis of no effect and evidence against a prespecified alternative hypothesis is measured to control both false positives and false negatives directly. The procedure was validated in the context of localizer tasks on simulated brain images and using a real data set of 100 runs per subject. Voxels categorized as active with ABT can be confidently included in the definition of the fROI, while inactive voxels can be confidently excluded. Additionally, the ABT method complements classic null hypothesis significance testing with valuable information by making a distinction between voxels that show evidence against both the null and alternative and voxels for which the alternative hypothesis cannot be rejected despite lack of evidence against the null.

The Influence of Study-Level Inference Models and Study Set Size on Coordinate-Based fMRI Meta-Analyses.

Given the increasing amount of neuroimaging studies, there is a growing need to summarize published results. Coordinate-based meta-analyses use the locations of statistically significant local maxima with possibly the associated effect sizes to aggregate studies. In this paper, we investigate the influence of key characteristics of a coordinate-based meta-analysis on (1) the balance between false and true positives and (2) the activation reliability of the outcome from a coordinate-based meta-analysis. More particularly, we consider the influence of the chosen group level model at the study level [fixed effects, ordinary least squares (OLS), or mixed effects models], the type of coordinate-based meta-analysis [Activation Likelihood Estimation (ALE) that only uses peak locations, fixed effects, and random effects meta-analysis that take into account both peak location and height] and the amount of studies included in the analysis (from 10 to 35). To do this, we apply a resampling scheme on a large dataset (N = 1,400) to create a test condition and compare this with an independent evaluation condition. The test condition corresponds to subsampling participants into studies and combine these using meta-analyses. The evaluation condition corresponds to a high-powered group analysis. We observe the best performance when using mixed effects models in individual studies combined with a random effects meta-analysis. Moreover the performance increases with the number of studies included in the meta-analysis. When peak height is not taken into consideration, we show that the popular ALE procedure is a good alternative in terms of the balance between type I and II errors. However, it requires more studies compared to other procedures in terms of activation reliability. Finally, we discuss the differences, interpretations, and limitations of our results.

Assessing moderated mediation in linear models requires fewer confounding assumptions than assessing mediation.

It is well known from the mediation analysis literature that the identification of direct and indirect effects relies on strong no unmeasured confounding assumptions of no unmeasured confounding. Even in randomized studies the mediator may still be correlated with unobserved prognostic variables that affect the outcome, in which case the mediator's role in the causal process may not be inferred without bias. In the behavioural and social science literature very little attention has been given so far to the causal assumptions required for moderated mediation analysis. In this paper we focus on the index for moderated mediation, which measures by how much the mediated effect is larger or smaller for varying levels of the moderator. We show that in linear models this index can be estimated without bias in the presence of unmeasured common causes of the moderator, mediator and outcome under certain conditions. Importantly, one can thus use the test for moderated mediation to support evidence for mediation under less stringent confounding conditions. We illustrate our findings with data from a randomized experiment assessing the impact of being primed with social deception upon observer responses to others' pain, and from an observational study of individuals who ended a romantic relationship assessing the effect of attachment anxiety during the relationship on mental distress 2 years after the break-up.

Evaluation of Second-Level Inference in fMRI Analysis.

We investigate the impact of decisions in the second-level (i.e., over subjects) inferential process in functional magnetic resonance imaging on (1) the balance between false positives and false negatives and on (2) the data-analytical stability, both proxies for the reproducibility of results. Second-level analysis based on a mass univariate approach typically consists of 3 phases. First, one proceeds via a general linear model for a test image that consists of pooled information from different subjects. We evaluate models that take into account first-level (within-subjects) variability and models that do not take into account this variability. Second, one proceeds via inference based on parametrical assumptions or via permutation-based inference. Third, we evaluate 3 commonly used procedures to address the multiple testing problem: familywise error rate correction, False Discovery Rate (FDR) correction, and a two-step procedure with minimal cluster size. Based on a simulation study and real data we find that the two-step procedure with minimal cluster size results in most stable results, followed by the familywise error rate correction. The FDR results in most variable results, for both permutation-based inference and parametrical inference. Modeling the subject-specific variability yields a better balance between false positives and false negatives when using parametric inference.

Structural equation modeling versus marginal structural modeling for assessing mediation in the presence of posttreatment confounding.

Inverse probability weighting for marginal structural models has been suggested as a strategy to estimate the direct effect of a treatment or exposure on an outcome in studies where the effect of mediator on outcome is subject to posttreatment confounding. This type of confounding, whereby confounders of the effect of mediator on outcome are themselves affected by the exposure, complicates mediation analyses and necessitates apt analysis strategies. In this article, we contrast the inverse probability weighting approach with the traditional path analysis approach to mediation analysis. We show that in a particular class of linear models, adjustment for posttreatment confounding can be realized via a fairly standard modification of the traditional path analysis approach. The resulting approach is simpler; by avoiding inverse probability weighting, it moreover results in direct effect estimators with smaller finite sample bias and greater precision. We further show that a particular variant of the G-estimation approach from the causal inference literature is equivalent with the path analysis approach in simple linear settings but is more generally applicable in settings with interactions and/or noncontinuous mediators and confounders. We conclude that the use of inverse probability weighting for marginal structural models to adjust for posttreatment confounding in mediation analysis is primarily indicated in nonlinear models for the outcome.

Bootstrapping fMRI Data: Dealing with Misspecification.

The validity of inference based on the General Linear Model (GLM) for the analysis of functional magnetic resonance imaging (fMRI) time series has recently been questioned. Bootstrap procedures that partially avoid modeling assumptions may offer a welcome solution. We empirically compare two voxelwise GLM-based bootstrap approaches: a semi-parametric approach, relying solely on a model for the expected signal; and a fully parametric bootstrap approach, requiring an additional parameterization of the temporal structure. While the fully parametric approach assumes independent whitened residuals, the semi-parametric approach relies on independent blocks of residuals. The evaluation is based on inferential properties and the potential to reproduce important data characteristics. Different noise structures and data-generating mechanisms for the signal are simulated. When the model for the noise and expected signal is correct, we find that the fully parametric approach works well, with respect to both inference and reproduction of data characteristics. However, in the presence of misspecification, the fully parametric approach can be improved with additional blocking. The semi-parametric approach performs worse than the (fully) parametric approach with respect to inference but achieves comparable results as the parametric approach with additional blocking with respect to image reproducibility. We demonstrate that when the expected signal is incorrect GLM-based bootstrapping can overcome the poor performance of classical (non-bootstrap) parametric inference. We illustrate both approaches on a study exploring the neural representation of object representation in the visual pathway.

Mindful attention and awareness mediate the association between age and negative affect.

OBJECTIVES: Later life is often accompanied by experiences of loss and bereavement in several life domains. In spite of this, older adults experience less negative affect than their younger counterparts. Several explanations for this paradoxical finding have been put forward, but the mechanisms underlying the association between age and negative affect remain largely unclear. In the present study, we propose that mindfulness may be an important mediator of this association. METHOD: A cross-sectional sample of 507 participants (age range 18-85 years) was used to investigate this question. Participants completed a range of self-report questionnaires on demographic variables, mindfulness, affect, quality of life (QoL), and personality. In our mediation analysis, we used an advanced statistical technique called G-estimation to control for the impact of confounding variables such as personality dimensions and QoL. RESULTS: Our findings indicate that the age-related decrease in negative affect is mediated by mindfulness. The results remain significant when we control for QoL and personality. DISCUSSION: These findings imply that mindfulness skills may be an important link between age and negative affect. Implications of these findings for the understanding of the well-being paradox are discussed.

Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability.

Functional Magnetic Resonance Imaging is a widespread technique in cognitive psychology that allows visualizing brain activation. The data analysis encompasses an enormous number of simultaneous statistical tests. Procedures that either control the familywise error rate or the false discovery rate have been applied to these data. These methods are mostly validated in terms of average sensitivity and specificity. However, procedures are not comparable if requirements on their error rates differ. Moreover, less attention has been given to the instability or variability of results. In a simulation study in the context of imaging, we first compare the Bonferroni and Benjamini-Hochberg procedures. Considering Bonferroni as a way to control the expected number of type I errors enables more lenient thresholding compared to familywise error rate control and a direct comparison between both procedures. We point out that while the same balance is obtained between average sensitivity and specificity, the Benjamini-Hochberg procedure appears less stable. Secondly, we have implemented the procedure of Gordon et al. () (originally proposed for gene selection) that includes stability, measured through bootstrapping, in the decision criterion. Simulations indicate that the method attains the same balance between sensitivity and specificity. It improves the stability of Benjamini-Hochberg but does not outperform Bonferroni, making this computationally heavy bootstrap procedure less appealing. Third, we show how stability of thresholding procedures can be assessed using real data. In a dataset on face recognition, we again find that Bonferroni renders more stable results.

Post-hoc power estimation for topological inference in fMRI.

When analyzing functional MRI data, several thresholding procedures are available to account for the huge number of volume units or features that are tested simultaneously. The main focus of these methods is to prevent an inflation of false positives. However, this comes with a serious decrease in power and leads to a problematic imbalance between type I and type II errors. In this paper, we show how estimating the number of activated peaks or clusters enables one to estimate post-hoc how powerful the selection procedure performs. This procedure can be used in real studies as a diagnostics tool, and raises awareness on how much activation is potentially missed. The method is evaluated and illustrated using simulations and a real data example. Our real data example illustrates the lack of power in current fMRI research.