Highlight results, don't hide them: Enhance interpretation, reduce biases and improve reproducibility.

Most neuroimaging studies display results that represent only a tiny fraction of the collected data. While it is conventional to present "only the significant results" to the reader, here we suggest that this practice has several negative consequences for both reproducibility and understanding. This practice hides away most of the results of the dataset and leads to problems of selection bias and irreproducibility, both of which have been recognized as major issues in neuroimaging studies recently. Opaque, all-or-nothing thresholding, even if well-intentioned, places undue influence on arbitrary filter values, hinders clear communication of scientific results, wastes data, is antithetical to good scientific practice, and leads to conceptual inconsistencies. It is also inconsistent with the properties of the acquired data and the underlying biology being studied. Instead of presenting only a few statistically significant locations and hiding away the remaining results, studies should "highlight" the former while also showing as much as possible of the rest. This is distinct from but complementary to utilizing data sharing repositories: the initial presentation of results has an enormous impact on the interpretation of a study. We present practical examples and extensions of this approach for voxelwise, regionwise and cross-study analyses using publicly available data that was analyzed previously by 70 teams (NARPS; Botvinik-Nezer, et al., 2020), showing that it is possible to balance the goals of displaying a full set of results with providing the reader reasonably concise and "digestible" findings. In particular, the highlighting approach sheds useful light on the kind of variability present among the NARPS teams' results, which is primarily a varied strength of agreement rather than disagreement. Using a meta-analysis built on the informative "highlighting" approach shows this relative agreement, while one using the standard "hiding" approach does not. We describe how this simple but powerful change in practice-focusing on highlighting results, rather than hiding all but the strongest ones-can help address many large concerns within the field, or at least to provide more complete information about them. We include a list of practical suggestions for results reporting to improve reproducibility, cross-study comparisons and meta-analyses.

BOLD Response is more than just magnitude: Improving detection sensitivity through capturing hemodynamic profiles.

Typical fMRI analyses often assume a canonical hemodynamic response function (HRF) that primarily focuses on the peak height of the overshoot, neglecting other morphological aspects. Consequently, reported analyses often reduce the overall response curve to a single scalar value. In this study, we take a data-driven approach to HRF estimation at the whole-brain voxel level, without assuming a response profile at the individual level. We then employ a roughness penalty at the population level to estimate the response curve, aiming to enhance predictive accuracy, inferential efficiency, and cross-study reproducibility. By examining a fast event-related FMRI dataset, we demonstrate the shortcomings and information loss associated with adopting the canonical approach. Furthermore, we address the following key questions: 1) To what extent does the HRF shape vary across different regions, conditions, and participant groups? 2) Does the data-driven approach improve detection sensitivity compared to the canonical approach? 3) Can analyzing the HRF shape help validate the presence of an effect in conjunction with statistical evidence? 4) Does analyzing the HRF shape offer evidence for whole-brain response during a simple task?

LayNii: A software suite for layer-fMRI.

High-resolution fMRI in the sub-millimeter regime allows researchers to resolve brain activity across cortical layers and columns non-invasively. While these high-resolution data make it possible to address novel questions of directional information flow within and across brain circuits, the corresponding data analyses are challenged by MRI artifacts, including image blurring, image distortions, low SNR, and restricted coverage. These challenges often result in insufficient spatial accuracy of conventional analysis pipelines. Here we introduce a new software suite that is specifically designed for layer-specific functional MRI: LayNii. This toolbox is a collection of command-line executable programs written in C/C++ and is distributed opensource and as pre-compiled binaries for Linux, Windows, and macOS. LayNii is designed for layer-fMRI data that suffer from SNR and coverage constraints and thus cannot be straightforwardly analyzed in alternative software packages. Some of the most popular programs of LayNii contain 'layerification' and columnarization in the native voxel space of functional data as well as many other layer-fMRI specific analysis tasks: layer-specific smoothing, model-based vein mitigation of GE-BOLD data, quality assessment of artifact dominated sub-millimeter fMRI, as well as analyses of VASO data.

Neural mechanisms of reward processing in adolescent irritability.

Irritability is impairing and prevalent across pediatric psychiatric disorders and typical development, yet its neural mechanisms are largely unknown. This study evaluated the relation between adolescent irritability and reward-related brain function as a candidate neural mechanism. Adolescents from intervention-seeking families in the community (N = 52; mean age = 13.80, SD = 1.94) completed a monetary incentive delay task to assess reward anticipation and feedback (reward receipt and omission) during fMRI acquisition. Whole-brain analyses, controlling for age, examined brain activation and striatal and amygdala connectivity in relation to irritability. Irritability was measured using the parent- and youth-reported Affective Reactivity Index. Irritability was associated with altered reward processing-related activation and connectivity in multiple networks during reward anticipation and feedback, including increased striatal activation and altered ventral striatum connectivity with prefrontal areas. Our findings suggest that irritability is associated with altered neural patterns during reward processing and that aberrant prefrontal cortex-mediated top-down control may be related to irritability. These findings inform our understanding of the etiology of youth irritability and the development of mechanism-based interventions.

Language lateralization from task-based and resting state functional MRI in patients with epilepsy.

We compared resting state (RS) functional connectivity and task-based fMRI to lateralize language dominance in 30 epilepsy patients (mean age = 33; SD = 11; 12 female), a measure used for presurgical planning. Language laterality index (LI) was calculated from task fMRI in frontal, temporal, and frontal + temporal regional masks using LI bootstrap method from SPM12. RS language LI was assessed using two novel methods of calculating RS language LI from bilateral Broca's area seed based connectivity maps across regional masks and multiple thresholds (p < .05, p < .01, p < .001, top 10% connections). We compared LI from task and RS fMRI continuous values and dominance classifications. We found significant positive correlations between task LI and RS LI when functional connectivity thresholds were set to the top 10% of connections. Concordance of dominance classifications ranged from 20% to 30% for the intrahemispheric resting state LI method and 50% to 63% for the resting state LI intra- minus interhemispheric difference method. Approximately 40% of patients left dominant on task showed RS bilateral dominance. There was no difference in LI concordance between patients with right-sided and left-sided resections. Early seizure onset (<6 years old) was not associated with atypical language dominance during task-based or RS fMRI. While a relationship between task LI and RS LI exists in patients with epilepsy, language dominance is less lateralized on RS than task fMRI. Concordance of language dominance classifications between task and resting state fMRI depends on brain regions surveyed and RS LI calculation method.

Differential Sampling of Visual Space in Ventral and Dorsal Early Visual Cortex.

A fundamental feature of cortical visual processing is the separation of visual processing for the upper and lower visual fields. In early visual cortex (EVC), the upper visual field is processed ventrally, with the lower visual field processed dorsally. This distinction persists into several category-selective regions of occipitotemporal cortex, with ventral and lateral scene-, face-, and object-selective regions biased for the upper and lower visual fields, respectively. Here, using an elliptical population receptive field (pRF) model, we systematically tested the sampling of visual space within ventral and dorsal divisions of human EVC in both male and female participants. We found that (1) pRFs tend to be elliptical and oriented toward the fovea with distinct angular distributions for ventral and dorsal divisions of EVC, potentially reflecting a radial bias; and (2) pRFs in ventral areas were larger (∼1.5×) and more elliptical (∼1.2×) than those in dorsal areas. These differences potentially reflect a tendency for receptive fields in ventral temporal cortex to overlap the fovea with less emphasis on precise localization and isotropic representation of space compared with dorsal areas. Collectively, these findings suggest that ventral and dorsal divisions of EVC sample visual space differently, likely contributing to and/or stemming from the functional differentiation of visual processing observed in higher-level regions of the ventral and dorsal cortical visual pathways.SIGNIFICANCE STATEMENT The processing of visual information from the upper and lower visual fields is separated in visual cortex. Although ventral and dorsal divisions of early visual cortex (EVC) are commonly assumed to sample visual space equivalently, we demonstrate systematic differences using an elliptical population receptive field (pRF) model. Specifically, we demonstrate that (1) ventral and dorsal divisions of EVC exhibit diverging distributions of pRF angle, which are biased toward the fovea; and (2) ventral pRFs exhibit higher aspect ratios and cover larger areas than dorsal pRFs. These results suggest that ventral and dorsal divisions of EVC sample visual space differently and that such differential sampling likely contributes to different functional roles attributed to the ventral and dorsal pathways, such as object recognition and visually guided attention, respectively.

A tail of two sides: Artificially doubled false positive rates in neuroimaging due to the sidedness choice with t-tests.

One-sided t-tests are widely used in neuroimaging data analysis. While such a test may be applicable when investigating specific regions and prior information about directionality is present, we argue here that it is often mis-applied, with severe consequences for false positive rate (FPR) control. Conceptually, a pair of one-sided t-tests conducted in tandem (e.g., to test separately for both positive and negative effects), effectively amounts to a two-sided t-test. However, replacing the two-sided test with a pair of one-sided tests without multiple comparisons correction essentially doubles the intended FPR of statements made about the same study; that is, the actual family-wise error (FWE) of results at the whole brain level would be 10% instead of the 5% intended by the researcher. Therefore, we strongly recommend that, unless otherwise explicitly justified, two-sided t-tests be applied instead of two simultaneous one-sided t-tests.

Neural mechanisms of face emotion processing in youths and adults with bipolar disorder.

OBJECTIVES: Little is known about potential differences in the pathophysiology of bipolar disorder (BD) across development. The present study aimed to characterize age-related neural mechanisms of BD. METHODS: Youths and adults with and without BD (N = 108, age range = 9.8-55.9 years) completed an emotional face labeling task during fMRI acquisition. We leveraged three different fMRI analytic tools to identify age-related neural mechanisms of BD, investigating (a) change in neural responses over the course of the task, (b) neural activation averaged across the entire task, and (c) amygdala functional connectivity. RESULTS: We found converging Age Group × Diagnosis patterns across all three analytic methods. Compared to healthy youths vs adults, youths vs adults with BD show an altered pattern in response to repeated presentation of emotional faces in medial prefrontal, amygdala, and temporoparietal regions, as well as amygdala-temporoparietal connectivity. Specifically, medial prefrontal and lingual activation decreases over the course of repeated emotional face presentations in healthy youths vs adults but increases in youths with BD compared to adults with BD. Moreover, youths vs adults with BD show less medial prefrontal activation and amygdala-temporoparietal junction connectivity averaged over the task, but this difference is not found for healthy youths vs adults. CONCLUSION: Although longitudinal confirmation and replication will be necessary, these findings suggest that neural development may be aberrant in BD and that some neural mechanisms mediating BD may differ in adults vs children with the illness.

Comparison of heritability estimates on resting state fMRI connectivity phenotypes using the ENIGMA analysis pipeline.

We measured and compared heritability estimates for measures of functional brain connectivity extracted using the Enhancing Neuroimaging Genetics through Meta-Analysis (ENIGMA) rsfMRI analysis pipeline in two cohorts: the genetics of brain structure (GOBS) cohort and the HCP (the Human Connectome Project) cohort. These two cohorts were assessed using conventional (GOBS) and advanced (HCP) rsfMRI protocols, offering a test case for harmonization of rsfMRI phenotypes, and to determine measures that show consistent heritability for in-depth genome-wide analysis. The GOBS cohort consisted of 334 Mexican-American individuals (124M/210F, average age = 47.9 ± 13.2 years) from 29 extended pedigrees (average family size = 9 people; range 5-32). The GOBS rsfMRI data was collected using a 7.5-min acquisition sequence (spatial resolution = 1.72 × 1.72 × 3 mm3 ). The HCP cohort consisted of 518 twins and family members (240M/278F; average age = 28.7 ± 3.7 years). rsfMRI data was collected using 28.8-min sequence (spatial resolution = 2 × 2 × 2 mm3 ). We used the single-modality ENIGMA rsfMRI preprocessing pipeline to estimate heritability values for measures from eight major functional networks, using (1) seed-based connectivity and (2) dual regression approaches. We observed significant heritability (h2 = 0.2-0.4, p < .05) for functional connections from seven networks across both cohorts, with a significant positive correlation between heritability estimates across two cohorts. The similarity in heritability estimates for resting state connectivity measurements suggests that the additive genetic contribution to functional connectivity is robustly detectable across populations and imaging acquisition parameters. The overarching genetic influence, and means to consistently detect it, provides an opportunity to define a common genetic search space for future gene discovery studies.

Methods for cleaning the BOLD fMRI signal.

Blood oxygen-level-dependent functional magnetic resonance imaging (BOLD fMRI) has rapidly become a popular technique for the investigation of brain function in healthy individuals, patients as well as in animal studies. However, the BOLD signal arises from a complex mixture of neuronal, metabolic and vascular processes, being therefore an indirect measure of neuronal activity, which is further severely corrupted by multiple non-neuronal fluctuations of instrumental, physiological or subject-specific origin. This review aims to provide a comprehensive summary of existing methods for cleaning the BOLD fMRI signal. The description is given from a methodological point of view, focusing on the operation of the different techniques in addition to pointing out the advantages and limitations in their application. Since motion-related and physiological noise fluctuations are two of the main noise components of the signal, techniques targeting their removal are primarily addressed, including both data-driven approaches and using external recordings. Data-driven approaches, which are less specific in the assumed model and can simultaneously reduce multiple noise fluctuations, are mainly based on data decomposition techniques such as principal and independent component analysis. Importantly, the usefulness of strategies that benefit from the information available in the phase component of the signal, or in multiple signal echoes is also highlighted. The use of global signal regression for denoising is also addressed. Finally, practical recommendations regarding the optimization of the preprocessing pipeline for the purpose of denoising and future venues of research are indicated. Through the review, we summarize the importance of signal denoising as an essential step in the analysis pipeline of task-based and resting state fMRI studies.

An improved model of motion-related signal changes in fMRI.

Head motion is a significant source of noise in the estimation of functional connectivity from resting-state functional MRI (rs-fMRI). Current strategies to reduce this noise include image realignment, censoring time points corrupted by motion, and including motion realignment parameters and their derivatives as additional nuisance regressors in the general linear model. However, this nuisance regression approach assumes that the motion-induced signal changes are linearly related to the estimated realignment parameters, which is not always the case. In this study we develop an improved model of motion-related signal changes, where nuisance regressors are formed by first rotating and translating a single brain volume according to the estimated motion, re-registering the data, and then performing a principal components analysis (PCA) on the resultant time series of both moved and re-registered data. We show that these "Motion Simulated (MotSim)" regressors account for significantly greater fraction of variance, result in higher temporal signal-to-noise, and lead to functional connectivity estimates that are less affected by motion compared to the most common current approach of using the realignment parameters and their derivatives as nuisance regressors. This improvement should lead to more accurate estimates of functional connectivity, particularly in populations where motion is prevalent, such as patients and young children.

Untangling the relatedness among correlations, Part II: Inter-subject correlation group analysis through linear mixed-effects modeling.

It has been argued that naturalistic conditions in FMRI studies provide a useful paradigm for investigating perception and cognition through a synchronization measure, inter-subject correlation (ISC). However, one analytical stumbling block has been the fact that the ISC values associated with each single subject are not independent, and our previous paper (Chen et al., 2016) used simulations and analyses of real data to show that the methodologies adopted in the literature do not have the proper control for false positives. In the same paper, we proposed nonparametric subject-wise bootstrapping and permutation testing techniques for one and two groups, respectively, which account for the correlation structure, and these greatly outperformed the prior methods in controlling the false positive rate (FPR); that is, subject-wise bootstrapping (SWB) worked relatively well for both cases with one and two groups, and subject-wise permutation (SWP) testing was virtually ideal for group comparisons. Here we seek to explicate and adopt a parametric approach through linear mixed-effects (LME) modeling for studying the ISC values, building on the previous correlation framework, with the benefit that the LME platform offers wider adaptability, more powerful interpretations, and quality control checking capability than nonparametric methods. We describe both theoretical and practical issues involved in the modeling and the manner in which LME with crossed random effects (CRE) modeling is applied. A data-doubling step further allows us to conveniently track the subject index, and achieve easy implementations. We pit the LME approach against the best nonparametric methods, and find that the LME framework achieves proper control for false positives. The new LME methodologies are shown to be both efficient and robust, and they will be publicly available in AFNI (http://afni.nimh.nih.gov).

Imperfect (de)convolution may introduce spurious psychophysiological interactions and how to avoid it.

Psychophysiological interaction (PPI) is a widely used regression-based method to study connectivity changes in different experimental conditions. A PPI effect is generated by point-by-point multiplication of a psychological variable (experimental design) and a physiological variable (time series of a seed region). If the psychological variable is non-centered with a constant component, the constant component will add a physiological variable to the PPI term. The physiological component would in theory be accounted for by the physiological main effect in the model. But due to imperfect deconvolution and convolution with hemodynamic response function, the physiological component in PPI may no longer be exactly the same as the physiological main effect. This issue was illustrated by analyzing two block-designed fMRI datasets, one simple visual checkerboard task and a set of different tasks designed to activate different hemispheres. When PPI was calculated with deconvolution but without centering, significant results were usually observed between regions that are known to have baseline functional connectivity. These results could be suppressed by simply centering the psychological variable when calculating the PPI term or adding a deconvolve-reconvolved version of the physiological covariate. The PPI results with centering and with deconvolve-reconvolved physiological covariate are consistent with an explicit test for differences in coupling between conditions. It was, therefore, suggested that centering of the psychological variable or the addition of a deconvolve-reconvolved covariate is necessary for PPI analysis. Hum Brain Mapp 38:1723-1740, 2017. © 2017 Wiley Periodicals, Inc.

Untangling the relatedness among correlations, part I: Nonparametric approaches to inter-subject correlation analysis at the group level.

FMRI data acquisition under naturalistic and continuous stimuli (e.g., watching a video or listening to music) has become popular recently due to the fact that it entails less manipulation and more realistic/complex contexts involved in the task, compared to the conventional task-based experimental designs. The synchronization or response similarities among subjects are typically measured through inter-subject correlation (ISC) between any pair of subjects. At the group level, summarizing the collection of ISC values is complicated by their intercorrelations, which necessarily lead to the violation of independence assumed in typical parametric approaches such as Student's t-test. Nonparametric methods, such as bootstrapping and permutation testing, have previously been adopted for testing purposes by resampling the time series of each subject, but the quantitative validity of these specific approaches in terms of controllability of false positive rate (FPR) has never been explored before. Here we survey the methods of ISC group analysis that have been employed in the literature, and discuss the issues involved in those methods. We then propose less computationally intensive nonparametric methods that can be performed at the group level (for both one- and two-sample analyses), as compared to the popular method of circularly shifting the EPI time series at the individual level. As part of the new approaches, subject-wise (SW) resampling is adopted instead of element-wise (EW) resampling, so that exchangeability and independence assumptions are satisfied, and the patterned correlation structure among the ISC values can be more accurately captured. We examine the FPR controllability and power achievement of all the methods through simulations, as well as their performance when applied to a real experimental dataset.

Processing, evaluating and understanding FMRI data with afni_proc.py.

FMRI data are noisy, complicated to acquire, and typically go through many steps of processing before they are used in a study or clinical practice. Being able to visualize and understand the data from the start through the completion of processing, while being confident that each intermediate step was successful, is challenging. AFNI's afni_proc.py is a tool to create and run a processing pipeline for FMRI data. With its flexible features, afni_proc.py allows users to both control and evaluate their processing at a detailed level. It has been designed to keep users informed about all processing steps: it does not just process the data, but first outputs a fully commented processing script that the users can read, query, interpret and refer back to. Having this full provenance is important for being able to understand each step of processing; it also promotes transparency and reproducibility by keeping the record of individual-level processing and modeling specifics in a single, shareable place. Additionally, afni_proc.py creates pipelines that contain several automatic self-checks for potential problems during runtime. The output directory contains a dictionary of relevant quantities that can be programmatically queried for potential issues and a systematic, interactive quality control (QC) HTML. All of these features help users evaluate and understand their data and processing in detail. We describe these and other aspects of afni_proc.py here using a set of task-based and resting state FMRI example commands.

A Set of FMRI Quality Control Tools in AFNI: Systematic, in-depth and interactive QC with afni_proc.py and more.

Quality control (QC) assessment is a vital part of FMRI processing and analysis, and a typically under-discussed aspect of reproducibility. This includes checking datasets at their very earliest stages (acquisition and conversion) through their processing steps (e.g., alignment and motion correction) to regression modeling (correct stimuli, no collinearity, valid fits, enough degrees of freedom, etc.) for each subject. There are a wide variety of features to verify throughout any single subject processing pipeline, both quantitatively and qualitatively. We present several FMRI preprocessing QC features available in the AFNI toolbox, many of which are automatically generated by the pipeline-creation tool, afni_proc.py. These items include: a modular HTML document that covers full single subject processing from the raw data through statistical modeling; several review scripts in the results directory of processed data; and command line tools for identifying subjects with one or more quantitative properties across a group (such as triaging warnings, making exclusion criteria or creating informational tables). The HTML itself contains several buttons that efficiently facilitate interactive investigations into the data, when deeper checks are needed beyond the systematic images. The pages are linkable, so that users can evaluate individual items across a group, for increased sensitivity to differences (e.g., in alignment or regression modeling images). Finally, the QC document contains rating buttons for each "QC block", as well as comment fields for each, to facilitate both saving and sharing the evaluations. This increases the specificity of QC, as well as its shareability, as these files can be shared with others and potentially uploaded into repositories, promoting transparency and open science. We describe the features and applications of these QC tools for FMRI.

A Set of FMRI Quality Control Tools in AFNI: Systematic, in-depth, and interactive QC with afni_proc.py and more.

Quality control (QC) assessment is a vital part of FMRI processing and analysis, and a typically underdiscussed aspect of reproducibility. This includes checking datasets at their very earliest stages (acquisition and conversion) through their processing steps (e.g., alignment and motion correction) to regression modeling (correct stimuli, no collinearity, valid fits, enough degrees of freedom, etc.) for each subject. There are a wide variety of features to verify throughout any single-subject processing pipeline, both quantitatively and qualitatively. We present several FMRI preprocessing QC features available in the AFNI toolbox, many of which are automatically generated by the pipeline-creation tool, afni_proc.py. These items include a modular HTML document that covers full single-subject processing from the raw data through statistical modeling, several review scripts in the results directory of processed data, and command line tools for identifying subjects with one or more quantitative properties across a group (such as triaging warnings, making exclusion criteria, or creating informational tables). The HTML itself contains several buttons that efficiently facilitate interactive investigations into the data, when deeper checks are needed beyond the systematic images. The pages are linkable, so that users can evaluate individual items across a group, for increased sensitivity to differences (e.g., in alignment or regression modeling images). Finally, the QC document contains rating buttons for each "QC block," as well as comment fields for each, to facilitate both saving and sharing the evaluations. This increases the specificity of QC, as well as its shareability, as these files can be shared with others and potentially uploaded into repositories, promoting transparency and open science. We describe the features and applications of these QC tools for FMRI.

SUMA.

Surface-based brain imaging analysis offers the advantages of preserving the topology of cortical activation, increasing statistical power of group-level statistics, estimating cortical thickness, and visualizing with ease the pattern of activation across the whole cortex. SUMA is an open-source suite of programs for performing surface-based analysis and visualization. It was designed since its inception to allow for a fine control over the mapping between volume and surface domains, and for very fast and simultaneous display of multiple surface models and corresponding multitudes of datasets, all while maintaining a direct two-way link to volumetric data from which surface models and data originated. SUMA provides tools for performing spatial operations such as controlled smoothing, clustering, and interactive ROI drawing on folded surfaces in 3D, in addition to the various level-1 and level-2 FMRI statistics including FDR and FWE correction for multiple comparisons. In our contribution to this commemorative issue of Neuroimage we touch on the importance of surface-based analysis and provide a historic backdrop that motivated the creation of SUMA. We also highlight features that are particular to SUMA, notably the standardization procedure of meshes to greatly facilitate group-level analyses, and the ability to control SUMA's graphical interface from external programs making it possible to handle large collections of data with relative ease.