Variability in the analysis of a single neuroimaging dataset by many teams.

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.

Not All Stroop-Type Tasks Are Alike: Assessing the Impact of Stimulus Material, Task Design, and Cognitive Demand via Meta-analyses Across Neuroimaging Studies.

The Stroop effect is one of the most often studied examples of cognitive conflict processing. Over time, many variants of the classic Stroop task were used, including versions with different stimulus material, control conditions, presentation design, and combinations with additional cognitive demands. The neural and behavioral impact of this experimental variety, however, has never been systematically assessed. We used activation likelihood meta-analysis to summarize neuroimaging findings with Stroop-type tasks and to investigate whether involvement of the multiple-demand network (anterior insula, lateral frontal cortex, intraparietal sulcus, superior/inferior parietal lobules, midcingulate cortex, and pre-supplementary motor area) can be attributed to resolving some higher-order conflict that all of the tasks have in common, or if aspects that vary between task versions lead to specialization within this network. Across 133 neuroimaging experiments, incongruence processing in the color-word Stroop variant consistently recruited regions of the multiple-demand network, with modulation of spatial convergence by task variants. In addition, the neural patterns related to solving Stroop-like interference differed between versions of the task that use different stimulus material, with the only overlap between color-word, emotional picture-word, and other types of stimulus material in the posterior medial frontal cortex and right anterior insula. Follow-up analyses on behavior reported in these studies (in total 164 effect sizes) revealed only little impact of task variations on the mean effect size of reaction time. These results suggest qualitative processing differences among the family of Stroop variants, despite similar task difficulty levels, and should carefully be considered when planning or interpreting Stroop-type neuroimaging experiments.

Brain functional characterization of response-code conflict in dual-tasking and its modulation by age.

Crosstalk between conflicting response codes contributes to interference in dual-tasking, an effect exacerbated in advanced age. Here, we investigated (i) brain activity correlates of such response-code conflicts, (ii) activity modulations by individual dual-task performance and related cognitive abilities, (iii) task-modulated connectivity within the task network, and (iv) age-related differences in all these aspects. Young and older adults underwent fMRI while responding to the pitch of tones through spatially mapped speeded button presses with one or two hands concurrently. Using opposing stimulus-response mappings between hands, we induced conflict between simultaneously activated response codes. These response-code conflicts elicited activation in key regions of the multiple-demand network. While thalamic and parietal areas of the conflict-related network were modulated by attentional, working-memory and task-switching abilities, efficient conflict resolution in dual-tasking mainly relied on increasing supplementary motor activity. Older adults showed non-compensatory hyperactivity in left superior frontal gyrus, and higher right premotor activity was modulated by working-memory capacity. Finally, connectivity between premotor or parietal seed regions and the conflict-sensitive network was neither conflict-specific nor age-sensitive. Overall, resolving dual-task response-code conflict recruited substantial parts of the multiple-demand network, whose activity and coupling, however, were only little affected by individual differences in task performance or age.

Predicting executive functioning from functional brain connectivity: network specificity and age effects.

Healthy aging is associated with altered executive functioning (EF). Earlier studies found age-related differences in EF performance to be partially accounted for by changes in resting-state functional connectivity (RSFC) within brain networks associated with EF. However, it remains unclear which role RSFC in EF-associated networks plays as a marker for individual differences in EF performance. Here, we investigated to what degree individual abilities across 3 different EF tasks can be predicted from RSFC within EF-related, perceptuo-motor, whole-brain, and random networks separately in young and old adults. Specifically, we were interested if (i) young and old adults differ in predictability depending on network or EF demand level (high vs. low), (ii) an EF-related network outperforms EF-unspecific networks when predicting EF abilities, and (iii) this pattern changes with demand level. Both our uni- and multivariate analysis frameworks analyzing interactions between age × demand level × networks revealed overall low prediction accuracies and a general lack of specificity regarding neurobiological networks for predicting EF abilities. This questions the idea of finding markers for individual EF performance in RSFC patterns and calls for future research replicating the current approach in different task states, brain modalities, different, larger samples, and with more comprehensive behavioral measures.

Predicting executive functioning from brain networks: modality specificity and age effects.

Healthy aging is associated with structural and functional network changes in the brain, which have been linked to deterioration in executive functioning (EF), while their neural implementation at the individual level remains unclear. As the biomarker potential of individual resting-state functional connectivity (RSFC) patterns has been questioned, we investigated to what degree individual EF abilities can be predicted from the gray-matter volume (GMV), regional homogeneity, fractional amplitude of low-frequency fluctuations (fALFF), and RSFC within EF-related, perceptuo-motor, and whole-brain networks in young and old adults. We examined whether the differences in out-of-sample prediction accuracy were modality-specific and depended on age or task-demand levels. Both uni- and multivariate analysis frameworks revealed overall low prediction accuracies and moderate-to-weak brain-behavior associations (R2 < 0.07, r < 0.28), further challenging the idea of finding meaningful markers for individual EF performance with the metrics used. Regional GMV, well linked to overall atrophy, carried the strongest information about individual EF differences in older adults, whereas fALFF, measuring functional variability, did so for younger adults. Our study calls for future research analyzing more global properties of the brain, different task-states and applying adaptive behavioral testing to result in sensitive predictors for young and older adults, respectively.

Evaluation of thresholding methods for activation likelihood estimation meta-analysis via large-scale simulations.

In recent neuroimaging studies, threshold-free cluster enhancement (TFCE) gained popularity as a sophisticated thresholding method for statistical inference. It was shown to feature higher sensitivity than the frequently used approach of controlling the cluster-level family-wise error (cFWE) and it does not require setting a cluster-forming threshold at voxel level. Here, we examined the applicability of TFCE to a widely used method for coordinate-based neuroimaging meta-analysis, Activation Likelihood Estimation (ALE), by means of large-scale simulations. We created over 200,000 artificial meta-analysis datasets by independently varying the total number of experiments included and the amount of spatial convergence across experiments. Next, we applied ALE to all datasets and compared the performance of TFCE to both voxel-level and cluster-level FWE correction approaches. All three multiple-comparison correction methods yielded valid results, with only about 5% of the significant clusters being based on spurious convergence, which corresponds to the nominal level the methods were controlling for. On average, TFCE's sensitivity was comparable to that of cFWE correction, but it was slightly worse for a subset of parameter combinations, even after TFCE parameter optimization. cFWE yielded the largest significant clusters, closely followed by TFCE, while voxel-level FWE correction yielded substantially smaller clusters, showcasing its high spatial specificity. Given that TFCE does not outperform the standard cFWE correction but is computationally much more expensive, we conclude that employing TFCE for ALE cannot be recommended to the general user.

The Aging Brain and Executive Functions Revisited: Implications from Meta-analytic and Functional-Connectivity Evidence.

Healthy aging is associated with changes in cognitive performance, including executive functions (EFs) and their associated brain activation patterns. However, it has remained unclear which EF-related brain regions are affected consistently, because the results of pertinent neuroimaging studies and earlier meta-analyses vary considerably. We, therefore, conducted new rigorous meta-analyses of published age differences in EF-related brain activity. Out of a larger set of regions associated with EFs, only left inferior frontal junction and left anterior cuneus/precuneus were found to show consistent age differences. To further characterize these two age-sensitive regions, we performed seed-based resting-state functional connectivity (RS-FC) analyses using fMRI data from a large adult sample with a wide age range. We also assessed associations of the two regions' whole-brain RS-FC patterns with age and EF performance. Although our results largely point toward a domain-general role of left inferior frontal junction in EFs, the pattern of individual study contributions to the meta-analytic results suggests process-specific modulations by age. Our analyses further indicate that the left anterior cuneus/precuneus is recruited differently by older (compared with younger) adults during EF tasks, potentially reflecting inefficiencies in switching the attentional focus. Overall, our findings question earlier meta-analytic results and suggest a larger heterogeneity of age-related differences in brain activity associated with EFs. Hence, they encourage future research that pays greater attention to replicability, investigates age-related differences in deactivation, and focuses on more narrowly defined EF subprocesses, combining multiple behavioral assessments with multimodal imaging.

The heterogeneity of the left dorsal premotor cortex evidenced by multimodal connectivity-based parcellation and functional characterization.

Despite the common conception of the dorsal premotor cortex (PMd) as a single brain region, its diverse connectivity profiles and behavioral heterogeneity argue for a differentiated organization of the PMd. A previous study revealed that the right PMd is characterized by a rostro-caudal and a ventro-dorsal distinction dividing it into five subregions: rostral, central, caudal, ventral and dorsal. The present study assessed whether a similar organization is present in the left hemisphere, by capitalizing on a multimodal data-driven approach combining connectivity-based parcellation (CBP) based on meta-analytic modeling, resting-state functional connectivity, and probabilistic diffusion tractography. The resulting PMd modules were then characterized based on multimodal functional connectivity and a quantitative analysis of associated behavioral functions. Analyzing the clusters consistent across all modalities revealed an organization of the left PMd that mirrored its right counterpart to a large degree. Again, caudal, central and rostral modules reflected a cognitive-motor gradient and a premotor eye-field was found in the ventral part of the left PMd. In addition, a distinct module linked to abstract cognitive functions was observed in the rostro-ventral left PMd across all CBP modalities, implying greater differentiation of higher cognitive functions for the left than the right PMd.

The Right Dorsal Premotor Mosaic: Organization, Functions, and Connectivity.

The right dorsal premotor cortex (PMd) of humans has been reported to be involved in a broad range of motor and cognitive functions. We explored the basis of this behavioral heterogeneity by performing a connectivity-based parcellation using meta-analytic approach applied to PMd coactivations. We compared our connectivity-based parcellation results with parcellations obtained through resting-state functional connectivity and probabilistic diffusion tractography. Functional connectivity profiles and behavioral decoding of the resulting PMd subregions allowed characterizing their respective behavior profile. These procedures divided the right PMd into 5 distinct subregions that formed a cognitive-motor gradient along a rostro-caudal axis. In particular, we found 1) a rostral subregion functionally connected with prefrontal cortex, which likely supports high-level cognitive processes, such as working memory, 2) a central subregion showing a mixed behavioral profile and functional connectivity to parietal regions of the dorsal attention network, and 3) a caudal subregion closely integrated with the motor system. Additionally, we found 4) a dorsal subregion, preferentially related to hand movements and connected to both cognitive and motor regions, and 5) a ventral subregion, whose functional profile fits the concept of an eye movement-related field. In conclusion, right PMd may be considered as a functional mosaic formed by 5 subregions.

Neural correlates of formal thought disorder: An activation likelihood estimation meta-analysis.

Formal thought disorder (FTD) refers to a psychopathological dimension characterized by disorganized and incoherent speech. Whether symptoms of FTD arise from aberrant processing in language-related regions or more general cognitive networks, however, remains debated. Here, we addressed this question by a quantitative meta-analysis of published functional neuroimaging studies on FTD. The revised Activation Likelihood Estimation (ALE) algorithm was used to test for convergent aberrant activation changes in 18 studies (30 experiments) investigating FTD, of which 17 studies comprised schizophrenia patients and one study healthy subjects administered to S-ketamine. Additionally, we analyzed task-dependent and task-independent (resting-state) functional connectivity (FC) of brain regions showing convergence in activation changes. Subsequent functional characterization was performed for the initial clusters and the delineated connectivity networks by reference to the BrainMap database. Consistent activation changes were found in the left superior temporal gyrus (STG) and two regions within the left posterior middle temporal gyrus (p-MTG), ventrally (vp-MTG) and dorsally (dp-MTG). Functional characterization revealed a prominent functional association of ensuing clusters from our ALE meta-analysis with language and speech processing, as well as auditory perception in STG and with social cognition in dp-MTG. FC analysis identified task-dependent and task-independent networks for all three seed regions, which were mainly related to language and speech processing, but showed additional involvement in higher order cognitive functions. Our findings suggest that FTD is mainly characterized by abnormal activation in brain regions of the left hemisphere that are associated with language and speech processing, but also extend to higher order cognitive functions. Hum Brain Mapp 38:4946-4965, 2017. © 2017 Wiley Periodicals, Inc.

On the integrity of functional brain networks in schizophrenia, Parkinson's disease, and advanced age: Evidence from connectivity-based single-subject classification.

Previous whole-brain functional connectivity studies achieved successful classifications of patients and healthy controls but only offered limited specificity as to affected brain systems. Here, we examined whether the connectivity patterns of functional systems affected in schizophrenia (SCZ), Parkinson's disease (PD), or normal aging equally translate into high classification accuracies for these conditions. We compared classification performance between pre-defined networks for each group and, for any given network, between groups. Separate support vector machine classifications of 86 SCZ patients, 80 PD patients, and 95 older adults relative to their matched healthy/young controls, respectively, were performed on functional connectivity in 12 task-based, meta-analytically defined networks using 25 replications of a nested 10-fold cross-validation scheme. Classification performance of the various networks clearly differed between conditions, as those networks that best classified one disease were usually non-informative for the other. For SCZ, but not PD, emotion-processing, empathy, and cognitive action control networks distinguished patients most accurately from controls. For PD, but not SCZ, networks subserving autobiographical or semantic memory, motor execution, and theory-of-mind cognition yielded the best classifications. In contrast, young-old classification was excellent based on all networks and outperformed both clinical classifications. Our pattern-classification approach captured associations between clinical and developmental conditions and functional network integrity with a higher level of specificity than did previous whole-brain analyses. Taken together, our results support resting-state connectivity as a marker of functional dysregulation in specific networks known to be affected by SCZ and PD, while suggesting that aging affects network integrity in a more global way. Hum Brain Mapp 38:5845-5858, 2017. © 2017 Wiley Periodicals, Inc.

Is there "one" DLPFC in cognitive action control? Evidence for heterogeneity from co-activation-based parcellation.

The dorsolateral prefrontal cortex (DLPFC) has consistently been implicated in cognitive control of motor behavior. There is, however, considerable variability in the exact location and extension of these activations across functional magnetic resonance imaging (fMRI) experiments. This poses the question of whether this variability reflects sampling error and spatial uncertainty in fMRI experiments or structural and functional heterogeneity of this region. This study shows that the right DLPFC as observed in 4 different experiments tapping executive action control may be subdivided into 2 distinct subregions-an anterior-ventral and a posterior-dorsal one -based on their whole-brain co-activation patterns across neuroimaging studies. Investigation of task-dependent and task-independent connectivity revealed both clusters to be involved in distinct neural networks. The posterior subregion showed increased connectivity with bilateral intraparietal sulci, whereas the anterior subregion showed increased connectivity with the anterior cingulate cortex. Functional characterization with quantitative forward and reverse inferences revealed the anterior network to be more strongly associated with attention and action inhibition processes, whereas the posterior network was more strongly related to action execution and working memory. The present data provide evidence that cognitive action control in the right DLPFC may rely on differentiable neural networks and cognitive functions.

Success versus failure in cognitive control: Meta-analytic evidence from neuroimaging studies on error processing.

Brain mechanisms of error processing have often been investigated using response interference tasks and focusing on the posterior medial frontal cortex, which is also implicated in resolving response conflict in general. Thereby, the role other brain regions may play has remained undervalued. Here, activation likelihood estimation meta-analyses were used to synthesize the neuroimaging literature on brain activity related to committing errors versus responding successfully in interference tasks and to test for commonalities and differences. The salience network and the temporoparietal junction were commonly recruited irrespective of whether responses were correct or incorrect, pointing towards a general involvement in coping with situations that call for increased cognitive control. The dorsal posterior cingulate cortex, posterior thalamus, and left superior frontal gyrus showed error-specific convergence, which underscores their consistent involvement when performance goals are not met. In contrast, successful responding revealed stronger convergence in the dorsal attention network and lateral prefrontal regions. Underrecruiting these regions in error trials may reflect failures in activating the task-appropriate stimulus-response contingencies necessary for successful response execution.

The effect of task complexity on the neural network for response inhibition: An ALE meta-analysis.

Response inhibition is classically investigated using the go/no-go (GNGT) and stop-signal task (SST), which conceptually measure different subprocesses of inhibition. Further, different task versions with varying levels of additional executive control demands exist, making it difficult to identify the core neural correlates of response inhibition independent of variations in task complexity. Using neuroimaging meta-analyses, we show that a divergent pattern of regions is consistently involved in the GNGT versus SST, arguing for different mechanisms involved when performing the two tasks. Further, for the GNGT a strong effect of task complexity was found, with regions of the multiple demand network (MDN) consistently involved particularly in the complex GNGT. In contrast, both standard and complex SST recruited the MDN to a similar degree. These results complement behavioral evidence suggesting that inhibitory control becomes automatic after some practice and is performed without input of higher control regions in the classic, standard GNGT, but continues to be implemented in a top-down controlled fashion in the SST.

Characterizing functional modules in the human thalamus: coactivation-based parcellation and systems-level functional decoding.

The human thalamus relays sensory signals to the cortex and facilitates brain-wide communication. The thalamus is also more directly involved in sensorimotor and various cognitive functions but a full characterization of its functional repertoire, particularly in regard to its internal anatomical structure, is still outstanding. As a putative hub in the human connectome, the thalamus might reveal its functional profile only in conjunction with interconnected brain areas. We therefore developed a novel systems-level Bayesian reverse inference decoding that complements the traditional neuroinformatics approach towards a network account of thalamic function. The systems-level decoding considers the functional repertoire (i.e., the terms associated with a brain region) of all regions showing co-activations with a predefined seed region in a brain-wide fashion. Here, we used task-constrained meta-analytic connectivity-based parcellation (MACM-CBP) to identify thalamic subregions as seed regions and applied the systems-level decoding to these subregions in conjunction with functionally connected cortical regions. Our results confirm thalamic structure-function relationships known from animal and clinical studies and revealed further associations with language, memory, and locomotion that have not been detailed in the cognitive neuroscience literature before. The systems-level decoding further uncovered large systems engaged in autobiographical memory and nociception. We propose this novel decoding approach as a useful tool to detect previously unknown structure-function relationships at the brain network level, and to build viable starting points for future studies.

Success versus failure in cognitive control: meta-analytic evidence from neuroimaging studies on error processing.

Brain mechanisms of error processing have often been investigated using response interference tasks and focusing on the posterior medial frontal cortex, which is also implicated in resolving response conflict in general. Thereby, the role other brain regions may play has remained undervalued. Here, activation likelihood estimation meta-analyses were used to synthesize the neuroimaging literature on brain activity related to committing errors versus responding successfully in interference tasks and to test for commonalities and differences. The salience network and the temporoparietal junction were commonly recruited irrespective of whether responses were correct or incorrect, pointing towards a general involvement in coping with situations that call for increased cognitive control. The dorsal posterior cingulate cortex, posterior thalamus, and left superior frontal gyrus showed error-specific convergence, which underscores their consistent involvement when performance goals are not met. In contrast, successful responding revealed stronger convergence in the dorsal attention network and lateral prefrontal regions. Underrecruiting these regions in error trials may reflect failures in activating the task-appropriate stimulus-response contingencies necessary for successful response execution.

Predicting Executive Functioning from Brain Networks: Modality Specificity and Age Effects.

Healthy aging is associated with structural and functional network changes in the brain, which have been linked to deterioration in executive functioning (EF), while their neural implementation at the individual level remains unclear. As the biomarker potential of individual resting-state functional connectivity (RSFC) patterns has been questioned, we investigated to what degree individual EF abilities can be predicted from gray-matter volume (GMV), regional homogeneity, fractional amplitude of low-frequency fluctuations (fALFF), and RSFC within EF-related, perceptuo-motor, and whole-brain networks in young and old adults. We examined whether differences in out-of-sample prediction accuracy were modality-specific and depended on age or task-demand levels. Both uni- and multivariate analysis frameworks revealed overall low prediction accuracies and moderate to weak brain-behavior associations (R2 < .07, r < .28), further challenging the idea of finding meaningful markers for individual EF performance with the metrics used. Regional GMV, well linked to overall atrophy, carried the strongest information about individual EF differences in older adults, whereas fALFF, measuring functional variability, did so for younger adults. Our study calls for future research analyzing more global properties of the brain, different task-states and applying adaptive behavioral testing to result in sensitive predictors for young and older adults, respectively.

Crossmodal interactions in audiovisual emotion processing.

Emotion in daily life is often expressed in a multimodal fashion. Consequently emotional information from one modality can influence processing in another. In a previous fMRI study we assessed the neural correlates of audio-visual integration and found that activity in the left amygdala is significantly attenuated when a neutral stimulus is paired with an emotional one compared to conditions where emotional stimuli were present in both channels. Here we used dynamic causal modelling to investigate the effective connectivity in the neuronal network underlying this emotion presence congruence effect. Our results provided strong evidence in favor of a model family, differing only in the interhemispheric interactions. All winning models share a connection from the bilateral fusiform gyrus (FFG) into the left amygdala and a non-linear modulatory influence of bilateral posterior superior temporal sulcus (pSTS) on these connections. This result indicates that the pSTS not only integrates multi-modal information from visual and auditory regions (as reflected in our model by significant feed-forward connections) but also gates the influence of the sensory information on the left amygdala, leading to attenuation of amygdala activity when a neutral stimulus is integrated. Moreover, we found a significant lateralization of the FFG due to stronger driving input by the stimuli (faces) into the right hemisphere, whereas such lateralization was not present for sound-driven input into the superior temporal gyrus. In summary, our data provides further evidence for a rightward lateralization of the FFG and in particular for a key role of the pSTS in the integration and gating of audio-visual emotional information.

Across-study and within-subject functional connectivity of a right temporo-parietal junction subregion involved in stimulus-context integration.

Bidirectional integration between sensory stimuli and contextual framing is fundamental to action control. Stimuli may entail context-dependent actions, while temporal or spatial characteristics of a stimulus train may establish a contextual framework for upcoming stimuli. Here we aimed at identifying core areas for stimulus-context integration and delineated their functional connectivity (FC) using meta-analytic connectivity modeling (MACM) and analysis of resting-state networks. In a multi-study conjunction, consistently increased activity under higher demands on stimulus-context integration was predominantly found in the right temporo-parietal junction (TPJ), which represented the largest cluster of overlap and was thus used as the seed for the FC analyses. The conjunction between task-dependent (MACM) and task-free (resting state) FC of the right TPJ revealed a shared network comprising bilaterally inferior parietal and frontal cortices, anterior insula, premotor cortex, putamen and cerebellum, i.e., a 'ventral' action/attention network. Stronger task-dependent (vs. task-free) connectivity was observed with the pre-SMA, dorsal premotor cortex, intraparietal sulcus, basal ganglia and primary sensori motor cortex, while stronger resting-state (vs. task-dependent) connectivity was found with the dorsolateral prefrontal and medial parietal cortex. Our data provide strong evidence that the right TPJ may represent a key region for the integration of sensory stimuli and contextual frames in action control. Task-dependent associations with regions related to stimulus processing and motor responses indicate that the right TPJ may integrate 'collaterals' of sensory processing and apply (ensuing) contextual frames, most likely via modulation of preparatory loops. Given the pattern of resting-state connectivity, internal states and goal representations may provide the substrates for the contextual integration within the TPJ in the absence of a specific task.