Variability in the analysis of a single neuroimaging dataset by many teams.

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.

Higher resting-state BNST-CeA connectivity is associated with greater corrugator supercilii reactivity to negatively valenced images.

The bed nucleus of the stria terminalis (BNST) and central nucleus of the amygdala (CeA) are hypothesized to be the output nodes of the extended amygdala threat response, integrating multiple signals to coordinate the threat response via outputs to the hypothalamus and brainstem. The BNST and CeA are structurally and functionally connected, suggesting interactions between these regions may regulate how the response to provocation unfolds. However, the relationship between human BNST-CeA connectivity and the behavioral response to affective stimuli is little understood. To investigate whether individual differences in BNST-CeA connectivity are related to the affective response to negatively valenced stimuli, we tested relations between resting-state BNST-CeA connectivity and both facial electromyographic (EMG) activity of the corrugator supercilii muscle and eyeblink startle magnitude during affective image presentation within the Refresher sample of the Midlife in the United States (MIDUS) study. We found that higher right BNST-CeA connectivity was associated with greater corrugator activity to negative, but not positive, images. There was a trend-level association between right BNST-CeA connectivity and trait negative affect. Eyeblink startle magnitude was not significantly related to BNST-CeA connectivity. These results suggest that functional interactions between BNST and CeA contribute to the behavioral response to negative emotional events.

Control Engagement During Sentence and Inhibition fMRI Tasks in Children With Reading Difficulties.

Recent reading research implicates executive control regions as sites of difference in struggling readers. However, as studies often employ only reading or language tasks, the extent of deviation in control engagement in children with reading difficulties is not known. The current study investigated activation in reading and executive control brain regions during both a sentence comprehension task and a nonlexical inhibitory control task in third-fifth grade children with and without reading difficulties. We employed both categorical (group-based) and individual difference approaches to relate reading ability to brain activity. During sentence comprehension, struggling readers had less activation in the left posterior temporal cortex, previously implicated in language, semantic, and reading research. Greater negative activity (relative to fixation) during sentence comprehension in a left inferior parietal region from the executive control literature correlated with poorer reading ability. Greater comprehension scores were associated with less dorsal anterior cingulate activity during the sentence comprehension task. Unlike the sentence task, there were no significant differences between struggling and nonstruggling readers for the nonlexical inhibitory control task. Thus, differences in executive control engagement were largely specific to reading, rather than a general control deficit across tasks in children with reading difficulties, informing future intervention research.

Impact of short- and long-term mindfulness meditation training on amygdala reactivity to emotional stimuli.

Meditation training can improve mood and emotion regulation, yet the neural mechanisms of these affective changes have yet to be fully elucidated. We evaluated the impact of long- and short-term mindfulness meditation training on the amygdala response to emotional pictures in a healthy, non-clinical population of adults using blood-oxygen level dependent functional magnetic resonance imaging. Long-term meditators (N = 30, 16 female) had 9081 h of lifetime practice on average, primarily in mindfulness meditation. Short-term training consisted of an 8-week Mindfulness- Based Stress Reduction course (N = 32, 22 female), which was compared to an active control condition (N = 35, 19 female) in a randomized controlled trial. Meditation training was associated with less amygdala reactivity to positive pictures relative to controls, but there were no group differences in response to negative pictures. Reductions in reactivity to negative stimuli may require more practice experience or concentrated practice, as hours of retreat practice in long-term meditators was associated with lower amygdala reactivity to negative pictures - yet we did not see this relationship for practice time with MBSR. Short-term training, compared to the control intervention, also led to increased functional connectivity between the amygdala and a region implicated in emotion regulation - ventromedial prefrontal cortex (VMPFC) - during affective pictures. Thus, meditation training may improve affective responding through reduced amygdala reactivity, and heightened amygdala-VMPFC connectivity during affective stimuli may reflect a potential mechanism by which MBSR exerts salutary effects on emotion regulation ability.

How Do You See Me? The Neural Basis of Motivated Meta-perception.

We cannot see the minds of others, yet people often spontaneously interpret how they are viewed by other people (i.e., meta-perceptions) and often in a self-flattering manner. Very little is known about the neural associations of meta-perceptions, but a likely candidate is the ventromedial pFC (VMPFC). VMPFC has been associated with both self- and other-perception as well as motivated self-perception. Does this function extend to meta-perceptions? The current study examined neural activity while participants made meta-perceptive interpretations in various social scenarios. A drift-diffusion model was used to test whether the VMPFC is associated with two processes involved in interpreting meta-perceptions in a self-flattering manner: the extent to which the interpretation process involves the preferential accumulation of evidence in favor of a self-flattering interpretation versus the extent to which the interpretation process begins with an expectation that favors a self-flattering outcome. Increased VMPFC activity was associated with the extent to which people preferentially accumulate information when interpreting meta-perceptions under ambiguous conditions and marginally associated with self-flattering meta-perceptions. Together, the present findings illuminate the neural underpinnings of a social cognitive process that has received little attention to date: how we make meaning of others' minds when we think those minds are pointed at us.

A comprehensive review of group level model performance in the presence of heteroscedasticity: Can a single model control Type I errors in the presence of outliers?

Even after thorough preprocessing and a careful time series analysis of functional magnetic resonance imaging (fMRI) data, artifact and other issues can lead to violations of the assumption that the variance is constant across subjects in the group level model. This is especially concerning when modeling a continuous covariate at the group level, as the slope is easily biased by outliers. Various models have been proposed to deal with outliers including models that use the first level variance or that use the group level residual magnitude to differentially weight subjects. The most typically used robust regression, implementing a robust estimator of the regression slope, has been previously studied in the context of fMRI studies and was found to perform well in some scenarios, but a loss of Type I error control can occur for some outlier settings. A second type of robust regression using a heteroscedastic autocorrelation consistent (HAC) estimator, which produces robust slope and variance estimates has been shown to perform well, with better Type I error control, but with large sample sizes (500-1000 subjects). The Type I error control with smaller sample sizes has not been studied in this model and has not been compared to other modeling approaches that handle outliers such as FSL's Flame 1 and FSL's outlier de-weighting. Focusing on group level inference with a continuous covariate over a range of sample sizes and degree of heteroscedasticity, which can be driven either by the within- or between-subject variability, both styles of robust regression are compared to ordinary least squares (OLS), FSL's Flame 1, Flame 1 with outlier de-weighting algorithm and Kendall's Tau. Additionally, subject omission using the Cook's Distance measure with OLS and nonparametric inference with the OLS statistic are studied. Pros and cons of these models as well as general strategies for detecting outliers in data and taking precaution to avoid inflated Type I error rates are discussed.

Reprint of 'A Comprehensive review of group level model performance in the presence of heteroscedasticity: Can a single model control Type I errors in the presence of outliers?'

Even after thorough preprocessing and a careful time series analysis of functional magnetic resonance imaging (fMRI) data, artifact and other issues can lead to violations of the assumption that the variance is constant across subjects in the group level model. This is especially concerning when modeling a continuous covariate at the group level, as the slope is easily biased by outliers. Various models have been proposed to deal with outliers including models that use the first level variance or that use the group level residual magnitude to differentially weight subjects. The most typically used robust regression, implementing a robust estimator of the regression slope, has been previously studied in the context of fMRI studies and was found to perform well in some scenarios, but a loss of Type I error control can occur for some outlier settings. A second type of robust regression using a heteroscedastic autocorrelation consistent (HAC) estimator, which produces robust slope and variance estimates has been shown to perform well, with better Type I error control, but with large sample sizes (500-1000 subjects). The Type I error control with smaller sample sizes has not been studied in this model and has not been compared to other modeling approaches that handle outliers such as FSL's Flame 1 and FSL's outlier de-weighting. Focusing on group level inference with a continuous covariate over a range of sample sizes and degree of heteroscedasticity, which can be driven either by the within- or between-subject variability, both styles of robust regression are compared to ordinary least squares (OLS), FSL's Flame 1, Flame 1 with outlier de-weighting algorithm and Kendall's Tau. Additionally, subject omission using the Cook's Distance measure with OLS and nonparametric inference with the OLS statistic are studied. Pros and cons of these models as well as general strategies for detecting outliers in data and taking precaution to avoid inflated Type I error rates are discussed.

The Role of Corticostriatal Systems in Speech Category Learning.

One of the most difficult category learning problems for humans is learning nonnative speech categories. While feedback-based category training can enhance speech learning, the mechanisms underlying these benefits are unclear. In this functional magnetic resonance imaging study, we investigated neural and computational mechanisms underlying feedback-dependent speech category learning in adults. Positive feedback activated a large corticostriatal network including the dorsolateral prefrontal cortex, inferior parietal lobule, middle temporal gyrus, caudate, putamen, and the ventral striatum. Successful learning was contingent upon the activity of domain-general category learning systems: the fast-learning reflective system, involving the dorsolateral prefrontal cortex that develops and tests explicit rules based on the feedback content, and the slow-learning reflexive system, involving the putamen in which the stimuli are implicitly associated with category responses based on the reward value in feedback. Computational modeling of response strategies revealed significant use of reflective strategies early in training and greater use of reflexive strategies later in training. Reflexive strategy use was associated with increased activation in the putamen. Our results demonstrate a critical role for the reflexive corticostriatal learning system as a function of response strategy and proficiency during speech category learning.

Predicting risky choices from brain activity patterns.

Previous research has implicated a large network of brain regions in the processing of risk during decision making. However, it has not yet been determined if activity in these regions is predictive of choices on future risky decisions. Here, we examined functional MRI data from a large sample of healthy subjects performing a naturalistic risk-taking task and used a classification analysis approach to predict whether individuals would choose risky or safe options on upcoming trials. We were able to predict choice category successfully in 71.8% of cases. Searchlight analysis revealed a network of brain regions where activity patterns were reliably predictive of subsequent risk-taking behavior, including a number of regions known to play a role in control processes. Searchlights with significant predictive accuracy were primarily located in regions more active when preparing to avoid a risk than when preparing to engage in one, suggesting that risk taking may be due, in part, to a failure of the control systems necessary to initiate a safe choice. Additional analyses revealed that subject choice can be successfully predicted with minimal decrements in accuracy using highly condensed data, suggesting that information relevant for risky choice behavior is encoded in coarse global patterns of activation as well as within highly local activation within searchlights.

Neural correlates of state-based decision-making in younger and older adults.

Older and younger adults performed a state-based decision-making task while undergoing functional MRI (fMRI). We proposed that younger adults would be more prone to base their decisions on expected value comparisons, but that older adults would be more reactive decision-makers who would act in response to recent changes in rewards or states, rather than on a comparison of expected values. To test this we regressed BOLD activation on two measures from a sophisticated reinforcement learning (RL) model. A value-based regressor was computed by subtracting the immediate value of the selected alternative from its long-term value. The other regressor was a state-change uncertainty signal that served as a proxy for whether the participant's state improved or declined, relative to the previous trial. Younger adults' activation was modulated by the value-based regressor in ventral striatal and medial PFC regions implicated in reinforcement learning. Older adults' activation was modulated by state-change uncertainty signals in right dorsolateral PFC, and activation in this region was associated with improved performance in the task. This suggests that older adults may depart from standard expected-value based strategies and recruit lateral PFC regions to engage in reactive decision-making strategies.

Influencing food choices by training: evidence for modulation of frontoparietal control signals.

To overcome unhealthy behaviors, one must be able to make better choices. Changing food preferences is an important strategy in addressing the obesity epidemic and its accompanying public health risks. However, little is known about how food preferences can be effectively affected and what neural systems support such changes. In this study, we investigated a novel extensive training paradigm where participants chose from specific pairs of palatable junk food items and were rewarded for choosing the items with lower subjective value over higher value ones. In a later probe phase, when choices were made for real consumption, participants chose the lower-valued item more often in the trained pairs compared with untrained pairs. We replicated the behavioral results in an independent sample of participants while they were scanned with fMRI. We found that, as training progressed, there was decreased recruitment of regions that have been previously associated with cognitive control, specifically the left dorsolateral pFC and bilateral parietal cortices. Furthermore, we found that connectivity of the left dorsolateral pFC was greater with primary motor regions by the end of training for choices of lower-valued items that required exertion of self-control, suggesting a formation of a stronger stimulus-response association. These findings demonstrate that it is possible to influence food choices through training and that this training is associated with a decreasing need for top-down frontoparietal control. The results suggest that training paradigms may be promising as the basis for interventions to influence real-world food preferences.

Decomposing decision components in the stop-signal task: a model-based approach to individual differences in inhibitory control.

The stop-signal task, in which participants must inhibit prepotent responses, has been used to identify neural systems that vary with individual differences in inhibitory control. To explore how these differences relate to other aspects of decision making, a drift-diffusion model of simple decisions was fitted to stop-signal task data from go trials to extract measures of caution, motor execution time, and stimulus processing speed for each of 123 participants. These values were used to probe fMRI data to explore individual differences in neural activation. Faster processing of the go stimulus correlated with greater activation in the right frontal pole for both go and stop trials. On stop trials, stimulus processing speed also correlated with regions implicated in inhibitory control, including the right inferior frontal gyrus, medial frontal gyrus, and BG. Individual differences in motor execution time correlated with activation of the right parietal cortex. These findings suggest a robust relationship between the speed of stimulus processing and inhibitory processing at the neural level. This model-based approach provides novel insight into the interrelationships among decision components involved in inhibitory control and raises interesting questions about strategic adjustments in performance and inhibitory deficits associated with psychopathology.

Bayesian networks for fMRI: a primer.

Bayesian network analysis is an attractive approach for studying the functional integration of brain networks, as it includes both the locations of connections between regions of the brain (functional connectivity) and more importantly the direction of the causal relationship between the regions (directed functional connectivity). Further, these approaches are more attractive than other functional connectivity analyses in that they can often operate on larger sets of nodes and run searches over a wide range of candidate networks. An important study by Smith et al. (2011) illustrated that many Bayesian network approaches did not perform well in identifying the directionality of connections in simulated single-subject data. Since then, new Bayesian network approaches have been developed that have overcome the failures in the Smith work. Additionally, an important discovery was made that shows a preprocessing step used in the Smith data puts some of the Bayesian network methods at a disadvantage. This work provides a review of Bayesian network analyses, focusing on the methods used in the Smith work as well as methods developed since 2011 that have improved estimation performance. Importantly, only approaches that have been specifically designed for fMRI data perform well, as they have been tailored to meet the challenges of fMRI data. Although this work does not suggest a single best model, it describes the class of models that perform best and highlights the features of these models that allow them to perform well on fMRI data. Specifically, methods that rely on non-Gaussianity to direct causal relationships in the network perform well.

The impact of study design on pattern estimation for single-trial multivariate pattern analysis.

A prerequisite for a pattern analysis using functional magnetic resonance imaging (fMRI) data is estimating the patterns from time series data, which then are input into the pattern analysis. Here we focus on how the combination of study design (order and spacing of trials) with pattern estimator impacts the Type I error rate of the subsequent pattern analysis. When Type I errors are inflated, the results are no longer valid, so this work serves as a guide for designing and analyzing MVPA studies with controlled false positive rates. The MVPA strategies examined are pattern classification and similarity, utilizing single trial activation patterns from the same functional run. Primarily focusing on the Least Squares Single and Least Square All pattern estimators, we show that collinearities in the models, along with temporal autocorrelation, can cause false positive correlations between activation pattern estimates that adversely impact the false positive rates of pattern similarity and classification analyses. It may seem intuitive that increasing the interstimulus interval (ISI) would alleviate this issue, but remaining weak correlations between activation patterns persist and have a strong influence in pattern similarity analyses. Pattern similarity analyses using only activation patterns estimated from the same functional run of data are susceptible to inflated false positives unless trials are randomly ordered, with a different randomization for each subject. In other cases, where there is any structure to trial order, valid pattern similarity analysis results can only be obtained if similarity computations are restricted to pairs of activation patterns from independent runs. Likewise, for pattern classification, false positives are minimized when the testing and training sets in cross validation do not contain patterns estimated from the same run.

What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis.

Multi-voxel pattern analysis (MVPA) has led to major changes in how fMRI data are analyzed and interpreted. Many studies now report both MVPA results and results from standard univariate voxel-wise analysis, often with the goal of drawing different conclusions from each. Because MVPA results can be sensitive to latent multidimensional representations and processes whereas univariate voxel-wise analysis cannot, one conclusion that is often drawn when MVPA and univariate results differ is that the activation patterns underlying MVPA results contain a multidimensional code. In the current study, we conducted simulations to formally test this assumption. Our findings reveal that MVPA tests are sensitive to the magnitude of voxel-level variability in the effect of a condition within subjects, even when the same linear relationship is coded in all voxels. We also find that MVPA is insensitive to subject-level variability in mean activation across an ROI, which is the primary variance component of interest in many standard univariate tests. Together, these results illustrate that differences between MVPA and univariate tests do not afford conclusions about the nature or dimensionality of the neural code. Instead, targeted tests of the informational content and/or dimensionality of activation patterns are critical for drawing strong conclusions about the representational codes that are indicated by significant MVPA results.

Greater BOLD response to working memory in endurance-trained adults revealed by breath-hold calibration.

BACKGROUND: Cardiorespiratory fitness is associated with increased frontal and parietal activation during executive function tasks. While these findings suggest fitness-related enhancement of neuronal response, the utility of functional magnetic resonance imaging (fMRI) may be limited by potential fitness-related differences in global vascular reactivity. The aim of this study was to determine if highly fit adults display differential activation during working memory after calibration for vascular reactivity relative to their sedentary counterparts. METHODS: Thirty-two endurance-trained and 24 sedentary adults, aged 40-65 years, completed a 2-Back verbal working memory task and a breath-hold challenge during fMRI. Group differences in blood oxygen level-dependent (BOLD) response during working memory were examined across the whole brain and in a priori regions of interest (ROI) before and after breath-hold calibration using non-parametric permutation testing. Multiple regression was used to explore the association between cardiorespiratory fitness (VO2 max), age, and calibrated 2-Back-related activation within the one a priori ROI with significant group effects. RESULTS: In comparison to the endurance-trained group, the sedentary group exhibited greater BOLD signal changes in response to the breath-hold task. After, but not before calibration, the endurance-trained group displayed significantly higher 2-Back-related activation in the right middle frontal gyrus (P = 0.049). Older age predicted lower 2-Back-related activation (ß = -0.308, P = 0.031), whereas fitness predicted higher activation (ß = 0.372, P = 0.021) in this region. CONCLUSIONS: Breath-hold calibration increased detection of working memory-related BOLD response differences between sedentary and endurance-trained adults. Moreover, cardiorespiratory fitness appeared to mitigate age-related changes in BOLD during working memory in this region.

Complementary role of frontoparietal activity and cortical pattern similarity in successful episodic memory encoding.

One central goal in cognitive neuroscience of learning and memory is to characterize the neural processes that lead to long-lasting episodic memory. In addition to the stronger frontoparietal activity, greater category- or item-specific cortical representation during encoding, as measured by pattern similarity (PS), is also associated with better subsequent episodic memory. Nevertheless, it is unknown whether frontoparietal activity and cortical PS reflect distinct mechanisms. To address this issue, we reanalyzed previous data (Xue G, Dong Q, Chen C, Lu ZL, Mumford JA, Poldrack RA. 2010. Greater neural pattern similarity across repetitions is associated with better memory. Science. 330:97, Experiment 3) using a novel approach based on combined activation-based and information-based analyses. The results showed that across items, stronger frontoparietal activity was associated with greater PS in distributed brain regions, including those where the PS was predictive of better subsequent memory. Nevertheless, the item-specific PS was still associated with later episodic memory after controlling the effect of frontoparietal activity. Our results suggest that one possible mechanism of frontoparietal activity on episodic memory encoding is via enhancing PS, resulting in more unique and consistent input to the medial temporal lobe. In addition, they suggest that PS might index additional processes, such as pattern reinstatement as a result of study-phase retrieval, that contribute to episodic memory encoding.

A multi-sample evaluation of the measurement structure and function of the modified monetary incentive delay task in adolescents.

Interpreting the neural response elicited during task functional magnetic resonance imaging (fMRI) remains a challenge in neurodevelopmental research. The monetary incentive delay (MID) task is an fMRI reward processing task that is extensively used in the literature. However, modern psychometric tools have not been used to evaluate measurement properties of the MID task fMRI data. The current study uses data for a similar task design across three adolescent samples (N = 346 [Agemean 12.0; 44 % Female]; N = 97 [19.3; 58 %]; N = 112 [20.2; 38 %]) to evaluate multiple measurement properties of fMRI responses on the MID task. Confirmatory factor analysis (CFA) is used to evaluate an a priori theoretical model for the task and its measurement invariance across three samples. Exploratory factor analysis (EFA) is used to identify the data-driven measurement structure across the samples. CFA results suggest that the a priori model is a poor representation of these MID task fMRI data. Across the samples, the data-driven EFA models consistently identify a six-to-seven factor structure with run and bilateral brain region factors. This factor structure is moderately-to-highly congruent across the samples. Altogether, these findings demonstrate a need to evaluate theoretical frameworks for popular fMRI task designs to improve our understanding and interpretation of brain-behavior associations.

The response time paradox in functional magnetic resonance imaging analyses.

Response times (RTs) are often the main signal of interest in cognitive psychology but are often ignored in functional MRI (fMRI) analyses. In fMRI analysis the intensity of the signal serves as a proxy for the intensity of local neuronal activity, but changes in either the intensity or the duration of neuronal activity can yield identical fMRI signals. Therefore, if RTs are ignored and pair with neuronal durations, fMRI results claiming intensity differences may be confounded by RTs. We show how ignoring RTs goes beyond this confound, where longer RTs are paired with larger activation estimates, to lesser-known issues where RTs become confounds in group-level analyses and, surprisingly, how the RT confound can induce other artificial group-level associations with variables that are not related to the condition contrast or RTs. We propose a new time-series model to address these issues and encourage increasing focus on what the widespread RT-based signal represents.

Perceptual criteria in the human brain.

A critical component of decision making is the ability to adjust criteria for classifying stimuli. fMRI and drift diffusion models were used to explore the neural representations of perceptual criteria in decision making. The specific focus was on the relative engagement of perceptual- and decision-related neural systems in response to adjustments in perceptual criteria. Human participants classified visual stimuli as big or small based on criteria of different sizes, which effectively biased their choices toward one response over the other. A drift diffusion model was fit to the behavioral data to extract estimates of stimulus size, criterion size, and difficulty for each participant and condition. These parameter values were used as modulated regressors to create a highly constrained model for the fMRI analysis that accounted for several components of the decision process. The results show that perceptual criteria values were reflected by activity in left inferior temporal cortex, a region known to represent objects and their physical properties, whereas stimulus size was reflected by activation in occipital cortex. A frontoparietal network of regions, including dorsolateral prefrontal cortex and superior parietal lobule, corresponded to the decision variables resulting from the downstream stimulus-criterion comparison, independent of stimulus type. The results provide novel evidence that perceptual criteria are represented in stimulus space and serve as inputs to be compared with the presented stimulus, recruiting a common network of decision regions shown to be active in other simple decisions. This work advances our understanding of the neural correlates of decision flexibility and adjustments of behavioral bias.