Variability in the analysis of a single neuroimaging dataset by many teams.

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.

fMRI data of mixed gambles from the Neuroimaging Analysis Replication and Prediction Study.

There is an ongoing debate about the replicability of neuroimaging research. It was suggested that one of the main reasons for the high rate of false positive results is the many degrees of freedom researchers have during data analysis. In the Neuroimaging Analysis Replication and Prediction Study (NARPS), we aim to provide the first scientific evidence on the variability of results across analysis teams in neuroscience. We collected fMRI data from 108 participants during two versions of the mixed gambles task, which is often used to study decision-making under risk. For each participant, the dataset includes an anatomical (T1 weighted) scan and fMRI as well as behavioral data from four runs of the task. The dataset is shared through OpenNeuro and is formatted according to the Brain Imaging Data Structure (BIDS) standard. Data pre-processed with fMRIprep and quality control reports are also publicly shared. This dataset can be used to study decision-making under risk and to test replicability and interpretability of previous results in the field.

The Cue-Approach Task as a General Mechanism for Long-Term Non-Reinforced Behavioral Change.

Recent findings show that preferences for food items can be modified without external reinforcements using the cue-approach task. In the task, the mere association of food item images with a neutral auditory cue and a speeded button press, resulted in enhanced preferences for the associated stimuli. In a series of 10 independent samples with a total of 255 participants, we show for the first time that using this non-reinforced method we can enhance preferences for faces, fractals and affective images, as well as snack foods, using auditory, visual and even aversive cues. This change was highly durable in follow-up sessions performed one to six months after training. Preferences were successfully enhanced for all conditions, except for negative valence items. These findings promote our understanding of non-reinforced change, suggest a boundary condition for the effect and lay the foundation for development of novel applications.