Variability in the analysis of a single neuroimaging dataset by many teams.

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.

Open and reproducible neuroimaging: From study inception to publication.

Empirical observations of how labs conduct research indicate that the adoption rate of open practices for transparent, reproducible, and collaborative science remains in its infancy. This is at odds with the overwhelming evidence for the necessity of these practices and their benefits for individual researchers, scientific progress, and society in general. To date, information required for implementing open science practices throughout the different steps of a research project is scattered among many different sources. Even experienced researchers in the topic find it hard to navigate the ecosystem of tools and to make sustainable choices. Here, we provide an integrated overview of community-developed resources that can support collaborative, open, reproducible, replicable, robust and generalizable neuroimaging throughout the entire research cycle from inception to publication and across different neuroimaging modalities. We review tools and practices supporting study inception and planning, data acquisition, research data management, data processing and analysis, and research dissemination. An online version of this resource can be found at https://oreoni.github.io. We believe it will prove helpful for researchers and institutions to make a successful and sustainable move towards open and reproducible science and to eventually take an active role in its future development.

Memory for individual items is related to nonreinforced preference change.

It is commonly assumed that memories contribute to value-based decisions. Nevertheless, most theories of value-based decision-making do not account for memory influences on choice. Recently, new interest has emerged in the interactions between these two fundamental processes, mainly using reinforcement-based paradigms. Here, we aimed to study the role memory processes play in preference change following the nonreinforced cue-approach training (CAT) paradigm. In CAT, the mere association of cued items with a speeded motor response influences choices. Previous studies with this paradigm showed that a single training session induces a long-lasting effect of enhanced preferences for high-value trained stimuli, that is maintained for several months. We hypothesized that CAT increases memory of trained items, leading to enhanced accessibility of their positive associative memories and in turn to preference changes. In two preregistered experiments, we found evidence that memory is enhanced for trained items and that better memory is correlated with enhanced preferences at the individual item level, both immediately and 1 mo following CAT. Our findings suggest that memory plays a central role in value-based decision-making following CAT, even in the absence of external reinforcements. These findings contribute to new theories relating memory and value-based decision-making and set the groundwork for the implementation of novel nonreinforced behavioral interventions that lead to long-lasting behavioral change.

Brain volumetric changes in the general population following the COVID-19 outbreak and lockdown.

The coronavirus disease 2019 (COVID-19) outbreak introduced unprecedented health-risks, as well as pressure on the economy, society, and psychological well-being due to the response to the outbreak. In a preregistered study, we hypothesized that the intense experience of the outbreak potentially induced stress-related brain modifications in the healthy population, not infected with the virus. We examined volumetric changes in 50 participants who underwent MRI scans before and after the COVID-19 outbreak and lockdown in Israel. Their scans were compared with those of 50 control participants who were scanned twice prior to the pandemic. Following COVID-19 outbreak and lockdown, the test group participants uniquely showed volumetric increases in bilateral amygdalae, putamen, and the anterior temporal cortices. Changes in the amygdalae diminished as time elapsed from lockdown relief, suggesting that the intense experience associated with the pandemic induced transient volumetric changes in brain regions commonly associated with stress and anxiety. The current work utilizes a rare opportunity for real-life natural experiment, showing evidence for brain plasticity following the COVID-19 global pandemic. These findings have broad implications, relevant both for the scientific community as well as the general public.

Enhanced Bottom-Up and Reduced Top-Down fMRI Activity Is Related to Long-Lasting Nonreinforced Behavioral Change.

Behavioral change studies and interventions focus on self-control and external reinforcements to influence preferences. Cue-approach training (CAT) has been shown to induce preference changes lasting months by merely associating items with neutral cues and speeded responses. We utilized this paradigm to study neural representation of preferences and their modification without external reinforcements. We scanned 36 participants with fMRI during a novel passive viewing task before, after and 30 days following CAT. We preregistered the predictions that activity in memory, top-down attention, and value-processing regions will underlie preference modification. While most theories associate preferences with prefrontal regions, we found that "bottom-up" perceptual mechanisms were associated with immediate change, whereas reduced "top-down" parietal activity was related to long-term change. Activity in value-related prefrontal regions was enhanced immediately after CAT for trained items and 1 month after for all items. Our findings suggest a novel neural mechanism of preference representation and modification. We suggest that nonreinforced change of preferences occurs initially in perceptual representation of items, putatively leading to long-term changes in "top-down" processes. These findings offer implementation of bottom-up instead of top-down targeted interventions for long-lasting behavioral change.

Short-term plasticity following motor sequence learning revealed by diffusion magnetic resonance imaging.

Current noninvasive methods to detect structural plasticity in humans are mainly used to study long-term changes. Diffusion magnetic resonance imaging (MRI) was recently proposed as a novel approach to reveal gray matter changes following spatial navigation learning and object-location memory tasks. In the present work, we used diffusion MRI to investigate the short-term neuroplasticity that accompanies motor sequence learning. Following a 45-min training session in which participants learned to accurately play a short sequence on a piano keyboard, changes in diffusion properties were revealed mainly in motor system regions such as the premotor cortex and cerebellum. In a second learning session taking place immediately afterward, feedback was given on the timing of key pressing instead of accuracy, while participants continued to learn. This second session induced a different plasticity pattern, demonstrating the dynamic nature of learning-induced plasticity, formerly thought to require months of training in order to be detectable. These results provide us with an important reminder that the brain is an extremely dynamic structure. Furthermore, diffusion MRI offers a novel measure to follow tissue plasticity particularly over short timescales, allowing new insights into the dynamics of structural brain plasticity.

Enhanced striatal and prefrontal activity is associated with individual differences in nonreinforced preference change for faces.

Developing effective preference modification paradigms is crucial to improve the quality of life in a wide range of behaviors. The cue-approach training (CAT) paradigm has been introduced as an effective tool to modify preferences lasting months, without external reinforcements, using the mere association of images with a cue and a speeded button response. In the current work for the first time, we used fMRI with faces as stimuli in the CAT paradigm, focusing on face-selective brain regions. We found a behavioral change effect of CAT with faces immediately and 1-month after training, however face-selective regions were not indicative of behavioral change and thus preference change is less likely to rely on face processing brain regions. Nevertheless, we found that during training, fMRI activations in the ventral striatum were correlated with individual preference change. We also found a correlation between preference change and activations in the ventromedial prefrontal cortex during the binary choice phase. Functional connectivity among striatum, prefrontal regions, and high-level visual regions was also related to individual preference change. Our work sheds new light on the involvement of neural mechanisms in the process of valuation. This could lead to development of novel real-world interventions.

Expectation generation and its effect on subsequent pain and visual perception.

Bayesian accounts of perception, such as predictive processing, suggest that perceptions integrate expectations and sensory experience, and thus assimilate to expected values. Furthermore, more precise expectations should have stronger influences on perception. We tested these hypotheses in a paradigm that manipulates both the mean value and the precision of cues within-person. Forty-five participants observed cues-presented as ratings from 10 previous participants-with varying cue means, variances (precision), and skewness across trials. Participants reported expectations regarding the painfulness of thermal stimuli or the visual contrast of flickering checkerboards. Subsequently, similar cues were each followed by a visual or noxious thermal stimulus. While perceptions assimilated to expected values in both modalities, cues' precision mainly affected visual ratings. Furthermore, behavioral and computational models revealed that expectations were biased towards extreme values in both modalities, and towards low-pain cues specifically. fMRI analysis revealed that the cues affected systems related to higher-level affective and cognitive processes-including assimilation to the cue mean in a neuromarker of endogenous contributions to pain and in the nucleus accumbens, and activity consistent with aversive prediction-error-like encoding in the periaqueductal gray during pain perception-but not systems related to early perceptual processing. Our findings suggest that predictive processing theories should be combined with mechanisms such as selective attention to better fit empirical findings, and that expectation generation and its perceptual effects are mostly modality-specific and operate on higher-level processes rather than early perception.

Placebo treatment affects brain systems related to affective and cognitive processes, but not nociceptive pain.

Drug treatments for pain often do not outperform placebo, and a better understanding of placebo mechanisms is needed to improve treatment development and clinical practice. In a large-scale fMRI study (N = 392) with pre-registered analyses, we tested whether placebo analgesic treatment modulates nociceptive processes, and whether its effects generalize from conditioned to unconditioned pain modalities. Placebo treatment caused robust analgesia in conditioned thermal pain that generalized to unconditioned mechanical pain. However, placebo did not decrease pain-related fMRI activity in brain measures linked to nociceptive pain, including the Neurologic Pain Signature (NPS) and spinothalamic pathway regions, with strong support for null effects in Bayes Factor analyses. In addition, surprisingly, placebo increased activity in some spinothalamic regions for unconditioned mechanical pain. In contrast, placebo reduced activity in a neuromarker associated with higher-level contributions to pain, the Stimulus Intensity Independent Pain Signature (SIIPS), and affected activity in brain regions related to motivation and value, in both pain modalities. Individual differences in behavioral analgesia were correlated with neural changes in both modalities. Our results indicate that cognitive and affective processes primarily drive placebo analgesia, and show the potential of neuromarkers for separating treatment influences on nociception from influences on evaluative processes.

Subjective evidence evaluation survey for many-analysts studies.

Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.

Reproducibility in Neuroimaging Analysis: Challenges and Solutions.

Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience, and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research-one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish 3 main types of reproducibility, discussing each in turn. Analytical reproducibility is the ability to reproduce findings using the same data and methods. Replicability is the ability to find an effect in new datasets, using the same or similar methods. Finally, robustness to analytical variability refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in more reproducible, replicable, and robust psychological and brain research and a stronger scientific foundation across fields of inquiry.

A belief systems analysis of fraud beliefs following the 2020 US election.

Beliefs that the US 2020 Presidential election was fraudulent are prevalent despite substantial contradictory evidence. Why are such beliefs often resistant to counter-evidence? Is this resistance rational, and thus subject to evidence-based arguments, or fundamentally irrational? Here we surveyed 1,642 Americans during the 2020 vote count, testing fraud belief updates given hypothetical election outcomes. Participants' fraud beliefs increased when their preferred candidate lost and decreased when he won, and both effects scaled with partisan preferences, demonstrating partisan asymmetry (desirability effects). A Bayesian model of rational updating of a system of beliefs-beliefs in the true vote winner, fraud prevalence and beneficiary of fraud-accurately accounted for this partisan asymmetry, outperforming alternative models of irrational, motivated updating and models lacking the full belief system. Partisan asymmetries may not reflect motivated reasoning, but rather rational attributions over multiple potential causes of evidence. Changing such beliefs may require targeting multiple key beliefs simultaneously rather than direct debunking attempts.

Placebo treatment affects brain systems related to affective and cognitive processes, but not nociceptive pain.

Placebo analgesia is a replicable and well-studied phenomenon, yet it remains unclear to what degree it includes modulation of nociceptive processes. Some studies find effects consistent with nociceptive effects, but meta-analyses show that these effects are often small. We analyzed placebo analgesia in a large fMRI study (N = 392), including placebo effects on brain responses to noxious stimuli. Placebo treatment caused robust analgesia in both conditioned thermal and unconditioned mechanical pain. Placebo did not decrease fMRI activity in nociceptive pain regions, including the Neurologic Pain Signature (NPS) and pre-registered spinothalamic pathway regions, with strong support from Bayes Factor analyses. However, placebo treatment affected activity in pre-registered analyses of a second neuromarker, the Stimulus Intensity Independent Pain Signature (SIIPS), and several associated a priori brain regions related to motivation and value, in both thermal and mechanical pain. Individual differences in behavioral analgesia were correlated with neural changes in both thermal and mechanical pain. Our results indicate that processes related to affective and cognitive aspects of pain primarily drive placebo analgesia.

Centering inclusivity in the design of online conferences-An OHBM-Open Science perspective.

As the global health crisis unfolded, many academic conferences moved online in 2020. This move has been hailed as a positive step towards inclusivity in its attenuation of economic, physical, and legal barriers and effectively enabled many individuals from groups that have traditionally been underrepresented to join and participate. A number of studies have outlined how moving online made it possible to gather a more global community and has increased opportunities for individuals with various constraints, e.g., caregiving responsibilities. Yet, the mere existence of online conferences is no guarantee that everyone can attend and participate meaningfully. In fact, many elements of an online conference are still significant barriers to truly diverse participation: the tools used can be inaccessible for some individuals; the scheduling choices can favour some geographical locations; the set-up of the conference can provide more visibility to well-established researchers and reduce opportunities for early-career researchers. While acknowledging the benefits of an online setting, especially for individuals who have traditionally been underrepresented or excluded, we recognize that fostering social justice requires inclusivity to actively be centered in every aspect of online conference design. Here, we draw from the literature and from our own experiences to identify practices that purposefully encourage a diverse community to attend, participate in, and lead online conferences. Reflecting on how to design more inclusive online events is especially important as multiple scientific organizations have announced that they will continue offering an online version of their event when in-person conferences can resume.

Consensus-based guidance for conducting and reporting multi-analyst studies.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

fMRI data of mixed gambles from the Neuroimaging Analysis Replication and Prediction Study.

There is an ongoing debate about the replicability of neuroimaging research. It was suggested that one of the main reasons for the high rate of false positive results is the many degrees of freedom researchers have during data analysis. In the Neuroimaging Analysis Replication and Prediction Study (NARPS), we aim to provide the first scientific evidence on the variability of results across analysis teams in neuroscience. We collected fMRI data from 108 participants during two versions of the mixed gambles task, which is often used to study decision-making under risk. For each participant, the dataset includes an anatomical (T1 weighted) scan and fMRI as well as behavioral data from four runs of the task. The dataset is shared through OpenNeuro and is formatted according to the Brain Imaging Data Structure (BIDS) standard. Data pre-processed with fMRIprep and quality control reports are also publicly shared. This dataset can be used to study decision-making under risk and to test replicability and interpretability of previous results in the field.

Spacing of cue-approach training leads to better maintenance of behavioral change.

The maintenance of behavioral change over the long term is essential to achieve public health goals such as combatting obesity and drug use. Previous work by our group has demonstrated a reliable shift in preferences for appetitive foods following a novel non-reinforced training paradigm. In the current studies, we tested whether distributing training trials over two consecutive days would affect preferences immediately after training as well as over time at a one-month follow-up. In four studies, three different designs and an additional pre-registered replication of one sample, we found that spacing of cue-approach training induced a shift in food choice preferences over one month. The spacing and massing schedule employed governed the long-term changes in choice behavior. Applying spacing strategies to training paradigms that target automatic processes could prove a useful tool for the long-term maintenance of health improvement goals with the development of real-world behavioral change paradigms that incorporate distributed practice principles.

The Cue-Approach Task as a General Mechanism for Long-Term Non-Reinforced Behavioral Change.

Recent findings show that preferences for food items can be modified without external reinforcements using the cue-approach task. In the task, the mere association of food item images with a neutral auditory cue and a speeded button press, resulted in enhanced preferences for the associated stimuli. In a series of 10 independent samples with a total of 255 participants, we show for the first time that using this non-reinforced method we can enhance preferences for faces, fractals and affective images, as well as snack foods, using auditory, visual and even aversive cues. This change was highly durable in follow-up sessions performed one to six months after training. Preferences were successfully enhanced for all conditions, except for negative valence items. These findings promote our understanding of non-reinforced change, suggest a boundary condition for the effect and lay the foundation for development of novel applications.