Open and reproducible neuroimaging: From study inception to publication.

Empirical observations of how labs conduct research indicate that the adoption rate of open practices for transparent, reproducible, and collaborative science remains in its infancy. This is at odds with the overwhelming evidence for the necessity of these practices and their benefits for individual researchers, scientific progress, and society in general. To date, information required for implementing open science practices throughout the different steps of a research project is scattered among many different sources. Even experienced researchers in the topic find it hard to navigate the ecosystem of tools and to make sustainable choices. Here, we provide an integrated overview of community-developed resources that can support collaborative, open, reproducible, replicable, robust and generalizable neuroimaging throughout the entire research cycle from inception to publication and across different neuroimaging modalities. We review tools and practices supporting study inception and planning, data acquisition, research data management, data processing and analysis, and research dissemination. An online version of this resource can be found at https://oreoni.github.io. We believe it will prove helpful for researchers and institutions to make a successful and sustainable move towards open and reproducible science and to eventually take an active role in its future development.

Isolating the sources of pipeline-variability in group-level task-fMRI results.

Task-fMRI researchers have great flexibility as to how they analyze their data, with multiple methodological options to choose from at each stage of the analysis workflow. While the development of tools and techniques has broadened our horizons for comprehending the complexities of the human brain, a growing body of research has highlighted the pitfalls of such methodological plurality. In a recent study, we found that the choice of software package used to run the analysis pipeline can have a considerable impact on the final group-level results of a task-fMRI investigation (Bowring et al., 2019, BMN). Here we revisit our work, seeking to identify the stages of the pipeline where the greatest variation between analysis software is induced. We carry out further analyses on the three datasets evaluated in BMN, employing a common processing strategy across parts of the analysis workflow and then utilizing procedures from three software packages (AFNI, FSL, and SPM) across the remaining steps of the pipeline. We use quantitative methods to compare the statistical maps and isolate the main stages of the workflow where the three packages diverge. Across all datasets, we find that variation between the packages' results is largely attributable to a handful of individual analysis stages, and that these sources of variability were heterogeneous across the datasets (e.g., choice of first-level signal model had the most impact for the balloon analog risk task dataset, while first-level noise model and group-level model were more influential for the false belief and antisaccade task datasets, respectively). We also observe areas of the analysis workflow where changing the software package causes minimal differences in the final results, finding that the group-level results were largely unaffected by which software package was used to model the low-frequency fMRI drifts.

The Open Brain Consent: Informing research participants and obtaining consent to share brain imaging data.

Having the means to share research data openly is essential to modern science. For human research, a key aspect in this endeavor is obtaining consent from participants, not just to take part in a study, which is a basic ethical principle, but also to share their data with the scientific community. To ensure that the participants' privacy is respected, national and/or supranational regulations and laws are in place. It is, however, not always clear to researchers what the implications of those are, nor how to comply with them. The Open Brain Consent (https://open-brain-consent.readthedocs.io) is an international initiative that aims to provide researchers in the brain imaging community with information about data sharing options and tools. We present here a short history of this project and its latest developments, and share pointers to consent forms, including a template consent form that is compliant with the EU general data protection regulation. We also share pointers to an associated data user agreement that is not only useful in the EU context, but also for any researchers dealing with personal (clinical) data elsewhere.

GliMR: Cross-Border Collaborations to Promote Advanced MRI Biomarkers for Glioma.

Purpose: There is an annual incidence of 50,000 glioma cases in Europe. The optimal treatment strategy is highly personalised, depending on tumour type, grade, spatial localization, and the degree of tissue infiltration. In research settings, advanced magnetic resonance imaging (MRI) has shown great promise as a tool to inform personalised treatment decisions. However, the use of advanced MRI in clinical practice remains scarce due to the downstream effects of siloed glioma imaging research with limited representation of MRI specialists in established consortia; and the associated lack of available tools and expertise in clinical settings. These shortcomings delay the translation of scientific breakthroughs into novel treatment strategy. As a response we have developed the network "Glioma MR Imaging 2.0" (GliMR) which we present in this article. Methods: GliMR aims to build a pan-European and multidisciplinary network of experts and accelerate the use of advanced MRI in glioma beyond the current "state-of-the-art" in glioma imaging. The Action Glioma MR Imaging 2.0 (GliMR) was granted funding by the European Cooperation in Science and Technology (COST) in June 2019. Results: GliMR's first grant period ran from September 2019 to April 2020, during which several meetings were held and projects were initiated, such as reviewing the current knowledge on advanced MRI; developing a General Data Protection Regulation (GDPR) compliant consent form; and setting up the website. Conclusion: The Action overcomes the pre-existing limitations of glioma research and is funded until September 2023. New members will be accepted during its entire duration.

Exploring the impact of analysis software on task fMRI results.

A wealth of analysis tools are available to fMRI researchers in order to extract patterns of task variation and, ultimately, understand cognitive function. However, this "methodological plurality" comes with a drawback. While conceptually similar, two different analysis pipelines applied on the same dataset may not produce the same scientific results. Differences in methods, implementations across software, and even operating systems or software versions all contribute to this variability. Consequently, attention in the field has recently been directed to reproducibility and data sharing. In this work, our goal is to understand how choice of software package impacts on analysis results. We use publicly shared data from three published task fMRI neuroimaging studies, reanalyzing each study using the three main neuroimaging software packages, AFNI, FSL, and SPM, using parametric and nonparametric inference. We obtain all information on how to process, analyse, and model each dataset from the publications. We make quantitative and qualitative comparisons between our replications to gauge the scale of variability in our results and assess the fundamental differences between each software package. Qualitatively we find similarities between packages, backed up by Neurosynth association analyses that correlate similar words and phrases to all three software package's unthresholded results for each of the studies we reanalyse. However, we also discover marked differences, such as Dice similarity coefficients ranging from 0.000 to 0.684 in comparisons of thresholded statistic maps between software. We discuss the challenges involved in trying to reanalyse the published studies, and highlight our efforts to make this research reproducible.

Accelerating permutation testing in voxel-wise analysis through subspace tracking: A new plugin for SnPM.

Permutation testing is a non-parametric method for obtaining the max null distribution used to compute corrected p-values that provide strong control of false positives. In neuroimaging, however, the computational burden of running such an algorithm can be significant. We find that by viewing the permutation testing procedure as the construction of a very large permutation testing matrix, T, one can exploit structural properties derived from the data and the test statistics to reduce the runtime under certain conditions. In particular, we see that T is low-rank plus a low-variance residual. This makes T a good candidate for low-rank matrix completion, where only a very small number of entries of T (∼0.35% of all entries in our experiments) have to be computed to obtain a good estimate. Based on this observation, we present RapidPT, an algorithm that efficiently recovers the max null distribution commonly obtained through regular permutation testing in voxel-wise analysis. We present an extensive validation on a synthetic dataset and four varying sized datasets against two baselines: Statistical NonParametric Mapping (SnPM13) and a standard permutation testing implementation (referred as NaivePT). We find that RapidPT achieves its best runtime performance on medium sized datasets (50≤n≤200), with speedups of 1.5× - 38× (vs. SnPM13) and 20x-1000× (vs. NaivePT). For larger datasets (n≥200) RapidPT outperforms NaivePT (6× - 200×) on all datasets, and provides large speedups over SnPM13 when more than 10000 permutations (2× - 15×) are needed. The implementation is a standalone toolbox and also integrated within SnPM13, able to leverage multi-core architectures when available.

An a contrario approach for the detection of patient-specific brain perfusion abnormalities with arterial spin labelling.

In this paper, we introduce a new locally multivariate procedure to quantitatively extract voxel-wise patterns of abnormal perfusion in individual patients. This a contrario approach uses a multivariate metric from the computer vision community that is suitable to detect abnormalities even in the presence of closeby hypo- and hyper-perfusions. This method takes into account local information without applying Gaussian smoothing to the data. Furthermore, to improve on the standard a contrario approach, which assumes white noise, we introduce an updated a contrario approach that takes into account the spatial coherency of the noise in the probability estimation. Validation is undertaken on a dataset of 25 patients diagnosed with brain tumours and 61 healthy volunteers. We show how the a contrario approach outperforms the massively univariate general linear model usually employed for this type of analysis.

NeuroVault.org: A repository for sharing unthresholded statistical maps, parcellations, and atlases of the human brain.

NeuroVault.org is dedicated to storing outputs of analyses in the form of statistical maps, parcellations and atlases, a unique strategy that contrasts with most neuroimaging repositories that store raw acquisition data or stereotaxic coordinates. Such maps are indispensable for performing meta-analyses, validating novel methodology, and deciding on precise outlines for regions of interest (ROIs). NeuroVault is open to maps derived from both healthy and clinical populations, as well as from various imaging modalities (sMRI, fMRI, EEG, MEG, PET, etc.). The repository uses modern web technologies such as interactive web-based visualization, cognitive decoding, and comparison with other maps to provide researchers with efficient, intuitive tools to improve the understanding of their results. Each dataset and map is assigned a permanent Universal Resource Locator (URL), and all of the data is accessible through a REST Application Programming Interface (API). Additionally, the repository supports the NIDM-Results standard and has the ability to parse outputs from popular FSL and SPM software packages to automatically extract relevant metadata. This ease of use, modern web-integration, and pioneering functionality holds promise to improve the workflow for making inferences about and sharing whole-brain statistical maps.

Corticospinal tract asymmetry and handedness in right- and left-handers by diffusion tensor tractography.

PURPOSE: Cerebral hemispheres represent both structural and functional asymmetry, which differs among right- and left-handers. The left hemisphere is specialised for language and task execution of the right hand in right-handers. We studied the corticospinal tract in right- and left-handers by diffusion tensor imaging and tractography. The present study aimed at revealing a morphological difference resulting from a region of interest (ROI) obtained by functional MRI (fMRI). METHODS: Twenty-five healthy participants (right-handed: 15, left-handed: 10) were enrolled in our assessment of morphological, functional and diffusion tensor MRI. Assessment of brain fibre reconstruction (tractography) was done using a deterministic algorithm. Fractional anisotropy (FA) and mean diffusivity (MD) were studied on the tractography traces of the reference slices. RESULTS: We observed a significant difference in number of leftward fibres based on laterality. The significant difference in regard to FA and MD was based on the slices obtained at different levels and the laterality index. We found left-hand asymmetry and right-hand asymmetry, respectively, for the MD and FA. CONCLUSIONS: Our study showed the presence of hemispheric asymmetry based on laterality index in right- and left-handers. These results are inconsistent with some studies and consistent with others. The reported difference in hemispheric asymmetry could be related to dexterity (manual skill).

Improving the Eligibility of Task-Based fMRI Studies for Meta-Analysis: A Review and Reporting Recommendations.

Decisions made during the analysis or reporting of an fMRI study influence the eligibility of that study to be entered into a meta-analysis. In a meta-analysis, results of different studies on the same topic are combined. To combine the results, it is necessary that all studies provide equivalent pieces of information. However, in task-based fMRI studies we see a large variety in reporting styles. Several specific meta-analysis methods have been developed to deal with the reporting practices occurring in task-based fMRI studies, therefore each requiring a specific type of input. In this manuscript we provide an overview of the meta-analysis methods and the specific input they require. Subsequently we discuss how decisions made during the study influence the eligibility of a study for a meta-analysis and finally we formulate some recommendations about how to report an fMRI study so that it complies with as many meta-analysis methods as possible.

Patient-specific detection of perfusion abnormalities combining within-subject and between-subject variances in Arterial Spin Labeling.

In this paper, patient-specific perfusion abnormalities in Arterial Spin Labeling (ASL) were identified by comparing a single patient to a group of healthy controls using a mixed-effect hierarchical General Linear Model (GLM). Two approaches are currently in use to solve hierarchical GLMs: (1) the homoscedastic approach assumes homogeneous variances across subjects and (2) the heteroscedastic approach is theoretically more efficient in the presence of heterogeneous variances but algorithmically more demanding. In practice, in functional magnetic resonance imaging studies, the superiority of the heteroscedastic approach is still under debate. Due to the low signal-to-noise ratio of ASL sequences, within-subject variances have a significant impact on the estimated perfusion maps and the heteroscedastic model might be better suited in this context. In this paper we studied how the homoscedastic and heteroscedastic approaches behave in terms of specificity and sensitivity in the detection of patient-specific ASL perfusion abnormalities. Validation was undertaken on a dataset of 25 patients diagnosed with brain tumors and 36 healthy volunteers. We showed evidence of heterogeneous within-subject variances in ASL and pointed out an increased false positive rate of the homoscedastic model. In the detection of patient-specific brain perfusion abnormalities with ASL, modeling heterogeneous variances increases the sensitivity at the same specificity level.

Review Paper: Reporting Practices for Task fMRI Studies.

What are the standards for the reporting methods and results of fMRI studies, and how have they evolved over the years? To answer this question we reviewed 160 papers published between 2004 and 2019. Reporting styles for methods and results of fMRI studies can differ greatly between published studies. However, adequate reporting is essential for the comprehension, replication and reuse of the study (for instance in a meta-analysis). To aid authors in reporting the methods and results of their task-based fMRI study the COBIDAS report was published in 2016, which provides researchers with clear guidelines on how to report the design, acquisition, preprocessing, statistical analysis and results (including data sharing) of fMRI studies (Nichols et al. in Best Practices in Data Analysis and Sharing in Neuroimaging using fMRI, 2016). In the past reviews have been published that evaluate how fMRI methods are reported based on the 2008 guidelines, but they did not focus on how task based fMRI results are reported. This review updates reporting practices of fMRI methods, and adds an extra focus on how fMRI results are reported. We discuss reporting practices about the design stage, specific participant characteristics, scanner characteristics, data processing methods, data analysis methods and reported results.

Abnormal functional lateralization and activity of language brain areas in typical specific language impairment (developmental dysphasia).

Atypical functional lateralization and specialization for language have been proposed to account for developmental language disorders, yet results from functional neuroimaging studies are sparse and inconsistent. This functional magnetic resonance imaging study compared children with a specific subtype of specific language impairment affecting structural language (n = 21), to a matched group of typically developing children using a panel of four language tasks neither requiring reading nor metalinguistic skills, including two auditory lexico-semantic tasks (category fluency and responsive naming) and two visual phonological tasks based on picture naming. Data processing involved normalizing the data with respect to a matched pairs paediatric template, groups and between-groups analysis, and laterality indices assessment within regions of interest using single and combined task analysis. Children with specific language impairment exhibited a significant lack of left lateralization in all core language regions (inferior frontal gyrus-opercularis, inferior frontal gyrus-triangularis, supramarginal gyrus and superior temporal gyrus), across single or combined task analysis, but no difference of lateralization for the rest of the brain. Between-group comparisons revealed a left hypoactivation of Wernicke's area at the posterior superior temporal/supramarginal junction during the responsive naming task, and a right hyperactivation encompassing the anterior insula with adjacent inferior frontal gyrus and the head of the caudate nucleus during the first phonological task. This study thus provides evidence that this subtype of specific language impairment is associated with atypical lateralization and functioning of core language areas.

Corticospinal tractography with morphological, functional and diffusion tensor MRI: a comparative study of four deterministic algorithms used in clinical routine.

PURPOSE: Diffusion tensor imaging permits study of white matter fibre bundles; however, its main limitation is lack of validation on anatomical data, especially in crossing fibre regions. Our study aimed to compare four deterministic tractography algorithms used in clinical routine. We studied the corticospinal tract, the bundle mediating voluntary movement. Our study seeks to evaluate tractography provided by algorithms through comparative analysis by expert neuroradiologists. METHODS: MRI data from 15 right-handed volunteers (30.8 years) were studied. Regions of interest (ROIs) were segmented on morphological and functional MRI. Diffusion weighted images (15 directions) were performed, then for each voxel the tensor was estimated. Tractography of the corticospinal tract was performed using four fibre-tracking algorithms. Three numerical integration methods Euler, Runge-Kutta second (RK2) and fourth order (RK4), and a tensor deflection method (TEND). Quantitative measurement was performed. Qualitative evaluation was carried out by two expert neuroradiologists using Kappa test concordance. RESULTS: For the quantitative aspect, only RK2 and TEND presented no significant difference concerning the number of fibres (p = 0.58). There was no difference between right and left side for each algorithm. Regarding the qualitative aspects, there was a lack of fibres from the ventrolateral part of the functional ROIs. Comparison by expert neuroradiologists revealed low rather than high concordance. The algorithm ranked first was RK2 according to expert preferences. CONCLUSIONS: Different algorithms used in clinical routine failed to show realistic anatomical bundles. The most mathematically robust algorithm was not selected, nor was the algorithm defining more fibres. Validation of anatomical data provided by tractography remains a challenge.

On the benefits of self-taught learning for brain decoding.

CONTEXT: We study the benefits of using a large public neuroimaging database composed of functional magnetic resonance imaging (fMRI) statistic maps, in a self-taught learning framework, for improving brain decoding on new tasks. First, we leverage the NeuroVault database to train, on a selection of relevant statistic maps, a convolutional autoencoder to reconstruct these maps. Then, we use this trained encoder to initialize a supervised convolutional neural network to classify tasks or cognitive processes of unseen statistic maps from large collections of the NeuroVault database. RESULTS: We show that such a self-taught learning process always improves the performance of the classifiers, but the magnitude of the benefits strongly depends on the number of samples available both for pretraining and fine-tuning the models and on the complexity of the targeted downstream task. CONCLUSION: The pretrained model improves the classification performance and displays more generalizable features, less sensitive to individual differences.

FMRI language mapping in children: a panel of language tasks using visual and auditory stimulation without reading or metalinguistic requirements.

In the context of presurgical mapping or investigation of neurological and developmental disorders in children, language fMRI raises the issue of the design of a tasks panel achievable by young disordered children. Most language tasks shown to be efficient with healthy children require metalinguistic or reading abilities, therefore adding attentional, cognitive and academic constraints that may be problematic in this context. This study experimented a panel of four language tasks that did not require high attentional skills, reading, or metalinguistic abilities. Two reference tasks involving auditory stimulation (words generation from category, "category"; auditory responsive naming, "definition") were compared with two new tasks involving visual stimulation. These later were designed to tap spontaneous phonological production, in which the names of pictures to be named involve a phonological difference (e.g. in French poule/boule/moule; "phon-diff") or change of segmentation (e.g. in French car/car-te/car-t-on; "phon-seg"). Eighteen healthy children participated (mean age: 12.7+/-3 years). Data processing involved normalizing the data via a matched pairs pediatric template, and inter-task and region of interest analyses with laterality assessment. The reference tasks predominantly activated the left frontal and temporal core language regions, respectively. The new tasks activated these two regions simultaneously, more strongly for the phon-seg task. The union and intersection of all tasks provided more sensitive or specific maps. The study demonstrates that both reference and new tasks highlight core language regions in children, and that the latter are useful for the mapping of spontaneous phonological processing. The use of several different tasks may improve the sensitivity and specificity of fMRI.

When the Single Matters more than the Group (II): Addressing the Problem of High False Positive Rates in Single Case Voxel Based Morphometry Using Non-parametric Statistics.

In recent years, an increasing number of studies have used Voxel Based Morphometry (VBM) to compare a single patient with a psychiatric or neurological condition of interest against a group of healthy controls. However, the validity of this approach critically relies on the assumption that the single patient is drawn from a hypothetical population with a normal distribution and variance equal to that of the control group. In a previous investigation, we demonstrated that family-wise false positive error rate (i.e., the proportion of statistical comparisons yielding at least one false positive) in single case VBM are much higher than expected (Scarpazza et al., 2013). Here, we examine whether the use of non-parametric statistics, which does not rely on the assumptions of normal distribution and equal variance, would enable the investigation of single subjects with good control of false positive risk. We empirically estimated false positive rates (FPRs) in single case non-parametric VBM, by performing 400 statistical comparisons between a single disease-free individual and a group of 100 disease-free controls. The impact of smoothing (4, 8, and 12 mm) and type of pre-processing (Modulated, Unmodulated) was also examined, as these factors have been found to influence FPRs in previous investigations using parametric statistics. The 400 statistical comparisons were repeated using two independent, freely available data sets in order to maximize the generalizability of the results. We found that the family-wise error rate was 5% for increases and 3.6% for decreases in one data set; and 5.6% for increases and 6.3% for decreases in the other data set (5% nominal). Further, these results were not dependent on the level of smoothing and modulation. Therefore, the present study provides empirical evidence that single case VBM studies with non-parametric statistics are not susceptible to high false positive rates. The critical implication of this finding is that VBM can be used to characterize neuroanatomical alterations in individual subjects as long as non-parametric statistics are employed.

Sharing brain mapping statistical results with the neuroimaging data model.

Only a tiny fraction of the data and metadata produced by an fMRI study is finally conveyed to the community. This lack of transparency not only hinders the reproducibility of neuroimaging results but also impairs future meta-analyses. In this work we introduce NIDM-Results, a format specification providing a machine-readable description of neuroimaging statistical results along with key image data summarising the experiment. NIDM-Results provides a unified representation of mass univariate analyses including a level of detail consistent with available best practices. This standardized representation allows authors to relay methods and results in a platform-independent regularized format that is not tied to a particular neuroimaging software package. Tools are available to export NIDM-Result graphs and associated files from the widely used SPM and FSL software packages, and the NeuroVault repository can import NIDM-Results archives. The specification is publically available at: http://nidm.nidash.org/specs/nidm-results.html.

The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments.

The development of magnetic resonance imaging (MRI) techniques has defined modern neuroimaging. Since its inception, tens of thousands of studies using techniques such as functional MRI and diffusion weighted imaging have allowed for the non-invasive study of the brain. Despite the fact that MRI is routinely used to obtain data for neuroscience research, there has been no widely adopted standard for organizing and describing the data collected in an imaging experiment. This renders sharing and reusing data (within or between labs) difficult if not impossible and unnecessarily complicates the application of automatic pipelines and quality assurance protocols. To solve this problem, we have developed the Brain Imaging Data Structure (BIDS), a standard for organizing and describing MRI datasets. The BIDS standard uses file formats compatible with existing software, unifies the majority of practices already common in the field, and captures the metadata necessary for most common data processing operations.