Tractography reproducibility challenge with empirical data (TraCED): The 2017 ISMRM diffusion study group challenge.

BACKGROUND: Fiber tracking with diffusion-weighted MRI has become an essential tool for estimating in vivo brain white matter architecture. Fiber tracking results are sensitive to the choice of processing method and tracking criteria. PURPOSE: To assess the variability for an algorithm in group studies reproducibility is of critical context. However, reproducibility does not assess the validity of the brain connections. Phantom studies provide concrete quantitative comparisons of methods relative to absolute ground truths, yet do no capture variabilities because of in vivo physiological factors. The ISMRM 2017 TraCED challenge was created to fulfill the gap. STUDY TYPE: A systematic review of algorithms and tract reproducibility studies. SUBJECTS: Single healthy volunteers. FIELD STRENGTH/SEQUENCE: 3.0T, two different scanners by the same manufacturer. The multishell acquisition included b-values of 1000, 2000, and 3000 s/mm2 with 20, 45, and 64 diffusion gradient directions per shell, respectively. ASSESSMENT: Nine international groups submitted 46 tractography algorithm entries each consisting 16 tracts per scan. The algorithms were assessed using intraclass correlation (ICC) and the Dice similarity measure. STATISTICAL TESTS: Containment analysis was performed to assess if the submitted algorithms had containment within tracts of larger volume submissions. This also serves the purpose to detect if spurious submissions had been made. RESULTS: The top five submissions had high ICC and Dice >0.88. Reproducibility was high within the top five submissions when assessed across sessions or across scanners: 0.87-0.97. Containment analysis shows that the top five submissions are contained within larger volume submissions. From the total of 16 tracts as an outcome relatively the number of tracts with high, moderate, and low reproducibility were 8, 4, and 4. DATA CONCLUSION: The different methods clearly result in fundamentally different tract structures at the more conservative specificity choices. Data and challenge infrastructure remain available for continued analysis and provide a platform for comparison. LEVEL OF EVIDENCE: 5 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:234-249.

Empirical single sample quantification of bias and variance in Q-ball imaging.

PURPOSE: The bias and variance of high angular resolution diffusion imaging methods have not been thoroughly explored in the literature and may benefit from the simulation extrapolation (SIMEX) and bootstrap techniques to estimate bias and variance of high angular resolution diffusion imaging metrics. METHODS: The SIMEX approach is well established in the statistics literature and uses simulation of increasingly noisy data to extrapolate back to a hypothetical case with no noise. The bias of calculated metrics can then be computed by subtracting the SIMEX estimate from the original pointwise measurement. The SIMEX technique has been studied in the context of diffusion imaging to accurately capture the bias in fractional anisotropy measurements in DTI. Herein, we extend the application of SIMEX and bootstrap approaches to characterize bias and variance in metrics obtained from a Q-ball imaging reconstruction of high angular resolution diffusion imaging data. RESULTS: The results demonstrate that SIMEX and bootstrap approaches provide consistent estimates of the bias and variance of generalized fractional anisotropy, respectively. The RMSE for the generalized fractional anisotropy estimates shows a 7% decrease in white matter and an 8% decrease in gray matter when compared with the observed generalized fractional anisotropy estimates. On average, the bootstrap technique results in SD estimates that are approximately 97% of the true variation in white matter, and 86% in gray matter. CONCLUSION: Both SIMEX and bootstrap methods are flexible, estimate population characteristics based on single scans, and may be extended for bias and variance estimation on a variety of high angular resolution diffusion imaging metrics.

A deep learning approach to estimation of subject-level bias and variance in high angular resolution diffusion imaging.

The ability to evaluate empirical diffusion MRI acquisitions for quality and to correct the resulting imaging metrics allows for improved inference and increased replicability. Previous work has shown promise for estimation of bias and variance of generalized fractional anisotropy (GFA) but comes at the price of computational complexity. This paper aims to provide methods for estimating GFA, bias of GFA and standard deviation of GFA quickly and accurately. In order to provide a method for bias and variance estimation that can return results faster than the previously studied statistical techniques, three deep, fully-connected neural networks are developed for GFA, bias of GFA, and standard deviation of GFA. The results of these networks are compared to the observed values of the metrics as well as those fit from the statistical techniques (i.e. Simulation Extrapolation (SIMEX) for bias estimation and wild bootstrap for variance estimation). Our GFA network provides predictions that are closer to the true GFA values than a Q-ball fit of the observed data (root-mean-square error (RMSE) 0.0077 vs 0.0082, p < .001). The bias network also shows statistically significant improvement in comparison to the SIMEX-estimated error of GFA (RMSE 0.0071 vs. 0.01, p < .001).

Improved gray matter surface based spatial statistics in neuroimaging studies.

Neuroimaging often involves acquiring high-resolution anatomical images along with other low-resolution image modalities, like diffusion and functional magnetic resonance imaging. Performing gray matter statistics with low-resolution image modalities is a challenge due to registration artifacts and partial volume effects. Gray matter surface based spatial statistics (GS-BSS) has been shown to provide higher sensitivity using gray matter surfaces compared to that of skeletonization approach of gray matter based spatial statistics which is adapted from tract based spatial statistics in diffusion studies. In this study, we improve upon GS-BSS incorporating neurite orientation dispersion and density imaging (NODDI) based search (denoted N-GSBSS) by 1) enhancing metrics mapping from native space, 2) incorporating maximum orientation dispersion index (ODI) search along surface normal, and 3) proposing applicability to other modalities, such as functional MRI (fMRI). We evaluated the performance of N-GSBSS against three baseline pipelines: volume-based registration, FreeSurfer's surface registration and ciftify pipeline for fMRI and simulation studies. First, qualitative mean ODI results are shown for N-GSBSS with and without NODDI based search in comparison with ciftify pipeline. Second, we conducted one-sample t-tests on working memory activations in fMRI to show that the proposed method can aid in the analysis of low resolution fMRI data. Finally we performed a sensitivity test in a simulation study by varying percentage change of intensity values within a region of interest in gray matter probability maps. N-GSBSS showed higher sensitivity in the simulation test compared to the other methods capturing difference between the groups starting at 10% change in the intensity values. The computational time of N-GSBSS is 68 times faster than that of traditional surface-based or 86 times faster than that of ciftify pipeline analysis.

Towards Machine Learning Prediction of Deep Brain Stimulation (DBS) Intra-operative Efficacy Maps.

Deep brain stimulation (DBS) has the potential to improve the quality of life of people with a variety of neurological diseases. A key challenge in DBS is in the placement of a stimulation electrode in the anatomical location that maximizes efficacy and minimizes side effects. Pre-operative localization of the optimal stimulation zone can reduce surgical times and morbidity. Current methods of producing efficacy probability maps follow an anatomical guidance on magnetic resonance imaging (MRI) to identify the areas with the highest efficacy in a population. In this work, we propose to revisit this problem as a classification problem, where each voxel in the MRI is a sample informed by the surrounding anatomy. We use a patch-based convolutional neural network to classify a stimulation coordinate as having a positive reduction in symptoms during surgery. We use a cohort of 187 patients with a total of 2,869 stimulation coordinates, upon which 3D patches were extracted and associated with an efficacy score. We compare our results with a registration-based method of surgical planning. We show an improvement in the classification of intraoperative stimulation coordinates as a positive response in reduction of symptoms with AUC of 0.670 compared to a baseline registration-based approach, which achieves an AUC of 0.627 (p < 0.01). Although additional validation is needed, the proposed classification framework and deep learning method appear well-suited for improving pre-surgical planning and personalize treatment strategies.

Characterization and correlation of signal drift in diffusion weighted MRI.

Diffusion weighted MRI (DWMRI) and the myriad of analysis approaches (from tensors to spherical harmonics and brain tractography to body multi-compartment models) depend on accurate quantification of the apparent diffusion coefficient (ADC). Signal drift during imaging (e.g., due to b0 drift associated with heating) can cause systematic non-linearities that manifest as ADC changes if not corrected. Herein, we present a case study on two phantoms on one scanner. Different scan protocols exhibit different degrees of drift during similar scans and may be sensitive to the order of scans within an exam. Vos et al. recently reviewed the effects of signal drift in DWMRI acquisitions and proposed a temporal model for correction. We propose a novel spatial-temporal model to correct for higher order aspects of the signal drift and derive a statistically robust variant. We evaluate the Vos model and propose a method using two phantoms that mimic the ADC of the relevant brain tissue (0.36-2.2 × 10-3 mm2/s) on a single 3 T scanner. The phantoms are (1) a spherical isotropic sphere consisting of a single concentration of polyvinylpyrrolidone (PVP) and (2) an ice-water phantom with 13 vials of varying PVP concentrations. To characterize the impact of interspersed minimally weighted volumes ("b0's"), image volumes with b-value equal to 0.1 s/mm2 are interspersed every 8, 16, 32, 48, and 96 diffusion weighted volumes in different trials. Signal drift is found to have spatially varying effects that are not accounted for with temporal-only models. The novel model captures drift more accurately (i.e., reduces the overall change per-voxel over the course of a scan) and results in more consistent ADC metrics.

Deep learning reveals untapped information for local white-matter fiber reconstruction in diffusion-weighted MRI.

PURPOSE: Diffusion-weighted magnetic resonance imaging (DW-MRI) is of critical importance for characterizing in-vivo white matter. Models relating microarchitecture to observed DW-MRI signals as a function of diffusion sensitization are the lens through which DW-MRI data are interpreted. Numerous modern approaches offer opportunities to assess more complex intra-voxel structures. Nevertheless, there remains a substantial gap between intra-voxel estimated structures and ground truth captured by 3-D histology. METHODS: Herein, we propose a novel data-driven approach to model the non-linear mapping between observed DW-MRI signals and ground truth structures using a sequential deep neural network regression using residual block deep neural network (ResDNN). Training was performed on two 3-D histology datasets of squirrel monkey brains and validated on a third. A second validation was performed using scan-rescan datasets of 12 subjects from Human Connectome Project. The ResDNN was compared with multiple micro-structure reconstruction methods and super resolved-constrained spherical deconvolution (sCSD) in particular as baseline for both the validations. RESULTS: Angular correlation coefficient (ACC) is a correlation/similarity measure and can be interpreted as accuracy when compared with a ground truth. The median ACC of ResDNN is 0.82 and median ACC's of different variants of CSD are 0.75, 0.77, 0.79. The mean, median and std. of ResDNN & sCSD ACC across 12 subjects from HCP are 0.74, 0.88, 0.31 and 0.61, 0.71, 0.31 respectively. CONCLUSION: This work highlights the ability of deep learning to capture linkages between ex-vivo ground truth data with feasible MRI sequences. The data-driven approach is applicable to human in-vivo data and results in intriguingly high reproducibility of orientation structure.

Consideration of Cerebrospinal Fluid Intensity Variation in Diffusion Weighted MRI.

Diffusion weighted MRI (DW-MRI) depends on accurate quantification signal intensities that reflect directional apparent diffusion coefficients (ADC). Signal drift and fluctuations during imaging can cause systematic non-linearities that manifest as ADC changes if not corrected. Here, we present a case study on a large longitudinal dataset of typical diffusion tensor imaging. We investigate observed variation in the cerebral spinal fluid (CSF) regions of the brain, which should represent compartments with isotropic diffusivity. The study contains 3949 DW-MRI acquisitions of the human brain with 918 subjects and 542 with repeated scan sessions. We provide an analysis of the inter-scan, inter-session, and intra-session variation and an analysis of the associations with the applied diffusion gradient directions. We investigate a hypothesis that CSF models could be used in lieu of an interspersed minimally diffusion-weighted image (b0) correction. Variation in CSF signal is not largely attributable to within-scan dynamic anatomical changes (3.6%), but rather has substantial variation across scan sessions (10.6%) and increased variation across individuals (26.6%). Unfortunately, CSF intensity is not solely explained by a main drift model or a gradient model, but rather has statistically significant associations with both possible explanations. Further exploration is necessary for CSF drift to be used as an effective harmonization technique.

Inter-Scanner Harmonization of High Angular Resolution DW-MRI using Null Space Deep Learning.

Diffusion-weighted magnetic resonance imaging (DW-MRI) allows for non-invasive imaging of the local fiber architecture of the human brain at a millimetric scale. Multiple classical approaches have been proposed to detect both single (e.g., tensors) and multiple (e.g., constrained spherical deconvolution, CSD) fiber population orientations per voxel. However, existing techniques generally exhibit low reproducibility across MRI scanners. Herein, we propose a data-driven technique using a neural network design which exploits two categories of data. First, training data were acquired on three squirrel monkey brains using ex-vivo DW-MRI and histology of the brain. Second, repeated scans of human subjects were acquired on two different scanners to augment the learning of the network proposed. To use these data, we propose a new network architecture, the null space deep network (NSDN), to simultaneously learn on traditional observed/truth pairs (e.g., MRI-histology voxels) along with repeated observations without a known truth (e.g., scan-rescan MRI). The NSDN was tested on twenty percent of the histology voxels that were kept completely blind to the network. NSDN significantly improved absolute performance relative to histology by 3.87% over CSD and 1.42% over a recently proposed deep neural network approach. Moreover, it improved reproducibility on the paired data by 21.19% over CSD and 10.09% over a recently proposed deep approach. Finally, NSDN improved generalizability of the model to a third in vivo human scanner (which was not used in training) by 16.08% over CSD and 10.41% over a recently proposed deep learning approach. This work suggests that data-driven approaches for local fiber reconstruction are more reproducible, informative and precise and offers a novel, practical method for determining these models.

Empirical estimation of intravoxel structure with persistent angular structure and Q-ball models of diffusion weighted MRI.

The diffusion tensor model is nonspecific in regions where micrometer structural patterns are inconsistent at the millimeter scale (i.e., brain regions with pathways that cross, bend, branch, fan, etc.). Numerous models have been proposed to represent crossing fibers and complex intravoxel structure from in vivo diffusion weighted magnetic resonance imaging (e.g., high angular resolution diffusion imaging-HARDI). Here, we present an empirical comparison of two HARDI approaches-persistent angular structure MRI (PAS-MRI) and Q-ball-using a newly acquired reproducibility dataset. Briefly, a single subject was scanned 11 times with 96 diffusion weighted directions and 10 reference volumes for each of two [Formula: see text] values (1000 and [Formula: see text] for a total of 2144 volumes). Empirical reproducibility of intravoxel fiber fractions (number/strength of peaks), angular orientation, and fractional anisotropy was compared with metrics from a traditional tensor analysis approach, focusing on [Formula: see text] values of 1000 and [Formula: see text]. PAS-MRI is shown to be more reproducible than Q-ball and offers advantages at low [Formula: see text] values. However, there are substantial and biologically meaningful differences between the intravoxel structures estimated both in terms of analysis method as well as by [Formula: see text] value. The two methods suggest a fundamentally different microarchitecture of the human brain; therefore, it is premature to perform meta-analysis or combine results across HARDI studies using a different analysis model or acquisition sequences.

Constructing Statistically Unbiased Cortical Surface Templates Using Feature-Space Covariance.

The choice of surface template plays an important role in cross-sectional subject analyses involving cortical brain surfaces because there is a tendency toward registration bias given variations in inter-individual and inter-group sulcal and gyral patterns. In order to account for the bias and spatial smoothing, we propose a feature-based unbiased average template surface. In contrast to prior approaches, we factor in the sample population covariance and assign weights based on feature information to minimize the influence of covariance in the sampled population. The mean surface is computed by applying the weights obtained from an inverse covariance matrix, which guarantees that multiple representations from similar groups (e.g., involving imaging, demographic, diagnosis information) are down-weighted to yield an unbiased mean in feature space. Results are validated by applying this approach in two different applications. For evaluation, the proposed unbiased weighted surface mean is compared with un-weighted means both qualitatively and quantitatively (mean squared error and absolute relative distance of both the means with baseline). In first application, we validated the stability of the proposed optimal mean on a scan-rescan reproducibility dataset by incrementally adding duplicate subjects. In the second application, we used clinical research data to evaluate the difference between the weighted and unweighted mean when different number of subjects were included in control versus schizophrenia groups. In both cases, the proposed method achieved greater stability that indicated reduced impacts of sampling bias. The weighted mean is built based on covariance information in feature space as opposed to spatial location, thus making this a generic approach to be applicable to any feature of interest.

SHARD: Spherical Harmonic-based Robust Outlier Detection for HARDI Methods.

High Angular Resolution Diffusion Imaging (HARDI) models are used to capture complex intra-voxel microarchitectures. The magnetic resonance imaging sequences that are sensitized to diffusion are often highly accelerated and prone to motion, physiologic, and imaging artifacts. In diffusion tensor imaging, robust statistical approaches have been shown to greatly reduce these adverse factors without human intervention. Similar approaches would be possible with HARDI methods, but robust versions of each distinct HARDI approach would be necessary. To avoid the computational and pragmatic burdens of creating individual robust HARDI analysis variants, we propose a robust outlier imputation model to mitigate outliers prior to traditional HARDI analysis. This model uses a weighted spherical harmonic fit of diffusion weighted magnetic resonance imaging scans to estimate the values which had been corrupted during acquisition to restore them. Briefly, spherical harmonics of 6th order were used to generate basis function which were weighted by diffusion signal for detection of outliers. For validation, a single healthy volunteer was scanned for a single session comprising of two scans one without head movement and the other with deliberate head movement at a b-value of 3000 s/mm2 with 64 diffusion weighted directions with a single b0 (5 averages) per scan. The deliberate motion from the volunteer created natural artifacts in the acquisition of one of the scans. The imputation model shows reduction in root mean squared error of the raw signal intensities and improvement for the HARDI method Q-ball in terms of the Angular Correlation Coefficient. The results reveal that there is quantitative and qualitative improvement. The proposed model can be used as general pre-processing model before implementing any HARDI model in general to restore the artifacts which are created because of the outlier diffusion signal in certain gradient volumes.

Evaluation of inter-site bias and variance in diffusion-weighted MRI.

An understanding of the bias and variance of diffusion weighted magnetic resonance imaging (DW-MRI) acquisitions across scanners, study sites, or over time is essential for the incorporation of multiple data sources into a single clinical study. Studies that combine samples from various sites may be introducing confounding due to site-specific artifacts and patterns. Differences in bias and variance across sites may render the scans incomparable, and, without correction, any inferences obtained from these data are misleading. We present an analysis of the bias and variance of scans of the same subjects across different sites and evaluate their impact on statistical analyses. In previous work, we presented a simulation extrapolation (SIMEX) technique for bias estimation as well as a wild bootstrap technique for variance estimation in metrics obtained from a Q-ball imaging (QBI) reconstruction of empirical high angular resolution diffusion imaging (HARDI) data. We now apply those techniques to data acquired from 5 healthy volunteers on 3 independent scanners under closely matched acquisition protocols. The bias and variance of GFA measurements were estimated on a voxel-wise basis for each scan and compared across study sites to identify site-specific differences. Further, we provide model recommendations that can be used to determine the extent of the impact of bias and variance as well as aspects of the analysis to account for these differences. We include a decision tree to help researchers determine if model adjustments are necessary based on the bias and variance results.

Empirical reproducibility, sensitivity, and optimization of acquisition protocol, for Neurite Orientation Dispersion and Density Imaging using AMICO.

Neurite Orientation Dispersion and Density Imaging (NODDI) has been gaining prominence for estimating multiple diffusion compartments from MRI data acquired in a clinically feasible time. To establish a pathway for adoption of NODDI in clinical studies, it is important to understand the sensitivity and reproducibility of NODDI metrics on empirical data in the context of acquisition protocol and brain anatomy. Previous studies addressed reproducibility across the 3 T scanners and within session and between subject reproducibility at 1.5 T and 3 T. However, empirical reproducibility on the performance of NODDI metrics based on b-value and diffusion-sensitized directions has not yet been addressed. In this study, we investigate a high angular resolution dataset with 11 repeats of a study with five b-values shells (1000, 1500, 2000, 2500 and 3000 s/mm2) and 96 directions per shell on a single subject. We validated the findings with a dataset from second subject with 10 repeats and 3 b-value shells (1000, 2000, 3000 s/mm2). The NODDI model was estimated using Accelerated Microstructure Imaging via Convex Optimization (AMICO) for different b-values and gradient directions on two-shell High Angular Resolution Density Imaging (HARDI) data fixing the lower shell at b = 1000 s/mm2. NODDI model applied to all acquired imaging data was used as a baseline gold standard for comparison. Additionally, we characterize orientation dispersion index (ODI) reproducibility using single-shell data. The experimental findings confirmed the sensitivity of intracellular volume fraction (Vic) with the choice of outer shell b-value more than with the choice of gradient directions. On the other hand, ODI is more sensitive to the number of gradient directions compared to b-value selection. Single-shell results for ODI are more comparable to 2-shell data at lower b-values than higher b-values. Recommended settings by region of interest and acquisition time are reported for the researchers considering using NODDI in human studies and/or comparing results across acquisition protocols.

Gray Matter Surface based Spatial Statistics (GS-BSS) in Diffusion Microstructure.

Tract-based spatial statistics (TBSS) has proven to be a popular technique for performing voxel-wise statistical analysis that aims to improve sensitivity and interpretability of analysis of multi-subject diffusion imaging studies in white matter. With the advent of advanced diffusion MRI models - e.g., the neurite orientation dispersion density imaging (NODDI), it is of interest to analyze microstructural changes within gray matter (GM). A recent study has proposed using NODDI in gray matter based spatial statistics (N-GBSS) to perform voxel-wise statistical analysis on GM microstructure. N-GBSS adapts TBSS by skeletonizing the GM and projecting diffusion metrics to a cortical ribbon. In this study, we propose an alternate approach, known as gray matter surface based spatial statistics (GS-BSS), to perform statistical analysis using gray matter surfaces by incorporating established methods of registration techniques of GM surface segmentation on structural images. Diffusion microstructure features from NODDI and GM surfaces are transferred to standard space. All the surfaces are then projected onto a common GM surface non-linearly using diffeomorphic spectral matching on cortical surfaces. Prior post-mortem studies have shown reduced dendritic length in prefrontal cortex region in schizophrenia and bipolar disorder population. To validate the results, statistical tests are compared between GS-BSS and N-GBSS to study the differences between healthy and psychosis population. Significant results confirming the microstructural changes are presented. GS-BSS results show higher sensitivity to group differences between healthy and psychosis population in previously known regions.