E(3) × SO(3)-Equivariant Networks for Spherical Deconvolution in Diffusion MRI.

We present Roto-Translation Equivariant Spherical Deconvolution (RT-ESD), an E(3)×SO(3) equivariant framework for sparse deconvolution of volumes where each voxel contains a spherical signal. Such 6D data naturally arises in diffusion MRI (dMRI), a medical imaging modality widely used to measure microstructure and structural connectivity. As each dMRI voxel is typically a mixture of various overlapping structures, there is a need for blind deconvolution to recover crossing anatomical structures such as white matter tracts. Existing dMRI work takes either an iterative or deep learning approach to sparse spherical deconvolution, yet it typically does not account for relationships between neighboring measurements. This work constructs equivariant deep learning layers which respect to symmetries of spatial rotations, reflections, and translations, alongside the symmetries of voxelwise spherical rotations. As a result, RT-ESD improves on previous work across several tasks including fiber recovery on the DiSCo dataset, deconvolution-derived partial volume estimation on real-world in vivo human brain dMRI, and improved downstream reconstruction of fiber tractograms on the Tractometer dataset. Our implementation is available at https://github.com/AxelElaldi/e3so3_conv.

White matter development and language abilities during infancy in autism spectrum disorder.

White matter (WM) fiber tract differences are present in autism spectrum disorder (ASD) and could be important markers of behavior. One of the earliest phenotypic differences in ASD are language atypicalities. Although language has been linked to WM in typical development, no work has evaluated this association in early ASD. Participants came from the Infant Brain Imaging Study and included 321 infant siblings of children with ASD at high likelihood (HL) for developing ASD; 70 HL infants were later diagnosed with ASD (HL-ASD), and 251 HL infants were not diagnosed with ASD (HL-Neg). A control sample of 140 low likelihood infants not diagnosed with ASD (LL-Neg) were also included. Infants contributed expressive language, receptive language, and diffusion tensor imaging data at 6-, 12-, and 24 months. Mixed effects regression models were conducted to evaluate associations between WM and language trajectories. Trajectories of microstructural changes in the right arcuate fasciculus were associated with expressive language development. HL-ASD infants demonstrated a different developmental pattern compared to the HL-Neg and LL-Neg groups, wherein the HL-ASD group exhibited a positive association between WM fractional anisotropy and language whereas HL-Neg and LL-Neg groups showed weak or no association. No other fiber tracts demonstrated significant associations with language. In conclusion, results indicated arcuate fasciculus WM is linked to language in early toddlerhood for autistic toddlers, with the strongest associations emerging around 24 months. To our knowledge, this is the first study to evaluate associations between language and WM development during the pre-symptomatic period in ASD.

Hierarchical Geodesic Polynomial Model for Multilevel Analysis of Longitudinal Shape.

Longitudinal analysis is a core aspect of many medical applications for understanding the relationship between an anatomical subject's function and its trajectory of shape change over time. Whereas mixed-effects (or hierarchical) modeling is the statistical method of choice for analysis of longitudinal data, we here propose its extension as hierarchical geodesic polynomial model (HGPM) for multilevel analyses of longitudinal shape data. 3D shapes are transformed to a non-Euclidean shape space for regression analysis using geodesics on a high dimensional Riemannian manifold. At the subject-wise level, each individual trajectory of shape change is represented by a univariate geodesic polynomial model on timestamps. At the population level, multivariate polynomial expansion is applied to uni/multivariate geodesic polynomial models for both anchor points and tangent vectors. As such, the trajectory of an individual subject's shape changes over time can be modeled accurately with a reduced number of parameters, and population-level effects from multiple covariates on trajectories can be well captured. The implemented HGPM is validated on synthetic examples of points on a unit 3D sphere. Further tests on clinical 4D right ventricular data show that HGPM is capable of capturing observable effects on shapes attributed to changes in covariates, which are consistent with qualitative clinical evaluations. HGPM demonstrates its effectiveness in modeling shape changes at both subject-wise and population levels, which is promising for future studies of the relationship between shape changes over time and the level of dysfunction severity on anatomical objects associated with disease.

A Prospective Evaluation of Infant Cerebellar-Cerebral Functional Connectivity in Relation to Behavioral Development in Autism Spectrum Disorder.

Background: Autism spectrum disorder (ASD) is a neurodevelopmental disorder diagnosed based on social impairment, restricted interests, and repetitive behaviors. Contemporary theories posit that cerebellar pathology contributes causally to ASD by disrupting error-based learning (EBL) during infancy. The present study represents the first test of this theory in a prospective infant sample, with potential implications for ASD detection. Methods: Data from the Infant Brain Imaging Study (n = 94, 68 male) were used to examine 6-month cerebellar functional connectivity magnetic resonance imaging in relation to later (12/24-month) ASD-associated behaviors and outcomes. Hypothesis-driven univariate analyses and machine learning-based predictive tests examined cerebellar-frontoparietal network (FPN; subserves error signaling in support of EBL) and cerebellar-default mode network (DMN; broadly implicated in ASD) connections. Cerebellar-FPN functional connectivity was used as a proxy for EBL, and cerebellar-DMN functional connectivity provided a comparative foil. Data-driven functional connectivity magnetic resonance imaging enrichment examined brain-wide behavioral associations, with post hoc tests of cerebellar connections. Results: Cerebellar-FPN and cerebellar-DMN connections did not demonstrate associations with ASD. Functional connectivity magnetic resonance imaging enrichment identified 6-month correlates of later ASD-associated behaviors in networks of a priori interest (FPN, DMN), as well as in cingulo-opercular (also implicated in error signaling) and medial visual networks. Post hoc tests did not suggest a role for cerebellar connections. Conclusions: We failed to identify cerebellar functional connectivity-based contributions to ASD. However, we observed prospective correlates of ASD-associated behaviors in networks that support EBL. Future studies may replicate and extend network-level positive results, and tests of the cerebellum may investigate brain-behavior associations at different developmental stages and/or using different neuroimaging modalities.

SlicerSALT: From Medical Images to Quantitative Insights of Anatomy.

Three-dimensional (3D) shape lies at the core of understanding the physical objects that surround us. In the biomedical field, shape analysis has been shown to be powerful in quantifying how anatomy changes with time and disease. The Shape AnaLysis Toolbox (SALT) was created as a vehicle for disseminating advanced shape methodology as an open source, free, and comprehensive software tool. We present new developments in our shape analysis software package, including easy-to-interpret statistical methods to better leverage the quantitative information contained in SALT's shape representations. We also show SlicerPipelines, a module to improve the usability of SALT by facilitating the analysis of large-scale data sets, automating workflows for non-expert users, and allowing the distribution of reproducible workflows.

Infant Visual Brain Development and Inherited Genetic Liability in Autism.

OBJECTIVE: Autism spectrum disorder (ASD) is heritable, and younger siblings of ASD probands are at higher likelihood of developing ASD themselves. Prospective MRI studies of siblings report that atypical brain development precedes ASD diagnosis, although the link between brain maturation and genetic factors is unclear. Given that familial recurrence of ASD is predicted by higher levels of ASD traits in the proband, the authors investigated associations between proband ASD traits and brain development among younger siblings. METHODS: In a sample of 384 proband-sibling pairs (89 pairs concordant for ASD), the authors examined associations between proband ASD traits and sibling brain development at 6, 12, and 24 months in key MRI phenotypes: total cerebral volume, cortical surface area, extra-axial cerebrospinal fluid, occipital cortical surface area, and splenium white matter microstructure. Results from primary analyses led the authors to implement a data-driven approach using functional connectivity MRI at 6 months. RESULTS: Greater levels of proband ASD traits were associated with larger total cerebral volume and surface area and larger surface area and reduced white matter integrity in components of the visual system in siblings who developed ASD. This aligned with weaker functional connectivity between several networks and the visual system among all siblings during infancy. CONCLUSIONS: The findings provide evidence that specific early brain MRI phenotypes of ASD reflect quantitative variation in familial ASD traits. Multimodal anatomical and functional convergence on cortical regions, fiber pathways, and functional networks involved in visual processing suggest that inherited liability has a role in shaping the prodromal development of visual circuitry in ASD.

Subcortical Brain Development in Autism and Fragile X Syndrome: Evidence for Dynamic, Age- and Disorder-Specific Trajectories in Infancy.

OBJECTIVE: Previous research has demonstrated that the amygdala is enlarged in children with autism spectrum disorder (ASD). However, the precise onset of this enlargement during infancy, how it relates to later diagnostic behaviors, whether the timing of enlargement in infancy is specific to the amygdala, and whether it is specific to ASD (or present in other neurodevelopmental disorders, such as fragile X syndrome) are all unknown. METHODS: Longitudinal MRIs were acquired at 6-24 months of age in 29 infants with fragile X syndrome, 58 infants at high likelihood for ASD who were later diagnosed with ASD, 212 high-likelihood infants not diagnosed with ASD, and 109 control infants (1,099 total scans). RESULTS: Infants who developed ASD had typically sized amygdala volumes at 6 months, but exhibited significantly faster amygdala growth between 6 and 24 months, such that by 12 months the ASD group had significantly larger amygdala volume (Cohen's d=0.56) compared with all other groups. Amygdala growth rate between 6 and 12 months was significantly associated with greater social deficits at 24 months when the infants were diagnosed with ASD. Infants with fragile X syndrome had a persistent and significantly enlarged caudate volume at all ages between 6 and 24 months (d=2.12), compared with all other groups, which was significantly associated with greater repetitive behaviors. CONCLUSIONS: This is the first MRI study comparing fragile X syndrome and ASD in infancy, demonstrating strikingly different patterns of brain and behavior development. Fragile X syndrome-related changes were present from 6 months of age, whereas ASD-related changes unfolded over the first 2 years of life, starting with no detectable group differences at 6 months. Increased amygdala growth rate between 6 and 12 months occurs prior to social deficits and well before diagnosis. This gradual onset of brain and behavior changes in ASD, but not fragile X syndrome, suggests an age- and disorder-specific pattern of cascading brain changes preceding autism diagnosis.

ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration.

Establishing voxelwise semantic correspondence across distinct imaging modalities is a foundational yet formidable computer vision task. Current multi-modality registration techniques maximize hand-crafted inter-domain similarity functions, are limited in modeling nonlinear intensity-relationships and deformations, and may require significant re-engineering or underperform on new tasks, datasets, and domain pairs. This work presents ContraReg, an unsupervised contrastive representation learning approach to multi-modality deformable registration. By projecting learned multi-scale local patch features onto a jointly learned inter-domain embedding space, ContraReg obtains representations useful for non-rigid multi-modality alignment. Experimentally, ContraReg achieves accurate and robust results with smooth and invertible deformations across a series of baselines and ablations on a neonatal T1-T2 brain MRI registration task with all methods validated over a wide range of deformation regularization strengths.

Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis.

Recent self-supervised advances in medical computer vision exploit the global and local anatomical self-similarity for pretraining prior to downstream tasks such as segmentation. However, current methods assume i.i.d. image acquisition, which is invalid in clinical study designs where follow-up longitudinal scans track subject-specific temporal changes. Further, existing self-supervised methods for medically-relevant image-to-image architectures exploit only spatial or temporal self-similarity and do so via a loss applied only at a single image-scale, with naive multi-scale spatiotemporal extensions collapsing to degenerate solutions. To these ends, this paper makes two contributions: (1) It presents a local and multi-scale spatiotemporal representation learning method for image-to-image architectures trained on longitudinal images. It exploits the spatiotemporal self-similarity of learned multi-scale intra-subject image features for pretraining and develops several feature-wise regularizations that avoid degenerate representations; (2) During finetuning, it proposes a surprisingly simple self-supervised segmentation consistency regularization to exploit intra-subject correlation. Benchmarked across various segmentation tasks, the proposed framework outperforms both well-tuned randomly-initialized baselines and current self-supervised techniques designed for both i.i.d. and longitudinal datasets. These improvements are demonstrated across both longitudinal neurodegenerative adult MRI and developing infant brain MRI and yield both higher performance and longitudinal consistency.

Longitudinal Prediction of Infant MR Images With Multi-Contrast Perceptual Adversarial Learning.

The infant brain undergoes a remarkable period of neural development that is crucial for the development of cognitive and behavioral capacities (Hasegawa et al., 2018). Longitudinal magnetic resonance imaging (MRI) is able to characterize the developmental trajectories and is critical in neuroimaging studies of early brain development. However, missing data at different time points is an unavoidable occurrence in longitudinal studies owing to participant attrition and scan failure. Compared to dropping incomplete data, data imputation is considered a better solution to address such missing data in order to preserve all available samples. In this paper, we adapt generative adversarial networks (GAN) to a new application: longitudinal image prediction of structural MRI in the first year of life. In contrast to existing medical image-to-image translation applications of GANs, where inputs and outputs share a very close anatomical structure, our task is more challenging as brain size, shape and tissue contrast vary significantly between the input data and the predicted data. Several improvements over existing GAN approaches are proposed to address these challenges in our task. To enhance the realism, crispness, and accuracy of the predicted images, we incorporate both a traditional voxel-wise reconstruction loss as well as a perceptual loss term into the adversarial learning scheme. As the differing contrast changes in T1w and T2w MR images in the first year of life, we incorporate multi-contrast images leading to our proposed 3D multi-contrast perceptual adversarial network (MPGAN). Extensive evaluations are performed to assess the qualityand fidelity of the predicted images, including qualitative and quantitative assessments of the image appearance, as well as quantitative assessment on two segmentation tasks. Our experimental results show that our MPGAN is an effective solution for longitudinal MR image data imputation in the infant brain. We further apply our predicted/imputed images to two practical tasks, a regression task and a classification task, in order to highlight the enhanced task-related performance following image imputation. The results show that the model performance in both tasks is improved by including the additional imputed data, demonstrating the usability of the predicted images generated from our approach.

Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization.

Deep networks are now ubiquitous in large-scale multi-center imaging studies. However, the direct aggregation of images across sites is contraindicated for downstream statistical and deep learning-based image analysis due to inconsistent contrast, resolution, and noise. To this end, in the absence of paired data, variations of Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. Importantly, these methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which limit their reliable adoption in real-world medical imaging. In this work, based on an underlying assumption that morphological shape is consistent across imaging sites, we propose a segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout. We replace the affine transformations used in the normalization layers within generative networks with trainable scale and shift parameters conditioned on jointly learned anatomical segmentation embeddings to modulate features at every level of translation. We evaluate our methodologies against recent baselines across several imaging modalities (T1w MRI, FLAIR MRI, and OCT) on datasets with and without lesions. Segmentation-renormalization for translation GANs yields superior image harmonization as quantified by Inception distances, demonstrates improved downstream utility via post-hoc segmentation accuracy, and improved robustness to translation perturbation and self-adversarial attacks.

Equivariant Spherical Deconvolution: Learning Sparse Orientation Distribution Functions from Spherical Data.

We present a rotation-equivariant self-supervised learning framework for the sparse deconvolution of non-negative scalar fields on the unit sphere. Spherical signals with multiple peaks naturally arise in Diffusion MRI (dMRI), where each voxel consists of one or more signal sources corresponding to anisotropic tissue structure such as white matter. Due to spatial and spectral partial voluming, clinically-feasible dMRI struggles to resolve crossing-fiber white matter configurations, leading to extensive development in spherical deconvolution methodology to recover underlying fiber directions. However, these methods are typically linear and struggle with small crossing-angles and partial volume fraction estimation. In this work, we improve on current methodologies by nonlinearly estimating fiber structures via self-supervised spherical convolutional networks with guaranteed equivariance to spherical rotation. We perform validation via extensive single and multi-shell synthetic benchmarks demonstrating competitive performance against common base-lines. We further show improved downstream performance on fiber tractography measures on the Tractometer benchmark dataset. Finally, we show downstream improvements in terms of tractography and partial volume estimation on a multi-shell dataset of human subjects.

Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI.

Current deep learning approaches for diffusion MRI modeling circumvent the need for densely-sampled diffusion-weighted images (DWIs) by directly predicting microstructural indices from sparsely-sampled DWIs. However, they implicitly make unrealistic assumptions of static q-space sampling during training and reconstruction. Further, such approaches can restrict downstream usage of variably sampled DWIs for usages including the estimation of microstructural indices or tractography. We propose a generative adversarial translation framework for high-quality DWI synthesis with arbitrary q-space sampling given commonly acquired structural images (e.g., B0, T1, T2). Our translation network linearly modulates its internal representations conditioned on continuous q-space information, thus removing the need for fixed sampling schemes. Moreover, this approach enables downstream estimation of high-quality microstructural maps from arbitrarily subsampled DWIs, which may be particularly important in cases with sparsely sampled DWIs. Across several recent methodologies, the proposed approach yields improved DWI synthesis accuracy and fidelity with enhanced downstream utility as quantified by the accuracy of scalar microstructure indices estimated from the synthesized images. Code is available at https://github.com/mengweiren/q-space-conditioned-dwi-synthesis.

A Novel Method for High-Dimensional Anatomical Mapping of Extra-Axial Cerebrospinal Fluid: Application to the Infant Brain.

Cerebrospinal fluid (CSF) plays an essential role in early postnatal brain development. Extra-axial CSF (EA-CSF) volume, which is characterized by CSF in the subarachnoid space surrounding the brain, is a promising marker in the early detection of young children at risk for neurodevelopmental disorders. Previous studies have focused on global EA-CSF volume across the entire dorsal extent of the brain, and not regionally-specific EA-CSF measurements, because no tools were previously available for extracting local EA-CSF measures suitable for localized cortical surface analysis. In this paper, we propose a novel framework for the localized, cortical surface-based analysis of EA-CSF. The proposed processing framework combines probabilistic brain tissue segmentation, cortical surface reconstruction, and streamline-based local EA-CSF quantification. The quantitative analysis of local EA-CSF was applied to a dataset of typically developing infants with longitudinal MRI scans from 6 to 24 months of age. There was a high degree of consistency in the spatial patterns of local EA-CSF across age using the proposed methods. Statistical analysis of local EA-CSF revealed several novel findings: several regions of the cerebral cortex showed reductions in EA-CSF from 6 to 24 months of age, and specific regions showed higher local EA-CSF in males compared to females. These age-, sex-, and anatomically-specific patterns of local EA-CSF would not have been observed if only a global EA-CSF measure were utilized. The proposed methods are integrated into a freely available, open-source, cross-platform, user-friendly software tool, allowing neuroimaging labs to quantify local extra-axial CSF in their neuroimaging studies to investigate its role in typical and atypical brain development.

Sex differences associated with corpus callosum development in human infants: A longitudinal multimodal imaging study.

The corpus callosum (CC) is the largest connective pathway in the human brain, linking cerebral hemispheres. There is longstanding debate in the scientific literature whether sex differences are evident in this structure, with many studies indicating the structure is larger in females. However, there are few data pertaining to this issue in infancy, during which time the most rapid developmental changes to the CC occur. In this study, we examined longitudinal brain imaging data collected from 104 infants at ages 6, 12, and 24 months. We identified sex differences in brain-size adjusted CC area and thickness characterized by a steeper rate of growth in males versus females from ages 6-24 months. In contrast to studies of older children and adults, CC size was larger for male compared to female infants. Based on diffusion tensor imaging data, we found that CC thickness is significantly associated with underlying microstructural organization. However, we observed no sex differences in the association between microstructure and thickness, suggesting that the role of factors such as axon density and/or myelination in determining CC size is generally equivalent between sexes. Finally, we found that CC length was negatively associated with nonverbal ability among females.

Hierarchical geodesic modeling on the diffusion orientation distribution function for longitudinal DW-MRI analysis.

The analysis of anatomy that undergoes rapid changes, such as neuroimaging of the early developing brain, greatly benefits from spatio-temporal statistical analysis methods to represent population variations but also subject-wise characteristics over time. Methods for spatio-temporal modeling and for analysis of longitudinal shape and image data have been presented before, but, to our knowledge, not for diffusion weighted MR images (DW-MRI) fitted with higher-order diffusion models. To bridge the gap between rapidly evolving DW-MRI methods in longitudinal studies and the existing frameworks, which are often limited to the analysis of derived measures like fractional anisotropy (FA), we propose a new framework to estimate a population trajectory of longitudinal diffusion orientation distribution functions (dODFs) along with subject-specific changes by using hierarchical geodesic modeling. The dODF is an angular profile of the diffusion probability density function derived from high angular resolution diffusion imaging (HARDI) and we consider the dODF with the square-root representation to lie on the unit sphere in a Hilbert space, which is a well-known Riemannian manifold, to respect the nonlinear characteristics of dODFs. The proposed method is validated on synthetic longitudinal dODF data and tested on a longitudinal set of 60 HARDI images from 25 healthy infants to characterize dODF changes associated with early brain development.

A framework to construct a longitudinal DW-MRI infant atlas based on mixed effects modeling of dODF coefficients.

Building of atlases plays a crucial role in the analysis of brain images. In scenarios where early growth, aging or disease trajectories are of key importance, longitudinal atlases become necessary as references, most often created from cross-sectional data. New opportunities will be offered by creating longitudinal brain atlases from longitudinal subject-specific image data, where explicit modeling of subject's variability in slope and intercept leads to a more robust estimation of average trajectories but also to estimates of confidence bounds. This work focuses on a framework to build a continuous 4D atlas from longitudinal high angular resolution diffusion images (HARDI) where, unlike atlases of derived scalar diffusion indices such as FA, statistics on dODFs is preserved. Multi-scalar images obtained from DW images are used for geometric alignment, and linear mixed-effects modeling from longitudinal diffusion orientation distribution functions (dODF) leads to estimation of continuous dODF changes. The proposed method is applied to a longitudinal dataset of HARDI images from healthy developing infants in the age range of 3 to 36 months. Verification of mixed-effects modeling is obtained by voxel-wise goodness of fit calculations. To demonstrate the potential of our method, we display changes of longitudinal atlas using dODF and derived generalized fractional anisotropy (GFA) of dODF. We also investigate white matter maturation patterns in genu, body, and splenium of the corpus callosum. The framework can be used to build an average dODF atlas from HARDI data and to derive subject-specific and population-based longitudinal change trajectories.

Trajectories from Distribution-valued Functional Curves: A Unified Wasserstein Framework☠.

Temporal changes in medical images are often evaluated along a parametrized function that represents a structure of interest (e.g. white matter tracts). By attributing samples along these functions with distributions of image properties in the local neighborhood, we create distribution-valued signatures for these functions. We propose a novel and comprehensive framework which models their temporal evolution trajectories. This is achieved under the unifying scheme of Wasserstein distance metric. The regression problem is formulated as a constrained optimization problem and solved using an alternating projection algorithm. The solution simultaneously preserves the functional characteristics of the curve, models the temporal change in distribution profiles and forces the estimated distributions to be valid. Hypothesis testing is applied in two ways using Wasserstein based test statistics. Validation is presented on synthetic data. Detection of delayed growth is shown on DTI tracts, for a pediatric subject with respect to a healthy population of infants.

Multi-modal Image Fusion for Multispectral Super-resolution in Microscopy.

Spectral imaging is a ubiquitous tool in modern biochemistry. Despite acquiring dozens to thousands of spectral channels, existing technology cannot capture spectral images at the same spatial resolution as structural microscopy. Due to partial voluming and low light exposure, spectral images are often difficult to interpret and analyze. This highlights a need to upsample the low-resolution spectral image by using spatial information contained in the high-resolution image, thereby creating a fused representation with high specificity both spatially and spectrally. In this paper, we propose a framework for the fusion of co-registered structural and spectral microscopy images to create super-resolved representations of spectral images. As a first application, we super-resolve spectral images of retinal tissue imaged with confocal laser scanning microscopy, by using spatial information from structured illumination microscopy. Second, we super-resolve mass spectroscopic images of mouse brain tissue, by using spatial information from high-resolution histology images. We present a systematic validation of model assumptions crucial towards maintaining the original nature of spectra and the applicability of super-resolution. Goodness-of-fit for spectral predictions are evaluated through functional R 2 values, and the spatial quality of the super-resolved images are evaluated using normalized mutual information.