Covariant Image Representation with Applications to Classification Problems in Medical Imaging.

Images are often considered as functions defined on the image domains, and as functions, their (intensity) values are usually considered to be invariant under the image domain transforms. This functional viewpoint is both influential and prevalent, and it provides the justification for comparing images using functional Lp -norms. However, with the advent of more advanced sensing technologies and data processing methods, the definition and the variety of images has been broadened considerably, and the long-cherished functional paradigm for images is becoming inadequate and insufficient. In this paper, we introduce the formal notion of covariant images and study two types of covariant images that are important in medical image analysis, symmetric positive-definite tensor fields and Gaussian mixture fields, images whose sample values covary i.e., jointly vary with image domain transforms rather than being invariant to them. We propose a novel similarity measure between a pair of covariant images considered as embedded shapes (manifolds) in the ambient space, a Cartesian product of the image and its sample-value domains. The similarity measure is based on matching the two embedded low-dimensional shapes, and both the extrinsic geometry of the ambient space and the intrinsic geometry of the shapes are incorporated in computing the similarity measure. Using this similarity as an affinity measure in a supervised learning framework, we demonstrate its effectiveness on two challenging classification problems: classification of brain MR images based on patients' age and (Alzheimer's) disease status and seizure detection from high angular resolution diffusion magnetic resonance scans of rat brains.

An Empirical Bayes Approach to Shrinkage Estimation on the Manifold of Symmetric Positive-Definite Matrices.

The James-Stein estimator is an estimator of the multivariate normal mean and dominates the maximum likelihood estimator (MLE) under squared error loss. The original work inspired great interest in developing shrinkage estimators for a variety of problems. Nonetheless, research on shrinkage estimation for manifold-valued data is scarce. In this article, we propose shrinkage estimators for the parameters of the Log-Normal distribution defined on the manifold of N × N symmetric positive-definite matrices. For this manifold, we choose the Log-Euclidean metric as its Riemannian metric since it is easy to compute and has been widely used in a variety of applications. By using the Log-Euclidean distance in the loss function, we derive a shrinkage estimator in an analytic form and show that it is asymptotically optimal within a large class of estimators that includes the MLE, which is the sample Fréchet mean of the data. We demonstrate the performance of the proposed shrinkage estimator via several simulated data experiments. Additionally, we apply the shrinkage estimator to perform statistical inference in both diffusion and functional magnetic resonance imaging problems.

A robust variational approach for simultaneous smoothing and estimation of DTI.

Estimating diffusion tensors is an essential step in many applications - such as diffusion tensor image (DTI) registration, segmentation and fiber tractography. Most of the methods proposed in the literature for this task are not simultaneously statistically robust and feature preserving techniques. In this paper, we propose a novel and robust variational framework for simultaneous smoothing and estimation of diffusion tensors from diffusion MRI. Our variational principle makes use of a recently introduced total Kullback-Leibler (tKL) divergence for DTI regularization. tKL is a statistically robust dissimilarity measure for diffusion tensors, and regularization by using tKL ensures the symmetric positive definiteness of tensors automatically. Further, the regularization is weighted by a non-local factor adapted from the conventional non-local means filters. Finally, for the data fidelity, we use the nonlinear least-squares term derived from the Stejskal-Tanner model. We present experimental results depicting the positive performance of our method in comparison to competing methods on synthetic and real data examples.

Horospherical Decision Boundaries for Large Margin Classification in Hyperbolic Space.

Hyperbolic spaces have been quite popular in the recent past for representing hierarchically organized data. Further, several classification algorithms for data in these spaces have been proposed in the literature. These algorithms mainly use either hyperplanes or geodesics for decision boundaries in a large margin classifiers setting leading to a non-convex optimization problem. In this paper, we propose a novel large margin classifier based on horospherical decision boundaries that leads to a geodesically convex optimization problem that can be optimized using any Riemannian gradient descent technique guaranteeing a globally optimal solution. We present several experiments depicting the competitive performance of our classifier in comparison to SOTA.

Geometric Deep Learning for Unsupervised Registration of Diffusion Magnetic Resonance Images.

Deep learning based models for registration predict a transformation directly from moving and fixed image appearances. These models have revolutionized the field of medical image registration, achieving accuracy on-par with classical registration methods at a fraction of the computation time. Unfortunately, most deep learning based registration methods have focused on scalar imaging modalities such as T1/T2 MRI and CT, with less attention given to more complex modalities such as diffusion MRI. In this paper, to the best of our knowledge, we present the first end-to-end geometric deep learning based model for the non-rigid registration of fiber orientation distribution fields (fODF) derived from diffusion MRI (dMRI). Our method can be trained in a fully-unsupervised fashion using only input fODF image pairs, i.e. without ground truth deformation fields. Our model introduces several novel differentiable layers for local Jacobian estimation and reorientation that can be seamlessly integrated into the recently introduced manifold-valued convolutional network in literature. The results of this work are accurate deformable registration algorithms for dMRI data that can execute in the order of seconds, as opposed to dozens of minutes to hours consumed by their classical counterparts.

Construction of a neuroanatomical shape complex atlas from 3D MRI brain structures.

Brain atlas construction has attracted significant attention lately in the neuroimaging community due to its application to the characterization of neuroanatomical shape abnormalities associated with various neurodegenerative diseases or neuropsychiatric disorders. Existing shape atlas construction techniques usually focus on the analysis of a single anatomical structure in which the important inter-structural information is lost. This paper proposes a novel technique for constructing a neuroanatomical shape complex atlas based on an information geometry framework. A shape complex is a collection of neighboring shapes - for example, the thalamus, amygdala and the hippocampus circuit - which may exhibit changes in shape across multiple structures during the progression of a disease. In this paper, we represent the boundaries of the entire shape complex using the zero level set of a distance transform function S(x). We then re-derive the relationship between the stationary state wave function ψ(x) of the Schrödinger equation [formula in text] and the eikonal equation [formula in text] satisfied by any distance function. This leads to a one-to-one map (up to scale) between ψ(x) and S(x) via an explicit relationship. We further exploit this relationship by mapping ψ(x) to a unit hypersphere whose Riemannian structure is fully known, thus effectively turn ψ(x) into the square-root of a probability density function. This allows us to make comparisons - using elegant, closed-form analytic expressions - between shape complexes represented as square-root densities. A shape complex atlas is constructed by computing the Karcher mean ψ¯(x) in the space of square-root densities and then inversely mapping it back to the space of distance transforms in order to realize the atlas shape. We demonstrate the shape complex atlas computation technique via a set of experiments on a population of brain MRI scans including controls and epilepsy patients with either right anterior medial temporal or left anterior medial temporal lobectomies.

Nested Grassmannians for Dimensionality Reduction with Applications.

In the recent past, nested structures in Riemannian manifolds has been studied in the context of dimensionality reduction as an alternative to the popular principal geodesic analysis (PGA) technique, for example, the principal nested spheres. In this paper, we propose a novel framework for constructing a nested sequence of homogeneous Riemannian manifolds. Common examples of homogeneous Riemannian manifolds include the n-sphere, the Stiefel manifold, the Grassmann manifold and many others. In particular, we focus on applying the proposed framework to the Grassmann manifold, giving rise to the nested Grassmannians (NG). An important application in which Grassmann manifolds are encountered is planar shape analysis. Specifically, each planar (2D) shape can be represented as a point in the complex projective space which is a complex Grassmann manifold. Some salient features of our framework are: (i) it explicitly exploits the geometry of the homogeneous Riemannian manifolds and (ii) the nested lower-dimensional submanifolds need not be geodesic. With the proposed NG structure, we develop algorithms for the supervised and unsupervised dimensionality reduction problems respectively. The proposed algorithms are compared with PGA via simulation studies and real data experiments and are shown to achieve a higher ratio of expressed variance compared to PGA.

Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Design.

Hyperbolic neural networks have been popular in the recent past due to their ability to represent hierarchical data sets effectively and efficiently. The challenge in developing these networks lies in the nonlinearity of the embedding space namely, the Hyperbolic space. Hyperbolic space is a homogeneous Riemannian manifold of the Lorentz group which is a semi-Riemannian manifold, i.e. a manifold equipped with an indefinite metric. Most existing methods (with some exceptions) use local linearization to define a variety of operations paralleling those used in traditional deep neural networks in Euclidean spaces. In this paper, we present a novel fully hyperbolic neural network which uses the concept of projections (embeddings) followed by an intrinsic aggregation and a nonlinearity all within the hyperbolic space. The novelty here lies in the projection which is designed to project data on to a lower-dimensional embedded hyperbolic space and hence leads to a nested hyperbolic space representation independently useful for dimensionality reduction. The main theoretical contribution is that the proposed embedding is proved to be isometric and equivariant under the Lorentz transformations, which are the natural isometric transformations in hyperbolic spaces. This projection is computationally efficient since it can be expressed by simple linear operations, and, due to the aforementioned equivariance property, it allows for weight sharing. The nested hyperbolic space representation is the core component of our network and therefore, we first compare this representation - independent of the network - with other dimensionality reduction methods such as tangent PCA, principal geodesic analysis (PGA) and HoroPCA. Based on this equivariant embedding, we develop a novel fully hyperbolic graph convolutional neural network architecture to learn the parameters of the projection. Finally, we present experiments demonstrating comparative performance of our network on several publicly available data sets.

VolterraNet: A Higher Order Convolutional Network With Group Equivariance for Homogeneous Manifolds.

Convolutional neural networks have been highly successful in image-based learning tasks due to their translation equivariance property. Recent work has generalized the traditional convolutional layer of a convolutional neural network to non-euclidean spaces and shown group equivariance of the generalized convolution operation. In this paper, we present a novel higher order Volterra convolutional neural network (VolterraNet) for data defined as samples of functions on Riemannian homogeneous spaces. Analagous to the result for traditional convolutions, we prove that the Volterra functional convolutions are equivariant to the action of the isometry group admitted by the Riemannian homogeneous spaces, and under some restrictions, any non-linear equivariant function can be expressed as our homogeneous space Volterra convolution, generalizing the non-linear shift equivariant characterization of Volterra expansions in euclidean space. We also prove that second order functional convolution operations can be represented as cascaded convolutions which leads to an efficient implementation. Beyond this, we also propose a dilated VolterraNet model. These advances lead to large parameter reductions relative to baseline non-euclidean CNNs. To demonstrate the efficacy of the VolterraNet performance, we present several real data experiments involving classification tasks on spherical-MNIST, atomic energy, Shrec17 data sets, and group testing on diffusion MRI data. Performance comparisons to the state-of-the-art are also presented.

ManifoldNet: A Deep Neural Network for Manifold-Valued Data With Applications.

Geometric deep learning is a relatively nascent field that has attracted significant attention in the past few years. This is partly due to the availability of data acquired from non-euclidean domains or features extracted from euclidean-space data that reside on smooth manifolds. For instance, pose data commonly encountered in computer vision reside in Lie groups, while covariance matrices that are ubiquitous in many fields and diffusion tensors encountered in medical imaging domain reside on the manifold of symmetric positive definite matrices. Much of this data is naturally represented as a grid of manifold-valued data. In this paper we present a novel theoretical framework for developing deep neural networks to cope with these grids of manifold-valued data inputs. We also present a novel architecture to realize this theory and call it the ManifoldNet. Analogous to vector spaces where convolutions are equivalent to computing weighted sums, manifold-valued data 'convolutions' can be defined using the weighted Fréchet Mean ([Formula: see text]). (This requires endowing the manifold with a Riemannian structure if it did not already come with one.) The hidden layers of ManifoldNet compute [Formula: see text]s of their inputs, where the weights are to be learnt. This means the data remain manifold-valued as they propagate through the hidden layers. To reduce computational complexity, we present a provably convergent recursive algorithm for computing the [Formula: see text]. Further, we prove that on non-constant sectional curvature manifolds, each [Formula: see text] layer is a contraction mapping and provide constructive evidence for its non-collapsibility when stacked in layers. This captures the two fundamental properties of deep network layers. Analogous to the equivariance of convolution in euclidean space to translations, we prove that the [Formula: see text] is equivariant to the action of the group of isometries admitted by the Riemannian manifold on which the data reside. To showcase the performance of ManifoldNet, we present several experiments using both computer vision and medical imaging data sets.

Intrinsic Grassmann Averages for Online Linear, Robust and Nonlinear Subspace Learning.

Principal component analysis (PCA) and Kernel principal component analysis (KPCA) are fundamental methods in machine learning for dimensionality reduction. The former is a technique for finding this approximation in finite dimensions and the latter is often in an infinite dimensional reproducing Kernel Hilbert-space (RKHS). In this paper, we present a geometric framework for computing the principal linear subspaces in both (finite and infinite) situations as well as for the robust PCA case, that amounts to computing the intrinsic average on the space of all subspaces: the Grassmann manifold. Points on this manifold are defined as the subspaces spanned by K-tuples of observations. The intrinsic Grassmann average of these subspaces are shown to coincide with the principal components of the observations when they are drawn from a Gaussian distribution. We show similar results in the RKHS case and provide an efficient algorithm for computing the projection onto the this average subspace. The result is a method akin to KPCA which is substantially faster. Further, we present a novel online version of the KPCA using our geometric framework. Competitive performance of all our algorithms are demonstrated on a variety of real and synthetic data sets.

Group-wise Point-set registration using a novel CDF-based Havrda-Charvát Divergence.

This paper presents a novel and robust technique for group-wise registration of point sets with unknown correspondence. We begin by defining a Havrda-Charvát (HC) entropy valid for cumulative distribution functions (CDFs) which we dub the HC Cumulative Residual Entropy (HC-CRE). Based on this definition, we propose a new measure called the CDF-HC divergence which is used to quantify the dis-similarity between CDFs estimated from each point-set in the given population of point sets. This CDF-HC divergence generalizes the CDF based Jensen-Shannon (CDF-JS) divergence introduced earlier in the literature, but is much simpler in implementation and computationally more efficient.A closed-form formula for the analytic gradient of the cost function with respect to the non-rigid registration parameters has been derived, which is conducive for efficient quasi-Newton optimization. Our CDF-HC algorithm is especially useful for unbiased point-set atlas construction and can do so without the need to establish correspondences. Mathematical analysis and experimental results indicate that this CDF-HC registration algorithm outperforms the previous group-wise point-set registration algorithms in terms of efficiency, accuracy and robustness.

Regularized positive-definite fourth order tensor field estimation from DW-MRI.

In Diffusion Weighted Magnetic Resonance Image (DW-MRI) processing, a 2nd order tensor has been commonly used to approximate the diffusivity function at each lattice point of the DW-MRI data. From this tensor approximation, one can compute useful scalar quantities (e.g. anisotropy, mean diffusivity) which have been clinically used for monitoring encephalopathy, sclerosis, ischemia and other brain disorders. It is now well known that this 2nd-order tensor approximation fails to capture complex local tissue structures, e.g. crossing fibers, and as a result, the scalar quantities derived from these tensors are grossly inaccurate at such locations. In this paper we employ a 4th order symmetric positive-definite (SPD) tensor approximation to represent the diffusivity function and present a novel technique to estimate these tensors from the DW-MRI data guaranteeing the SPD property. Several articles have been reported in literature on higher order tensor approximations of the diffusivity function but none of them guarantee the positivity of the estimates, which is a fundamental constraint since negative values of the diffusivity are not meaningful. In this paper we represent the 4th-order tensors as ternary quartics and then apply Hilbert's theorem on ternary quartics along with the Iwasawa parametrization to guarantee an SPD 4th-order tensor approximation from the DW-MRI data. The performance of this model is depicted on synthetic data as well as real DW-MRIs from a set of excised control and injured rat spinal cords, showing accurate estimation of scalar quantities such as generalized anisotropy and trace as well as fiber orientations.

Exploiting structural redundancy in q-space for improved EAP reconstruction from highly undersampled (k, q)-space in DMRI.

Accurate reconstruction of the ensemble average propagators (EAPs) from undersampled diffusion MRI (dMRI) measurements is a well-motivated, actively researched problem in the field of dMRI acquisition and analysis. A number of approaches based on compressed sensing (CS) principles have been developed for this problem, achieving a considerable acceleration in the acquisition by leveraging sparse representations of the signal. Most recent methods in literature apply undersampling techniques in the (k, q)-space for the recovery of EAP in the joint (x, r)-space. Yet, the majority of these methods follow a pipeline of first reconstructing the diffusion images in the (x, q)-space and subsequently estimating the EAPs through a 3D Fourier transform. In this work, we present a novel approach to achieve the direct reconstruction of P(x, r) from partial (k, q)-space measurements, with geometric constraints involving the parallelism of level-sets of diffusion images from proximal q-space points. By directly reconstructing P(x, r)) from (k, q)-space data, we exploit the incoherence between the 6D sensing and reconstruction domains to the fullest, which is consistent with the CS-theory. Further, our approach aims to utilize the inherent structural similarity (parallelism) of the level-sets in the diffusion images corresponding to proximally-located q-space points in a CS framework to achieve further reduction in sample complexity that could facilitate faster acquisition in dMRI. We compare the proposed method to a state-of-the-art CS based EAP reconstruction method (from joint (k, q)-space) on simulated, phantom and real dMRI data demonstrating the benefits of exploiting the structural similarity in the q-space.

A geometric framework for ensemble average propagator reconstruction from diffusion MRI.

Diffusion-weighted magnetic resonance imaging (dMRI) is a non-invasive technique to probe the complex micro-architecture of the tissue being imaged. The diffusional properties of the tissue at the imaged resolution are well captured by the ensemble average propagator (EAP), which is a probability density function characterizing the probability of water molecule diffusion. Many properties in the form of imaging 'stains' can then be computed from the EAP that can serve as bio-markers for a variety of diseases. This motivates the development of methods for the accurate estimation of the EAPs from dMRI, which is an actively researched area in dMRI analysis. To this end, in the recent past, dictionary learning (DL) techniques have been applied by many researchers for accurate reconstruction of the EAP fields from dMRI scans of the central nervous system (CNS). However, most of the DL-based methods did not exploit the geometry of the space of the EAPs, which are probability density functions. By exploiting the geometry of the space of probability density functions, it is possible to reconstruct EAPs that satisfy the mathematical properties of a density function and hence yield better accuracy in the EAP field reconstruction. Using a square root density parameterization, the EAPs can be mapped to a unit Hilbert sphere, which is a smooth manifold with well known geometry that we will exploit in our formulation of the DL problem. Thus, in this paper, we present a general formulation of the DL problem for data residing on smooth manifolds and in particular the manifold of EAPs, along with a numerical solution using an alternating minimization method. We then showcase the properties and the performance of our algorithm on the reconstruction of the EAP field in a patch-wise manner from the dMRI data. Through several synthetic, phantom and real data examples, we demonstrate that our non-linear DL-based approach produces accurate and spatially smooth estimates of the EAP field from dMRI in comparison to the state-of-the-art EAP reconstruction method called the MAPL method, as well as the linear DL-based EAP reconstruction approaches. To further demonstrate the accuracy and utility of our approach, we compute an entropic anisotropy measure (HA), that is a function of the well known Rényi entropy, from the EAP fields of control and injured rat spinal cords respectively. We demonstrate its utility as an imaging 'stain' via a quantitative comparison of HA maps computed from EAP fields estimated using our method and competing methods. The quantitative comparison is achieved using a two sample t-test and the results of significance are displayed for a visualization of regions of the spinal cord affected most by the injury.

A unified computational framework for deconvolution to reconstruct multiple fibers from diffusion weighted MRI.

Diffusion magnetic resonance imaging (MRI) is a relatively new imaging modality which is capable of measuring the diffusion of water molecules in biological systems noninvasively. The measurements from diffusion MRI provide unique clues for extracting orientation information of brain white matter fibers and can be potentially used to infer the brain connectivity in vivo using tractography techniques. Diffusion tensor imaging (DTI), currently the most widely used technique, fails to extract multiple fiber orientations in regions with complex microstructure. In order to overcome this limitation of DTI, a variety of reconstruction algorithms have been introduced in the recent past. One of the key ingredients in several model-based approaches is deconvolution operation which is presented in a unified deconvolution framework in this paper. Additionally, some important computational issues in solving the deconvolution problem that are not addressed adequately in previous studies are described in detail here. Further, we investigate several deconvolution schemes towards achieving stable, sparse, and accurate solutions. Experimental results on both simulations and real data are presented. The comparisons empirically suggest that nonnegative least squares method is the technique of choice for the multifiber reconstruction problem in the presence of intravoxel orientational heterogeneity.

Tensor splines for interpolation and approximation of DT-MRI with applications to segmentation of isolated rat hippocampi.

In this paper, we present novel algorithms for statistically robust interpolation and approximation of diffusion tensors-which are symmetric positive definite (SPD) matrices-and use them in developing a significant extension to an existing probabilistic algorithm for scalar field segmentation, in order to segment diffusion tensor magnetic resonance imaging (DT-MRI) datasets. Using the Riemannian metric on the space of SPD matrices, we present a novel and robust higher order (cubic) continuous tensor product of B-splines algorithm to approximate the SPD diffusion tensor fields. The resulting approximations are appropriately dubbed tensor splines. Next, we segment the diffusion tensor field by jointly estimating the label (assigned to each voxel) field, which is modeled by a Gauss Markov measure field (GMMF) and the parameters of each smooth tensor spline model representing the labeled regions. Results of interpolation, approximation, and segmentation are presented for synthetic data and real diffusion tensor fields from an isolated rat hippocampus, along with validation. We also present comparisons of our algorithms with existing methods and show significantly improved results in the presence of noise as well as outliers.

Simultaneous registration and parcellation of bilateral hippocampal surface pairs for local asymmetry quantification.

In clinical applications where structural asymmetries between homologous shapes have been correlated with pathology, the questions of definition and quantification of "asymmetry" arise naturally. When not only the degree but the position of deformity is thought relevant, asymmetry localization must also be addressed. Asymmetries between paired shapes have already been formulated in terms of (nonrigid) diffeomorphisms between the shapes. For the infinity of such maps possible for a given pair, we define optimality as the minimization of deviation from isometry under the constraint of piecewise deformation homogeneity. We propose a novel variational formulation for segmenting asymmetric regions from surface pairs based on the minimization of a functional of both the deformation map and the segmentation boundary, which defines the regions within which the homogeneity constraint is to be enforced. The functional minimization is achieved via a quasi-simultaneous evolution of the map and the segmenting curve, conducted on and between two-dimensional surface parametric domains. We present examples using both synthetic data and pairs of left and right hippocampal structures and demonstrate the relevance of the extracted features through a clinical epilepsy classification analysis.

Diffusion basis functions decomposition for estimating white matter intravoxel fiber geometry.

In this paper, we present a new formulation for recovering the fiber tract geometry within a voxel from diffusion weighted magnetic resonance imaging (MRI) data, in the presence of single or multiple neuronal fibers. To this end, we define a discrete set of diffusion basis functions. The intravoxel information is recovered at voxels containing fiber crossings or bifurcations via the use of a linear combination of the above mentioned basis functions. Then, the parametric representation of the intravoxel fiber geometry is a discrete mixture of Gaussians. Our synthetic experiments depict several advantages by using this discrete schema: the approach uses a small number of diffusion weighted images (23) and relatively small b values (1250 s/mm2), i.e., the intravoxel information can be inferred at a fraction of the acquisition time required for datasets involving a large number of diffusion gradient orientations. Moreover our method is robust in the presence of more than two fibers within a voxel, improving the state-of-the-art of such parametric models. We present two algorithmic solutions to our formulation: by solving a linear program or by minimizing a quadratic cost function (both with non-negativity constraints). Such minimizations are efficiently achieved with standard iterative deterministic algorithms. Finally, we present results of applying the algorithms to synthetic as well as real data.

Non-Rigid Multi-Modal Image Registration Using Cross-Cumulative Residual Entropy.

In this paper we present a new approach for the non-rigid registration of multi-modality images. Our approach is based on an information theoretic measure called the cumulative residual entropy (CRE), which is a measure of entropy defined using cumulative distributions. Cross-CRE between two images to be registered is defined and maximized over the space of smooth and unknown non-rigid transformations. For efficient and robust computation of the non-rigid deformations, a tri-cubic B-spline based representation of the deformation function is used. The key strengths of combining CCRE with the tri-cubic B-spline representation in addressing the non-rigid registration problem are that, not only do we achieve the robustness due to the nature of the CCRE measure, we also achieve computational efficiency in estimating the non-rigid registration. The salient features of our algorithm are: (i) it accommodates images to be registered of varying contrast+brightness, (ii) faster convergence speed compared to other information theory-based measures used for non-rigid registration in literature, (iii) analytic computation of the gradient of CCRE with respect to the non-rigid registration parameters to achieve efficient and accurate registration, (iv) it is well suited for situations where the source and the target images have field of views with large non-overlapping regions. We demonstrate these strengths via experiments on synthesized and real image data.