ICON: Learning Regular Maps Through Inverse Consistency.

Learning maps between data samples is fundamental. Applications range from representation learning, image translation and generative modeling, to the estimation of spatial deformations. Such maps relate feature vectors, or map between feature spaces. Well-behaved maps should be regular, which can be imposed explicitly or may emanate from the data itself. We explore what induces regularity for spatial transformations, e.g., when computing image registrations. Classical optimization-based models compute maps between pairs of samples and rely on an appropriate regularizer for well-posedness. Recent deep learning approaches have attempted to avoid using such regularizers altogether by relying on the sample population instead. We explore if it is possible to obtain spatial regularity using an inverse consistency loss only and elucidate what explains map regularity in such a context. We find that deep networks combined with an inverse consistency loss and randomized off-grid interpolation yield well behaved, approximately diffeomorphic, spatial transformations. Despite the simplicity of this approach, our experiments present compelling evidence, on both synthetic and real data, that regular maps can be obtained without carefully tuned explicit regularizers, while achieving competitive registration performance.

Metric Learning for Image Registration.

Image registration is a key technique in medical image analysis to estimate deformations between image pairs. A good deformation model is important for high-quality estimates. However, most existing approaches use ad-hoc deformation models chosen for mathematical convenience rather than to capture observed data variation. Recent deep learning approaches learn deformation models directly from data. However, they provide limited control over the spatial regularity of transformations. Instead of learning the entire registration approach, we learn a spatially-adaptive regularizer within a registration model. This allows controlling the desired level of regularity and preserving structural properties of a registration model. For example, diffeomorphic transformations can be attained. Our approach is a radical departure from existing deep learning approaches to image registration by embedding a deep learning model in an optimization-based registration algorithm to parameterize and data-adapt the registration model itself. Source code is publicly-available at https://github.com/uncbiag/registration.

Region-specific Diffeomorphic Metric Mapping.

We introduce a region-specific diffeomorphic metric mapping (RDMM) registration approach. RDMM is non-parametric, estimating spatio-temporal velocity fields which parameterize the sought-for spatial transformation. Regularization of these velocity fields is necessary. In contrast to existing non-parametric registration approaches using a fixed spatially-invariant regularization, for example, the large displacement diffeomorphic metric mapping (LDDMM) model, our approach allows for spatially-varying regularization which is advected via the estimated spatio-temporal velocity field. Hence, not only can our model capture large displacements, it does so with a spatio-temporal regularizer that keeps track of how regions deform, which is a more natural mathematical formulation. We explore a family of RDMM registration approaches: 1) a registration model where regions with separate regularizations are pre-defined (e.g., in an atlas space or for distinct foreground and background regions), 2) a registration model where a general spatially-varying regularizer is estimated, and 3) a registration model where the spatially-varying regularizer is obtained via an end-to-end trained deep learning (DL) model. We provide a variational derivation of RDMM, showing that the model can assure diffeomorphic transformations in the continuum, and that LDDMM is a particular instance of RDMM. To evaluate RDMM performance we experiment 1) on synthetic 2D data and 2) on two 3D datasets: knee magnetic resonance images (MRIs) of the Osteoarthritis Initiative (OAI) and computed tomography images (CT) of the lung. Results show that our framework achieves comparable performance to state-of-the-art image registration approaches, while providing additional information via a learned spatio-temporal regularizer. Further, our deep learning approach allows for very fast RDMM and LDDMM estimations. Code is available at https://github.com/uncbiag/registration.

Splines for diffeomorphisms.

This paper develops a method for higher order parametric regression on diffeomorphisms for image regression. We present a principled way to define curves with nonzero acceleration and nonzero jerk. This work extends methods based on geodesics which have been developed during the last decade for computational anatomy in the large deformation diffeomorphic image analysis framework. In contrast to previously proposed methods to capture image changes over time, such as geodesic regression, the proposed method can capture more complex spatio-temporal deformations. We take a variational approach that is governed by an underlying energy formulation, which respects the nonflat geometry of diffeomorphisms. Such an approach of minimal energy curve estimation also provides a physical analogy to particle motion under a varying force field. This gives rise to the notion of the quadratic, the cubic and the piecewise cubic splines on the manifold of diffeomorphisms. The variational formulation of splines also allows for the use of temporal control points to control spline behavior. This necessitates the development of a shooting formulation for splines. The initial conditions of our proposed shooting polynomial paths in diffeomorphisms are analogous to the Euclidean polynomial coefficients. We experimentally demonstrate the effectiveness of using the parametric curves both for synthesizing polynomial paths and for regression of imaging data. The performance of the method is compared to geodesic regression.

Spatially-varying metric learning for diffeomorphic image registration: a variational framework.

This paper introduces a variational strategy to learn spatially-varying metrics on large groups of images, in the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework. Spatially-varying metrics we learn not only favor local deformations but also correlated deformations in different image regions and in different directions. In addition, metric parameters can be efficiently estimated using a gradient descent method. We first describe the general strategy and then show how to use it on 3D medical images with reasonable computational ressources. Our method is assessed on the 3D brain images of the LPBA40 dataset. Results are compared with ANTS-SyN and LDDMM with spatially-homogeneous metrics.

Longitudinal deformation models, spatial regularizations and learning strategies to quantify Alzheimer's disease progression.

In the context of Alzheimer's disease, two challenging issues are (1) the characterization of local hippocampal shape changes specific to disease progression and (2) the identification of mild-cognitive impairment patients likely to convert. In the literature, (1) is usually solved first to detect areas potentially related to the disease. These areas are then considered as an input to solve (2). As an alternative to this sequential strategy, we investigate the use of a classification model using logistic regression to address both issues (1) and (2) simultaneously. The classification of the patients therefore does not require any a priori definition of the most representative hippocampal areas potentially related to the disease, as they are automatically detected. We first quantify deformations of patients' hippocampi between two time points using the large deformations by diffeomorphisms framework and transport these deformations to a common template. Since the deformations are expected to be spatially structured, we perform classification combining logistic loss and spatial regularization techniques, which have not been explored so far in this context, as far as we know. The main contribution of this paper is the comparison of regularization techniques enforcing the coefficient maps to be spatially smooth (Sobolev), piecewise constant (total variation) or sparse (fused LASSO) with standard regularization techniques which do not take into account the spatial structure (LASSO, ridge and ElasticNet). On a dataset of 103 patients out of ADNI, the techniques using spatial regularizations lead to the best classification rates. They also find coherent areas related to the disease progression.

Piecewise-diffeomorphic image registration: application to the motion estimation between 3D CT lung images with sliding conditions.

In this paper, we propose a new strategy for modelling sliding conditions when registering 3D images in a piecewise-diffeomorphic framework. More specifically, our main contribution is the development of a mathematical formalism to perform Large Deformation Diffeomorphic Metric Mapping registration with sliding conditions. We also show how to adapt this formalism to the LogDemons diffeomorphic registration framework. We finally show how to apply this strategy to estimate the respiratory motion between 3D CT pulmonary images. Quantitative tests are performed on 2D and 3D synthetic images, as well as on real 3D lung images from the MICCAI EMPIRE10 challenge. Results show that our strategy estimates accurate mappings of entire 3D thoracic image volumes that exhibit a sliding motion, as opposed to conventional registration methods which are not capable of capturing discontinuous deformations at the thoracic cage boundary. They also show that although the deformations are not smooth across the location of sliding conditions, they are almost always invertible in the whole image domain. This would be helpful for radiotherapy planning and delivery.

Left-invariant metrics for diffeomorphic image registration with spatially-varying regularisation.

We present a new framework for diffeomorphic image registration which supports natural interpretations of spatially-varying metrics. This framework is based on left-invariant diffeomorphic metrics (LIDM) and is closely related to the now standard large deformation diffeomorphic metric mapping (LDDMM). We discuss the relationship between LIDM and LDDMM and introduce a computationally convenient class of spatially-varying metrics appropriate for both frameworks. Finally, we demonstrate the effectiveness of our method on a 2D toy example and on the 40 3D brain images of the LPBA40 dataset.

Geodesic regression for image time-series.

Registration of image-time series has so far been accomplished (i) by concatenating registrations between image pairs, (ii) by solving a joint estimation problem resulting in piecewise geodesic paths between image pairs, (iii) by kernel based local averaging or (iv) by augmenting the joint estimation with additional temporal irregularity penalties. Here, we propose a generative model extending least squares linear regression to the space of images by using a second-order dynamic formulation for image registration. Unlike previous approaches, the formulation allows for a compact representation of an approximation to the full spatio-temporal trajectory through its initial values. The method also opens up possibilities to design image-based approximation algorithms. The resulting optimization problem is solved using an adjoint method.

Simultaneous multi-scale registration using large deformation diffeomorphic metric mapping.

In the framework of large deformation diffeomorphic metric mapping (LDDMM), we present a practical methodology to integrate prior knowledge about the registered shapes in the regularizing metric. Our goal is to perform rich anatomical shape comparisons from volumetric images with the mathematical properties offered by the LDDMM framework. We first present the notion of characteristic scale at which image features are deformed. We then propose a methodology to compare anatomical shape variations in a multi-scale fashion, i.e., at several characteristic scales simultaneously. In this context, we propose a strategy to quantitatively measure the feature differences observed at each characteristic scale separately. After describing our methodology, we illustrate the performance of the method on phantom data. We then compare the ability of our method to segregate a group of subjects having Alzheimer's disease and a group of controls with a classical coarse to fine approach, on standard 3D MR longitudinal brain images. We finally apply the approach to quantify the anatomical development of the human brain from 3D MR longitudinal images of pre-term babies. Results show that our method registers accurately volumetric images containing feature differences at several scales simultaneously with smooth deformations.

Simultaneous fine and coarse diffeomorphic registration: application to atrophy measurement in Alzheimer's disease.

In this paper, we present a fine and coarse approach for the multiscale registration of 3D medical images using Large Deformation Diffeomorphic Metric Mapping (LDDMM). This approach has particularly interesting properties since it estimates large, smooth and invertible optimal deformations having a rich descriptive power for the quantification of temporal changes in the images. First, we show the importance of the smoothing kernel and its influence on the final solution. We then propose a new strategy for the spatial regularization of the deformations, which uses simultaneously fine and coarse smoothing kernels. We have evaluated the approach on both 2D synthetic images as well as on 3D MR longitudinal images out of the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. Results highlight the regularizing properties of our approach for the registration of complex shapes. More importantly, the results also demonstrate its ability to measure shape variations at several scales simultaneously while keeping the desirable properties of LDDMM. This opens new perspectives for clinical applications.