Double Transformer Super-Resolution for Breast Cancer ADC Images.

Diffusion-weighted imaging (DWI) has been extensively explored in guiding the clinic management of patients with breast cancer. However, due to the limited resolution, accurately characterizing tumors using DWI and the corresponding apparent diffusion coefficient (ADC) is still a challenging problem. In this paper, we aim to address the issue of super-resolution (SR) of ADC images and evaluate the clinical utility of SR-ADC images through radiomics analysis. To this end, we propose a novel double transformer-based network (DTformer) to enhance the resolution of ADC images. More specifically, we propose a symmetric U-shaped encoder-decoder network with two different types of transformer blocks, named as UTNet, to extract deep features for super-resolution. The basic backbone of UTNet is composed of a locally-enhanced Swin transformer block (LeSwin-T) and a convolutional transformer block (Conv-T), which are responsible for capturing long-range dependencies and local spatial information, respectively. Additionally, we introduce a residual upsampling network (RUpNet) to expand image resolution by leveraging initial residual information from the original low-resolution (LR) images. Extensive experiments show that DTformer achieves superior SR performance. Moreover, radiomics analysis reveals that improving the resolution of ADC images is beneficial for tumor characteristic prediction, such as histological grade and human epidermal growth factor receptor 2 (HER2) status.

Automatic landmark detection and registration of brain cortical surfaces via quasi-conformal geometry and convolutional neural networks.

In medical imaging, surface registration is extensively used for performing systematic comparisons between anatomical structures, with a prime example being the highly convoluted brain cortical surfaces. To obtain a meaningful registration, a common approach is to identify prominent features on the surfaces and establish a low-distortion mapping between them with the feature correspondence encoded as landmark constraints. Prior registration works have primarily focused on using manually labeled landmarks and solving highly nonlinear optimization problems, which are time-consuming and hence hinder practical applications. In this work, we propose a novel framework for the automatic landmark detection and registration of brain cortical surfaces using quasi-conformal geometry and convolutional neural networks. We first develop a landmark detection network (LD-Net) that allows for the automatic extraction of landmark curves given two prescribed starting and ending points based on the surface geometry. We then utilize the detected landmarks and quasi-conformal theory for achieving the surface registration. Specifically, we develop a coefficient prediction network (CP-Net) for predicting the Beltrami coefficients associated with the desired landmark-based registration and a mapping network called the disk Beltrami solver network (DBS-Net) for generating quasi-conformal mappings from the predicted Beltrami coefficients, with the bijectivity guaranteed by quasi-conformal theory. Experimental results are presented to demonstrate the effectiveness of our proposed framework. Altogether, our work paves a new way for surface-based morphometry and medical shape analysis.

Shape analysis via inconsistent surface registration.

In this work, we develop a framework for shape analysis using inconsistent surface mapping. Traditional landmark-based geometric morphometr- ics methods suffer from the limited degrees of freedom, while most of the more advanced non-rigid surface mapping methods rely on a strong assumption of the global consistency of two surfaces. From a practical point of view, given two anatomical surfaces with prominent feature landmarks, it is more desirable to have a method that automatically detects the most relevant parts of the two surfaces and finds the optimal landmark-matching alignment between these parts, without assuming any global 1-1 correspondence between the two surfaces. Our method is capable of solving this problem using inconsistent surface registration based on quasi-conformal theory. It further enables us to quantify the dissimilarity of two shapes using quasi-conformal distortion and differences in mean and Gaussian curvatures, thereby providing a natural way for shape classification. Experiments on Platyrrhine molars demonstrate the effectiveness of our method and shed light on the interplay between function and shape in nature.

Image Retargeting via Beltrami Representation.

Image retargeting aims to resize an image to one with a prescribed aspect ratio. Simple scaling inevitably introduces unnatural geometric distortions on the important content of the image. In this paper, we propose a simple and yet effective method to resize an image, which preserves the geometry of the important content, using the Beltrami representation. Our algorithm allows users to interactively label content regions as well as line structures. Image resizing can then be achieved by warping the image by an orientation-preserving bijective warping map with controlled distortion. The warping map is represented by its Beltrami representation, which captures the local geometric distortion of the map. By carefully prescribing the values of the Beltrami representation, images with different complexity can be effectively resized. Our method does not require solving any optimization problems and tuning parameters throughout the process. This results in a simple and efficient algorithm to solve the image retargeting problem. Extensive experiments have been carried out, which demonstrate the efficacy of our proposed method.

Conformal mapping of carotid vessel wall and plaque thickness measured from 3D ultrasound images.

Measurements of vessel-wall-plus-plaque thickness (VWT) from 3D carotid ultrasound have been shown to be sensitive to the effect of pharmaceutical interventions. Since the geometry of carotid arteries is highly subject-specific, quantitative comparison of the distributions of point-wise VWT measured for different patients or for the same patients at different ultrasound scanning sessions requires the development of a mapping strategy to adjust for the geometric variability of different carotid surface models. In this paper, we present an algorithm mapping each 3D carotid surface to a 2D carotid template with an emphasis on preserving the local geometry of the carotid surface by minimizing local angular distortion. The previously described arc-length scaling (AL) approach was applied to generate an initial 2D VWT map. Using results established in the quasi-conformal theory, a new map was computed to compensate for the angular distortion incurred in AL mapping. As the 2D carotid template lies on an L-shaped non-convex domain, one-to-one correspondence of the mapping operation was not guaranteed. To address this issue, an iterative Beltrami differential chopping and smoothing procedure was developed to enforce bijectivity. Evaluations performed in the 20 carotid surface models showed that the reduction in average angular distortion made by the proposed algorithm was highly significant (P = 2.06 × 10-5). This study is the first study showing that a bijective conformal map to a non-convex domain can be obtained using the iterative Beltrami differential chopping and smoothing procedure. The improved consistency exhibited in the 2D VWT map generated by the proposed algorithm will allow for unbiased quantitative comparisons of VWT as well as local geometric and hemodynamic quantities in population studies.

Landmark constrained registration of high-genus surfaces applied to vestibular system morphometry.

The analysis of the vestibular system (VS) is an important research topic in medical image analysis. VS is a sensory structure in the inner ear for the perception of spatial orientation. It is believed several diseases, such as the Adolescent Idiopathic Scoliosis (AIS), are due to the impairment of the VS function. The morphology of the VS is thus of great research significance. A major challenge is that the VS is a genus-3 surface. The high-genus topology of the VS poses great challenges to find accurate pointwise correspondences between the surfaces and whereby perform accurate shape analysis. In this paper, we present a method to obtain the landmark constrained diffeomorphic registration between the VS surfaces based on the quasi-conformal theory. Given a set of corresponding landmarks on the VS surfaces, a diffeomorphism between the VS surfaces that matches the features consistently can be obtained. The basic idea is to iteratively search for an admissible Beltrami coefficient, which is associated to our desired landmark matching registration. With the obtained surface registrations, vertex-wise morphometric analysis can be carried out. Two types of geometric features are used for shape comparison. One is the collection of homotopic loops on each canals of the VS, which can be used to measure the local thickness of the canals. From the homotopic loops, centerlines can be extracted. By examining the deviations of the centerlines from the best fit planes, bendings of the canals can be detected. The second geometric feature is the minimal surface enclosed by the homotopic loop. From the minimal surfaces of each homotopic loops, cross-sectional area of the canals can be evaluated. To study the local shape difference more comprehensively, a complete shape index, which is defined using the Beltrami coefficients and surface curvatures, is used. We test proposed registration method on 15 VS of normal control subjects and 12 VS of patients suffering from AIS. Experimental results show the efficacy and accuracy of the proposed algorithm to compute the VS surface registration. Shape analysis has also been carried out using the proposed geometric features and shape index, which reveals shape differences in the posterior canal between normal and diseased AIS groups.

Landmark constrained genus-one surface Teichmüller map applied to surface registration in medical imaging.

We address the registration problem of genus-one surfaces (such as vertebrae bones) with prescribed landmark constraints. The high-genus topology of the surfaces makes it challenging to obtain a unique and bijective surface mapping that matches landmarks consistently. This work proposes to tackle this registration problem using a special class of quasi-conformal maps called Teichmüller maps (T-Maps). A landmark constrained T-Map is the unique mapping between genus-1 surfaces that minimizes the maximal conformality distortion while matching the prescribed feature landmarks. Existence and uniqueness of the landmark constrained T-Map are theoretically guaranteed. This work presents an iterative algorithm to compute the T-Map. The main idea is to represent the set of diffeomorphism using the Beltrami coefficients (BC). The BC is iteratively adjusted to an optimal one, which corresponds to our desired T-Map that matches the prescribed landmarks and satisfies the periodic boundary condition on the universal covering space. Numerical experiments demonstrate the effectiveness of our proposed algorithm. The method has also been applied to register vertebrae bones with prescribed landmark points and curves, which gives accurate surface registrations.

Genus-one surface registration via teichmüller extremal mapping.

This paper presents a novel algorithm to obtain landmark-based genus-1 surface registration via a special class of quasi-conformal maps called the Teichmüller maps. Registering shapes with important features is an important process in medical imaging. However, it is challenging to obtain a unique and bijective genus-1surface matching that satisfies the prescribed landmark constraints. In addition, as suggested by [11], conformal transformation provides the most natural way to describe the deformation or growth of anatomical structures. This motivates us to look for the unique mapping between genus-1 surfaces that matches the features while minimizing the maximal conformality distortion. Existence and uniqueness of such optimal diffeomorphism is theoretically guaranteed and is called the Teichmüller extremal mapping. In this work, we propose an iterative algorithm, called the Quasi-conformal (QC) iteration, to find the Teichmüller extremal mapping between the covering spaces of genus-1 surfaces. By representing the set of diffeomorphisms using Beltrami coefficients (BCs), we look for an optimal BC which corresponds to our desired diffeomorphism that matches prescribed features and satisfies the periodic boundary condition on the covering space. Numerical experiments show that our proposed algorithm is efficient and stable for registering genus-1 surfaces even with large amount of landmarks. We have also applied the algorithm on registering vertebral bones with prescribed feature curves, which demonstrates the usefulness of the proposed algorithm.

Registration of brainstem surfaces in adolescent idiopathic scoliosis using discrete Ricci flow.

Adolescent Idiopathic Scoliosis (AIS) characterized by the 3D spine deformity affects about 4% schoolchildren worldwide. Several studies have demonstrated the malfunctioning of postural balance, proprioception, and equilibrium control in patients with AIS. Since these functions are closely related to structures in and around the brainstem, the morphometry of the brainstem surface is of utmost importance. In this paper, we propose an effective method to accurately compute the registration between brainstem surfaces. Four consistent features, which describe the global geometry of the brainstem, are automatically extracted to guide the surface registration. Using the discrete Ricci flow method, brainstem surfaces are parameterized conformally onto the quadrilaterally-faced hexahedron, through which the brainstem registration can be obtained. Our registration algorithm can guarantee the exact landmark correspondence between brainstem surfaces. With the obtained registration, a shape energy can be defined to measure the local shape difference between different brainstem surfaces. We have tested our algorithms on 30 real brainstem surfaces extracted from MRIs of 15 normal subjects and 15 AIS patients. Experimental results show the efficacy of the proposed algorithm to register brainstem surfaces, which matches landmark features consistently. The computed registration can be used for the morphometry of brainstems.

Euclidean geodesic loops on high-genus surfaces applied to the morphometry of vestibular systems.

This paper proposes a novel algorithm to extract feature landmarks on the vestibular system (VS), for the analysis of Adolescent Idiopathic Scoliosis (AIS) disease. AIS is a 3-D spinal deformity commonly occurred in adolescent girls with unclear etiology. One popular hypothesis was suggested to be the structural changes in the VS that induce the disturbed balance perception, and further cause the spinal deformity. The morphometry of VS to study the geometric differences between the healthy and AIS groups is of utmost importance. However, the VS is a genus-3 structure situated in the inner ear. The high-genus topology of the surface poses great challenge for shape analysis. In this work, we present a new method to compute exact geodesic loops on the VS. The resultant geodesic loops are in Euclidean metric, thus characterizing the intrinsic geometric properties of the VS based on the real background geometry. This leads to more accurate results than existing methods, such as the hyperbolic Ricci flow method. Furthermore, our method is fully automatic and highly efficient, e.g., one order of magnitude faster than. We applied our algorithm to the VS of normal and AIS subjects. The promising experimental results demonstrate the efficacy of our method and reveal more statistically significant shape difference in the VS between right-thoracic AIS and normal subjects.

Shape-based diffeomorphic registration on hippocampal surfaces using Beltrami holomorphic flow.

We develop a new algorithm to automatically register hippocampal (HP) surfaces with complete geometric matching, avoiding the need to manually label landmark features. A good registration depends on a reasonable choice of shape energy that measures the dissimilarity between surfaces. In our work, we first propose a complete shape index using the Beltrami coefficient and curvatures, which measures subtle local differences. The proposed shape energy is zero if and only if two shapes are identical up to a rigid motion. We then seek the best surface registration by minimizing the shape energy. We propose a simple representation of surface diffeomorphisms using Beltrami coefficients, which simplifies the optimization process. We then iteratively minimize the shape energy using the proposed Beltrami Holomorphic flow (BHF) method. Experimental results on 212 HP of normal and diseased (Alzheimer's disease) subjects show our proposed algorithm is effective in registering HP surfaces with complete geometric matching. The proposed shape energy can also capture local shape differences between HP for disease analysis.

Shape analysis of vestibular systems in adolescent idiopathic scoliosis using geodesic spectra.

Adolescent Idiopathic Scoliosis (AIS) characterized by the 3D spine deformity affects about 4% schoolchildren worldwide. One of the prominent theories of the etiopathogenesis of AIS was proposed to be the poor postural balance control due to the impaired vestibular function. Thus, the morphometry of the vestibular system (VS) is of great importance for studying AIS. The VS is a genus-3 structure situated in the inner ear and consists of three semicircular canals lying perpendicular to each other. The high-genus topology of the surface poses great challenge for shape analysis. In this work, we propose an effective method to analyze shapes of high-genus surfaces by considering their geodesic spectra. The key is to compute the canonical hyperbolic geodesic loops of the surface, using the Ricci flow method. The Fuchsian group generators are then computed which can be used to determine the geodesic spectra. The geodesic spectra effectively measure shape differences between high-genus surfaces up to the hyperbolic isometry. We applied the proposed algorithm to the VS of 12 normal and 15 AIS subjects. Experimental results show the effectiveness of our algorithm and reveal statistical shape difference in the VS between right-thoracic AIS and normal subjects.

Optimized conformal parameterization of cortical surfaces using shape based matching of landmark curves.

In this work, we find meaningful parameterizations of cortical surfaces utilizing prior anatomical information in the form of anatomical landmarks (sulci curves) on the surfaces. Specifically we generate close to conformal parametrizations that also give a shape-based correspondence between the landmark curves. We propose a variational energy that measures the harmonic energy of the parameterization maps, and the shape dissimilarity between mapped points on the landmark curves. The novelty is that the computed maps are guaranteed to give a shape-based diffeomorphism between the landmark curves. We achieve this by intrinsically modelling our search space of maps as flows of smooth vector fields that do not flow across the landmark curves, and by using the local surface geometry on the curves to define a shape measure. Such parameterizations ensure consistent correspondence between anatomical features, ensuring correct averaging and comparison of data across subjects. The utility of our model is demonstrated in experiments on cortical surfaces with landmarks delineated, which show that our computed maps give a shape-based alignment of the sulcal curves without significantly impairing conformality.

Brain surface conformal parameterization using Riemann surface structure.

In medical imaging, parameterized 3-D surface models are useful for anatomical modeling and visualization, statistical comparisons of anatomy, and surface-based registration and signal processing. Here we introduce a parameterization method based on Riemann surface structure, which uses a special curvilinear net structure (conformal net) to partition the surface into a set of patches that can each be conformally mapped to a parallelogram. The resulting surface subdivision and the parameterizations of the components are intrinsic and stable (their solutions tend to be smooth functions and the boundary conditions of the Dirichlet problem can be enforced). Conformal parameterization also helps transform partial differential equations (PDEs) that may be defined on 3-D brain surface manifolds to modified PDEs on a two-dimensional parameter domain. Since the Jacobian matrix of a conformal parameterization is diagonal, the modified PDE on the parameter domain is readily solved. To illustrate our techniques, we computed parameterizations for several types of anatomical surfaces in 3-D magnetic resonance imaging scans of the brain, including the cerebral cortex, hippocampi, and lateral ventricles. For surfaces that are topologically homeomorphic to each other and have similar geometrical structures, we show that the parameterization results are consistent and the subdivided surfaces can be matched to each other. Finally, we present an automatic sulcal landmark location algorithm by solving PDEs on cortical surfaces. The landmark detection results are used as constraints for building conformal maps between surfaces that also match explicitly defined landmarks.

A landmark-based brain conformal parametrization with automatic landmark tracking technique.

In this paper, we present algorithms to automatically detect and match landmark curves on cortical surfaces to get an optimized brain conformal parametrization. First, we propose an automatic landmark curve tracing method based on the principal directions of the local Weingarten matrix. Our algorithm obtains a hypothesized landmark curves using the Chan-Vese segmentation method, which solves a Partial Differential Equation (PDE) on a manifold with global conformal parameterization. Based on the global conformal parametrization of a cortical surface, our method adjusts the landmark curves iteratively on the spherical or rectangular parameter domain of the cortical surface along its principal direction field, using umbilic points of the surface as anchors. The landmark curves can then be mapped back onto the cortical surface. Experimental results show that the landmark curves detected by our algorithm closely resemble these manually labeled curves. Next, we applied these automatically labeled landmark curves to generate an optimized conformal parametrization of the cortical surface, in the sense that homologous features across subjects are caused to lie at the same parameter locations in a conformal grid. Experimental results show that our method can effectively help in automatically matching cortical surfaces across subjects.

Optimization of brain conformal mapping with landmarks.

To compare and integrate brain data, data from multiple subjects are typically mapped into a canonical space. One method to do this is to conformally map cortical surfaces to the sphere. It is well known that any genus zero Riemann surface can be conformally mapped to a sphere. Therefore, conformal mapping offers a convenient method to parameterize cortical surfaces without angular distortion, generating an orthogonal grid on the cortex that locally preserves the metric. To compare cortical surfaces more effectively, it is advantageous to adjust the conformal parameterizations to match consistent anatomical features across subjects. This matching of cortical patterns improves the alignment of data across subjects, although it is more challenging to create a consistent conformal (orthogonal) parameterization of anatomy across subjects when landmarks are constrained to lie at specific locations in the spherical parameter space. Here we propose a new method, based on a new energy functional, to optimize the conformal parameterization of cortical surfaces by using landmarks. Experimental results on a dataset of 40 brain hemispheres showed that the landmark mismatch energy can be greatly reduced while effectively preserving conformality. The key advantage of this conformal parameterization approach is that any local adjustments of the mapping to match landmarks do not affect the conformality of the mapping significantly. We also examined how the parameterization changes with different weighting factors. As expected, the landmark matching error can be reduced if it is more heavily penalized, but conformality is progressively reduced.