Bundle geodesic convolutional neural network for diffusion-weighted imaging segmentation.

Purpose: Applying machine learning techniques to magnetic resonance diffusion-weighted imaging (DWI) data is challenging due to the size of individual data samples and the lack of labeled data. It is possible, though, to learn general patterns from a very limited amount of training data if we take advantage of the geometry of the DWI data. Therefore, we present a tissue classifier based on a Riemannian deep learning framework for single-shell DWI data. Approach: The framework consists of three layers: a lifting layer that locally represents and convolves data on tangent spaces to produce a family of functions defined on the rotation groups of the tangent spaces, i.e., a (not necessarily continuous) function on a bundle of rotational functions on the manifold; a group convolution layer that convolves this function with rotation kernels to produce a family of local functions over each of the rotation groups; a projection layer using maximization to collapse this local data to form manifold based functions. Results: Experiments show that our method achieves the performance of the same level as state-of-the-art while using way fewer parameters in the model ( < 10 % ). Meanwhile, we conducted a model sensitivity analysis for our method. We ran experiments using a proportion (69.2%, 53.3%, and 29.4%) of the original training set and analyzed how much data the model needs for the task. Results show that this does reduce the overall classification accuracy mildly, but it also boosts the accuracy for minority classes. Conclusions: This work extended convolutional neural networks to Riemannian manifolds, and it shows the potential in understanding structural patterns in the brain, as well as in aiding manual data annotation.

Dental artifacts in the head and neck region: implications for Dixon-based attenuation correction in PET/MR.

BACKGROUND: In the absence of CT or traditional transmission sources in combined clinical positron emission tomography/magnetic resonance (PET/MR) systems, MR images are used for MR-based attenuation correction (MR-AC). The susceptibility effects due to metal implants challenge MR-AC in the neck region of patients with dental implants. The purpose of this study was to assess the frequency and magnitude of subsequent PET image distortions following MR-AC. METHODS: A total of 148 PET/MR patients with clear visual signal voids on the attenuation map in the dental region were included in this study. Patients were injected with [(18)F]-FDG, [(11)C]-PiB, [(18)F]-FET, or [(64)Cu]-DOTATATE. The PET/MR data were acquired over a single-bed position of 25.8 cm covering the head and neck. MR-AC was based on either standard MR-ACDIXON or MR-ACINPAINTED where the susceptibility-induced signal voids were substituted with soft tissue information. Our inpainting algorithm delineates the outer contour of signal voids breaching the anatomical volume using the non-attenuation-corrected PET image and classifies the inner air regions based on an aligned template of likely dental artifact areas. The reconstructed PET images were evaluated visually and quantitatively using regions of interests in reference regions. The volume of the artifacts and the computed relative differences in mean and max standardized uptake value (SUV) between the two PET images are reported. RESULTS: The MR-based volume of the susceptibility-induced signal voids on the MR-AC attenuation maps was between 1.6 and 520.8 mL. The corresponding/resulting bias of the reconstructed tracer distribution was localized mainly in the area of the signal void. The mean and maximum SUVs averaged across all patients increased after inpainting by 52% (± 11%) and 28% (± 11%), respectively, in the corrected region. SUV underestimation decreased with the distance to the signal void and correlated with the volume of the susceptibility artifact on the MR-AC attenuation map. CONCLUSIONS: Metallic dental work may cause severe MR signal voids. The resulting PET/MR artifacts may exceed the actual volume of the dental fillings. The subsequent bias in PET is severe in regions in and near the signal voids and may affect the conspicuity of lesions in the mandibular region.

Automatic correction of dental artifacts in PET/MRI.

A challenge when using current magnetic resonance (MR)-based attenuation correction in positron emission tomography/MR imaging (PET/MRI) is that the MRIs can have a signal void around the dental fillings that is segmented as artificial air-regions in the attenuation map. For artifacts connected to the background, we propose an extension to an existing active contour algorithm to delineate the outer contour using the nonattenuation corrected PET image and the original attenuation map. We propose a combination of two different methods for differentiating the artifacts within the body from the anatomical air-regions by first using a template of artifact regions, and second, representing the artifact regions with a combination of active shape models and k-nearest-neighbors. The accuracy of the combined method has been evaluated using 25 [Formula: see text]-fluorodeoxyglucose PET/MR patients. Results showed that the approach was able to correct an average of [Formula: see text] of the artifact areas.

Femoral cartilage segmentation in knee MRI scans using two stage voxel classification.

Using more than one classification stage and exploiting class population imbalance allows for incorporating powerful classifiers in tasks requiring large scale training data, even if these classifiers scale badly with the number of training samples. This led us to propose a two-stage classifier for segmenting tibial cartilage in knee MRI scans combining nearest neighbor classification and support vector machines (SVMs). Here we apply it to femoral cartilage segmentation. We describe the similarities and differences between segmenting these two knee cartilages. For further speeding up batch SVM training, we propose loosening the stopping condition in the quadratic program solver before considering moving on to other approximation techniques such as online SVMs. The two-stage approach reached a higher accuracy in comparison to the one-stage state-of-the-art method. It also achieved better inter-scan segmentation reproducibility when compared to a radiologist as well as the current state-of-the-art method.

Toward a theory of statistical tree-shape analysis.

To develop statistical methods for shapes with a tree-structure, we construct a shape space framework for tree-shapes and study metrics on the shape space. This shape space has singularities which correspond to topological transitions in the represented trees. We study two closely related metrics on the shape space, TED and QED. QED is a quotient euclidean distance arising naturally from the shape space formulation, while TED is the classical tree edit distance. Using Gromov's metric geometry, we gain new insight into the geometries defined by TED and QED. We show that the new metric QED has nice geometric properties that are needed for statistical analysis: Geodesics always exist and are generically locally unique. Following this, we can also show the existence and generic local uniqueness of average trees for QED. TED, while having some algorithmic advantages, does not share these advantages. Along with the theoretical framework we provide experimental proof-of-concept results on synthetic data trees as well as small airway trees from pulmonary CT scans. This way, we illustrate that our framework has promising theoretical and qualitative properties necessary to build a theory of statistical tree-shape analysis.

Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network.

Segmentation of anatomical structures in medical images is often based on a voxel/pixel classification approach. Deep learning systems, such as convolutional neural networks (CNNs), can infer a hierarchical representation of images that fosters categorization. We propose a novel system for voxel classification integrating three 2D CNNs, which have a one-to-one association with the xy, yz and zx planes of 3D image, respectively. We applied our method to the segmentation of tibial cartilage in low field knee MRI scans and tested it on 114 unseen scans. Although our method uses only 2D features at a single scale, it performs better than a state-of-the-art method using 3D multi-scale features. In the latter approach, the features and the classifier have been carefully adapted to the problem at hand. That we were able to get better results by a deep learning architecture that autonomously learns the features from the images is the main insight of this study.

A multi-scale kernel bundle for LDDMM: towards sparse deformation description across space and scales.

The Large Deformation Diffeomorphic Metric Mapping framework constitutes a widely used and mathematically well-founded setup for registration in medical imaging. At its heart lies the notion of the regularization kernel, and the choice of kernel greatly affects the results of registrations. This paper presents an extension of the LDDMM framework allowing multiple kernels at multiple scales to be incorporated in each registration while preserving many of the mathematical properties of standard LDDMM. On a dataset of landmarks from lung CT images, we show by example the influence of the kernel size in standard LDDMM, and we demonstrate how our framework, LDDKBM, automatically incorporates the advantages of each scale to reach the same accuracy as the standard method optimally tuned with respect to scale. The framework, which is not limited to landmark data, thus removes the need for classical scale selection. Moreover, by decoupling the momentum across scales, it promises to provide better interpolation properties, to allow sparse descriptions of the total deformation, to remove the tradeoff between match quality and regularity, and to allow for momentum based statistics using scale information.

Video super-resolution using simultaneous motion and intensity calculations.

In this paper, we propose an energy-based algorithm for motion-compensated video super-resolution (VSR) targeted on upscaling of standard definition (SD) video to high-definition (HD) video. Since the motion (flow field) of the image sequence is generally unknown, we introduce a formulation for the joint estimation of a super-resolution (SR) sequence and its flow field. Via the calculus of variations, this leads to a coupled system of partial differential equations for image sequence and motion estimation. We solve a simplified form of this system and, as a by-product, we indeed provide a motion field for super-resolved sequences. To the best of our knowledge, computing super-resolved flows has not been done before. Most advanced SR methods found in literature cannot be applied to general video with arbitrary scene content and/or arbitrary optical flows, as it is possible with our simultaneous VSR method. A series of experiments shows that our method outperforms other VSR methods when dealing with general video input and that it continues to provide good results even for large scaling factors up to 8 × 8.

Distribution, size, shape, growth potential and extent of abdominal aortic calcified deposits predict mortality in postmenopausal women.

BACKGROUND: Aortic calcification is a major risk factor for death from cardiovascular disease. We investigated the relationship between mortality and the composite markers of number, size, morphology and distribution of calcified plaques in the lumbar aorta. METHODS: 308 postmenopausal women aged 48-76 were followed for 8.3 ± 0.3 years, with deaths related to cardiovascular disease, cancer, or other causes being recorded. From lumbar X-rays at baseline the number (NCD), size, morphology and distribution of aortic calcification lesions were scored and combined into one Morphological Atherosclerotic Calcification Distribution (MACD) index. The hazard ratio for mortality was calculated for the MACD and for three other commonly used predictors: the EU SCORE card, the Framingham Coronary Heart Disease Risk Score (Framingham score), and the gold standard Aortic Calcification Severity score (AC24) developed from the Framingham Heart Study cohorts. RESULTS: All four scoring systems showed increasing age, smoking, and raised triglyceride levels were the main predictors of mortality after adjustment for all other metabolic and physical parameters. The SCORE card and the Framingham score resulted in a mortality hazard ratio increase per standard deviation (HR/SD) of 1.8 (1.51-2.13) and 2.6 (1.87-3.71), respectively. Of the morphological x-ray based measures, NCD revealed a HR/SD >2 adjusted for SCORE/Framingham. The MACD index scoring the distribution, size, morphology and number of lesions revealed the best predictive power for identification of patients at risk of mortality, with a hazard ratio of 15.6 (p < 0.001) for the 10% at greatest risk of death. CONCLUSIONS: This study shows that it is not just the extent of aortic calcification that predicts risk of mortality, but also the distribution, shape and size of calcified lesions. The MACD index may provide a more sensitive predictor of mortality from aortic calcification than the commonly used AC24 and SCORE/Framingham point card systems.

The improbability of harris interest points.

An elementary characterization of the map underlying Harris corners, also known as Harris interest points or key points, is provided. Two principal and basic assumptions made are: 1) Local image structure is captured in an uncommitted way, simply using weighted raw image values around every image location to describe the local image information, and 2) the lower the probability of observing the image structure present in a particular point, the more salient, or interesting, this position is, i.e., saliency is related to how uncommon it is to see a certain image structure, how surprising it is. Through the latter assumption, the axiomatization proposed makes a sound link between image saliency in computer vision on the one hand and, on the other, computational models of preattentive human visual perception, where exactly the same definition of saliency has been proposed. Because of this link, the characterization provides a compelling case in favor of Harris interest points over other approaches.

Deinterlacing using variational methods.

We present a variational framework for deinterlacing that was originally used for inpainting and subsequently redeveloped for deinterlacing. From the framework, we derive a motion adaptive (MA) deinterlacer and a motion compensated (MC) deinterlacer and test them together with a selection of known deinterlacers. To illustrate the need for MC deinterlacing, the problem of details in motion (DIM) is introduced. It cannot be solved by MA deinterlacers or any simpler deinterlacers but only by MC deinterlacers. The major problem in MC deinterlacing is computing reliable optical flow [motion estimation (ME)] in interlaced video. We discuss a number of strategies for computing optical flows on interlaced video hoping to shed some light on this problem. We produce results on challenging real world video data with our variational MC deinterlacer that in most cases are indistinguishable from the ground truth.

A family of principal component analyses for dealing with outliers.

Principal Component Analysis (PCA) has been widely used for dimensionality reduction in shape and appearance modeling. There have been several attempts of making PCA robust against outliers. However, there are cases in which a small subset of samples may appear as outliers and still correspond to plausible data. The example of shapes corresponding to fractures when building a vertebra shape model is addressed in this study. In this case, the modeling of "outliers" is important, and it might be desirable not only not to disregard them, but even to enhance their importance. A variation on PCA that deals naturally with the importance of outliers is presented in this paper. The technique is utilized for building a shape model of a vertebra, aiming at segmenting the spine out of lateral X-ray images. The results show that the algorithm can implement both an outlier-enhancing and a robust PCA. The former improves the segmentation performance in fractured vertebrae, while the latter does so in the unfractured ones.

Quantifying calcification in the lumbar aorta on X-ray images.

In this paper we propose to use inpainting to estimate the severity of atherosclerotic plaques from X-ray projections. Inpainting allows to "remove" the plaque and estimate what the background image for an uncalcified aorta would have looked like. A measure of plaque severity can then be derived by subtracting the inpainting from the original image. In contrast to the current standard of categorical calcification scoring from X-rays, our method estimates both the size and the density of calcified areas and provides a continuous severity score, thus allowing for measurement of more subtle differences. We discuss a class of smooth inpainting methods, compare their ability to reconstruct the original images, and compare the inpainting based calcification score to the conventional categorical score in a longitudinal study on 49 patients addressing correlations of the calcification scores with hypertension, a known cardiovascular risk factor.