Skeletons, Object Shape, Statistics.

Objects and object complexes in 3D, as well as those in 2D, have many possible representations. Among them skeletal representations have special advantages and some limitations. For the special form of skeletal representation called "s-reps," these advantages include strong suitability for representing slabular object populations and statistical applications on these populations. Accomplishing these statistical applications is best if one recognizes that s-reps live on a curved shape space. Here we will lay out the definition of s-reps, their advantages and limitations, their mathematical properties, methods for fitting s-reps to single- and multi-object boundaries, methods for measuring the statistics of these object and multi-object representations, and examples of such applications involving statistics. While the basic theory, ideas, and programs for the methods are described in this paper and while many applications with evaluations have been produced, there remain many interesting open opportunities for research on comparisons to other shape representations, new areas of application and further methodological developments, many of which are explicitly discussed here.

Fitting unbranching skeletal structures to objects.

Representing an object by a skeletal structure can be powerful for statistical shape analysis if there is good correspondence of the representations within a population. Many anatomic objects have a genus-zero boundary and can be represented by a smooth unbranching skeletal structure that can be discretely approximated. We describe how to compute such a discrete skeletal structure ("d-s-rep") for an individual 3D shape with the desired correspondence across cases. The method involves fitting a d-s-rep to an input representation of an object's boundary. A good fit is taken to be one whose skeletally implied boundary well approximates the target surface in terms of low order geometric boundary properties: (1) positions, (2) tangent fields, (3) various curvatures. Our method involves a two-stage framework that first, roughly yet consistently fits a skeletal structure to each object and second, refines the skeletal structure such that the shape of the implied boundary well approximates that of the object. The first stage uses a stratified diffeomorphism to produce topologically non-self-overlapping, smooth and unbranching skeletal structures for each object of a population. The second stage uses loss terms that measure geometric disagreement between the skeletally implied boundary and the target boundary and avoid self-overlaps in the boundary. By minimizing the total loss, we end up with a good d-s-rep for each individual shape. We demonstrate such d-s-reps for various human brain structures. The framework is accessible and extensible by clinical users, researchers and developers as an extension of SlicerSALT, which is based on 3D Slicer.

A Novel Method for High-Dimensional Anatomical Mapping of Extra-Axial Cerebrospinal Fluid: Application to the Infant Brain.

Cerebrospinal fluid (CSF) plays an essential role in early postnatal brain development. Extra-axial CSF (EA-CSF) volume, which is characterized by CSF in the subarachnoid space surrounding the brain, is a promising marker in the early detection of young children at risk for neurodevelopmental disorders. Previous studies have focused on global EA-CSF volume across the entire dorsal extent of the brain, and not regionally-specific EA-CSF measurements, because no tools were previously available for extracting local EA-CSF measures suitable for localized cortical surface analysis. In this paper, we propose a novel framework for the localized, cortical surface-based analysis of EA-CSF. The proposed processing framework combines probabilistic brain tissue segmentation, cortical surface reconstruction, and streamline-based local EA-CSF quantification. The quantitative analysis of local EA-CSF was applied to a dataset of typically developing infants with longitudinal MRI scans from 6 to 24 months of age. There was a high degree of consistency in the spatial patterns of local EA-CSF across age using the proposed methods. Statistical analysis of local EA-CSF revealed several novel findings: several regions of the cerebral cortex showed reductions in EA-CSF from 6 to 24 months of age, and specific regions showed higher local EA-CSF in males compared to females. These age-, sex-, and anatomically-specific patterns of local EA-CSF would not have been observed if only a global EA-CSF measure were utilized. The proposed methods are integrated into a freely available, open-source, cross-platform, user-friendly software tool, allowing neuroimaging labs to quantify local extra-axial CSF in their neuroimaging studies to investigate its role in typical and atypical brain development.

SlicerSALT: Shape AnaLysis Toolbox.

SlicerSALT is an open-source platform for disseminating state-of-the-art methods for performing statistical shape analysis. These methods are developed as 3D Slicer extensions to take advantage of its powerful underlying libraries. SlicerSALT itself is a heavily customized 3D Slicer package that is designed to be easy to use for shape analysis researchers. The packaged methods include powerful techniques for creating and visualizing shape representations as well as performing various types of analysis.

Skeletal Shape Correspondence Through Entropy.

We present a novel approach for improving the shape statistics of medical image objects by generating correspondence of skeletal points. Each object's interior is modeled by an s-rep, i.e., by a sampled, folded, two-sided skeletal sheet with spoke vectors proceeding from the skeletal sheet to the boundary. The skeleton is divided into three parts: the up side, the down side, and the fold curve. The spokes on each part are treated separately and, using spoke interpolation, are shifted along that skeleton in each training sample so as to tighten the probability distribution on those spokes' geometric properties while sampling the object interior regularly. As with the surface/boundary-based correspondence method of Cates et al., entropy is used to measure both the probability distribution tightness and the sampling regularity, here of the spokes' geometric properties. Evaluation on synthetic and real world lateral ventricle and hippocampus data sets demonstrate improvement in the performance of statistics using the resulting probability distributions. This improvement is greater than that achieved by an entropy-based correspondence method on the boundary points.

Non-Euclidean classification of medically imaged objects via s-reps.

Classifying medically imaged objects, e.g., into diseased and normal classes, has been one of the important goals in medical imaging. We propose a novel classification scheme that uses a skeletal representation to provide rich non-Euclidean geometric object properties. Our statistical method combines distance weighted discrimination (DWD) with a carefully chosen Euclideanization which takes full advantage of the geometry of the manifold on which these non-Euclidean geometric object properties (GOPs) live. Our method is evaluated via the task of classifying 3D hippocampi between schizophrenics and healthy controls. We address three central questions. 1) Does adding shape features increase discriminative power over the more standard classification based only on global volume? 2) If so, does our skeletal representation provide greater discriminative power than a conventional boundary point distribution model (PDM)? 3) Especially, is Euclideanization of non-Euclidean shape properties important in achieving high discriminative power? Measuring the capability of a method in terms of area under the receiver operator characteristic (ROC) curve, we show that our proposed method achieves strongly better classification than both the classification method based on global volume alone and the s-rep-based classification method without proper Euclideanization of non-Euclidean GOPs. We show classification using Euclideanized s-reps is also superior to classification using PDMs, whether the PDMs are first Euclideanized or not. We also show improved performance with Euclideanized boundary PDMs over non-linear boundary PDMs. This demonstrates the benefit that proper Euclideanization of non-Euclidean GOPs brings not only to s-rep-based classification but also to PDM-based classification.

Entropy-based Correspondence Improvement of Interpolated Skeletal Models.

Statistical analysis of shape representations relies on having good correspondence across a population. Improving correspondence yields improved statistics. Point distribution models (PDMs) are often used to represent object boundaries. Skeletal representations (s-reps) model object widths and boundary directions as well as boundary positions, so they should yield better correspondence. We present two methods: one for continuously interpolating a discretely-sampled skeletal model and one for improving correspondence by using this interpolation to shift skeletal samples to new positions. The interpolation operates by an extension of the mathematics of medial structures. As with Cates' boundary-based method, we evaluate correspondence in terms of regularity and shape-feature population entropies. Evaluation on both synthetic and real data shows that our method both improves correspondence of s-rep models fit to segmented lateral ventricles and that the combined boundary-and-skeletal PDMs implied by these optimized s-reps have better correspondence than optimized boundary PDMs.

Skeletal shape correspondence via entropy minimization.

PURPOSE: Improving the shape statistics of medical image objects by generating correspondence of interior skeletal points. DATA: Synthetic objects and real world lateral ventricles segmented from MR images. METHODS: Each object's interior is modeled by a skeletal representation called the s-rep, which is a quadrilaterally sampled, folded 2-sided skeletal sheet with spoke vectors proceeding from the sheet to the boundary. The skeleton is divided into three parts: up-side, down-side and fold-curve. The spokes on each part are treated separately and, using spoke interpolation, are shifted along their skeletal parts in each training sample so as to tighten the probability distribution on those spokes' geometric properties while sampling the object interior regularly. As with the surface-based correspondence method of Cates et al., entropy is used to measure both the probability distribution tightness and sampling regularity. The spokes' geometric properties are skeletal position, spoke length and spoke direction. The properties used to measure the regularity are the volumetric subregions bounded by the spokes, their quadrilateral sub-area and edge lengths on the skeletal surface and on the boundary. RESULTS: Evaluation on synthetic and real world lateral ventricles demonstrated improvement in the performance of statistics using the resulting probability distributions, as compared to methods based on boundary models. The evaluation measures used were generalization, specificity, and compactness. CONCLUSIONS: S-rep models with the proposed improved correspondence provide significantly enhanced statistics as compared to standard boundary models.

Fitting Skeletal Object Models Using Spherical Harmonics Based Template Warping.

We present a scheme that propagates a reference skeletal model (s-rep) into a particular case of an object, thereby propagating the initial shape-related layout of the skeleton-to-boundary vectors, called spokes. The scheme represents the surfaces of the template as well as the target objects by spherical harmonics and computes a warp between these via a thin plate spline. To form the propagated s-rep, it applies the warp to the spokes of the template s-rep and then statistically refines. This automatic approach promises to make s-rep fitting robust for complicated objects, which allows s-rep based statistics to be available to all. The improvement in fitting and statistics is significant compared with the previous methods and in statistics compared with a state-of-the-art boundary based method.

Multi-object analysis of volume, pose, and shape using statistical discrimination.

One goal of statistical shape analysis is the discrimination between two populations of objects. Whereas traditional shape analysis was mostly concerned with single objects, analysis of multi-object complexes presents new challenges related to alignment and pose. In this paper, we present a methodology for discriminant analysis of multiple objects represented by sampled medial manifolds. Non-euclidean metrics that describe geodesic distances between sets of sampled representations are used for alignment and discrimination. Our choice of discriminant method is the distance-weighted discriminant because of its generalization ability in high-dimensional, low sample size settings. Using an unbiased, soft discrimination score, we associate a statistical hypothesis test with the discrimination results. We explore the effectiveness of different choices of features as input to the discriminant analysis, using measures like volume, pose, shape, and the combination of pose and shape. Our method is applied to a longitudinal pediatric autism study with 10 subcortical brain structures in a population of 70 subjects. It is shown that the choices of type of global alignment and of intrinsic versus extrinsic shape features, the latter being sensitive to relative pose, are crucial factors for group discrimination and also for explaining the nature of shape change in terms of the application domain.

Shape-correlated Deformation Statistics for Respiratory Motion Prediction in 4D Lung.

4D image-guided radiation therapy (IGRT) for free-breathing lungs is challenging due to the complicated respiratory dynamics. Effective modeling of respiratory motion is crucial to account for the motion affects on the dose to tumors. We propose a shape-correlated statistical model on dense image deformations for patient-specic respiratory motion estimation in 4D lung IGRT. Using the shape deformations of the high-contrast lungs as the surrogate, the statistical model trained from the planning CTs can be used to predict the image deformation during delivery verication time, with the assumption that the respiratory motion at both times are similar for the same patient. Dense image deformation fields obtained by diffeomorphic image registrations characterize the respiratory motion within one breathing cycle. A point-based particle optimization algorithm is used to obtain the shape models of lungs with group-wise surface correspondences. Canonical correlation analysis (CCA) is adopted in training to maximize the linear correlation between the shape variations of the lungs and the corresponding dense image deformations. Both intra- and inter-session CT studies are carried out on a small group of lung cancer patients and evaluated in terms of the tumor location accuracies. The results suggest potential applications using the proposed method.

A SHAPE-NAVIGATED IMAGE DEFORMATION MODEL FOR 4D LUNG RESPIRATORY MOTION ESTIMATION.

Intensity modulated radiation therapy (IMRT) for cancers in the lung remains challenging due to the complicated respiratory dynamics. We propose a shape-navigated dense image deformation model to estimate the patient-specific breathing motion using 4D respiratory correlated CT (RCCT) images. The idea is to use the shape change of the lungs, the major motion feature in the thorax image, as a surrogate to predict the corresponding dense image deformation from training.To build the statistical model, dense diffeomorphic deformations between images of all other time points to the image at end expiration are calculated, and the shapes of the lungs are automatically extracted. By correlating the shape variation with the temporally corresponding image deformation variation, a linear mapping function that maps a shape change to its corresponding image deformation is calculated from the training sample. Finally, given an extracted shape from the image at an arbitrary time point, its dense image deformation can be predicted from the pre-computed statistics.The method is carried out on two patients and evaluated in terms of the tumor and lung estimation accuracies. The result shows robustness of the model and suggests its potential for 4D lung radiation treatment planning.

Automatic shape model building based on principal geodesic analysis bootstrapping.

We present a novel method for automatic shape model building from a collection of training shapes. The result is a shape model consisting of the mean model and the major modes of variation with a dense correspondence map between individual shapes. The framework consists of iterations where a medial shape representation is deformed into the training shapes followed by computation of the shape mean and modes of shape variation. In the first iteration, a generic shape model is used as starting point - in the following iterations in the bootstrap method, the resulting mean and modes from the previous iteration are used. Thereby, we gradually capture the shape variation in the training collection better and better. Convergence of the method is explicitly enforced. The method is evaluated on collections of artificial training shapes where the expected shape mean and modes of variation are known by design. Furthermore, collections of real prostates and cartilage sheets are used in the evaluation. The evaluation shows that the method is able to capture the training shapes close to the attainable accuracy already in the first iteration. Furthermore, the correspondence properties measured by generality, specificity, and compactness are improved during the shape model building iterations.

Automated finite-element analysis for deformable registration of prostate images.

Two major factors preventing the routine clinical use of finite-element analysis for image registration are: 1) the substantial labor required to construct a finite-element model for an individual patient's anatomy and 2) the difficulty of determining an appropriate set of finite-element boundary conditions. This paper addresses these issues by presenting algorithms that automatically generate a high quality hexahedral finite-element mesh and automatically calculate boundary conditions for an imaged patient. Medial shape models called m-reps are used to facilitate these tasks and reduce the effort required to apply finite-element analysis to image registration. Encouraging results are presented for the registration of CT image pairs which exhibit deformation caused by pressure from an endorectal imaging probe and deformation due to swelling.

Regional appearance in deformable model segmentation.

Automated medical image segmentation is a challenging task that benefits from the use of effective image appearance models. In this paper, we compare appearance models at three regional scales for statistically characterizing image intensity near object boundaries in the context of segmentation via deformable models. The three models capture appearance in the form of regional intensity quantile functions. These distribution-based regional image descriptors are amenable to Euclidean methods such as principal component analysis, which we use to build the statistical appearance models. The first model uses two regions, the interior and exterior of the organ of interest. The second model accounts for exterior inhomogeneity by clustering on object-relative local intensity quantile functions to determine tissue-consistent regions relative to the organ boundary. The third model analyzes these image descriptors per geometrically defined local region. To evaluate the three models, we present segmentation results on bladders and prostates in CT in the context of day-to-day adaptive radiotherapy for the treatment of prostate cancer. Results show improved segmentations with more local regions, probably because smaller regions better represent local inhomogeneity in the intensity distribution near the organ boundary.

Population-based fitting of medial shape models with correspondence optimization.

A crucial problem in statistical shape analysis is establishing the correspondence of shape features across a population. While many solutions are easy to express using boundary representations, this has been a considerable challenge for medial representations. This paper uses a new 3-D medial model that allows continuous interpolation of the medial manifold and provides a map back and forth between it and the boundary. A measure defined on the medial surface then allows one to write integrals over the boundary and the object interior in medial coordinates, enabling the expression of important object properties in an object-relative coordinate system. We use these integrals to optimize correspondence during model construction, reducing variability due to the model parameterization that could potentially mask true shape change effects. Discrimination and hypothesis testing of populations of shapes are expected to benefit, potentially resulting in improved significance of shape differences between populations even with a smaller sample size.

Geometrically proper models in statistical training.

In deformable model segmentation, the geometric training process plays a crucial role in providing shape statistical priors and appearance statistics that are used as likelihoods. Also, the geometric training process plays a crucial role in providing shape probability distributions in methods finding significant differences between classes. The quality of the training seriously affects the final results of segmentation or of significant difference finding between classes. However, the lack of shape priors in the training stage itself makes it difficult to enforce shape legality, i.e., making the model free of local self-intersection or creases. Shape legality not only yields proper shape statistics but also increases the consistency of parameterization of the object volume and thus proper appearance statistics. In this paper we propose a method incorporating explicit legality constraints in training process. The method is mathematically sound and has proved in practice to lead to shape probability distributions over only proper objects and most importantly to better segmentation results.

A method and software for segmentation of anatomic object ensembles by deformable m-reps.

Deformable shape models (DSMs) comprise a general approach that shows great promise for automatic image segmentation. Published studies by others and our own research results strongly suggest that segmentation of a normal or near-normal object from 3D medical images will be most successful when the DSM approach uses (1) knowledge of the geometry of not only the target anatomic object but also the ensemble of objects providing context for the target object and (2) knowledge of the image intensities to be expected relative to the geometry of the target and contextual objects. The segmentation will be most efficient when the deformation operates at multiple object-related scales and uses deformations that include not just local translations but the biologically important transformations of bending and twisting, i.e., local rotation, and local magnification. In computer vision an important class of DSM methods uses explicit geometric models in a Bayesian statistical framework to provide a priori information used in posterior optimization to match the DSM against a target image. In this approach a DSM of the object to be segmented is placed in the target image data and undergoes a series of rigid and nonrigid transformations that deform the model to closely match the target object. The deformation process is driven by optimizing an objective function that has terms for the geometric typicality and model-to-image match for each instance of the deformed model. The success of this approach depends strongly on the object representation, i.e., the structural details and parameter set for the DSM, which in turn determines the analytic form of the objective function. This paper describes a form of DSM called m-reps that has or allows these properties, and a method of segmentation consisting of large to small scale posterior optimization of m-reps. Segmentation by deformable m-reps, together with the appropriate data representations, visualizations, and user interface, has been implemented in software that accomplishes 3D segmentations in a few minutes. Software for building and training models has also been developed. The methods underlying this software and its abilities are the subject of this paper.

Comparison of human and automatic segmentations of kidneys from CT images.

PURPOSE: A controlled observer study was conducted to compare a method for automatic image segmentation with conventional user-guided segmentation of right and left kidneys from planning computerized tomographic (CT) images. METHODS AND MATERIALS: Deformable shape models called m-reps were used to automatically segment right and left kidneys from 12 target CT images, and the results were compared with careful manual segmentations performed by two human experts. M-rep models were trained based on manual segmentations from a collection of images that did not include the targets. Segmentation using m-reps began with interactive initialization to position the kidney model over the target kidney in the image data. Fully automatic segmentation proceeded through two stages at successively smaller spatial scales. At the first stage, a global similarity transformation of the kidney model was computed to position the model closer to the target kidney. The similarity transformation was followed by large-scale deformations based on principal geodesic analysis (PGA). During the second stage, the medial atoms comprising the m-rep model were deformed one by one. This procedure was iterated until no changes were observed. The transformations and deformations at both stages were driven by optimizing an objective function with two terms. One term penalized the currently deformed m-rep by an amount proportional to its deviation from the mean m-rep derived from PGA of the training segmentations. The second term computed a model-to-image match term based on the goodness of match of the trained intensity template for the currently deformed m-rep with the corresponding intensity data in the target image. Human and m-rep segmentations were compared using quantitative metrics provided in a toolset called Valmet. Metrics reported in this article include (1) percent volume overlap; (2) mean surface distance between two segmentations; and (3) maximum surface separation (Hausdorff distance). RESULTS: Averaged over all kidneys the mean surface separation was 0.12 cm, the mean Hausdorff distance was 0.99 cm, and the mean volume overlap for human segmentations was 88.8%. Between human and m-rep segmentations the mean surface separation was 0.18-0.19 cm, the mean Hausdorff distance was 1.14-1.25 cm, and the mean volume overlap was 82-83%. CONCLUSIONS: Overall in this study, the best m-rep kidney segmentations were at least as good as careful manual slice-by-slice segmentations performed by two experienced humans, and the worst performance was no worse than typical segmentations from our clinical setting. The mean surface separations for human-m-rep segmentations were slightly larger than for human-human segmentations but still in the subvoxel range, and volume overlap and maximum surface separation were slightly better for human-human comparisons. These results were expected because of experimental factors that favored comparison of the human-human segmentations. In particular, m-rep agreement with humans appears to have been limited largely by fundamental differences between manual slice-by-slice and true three-dimensional segmentation, imaging artifacts, image voxel dimensions, and the use of an m-rep model that produced a smooth surface across the renal pelvis.