Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets.

Radiomic features are being increasingly studied for clinical applications. We aimed to assess the agreement among radiomic features when computed by several groups by using different software packages under very tightly controlled conditions, which included standardized feature definitions and common image data sets. Ten sites (9 from the NCI's Quantitative Imaging Network] positron emission tomography-computed tomography working group plus one site from outside that group) participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape, and texture. The common image data sets were: three 3D digital reference objects (DROs) and 10 patient image scans from the Lung Image Database Consortium data set using a specific lesion in each scan. Each object (DRO or lesion) was accompanied by an already-defined volume of interest, from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The coefficient of variation (CV), expressed as a percentage, was calculated across software packages for each feature on each object. Thirteen sets of results were obtained for the DROs and patient data sets. Five of the 9 features showed excellent agreement with CV < 1%; 1 feature had moderate agreement (CV < 10%), and 3 features had larger variations (CV ≥ 10%) even after attempts at harmonization of feature calculations. This work highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.

Radiomics in Brain Tumor: Image Assessment, Quantitative Feature Descriptors, and Machine-Learning Approaches.

Radiomics describes a broad set of computational methods that extract quantitative features from radiographic images. The resulting features can be used to inform imaging diagnosis, prognosis, and therapy response in oncology. However, major challenges remain for methodologic developments to optimize feature extraction and provide rapid information flow in clinical settings. Equally important, to be clinically useful, predictive radiomic properties must be clearly linked to meaningful biologic characteristics and qualitative imaging properties familiar to radiologists. Here we use a cross-disciplinary approach to highlight studies in radiomics. We review brain tumor radiologic studies (eg, imaging interpretation) through computational models (eg, computer vision and machine learning) that provide novel clinical insights. We outline current quantitative image feature extraction and prediction strategies with different levels of available clinical classes for supporting clinical decision-making. We further discuss machine-learning challenges and data opportunities to advance radiomic studies.

Automatic section thickness determination using an absolute gradient focus function.

Quantitative analysis of microstructures using computerized stereology systems is an essential tool in many disciplines of bioscience research. Section thickness determination in current nonautomated approaches requires manual location of upper and lower surfaces of tissue sections. In contrast to conventional autofocus functions that locate the optimally focused optical plane using the global maximum on a focus curve, this study identified by two sharp 'knees' on the focus curve as the transition from unfocused to focused optical planes. Analysis of 14 grey-scale focus functions showed, the thresholded absolute gradient function, was best for finding detectable bends that closely correspond to the bounding optical planes at the upper and lower tissue surfaces. Modifications to this function generated four novel functions that outperformed the original. The 'modified absolute gradient count' function outperformed all others with an average error of 0.56 µm on a test set of images similar to the training set; and, an average error of 0.39 µm on a test set comprised of images captured from a different case, that is, different staining methods on a different brain region from a different subject rat. We describe a novel algorithm that allows for automatic section thickness determination based on just out-of-focus planes, a prerequisite for fully automatic computerized stereology.

Comprehensive processing, display and analysis for in vivo MR spectroscopic imaging.

Image reconstruction for magnetic resonance spectroscopic imaging (MRSI) requires specialized spatial and spectral data processing methods and benefits from the use of several sources of prior information that are not commonly available, including MRI-derived tissue segmentation, morphological analysis and spectral characteristics of the observed metabolites. In addition, incorporating information obtained from MRI data can enhance the display of low-resolution metabolite images and multiparametric and regional statistical analysis methods can improve detection of altered metabolite distributions. As a result, full MRSI processing and analysis can involve multiple processing steps and several different data types. In this paper, a processing environment is described that integrates and automates these data processing and analysis functions for imaging of proton metabolite distributions in the normal human brain. The capabilities include normalization of metabolite signal intensities and transformation into a common spatial reference frame, thereby allowing the formation of a database of MR-measured human metabolite values as a function of acquisition, spatial and subject parameters. This development is carried out under the MIDAS project (Metabolite Imaging and Data Analysis System), which provides an integrated set of MRI and MRSI processing functions. It is anticipated that further development and distribution of these capabilities will facilitate more widespread use of MRSI for diagnostic imaging, encourage the development of standardized MRSI acquisition, processing and analysis methods and enable improved mapping of metabolite distributions in the human brain.

Integration of clinical information and gene expression profiles for prediction of chemo-response for ovarian cancer.

Ovarian cancer is the fifth leading cause of cancer death among women in the United States and western Europe. Platinum drugs are the most active agents in epithelial ovarian cancer therapy. In order to improve the prediction of response to platinum-based chemotherapy for advanced-stage ovarian cancers, we describe an integrated model which combines clinical information tumor and treatment information, with gene expression profile. This integrated modeling framework is based on the support vector machine classifier that evaluates the contributions of both clinical and gene expression data. The results show that the integrated model combining clinical information and gene expression profiles improve the prediction accuracy compared to those made by using gene expression predictor alone.

Improving Reliability of Response Prediction to Platinum-Based Therapy by AdaBoost and Multiple Classifiers.

It is a challenge to construct a reliable classifier based on microarray gene expression data for prediction of chemotherapy response, because usually only a small number of samples are available and each sample has thousands of gene expressions. This paper uses boosting and bootstrap approaches to improve the reliability of prediction. Specifically, AdaBoost and multiple classifiers based methods are used, in which support vector machines (SVMs) are utilized as the classifiers due to their good generalization ability. We compare the performance of proposed methods with a single SVM classifier system using MAS gene expression dataset in prediction of the response to platinum-based therapy for advanced-stage ovarian cancers. Statistical tests show both of the proposed methods achieve better prediction performance and have good reliability in terms of mean and standard deviation of the prediction performance for different number of selected features.

Egomotion estimation of a range camera using the space envelope.

In this paper we present a method to compute the egomotion of a range camera using the space envelope. The space envelope is a geometric model that provides more information than a simple segmentation for correspondences and motion estimation. We describe a novel variation of the maximal matching algorithm that matches surface normals to find correspondences. These correspondences are used to compute rotation and translation estimates of the egomotion. We demonstrate our methods on two image sequences containing 70 images. We also discuss the cases where our methods fail, and additional possible methods for exploiting the space envelope.

A generic knowledge-guided image segmentation and labeling system using fuzzy clustering algorithms.

Segmentation of an image into regions and the labeling of the regions is a challenging problem. In this paper, an approach that is applicable to any set of multifeature images of the same location is derived. Our approach applies to, for example, medical images of a region of the body; repeated camera images of the same area; and satellite images of a region. The segmentation and labeling approach described here uses a set of training images and domain knowledge to produce an image segmentation system that can be used without change on images of the same region collected over time. How to obtain training images, integrate domain knowledge, and utilize learning to segment and label images of the same region taken under any condition for which a training image exists is detailed. It is shown that clustering in conjunction with image processing techniques utilizing an iterative approach can effectively identify objects of interest in images. The segmentation and labeling approach described here is applied to color camera images and two other image domains are used to illustrate the applicability of the approach.

Automatic segmentation of non-enhancing brain tumors in magnetic resonance images.

Tumor segmentation from magnetic resonance (MR) images may aid in tumor treatment by tracking the progress of tumor growth and/or shrinkage. In this paper we present the first automatic segmentation method which separates non-enhancing brain tumors from healthy tissues in MR images to aid in the task of tracking tumor size over time. The MR feature images used for the segmentation consist of three weighted images (T1, T2 and proton density (PD)) for each axial slice through the head. An initial segmentation is computed using an unsupervised fuzzy clustering algorithm. Then, integrated domain knowledge and image processing techniques contribute to the final tumor segmentation. They are applied under the control of a knowledge-based system. The system knowledge was acquired by training on two patient volumes (14 images). Testing has shown successful tumor segmentations on four patient volumes (31 images). Our results show that we detected all six non-enhancing brain tumors, located tumor tissue in 35 of the 36 ground truth (radiologist labeled) slices containing tumor and successfully separated tumor regions from physically connected CSF regions in all the nine slices. Quantitative measurements are promising as correspondence ratios between ground truth and segmented tumor regions ranged between 0.368 and 0.871 per volume, with percent match ranging between 0.530 and 0.909 per volume.

Comparison of edge detection algorithms using a structure from motion task.

This paper presents an evaluation of edge detector performance. We use the task of structure from motion (SFM) as a "black box" through which to evaluate the performance of edge detection algorithms. Edge detector goodness is measured by how accurately the SFM could recover the known structure and motion from the edge detection of the image sequences. We use a variety of real image sequences with ground truth to evaluate eight different edge detectors from the literature. Our results suggest that ratings of edge detector performance based on pixel-level metrics and on the SFM are well correlated and that detectors such as the Canny detector and Heitger detector offer the best performance.

Fusion of physically-based registration and deformation modeling for nonrigid motion analysis.

In our previous work, we used finite element models to determine nonrigid motion parameters and recover unknown local properties of objects given correspondence data recovered with snakes or other tracking models. In this paper, we present a novel multiscale approach to recovery of nonrigid motion from sequences of registered intensity and range images. The main idea of our approach is that a finite element (FEM) model incorporating material properties of the object can naturally handle both registration and deformation modeling using a single model-driving strategy. The method includes a multiscale iterative algorithm based on analysis of the undirected Hausdorff distance to recover correspondences. The method is evaluated with respect to speed and accuracy. Noise sensitivity issues are addressed. Advantages of the proposed approach are demonstrated using man-made elastic materials and human skin motion. Experiments with regular grid features are used for performance comparison with a conventional approach (separate snakes and FEM models). It is shown, however, that the new method does not require a sampling/correspondence template and can adapt the model to available object features. Usefulness of the method is presented not only in the context of tracking and motion analysis, but also for a burn scar detection application.

Recovery of global nonrigid motion: a model-based approach without point correspondences

We present a novel technique for the estimation of global nonrigid motion of an object's boundary without using point correspondences. A complete description of the motion of an object's boundary involves specifying a displacement vector at each point of the boundary. Such a description provides a large amount of information, which needs to be processed further for study of the global characteristics of the deformation. Nonrigid motion can be studied hierarchically in terms of global nonrigid motion and point-by-point local nonrigid motion. The technique presented gives a method for estimating a global affine or polynomial transformation between two object boundaries. The novelty of the technique lies in the fact that it does not use any point correspondences. Our method uses hyperquadric models to model the data and estimate the global deformation. We show that affine or polynomial transformation between two datasets can be recovered from the hyperquadric parameters. The usefulness of the technique is twofold. First, it paves the way for viewing nonrigid motion hierarchically in terms of global and local motion. Second, it can be used as a front end to other motion analysis techniques that assume small motion. For instance, most nonrigid motion analysis algorithms make some assumptions on the type of nonrigid motion (conformal motion, small motion, etc.) that are not always satisfied in practice. When the motion between two datasets is large, our algorithm can be used to estimate the affine transformation (which includes scale and shear) or a polynomial transformation between the two datasets, which can then be used to warp the first dataset closer to the second so as to satisfy the small-motion assumption. We present experimental results with real and synthetic two- and three-dimensional data.

Scar assessment: current problems and future solutions.

Current problems in the assessment of scars are discussed. The concept of subjective and objective aspects of scar assessment is introduced. The patient's own view of the scar (the subjective component) can currently be assessed and may be very influential in determining the patient's quality of life, irrespective of the actual physical characteristics of the scar. The objective aspects of the scar, including size, shape, texture, and pliability, are currently difficult to measure. Although the Vancouver Scar Scale has been used as the standard for objective measurements, there are problems with both the validity and reliability of this instrument. Various imaging techniques may permit more reliable and accurate methods for measuring the quantitative aspects of scars.

A vision-based technique for objective assessment of burn scars.

In this paper a method for the objective assessment of burn scars is proposed. The quantitative measures developed in this research provide an objective way to calculate elastic properties of burn scars relative to the surrounding areas. The approach combines range data and the mechanics and motion dynamics of human tissues. Active contours are employed to locate regions of interest and to find displacements of feature points using automatically established correspondences. Changes in strain distribution over time are evaluated. Given images at two time instances and their corresponding features, the finite element method is used to synthesize strain distributions of the underlying tissues. This results in a physically based framework for motion and strain analysis. Relative elasticity of the burn scar is then recovered using iterative descent search for the best nonlinear finite element model that approximates stretching behavior of the region containing the burn scar. The results from the skin elasticity experiments illustrate the ability to objectively detect differences in elasticity between normal and abnormal tissue. These estimated differences in elasticity are correlated against the subjective judgments of physicians that are presently the practice.

Automatic tumor segmentation using knowledge-based techniques.

A system that automatically segments and labels glioblastoma-multiforme tumors in magnetic resonance images (MRI's) of the human brain is presented. The MRI's consist of T1-weighted, proton density, and T2-weighted feature images and are processed by a system which integrates knowledge-based (KB) techniques with multispectral analysis. Initial segmentation is performed by an unsupervised clustering algorithm. The segmented image, along with cluster centers for each class are provided to a rule-based expert system which extracts the intracranial region. Multispectral histogram analysis separates suspected tumor from the rest of the intracranial region, with region analysis used in performing the final tumor labeling. This system has been trained on three volume data sets and tested on thirteen unseen volume data sets acquired from a single MRI system. The KB tumor segmentation was compared with supervised, radiologist-labeled "ground truth" tumor volumes and supervised k-nearest neighbors tumor segmentations. The results of this system generally correspond well to ground truth, both on a per slice basis and more importantly in tracking total tumor volume during treatment over time.

MRI measurement of brain tumor response: comparison of visual metric and automatic segmentation.

An automatic magnetic resonance imaging (MRI) multispectral segmentation method and a visual metric are compared for their effectiveness to measure tumor response to therapy. Automatic response measurements are important for multicenter clinical trials. A visual metric such as the product of the largest diameter and the largest perpendicular diameter of the tumor is a standard approach, and is currently used in the Radiation Treatment Oncology Group (RTOG) and the Eastern Cooperative Oncology Group (EGOG) clinical trials. In the standard approach, the tumor response is based on the percentage change in the visual metric and is categorized into cure, partial response, stable disease, or progression. Both visual and automatic methods are applied to six brain tumor cases (gliomas) of varying levels of segmentation difficulty. The analyzed data were serial multispectral MR images, collected using MR contrast enhancement. A fully automatic knowledge guided method (KG) was applied to the MRI multispectral data, while the visual metric was taken from the MRI films using the T1 gadolinium enhanced image, with repeat measurements done by two radiologists and two residents. Tumor measurements from both visual and automatic methods are compared to "ground truth," (GT) i.e., manually segmented tumor. The KG method was found to slightly overestimate tumor volume, but in a consistent manner, and the estimated tumor response compared very well to hand-drawn ground truth with a correlation coefficient of 0.96. In contrast, the visually estimated metric had a large variation between observers, particularly for difficult cases, where the tumor margins are not well delineated. The inter-observer variation for the measurement of the visual metric was only 16%, i.e., observers generally agreed on the lengths of the diameters. However, in 30% of the studied cases no consensus was found for the categorical tumor response measurement, indicating that the categories are very sensitive to variations in the diameter measurements. Moreover, the method failed to correctly identify the response in half of the cases. The data demonstrate that automatic 3D methods are clearly necessary for objective and clinically meaningful assessment of tumor volume in single or multicenter clinical trials.

Medical image analysis with fuzzy models.

This paper updates several recent surveys on the use of fuzzy models for segmentation and edge detection in medical image data. Our survey is divided into methods based on supervised and unsupervised learning (that is, on whether there are or are not labelled data available for supervising the computations), and is organized first and foremost by groups (that we know of!) that are active in this area. Our review is aimed more towards 'who is doing it' rather than 'how good it is'. This is partially dictated by the fact that direct comparisons of supervised and unsupervised methods is somewhat akin to comparing apples and oranges. There is a further subdivision into methods for two- and three-dimensional data and/or problems. We do not cover methods based on neural-like networks or fuzzy reasoning systems. These topics are covered in a recently published companion survey by keller et al.

Left ventricular boundary detection from spatio-temporal volumetric computed tomography images.

This paper presents a new automatic technique for left ventricle boundary detection from a set of three-dimensional (3D) computed tomography (CT) volumetric cardiac images. The goals of this paper are to incorporate the temporal information into LV boundary detection, to link the shape modeling and LV boundary detection together, and to provide a compact representation of recovered LV boundaries to cardiac imaging. The proposed technique introduces spatio-temporal boundary detection and iterative model-based boundary refinement to left ventricular boundary extraction. The proposed technique has been applied to two sets of four-dimensional (4D) computed tomography images. Experimental results are compared with the manually edited images.

Automatic tracking of SPAMM grid and the estimation of deformation parameters from cardiac MR images.

Presents a new approach for the automatic tracking of SPAMM (Spatial Modulation of Magnetization) grid in cardiac MR images and consequent estimation of deformation parameters. The tracking is utilized to extract grid points from MR images and to establish correspondences between grid points in images taken at consecutive frames. These correspondences are used with a thin plate spline model to establish a mapping from one image to the next. This mapping is then used for motion and deformation estimation. Spatio-temporal tracking of SPAMM grid is achieved by using snakes-active contour models with an associated energy functional. The authors present a minimizing strategy which is suitable for tracking the SPAMM grid. By continuously minimizing their energy functionals, the snakes lock on to and follow the in-slice motion and deformation of the SPAMM grid. The proposed algorithm was tested with excellent results on 123 images (three data sets each a multiple slice 2D, 16 phase Cine study, three data sets each a multiple slice 2D, 13 phase Cine study and three data sets each a multiple slice 2D, 12 phase Cine study).

Knowledge-based classification and tissue labeling of MR images of human brain.

Presents a knowledge-based approach to automatic classification and tissue labeling of 2D magnetic resonance (MR) images of the human brain. The system consists of 2 components: an unsupervised clustering algorithm and an expert system. MR brain data is initially segmented by the unsupervised algorithm, then the expert system locates a landmark tissue or cluster and analyzes it by matching it with a model or searching in it for an expected feature. The landmark tissue location and its analysis are repeated until a tumor is found or all tissues are labeled. The knowledge base contains information on cluster distribution in feature space and tissue models. Since tissue shapes are irregular, their models and matching are specially designed: 1) qualitative tissue models are defined for brain tissues such as white matter; 2) default reasoning is used to match a model with an MR image; that is, if there is no mismatch between a model and an image, they are taken as matched. The system has been tested with 53 slices of MR images acquired at different times by 2 different scanners. It accurately identifies abnormal slices and provides a partial labeling of the tissues. It provides an accurate complete labeling of all normal tissues in the absence of large amounts of data nonuniformity, as verified by radiologists. Thus the system can be used to provide automatic screening of slices for abnormality. It also provides a first step toward the complete description of abnormal images for use in automatic tumor volume determination.