Model-Agnostic Temporal Regularizer for Object Localization Using Motion Fields.

Video analysis often requires locating and tracking target objects. In some applications, the localization system has access to the full video, which allows fine-grain motion information to be estimated. This paper proposes capturing this information through motion fields and using it to improve the localization results. The learned motion fields act as a model-agnostic temporal regularizer that can be used with any localization system based on keypoints. Unlike optical flow-based strategies, our motion fields are estimated from the model domain, based on the trajectories described by the object keypoints. Therefore, they are not affected by poor imaging conditions. The benefits of the proposed strategy are shown on three applications: 1) segmentation of cardiac magnetic resonance; 2) facial model alignment; and 3) vehicle tracking. In each case, combining popular localization methods with the proposed regularizer leads to improvement in overall accuracies and reduces gross errors.

A Survey of Feature Extraction in Dermoscopy Image Analysis of Skin Cancer.

Dermoscopy image analysis (DIA) is a growing field, with works being published every week. This makes it difficult not only to keep track of all the contributions, but also for new researchers to identify relevant information and new directions to be explored. Several surveys have been written in the past decade, but these tend to cover all of the steps of a CAD system, which can be overwhelming. Moreover, in these works, each of the steps is briefly discussed due to lack of space. Among the different blocks of the CAD system, the most relevant is the one devoted to feature extraction. This is also the block where existing works exhibit the most variability. Therefore, we believe that it is important to review the state-of-the-art on this matter. This work thoroughly explores the several types of features that have been used in DIA. A discussion on their relevance and limitations, as well as suggestions for future research are provided.

Fast segmentation of the left ventricle in cardiac MRI using dynamic programming.

BACKGROUND AND OBJECTIVE: The segmentation of the left ventricle (LV) in cardiac magnetic resonance imaging is a necessary step for the analysis and diagnosis of cardiac function. In most clinical setups, this step is still manually performed by cardiologists, which is time-consuming and laborious. This paper proposes a fast system for the segmentation of the LV that significantly reduces human intervention. METHODS: A dynamic programming approach is used to obtain the border of the LV. Using very simple assumptions about the expected shape and location of the segmentation, this system is able to deal with many of the challenges associated with this problem. The system was evaluated on two public datasets: one with 33 patients, comprising a total of 660 magnetic resonance volumes and another with 45 patients, comprising a total of 90 volumes. Quantitative evaluation of the segmentation accuracy and computational complexity was performed. RESULTS: The proposed system is able to segment a whole volume in 1.5 seconds and achieves an average Dice similarity coefficient of 86.0% and an average perpendicular distance of 2.4 mm, which compares favorably with other state-of-the-art methods. CONCLUSIONS: A system for the segmentation of the left ventricle in cardiac magnetic resonance imaging is proposed. It is a fast framework that significantly reduces the amount of time and work required of cardiologists.

Automatic 3-D segmentation of endocardial border of the left ventricle from ultrasound images.

The segmentation of the left ventricle (LV) is an important task to assess the cardiac function in ultrasound images of the heart. This paper presents a novel methodology for the segmentation of the LV in three-dimensional (3-D) echocardiographic images based on the probabilistic data association filter (PDAF). The proposed methodology begins by initializing a 3-D deformable model either semiautomatically, with user input, or automatically, and it comprises the following feature hierarchical approach: 1) edge detection in the vicinity of the surface (low-level features); 2) edge grouping to obtain potential LV surface patches (mid-level features); and 3) patch filtering using a shape-PDAF framework (high-level features). This method provides good performance accuracy in 20 echocardiographic volumes, and compares favorably with the state-of-the-art segmentation methodologies proposed in the recent literature.

Improving dermoscopy image classification using color constancy.

Robustness is one of the most important characteristics of computer-aided diagnosis systems designed for dermoscopy images. However, it is difficult to ensure this characteristic if the systems operate with multisource images acquired under different setups. Changes in the illumination and acquisition devices alter the color of images and often reduce the performance of the systems. Thus, it is important to normalize the colors of dermoscopy images before training and testing any system. In this paper, we investigate four color constancy algorithms: Gray World, max-RGB, Shades of Gray, and General Gray World. Our results show that color constancy improves the classification of multisource images, increasing the sensitivity of a bag-of-features system from 71.0% to 79.7% and the specificity from 55.2% to 76% using only 1-D RGB histograms as features.

Segmentation of the left ventricle in cardiac MRI using a probabilistic data association active shape model.

The 3D segmentation of endocardium of the left ventricle (LV) in cardiac MRI volumes is a challenging problem due to the intrinsic properties of this image modality. Typically, the object shape and position are estimated to fit the observed features collected from the images. The difficulty inherent to the LV segmentation in MRI is that the images contain outliers (i.e., observations not belonging to the LV border) due to the presence of other structures. This paper proposes a robust approach based on the Active Shape Model (ASM) that is able to circumvent the above problem. More specifically, the ASM will be guided by probabilistic data association filtering (PDAF) of strokes (i.e. line segments) computed in the neighborhood of the shape model. Thus, the proposed approach, termed herein as ASM-PDAF, will perform the following main steps: 1) edge detection (low-level features) in the vicinity of the shape model; 2) edge grouping (mid-level features) to obtain potential LV strokes; and 3) filtering using a PDAF framework (high-level features) to update the ASM. Experimental results on a public cardiac MRI database show that the proposed approach outperforms previous literature research.

A clinically oriented system for melanoma diagnosis using a color representation.

Computer Aided-Diagnosis (CAD) systems have been proposed to help dermatologists diagnose melanomas. However, these systems fail to provide a medical explanation for the diagnosis. This makes the dermatologists unsure about their use, since they are not easy to understand. In this paper we propose a CAD system that extracts a clinically inspired color description of the lesion and then, uses this information to discriminate melanomas from benign lesions. The proposed system is also capable of showing the extracted color features, making the system and its decisions more comprehensible for practitioners. The development of this system is hampered by the lack of a database of detailed annotate dermoscopy images. Nonetheless, we are able to tackle this issue using an image annotation framework based on the Correspondence-LDA algorithm. This method is applied with success to the identification of relevant colors in dermoscopy images, obtaining an average Precision of 84.9% and a Recall of 85.5%. The proposed color representation is then used to classify skin lesions, resulting in a Sensitivity of 78.9% and Specificity of 76.7%, these values are promising and comparable with the state-of-the art.

Melanoma detection algorithm based on feature fusion.

A Computer Aided-Diagnosis (CAD) System for melanoma diagnosis usually makes use of different types of features to characterize the lesions. The features are often combined into a single vector that can belong to a high dimensional space (early fusion). However, it is not clear if this is the optimal strategy and works on other fields have shown that early fusion has some limitations. In this work, we address this issue and investigate which is the best approach to combine different features comparing early and late fusion. Experiments carried on the datasets PH2 (single source) and EDRA (multi source) show that late fusion performs better, leading to classification scores of Sensitivity = 98% and Specificity = 90% (PH(2)) and Sensitivity = 83% and Specificity = 76% (EDRA).

Information geometric algorithm for estimating switching probabilities in space-varying HMM.

This paper proposes an iterative natural gradient algorithm to perform the optimization of switching probabilities in a space-varying hidden Markov model, in the context of human activity recognition in long-range surveillance. The proposed method is a version of the gradient method, developed under an information geometric viewpoint, where the usual Euclidean metric is replaced by a Riemannian metric on the space of transition probabilities. It is shown that the change in metric provides advantages over more traditional approaches, namely: 1) it turns the original constrained optimization into an unconstrained optimization problem; 2) the optimization behaves asymptotically as a Newton method and yields faster convergence than other methods for the same computational complexity; and 3) the natural gradient vector is an actual contravariant vector on the space of probability distributions for which an interpretation as the steepest descent direction is formally correct. Experiments on synthetic and real-world problems, focused on human activity recognition in long-range surveillance settings, show that the proposed methodology compares favorably with the state-of-the-art algorithms developed for the same purpose.

Manifold learning for object tracking with multiple nonlinear models.

This paper presents a novel manifold learning algorithm for high-dimensional data sets. The scope of the application focuses on the problem of motion tracking in video sequences. The framework presented is twofold. First, it is assumed that the samples are time ordered, providing valuable information that is not presented in the current methodologies. Second, the manifold topology comprises multiple charts, which contrasts to the most current methods that assume one single chart, being overly restrictive. The proposed algorithm, Gaussian process multiple local models (GP-MLM), can deal with arbitrary manifold topology by decomposing the manifold into multiple local models that are probabilistic combined using Gaussian process regression. In addition, the paper presents a multiple filter architecture where standard filtering techniques are integrated within the GP-MLM. The proposed approach exhibits comparable performance of state-of-the-art trackers, namely multiple model data association and deep belief networks, and compares favorably with Gaussian process latent variable models. Extensive experiments are presented using real video data, including a publicly available database of lip sequences and left ventricle ultrasound images, in which the GP-MLM achieves state of the art results.

PH² - a dermoscopic image database for research and benchmarking.

The increasing incidence of melanoma has recently promoted the development of computer-aided diagnosis systems for the classification of dermoscopic images. Unfortunately, the performance of such systems cannot be compared since they are evaluated in different sets of images by their authors and there are no public databases available to perform a fair evaluation of multiple systems. In this paper, a dermoscopic image database, called PH², is presented. The PH² database includes the manual segmentation, the clinical diagnosis, and the identification of several dermoscopic structures, performed by expert dermatologists, in a set of 200 dermoscopic images. The PH² database will be made freely available for research and benchmarking purposes.

Performance evaluation of point matching algorithms for left ventricle motion analysis in MRI.

Finding correspondences between contour points in consecutive frames is crucial for the left ventricular motion analysis. In many medical applications, point correspondences can be determined by using distinctive anatomical features, called anatomical landmarks. However, in the case of cardiac images, these landmarks are scarce and insufficient for the registration. Several methods have been proposed using semi-landmarks, but this may lead to incorrect correspondences. This paper proposes and evaluates the performance of three point matching algorithm. Results show that the matching by resampling method leads to the best overall correspondences and compares favorably with the performance of a state of the art shape alignment algorithm [9].

Modeling and classifying human activities from trajectories using a class of space-varying parametric motion fields.

Many approaches to trajectory analysis, such as clustering or classification, use probabilistic generative models, thus not requiring trajectory alignment/registration. Switched linear dynamical models (e.g., HMMs) have been used in this context, due to their ability to describe different motion regimes. However, these models are not suitable for handling space-dependent dynamics that are more naturally captured by nonlinear models. As is well known, these are more difficult to identify. In this paper, we propose a new way of modeling trajectories, based on a mixture of parametric motion vector fields that depend on a small number of parameters. Switching among these fields follows a probabilistic mechanism, characterized by a field of stochastic matrices. This approach allows representing a wide variety of trajectories and modeling space-dependent behaviors without using global nonlinear dynamical models. Experimental evaluation is conducted in both synthetic and real scenarios. The latter concerning with human trajectory modeling for activity classification, a central task in video surveillance.

Activity recognition using a mixture of vector fields.

The analysis of moving objects in image sequences (video) has been one of the major themes in computer vision. In this paper, we focus on video-surveillance tasks; more specifically, we consider pedestrian trajectories and propose modeling them through a small set of motion/vector fields together with a space-varying switching mechanism. Despite the diversity of motion patterns that can occur in a given scene, we show that it is often possible to find a relatively small number of typical behaviors, and model each of these behaviors by a "simple" motion field. We increase the expressiveness of the formulation by allowing the trajectories to switch from one motion field to another, in a space-dependent manner. We present an expectation-maximization algorithm to learn all the parameters of the model, and apply it to trajectory classification tasks. Experiments with both synthetic and real data support the claims about the performance of the proposed approach.

A system for the detection of pigment network in dermoscopy images using directional filters.

A pigment network is one of the most important dermoscopic structures. This paper describes an automatic system that performs its detection in dermoscopy images. The proposed system involves a set of sequential steps. First, a preprocessing algorithm is applied to the dermoscopy image. Then, a bank of directional filters and a connected component analysis are used in order to detect the "lines" of the pigment network. Finally, features are extracted from the detected network and used to train an AdaBoost algorithm to classify each lesion regarding the presence of the pigment network. The algorithm was tested on a dataset of 200 medically annotated images from the database of Hospital Pedro Hispano (Matosinhos), achieving a sensitivity = 91.1% and a specificity = 82.1%.

On the role of texture and color in the classification of dermoscopy images.

This paper addresses the detection of melanoma lesions in dermoscopy images, using texture and color features. Although melanoma detection has been studied in several works, using different types of texture, color and shape features, it is not always clear what is the role of each set of features and which features are most discriminative. This papers aims at clarifying the role of texture and color features. Furthermore, the proposed systems is based on features which can be easily implemented and tested by other researchers. It is concluded that both types of features achieve good detection scores when used alone. The best results (SE=94.1%, SP=77.4%) are achieved by combining them both.

Detecting the pigment network in dermoscopy images: a directional approach.

Several algorithms have been recently proposed for the analysis of dermoscopy images and the detection of melanomas. However, the pigment network is not considered in most of these works, although this cue plays a major role in clinical diagnosis routines. This paper proposes an algorithm for the detection of the pigment network. The algorithm is based on a bank of directional filters (difference of Gaussians) and explores color, directionality and topological properties of the network.

Improved Gradient Vector Flow for robust shape estimation in medical imaging.

We propose a improved Gradient Vector Flow (iGVF) for active contour detection. The algorithm herein proposed allows to surpass the problems of the GVF, which occur in noisy images with cluttered background. We experimentally illustrate that the proposed modified version of the GVF algorithm has a better performance in noisy images. The main difference concerns the use of more robust and informative features (edge segments) which significantly reduce the influence of noise. Experiments with real data from several image modalities are presented to illustrate the performance of the proposed approach.

Trajectory classification using switched dynamical hidden Markov models.

This paper proposes an approach for recognizing human activities (more specifically, pedestrian trajectories) in video sequences, in a surveillance context. A system for automatic processing of video information for surveillance purposes should be capable of detecting, recognizing, and collecting statistics of human activity, reducing human intervention as much as possible. In the method described in this paper, human trajectories are modeled as a concatenation of segments produced by a set of low level dynamical models. These low level models are estimated in an unsupervised fashion, based on a finite mixture formulation, using the expectation-maximization (EM) algorithm; the number of models is automatically obtained using a minimum message length (MML) criterion. This leads to a parsimonious set of models tuned to the complexity of the scene. We describe the switching among the low-level dynamic models by a hidden Markov chain; thus, the complete model is termed a switched dynamical hidden Markov model (SD-HMM). The performance of the proposed method is illustrated with real data from two different scenarios: a shopping center and a university campus. A set of human activities in both scenarios is successfully recognized by the proposed system. These experiments show the ability of our approach to properly describe trajectories with sudden changes.

Medical image noise reduction using the Sylvester-Lyapunov equation.

Multiplicative noise is often present in medical and biological imaging, such as magnetic resonance imaging (MRI), Ultrasound, positron emission tomography (PET), single photon emission computed tomography (SPECT), and fluorescence microscopy. Noise reduction in medical images is a difficult task in which linear filtering algorithms usually fail. Bayesian algorithms have been used with success but they are time consuming and computationally demanding. In addition, the increasing importance of the 3-D and 4-D medical image analysis in medical diagnosis procedures increases the amount of data that must be efficiently processed. This paper presents a Bayesian denoising algorithm which copes with additive white Gaussian and multiplicative noise described by Poisson and Rayleigh distributions. The algorithm is based on the maximum a posteriori (MAP) criterion, and edge preserving priors which avoid the distortion of relevant anatomical details. The main contribution of the paper is the unification of a set of Bayesian denoising algorithms for additive and multiplicative noise using a well-known mathematical framework, the Sylvester-Lyapunov equation, developed in the context of the Control theory.