Semantic content analysis and annotation of histological images.

This paper presents a novel two-dimensional (2-D) stochastic method for semantic analysis of the content of histological images Specifically, we propose a 2-D generalization of the traditional hidden Markov model (HMM). The generalization is called spatial-hidden Markov model (SHMM) that captures the contextual characteristics of complex biological features in histological images The model employs a second-order neighborhood system and assumes the conditional independence of vertical and horizontal transitions between hidden states. The notion of 'past' in SHMM is defined as what have been observed in a row-wise raster scan. This paper focuses on two fundamental problems: the best states decoding problem and the estimation of generation probability of an image by a SHMM. Based on our independence assumption of horizontal and vertical transitions, we derive computational tractable solutions to those problems. These solutions are direct extensions of their counterparts, i.e., the Viterbi algorithm and Forward-Backward algorithm, for 1-D HMM. Our experiments were carried on a medical image database with 200 images and compared with a state-of-the-art approach that was run on the same database. The annotation results demonstrated that SHMM consistently outperforms the previous approach and ameliorates many of its drawbacks. In addition, performance comparison with HMM has also validated the superiority of SHMM.

Convolutional Neural Network Based Synthesized View Quality Enhancement for 3D Video Coding.

The quality of synthesized view plays an important role in the three dimensional (3D) video system. In this paper, to further improve the coding efficiency, a convolutional neural network (CNN) based synthesized view quality enhancement method for 3D High Efficiency Video Coding (HEVC) is proposed. Firstly, the distortion elimination in synthesized view is formulated as an image restoration task with the aim to reconstruct the latent distortion free synthesized image. Secondly, the learned CNN models are incorporated into 3D HEVC codec to improve the view synthesis performance for both view synthesis optimization (VSO) and the final synthesized view, where the geometric and compression distortions are considered according to the specific characteristics of synthesized view. Thirdly, a new Lagrange multiplier in the rate-distortion (RD) cost function is derived to adapt the CNN based VSO process to embrace a better 3D video coding performance. Extensive experimental results show that the proposed scheme can efficiently eliminate the artifacts in the synthesized image, and reduce 25.9% and 11.7% bit rate in terms of peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM) index, which significantly outperforms the state-of-theart methods.

Prediction of soft tissue deformations after CMF surgery with incremental kernel ridge regression.

Facial soft tissue deformation following osteotomy is associated with the corresponding biomechanical characteristics of bone and soft tissues. However, none of the methods devised to predict soft tissue deformation after osteotomy incorporates population-based statistical data. The aim of this study is to establish a statistical model to describe the relationship between biomechanical characteristics and soft tissue deformation after osteotomy. We proposed an incremental kernel ridge regression (IKRR) model to accomplish this goal. The input of the model is the biomechanical information computed by the Finite Element Method (FEM). The output is the soft tissue deformation generated from the paired pre-operative and post-operative 3D images. The model is adjusted incrementally with each new patient's biomechanical information. Therefore, the IKRR model enables us to predict potential soft tissue deformations for new patient by using both biomechanical and statistical information. The integration of these two types of data is critically important for accurate simulations of soft-tissue changes after surgery. The proposed method was evaluated by leave-one-out cross-validation using data from 11 patients. The average prediction error of our model (0.9103mm) was lower than some state-of-the-art algorithms. This model is promising as a reliable way to prevent the risk of facial distortion after craniomaxillofacial surgery.

Zeroing polynomials using modified constrained neural network approach.

This paper proposes new modified constrained learning neural root finders (NRFs) of polynomial constructed by backpropagation network (BPN). The technique is based on the relationships between the roots and the coefficients of polynomial as well as between the root moments and the coefficients of the polynomial. We investigated different resulting constrained learning algorithms (CLAs) based on the variants of the error cost functions (ECFs) in the constrained BPN and derived a new modified CLA (MCLA), and found that the computational complexities of the CLA and the MCLA based on the root-moment method (RMM) are the order of polynomial, and that the MCLA is simpler than the CLA. Further, we also discussed the effects of the different parameters with the CLA and the MCLA on the NRFs. In particular, considering the coefficients of the polynomials involved in practice to possibly be perturbed by noisy sources, thus, we also evaluated and discussed the effects of noises on the two NRFs. Finally, to demonstrate the advantage of our neural approaches over the nonneural ones, a series of simulating experiments are conducted.

Reconstruction and representation of caudal vasculature of zebrafish embryo from confocal scanning laser fluorescence microscopic images.

Three-dimensional (3D) reconstruction from a series of sections is an important technique in medical imaging, particularly for visualization of blood vessels from angiography. Here, we present a framework for automatic segmentation and registration of different kind of blood vessels from 2-day-old zebrafish embryos. Series of optical sections were acquired from confocal microscopy with the blood vessels labeled by fluorescent microbeads (0.02 microm) injected into blood stream of 2-day-old zebrafish embryos. Blood vessels were extracted and their morphological parameters, including length and diameter, were calculated. At the same time, individual blood vessels were registered automatically. Vasculature was represented by attributed vessel represent graph (AVRG), which contained morphological data and connectivity of every blood vessel. Using AVRG to represent a vasculature made the comparison between vasculatures of different embryos more easy. Visualization, as well as quantification, of reconstructed 3D model of AVRG was presented in an interactive interface. The framework was implemented by Visual C++ as Windows-based program.

InSPAL: A Novel Immersive Virtual Learning Programme.

In this paper we introduce The Interactive Sensory Program for Affective Learning (InSPAL) a pioneering virtual learning programme designed for the severely intellectually disabled (SID) students, who are having cognitive deficiencies and other sensory-motor handicaps, and thus need more help and attention in overcoming their learning difficulties. Through combining and integrating interactive media and virtual reality technology with the principles of art therapy and relevant pedagogical techniques, InSPAL aims to strengthen SID students' pre-learning abilities, promote their self-awareness, decrease behavioral interferences with learning as well as social interaction, enhance their communication and thus promote their quality of life. Results of our study show that students who went through our programme were more focused, and the ability to do things more independently increased by 15%. Moreover, 50% of the students showed a marked improvement in the ability to raise their hands in response, thus increasing their communication skills. The use of therapeutic interventions enabled a better control to the body, mind and emotions, resulting a greater performance and better participation.

Histological image retrieval based on semantic content analysis.

The demand for automatically recognizing and retrieving medical images for screening, reference, and management is growing faster than ever. In this paper, we present an intelligent content-based image retrieval system called I-Browse, which integrates both iconic and semantic content for histological image analysis. The I-Browse system combines low-level image processing technology with high-level semantic analysis of medical image content through different processing modules in the proposed system architecture. Similarity measures are proposed and their performance is evaluated. Furthermore, as a byproduct of semantic analysis, I-Browse allows textual annotations to be generated for unknown images. As an image browser, apart from retrieving images by image example, it also supports query by natural language.

A relational-tubular (ReTu) deformable model for vasculature quantification of zebrafish embryo from microangiography image series.

Embryonic cardiovascular system plays a vital role in embryonic development of human and animal. In this work, we introduce a novel deformable model, which we called Relational-tubular (ReTu) deformable model for segmenting and quantifying the embryonic vasculature of zebrafish embryo from microangiography image series. Particularly, to incorporate additional constraints on the spatial relationships among vessel branches, we introduce a new energy term called relation energy into the model energy function. This energy item acts as a repulsion force between neighboring vessels during the deformation to encourage them to move towards their respective volume data. Using the ReTu deformable model, the deformation process is an iterative two-stage procedure: vascular axis deformation and vascular surface deformation. The efficiency and robustness of this approach are demonstrated by experiments which show that satisfactory quantifications of the vasculature can be obtained after 3-4 iterations.

Incremental kernel ridge regression for the prediction of soft tissue deformations.

This paper proposes a nonlinear regression model to predict soft tissue deformation after maxillofacial surgery. The feature which served as input in the model is extracted with finite element model (FEM). The output in the model is the facial deformation calculated from the preoperative and postoperative 3D data. After finding the relevance between feature and facial deformation by using the regression model, we establish a general relationship which can be applied to all the patients. As a new patient comes, we predict his/her facial deformation by combining the general relationship and the new patient's biomechanical properties. Thus, our model is biomechanical relevant and statistical relevant. Validation on eleven patients demonstrates the effectiveness and efficiency of our method.

Contextual kernel and spectral methods for learning the semantics of images.

This paper presents contextual kernel and spectral methods for learning the semantics of images that allow us to automatically annotate an image with keywords. First, to exploit the context of visual words within images for automatic image annotation, we define a novel spatial string kernel to quantify the similarity between images. Specifically, we represent each image as a 2-D sequence of visual words and measure the similarity between two 2-D sequences using the shared occurrences of s -length 1-D subsequences by decomposing each 2-D sequence into two orthogonal 1-D sequences. Based on our proposed spatial string kernel, we further formulate automatic image annotation as a contextual keyword propagation problem, which can be solved very efficiently by linear programming. Unlike the traditional relevance models that treat each keyword independently, the proposed contextual kernel method for keyword propagation takes into account the semantic context of annotation keywords and propagates multiple keywords simultaneously. Significantly, this type of semantic context can also be incorporated into spectral embedding for refining the annotations of images predicted by keyword propagation. Experiments on three standard image datasets demonstrate that our contextual kernel and spectral methods can achieve significantly better results than the state of the art.

Hybrid associative retrieval of three-dimensional models.

In this paper, we propose a novel 3-D model retrieval framework, which is referred to as hybrid 3-D model associative retrieval. Unlike the conventional 3-D model similarity retrieval approach, the query model and the models obtained by 3-D model hybrid associative retrieval have the following properties: They belong to different model classes and have different shape characteristics in general but are semantically related and preassembled in a certain associative group. For instance, given a furniture associative group { desk, chair, bed}, we may probably like to use a desk as a query model to search for a list of matching models, which belong to the chair or bed class. We consider the following possibilities: 1) there can be more than two classes in an association group and 2) different association groups might have different numbers of classes. The hybrid associative retrieval is performed in two stages: 1) to establish the relationship between different 3-D model categories with semantic associations, we propose three approaches based on neural network learning and 2) to address the aforementioned two conditions, we use a cyclic-shift scheme to partition different associative groups into two-class pairwise associative groups and then adopt two different strategies to combine the final retrieval results. Experiments by using different data sets demonstrate the effectiveness and efficiency of our proposed framework on the new hybrid associative retrieval task.

Generalized competitive learning of gaussian mixture models.

When fitting Gaussian mixtures to multivariate data, it is crucial to select the appropriate number of Gaussians, which is generally referred to as the model selection problem. Under regularization theory, we aim to solve this model selection problem through developing an entropy regularized likelihood (ERL) learning on Gaussian mixtures. We further present a gradient algorithm for this ERL learning. Through some theoretic analysis, we have shown a mechanism of generalized competitive learning that is inherent in the ERL learning, which can lead to automatic model selection on Gaussian mixtures and also make our ERL learning algorithm less sensitive to the initialization as compared to the standard expectation-maximization algorithm. The experiments on simulated data using our algorithm verified our theoretic analysis. Moreover, our ERL learning algorithm has been shown to outperform other competitive learning algorithms in the application of unsupervised image segmentation.

A statistical assembled deformable model (SAMTUS) for vasculature reconstruction.

Although many deformable models have been proposed in medical applications for segmenting isolated structures in the human anatomy, not much of such work had been done on tubular structures such as the vasculature. In this paper, we propose a statistical assembled model for tubular structures (SAMTUS) to segment entire tubular structure from three-dimensional (3D) volumetric data. To our knowledge, there is no literature about the statistical deformable model for entire tubular structures. Specifically, the statistical tubular model is composed of a statistical axis model (SAM) and a statistical surface model (SSM). Both of them are assembled from a set of branch segments through the control points. Instead of searching for fuzzy correspondence along tubular axes or surfaces, we build point matching between feature points along tubular segments, and train SAM and SSM independently to characterize, respectively, the axial and the cross-sectional variation of the entire structure. In this way, more accurate point correspondence can be established, and a larger number of deformation modes can be captured. Our SAMTUS-based segmentation process consists of three stages: initialization, model fitting and final refinement. The experimental results demonstrate that the algorithm obtains good quantifications on the morphology and volume of the vasculature of the zebrafish which is being used increasingly as a specimen for drug screening and genomic research.

Iterative 3D point-set registration based on hierarchical vertex signature (HVS).

Robust 3D point registration is difficult for biomedical surfaces, especially for roundish and approximate symmetric soft tissues such as liver, stomach, etc. We present an Iterative Optimization Registration scheme (IOR) based on Hierarchical Vertex Signatures (HVS) between point-sets of medical surfaces. HVSs are distributions of concatenated neighborhood angles relative to the PCA axes of the surfaces which concisely describe global structures and local contexts around vertices in a hierarchical paradigm. The correspondences between point-sets are then established by Chi-Square test statistics. Specifically, to alleviate the sensitivity to axes directions that often affects robustness for other global axes based algorithms, IOR aligns surfaces gradually, and incrementally calibrates the directions of major axes in a multi-resolution manner. The experimental results demonstrate IOR is efficient and robust for liver registration. This method is also promising to other applications such as morphological pathological analysis, 3D model retrieval and object recognition.