A Deep Graph Cut Model For 3D Brain Tumor Segmentation.

Brain tumor segmentation plays a key role in tumor diagnosis and surgical planning. In this paper, we propose a solution to the 3D brain tumor segmentation problem using deep learning and graph cut from the MRI data. In particular, the probability maps of a voxel to belong to the object (tumor) and background classes from the UNet are used to improve the energy function of the graph cut. We derive new expressions for the data term, the region term and the weight factor balancing the data term and the region term for individual voxels in our proposed model. We validate the performance of our model on the publicly available BRATS 2018 dataset. Our segmentation accuracy with a dice similarity score of 0.92 is found to be higher than that of the graph cut and the UNet applied in isolation as well as over a number of state of the art approaches.

ADGAN: Attribute-Driven Generative Adversarial Network for Synthesis and Multiclass Classification of Pulmonary Nodules.

Lung cancer is the leading cause of cancer-related deaths worldwide. According to the American Cancer Society, early diagnosis of pulmonary nodules in computed tomography (CT) scans can improve the five-year survival rate up to 70 % with proper treatment planning. In this article, we propose an attribute-driven Generative Adversarial Network (ADGAN) for synthesis and multiclass classification of Pulmonary Nodules. A self-attention U-Net (SaUN) architecture is proposed to improve the generation mechanism of the network. The generator is designed with two modules, namely, self-attention attribute module (SaAM) and a self-attention spatial module (SaSM). SaAM generates a nodule image based on given attributes whereas SaSM specifies the nodule region of the input image to be altered. A reconstruction loss along with an attention localization loss (AL) is used to produce an attention map prioritizing the nodule regions. To avoid resemblance between a generated image and a real image, we further introduce an adversarial loss containing a regularization term based on KL divergence. The discriminator part of the proposed model is designed to achieve the multiclass nodule classification task. Our proposed approach is validated over two challenging publicly available datasets, namely LIDC-IDRI and LUNGX. Exhaustive experimentation on these two datasets clearly indicate that we have achieved promising classification accuracy as compared to other state-of-the-art methods.

An adaptive registration algorithm for zebrafish larval brain images.

BACKGROUND AND OBJECTIVE: Zebrafish (Danio rerio) in their larval stages have grown increasingly popular as excellent vertebrate models for neurobiological research. Researchers can apply various tools in order to decode the neural structure patterns which can aid the understanding of vertebrate brain development. In order to do so, it is essential to map the gene expression patterns to an anatomical reference precisely. However, high accuracy in sample registration is sometimes difficult to achieve due to laboratory- or protocol-dependent variabilities. METHODS: In this paper, we propose an accurate adaptive registration algorithm for volumetric zebrafish larval image datasets using a synergistic combination of attractive Free-Form-Deformation (FFD) and diffusive Demons algorithms. A coarse registration is achieved first for 3D volumetric data using a 3D affine transformation. A localized registration algorithm in form of a B-splines based FFD is applied next on the coarsely registered volume. Finally, the Demons algorithm is applied on this FFD registered volume for achieving fine registration by making the solution noise resilient. RESULTS: Results Experimental procedures are carried out on a number of 72 hpf (hours post fertilization) 3D confocal zebrafish larval datasets. Comparisons with state-of-the-art methods including some ablation studies clearly demonstrate the effectiveness of the proposed method. CONCLUSIONS: Our adaptive registration algorithm significantly aids Zebrafish imaging analysis over current methods for gene expression anatomical mapping, such as Vibe-Z. We believe the proposed solution would be able to overcome the requirement of high quality images which currently limits the applicability of Zebrafish in neuroimaging research.

Together Recognizing, Localizing and Summarizing Actions in Egocentric Videos.

Analysis of egocentric video has recently drawn attention of researchers in the computer vision as well as multimedia communities. In this paper, we propose a weakly supervised superpixel level joint framework for localization, recognition and summarization of actions in an egocentric video. We first recognize and localize single as well as multiple action(s) in each frame of an egocentric video and then construct a summary of these detected actions. The superpixel level solution helps in precise localization of actions in addition to improving the recognition accuracy. Superpixels are extracted within the central regions of the egocentric video frames; these central regions being determined through a previously developed center-surround model. A sparse spatio-temporal video representation graph is constructed in the deep feature space with the superpixels as nodes. A weakly supervised solution using random walks yields action labels for each superpixel. After determining action label(s) for each frame from its constituent superpixels, we apply a fractional knapsack type formulation for obtaining a summary (of actions). Experimental comparisons on publicly available ADL, GTEA, EGTEA Gaze+, EgoGesture, and EPIC-Kitchens datasets show the effectiveness of the proposed solution.

MDL-IWS: Multi-view Deep Learning with Iterative Watershed for Pulmonary Fissure Segmentation.

Pulmonary fissure segmentation is important for localization of lung lesions which include nodules at respective lobar territories. This can be very useful for diagnosis as well as treatment planning. In this paper, we propose a novel coarse-to-fine fissure segmentation approach by proposing a Multi-View Deep Learning driven Iterative WaterShed Algorithm (MDL-IWS). Coarse fissure segmentation obtained from multi-view deep learning yields incomplete fissure volume of interest (VOI) with additional false positives. An iterative watershed algorithm (IWS) is presented to achieve fine segmentation of fissure surfaces. As a part of the IWS algorithm, surface fitting is used to generate a more accurate fissure VOI with substantial reduction in false positives. Additionally, a weight map is used to reduce the over-segmentation of watershed in subsequent iterations. Experiments on the publicly available LOLA11 dataset clearly reveal that our method outperforms several state-of-the-art competitors.

An Iterative Spanning Forest Framework for Superpixel Segmentation.

Superpixel segmentation has emerged as an important research problem in the areas of image processing and computer vision. In this paper, we propose a framework, namely Iterative Spanning Forest (ISF), in which improved sets of connected superpixels (supervoxels in 3D) can be generated by a sequence of image foresting transforms. In this framework, one can choose the most suitable combination of ISF components for a given application-i.e., 1) a seed sampling strategy; 2) a connectivity function; 3) an adjacency relation; and 4) a seed pixel recomputation procedure. The superpixels in ISF structurally correspond to spanning trees rooted at those seeds. We present five ISF-based methods to illustrate different choices for those components. These methods are compared with a number of state-of-the-art approaches with respect to effectiveness and efficiency. Experiments are carried out on several datasets containing 2D and 3D objects with distinct texture and shape properties, including a high-level application, named sky image segmentation. The theoretical properties of ISF are demonstrated in the supplementary material and the results show ISF-based methods rank consistently among the best for all datasets.

CHAMPS: Cardiac health Hypergraph Analysis using Multimodal Physiological Signals.

State-of-the-art methods have reported various features for the non-invasive screening of Coronary Artery Disease (CAD). In this paper, we propose a novel approach to represent such features extracted from multiple physiological signals using hypergraph. Firstly, the biological and statistical interconnections among Photoplethysmogram (PPG) and Phonocardiogram (PCG) features are exploited by connecting them as hyperedges. Then, metadata features (age, weight and height) are connected using hyperedges with the rest of the features. Hypergraph based formalism provides greater flexibility in capturing the interrelationships among different features as compared to the graph counterpart. Finally, hypergraph laplacian as a derived feature is applied to classify CAD against non-CAD. The proposed method is validated on PPG and PCG data collected in a hospital setup. The results reveal 98% Sensitivity and 82% Specificity, leading to 92% classification accuracy.

CHANGE: Cardiac Health Analysis Using Graph Eigenvalues.

Coronary Artery Disease (CAD) is an important problem in cardiac health and is a leading cause of human mortality. Prior arts have shown that features extracted from non-invasive Photoplethysmogram (PPG) signal are effective in classifying CAD. In this paper, we represent cardiac health as a graph (CHG) in order to exploit the dependencies of PPG features as well as the metadata features. We then compute spectral features from the eigenvalues of the graph Laplacian of CHG. Finally, k-means algorithm is employed for classifying the data into CAD and non-CAD. Unsupervised experiments on a cohort with 32 participants yields 88% accuracy and demonstrates advantage of the proposed formulation over a baseline and two state-of-the-art approaches.

Nyström Approximated Temporally Constrained Multisimilarity Spectral Clustering Approach for Movie Scene Detection.

Movie scene detection has emerged as an important problem in present day multimedia applications. Since a movie typically consists of huge amount of video data with widespread content variations, detecting a movie scene has become extremely challenging. In this paper, we propose a fast yet accurate solution for movie scene detection using Nyström approximated multisimilarity spectral clustering with a temporal integrity constraint. We use multiple similarity matrices to model the wide content variations typically present in any movie dataset. Nyström approximation is employed to reduce the high computational cost of constructing multiple similarity measures. The temporal integrity constraint captures the inherent temporal cohesion of the movie shots. Experiments on five movie datasets from different genres clearly demonstrate the superiority of the proposed solution over the state-of-the-art methods.

A novel non-rigid registration algorithm for zebrafish larval images.

Precise three-dimensional mapping of a large number of gene expression patterns, neuronal types and connections to an anatomical reference helps us to understand the vertebrate brain and its development. Zebrafish has evolved as a model organism for such study. In this paper, we propose a novel non-rigid registration algorithm for volumetric zebrafish larval image datasets. A coarse affine registration using the L-BFGS algorithm is applied first on the moving dataset. We then divide this coarsely registered moving image and the reference image into a union of overlapping patches. Minimum weight bipartite graph matching algorithm is employed to find the correspondence between the two sets of patches. The corresponding patches are then registered using the diffeomorphic demons method with proper intra-patch regularization. For each voxel lying in the overlapping regions, we impose inter-patch regularization through a composite transformation obtained from the adjacent transformation fields. Experimental results on four multi-view confocal 3D datasets show the advantage of the proposed solution over the existing ViBE-Z software.

Cell tracking in microscopic video using matching and linking of bipartite graphs.

Automated visual tracking of cells from video microscopy has many important biomedical applications. In this paper, we track human monocyte cells in a fluorescent microscopic video using matching and linking of bipartite graphs. Tracking of cells over a pair of frames is modeled as a maximum cardinality minimum weight matching problem for a bipartite graph with a novel cost function. The tracking results are further refined using a rank-based filtering mechanism. Linking of cell trajectories over different frames are achieved through composition of bipartite matches. The proposed solution does not require any explicit motion model, is highly scalable, and, can effectively handle the entry and exit of cells. Our tracking accuracy of (97.97±0.94)% is superior than several existing methods [(95.66±2.39)%, (94.42±2.08)%, (81.22±5.75)%, (78.31±4.70)%] and is highly comparable (98.20±1.22)% to a recently published algorithm.

Multi-organ segmentation from multi-phase abdominal CT via 4D graphs using enhancement, shape and location optimization.

The interpretation of medical images benefits from anatomical and physiological priors to optimize computer-aided diagnosis (CAD) applications. Diagnosis also relies on the comprehensive analysis of multiple organs and quantitative measures of soft tissue. An automated method optimized for medical image data is presented for the simultaneous segmentation of four abdominal organs from 4D CT data using graph cuts. Contrast-enhanced CT scans were obtained at two phases: non-contrast and portal venous. Intra-patient data were spatially normalized by non-linear registration. Then 4D erosion using population historic information of contrast-enhanced liver, spleen, and kidneys was applied to multi-phase data to initialize the 4D graph and adapt to patient specific data. CT enhancement information and constraints on shape, from Parzen windows, and location, from a probabilistic atlas, were input into a new formulation of a 4D graph. Comparative results demonstrate the effects of appearance and enhancement, and shape and location on organ segmentation.

Reversible projection technique for colon unfolding.

Colon unfolding provides an efficient way to navigate the colon in computed tomographic colonography (CTC). Most existing unfolding techniques only compute forward projections. When radiologists find abnormalities or conduct measurements on the unfolded view (which is often quicker and easier), it is difficult to locate the corresponding region on the 3-D view for further examination (which is more accurate and reliable). To address this, we propose a reversible projection technique for colon unfolding. The method makes use of advanced algorithms including rotation-minimizing frames, recursive ring sets, mesh skinning, and cylindrical projection. Both forward and reverse mapping can be computed for points on the colon surface. Therefore, it allows for detecting and measuring polyps on the unfolded view and mapping them back to the 3-D surface. We generated realistic colon simulation data incorporating most colon characteristics, such as curved centerline, variable distention, haustral folds, teniae coli, and colonic polyps. Our method was tested on both simulated data and data from 110 clinical CTC studies. The results showed submillimeter accuracy in simulated data and -0.23 ± 1.67 mm in the polyp measurement using clinical CTC data. The major contributions of our technique are: 1) the use of a recursive ring set method to solve the centerline and surface correspondence problem; 2) reverse transformation from the unfolded view to the 3-D view; and 3) quantitative validation using a realistic colon simulation and clinical CTC polyp measurement.

Virtual multi-fracture craniofacial reconstruction using computer vision and graph matching.

The problem of computer vision-guided reconstruction of a fractured human mandible from a computed tomography (CT) image sequence exhibiting multiple broken fragments is addressed. The problem resembles 3D jigsaw puzzle assembly and hence is of general interest for a variety of applications dealing with automated reconstruction or assembly. The specific problem of automated multi-fracture craniofacial reconstruction is particularly challenging since the identification of opposable fracture surfaces followed by their pairwise registration needs to be performed expeditiously in order to minimize the operative trauma to the patient and also limit the operating costs. A polynomial time solution using graph matching is proposed. In the first phase of the proposed solution, the opposable fracture surfaces are identified using the Maximum Weight Graph Matching algorithm. The pairs of opposable fracture surfaces, identified in the first stage, are registered in the second phase using the Iterative Closest Point (ICP) algorithm. Correspondence for a given pair of fracture surfaces, needed for the Closest Set computation in the ICP algorithm, is established using the Maximum Cardinality Minimum Weight bipartite graph matching algorithm. The correctness of the reconstruction is constantly monitored by using constraints derived from a volumetric matching procedure guided by the computation of the Tanimoto Coefficient.

Computer vision guided virtual craniofacial reconstruction.

The problem of virtual craniofacial reconstruction from a sequence of computed tomography (CT) images is addressed and is modeled as a rigid surface registration problem. Two different classes of surface matching algorithms, namely the data aligned rigidity constrained exhaustive search (DARCES) algorithm and the iterative closest point (ICP) algorithm are first used in isolation. Since the human bone can be reasonably approximated as a rigid body, 3D rigid surface registration techniques such as the DARCES and ICP algorithms are deemed to be well suited for the purpose of aligning the fractured bone fragments. A synergistic combination of these two algorithms, termed as the hybrid DARCES-ICP algorithm, is proposed. The hybrid algorithm is shown to result in a more accurate mandibular reconstruction when compared to the individual algorithms used in isolation. The proposed scheme for virtual reconstructive surgery would prove to be of tremendous benefit to the operating surgeons as it would allow them to pre-visualize the reconstructed mandible (i.e., the end-product of their work), before performing the actual surgical procedure. Experimental results on both phantom and real (human) patient datasets are presented.