Similarity network fusion for aggregating data types on a genomic scale.

Recent technologies have made it cost-effective to collect diverse types of genome-wide data. Computational methods are needed to combine these data to create a comprehensive view of a given disease or a biological process. Similarity network fusion (SNF) solves this problem by constructing networks of samples (e.g., patients) for each available data type and then efficiently fusing these into one network that represents the full spectrum of underlying data. For example, to create a comprehensive view of a disease given a cohort of patients, SNF computes and fuses patient similarity networks obtained from each of their data types separately, taking advantage of the complementarity in the data. We used SNF to combine mRNA expression, DNA methylation and microRNA (miRNA) expression data for five cancer data sets. SNF substantially outperforms single data type analysis and established integrative approaches when identifying cancer subtypes and is effective for predicting survival.

Robust subspace discovery via relaxed rank minimization.

This letter examines the problem of robust subspace discovery from input data samples (instances) in the presence of overwhelming outliers and corruptions. A typical example is the case where we are given a set of images; each image contains, for example, a face at an unknown location of an unknown size; our goal is to identify or detect the face in the image and simultaneously learn its model. We employ a simple generative subspace model and propose a new formulation to simultaneously infer the label information and learn the model using low-rank optimization. Solving this problem enables us to simultaneously identify the ownership of instances to the subspace and learn the corresponding subspace model. We give an efficient and effective algorithm based on the alternating direction method of multipliers and provide extensive simulations and experiments to verify the effectiveness of our method. The proposed scheme can also be used to tackle many related high-dimensional combinatorial selection problems.

ELODI: Ensemble Logit Difference Inhibition for Positive-Congruent Training.

Negative flips are errors introduced in a classification system when a legacy model is updated. Existing methods to reduce the negative flip rate (NFR) either do so at the expense of overall accuracy by forcing a new model to imitate the old models, or use ensembles, which multiply inference cost prohibitively. We analyze the role of ensembles in reducing NFR and observe that they remove negative flips that are typically not close to the decision boundary, but often exhibit large deviations in the distance among their logits. Based on the observation, we present a method, called Ensemble Logit Difference Inhibition (ELODI), to train a classification system that achieves paragon performance in both error rate and NFR, at the inference cost of a single model. The method distills a homogeneous ensemble to a single student model which is used to update the classification system. ELODI also introduces a generalized distillation objective, Logit Difference Inhibition (LDI), which only penalizes the logit difference of a subset of classes with the highest logit values. On multiple image classification benchmarks, model updates with ELODI demonstrate superior accuracy retention and NFR reduction.

Automated hippocampal shape analysis predicts the onset of dementia in mild cognitive impairment.

The hippocampus is involved at the onset of the neuropathological pathways leading to Alzheimer's disease (AD). Individuals with mild cognitive impairment (MCI) are at increased risk of AD. Hippocampal volume has been shown to predict which MCI subjects will convert to AD. Our aim in the present study was to produce a fully automated prognostic procedure, scalable to high throughput clinical and research applications, for the prediction of MCI conversion to AD using 3D hippocampal morphology. We used an automated analysis for the extraction and mapping of the hippocampus from structural magnetic resonance scans to extract 3D hippocampal shape morphology, and we then applied machine learning classification to predict conversion from MCI to AD. We investigated the accuracy of prediction in 103 MCI subjects (mean age 74.1 years) from the longitudinal AddNeuroMed study. Our model correctly predicted MCI conversion to dementia within a year at an accuracy of 80% (sensitivity 77%, specificity 80%), a performance which is competitive with previous predictive models dependent on manual measurements. Categorization of MCI subjects based on hippocampal morphology revealed more rapid cognitive deterioration in MMSE scores (p<0.01) and CERAD verbal memory (p<0.01) in those subjects who were predicted to develop dementia relative to those predicted to remain stable. The pattern of atrophy associated with increased risk of conversion demonstrated initial degeneration in the anterior part of the cornus ammonis 1 (CA1) hippocampal subregion. We conclude that automated shape analysis generates sensitive measurements of early neurodegeneration which predates the onset of dementia and thus provides a prognostic biomarker for conversion of MCI to AD.

Modeling diffusion-weighted MRI as a spatially variant gaussian mixture: application to image denoising.

PURPOSE: This work describes a spatially variant mixture model constrained by a Markov random field to model high angular resolution diffusion imaging (HARDI) data. Mixture models suit HARDI well because the attenuation by diffusion is inherently a mixture. The goal is to create a general model that can be used in different applications. This study focuses on image denoising and segmentation (primarily the former). METHODS: HARDI signal attenuation data are used to train a Gaussian mixture model in which the mean vectors and covariance matrices are assumed to be independent of spatial locations, whereas the mixture weights are allowed to vary at different lattice positions. Spatial smoothness of the data is ensured by imposing a Markov random field prior on the mixture weights. The model is trained in an unsupervised fashion using the expectation maximization algorithm. The number of mixture components is determined using the minimum message length criterion from information theory. Once the model has been trained, it can be fitted to a noisy diffusion MRI volume by maximizing the posterior probability of the underlying noiseless data in a Bayesian framework, recovering a denoised version of the image. Moreover, the fitted probability maps of the mixture components can be used as features for posterior image segmentation. RESULTS: The model-based denoising algorithm proposed here was compared on real data with three other approaches that are commonly used in the literature: Gaussian filtering, anisotropic diffusion, and Rician-adapted nonlocal means. The comparison shows that, at low signal-to-noise ratio, when these methods falter, our algorithm considerably outperforms them. When tractography is performed on the model-fitted data rather than on the noisy measurements, the quality of the output improves substantially. Finally, ventricle and caudate nucleus segmentation experiments also show the potential usefulness of the mixture probability maps for classification tasks. CONCLUSIONS: The presented spatially variant mixture model for diffusion MRI provides excellent denoising results at low signal-to-noise ratios. This makes it possible to restore data acquired with a fast (i.e., noisy) pulse sequence to acceptable noise levels. This is the case in diffusion MRI, where a large number of diffusion-weighted volumes have to be acquired under clinical time constraints.

Identifying heritable brain phenotypes in an extended pedigree of vervet monkeys.

The area and volume of brain structural features, as assessed by high-resolution three-dimensional magnetic resonance imaging (MRI), are among the most heritable measures relating to the human CNS. We have conducted MRI scanning of all available monkeys >2 years of age (n = 357) from the extended multigenerational pedigree of the Vervet Research Colony (VRC). Using a combination of automated and manual segmentation we have quantified several correlated but distinct brain structural phenotypes. The estimated heritabilities (h(2)) for these measures in the VRC are higher than those reported previously for such features in humans or in other nonhuman primates: total brain volume (h(2) = 0.99, SE = 0.06), cerebral volume (h(2) = 0.98, SE = 0.06), cerebellar volume (h(2) = 0.86, SE = 0.09), hippocampal volume (h(2) = 0.95, SE = 0.07) and corpus callosum cross-sectional areas (h(2) = 0.87, SE = 0.07). These findings indicate that, in the controlled environment and with the inbreeding structure of the VRC, additive genetic factors account for almost all of the observed variance in brain structure, and suggest the potential of the VRC for genetic mapping of quantitative trait loci underlying such variance.

Automated mapping of hippocampal atrophy in 1-year repeat MRI data from 490 subjects with Alzheimer's disease, mild cognitive impairment, and elderly controls.

As one of the earliest structures to degenerate in Alzheimer's disease (AD), the hippocampus is the target of many studies of factors that influence rates of brain degeneration in the elderly. In one of the largest brain mapping studies to date, we mapped the 3D profile of hippocampal degeneration over time in 490 subjects scanned twice with brain MRI over a 1-year interval (980 scans). We examined baseline and 1-year follow-up scans of 97 AD subjects (49 males/48 females), 148 healthy control subjects (75 males/73 females), and 245 subjects with mild cognitive impairment (MCI; 160 males/85 females). We used our previously validated automated segmentation method, based on AdaBoost, to create 3D hippocampal surface models in all 980 scans. Hippocampal volume loss rates increased with worsening diagnosis (normal=0.66%/year; MCI=3.12%/year; AD=5.59%/year), and correlated with both baseline and interval changes in Mini-Mental State Examination (MMSE) scores and global and sum-of-boxes Clinical Dementia Rating scale (CDR) scores. Surface-based statistical maps visualized a selective profile of ongoing atrophy in all three diagnostic groups. Healthy controls carrying the ApoE4 gene atrophied faster than non-carriers, while more educated controls atrophied more slowly; converters from MCI to AD showed faster atrophy than non-converters. Hippocampal loss rates can be rapidly mapped, and they track cognitive decline closely enough to be used as surrogate markers of Alzheimer's disease in drug trials. They also reveal genetically greater atrophy in cognitively intact subjects.

Automated 3D mapping of hippocampal atrophy and its clinical correlates in 400 subjects with Alzheimer's disease, mild cognitive impairment, and elderly controls.

We used a new method we developed for automated hippocampal segmentation, called the auto context model, to analyze brain MRI scans of 400 subjects from the Alzheimer's disease neuroimaging initiative. After training the classifier on 21 hand-labeled expert segmentations, we created binary maps of the hippocampus for three age- and sex-matched groups: 100 subjects with Alzheimer's disease (AD), 200 with mild cognitive impairment (MCI) and 100 elderly controls (mean age: 75.84; SD: 6.64). Hippocampal traces were converted to parametric surface meshes and a radial atrophy mapping technique was used to compute average surface models and local statistics of atrophy. Surface-based statistical maps visualized links between regional atrophy and diagnosis (MCI versus controls: P = 0.008; MCI versus AD: P = 0.001), mini-mental state exam (MMSE) scores, and global and sum-of-boxes clinical dementia rating scores (CDR; all P < 0.0001, corrected). Right but not left hippocampal atrophy was associated with geriatric depression scores (P = 0.004, corrected); hippocampal atrophy was not associated with subsequent decline in MMSE and CDR scores, educational level, ApoE genotype, systolic or diastolic blood pressure measures, or homocysteine. We gradually reduced sample sizes and used false discovery rate curves to examine the method's power to detect associations with diagnosis and cognition in smaller samples. Forty subjects were sufficient to discriminate AD from normal and correlate atrophy with CDR scores; 104, 200, and 304 subjects, respectively, were required to correlate MMSE with atrophy, to distinguish MCI from normal, and MCI from AD.

Deep FisherNet for Image Classification.

Despite the great success of convolutional neural networks (CNNs) for the image classification task on data sets such as Cifar and ImageNet, CNN's representation power is still somewhat limited in dealing with images that have a large variation in size and clutter, where Fisher vector (FV) has shown to be an effective encoding strategy. FV encodes an image by aggregating local descriptors with a universal generative Gaussian mixture model (GMM). FV, however, has limited learning capability and its parameters are mostly fixed after constructing the codebook. To combine together the best of the two worlds, we propose in this brief a neural network structure with FV layer being part of an end-to-end trainable system that is differentiable; we name our network FisherNet that is learnable using back propagation. Our proposed FisherNet combines CNN training and FV encoding in a single end-to-end structure. We observe a clear advantage of FisherNet over plain CNN and standard FV in terms of both classification accuracy and computational efficiency on the challenging PASCAL visual object classes object classification and emotion image classification tasks.

Deeply Supervised Salient Object Detection with Short Connections.

Recent progress on salient object detection is substantial, benefiting mostly from the explosive development of Convolutional Neural Networks (CNNs). Semantic segmentation and salient object detection algorithms developed lately have been mostly based on Fully Convolutional Neural Networks (FCNs). There is still a large room for improvement over the generic FCN models that do not explicitly deal with the scale-space problem. The Holistically-Nested Edge Detector (HED) provides a skip-layer structure with deep supervision for edge and boundary detection, but the performance gain of HED on saliency detection is not obvious. In this paper, we propose a new salient object detection method by introducing short connections to the skip-layer structures within the HED architecture. Our framework takes full advantage of multi-level and multi-scale features extracted from FCNs, providing more advanced representations at each layer, a property that is critically needed to perform segment detection. Our method produces state-of-the-art results on 5 widely tested salient object detection benchmarks, with advantages in terms of efficiency (0.08 seconds per image), effectiveness, and simplicity over the existing algorithms. Beyond that, we conduct an exhaustive analysis of the role of training data on performance. We provide a training set for future research and fair comparisons.

Validation of a fully automated 3D hippocampal segmentation method using subjects with Alzheimer's disease mild cognitive impairment, and elderly controls.

We introduce a new method for brain MRI segmentation, called the auto context model (ACM), to segment the hippocampus automatically in 3D T1-weighted structural brain MRI scans of subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI). In a training phase, our algorithm used 21 hand-labeled segmentations to learn a classification rule for hippocampal versus non-hippocampal regions using a modified AdaBoost method, based on approximately 18,000 features (image intensity, position, image curvatures, image gradients, tissue classification maps of gray/white matter and CSF, and mean, standard deviation, and Haar filters of size 1x1x1 to 7x7x7). We linearly registered all brains to a standard template to devise a basic shape prior to capture the global shape of the hippocampus, defined as the pointwise summation of all the training masks. We also included curvature, gradient, mean, standard deviation, and Haar filters of the shape prior and the tissue classified images as features. During each iteration of ACM - our extension of AdaBoost - the Bayesian posterior distribution of the labeling was fed back in as an input, along with its neighborhood features as new features for AdaBoost to use. In validation studies, we compared our results with hand-labeled segmentations by two experts. Using a leave-one-out approach and standard overlap and distance error metrics, our automated segmentations agreed well with human raters; any differences were comparable to differences between trained human raters. Our error metrics compare favorably with those previously reported for other automated hippocampal segmentations, suggesting the utility of the approach for large-scale studies.

Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree.

In this paper, we seek to improve deep neural networks by generalizing the pooling operations that play a central role in the current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in: (1) learning a pooling function via (two strategies of) combining of max and average pooling, and (2) learning a pooling function in the form of a tree-structured fusion of pooling filters that are themselves learned. In our experiments every generalized pooling operation we explore improves performance when used in place of average or max pooling. We experimentally demonstrate that the proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets. These benefits come with only a light increase in computational overhead during training (ranging from additional 5 to 15 percent in time complexity) and a very modest increase in the number of model parameters (e.g., additional 1, 9, and 27 parameters for mixed, gated, and 2-level tree pooling operators, respectively). To gain more insights about our proposed pooling methods, we also visualize the learned pooling masks and the embeddings of the internal feature responses for different pooling operations. Our proposed pooling operations are easy to implement and can be applied within various deep neural network architectures.

Automated extraction of the cortical sulci based on a supervised learning approach.

It is important to detect and extract the major cortical sulci from brain images, but manually annotating these sulci is a time-consuming task and requires the labeler to follow complex protocols. This paper proposes a learning-based algorithm for automated extraction of the major cortical sulci from magnetic resonance imaging (MRI) volumes and cortical surfaces. Unlike alternative methods for detecting the major cortical sulci, which use a small number of predefined rules based on properties of the cortical surface such as the mean curvature, our approach learns a discriminative model using the probabilistic boosting tree algorithm (PBT). PBT is a supervised learning approach which selects and combines hundreds of features at different scales, such as curvatures, gradients and shape index. Our method can be applied to either MRI volumes or cortical surfaces. It first outputs a probability map which indicates how likely each voxel lies on a major sulcal curve. Next, it applies dynamic programming to extract the best curve based on the probability map and a shape prior. The algorithm has almost no parameters to tune for extracting different major sulci. It is very fast (it runs in under 1 min per sulcus including the time to compute the discriminative models) due to efficient implementation of the features (e.g., using the integral volume to rapidly compute the responses of 3-D Haar filters). Because the algorithm can be applied to MRI volumes directly, there is no need to perform preprocessing such as tissue segmentation or mapping to a canonical space. The learning aspect of our approach makes the system very flexible and general. For illustration, we use volumes of the right hemisphere with several major cortical sulci manually labeled. The algorithm is tested on two groups of data, including some brains from patients with Williams Syndrome, and the results are very encouraging.

Unsupervised Object Class Discovery via Saliency-Guided Multiple Class Learning.

In this paper, we tackle the problem of common object (multiple classes) discovery from a set of input images, where we assume the presence of one object class in each image. This problem is, loosely speaking, unsupervised since we do not know a priori about the object type, location, and scale in each image. We observe that the general task of object class discovery in a fully unsupervised manner is intrinsically ambiguous; here we adopt saliency detection to propose candidate image windows/patches to turn an unsupervised learning problem into a weakly-supervised learning problem. In the paper, we propose an algorithm for simultaneously localizing objects and discovering object classes via bottom-up (saliency-guided) multiple class learning (bMCL). Our contributions are three-fold: (1) we adopt saliency detection to convert unsupervised learning into multiple instance learning, formulated as bottom-up multiple class learning (bMCL); (2) we propose an integrated framework that simultaneously performs object localization, object class discovery, and object detector training; (3) we demonstrate that our framework yields significant improvements over existing methods for multi-class object discovery and possess evident advantages over competing methods in computer vision. In addition, although saliency detection has recently attracted much attention, its practical usage for high-level vision tasks has yet to be justified. Our method validates the usefulness of saliency detection to output "noisy input" for a top-down method to extract common patterns.

A two-layer structure prediction framework for microscopy cell detection.

The task of microscopy cell detection is of great biological and clinical importance. However, existing algorithms for microscopy cell detection usually ignore the large variations of cells and only focus on the shape feature/descriptor design. Here we propose a new two-layer model for cell centre detection by a two-layer structure prediction framework, which is respectively built on classification for the cell centres implicitly using rich appearances and contextual information and explicit structural information for the cells. Experimental results demonstrate the efficiency and effectiveness of the proposed method over competing state-of-the-art methods, providing a viable alternative for microscopy cell detection.

Range image segmentation by an effective jump-diffusion method.

This paper presents an effective jump-diffusion method for segmenting a range image and its associated reflectance image in the Bayesian framework. The algorithm works on complex real-world scenes (indoor and outdoor), which consist of an unknown number of objects (or surfaces) of various sizes and types, such as planes, conics, smooth surfaces, and cluttered objects (like trees and bushes). Formulated in the Bayesian framework, the posterior probability is distributed over a solution space with a countable number of subspaces of varying dimensions. The algorithm simulates Markov chains with both reversible jumps and stochastic diffusions to traverse the solution space. The reversible jumps realize the moves between subspaces of different dimensions, such as switching surface models and changing the number of objects. The stochastic Langevin equation realizes diffusions within each subspace. To achieve effective computation, the algorithm precomputes some importance proposal probabilities over multiple scales through Hough transforms, edge detection, and data clustering. The latter are used by the Markov chains for fast mixing. The algorithm is tested on 100 1D simulated data sets for performance analysis on both accuracy and speed. Then, the algorithm is applied to three data sets of range images under the same parameter setting. The results are satisfactory in comparison with manual segmentations.

Symmetry constraint for foreground extraction.

Symmetry as an intrinsic shape property is often observed in natural objects. In this paper, we discuss how explicitly taking into account the symmetry constraint can enhance the quality of foreground object extraction. In our method, a symmetry foreground map is used to represent the symmetry structure of the image, which includes the symmetry matching magnitude and the foreground location prior. Then, the symmetry constraint model is built by introducing this symmetry structure into the graph-based segmentation function. Finally, the segmentation result is obtained via graph cuts. Our method encourages objects with symmetric parts to be consistently extracted. Moreover, our symmetry constraint model is applicable to weak symmetric objects under the part-based framework. Quantitative and qualitative experimental results on benchmark datasets demonstrate the advantages of our approach in extracting the foreground. Our method also shows improved results in segmenting objects with weak, complex symmetry properties.

Exemplar-based human action pose correction.

The launch of Xbox Kinect has built a very successful computer vision product and made a big impact on the gaming industry. This sheds lights onto a wide variety of potential applications related to action recognition. The accurate estimation of human poses from the depth image is universally a critical step. However, existing pose estimation systems exhibit failures when facing severe occlusion. In this paper, we propose an exemplar-based method to learn to correct the initially estimated poses. We learn an inhomogeneous systematic bias by leveraging the exemplar information within a specific human action domain. Furthermore, as an extension, we learn a conditional model by incorporation of pose tags to further increase the accuracy of pose correction. In the experiments, significant improvements on both joint-based skeleton correction and tag prediction are observed over the contemporary approaches, including what is delivered by the current Kinect system. Our experiments for the facial landmark correction also illustrate that our algorithm can improve the accuracy of other detection/estimation systems.

Weakly supervised histopathology cancer image segmentation and classification.

Labeling a histopathology image as having cancerous regions or not is a critical task in cancer diagnosis; it is also clinically important to segment the cancer tissues and cluster them into various classes. Existing supervised approaches for image classification and segmentation require detailed manual annotations for the cancer pixels, which are time-consuming to obtain. In this paper, we propose a new learning method, multiple clustered instance learning (MCIL) (along the line of weakly supervised learning) for histopathology image segmentation. The proposed MCIL method simultaneously performs image-level classification (cancer vs. non-cancer image), medical image segmentation (cancer vs. non-cancer tissue), and patch-level clustering (different classes). We embed the clustering concept into the multiple instance learning (MIL) setting and derive a principled solution to performing the above three tasks in an integrated framework. In addition, we introduce contextual constraints as a prior for MCIL, which further reduces the ambiguity in MIL. Experimental results on histopathology colon cancer images and cytology images demonstrate the great advantage of MCIL over the competing methods.

Cluster-based co-saliency detection.

Co-saliency is used to discover the common saliency on the multiple images, which is a relatively underexplored area. In this paper, we introduce a new cluster-based algorithm for co-saliency detection. Global correspondence between the multiple images is implicitly learned during the clustering process. Three visual attention cues: contrast, spatial, and corresponding, are devised to effectively measure the cluster saliency. The final co-saliency maps are generated by fusing the single image saliency and multiimage saliency. The advantage of our method is mostly bottom-up without heavy learning, and has the property of being simple, general, efficient, and effective. Quantitative and qualitative experiments result in a variety of benchmark datasets demonstrating the advantages of the proposed method over the competing co-saliency methods. Our method on single image also outperforms most the state-of-the-art saliency detection methods. Furthermore, we apply the co-saliency method on four vision applications: co-segmentation, robust image distance, weakly supervised learning, and video foreground detection, which demonstrate the potential usages of the co-saliency map.