Towards Fully Mobile 3D Face, Body, and Environment Capture Using Only Head-worn Cameras.

We propose a new approach for 3D reconstruction of dynamic indoor and outdoor scenes in everyday environments, leveraging only cameras worn by a user. This approach allows 3D reconstruction of experiences at any location and virtual tours from anywhere. The key innovation of the proposed ego-centric reconstruction system is to capture the wearer's body pose and facial expression from near-body views, e.g. cameras on the user's glasses, and to capture the surrounding environment using outward-facing views. The main challenge of the ego-centric reconstruction, however, is the poor coverage of the near-body views - that is, the user's body and face are observed from vantage points that are convenient for wear but inconvenient for capture. To overcome these challenges, we propose a parametric-model-based approach to user motion estimation. This approach utilizes convolutional neural networks (CNNs) for near-view body pose estimation, and we introduce a CNN-based approach for facial expression estimation that combines audio and video. For each time-point during capture, the intermediate model-based reconstructions from these systems are used to re-target a high-fidelity pre-scanned model of the user. We demonstrate that the proposed self-sufficient, head-worn capture system is capable of reconstructing the wearer's movements and their surrounding environment in both indoor and outdoor situations without any additional views. As a proof of concept, we show how the resulting 3D-plus-time reconstruction can be immersively experienced within a virtual reality system (e.g., the HTC Vive). We expect that the size of the proposed egocentric capture-and-reconstruction system will eventually be reduced to fit within future AR glasses, and will be widely useful for immersive 3D telepresence, virtual tours, and general use-anywhere 3D content creation.

Multiple-network classification of childhood autism using functional connectivity dynamics.

Characterization of disease using stationary resting-state functional connectivity (FC) has provided important hallmarks of abnormal brain activation in many domains. Recent studies of resting-state functional magnetic resonance imaging (fMRI), however, suggest there is a considerable amount of additional knowledge to be gained by investigating the variability in FC over the course of a scan. While a few studies have begun to explore the properties of dynamic FC for characterizing disease, the analysis of dynamic FC over multiple networks at multiple time scales has yet to be fully examined. In this study, we combine dynamic connectivity features in a multi-network, multi-scale approach to evaluate the method's potential in better classifying childhood autism. Specifically, from a set of group-level intrinsic connectivity networks (ICNs), we use sliding window correlations to compute intra-network connectivity on the subject level. We derive dynamic FC features for all ICNs over a large range of window sizes and then use a multiple kernel support vector machine (MK-SVM) model to combine a subset of these features for classification. We compare the performance our multi-network, dynamic approach to the best results obtained from single-network dynamic FC features and those obtained from both single- and multi-network static FC features. Our experiments show that integrating multiple networks on different dynamic scales has a clear superiority over these existing methods.

Maximum-margin based representation learning from multiple atlases for Alzheimer's disease classification.

In order to establish the correspondences between different brains for comparison, spatial normalization based morphometric measurements have been widely used in the analysis of Alzheimer's disease (AD). In the literature, different subjects are often compared in one atlas space, which may be insufficient in revealing complex brain changes. In this paper, instead of deploying one atlas for feature extraction and classification, we propose a maximum-margin based representation learning (MMRL) method to learn the optimal representation from multiple atlases. Unlike traditional methods that perform the representation learning separately from the classification, we propose to learn the new representation jointly with the classification model, which is more powerful in discriminating AD patients from normal controls (NC). We evaluated the proposed method on the ADNI database, and achieved 90.69% for AD/NC classification and 73.69% for p-MCI/s-MCI classification.

Deformable segmentation of 3D MR prostate images via distributed discriminative dictionary and ensemble learning.

PURPOSE: Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. METHODS: To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. RESULTS: The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. CONCLUSIONS: A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

Disrupted brain functional network in internet addiction disorder: a resting-state functional magnetic resonance imaging study.

Internet addiction disorder (IAD) is increasingly recognized as a mental health disorder, particularly among adolescents. The pathogenesis associated with IAD, however, remains unclear. In this study, we aim to explore the encephalic functional characteristics of IAD adolescents at rest using functional magnetic resonance imaging data. We adopted a graph-theoretic approach to investigate possible disruptions of functional connectivity in terms of network properties including small-worldness, efficiency, and nodal centrality on 17 adolescents with IAD and 16 socio-demographically matched healthy controls. False discovery rate-corrected parametric tests were performed to evaluate the statistical significance of group-level network topological differences. In addition, a correlation analysis was performed to assess the relationships between functional connectivity and clinical measures in the IAD group. Our results demonstrate that there is significant disruption in the functional connectome of IAD patients, particularly between regions located in the frontal, occipital, and parietal lobes. The affected connections are long-range and inter-hemispheric connections. Although significant alterations are observed for regional nodal metrics, there is no difference in global network topology between IAD and healthy groups. In addition, correlation analysis demonstrates that the observed regional abnormalities are correlated with the IAD severity and behavioral clinical assessments. Our findings, which are relatively consistent between anatomically and functionally defined atlases, suggest that IAD causes disruptions of functional connectivity and, importantly, that such disruptions might link to behavioral impairments.

Survey: Enhancing protein complex prediction in PPI networks with GO similarity weighting.

Predicting protein complexes from protein-protein interaction (PPI) networks has been the focus of many computational approaches over the last decade. These methods tend to vary in performance based on the structure of the network and the parameters provided to the algorithm. Here, we evaluate the merits of enhancing PPI networks with semantic similarity edge weights using Gene Ontology (GO) and its annotation data. We compare the cluster features and predictive efficacy of six well-known unweighted protein complex detection methods (Clique Percolation, MCODE, DPClus, IPCA, Graph Entropy, and CoAch) against updated weighted implementations. We conclude that incorporating semantic similarity edge weighting in PPI network analysis unequivocally increases the performance of these methods.