Patch-Based Uncalibrated Photometric Stereo Under Natural Illumination.

This paper presents a photometric stereo method that works with unknown natural illumination without any calibration objects or initial guess of the target shape. To solve this challenging problem, we propose the use of an equivalent directional lighting model for small surface patches consisting of slowly varying normals, and solve each patch up to an arbitrary orthogonal ambiguity. We further build the patch connections by extracting consistent surface normal pairs via spatial overlaps among patches and intensity profiles. Guided by these connections, the local ambiguities are unified to a global orthogonal one through Markov Random Field optimization and rotation averaging. After applying the integrability constraint, our solution contains only a binary ambiguity, which could be easily removed. Experiments using both synthetic and real-world datasets show our method provides even comparable results to calibrated methods.

Ambiguity-Free Radiometric Calibration for Internet Photo Collections.

Radiometrically calibrating nonlinear images from Internet photo collections makes photometric analysis applicable not only to lab data but also to big image data in the wild. However, conventional calibration methods cannot be directly applied to such photo collections. This paper presents a method to jointly perform radiometric calibration for a set of nonlinear images in Internet photo collections. By incorporating the consistency of scene reflectance of corresponding pixels across nonlinear images, the proposed method first estimates radiometric response functions of all the nonlinear images up to a unique exponential ambiguity using a rank minimization framework. The ambiguity is then resolved using the linear edge color blending constraint. Quantitative evaluation using both synthetic and real-world data shows the effectiveness of the proposed method.

A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo.

Classic photometric stereo is often extended to deal with real-world materials and work with unknown lighting conditions for practicability. To quantitatively evaluate non-Lambertian and uncalibrated photometric stereo, a photometric stereo image dataset containing objects of various shapes with complex reflectance properties and high-quality ground truth normals is still missing. In this paper, we introduce the 'DiLiGenT' dataset with calibrated Directional Lightings, objects of General reflectance with different shininess, and 'ground Truth' normals from high-precision laser scanning. We use our dataset to quantitatively evaluate state-of-the-art photometric stereo methods for general materials and unknown lighting conditions, selected from a newly proposed photometric stereo taxonomy emphasizing non-Lambertian and uncalibrated methods. The dataset and evaluation results are made publicly available, and we hope it can serve as a benchmark platform that inspires future research.

A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation.

Recent advances of 3D acquisition devices have enabled large-scale acquisition of 3D scene data. Such data, if completely and well annotated, can serve as useful ingredients for a wide spectrum of computer vision and graphics works such as data-driven modeling and scene understanding, object detection and recognition. However, annotating a vast amount of 3D scene data remains challenging due to the lack of an effective tool and/or the complexity of 3D scenes (e.g. clutter, varying illumination conditions). This paper aims to build a robust annotation tool that effectively and conveniently enables the segmentation and annotation of massive 3D data. Our tool works by coupling 2D and 3D information via an interactive framework, through which users can provide high-level semantic annotation for objects. We have experimented our tool and found that a typical indoor scene could be well segmented and annotated in less than 30 minutes by using the tool, as opposed to a few hours if done manually. Along with the tool, we created a dataset of over a hundred 3D scenes associated with complete annotations using our tool. Both the tool and dataset will be available at http://scenenn.net.

GPF: GMM-Inspired Feature-Preserving Point Set Filtering.

Point set filtering, which aims at reconstructing noise-free point sets from their corresponding noisy inputs, is a fundamental problem in 3D geometry processing. The main challenge of point set filtering is to preserve geometric features of the underlying geometry while at the same time removing the noise. State-of-the-art point set filtering methods still struggle with this issue: some are not designed to recover sharp features, and others cannot well preserve geometric features, especially fine-scale features. In this paper, we propose a novel approach for robust feature-preserving point set filtering, inspired by the Gaussian Mixture Model (GMM). Taking a noisy point set and its filtered normals as input, our method can robustly reconstruct a high-quality point set which is both noise-free and feature-preserving. Various experiments show that our approach can soundly outperform the selected state-of-the-art methods, in terms of both filtering quality and reconstruction accuracy.

CODE: Coherence Based Decision Boundaries for Feature Correspondence.

A key challenge in feature correspondence is the difficulty in differentiating true and false matches at a local descriptor level. This forces adoption of strict similarity thresholds that discard many true matches. However, if analyzed at a global level, false matches are usually randomly scattered while true matches tend to be coherent (clustered around a few dominant motions), thus creating a coherence based separability constraint. This paper proposes a non-linear regression technique that can discover such a coherence based separability constraint from highly noisy matches and embed it into a correspondence likelihood model. Once computed, the model can filter the entire set of nearest neighbor matches (which typically contains over 90 percent false matches) for true matches. We integrate our technique into a full feature correspondence system which reliably generates large numbers of good quality correspondences over wide baselines where previous techniques provide few or no matches.

A Novel Riemannian Metric Based on Riemannian Structure and Scaling Information for Fixed Low-Rank Matrix Completion.

Riemannian optimization has been widely used to deal with the fixed low-rank matrix completion problem, and Riemannian metric is a crucial factor of obtaining the search direction in Riemannian optimization. This paper proposes a new Riemannian metric via simultaneously considering the Riemannian geometry structure and the scaling information, which is smoothly varying and invariant along the equivalence class. The proposed metric can make a tradeoff between the Riemannian geometry structure and the scaling information effectively. Essentially, it can be viewed as a generalization of some existing metrics. Based on the proposed Riemanian metric, we also design a Riemannian nonlinear conjugate gradient algorithm, which can efficiently solve the fixed low-rank matrix completion problem. By experimenting on the fixed low-rank matrix completion, collaborative filtering, and image and video recovery, it illustrates that the proposed method is superior to the state-of-the-art methods on the convergence efficiency and the numerical performance.

The Clutterpalette: An Interactive Tool for Detailing Indoor Scenes.

We introduce the Clutterpalette, an interactive tool for detailing indoor scenes with small-scale items. When the user points to a location in the scene, the Clutterpalette suggests detail items for that location. In order to present appropriate suggestions, the Clutterpalette is trained on a dataset of images of real-world scenes, annotated with support relations. Our experiments demonstrate that the adaptive suggestions presented by the Clutterpalette increase modeling speed and enhance the realism of indoor scenes.

3D Navigation on Impossible Figures via Dynamically Reconfigurable Maze.

Previous research on impossible figures focuses extensively on single view modeling and rendering. Existing computer games that employ impossible figures as navigation maze for gaming either use a fixed third-person view with axonometric projection to retain the figure's impossibility perception, or simply break the figure's impossibility upon view changes. In this paper, we present a new approach towards 3D gaming with impossible figures, delivering for the first time navigation in 3D mazes constructed from impossible figures. Such result cannot be achieved by previous research work in modeling impossible figures. To deliver seamless gaming navigation and interaction, we propose i) a set of guiding principles for bringing out subtle perceptions and ii) a novel computational approach to construct 3D structures from impossible figure images and then to dynamically construct the impossible-figure maze subjected to user's view. In the end, we demonstrate and discuss our method with a variety of generic maze types.

Normal Estimation of a Transparent Object Using a Video.

Reconstructing transparent objects is a challenging problem. While producing reasonable results for quite complex objects, existing approaches require custom calibration or somewhat expensive labor to achieve high precision. When an overall shape preserving salient and fine details is sufficient, we show in this paper a significant step toward solving the problem when the object's silhouette is available and simple user interaction is allowed, by using a video of a transparent object shot under varying illumination. Specifically, we estimate the normal map of the exterior surface of a given solid transparent object, from which the surface depth can be integrated. Our technical contribution lies in relating this normal estimation problem to one of graph-cut segmentation. Unlike conventional formulations, however, our graph is dual-layered, since we can see a transparent object's foreground as well as the background behind it. Quantitative and qualitative evaluation are performed to verify the efficacy of this practical solution.

Transmural imaging of ventricular action potentials and post-infarction scars in swine hearts.

The problem of using surface data to reconstruct transmural electrophysiological (EP) signals is intrinsically ill-posed without a unique solution in its unconstrained form. Incorporating physiological spatiotemporal priors through probabilistic integration of dynamic EP models, we have previously developed a Bayesian approach to transmural electrophysiological imaging (TEPI) using body-surface electrocardiograms. In this study, we generalize TEPI to using electrical signals collected from heart surfaces, and we test its feasibility on two pre-clinical swine models provided through the STACOM 2011 EP simulation Challenge. Since this new application of TEPI does not require whole-body imaging, there may be more immediate potential in EP laboratories where it could utilize catheter mapping data and produce transmural information for therapy guidance. Another focus of this study is to investigate the consistency among three modalities in delineating scar after myocardial infarction: TEPI, electroanatomical voltage mapping (EAVM), and magnetic resonance imaging (MRI). Our preliminary data demonstrate that, compared to the low-voltage scar area in EAVM, the 3-D electrical scar volume detected by TEPI is more consistent with anatomical scar volume delineated in MRI. Furthermore, TEPI could complement anatomical imaging by providing EP functional features related to both scar and healthy tissue.

A closed-form solution to tensor voting: theory and applications.

We prove a closed-form solution to tensor voting (CFTV): Given a point set in any dimensions, our closed-form solution provides an exact, continuous, and efficient algorithm for computing a structure-aware tensor that simultaneously achieves salient structure detection and outlier attenuation. Using CFTV, we prove the convergence of tensor voting on a Markov random field (MRF), thus termed as MRFTV, where the structure-aware tensor at each input site reaches a stationary state upon convergence in structure propagation. We then embed structure-aware tensor into expectation maximization (EM) for optimizing a single linear structure to achieve efficient and robust parameter estimation. Specifically, our EMTV algorithm optimizes both the tensor and fitting parameters and does not require random sampling consensus typically used in existing robust statistical techniques. We performed quantitative evaluation on its accuracy and robustness, showing that EMTV performs better than the original TV and other state-of-the-art techniques in fundamental matrix estimation for multiview stereo matching. The extensions of CFTV and EMTV for extracting multiple and nonlinear structures are underway.

Stochastic inverse consistency in medical image registration.

An essential goal in medical image registration is, the forward and reverse mapping matrices should be inverse to each other, i.e., inverse consistency. Conventional approaches enforce consistency in deterministic fashions, through incorporation of sub-objective cost function to impose source-destination symmetric property during the registration process. Assuming that the initial forward and reverse matching matrices have been computed and used as the inputs to our system, this paper presents a stochastic framework which yields perfect inverse consistency with the simultaneous considerations of the errors underneath the registration matrices and the imperfectness of the consistent constraint. An iterative generalized total least square (GTLS) strategy has been developed such that the inverse consistency is optimally imposed.