RESUMEN
We present incomplete gamma kernels, a generalization of Locally Optimal Projection (LOP) operators. In particular, we reveal the relation of the classical localized L1 estimator, used in the LOP operator for point cloud denoising, to the common Mean Shift framework via a novel kernel. Furthermore, we generalize this result to a whole family of kernels that are built upon the incomplete gamma function and each represents a localized Lp estimator. By deriving various properties of the kernel family concerning distributional, Mean Shift induced, and other aspects such as strict positive definiteness, we obtain a deeper understanding of the operator's projection behavior. From these theoretical insights, we illustrate several applications ranging from an improved Weighted LOP (WLOP) density weighting scheme and a more accurate Continuous LOP (CLOP) kernel approximation to the definition of a novel set of robust loss functions. These incomplete gamma losses include the Gaussian and LOP loss as special cases and can be applied to various tasks including normal filtering. Furthermore, we show that the novel kernels can be included as priors into neural networks. We demonstrate the effects of each application in a range of quantitative and qualitative experiments that highlight the benefits induced by our modifications.
RESUMEN
Previous work on interactive 3D labeling focused on improving user experience based on virtual/augmented reality and, thereby, speeding-up the labeling of scenes. In this article, we present a novel interactive, collaborative VR-based 3D labeling system for live-captured scenes by multiple remotely connected users based on sparse multi-user input with automatic label propagation and completion. Hence, our system is particularly beneficial in the case of multiple users that are able to label different scene parts from the respectively adequate views in parallel. Our proposed system relies on 1) the RGB-D capture of an environment by a user, 2) a reconstruction client that integrates this stream into a 3D model, 3) a server that gets scene updates and manages the global 3D scene model as well as client requests and the integration/propagation of labels, 4) labeling clients that allow an independent VR-based scene exploration and labeling for each user, and 5) remotely connected users that provide a sparse 3D labeling used to control the label propagation over objects and the label prediction to other scene parts. Our evaluation demonstrates the intuitive collaborative 3D labeling experience as well as its capability to meet the efficiency constraints regarding reconstruction speed, data streaming, visualization, and labeling.
RESUMEN
Real-time 3D scene reconstruction from RGB-D sensor data, as well as the exploration of such data in VR/AR settings, has seen tremendous progress in recent years. The combination of both these components into telepresence systems, however, comes with significant technical challenges. All approaches proposed so far are extremely demanding on input and output devices, compute resources and transmission bandwidth, and they do not reach the level of immediacy required for applications such as remote collaboration. Here, we introduce what we believe is the first practical client-server system for real-time capture and many-user exploration of static 3D scenes. Our system is based on the observation that interactive frame rates are sufficient for capturing and reconstruction, and real-time performance is only required on the client site to achieve lag-free view updates when rendering the 3D model. Starting from this insight, we extend previous voxel block hashing frameworks by introducing a novel thread-safe GPU hash map data structure that is robust under massively concurrent retrieval, insertion and removal of entries on a thread level. We further propose a novel transmission scheme for volume data that is specifically targeted to Marching Cubes geometry reconstruction and enables a 90% reduction in bandwidth between server and exploration clients. The resulting system poses very moderate requirements on network bandwidth, latency and client-side computation, which enables it to rely entirely on consumer-grade hardware, including mobile devices. We demonstrate that our technique achieves state-of-the-art representation accuracy while providing, for any number of clients, an immersive and fluid lag-free viewing experience even during network outages.