Semi-Supervised 3D Shape Segmentation via Self Refining.

3D shape segmentation is a fundamental and crucial task in the field of image processing and 3D shape analysis. To segment 3D shapes using data-driven methods, a fully labeled dataset is usually required. However, obtaining such a dataset can be a daunting task, as manual face-level labeling is both time-consuming and labor-intensive. In this paper, we present a semi-supervised framework for 3D shape segmentation that uses a small, fully labeled set of 3D shapes, as well as a weakly labeled set of 3D shapes with sparse scribble labels. Our framework first employs an auxiliary network to generate initial fully labeled segmentation labels for the sparsely labeled dataset, which helps in training the primary network. During training, the self-refine module uses increasingly accurate predictions of the primary network to improve the labels generated by the auxiliary network. Our proposed method achieves better segmentation performance than previous semi-supervised methods, as demonstrated by extensive benchmark tests, while also performing comparably to supervised methods.

A Multi-Modal Attention-Based Approach for Points of Interest Detection on 3D Shapes.

Identifying points of interest (POIs) on the surface of 3D shapes is a significant challenge in geometric processing research. The complex connection between POIs and their geometric descriptors, combined with the small percentage of POIs on the shape, makes detecting POIs on any given 3D shape a highly challenging task. Existing methods directly detect POIs from the entire 3D shape, resulting in low efficiency and accuracy. Therefore, we propose a novel multi-modal POI detection method using a coarse-to-fine approach, with the key idea of reducing data complexity and enabling more efficient and accurate subsequent POI detection by first identifying and processing important regions on the 3D shape. It first obtains important areas on the 3D shape through 2D projected images, then processes points within these regions using attention mechanisms. Extensive experiments demonstrate that our method outperforms existing POI detection techniques.

Computing Smooth and Integrable Cross Fields via Iterative Singularity Adjustment.

We propose a new method for computing smooth and integrable cross fields on 2D and 3D surfaces. We first compute smooth cross fields by minimizing the Dirichlet energy. Unlike the existing optimization based approaches, our method determines the singularity configuration, i.e., the number of singularities, their locations and indices, via iteratively adjusting singularities. The singularities can move, merge and split, as like charges repel and unlike charges attract. Once all singularities stop moving, we obtain a cross field with (locally) lowest Dirichlet energy. In simply connected domains, such a cross field is guaranteed to be integrable. However, this property does not hold in multiply connected domains. To make a smooth cross field integrable, we construct a vector field c, which characterizes how far the cross field is away from a curl-free field. Then we optimize the locations of singularities by moving them along the field lines of c. Our method is fundamentally different from the existing integer programming-based approaches, since it does not require any special numerical solver. It is fully automatic and also has a parameter to control the number of singularities. Our method is well suited for smooth models in which exact boundary alignment and sparse hard directional constraints are desired, and can guide seamless conformal parameterization and T-junction-free quadrangulation. We will make the source code publicly available.

A Variational Framework for Curve Shortening in Various Geometric Domains.

Geodesics measure the shortest distance (either locally or globally) between two points on a curved surface and serve as a fundamental tool in digital geometry processing. Suppose that we have a parameterized path γ(t)=x(u(t),v(t)) on a surface x=x(u,v) with γ(0)=p and γ(1)=q. We formulate the two-point geodesic problem into a minimization problem [Formula: see text], where H(s) satisfies and H''(s) ≥ 0 for . In our implementation, we choose H(s)=es2-1 and show that it has several unique advantages over other choices such as H(s)=s2 and H(s)=s. It is also a minimizer of the traditional geodesic length variational and able to guarantee the uniqueness and regularity in terms of curve parameterization. In the discrete setting, we construct the initial path by a sequence of moveable points {xi}i=1n and minimize ∑i=1n H(||xi - xi+1||). The resulting points are evenly spaced along the path. It's obvious that our algorithm can deal with parametric surfaces. Considering that meshes, point clouds and implicit surfaces can be transformed into a signed distance function (SDF), we also discuss its implementation on a general SDF. Finally, we show that our method can be extended to solve a general least-cost path problem. We validate the proposed algorithm in terms of accuracy, performance and scalability, and demonstrate the advantages by extensive comparisons.

Laplacian2Mesh: Laplacian-Based Mesh Understanding.

Geometric deep learning has sparked a rising interest in computer graphics to perform shape understanding tasks, such as shape classification and semantic segmentation. When the input is a polygonal surface, one has to suffer from the irregular mesh structure. Motivated by the geometric spectral theory, we introduce Laplacian2Mesh, a novel and flexible convolutional neural network (CNN) framework for coping with irregular triangle meshes (vertices may have any valence). By mapping the input mesh surface to the multi-dimensional Laplacian-Beltrami space, Laplacian2Mesh enables one to perform shape analysis tasks directly using the mature CNNs, without the need to deal with the irregular connectivity of the mesh structure. We further define a mesh pooling operation such that the receptive field of the network can be expanded while retaining the original vertex set as well as the connections between them. Besides, we introduce a channel-wise self-attention block to learn the individual importance of feature ingredients. Laplacian2Mesh not only decouples the geometry from the irregular connectivity of the mesh structure but also better captures the global features that are central to shape classification and segmentation. Extensive tests on various datasets demonstrate the effectiveness and efficiency of Laplacian2Mesh, particularly in terms of the capability of being vulnerable to noise to fulfill various learning tasks.

Neural-IMLS: Self-supervised Implicit Moving Least-Squares Network for Surface Reconstruction.

Surface reconstruction is a challenging task when input point clouds, especially real scans, are noisy and lack normals. Observing that the Multilayer Perceptron (MLP) and the implicit moving least-square function (IMLS) provide a dual representation of the underlying surface, we introduce Neural-IMLS, a novel approach that directly learns a noise-resistant signed distance function (SDF) from unoriented raw point clouds in a self-supervised manner. In particular, IMLS regularizes MLP by providing estimated SDFs near the surface and helps enhance its ability to represent geometric details and sharp features, while MLP regularizes IMLS by providing estimated normals. We prove that at convergence, our neural network produces a faithful SDF whose zero-level set approximates the underlying surface due to the mutual learning mechanism between the MLP and the IMLS. Extensive experiments on various benchmarks, including synthetic and real scans, show that Neural-IMLS can reconstruct faithful shapes even with noise and missing parts. The source code can be found at https://github.com/bearprin/Neural-IMLS.

GDR-Net: A Geometric Detail Recovering Network for 3D Scanned Objects.

This article addresses the problem of mesh super-resolution such that the geometry details which are not well represented in the low-resolution models can be recovered and well represented in the generated high-quality models. The main challenges of this problem are the nonregularity of 3D mesh representation and the high complexity of 3D shapes. We propose a deep neural network called GDR-Net to solve this ill-posed problem, which resolves the two challenges simultaneously. First, to overcome the nonregularity, we regress a displacement in radial basis function parameter space instead of the vertex-wise coordinates in the euclidean space. Second, to overcome the high complexity, we apply the detail recovery process to small surface patches extracted from the input surface and obtain the overall high-quality mesh by fusing the refined surface patches. To train the network, we constructed a dataset composed of both real-world and synthetic scanned models, including high/low-quality pairs. Our experimental results demonstrate that GDR-Net works well for general models and outperforms previous methods for recovering geometric details.

SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform.

Segmenting arbitrary 3D objects into constituent parts that are structurally meaningful is a fundamental problem encountered in a wide range of computer graphics applications. Existing methods for 3D shape segmentation suffer from complex geometry processing and heavy computation caused by using low-level features and fragmented segmentation results due to the lack of global consideration. We present an efficient method, called SEG-MAT, based on the medial axis transform (MAT) of the input shape. Specifically, with the rich geometrical and structural information encoded in the MAT, we are able to develop a simple and principled approach to effectively identify the various types of junctions between different parts of a 3D shape. Extensive evaluations and comparisons show that our method outperforms the state-of-the-art methods in terms of segmentation quality and is also one order of magnitude faster.

Geodesic Tracks: Computing Discrete Geodesics With Track-Based Steiner Point Propagation.

This article presents a simple yet effective method for computing geodesic distances on triangle meshes. Unlike the popular window propagation methods that partition mesh edges into intervals of varying lengths, our method places evenly-spaced, source-independent Steiner points on edges. Given a source vertex, our method constructs a Steiner-point graph that partitions the surface into mutually exclusive tracks, called geodesic tracks. Inside each triangle, the tracks form sub-regions in which the change of distance field is approximately linear. Our method does not require any pre-computation, and can effectively balance speed and accuracy. Experimental results show that with 5 Steiner points on each edge, the mean relative error is less than 0.3 % for common 3D models used in the graphics community. We propose a set of effective filtering rules to eliminate a large amount of useless broadcast events. For a 1000K-face model, our method runs 10 times faster than the conventional Steiner point method that examines a complete graph of Steiner points in each triangle. We also observe that using more Steiner points increases the accuracy at only a small extra computational cost. Our method works well for meshes with poor triangulation and non-manifold configuration, which often poses challenges to the existing PDE methods. We show that geodesic tracks, as a new data structure that encodes rich information of discrete geodesics, support accurate geodesic path and isoline tracing, and efficient distance query. Our method can be easily extended to meshes with non-constant density functions and/or anisotropic metrics.

Convex and Compact Superpixels by Edge- Constrained Centroidal Power Diagram.

Superpixel segmentation, as a central image processing task, has many applications in computer vision and computer graphics. Boundary alignment and shape compactness are leading indicators to evaluate a superpixel segmentation algorithm. Furthermore, convexity can make superpixels reflect more geometric structures in images and provide a more concise over-segmentation result. In this paper, we consider generating convex and compact superpixels while satisfying the constraints of adhering to the boundary as far as possible. We formulate the new superpixel segmentation into an edge-constrained centroidal power diagram (ECCPD) optimization problem. In the implementation, we optimize the superpixel configurations by repeatedly performing two alternative operations, which include site location updating and weight updating through a weight function defined by image features. Compared with existing superpixel methods, our method can partition an image into fully convex and compact superpixels with better boundary adherence. Extensive experimental results show that our approach outperforms existing superpixel segmentation methods in boundary alignment and compactness for generating convex superpixels.

Top-Down Shape Abstraction Based on Greedy Pole Selection.

Motivated by the fact that the medial axis transform is able to encode the shape completely, we propose to use as few medial balls as possible to approximate the original enclosed volume by the boundary surface. We progressively select new medial balls, in a top-down style, to enlarge the region spanned by the existing medial balls. The key spirit of the selection strategy is to encourage large medial balls while imposing given geometric constraints. We further propose a speedup technique based on a provable observation that the intersection of medial balls implies the adjacency of power cells (in the sense of the power crust).We further elaborate the selection rules in combination with two closely related applications. One application is to develop an easy-to-use ball-stick modeling system that helps non-professional users to quickly build a shape with only balls and wires, but any penetration between two medial balls must be suppressed. The other application is to generate porous structures with convex, compact (with a high isoperimetric quotient) and shape-aware pores where two adjacent spherical pores may have penetration as long as the mechanical rigidity can be well preserved.

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning.

Shape segmentation is a fundamental problem in shape analysis. Previous research shows that prior knowledge helps to improve the segmentation accuracy and quality. However, completely labeling each 3D shape in a large training data set requires a heavy manual workload. In this paper, we propose a novel weakly-supervised algorithm for segmenting 3D shapes using deep learning. Our method jointly propagates information from scribbles to unlabeled faces and learns deep neural network parameters. Therefore, it does not rely on completely labeled training shapes and only needs a really simple and convenient scribble-based partially labeling process, instead of the extremely time-consuming and tedious fully labeling processes. Various experimental results demonstrate the proposed method's superior segmentation performance over the previous unsupervised approaches and comparable segmentation performance to the state-of-the-art fully supervised methods.

Detecting 3D Points of Interest Using Multiple Features and Stacked Auto-encoder.

Considering the fact that points of interest on 3D shapes can be discriminated from a geometric perspective, it is reasonable to map the geometric signature of a point $p$p to a probability value encoding to what degree $p$p is a point of interest, especially for a specific class of 3D shapes. Based on the observation, we propose a three-phase algorithm for learning and predicting points of interest on 3D shapes by using multiple feature descriptors. Our algorithm requires two separate deep neural networks (stacked auto-encoders) to accomplish the task. During the first phase, we predict the membership of the given 3D shape according to a set of geometric descriptors using a deep neural network. After that, we train the other deep neural network to predict a probability distribution defined on the surface representing the possibility of a point being a point of interest. Finally, we use a manifold clustering technique to extract a set of points of interest as the output. Experimental results show superior detection performance of the proposed method over the previous state-of-the-art approaches.

Fast and robust shape diameter function.

The shape diameter function (SDF) is a scalar function defined on a closed manifold surface, measuring the neighborhood diameter of the object at each point. Due to its pose oblivious property, SDF is widely used in shape analysis, segmentation and retrieval. However, computing SDF is computationally expensive since one has to place an inverted cone at each point and then average the penetration distances for a number of rays inside the cone. Furthermore, the shape diameters are highly sensitive to local geometric features as well as the normal vectors, hence diminishing their applications to real-world meshes which often contain rich geometric details and/or various types of defects, such as noise and gaps. In order to increase the robustness of SDF and promote it to a wide range of 3D models, we define SDF by offsetting the input object a little bit. This seemingly minor change brings three significant benefits: First, it allows us to compute SDF in a robust manner since the offset surface is able to give reliable normal vectors. Second, it runs many times faster since at each point we only need to compute the penetration distance along a single direction, rather than tens of directions. Third, our method does not require watertight surfaces as the input-it supports both point clouds and meshes with noise and gaps. Extensive experimental results show that the offset-surface based SDF is robust to noise and insensitive to geometric details, and it also runs about 10 times faster than the existing method. We also exhibit its usefulness using two typical applications including shape retrieval and shape segmentation, and observe a significant improvement over the existing SDF.

Facial structure analysis separates autism spectrum disorders into meaningful clinical subgroups.

Varied cluster analysis were applied to facial surface measurements from 62 prepubertal boys with essential autism to determine whether facial morphology constitutes viable biomarker for delineation of discrete Autism Spectrum Disorders (ASD) subgroups. Earlier study indicated utility of facial morphology for autism subgrouping (Aldridge et al. in Mol Autism 2(1):15, 2011). Geodesic distances between standardized facial landmarks were measured from three-dimensional stereo-photogrammetric images. Subjects were evaluated for autism-related symptoms, neurologic, cognitive, familial, and phenotypic variants. The most compact cluster is clinically characterized by severe ASD, significant cognitive impairment and language regression. This verifies utility of facially-based ASD subtypes and validates Aldridge et al.'s severe ASD subgroup, notwithstanding different techniques. It suggests that language regression may define a unique ASD subgroup with potential etiologic differences.

An intrinsic algorithm for parallel Poisson disk sampling on arbitrary surfaces.

Poisson disk sampling has excellent spatial and spectral properties, and plays an important role in a variety of visual computing. Although many promising algorithms have been proposed for multidimensional sampling in euclidean space, very few studies have been reported with regard to the problem of generating Poisson disks on surfaces due to the complicated nature of the surface. This paper presents an intrinsic algorithm for parallel Poisson disk sampling on arbitrary surfaces. In sharp contrast to the conventional parallel approaches, our method neither partitions the given surface into small patches nor uses any spatial data structure to maintain the voids in the sampling domain. Instead, our approach assigns each sample candidate a random and unique priority that is unbiased with regard to the distribution. Hence, multiple threads can process the candidates simultaneously and resolve conflicts by checking the given priority values. Our algorithm guarantees that the generated Poisson disks are uniformly and randomly distributed without bias. It is worth noting that our method is intrinsic and independent of the embedding space. This intrinsic feature allows us to generate Poisson disk patterns on arbitrary surfaces in IR(n). To our knowledge, this is the first intrinsic, parallel, and accurate algorithm for surface Poisson disk sampling. Furthermore, by manipulating the spatially varying density function, we can obtain adaptive sampling easily.

Interactive applications for sketch-based editable polycube map.

In this paper, we propose a sketch-based editable polycube mapping method that, given a general mesh and a simple polycube that coarsely resembles the shape of the object, plus sketched features indicating relevant correspondences between the two, provides a uniform, regular, and user-controllable quads-only mesh that can be used as a basis structure for subdivision. Large scale models with complex geometry and topology can be processed efficiently with simple, intuitive operations. We show that the simple, intuitive nature of the polycube map is a substantial advantage from the point of view of the interface by demonstrating a series of applications, including kit-basing, shape morphing, painting over the parameterization domain, and GPU-friendly tessellated subdivision displacement, where the user is also able to control the number of patches in the base mesh by the construction of the base polycube.

Efficiently computing exact geodesic loops within finite steps.

Closed geodesics, or geodesic loops, are crucial to the study of differential topology and differential geometry. Although the existence and properties of closed geodesics on smooth surfaces have been widely studied in mathematics community, relatively little progress has been made on how to compute them on polygonal surfaces. Most existing algorithms simply consider the mesh as a graph and so the resultant loops are restricted only on mesh edges, which are far from the actual geodesics. This paper is the first to prove the existence and uniqueness of geodesic loop restricted on a closed face sequence; it contributes also with an efficient algorithm to iteratively evolve an initial closed path on a given mesh into an exact geodesic loop within finite steps. Our proposed algorithm takes only an O(k) space complexity and an O(mk) time complexity (experimentally), where m is the number of vertices in the region bounded by the initial loop and the resultant geodesic loop, and k is the average number of edges in the edge sequences that the evolving loop passes through. In contrast to the existing geodesic curvature flow methods which compute an approximate geodesic loop within a predefined threshold, our method is exact and can apply directly to triangular meshes without needing to solve any differential equation with a numerical solver; it can run at interactive speed, e.g., in the order of milliseconds, for a mesh with around 50K vertices, and hence, significantly outperforms existing algorithms. Actually, our algorithm could run at interactive speed even for larger meshes. Besides the complexity of the input mesh, the geometric shape could also affect the number of evolving steps, i.e., the performance. We motivate our algorithm with an interactive shape segmentation example shown later in the paper.

Euclidean geodesic loops on high-genus surfaces applied to the morphometry of vestibular systems.

This paper proposes a novel algorithm to extract feature landmarks on the vestibular system (VS), for the analysis of Adolescent Idiopathic Scoliosis (AIS) disease. AIS is a 3-D spinal deformity commonly occurred in adolescent girls with unclear etiology. One popular hypothesis was suggested to be the structural changes in the VS that induce the disturbed balance perception, and further cause the spinal deformity. The morphometry of VS to study the geometric differences between the healthy and AIS groups is of utmost importance. However, the VS is a genus-3 structure situated in the inner ear. The high-genus topology of the surface poses great challenge for shape analysis. In this work, we present a new method to compute exact geodesic loops on the VS. The resultant geodesic loops are in Euclidean metric, thus characterizing the intrinsic geometric properties of the VS based on the real background geometry. This leads to more accurate results than existing methods, such as the hyperbolic Ricci flow method. Furthermore, our method is fully automatic and highly efficient, e.g., one order of magnitude faster than. We applied our algorithm to the VS of normal and AIS subjects. The promising experimental results demonstrate the efficacy of our method and reveal more statistically significant shape difference in the VS between right-thoracic AIS and normal subjects.