Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.

The characterization of natural spaces by the precise observation of their material properties is highly demanded in remote sensing and computer vision. The production of novel sensors enables the collection of heterogeneous data to get a comprehensive knowledge of the living and non-living entities in the ecosystem. The high resolution of consumer-grade RGB cameras is frequently used for the geometric reconstruction of many types of environments. Nevertheless, the understanding of natural spaces is still challenging. The automatic segmentation of homogeneous materials in nature is a complex task because there are many overlapping structures and an indirect illumination, so the object recognition is difficult. In this paper, we propose a method based on fusing spatial and multispectral characteristics for the unsupervised classification of natural materials in a point cloud. A high-resolution camera and a multispectral sensor are mounted on a custom camera rig in order to simultaneously capture RGB and multispectral images. Our method is tested in a controlled scenario, where different natural objects coexist. Initially, the input RGB images are processed to generate a point cloud by applying the structure-from-motion (SfM) algorithm. Then, the multispectral images are mapped on the three-dimensional model to characterize the geometry with the reflectance captured from four narrow bands (green, red, red-edge and near-infrared). The reflectance, the visible colour and the spatial component are combined to extract key differences among all existing materials. For this purpose, a hierarchical cluster analysis is applied to pool the point cloud and identify the feature pattern for every material. As a result, the tree trunk, the leaves, different species of low plants, the ground and rocks can be clearly recognized in the scene. These results demonstrate the feasibility to perform a semantic segmentation by considering multispectral and spatial features with an unknown number of clusters to be detected on the point cloud. Moreover, our solution is compared to other method based on supervised learning in order to test the improvement of the proposed approach.

Modeling of the 3D Tree Skeleton Using Real-World Data: A Survey.

Tree modeling has been extensively studied in computer graphics. Recent advances in the development of high-resolution sensors and data processing techniques are extremely useful for collecting 3D datasets of real-world trees and generating increasingly plausible branching structures. The wide availability of versatile acquisition platforms allows us to capture multi-view images and scanned data that can be used for guided 3D tree modeling. In this paper, we carry out a comprehensive review of the state-of-the-art methods for the 3D modeling of botanical tree geometry by taking input data from real scenarios. A wide range of studies has been proposed following different approaches. The most relevant contributions are summarized and classified into three categories: (1) procedural reconstruction, (2) geometry-based extraction, and (3) image-based modeling. In addition, we describe other approaches focused on the reconstruction process by adding additional features to achieve a realistic appearance of the tree models. Thus, we provide an overview of the most effective procedures to assist researchers in the photorealistic modeling of trees in geometry and appearance. The article concludes with remarks and trends for promising research opportunities in 3D tree modeling using real-world data.