3D surface reconstruction of transparent objects using laser scanning with a four-layers refinement process.

Acquiring the 3D geometry of objects has been an active research topic, wherein the reconstruction of transparent objects poses a great challenge. In this paper, we present a fully automatic approach for reconstructing the exterior surface of a complex transparent scene. Through scanning a line laser by a galvo-mirror, images of the scene are captured from two viewing directions. Due to the light transmission inside the transparent object, the captured feature points and the calibrated laser plane can produce large number of 3D point candidates with large incorrect points through direct triangulation. Various situations of laser transmission inside the transparent object are analyzed and the reconstructed 3D laser point candidates are classified into two types: first-reflection points and non-first-reflection points. The first-reflection points means the first reflected laser points on the front surface of measured objects. Then, a novel four-layers refinement process is proposed to extract the first-reflection points step by step from the 3D point candidates through optical geometric constraints, including (1) Layer-1 : fake points removed by single camera, (2) Layer-2 : ambiguity points removed by the dual-camera joint constraint, (3) Layer-3 : retrieve the missing first-reflection exterior surface points by fusion and (4) Layer-4 : severe ambiguity points removed by contour-continuity. Besides, a novel calibration model about this imaging system is proposed for 3D point candidates reconstruction through triangulation. Compared with traditional laser scanning method, we pulled in the viewing angle information of the second camera and a novel four-layers refinement process is adopted for reconstruction of transparent objects. Various experiments on real objects demonstrate that proposed method can successfully extract the first-reflection points from the candidates and recover the complex shapes of transparent and semitransparent objects.

Accurate 3D Reconstruction of Dynamic Objects by Spatial-Temporal Multiplexing and Motion-Induced Error Elimination.

Three-dimensional (3D) reconstruction of dynamic objects has broad applications, including object recognition and robotic manipulation. However, achieving high-accuracy reconstruction and robustness to motion simultaneously is a challenging task. In this paper, we present a novel method for 3D reconstruction of dynamic objectS, whose main features are as follows. Firstly, a structured-light multiplexing method is developed that only requires 3 patterns to achieve high-accuracy encoding. Fewer projected patterns require shorter image acquisition time, thus, the object motion is reduced in each reconstruction cycle. The three patterns, i.e. spatial-temporally encoded patterns, are generated by embedding a specifically designed spatial-coded texture map into the temporal-encoded three-step phase-shifting fringes. A temporal codeword and three spatial codewords are extracted from the composite patterns using a proposed extraction algorithm. The two types of codewords are utilized separately in stereo matching: the temporal codeword ensures the high accuracy, while the spatial codewords are responsible for removing phase ambiguity. Secondly, we aim to eliminate the reconstruction error induced by motion between frames abbreviated as motion induced error (MiE). Instead of assuming the object to be static when acquiring the 3 images, we derive the motion of projection pixels among frames. Using the extracted spatial codewords, correspondences between different frames are found, i.e. pixels with the same codewords are traceable in the image sequences. Therefore, we can obtain the phase map at each image-acquisition moment without being affected by the object motion. Then the object surfaces corresponding to all the images can be recovered. Experimental results validate the high reconstruction accuracy and precision of the proposed method for dynamic objects with different motion speeds. Comparative experiments show that the presented method demonstrates superior performance with various types of motion, including translation in different directions and deformation.

3D reconstruction of objects with occlusion and surface reflection using a dual monocular structured light system.

Three-dimensional (3D) vision plays an important role in industrial vision, where occlusion and reflection have made it challenging to reconstruct the entire application scene. In this paper, we present a novel 3D reconstruction framework to solve the occlusion and reflection reconstruction issues in complex scenes. A dual monocular structured light system is adopted to obtain the point cloud from different viewing angles to fill the missing points in the complex scenes. To enhance the efficiency of point cloud fusion, we create a decision map that is able to avoid the reconstruction of repeating regions of the left and right system. Additionally, a compensation method based on the decision map is proposed for reducing the reconstruction error of the dual monocular system in the fusion area. Gray-code and phase-shifting patterns are utilized to encode the complex scenes, while the phase-jumping problem at the phase boundary is avoided by designing a unique compensation function. Various experiments including accuracy evaluation, comparison with the traditional fusion algorithm, and the reconstruction of real complex scenes are conducted to validate the method's accuracy and the robustness to the shiny surface and occlusion reconstruction problem.

A Real-Time 3D Laparoscopic Imaging System: Design, Method, and Validation.

OBJECTIVE: This paper aims to propose a 3D laparoscopic imaging system that can realize dense 3D reconstruction in real time. METHODS: Based on the active stereo technique which yields high-density, accurate and robust 3D reconstruction by combining structured light and stereo vision, we design a laparoscopic system consisting of two image feedback channels and one pattern projection channel. Remote high-speed image acquisition and pattern generation lay the foundation for the real-time dense 3D surface reconstruction and enable the miniaturization of the laparoscopic probe. To enhance the reconstruction efficiency and accuracy, we propose a novel active stereo method by which the dense 3D point cloud is obtained using only five patterns, while most existing multiple-shot structured light techniques require [Formula: see text] patterns. In our method, dual-frequency phase-shifting fringes are utilized to uniquely encode the pixels of the measured targets, and a dual-codeword matching scheme is developed to simplify the matching procedure and achieve high-precision reconstruction. RESULTS: Compared with the existing structured light techniques, the proposed method shows better real-time efficiency and accuracy in both quantitative and qualitative ways. Ex-vivo experiments demonstrate the robustness of the proposed method to different biological organs and the effectiveness to lesions and deformations of the organs. Feasibility of the proposed system for real-time dense 3D reconstruction is verified in dynamic experiments. According to the experimental results, the system acquires 3D point clouds with a speed of 12 frames per second. Each frame contains more than 40,000 points, and the average errors tested on standard objects are less than 0.2 mm. SIGNIFICANCE: This paper provides a new real-time dense 3D reconstruction method for 3D laparoscopic imaging. The established prototype system has shown good performance in reconstructing surface of biological tissues.