Accurate 3D Measurement of Complex Texture Objects by Height Compensation Using a Dual-Projector Structure.

Fringe projection profilometry is a widely used technique for 3D measurement due to its high accuracy and speed. However, the accuracy significantly decreases when measuring complex texture objects, especially in the junction of different colors. This paper analyzes the causes of errors resulting from complex textures and proposes a height compensation method to revise the error by employing a dual-projector structure. Moreover, the dual-projector is capable of acquiring a pair of errors with opposite signs, which can be utilized to calculate the accurate 3D information after determining the ratio of this pair of errors. Experiments provide significant improvement in measuring complex texture objects, demonstrating the proposed method's ability.

On Boundary Discontinuity in Angle Regression Based Arbitrary Oriented Object Detection.

With vigorous development e.g. in autonomous driving and remote sensing, oriented object detection has gradually been featured. The majority of existing methods directly perform regression on the rotation angle, which we argue has fundamental limitations of boundary discontinuity (even if using Gaussian or RotatedIoU-based losses). In this paper, a novel angle coder named phase-shifting coder (PSC) is proposed to address this issue. Different from another well-explored alternative i.e. angle classification, PSC achieves boundary-discontinuity-free in a continuous and differentiable manner and thus can work together with Gaussian or RotatedIoU-based methods to further boost their performance. Moreover, by rethinking the boundary discontinuity of elongated and square-like objects as rotational symmetry of different cycles, a dual-frequency version (PSCD) is proposed to accurately predict the orientation of both types of objects. Visual analysis and extensive experiments on several popular backbone detectors and datasets demonstrate the effectiveness and the potentiality of our approach. When facing scenarios requiring high-quality bounding boxes, the proposed methods are expected to give a competitive performance.

Defocused projection model for phase-shifting profilometry with a large depth range.

Phase-shifting 3D profilometry is widely combined with defocused projection, but the accuracy of defocused projection could be far below expectations especially in the case of large depth range measurement. In this paper, a new defocus-induced error related to the shape of the measured object is pinpointed and a novel defocused projection model is established to cope with such a error to improve the accuracy of defocusing phase-shifting profilometry. Supplemented with a specialized calibration and reconstruction procedure, the phase is well corrected to obtain accurate measurement results. Furthermore, the impact of the defocus-induced error is analyzed through simulations, and the feasibility of our method is verified by experiments. Faced with issues involving a large measurement range, the proposed method is expected to give a competitive performance.

Super-resolution technique for dense 3D reconstruction in fringe projection profilometry.

Fringe projection profilometry (FPP) is one of the most widely used 3D reconstruction techniques. A higher-resolution fringe pattern produces a more detailed and accurate 3D point cloud, which is critical for 3D sensing. However, there is no effective way to achieve FPP super-resolution except by using greater hardware. Therefore, this Letter proposes a dual-dense block super-resolution network (DdBSRN) to extend the fringe resolution and reconstruct a high-definition 3D shape. Especially, a novel dual-dense block structure is designed and embedded into a multi-path structure to fully utilize the local layers and fuse multiple discrete sinusoidal signals. Furthermore, a fully functional DdBSRN can be obtained even when training with a smaller data sample. Experiments demonstrate that the proposed DdBSRN method is stable and robust, and that it outperforms standard interpolation methods in terms of accuracy and 3D details.

Calibration for Camera-Projector Pairs Using Spheres.

A newly developed calibration algorithm for camera-projector system using spheres is presented in this paper. Previous studies have exploited image conics of sphere to calibrate the camera, whereas this approach can be strengthened to apply in the projector and ultimately achieve the overall calibration for single or multiple pairs of camera-projector. Following the concept of taking the projector as an inverse camera, we retrieve the image conic of the sphere on the projector plane based on a pole-polar relationship we found. At least 3 image conics on the image plane of each device are required to calculate the intrinsic parameters of the device. The extrinsic parameters for all devices in the system are determined by the position of sphere centers in each coordinates frame of the device. Based on the isotropy of the calibration object (sphere), this work is mainly interested in accomplishing the entire calibration for multiple camera-projector systems in which sensors surround a central observation volume. Experiments are conducted on both synthetic and real datasets to evaluate its performance.

Research on a hole filling algorithm of a point cloud based on structure from motion.

A point cloud can be obtained from three-dimensional measurement reconstruction based on fringe projection. However, there are holes in a point cloud due to objects with complicated shapes and the defect of the method. The holes have a profound impact on the subsequent data processing. A fitting approach to fill the holes based on structure from motion (SFM) is proposed in this paper. First, fringe projection with a two-dimensional phase is used to extract the hole boundary. Second, the registration of the SFM point cloud and the fringe projection point cloud is carried out. Then supplementary points are extracted. Third, the holes are filled based on a radial basis function on the point cloud added with the supplementary points. This method has been proven to be robust by experiments, and information of complex surface holes can be restored sufficiently.

Local blur analysis and phase error correction method for fringe projection profilometry systems.

We introduce a flexible error correction method for fringe projection profilometry (FPP) systems in the presence of local blur phenomenon. Local blur caused by global light transport such as camera defocus, projector defocus, and subsurface scattering will cause significant systematic errors in FPP systems. Previous methods, which adopt high-frequency patterns to separate the direct and global components, fail when the global light phenomenon occurs locally. In this paper, the influence of local blur on phase quality is thoroughly analyzed, and a concise error correction method is proposed to compensate the phase errors. For defocus phenomenon, this method can be directly applied. With the aid of spatially varying point spread functions and local frontal plane assumption, experiments show that the proposed method can effectively alleviate the system errors and improve the final reconstruction accuracy in various scenes. For a subsurface scattering scenario, if the translucent object is dominated by multiple scattering, the proposed method can also be applied to correct systematic errors once the bidirectional scattering-surface reflectance distribution function of the object material is measured.

Bi-tangent line based approach for multi-camera calibration using spheres.

This paper presents a novel approach for multi-camera calibration using spheres. We determine the projection of sphere centers first through bi-tangent lines of projection conics, which provides another solution for camera parameters on the basis of the relationship between the images of spheres and the image of the absolute conic in dual space. All parameters are refined by an optimization with the purpose of minimizing the reprojection error, which is divided into two independent parts associated with the corresponding shape parameters of the conics. Experimental results from the synthetic and real data show the feasibility and the accuracy achieved by our approach.

Multiple-gamma-value based phase error compensation method for phase measuring profilometry.

Three-dimensional measurement based on fringe projection has been widely used. However, the gamma nonlinearity and system nonlinearities usually result in significant phase error. Furthermore, there are various gamma values due to the non-uniform brightness distribution of the projector and nonlinear factors of the system, which makes the problem more complicated. To solve this problem, a sub-area compensation method based on multiple gamma values is proposed. In the beginning, a uniform image is projected on a standard whiteboard with a smooth surface. The obtained image is partitioned by using histogram statistics. Then, different phase error models are established for different regions. Finally, the phase error is compensated according to the regions. By applying this method, the accuracy of the phase algorithm is greatly improved. The method is simple and convenient compared with the existing methods.

Automatic fringe enhancement with novel bidimensional sinusoids-assisted empirical mode decomposition.

Fringe-based optical measurement techniques require reliable fringe analysis methods, where empirical mode decomposition (EMD) is an outstanding one due to its ability of analyzing complex signals and the merit of being data-driven. However, two challenging issues hinder the application of EMD in practical measurement. One is the tricky mode mixing problem (MMP), making the decomposed intrinsic mode functions (IMFs) have equivocal physical meaning; the other is the automatic and accurate extraction of the sinusoidal fringe from the IMFs when unpredictable and unavoidable background and noise exist in real measurements. Accordingly, in this paper, a novel bidimensional sinusoids-assisted EMD (BSEMD) is proposed to decompose a fringe pattern into mono-component bidimensional IMFs (BIMFs), with the MMP solved; properties of the resulted BIMFs are then analyzed to recognize and enhance the useful fringe component. The decomposition and the fringe recognition are integrated and the latter provides a feedback to the former, helping to automatically stop the decomposition to make the algorithm simpler and more reliable. A series of experiments show that the proposed method is accurate, efficient and robust to various fringe patterns even with poor quality, rendering it a potential tool for practical use.

Ternary Gray code-based phase unwrapping for 3D measurement using binary patterns with projector defocusing.

The three-dimensional measurement technique using binary pattern projection with projector defocusing has become increasingly important due to its high speed and high accuracy. To obtain even faster speed without sacrificing accuracy, a ternary Gray code-based phase-unwrapping method is proposed by using even fewer binary patterns, which makes it possible to efficiently and accurately unwrap the phase. Theoretical analysis, simulations, and experiments are presented to validate the proposed method's efficiency and robustness.

Phase-shifting profilometry combined with Gray-code patterns projection: unwrapping error removal by an adaptive median filter.

Phase-shifting profilometry combined with Gray-code patterns projection has been widely used for 3D measurement. In this technique, a phase-shifting algorithm is used to calculate the wrapped phase, and a set of Gray-code binary patterns is used to determine the unwrapped phase. In the real measurement, the captured Gray-code patterns are no longer binary, resulting in phase unwrapping errors at a large number of erroneous pixels. Although this problem has been attended and well resolved by a few methods, it remains challenging when a measured object has step-heights and the captured patterns contain invalid pixels. To effectively remove unwrapping errors and simultaneously preserve step-heights, in this paper, an effective method using an adaptive median filter is proposed. Both simulations and experiments can demonstrate its effectiveness.

A Novel Camera Calibration Method Based on Polar Coordinate.

A novel calibration method based on polar coordinate is proposed. The world coordinates are expressed in the form of polar coordinates, which are converted to world coordinates in the calibration process. In the beginning, the calibration points are obtained in polar coordinates. By transformation between polar coordinates and rectangular coordinates, the points turn into form of rectangular coordinates. Then, the points are matched with the corresponding image coordinates. At last, the parameters are obtained by objective function optimization. By the proposed method, the relationships between objects and cameras are expressed in polar coordinates easily. It is suitable for multi-camera calibration. Cameras can be calibrated with fewer points. The calibration images can be positioned according to the location of cameras. The experiment results demonstrate that the proposed method is an efficient calibration method. By the method, cameras are calibrated conveniently with high accuracy.

Novel 3D measurement system based on speckle and fringe pattern projection.

An efficient three-dimensional shape measurement system is proposed, which is based on the combining projection of single digital speckle pattern and phase-shifting fringe patterns. At the beginning, the initial corresponding point for each pixel is obtained by a novel speckle-phase combination method. The initial information can be calculated by the single speckle pattern in a short time, while the phase information is used to ensure the results. Unlike the conventional methods, it is not necessary to obtain the unwrapped phase, therefore the number of projected patterns is reduced greatly. Then accurate corresponding coordinates are obtained according to the wrapped phase. Three cases are analyzed while adjusting the initial corresponding coordinates locally. Thus accuracy coordinates are obtained without missing or incorrect points. Experiments demonstrate that we can achieve accurate reconstruction results with reduced measurement time by the proposed method.

Phase error analysis and compensation for phase shifting profilometry with projector defocusing.

Phase shifting profilometry (PSP) using binary fringe patterns with projector defocusing is promising for high-speed 3D shape measurement. To obtain a high-quality phase, the projector usually requires a high defocusing level, which leads to a drastic fall in fringe contrast. Due to its convenience and high speed, PSP using squared binary patterns with small phase shifting algorithms and slight defocusing is highly desirable. In this paper, the phase accuracies of the classical phase shifting algorithms are analyzed theoretically, and then compared using both simulation and experiment. We also adapt two algorithms for PSP using squared binary patterns, which include a Hilbert three-step PSP and a double three-step PSP. Both algorithms can increase phase accuracy, with the latter featuring additional invalid point detection. The adapted algorithms are also compared with the classical algorithms. Based on our analysis and comparison results, proper algorithm selection can be easily made according to the practical requirement.

Flexible calibration method for telecentric fringe projection profilometry systems.

A newly developed flexible calibration algorithm for telecentric 3D measurement systems is presented in this paper. We theoretically analyzed the similarities and differences between the telecentric and entocentric system. The telecentric system can be calibrated with the aid of the traditional 2D planar calibration method. An additional two-step refining process is proposed to improve the calibration accuracy effectively. With the calibration and refining algorithm, an affine camera can be calibrated with a reprojection error of 0.07 pixel. A projector with small field of view (FOV) is applied to achieve a full 3D reconstruction in our profilometry system. Experiments with a prototype demonstrate the validation and accuracy of the proposed calibration algorithm and system configuration. The reconstruction accuracy can achieve 5 µm with a measurement FOV of 28.43 mm×21.33 mm and a working distance of 110 mm.

Differential signal-assisted method for adaptive analysis of fringe pattern.

In dynamic 3D measurements, the recovering carrier signal as well as phase retrieval are important issues for single fringe image analysis. Local mean decomposition is a powerful signal demodulation tool, but it usually encounters an obstacle, namely, the mode-mixing problem. Then different components, especially noise and carrier signals, are all probably mixed in one of the decomposition results, thus confusing its physical meaning. Utilizing the characteristics of original noise, we design a pair of differential signals based on the conditions that two mixed components should meet to be separated completely and then add them to the original signal. Re-decomposing the newly formed signal, the differential signal, along with the original noise, will be separated from the carrier signal, leaving very little negative impact due to the characteristics of the same amplitude and opposite polarity of the differential signal. With the mode-mixing problem of high-frequency components being resolved, the decomposition of the following low-frequency components becomes more reasonable, facilitating the fringe pattern analysis and further phase retrieval. The proposed method is suitably used for the signal, even though it is not stable. Experiments illustrate the efficiency of this novel adaptive method.

Phase coding method for absolute phase retrieval with a large number of codewords.

A recently proposed phase coding method for absolute phase retrieval performs well because its codeword is embedded into phase domain rather than intensity. Then, the codeword can determine the fringe order for the phase unwrapping. However, for absolute phase retrieval with a large number of codewords, the traditional phase coding method becomes not so reliable. In this paper, we present a novel phase coding method to tackle this problem. Six additional fringe images can generate more than 64(2(6)) unique codewords for correct absolute phase retrieval. The novel phase coding method can be used for absolute phase retrieval with high frequency. Experiment results demonstrate the proposed method is effective.

Phase demodulation using adaptive windowed Fourier transform based on Hilbert-Huang transform.

The phase demodulation method of adaptive windowed Fourier transform (AWFT) is proposed based on Hilbert-Huang transform (HHT). HHT is analyzed and performed on fringe pattern to obtain instantaneous frequencies firstly. These instantaneous frequencies are further analyzed based on the condition of AWFT to locate local stationary areas where the fundamental spectrum will not be interfered by high-order spectrum. Within each local stationary area, the fundamental spectrum can be extracted accurately and adaptively by using AWFT with the background, which has been determined previously with the presented criterion during HHT, being eliminated to remove the zero-spectrum. This method is adaptive and unconstrained by any precondition for the measured phase. Experiments demonstrate its robustness and effectiveness for measuring the object with discontinuities or complex surface.

Nonrigid-deformation recovery for 3D face recognition using multiscale registration.

Nonrigid deformation is a fundamental feature in face recognition. The proposed method extracts nonrigid deformation by finding the mapping between two shapes. To improve registration accuracy, it integrates geometric shape decomposition and nonrigid point-set registration. On the basis of manifold harmonics, the method first decomposes shapes into low-frequency and high-frequency parts. Then, it applies the modified registration algorithm to obtain deformation parameters. An extensive set of experiments evaluated the method's performance on the FRGC (Face Recognition Grand Challenge) v2 database. The method achieved not only higher accuracy for fitting but also 98.2 percent identification and 97.4 percent verification at a 0.001 false-acceptance rate.