Modeling the measurement precision of Fringe Projection Profilometry.

Three-dimensional (3D) surface geometry provides elemental information in various sciences and precision engineering. Fringe Projection Profilometry (FPP) is one of the most powerful non-contact (thus non-destructive) and non-interferometric (thus less restrictive) 3D measurement techniques, featuring at its high precision. However, the measurement precision of FPP is currently evaluated experimentally, lacking a complete theoretical model for guidance. We propose the first complete FPP precision model chain including four stage models (camera intensity, fringe intensity, phase and 3D geometry) and two transfer models (from fringe intensity to phase and from phase to 3D geometry). The most significant contributions include the adoption of a non-Gaussian camera noise model, which, for the first time, establishes the connection between camera's electronics parameters (known in advance from the camera manufacturer) and the phase precision, and the formulation of the phase to geometry transfer, which makes the precision of the measured geometry representable in an explicit and concise form. As a result, we not only establish the full precision model of the 3D geometry to characterize the performance of an FPP system that has already been set up, but also explore the expression of the highest possible precision limit to guide the error distribution of an FPP system that is yet to build. Our theoretical models make FPP a more designable technique to meet the challenges from various measurement demands concerning different object sizes from macro to micro and requiring different measurement precisions from a few millimeters to a few micrometers.

Fringe projection profilometry method with high efficiency, precision, and convenience: theoretical analysis and development.

Fringe projector profilometry (FPP) is an important three-dimensional (3D) measurement technique, especially when high precision and speed are required. Thus, theoretical interrogation is critical to provide deep understanding and possible improvement of FPP. By dividing an FPP measurement process into four steps (system calibration, phase measurement, pixel correspondence, and 3D reconstruction), we give theoretical analysis on the entire process except for the extensively studied calibration step. Our study indeed reveals a series of important system properties, to the best of our knowledge, for the first time: (i) in phase measurement, the optimal and worst fringe angles are proven perpendicular and parallel to epipolar line, respectively, and can be considered as system parameters and can be directly made available during traditional calibration, highlighting the significance of the epipolar line; (ii) in correspondence, when two sets of fringes with different fringe orientations are projected, the highest correspondence precision can be achieved with arbitrary orientations as long as these two orientations are perpendicular to each other; (iii) in reconstruction, a higher reconstruction precision is given by the 4-equation methods, while we notice that the 3-equation methods are almost dominatingly used in literature. Based on these theoretical results, we propose a novel FPP measurement method which (i) only projects one set of fringes with optimal fringe angle to explicitly work together with the epipolar line for precise pixel correspondence; (ii) for the first time, the optimal fringe angle is determined directly from the calibration parameters, instead of being measured; (iii) uses 4 equations for precise 3D reconstruction but we can remove one equation which is equivalent to an epipolar line, making it the first algorithm that can use 3-equation solution to achieve 4-equation precision. Our method is efficient (only one set of fringe patterns is required in projection and the speed is doubled in reconstruction), precise (in both pixel correspondence and 3D reconstruction), and convenient (the computable optimal fringe angle and a closed-form 3-equation solution). We also believe that our work is insightful in revealing fundamental FPP properties, provides a more reasonable measurement for practice, and thus is beneficial to further FPP studies.

Phase-shifting algorithms with known and unknown phase shifts: comparison and hybrid.

The phase-shifting interferometry has been intensively studied for more than half a century, and is still actively investigated and improved for more demanding precision measurement requirements. A proper phase-shifting algorithm (PSA) for phase extraction should consider various error sources including (i) the phase-shift errors, (ii) the intensity harmonics, (iii) the non-uniform phase-shift distributions and (iv) the random additive intensity noise. Consequently, a large pool of PSAs has been developed, including those with known phase shifts (abbreviated as kPSA) and those with unknown phase shifts (abbreviated as uPSA). While numerous evaluation works have been done for the kPSAs, there are very few for the uPSAs, making the overall picture of the PSAs unclear. Specifically, there is a lack of (i) fringe pattern parameters' restriction analysis for the uPSAs and (ii) performance comparison within the uPSAs and between the uPSAs and the kPSAs. Thus, for the first time, we comprehensively evaluated the pre-requisites and performance of four representative uPSAs, the advanced iterative algorithm, the general iterative algorithm (GIA), the algorithm based on the principal component analysis and the algorithm based on VU factorization, and then compare the uPSAs with twelve benchmarking kPSAs. From this comparison, the demand for proper selection of a kPSA, and the restriction and attractive performance of the uPSAs are clearly depicted. Due to the outstanding performance of the GIA, a hybrid kPSA-GIA is proposed to boost the performance of a kPSA and relieve the fringe density restriction of the GIA.

Universal dwell time optimization for deterministic optics fabrication.

Computer-Controlled Optical Surfacing (CCOS) has been greatly developed and widely used for precision optical fabrication in the past three decades. It relies on robust dwell time solutions to determine how long the polishing tools must dwell at certain points over the surfaces to achieve the expected forms. However, as dwell time calculations are modeled as ill-posed deconvolution, it is always non-trivial to reach a reliable solution that 1) is non-negative, since CCOS systems are not capable of adding materials, 2) minimizes the residual in the clear aperture 3) minimizes the total dwell time to guarantee the stability and efficiency of CCOS processes, 4) can be flexibly adapted to different tool paths, 5) the parameter tuning of the algorithm is simple, and 6) the computational cost is reasonable. In this study, we propose a novel Universal Dwell time Optimization (UDO) model that universally satisfies these criteria. First, the matrix-based discretization of the convolutional polishing model is employed so that dwell time can be flexibly calculated for arbitrary dwell points. Second, UDO simplifies the inverse deconvolution as a forward scalar optimization for the first time, which drastically increases the solution stability and the computational efficiency. Finally, the dwell time solution is improved by a robust iterative refinement and a total dwell time reduction scheme. The superiority and general applicability of the proposed algorithm are verified on the simulations of different CCOS processes. A real application of UDO in improving a synchrotron X-ray mirror using Ion Beam Figuring (IBF) is then demonstrated. The simulation indicates that the estimated residual in the 92.3 mm × 15.7 mm CA can be reduced from 6.32 nm Root Mean Square (RMS) to 0.20 nm RMS in 3.37 min. After one IBF process, the measured residual in the CA converges to 0.19 nm RMS, which coincides with the simulation.

General iterative algorithm for phase-extraction from fringe patterns with random phase-shifts, intensity harmonics and non-uniform phase-shift distribution.

Advanced iterative algorithm (AIA) is a flexible and effective phase-shifting algorithm (PSA) which can extract phase from fringe patterns with random unknown phase-shifts, making it attractive in the scenarios where phase-shifts are unknown or not accurate. However, accuracy of AIA degrades when intensity harmonics and/or phase-shift non-uniformity are presented. To solve this problem, multiple PSAs have been proposed, but they restrict their fringe model in one way or another, and thus sacrifice the immunity to certain error source(s). In this paper, a general iterative algorithm (GIA) which adopts a most general fringe model is proposed. In GIA, the many unknowns in the fringe pattern model are divided into three groups including: (i) the fringe amplitudes, (ii) the phase and (iii) the phase-shifts related parameters, and alternatively optimized through univariate search technique group by group to improve accuracy and convergence. The Levenberg-Marquart method is used for the optimization of each group of unknowns due to its excellent accuracy and robustness. GIA is shown to have better accuracies than all of its relevant competitors through both a large number of simulations as well as real experiments with a Fizeau interferometer.

Single-shot 3D shape measurement using an end-to-end stereo matching network for speckle projection profilometry.

Speckle projection profilometry (SPP), which establishes the global correspondences between stereo images by projecting only a single speckle pattern, has the advantage of single-shot 3D reconstruction. Nevertheless, SPP suffers from the low matching accuracy of traditional stereo matching algorithms, which fundamentally limits its 3D measurement accuracy. In this work, we propose a single-shot 3D shape measurement method using an end-to-end stereo matching network for SPP. To build a high-quality SPP dataset for training the network, by combining phase-shifting profilometry (PSP) and temporal phase unwrapping techniques, high-precision absolute phase maps can be obtained to generate accurate and dense disparity maps with high completeness as the ground truth by phase matching. For the architecture of the network, a multi-scale residual subnetwork is first leveraged to synchronously extract compact feature tensors with 1/4 resolution from speckle images for constructing the 4D cost volume. Considering that the cost filtering based on 3D convolution is computationally costly, a lightweight 3D U-net network is proposed to implement efficient 4D cost aggregation. In addition, because the disparity maps in the SPP dataset should have valid values only in the foreground, a simple and fast saliency detection network is integrated to avoid predicting the invalid pixels in the occlusions and background regions, thereby implicitly enhancing the matching accuracy for valid pixels. Experiment results demonstrated that the proposed method improves the matching accuracy by about 50% significantly compared with traditional stereo matching methods. Consequently, our method achieves fast and absolute 3D shape measurement with an accuracy of about 100µm through a single speckle pattern.

RISE: robust iterative surface extension for sub-nanometer X-ray mirror fabrication.

Precision optics have been widely required in many advanced technological applications. X-ray mirrors, as an example, serve as the key optical components at synchrotron radiation and free electron laser facilities. They are rectangular silicon or glass substrates where a rectangular Clear Aperture (CA) needs to be polished to sub-nanometer Root Mean Squared (RMS) to keep the imaging capability of the incoming X-ray wavefront at the diffraction limit. The convolutional polishing model requires a CA to be extended with extra data, from which the dwell time is calculated via deconvolution. However, since deconvolution is very sensitive to boundary errors and noise, the existing surface extension methods can hardly fulfill the sub-nanometer requirement. On one hand, the figure errors in a CA were improperly modeled during the extension, leading to continuity issues along the boundary. On the other hand, uncorrectable high-frequency errors and noise were also extended. In this study, we propose a novel Robust Iterative Surface Extension (RISE) method that resolves these problems with a data fitting strategy. RISE models the figure errors in a CA with orthogonal polynomials and ensures that only correctable errors are fit and extended. Combined with boundary conditions, an iterative refinement of dwell time is then proposed to compensate the errors brought by the extension and deconvolution, which drastically reduces the estimated figure error residuals in a CA while the increase of total dwell time is negligible. To our best knowledge, RISE is the first data fitting-based surface extension method and is the first to optimize dwell time based on iterative extension. An experimental verification of RISE is given by fabricating two elliptic cylinders (10 mm × 80 mm CAs) starting from a sphere with a radius of curvature around 173 m using ion beam figuring. The figure errors in the two CAs greatly improved from 204.96 nm RMS and 190.28 nm RMS to 0.62 nm RMS and 0.71 nm RMS, respectively, which proves that RISE is an effective method for sub-nanometer level X-ray mirror fabrication.

Spatial pattern-shifting method for complete two-wavelength fringe projection profilometry: erratum.

We present an erratum and generalization to our Letter [Opt. Lett.45, 3115 (2020)OPLEDP0146-959210.1364/OL.392102]. This erratum corrects an error in Eq. (12), and the generalization converts Rh to kh for more general situations of wavelengths. Neither has any influence on the conclusions of the original Letter.

Y4-Net: a deep learning solution to one-shot dual-wavelength digital holographic reconstruction.

In this Letter, a deep learning solution (Y4-Net, four output channels network) to one-shot dual-wavelength digital holography is proposed to simultaneously reconstruct the complex amplitude information of both wavelengths from a single digital hologram with high efficiency. In the meantime, by using single-wavelength results as network ground truth to train the Y4-Net, the challenging spectral overlapping problem in common-path situations is solved with high accuracy.

Spatial pattern-shifting method for complete two-wavelength fringe projection profilometry.

Two-wavelength fringe projection profilometry (FPP) unwraps a phase with the unambiguous phase range (UPR) of the least common multiple (LCM) of the two wavelengths. It is accurate, convenient, and robust, and thus plays an important role in shape measurement. However, when two non-coprime wavelengths are used, only a small UPR can be generated, and the unwrapping performance is compromised. In this Letter, a spatial pattern-shifting method (SPSM) is proposed to generate the maximum UPR (i.e., the product of the two wavelengths) from two non-coprime wavelengths. For the first time, to the best of our knowledge, the SPSM breaks the constraint of wavelength selection and enables a complete (i.e., either coprime or non-coprime) two-wavelength FPP. The SPSM, on the other hand, only requires spatially shift of the low-frequency pattern with the designed amounts and accordingly adjusting the fringe order determination, which is extremely convenient in implementation. Both numerical and experimental analyses verify its flexibility and correctness.

Advanced iterative algorithm for phase extraction: performance evaluation and enhancement.

Phase-shifting techniques are extremely important in modern optical metrology. The advanced iterative algorithm (AIA) is an elegant, flexible and effective phase-shifting algorithm that can extract phase from fringe patterns with arbitrary unknown phase-shifts. However, comparing it with traditional phase-shifting algorithms, AIA has not been sufficiently investigated on (i) its applicability to different types of fringe patterns; (ii) its performance with respect to different phase-shifts, frame numbers and noise levels and thus the possibility of further improvement; and (iii) the predictability of its accuracy. To solve these problems, a series of innovations are proposed in this paper. First, condition numbers are introduced to characterize the least squares matrices used in AIA, and subsequently a fringe density requirement is suggested for the success of AIA. Second, the performance of AIA regarding different phase-shifts, frame numbers and noise levels is thoroughly evaluated by simulations, based on which, an overall phase error model is established. With such understanding, three individual improvements of AIA, i.e., controlling phase-shifts, controlling frame numbers and suppressing noise, are proposed for better performance of AIA. Third, practical methods for estimating the overall phase errors are developed to make the AIA performance predictable even before AIA is executed. We then integrate all these three innovations into an enhanced AIA (eAIA), which solves all the problems we mentioned earlier. The significant contributions of eAIA include the insurability of the convergence, the controllability of the performance, and achievability of a desired accuracy. An experiment is carried out to demonstrate the effectiveness of eAIA.

Y-Net: a one-to-two deep learning framework for digital holographic reconstruction.

In this Letter, for the first time, to the best of our knowledge, we propose a digital holographic reconstruction method with a one-to-two deep learning framework (Y-Net). Perfectly fitting the holographic reconstruction process, the Y-Net can simultaneously reconstruct intensity and phase information from a single digital hologram. As a result, this compact network with reduced parameters brings higher performance than typical network variants. The experimental results of the mouse phagocytes demonstrate the advantages of the proposed Y-Net.

One-step robust deep learning phase unwrapping.

Phase unwrapping is an important but challenging issue in phase measurement. Even with the research efforts of a few decades, unfortunately, the problem remains not well solved, especially when heavy noise and aliasing (undersampling) are present. We propose a database generation method for phase-type objects and a one-step deep learning phase unwrapping method. With a trained deep neural network, the unseen phase fields of living mouse osteoblasts and dynamic candle flame are successfully unwrapped, demonstrating that the complicated nonlinear phase unwrapping task can be directly fulfilled in one step by a single deep neural network. Excellent anti-noise and anti-aliasing performances outperforming classical methods are highlighted in this paper.

Windowed Fourier ridges for demodulation of carrier fringe patterns with nonlinearity: a theoretical analysis.

Accurately extracting phase or phase derivative is the most important requirement in optical metrology. However, in practice, there are many error sources, among which nonlinear distortion in fringe patterns is often encountered. Several techniques have been proposed over time to remove the nonlinearity error. Among these techniques, the windowed Fourier ridges (WFR) algorithm has been shown to be an effective solution insensitive to nonlinearity, but it lacks a theoretical justification. In this paper, we theoretically analyze the local frequency estimation error and phase extraction error, which not only proves the mentioned insensitivity, but also supports the performance prediction and error control, and thus is very important and useful in optical measurement. The theoretical results have been verified by computer simulations. Other error sources such as model error and noise are also compared and discussed.

Comparative study of sampling moiré and windowed Fourier transform techniques for demodulation of a single-fringe pattern.

Phase measurement techniques using a single-shot carrier fringe pattern play an important role in optical science and technology and have been widely used for various applications. In this paper, we focus on the comparative study of two major fringe analysis techniques, the sampling moiré (SM) and the windowed Fourier transform (WFT). While SM converts a single-fringe pattern to multiple phase-shifted moiré fringe patterns to extract the phase information in the spatial domain, WFT obtains the phase information in the windowed Fourier domain; thus, the two methods look entirely different. We evaluate the phase extraction errors of SM and windowed Fourier ridges (WFRs) as a typical WFT method for both linear and nonlinear phases with/without noise against the reference Fourier transform (FT) technique. For the simulated fringe patterns with linear or nonlinear phase and different random noise level, all the methods have high phase extraction accuracies. For a real experiment with more complicated phase and discontinuities, SM and WFR, both local methods, yield quite similar results and outperform FT.

Real-time reference-based dynamic phase retrieval algorithm for optical measurement.

To study dynamic behaviors of a phenomenon, measuring the evolving field of a specimen/material/structure is required. Optical interferometry, as a full-field, non-contact, and highly sensitive optical measurement technique, has been applied, where the evolving field is represented as dynamic phase distribution. A dynamic phase retrieval algorithm, called least-squares with 3 unknowns (LS3U), which estimates the phase change between each two consecutive patterns by a least-squares fitting method and denoises the phase change by a windowed Fourier filtering (WFF) algorithm, has been shown to be a simple yet effective algorithm. However, LS3U is computationally expensive, restricting its potential application in real-time dynamic phase retrieval systems. In this paper, a real-time LS3U algorithm powered by GPU parallel computing is proposed, with which frame rates of up to 64.5 frames per second (fps) and 131.8 fps are achieved on NVIDIA's GTX 680 and GTX 1080 graphics cards, respectively.

Automatic fringe enhancement with novel bidimensional sinusoids-assisted empirical mode decomposition.

Fringe-based optical measurement techniques require reliable fringe analysis methods, where empirical mode decomposition (EMD) is an outstanding one due to its ability of analyzing complex signals and the merit of being data-driven. However, two challenging issues hinder the application of EMD in practical measurement. One is the tricky mode mixing problem (MMP), making the decomposed intrinsic mode functions (IMFs) have equivocal physical meaning; the other is the automatic and accurate extraction of the sinusoidal fringe from the IMFs when unpredictable and unavoidable background and noise exist in real measurements. Accordingly, in this paper, a novel bidimensional sinusoids-assisted EMD (BSEMD) is proposed to decompose a fringe pattern into mono-component bidimensional IMFs (BIMFs), with the MMP solved; properties of the resulted BIMFs are then analyzed to recognize and enhance the useful fringe component. The decomposition and the fringe recognition are integrated and the latter provides a feedback to the former, helping to automatically stop the decomposition to make the algorithm simpler and more reliable. A series of experiments show that the proposed method is accurate, efficient and robust to various fringe patterns even with poor quality, rendering it a potential tool for practical use.

Ternary Gray code-based phase unwrapping for 3D measurement using binary patterns with projector defocusing.

The three-dimensional measurement technique using binary pattern projection with projector defocusing has become increasingly important due to its high speed and high accuracy. To obtain even faster speed without sacrificing accuracy, a ternary Gray code-based phase-unwrapping method is proposed by using even fewer binary patterns, which makes it possible to efficiently and accurately unwrap the phase. Theoretical analysis, simulations, and experiments are presented to validate the proposed method's efficiency and robustness.

Phase-shifting profilometry combined with Gray-code patterns projection: unwrapping error removal by an adaptive median filter.

Phase-shifting profilometry combined with Gray-code patterns projection has been widely used for 3D measurement. In this technique, a phase-shifting algorithm is used to calculate the wrapped phase, and a set of Gray-code binary patterns is used to determine the unwrapped phase. In the real measurement, the captured Gray-code patterns are no longer binary, resulting in phase unwrapping errors at a large number of erroneous pixels. Although this problem has been attended and well resolved by a few methods, it remains challenging when a measured object has step-heights and the captured patterns contain invalid pixels. To effectively remove unwrapping errors and simultaneously preserve step-heights, in this paper, an effective method using an adaptive median filter is proposed. Both simulations and experiments can demonstrate its effectiveness.

Phase error analysis and compensation for phase shifting profilometry with projector defocusing.

Phase shifting profilometry (PSP) using binary fringe patterns with projector defocusing is promising for high-speed 3D shape measurement. To obtain a high-quality phase, the projector usually requires a high defocusing level, which leads to a drastic fall in fringe contrast. Due to its convenience and high speed, PSP using squared binary patterns with small phase shifting algorithms and slight defocusing is highly desirable. In this paper, the phase accuracies of the classical phase shifting algorithms are analyzed theoretically, and then compared using both simulation and experiment. We also adapt two algorithms for PSP using squared binary patterns, which include a Hilbert three-step PSP and a double three-step PSP. Both algorithms can increase phase accuracy, with the latter featuring additional invalid point detection. The adapted algorithms are also compared with the classical algorithms. Based on our analysis and comparison results, proper algorithm selection can be easily made according to the practical requirement.