Robust active stereo vision using Kullback-Leibler divergence.

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.

Period coded phase shifting strategy for real-time 3-D structured light illumination.

Phase shifting structured light illumination for range sensing involves projecting a set of grating patterns where accuracy is determined, in part, by the number of stripes. However, high pattern frequencies introduce ambiguities during phase unwrapping. This paper proposes a process for embedding a period cue into the projected pattern set without reducing the signal-to-noise ratio. As a result, each period of the high frequency signal can be identified. The proposed method can unwrap high frequency phase and achieve high measurement precision without increasing the pattern number. Therefore, the proposed method can significantly benefit real-time applications. The method is verified by theoretical and experimental analysis using prototype system built to achieve 120 fps at 640 × 480 resolution.

Multifeature distortion-insensitive constellation detection.

Many applications require detection of multiple features that locally remain consistent in shape and intensity characteristics, but may globally change position with respect to one another over time or under different circumstances. We refer to these feature sets, defined by their characteristic relative positioning, as multifeature constellations. We introduce a method of processing in which multiple levels of correlation, using specially designed composite feature detection filters, are used to first detect local features, and then to detect constellations of these local features. We include experimental procedures and results indicating how the use of multifeature constellation detection may be utilized in applications such as sign language recognition and fingerprint matching.

Maximum SNR pattern strategy for phase shifting methods in structured light illumination.

Structured light illumination by means of phase shifting patterns is a widely employed method for three-dimensional (3-D) image acquisition that is robust to ambient light and object albedo but may be especially susceptible to sensor and environment noise. In this paper, we study the specific technique of phase measuring profilometry (PMP) and the maximization of a pattern's signal to noise ratio (SNR). By treating the design of an N-pattern PMP process as placing points in an N-dimensional coding space, we define a pattern's SNR in terms of a pattern set's computational length and the number of coded phase periods in the projected patterns. Then, without introducing phase ambiguities, we propose a so-called edge pattern strategy that maximizes the computational length and number of periods. Theoretically, the edge pattern technique improves the SNR by 1.2381 times when using three component patterns and by 15.5421 times when using five patterns. Experimental results further demonstrate the improved SNR of the proposed edge pattern technique such that more accurate 3-D results are achieved using fewer component patterns.

Real-time three-dimensional shape measurement of moving objects without edge errors by time-synchronized structured illumination.

Structured-light illumination is a process of three-dimensional imaging where a series of time-multiplexed, striped patterns are projected onto a target scene with the corresponding captured images used to determine surface shape according to the warping of the projected patterns around the target. In a real-time system, a high-speed projector/camera pair is used such that any surface motion is small over the projected pattern sequence, but regardless of acquisition speed, there are always those pixels near the edge of a moving surface that capture the projected patterns on both fore- and background surfaces. These edge pixels then create unpredictable results that typically require expensive processing steps to remove, but in this Letter, we introduce a filtering process that identifies motion artifacts based upon the discrete Fourier transform applied to the time axis of the captured pattern sequence. The process is of very low computational complexity, and in this Letter, we demonstrate that in a real-time structured-light illumination (SLI) system, the process comes at a cost of 15 frames per second (fps), where our SLI system drops from 180 to 165 fps after deleting those edge pixels where motion was detected.

Dual-frequency pattern scheme for high-speed 3-D shape measurement.

A novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information, and the unit-frequency component is used to reduce phase unwrapping ambiguities. With our proposed pattern scheme, phase unwrapping can overcome the major shortcomings of conventional spatial phase unwrapping: phase jumping and discontinuities. Compared with conventional temporal phase unwrapped approaches, the proposed pattern scheme can achieve higher quality phase data using a less number of patterns. To process data in real time, we also propose and develop look-up table based fast and accurate algorithms for phase generation and 3-D reconstruction. Those fast algorithms can be applied to our pattern scheme as well as traditional phase measuring profilometry. For a 640 x 480 video stream, we can generate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 8.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies.

Gamma model and its analysis for phase measuring profilometry.

Phase measuring profilometry is a method of structured light illumination whose three-dimensional reconstructions are susceptible to error from nonunitary gamma in the associated optical devices. While the effects of this distortion diminish with an increasing number of employed phase-shifted patterns, gamma distortion may be unavoidable in real-time systems where the number of projected patterns is limited by the presence of target motion. A mathematical model is developed for predicting the effects of nonunitary gamma on phase measuring profilometry, while also introducing an accurate gamma calibration method and two strategies for minimizing gamma's effect on phase determination. These phase correction strategies include phase corrections with and without gamma calibration. With the reduction in noise, for three-step phase measuring profilometry, analysis of the root mean squared error of the corrected phase will show a 60x reduction in phase error when the proposed gamma calibration is performed versus 33x reduction without calibration.

Fit-sphere unwrapping and performance analysis of 3D fingerprints.

To solve problems associated with conventional 2D fingerprint acquisition processes including skin deformations and print smearing, we developed a noncontact 3D fingerprint scanner employing structured light illumination that, in order to be backwards compatible with existing 2D fingerprint recognition systems, requires a method of unwrapping the 3D scans into 2D equivalent prints. For the latter purpose of virtually flattening a 3D print, this paper introduces a fit-sphere unwrapping algorithm. Taking advantage of detailed 3D information, the proposed method defuses the unwrapping distortion by controlling the distances between neighboring points. Experimental results will demonstrate the high quality and recognition performance of the 3D unwrapped prints versus traditionally collected 2D prints. Furthermore, by classifying the 3D database into high- and low-quality data sets, we demonstrate that the relationship between quality and recognition performance holding for conventional 2D prints is achieved for 3D unwrapped fingerprints.

Pose and position tracking with super image vector inner products.

We introduce a new and efficient distortion-invariant super image tracker and pose estimator based on a linear phase coefficient composite filter. The super image consists of a weighted sum of training images chosen to span the distortion range under analysis. Unlike correlation-based composite filter design, the super image is implemented by means of a complex vector inner product operation. A super image vector inner product is implemented by elementwise multiplication of a super image template by a window of interest in the input scene and summation of the elementwise operations. The resulting amplitude indicates target detection, and the resulting phase indicates the value of scale, orientation, or movement of the target object. The mathematical characteristics of the super image vector inner product are presented, and its application is demonstrated.

Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity.

Temporal frame-to-frame noise in multipattern structured light projection can significantly corrupt depth measurement repeatability. We present a rigorous stochastic analysis of phase-measuring-profilometry temporal noise as a function of the pattern parameters and the reconstruction coefficients. The analysis is used to optimize the two-frequency phase measurement technique. In phase-measuring profilometry, a sequence of phase-shifted sine-wave patterns is projected onto a surface. In two-frequency phase measurement, two sets of pattern sequences are used. The first, low-frequency set establishes a nonambiguous depth estimate, and the second, high-frequency set is unwrapped, based on the low-frequency estimate, to obtain an accurate depth estimate. If the second frequency is too low, then depth error is caused directly by temporal noise in the phase measurement. If the second frequency is too high, temporal noise triggers ambiguous unwrapping, resulting in depth measurement error. We present a solution for finding the second frequency, where intensity noise variance is at its minimum.