Event-based depth estimation with dense occlusion.

Occlusions pose a significant challenge to depth estimation in various fields, including automatic driving, remote sensing observation, and video surveillance. In this Letter, we propose a novel, to the best of our knowledge, depth estimation method for dense occlusion to estimate the depth behind occlusions. We design a comprehensive procedure using an event camera that consists of two steps: rough estimation and precise estimation. In the rough estimation, we reconstruct two segments of the event stream to remove occlusions and subsequently employ a binocular intersection measurement to estimate the rough depth. In the precise estimation, we propose a criterion that the maximum total length of edges of reconstructed images corresponds to the actual depth and search for the precise depth around the rough depth. The experimental results demonstrate that our method is implemented with relative errors of depth estimation below 1.05%.

Motion measurements of explosive shock waves based on an event camera.

Shock wave measurement is vital in assessing explosive power and designing warheads. To obtain satisfactory observation data of explosive shock waves, it is preferable for optical sensors to possess high-dynamic range and high-time resolution capabilities. In this paper, the event camera is first employed to observe explosive shock waves, leveraging its high dynamic range and low latency. A comprehensive procedure is devised to measure the motion parameters of shock waves accurately. Firstly, the plane lines-based calibration method is proposed to compute the calibration parameters of the event camera, which utilizes the edge-sensitive characteristic of the event camera. Then, the fitted ellipse parameters of the shock wave are estimated based on the concise event data, which are gained by utilizing the characteristics of the event triggering and shock waves' morphology. Finally, the geometric relationship between the ellipse parameters and the radius of the shock wave is derived, and the motion parameters of the shock wave are estimated. To verify the performance of our method, we compare our measurement results in the TNT explosion test with the pressure sensor results and empirical formula prediction. The relative measurement error compared to pressure sensors is the lowest at 0.33% and the highest at 7.58%. The experimental results verify the rationality and effectiveness of our methods.

Two large-exposure-ratio image fusion by improved morphological segmentation.

The fusion of two large-exposure-ratio images, especially in the rocket launch field, is a challenging task because of fast-moving objects and differential features from daily scenes. Based on the large-exposure-ratio images, we propose a principle of halo formation at the boundaries of over-exposed areas. To avoid the halos in the fusion images, an improved morphological segmentation (IMS) method is developed to segment the over-exposed regions and boundaries. The IMS method is inspired by the mountain topography and builds a bridge between the 3D patches and quadratic polynomial coefficients. An improved multiscale method with segmentation in high-exposed images is proposed. In the rocket launch observation experiment, we constructed a two-camera simultaneous imaging system to avoid the dynamic scenes. The result of our proposed fusion method could best preserve the details and colors of the flames in low-exposed images and has the best subjective observation. The objective matrices also demonstrate superior edge and contrast performances over mainstream methods.

Relative Pose Estimation With a Single Affine Correspondence.

In this article, we present four cases of minimal solutions for two-view relative pose estimation by exploiting the affine transformation between feature points, and we demonstrate efficient solvers for these cases. It is shown that under the planar motion assumption or with knowledge of a vertical direction, a single affine correspondence is sufficient to recover the relative camera pose. The four cases considered are two-view planar relative motion for calibrated cameras as a closed-form and least-squares solutions, a closed-form solution for unknown focal length, and the case of a known vertical direction. These algorithms can be used efficiently for outlier detection within a RANSAC loop and for initial motion estimation. All the methods are evaluated on both synthetic data and real-world datasets. The experimental results demonstrate that our methods outperform comparable state-of-the-art methods in accuracy with the benefit of a reduced number of needed RANSAC iterations. The source code is released at https://github.com/jizhaox/relative_pose_from_affine.

Rotation alignment of a camera-IMU system using a single affine correspondence.

We propose an accurate and easy-to-implement method on rotation alignment of a camera-inertial measurement unit (IMU) system using only a single affine correspondence in the minimal case. The known initial rotation angles between the camera and IMU are utilized; thus, the alignment model can be formulated as a polynomial equation system based on homography constraints by expressing the rotation matrix in a first-order approximation. By solving the equation system, we can recover the rotation alignment parameters. Furthermore, more accurate alignment results can be achieved with the joint optimization of multiple stereo image pairs. The proposed method does not require additional auxiliary equipment or a camera's particular motion. The experimental results on synthetic data and two real-world data sets demonstrate that our method is efficient and precise for the camera-IMU system's rotation alignment.

Self-calibration of cameras using affine correspondences and known relative rotation angle.

This paper proposes a flexible method for camera self-calibration using affine correspondences and known relative rotation angle, which applies to the case that camera and inertial measurement unit (IMU) are tightly fixed. An affine correspondence provides two more constraints for the self-calibration problem than a traditional point correspondence, and the relative rotation angle can be derived from the IMU. Therefore, calibrating intrinsic camera parameters needs fewer correspondences, which can reduce the iterations and improve the algorithm's robustness in the random sample consensus framework. The proposed method does not require rotational alignment between the camera and the IMU. This advantage makes our method more convenient and flexible. The experimental results of both synthetic data and publicly available real datasets demonstrate that our method is effective and accurate for camera self-calibration.

Reconstruction-based 6D pose estimation for robotic assembly.

Pose estimation is important for many robotic applications including bin picking and robotic assembly and collaboration. However, robust and accurate estimation of the poses of industrial objects is a challenging task owing to the various object shapes and complex working environments. This paper presents a method of estimating the poses of narrow and elongated industrial objects with a low-cost RGB-D (depth and color) camera to guide the process of robotic assembly. The proposed method comprises three main steps: reconstruction involved in preprocessing, pose initialization with geometric features, and tracking aided by contour cues. Pose tracking is coupled with real-time dense reconstruction, which can synthesize a smooth depth image as a substitute for the raw depth image. Because industrial objects (e.g., fork and adapter) feature mostly planar structures, primitive geometric features, such as three-dimensional planes, are extracted from the point cloud and utilized to induce a promising initial pose. For robust tracking of the adapter consisting of narrow and elongated planes, the dense surface correspondences are combined with sparse contour correspondences in the refinement scheme. This combination allows for a satisfactory tolerance to the initial guess in the pose tracking phase. The experimental results demonstrate the feasibility of the proposed method.

Self-calibration approach to stereo cameras with radial distortion based on epipolar constraint.

In this paper, we propose a self-calibration approach to stereo cameras with radial distortion from stereo image pairs of a common 3D scene. Based on the epipolar constraint in the stereo image pair, the intrinsic and extrinsic parameters of stereo cameras are estimated synchronously with a minimum number of nine image point correspondences. It is significant within a random sample consensus (RANSAC) scheme to cope with the outliers of feature matches efficiently and robustly. Then the inliers of the stereo image pair that have been determined after RANSAC are used to optimize the calibration parameters of stereo cameras. Furthermore, more accurate calibration results can be achieved with the joint optimization of multiple stereo image pairs. Both synthetic and real data are used to evaluate the performance of the proposed method, demonstrating that our method can calibrate stereo cameras with radial distortions efficiently and accurately.

Planar self-calibration for stereo cameras with radial distortion.

In this paper, we present a robust technique of stereo calibration using homography constraints. Our method is novel as stereo calibration is performed by solving a polynomial equation system including two radial distortion parameters, using a minimal number of five image point correspondences. This enables us to calibrate from only a pair of stereo images of a planar scene, and to provide the exact algebraic solution to the stereo calibration problem. The minimal case solution is useful to reduce the computation time and increase the calibration robustness when using random sample consensus (RANSAC) from the correspondences of the stereo image pair. Further, a non-linear parameter optimization for the intrinsic and extrinsic parameters of stereo cameras is performed using the inliers, which are determined after RANSAC. In addition, our method can achieve more robust calibration results with multiple stereo image pairs by performing joint optimization. In contrast to the previous stereo calibration methods, our method works without requiring any special hardware and has no problems with one stereo image pair, even corrupted by severe radial distortions. Finally, by evaluating our method on both synthetic and real scene data, we demonstrate that our method is both efficient and accurate for stereo calibration.

Automatic three-dimensional measurement of large-scale structure based on vision metrology.

All relevant key techniques involved in photogrammetric vision metrology for fully automatic 3D measurement of large-scale structure are studied. A new kind of coded target consisting of circular retroreflective discs is designed, and corresponding detection and recognition algorithms based on blob detection and clustering are presented. Then a three-stage strategy starting with view clustering is proposed to achieve automatic network orientation. As for matching of noncoded targets, the concept of matching path is proposed, and matches for each noncoded target are found by determination of the optimal matching path, based on a novel voting strategy, among all possible ones. Experiments on a fixed keel of airship have been conducted to verify the effectiveness and measuring accuracy of the proposed methods.