Object phase-valid region segmentation method for FPP-based three-dimensional measurement.

In most existing studies based on fringe projector profilometry (FPP), the whole scenario is reconstructed, or the ideal experimental settings are established to segment the object easily. However, in real industrial scenarios, automated object detection and segmentation are essential to perform object-level measurement. To address the problem, a dual-wavelet feature interaction network (DWFI-Net) is developed in this paper to perform object phase-valid region segmentation, where both the background and shadow are removed. In our work, the modulation and wrapped phase maps are considered as inputs innovatively. The modulation maps provide abundant structures and textures, while the wrapped phase maps complement and enhance shadows and edges. An adaptive wavelet feature interaction (AWFI) module is presented to learn and fuse the features, where discrete wavelet transformation (DWT) is applied to decompose the features. An edge-aware discrete cosine transformation (EDCT) module is developed as a decoder, where the discrete cosine transformation (DCT) is applied to interpret the fused features. Qualitative and quantitative experiments are performed to verify the superiority of our DWFI-Net and its effectiveness on object-level three-dimensional measurement based on FPP.

LiDAR-assisted accuracy improvement strategy for binocular visual measurement.

The measurement model of binocular vision is inaccurate when the measurement distance is much different from the calibration distance, which affects its practicality. To tackle this challenge, we proposed what we believe to be a novel LiDAR-assisted accuracy improvement strategy for binocular visual measurement. First, the 3D points cloud and 2D images were aligned by the Perspective-n-Point (PNP) algorithm to realize calibration between LiDAR and binocular camera. Then, we established a nonlinear optimization function and proposed a depth-optimization strategy to lessen the error of binocular depth. Finally, the size measurement model of binocular vision based on the optimized depth is built to verify the effectiveness of our strategy. The experimental results show that our strategy can improve the depth accuracy compared to three stereo matching methods. The mean error of binocular visual measurement decreased from 33.46% to 1.70% at different distances. This paper provides an effective strategy for improving the measurement accuracy of binocular vision at different distances.

Polarized reflection removal with difference feature attention guidance.

Reflection removal is of great significance for high-level computer vision tasks. Most existing methods separate reflections relying heavily on the quality of intermediate prediction or under certain special constraints. However, these methods ignore the inherent correlation between the background and reflection, which may lead to unsatisfactory results with undesired artifacts. Polarized images contain unique optical characteristics that can facilitate reflection removal. In this paper, we present, to the best of our knowledge, a novel two-stage polarized image reflection removal network with difference feature attention guidance. Specifically, our model takes multi-channel polarized images and Stokes parameters as input and utilizes the optical characteristics of reflected and transmitted light to alleviate the ill-posed nature. It adopts a simple yet effective two-stage structure that first predicts the reflection layer and then refines the transmission layer capitalizing on the special relationship between reflection and transmission light. The difference feature attention guidance module (DFAG) is elaborated to diminish the dependence on intermediate consequences and better suppress reflection. It mitigates the reflection components from the observation and generates the supplement and enhancement to the transmission features. Extensive experiments on the real-world polarized dataset demonstrate the superiority of our method in comparison to the state-of-the-art methods.

UAV system with trinocular vision for external obstacle detection of transmission lines.

External obstacle detection is a significant task in transmission line inspection and is related to the safe operation of the power transmission grid. In recent years, unmanned aerial vehicles (UAVs) equipped with different devices have been widely used for transmission line inspection. However, because of the complex environment of transmission lines and weak power line textures in the obtained images, most existing methods and systems cannot meet the requirements for real-time and high-accuracy external obstacle detection of transmission lines. In this paper, a novel, to the best of our knowledge, UAV system integrated trinocular vision technology with remote sensing is developed to achieve better external obstacle detection of transmission lines in real time, which is composed of a DJ-Innovations (DJI) UAV equipped with a global positioning system (GPS), angle sensors, trinocular vision including three visible cameras with the same parameters, and a small processor with a pre-implanted software algorithm. In this paper, a new method for external obstacle detection of transmission lines is proposed to satisfy the requirements for real-time and high-accuracy practical inspection applications. First, the original trinocular images need to be rectified. Then, the rectified trinocular images are adopted to achieve three-dimensional reconstruction of power lines. Finally, based on trinocular vision, bag of feature, and GPS, the clearance distance measurement, obstacle classification, and obstacle location are realized. Experimental tests on 220 kV transmission lines reveal that our proposed system can be applied in practical inspection environments and has good performance.

Trinocular vision and spatial prior based method for ground clearance measurement of transmission lines.

It is an essential task to inspect ground clearance of transmission lines in time. However, the weak texture of transmission lines and high complexity of the background make it difficult to balance efficiency and accuracy. To solve the problem, a trinocular vision and spatial prior based method is proposed, which is specifically designed for ground clearance measurement of transmission lines with unmanned aerial vehicles (UAVs). In this novel method, a perpendicular double-baseline trinocular vision module is applied to improve the accuracy of transmission line reconstruction. Then the spatial prior information of geometric models under different shooting attitudes is analyzed in detail, and it is adopted to determine the ground crossing points and compute ground clearance efficiently. Also, an interactive software is developed and tested in the simulation environment of UAV inspection. Experimental results verify the feasibility of the measurement method. Finally, we discuss in detail how to apply the method effectively in practice and give a set of recommended camera parameters.

Defect inspection for underwater structures based on line-structured light and binocular vision.

Surface defect inspection for underwater structures is important. However, the inspection technologies based on passive vision cannot meet accuracy requirements. In this paper, we propose a two-stage method based on structured light images for defect detection. In the first stage, light stripes are extracted based on the analysis of hue, saturation, value (HSV) space and gray space. Then a hole-filling method is applied to ensure stripe integrity. In the second stage, depth information for all light stripes is calculated to synthesize a depth map, which is segmented for defect localization and measurement. Experimental results have verified the feasibility and effectiveness of our method.

A dual-modal dynamic contour-based method for cervical vascular ultrasound image instance segmentation.

OBJECTIVES: We intend to develop a dual-modal dynamic contour-based instance segmentation method that is based on carotid artery and jugular vein ultrasound and its optical flow image, then we evaluate its performance in comparison with the classic single-modal deep learning networks. METHOD: We collected 2432 carotid artery and jugular vein ultrasound images and divided them into training, validation and test dataset by the ratio of 8:1:1. We then used these ultrasound images to generate optical flow images with clearly defined contours. We also proposed a dual-stream information fusion module to fuse complementary features between different levels extracted from ultrasound and optical flow images. In addition, we proposed a learnable contour initialization method that eliminated the need for manual design of the initial contour, facilitating the rapid regression of nodes on the contour to the ground truth points. RESULTS: We verified our method by using a self-built dataset of carotid artery and jugular vein ultrasound images. The quantitative metrics demonstrated a bounding box detection mean average precision of 0.814 and a mask segmentation mean average precision of 0.842. Qualitative analysis of our results showed that our method achieved smoother segmentation boundaries for blood vessels. CONCLUSIONS: The dual-modal network we proposed effectively utilizes the complementary features of ultrasound and optical flow images. Compared to traditional single-modal instance segmentation methods, our approach more accurately segments the carotid artery and jugular vein in ultrasound images, demonstrating its potential for reliable and precise medical image analysis.

3D reconstruction method based on second-order semiglobal stereo matching and fast point positioning Delaunay triangulation.

Binocular vision uses the parallax principle of the human eye to obtain 3D information of an object, which is widely used as an important means of acquiring 3D information for 3D reconstruction tasks. To improve the accuracy and efficiency of 3D reconstruction, we propose a 3D reconstruction method that combines second-order semiglobal matching, guided filtering and Delaunay triangulation. First, the existing second-order semiglobal matching method is improved, and the smoothness constraint of multiple angle directions is added to the matching cost to generate a more robust disparity map. Second, the 3D coordinates of all points are calculated by combining camera parameters and disparity maps to obtain the 3D point cloud, which is smoothed by guided filtering to remove noise points and retain details. Finally, a method to quickly locate the insertion point and accelerate Delaunay triangulation is proposed. The surface of the point cloud is reconstructed by Delaunay triangulation based on fast point positioning to improve the visibility of the 3D model. The proposed approach was evaluated using the Middlebury and KITTI datasets. The experimental results show that the proposed second-order semiglobal matching method has higher accuracy than other stereo matching methods and that the proposed Delaunay triangulation method based on fast point location requires less time than the original Delaunay triangulation.

Uncertainty Quantification Enforced Flash Radiography Reconstruction by Two-Level Efficient MCMC.

Flash Radiography inspections stand to gain from inversion to infer density distribution of object based on X-ray transmission image. It is indispensable to be able to reliably provide uncertainties associated with the inversions. Although many inversion algorithms have been devised, they often perform poorly due to either their sensitivity to regularization parameter chosen in variational optimization or prohibitive computation and noisy results in stochastic simulation. In this paper, we present a gradual reconstruction algorithm, called TLE-Gibbs (two-level efficient Gibbs sampling), for flash radiography. At its core, TLE-Gibbs is a stochastic approach based on efficient Gibbs sampling and reconstruction refinement. A two-level scheme is proposed that enables high-resolution image to be constrained with uncertainty estimation from high-level reconstruction. Furthermore, a splitting variant that increases flexibility and precision is considered in the two-level scheme. An efficient Markov chain Monte Carlo (MCMC) endowed with first-order truncated conjugate gradient (CG) optimizer is developed to achieve minimal cost per sample and to approximate the posterior distribution. Finally, we adopt an effective refinement method to remove noises remained in the sample meanwhile maintaining sharp edges. For performance evaluation, TLE-Gibbs is applied on both synthetic data in which the influence of system blur is specially investigated and real data, and comparison with state-of-the-art reconstruction methods demonstrates the superiority of the proposed method.

Novel coronavirus pneumonia detection and segmentation based on the deep-learning method.

BACKGROUND: Segmentation of coronavirus disease 2019 (COVID-19) lesions is a difficult task due to high uncertainty in the shape, size and location of the lesions. CT scan image is an important means of diagnosing COVID-19, but it requires doctors to observe a large number of scan images repeatedly to determine the patient's condition. Moreover, the low contrast of CT scan and the presence of tissues such as blood vessels in the background increase the difficulty of diagnosis. To solve this problem, we proposed an improved segmentation model called the residual attention U-shaped network (ResAU-Net). METHODS: A novel method to detect and segment coronavirus pneumonia was established based on the deep-learning algorithm. Firstly, the CT scan image was input, and lung segmentation was then realized by U-net. Then, the region of interest was selected by the minimum circumscribed rectangle clipping method. Finally, the proposed ResAU-Net, which includes attention module (AMB), residual module (RBM) and sub-pixel convolution module (SPCBM), was used to segment the infected area and generate the segmentation results. RESULTS: We evaluated our model using cross-validation on 100 chest CT scans test images. The experimental results showed that our method achieved start-of-the-art performance on the pneumonia dataset. The mIoU and Dice cofficients of Lesion segmentation were 73.40%±2.24% and 84.5%±2.46%, and realize fast real-time processing. CONCLUSIONS: Our model can effectively solve the problems of poor segmentation accuracy in the segmentation of COVID-19 lesions, and the segmentation result image can effectively assist medical staff in the diagnosis and quantitative analysis of infection degree, and improve the screening and diagnosis efficiency of pneumonia.

Multiple-Instance Discriminant Analysis for Weakly Supervised Segment Annotation.

In this paper, we propose a multiple-instance discriminant analysis algorithm for weakly supervised segment annotation. We introduce a selection parameter for each image/video with weak labels and expect that it can sift out object regions from the background clutter to train a better transformation vector. The selection parameter and the transformation parameter are incorporated into a single objective function and optimized in an alternate way. The optimization is an iteration between the eigenvalue decomposition and a set of quadratic programming. We also integrate a regularization term into the objective function to formulate the spatial constraint of segments, which is ignored in ordinary multiple-instance learning methods. The algorithm is able to overcome the limitations that arise when applying ordinary multiple-instance methods to the task. The experimental results validate the effectiveness of our method.

A Robust and Fast Method for Sidescan Sonar Image Segmentation Using Nonlocal Despeckling and Active Contour Model.

Sidescan sonar image segmentation is a very important issue in underwater object detection and recognition. In this paper, a robust and fast method for sidescan sonar image segmentation is proposed, which deals with both speckle noise and intensity inhomogeneity that may cause considerable difficulties in image segmentation. The proposed method integrates the nonlocal means-based speckle filtering (NLMSF), coarse segmentation using k -means clustering, and fine segmentation using an improved region-scalable fitting (RSF) model. The NLMSF is used before the segmentation to effectively remove speckle noise while preserving meaningful details such as edges and fine features, which can make the segmentation easier and more accurate. After despeckling, a coarse segmentation is obtained by using k -means clustering, which can reduce the number of iterations. In the fine segmentation, to better deal with possible intensity inhomogeneity, an edge-driven constraint is combined with the RSF model, which can not only accelerate the convergence speed but also avoid trapping into local minima. The proposed method has been successfully applied to both noisy and inhomogeneous sonar images. Experimental and comparative results on real and synthetic sonar images demonstrate that the proposed method is robust against noise and intensity inhomogeneity, and is also fast and accurate.

Image Classification Using Biomimetic Pattern Recognition with Convolutional Neural Networks Features.

As a typical deep-learning model, Convolutional Neural Networks (CNNs) can be exploited to automatically extract features from images using the hierarchical structure inspired by mammalian visual system. For image classification tasks, traditional CNN models employ the softmax function for classification. However, owing to the limited capacity of the softmax function, there are some shortcomings of traditional CNN models in image classification. To deal with this problem, a new method combining Biomimetic Pattern Recognition (BPR) with CNNs is proposed for image classification. BPR performs class recognition by a union of geometrical cover sets in a high-dimensional feature space and therefore can overcome some disadvantages of traditional pattern recognition. The proposed method is evaluated on three famous image classification benchmarks, that is, MNIST, AR, and CIFAR-10. The classification accuracies of the proposed method for the three datasets are 99.01%, 98.40%, and 87.11%, respectively, which are much higher in comparison with the other four methods in most cases.

Bionic Vision-Based Intelligent Power Line Inspection System.

Detecting the threats of the external obstacles to the power lines can ensure the stability of the power system. Inspired by the attention mechanism and binocular vision of human visual system, an intelligent power line inspection system is presented in this paper. Human visual attention mechanism in this intelligent inspection system is used to detect and track power lines in image sequences according to the shape information of power lines, and the binocular visual model is used to calculate the 3D coordinate information of obstacles and power lines. In order to improve the real time and accuracy of the system, we propose a new matching strategy based on the traditional SURF algorithm. The experimental results show that the system is able to accurately locate the position of the obstacles around power lines automatically, and the designed power line inspection system is effective in complex backgrounds, and there are no missing detection instances under different conditions.

Human Action Recognition Using Improved Salient Dense Trajectories.

Human action recognition in videos is a topic of active research in computer vision. Dense trajectory (DT) features were shown to be efficient for representing videos in state-of-the-art approaches. In this paper, we present a more effective approach of video representation using improved salient dense trajectories: first, detecting the motion salient region and extracting the dense trajectories by tracking interest points in each spatial scale separately and then refining the dense trajectories via the analysis of the motion saliency. Then, we compute several descriptors (i.e., trajectory displacement, HOG, HOF, and MBH) in the spatiotemporal volume aligned with the trajectories. Finally, in order to represent the videos better, we optimize the framework of bag-of-words according to the motion salient intensity distribution and the idea of sparse coefficient reconstruction. Our architecture is trained and evaluated on the four standard video actions datasets of KTH, UCF sports, HMDB51, and UCF50, and the experimental results show that our approach performs competitively comparing with the state-of-the-art results.