A Visual Positioning Method of UAV in a Large-Scale Outdoor Environment.

Visual positioning is a basic component for UAV operation. The structure-based methods are, widely applied in most literature, based on local feature matching between a query image that needs to be localized and a reference image with a known pose and feature points. However, the existing methods still struggle with the different illumination and seasonal changes. In outdoor regions, the feature points and descriptors are similar, and the number of mismatches will increase rapidly, leading to the visual positioning becoming unreliable. Moreover, with the database growing, the image retrieval and feature matching are time-consuming. Therefore, in this paper, we propose a novel hierarchical visual positioning method, which includes map construction, landmark matching and pose calculation. First, we combine brain-inspired mechanisms and landmarks to construct a cognitive map, which can make image retrieval efficient. Second, the graph neural network is utilized to learn the inner relations of the feature points. To improve matching accuracy, the network uses the semantic confidence in matching score calculations. Besides, the system can eliminate the mismatches by analyzing all the matching results in the same landmark. Finally, we calculate the pose by using a PnP solver. Furthermore, we evaluate both the matching algorithm and the visual positioning method experimentally in the simulation datasets, where the matching algorithm performs better in some scenes. The results demonstrate that the retrieval time can be shortened by three-thirds with an average positioning error of 10.8 m.

Line-Features-Based Pose Estimation Method for the Disc Cutter Holder of Shield Machine.

To achieve automatic disc cutter replacement of shield machines, measuring the accurate pose of the disc cutter holder by machine vision is crucial. However, under polluted and restricted illumination conditions, achieving pose estimation by vision is a great challenge. This paper proposes a line-features-based pose estimation method for the disc cutter holder of the shield machine by using a monocular camera. For the blurring effect of rounded corners on the image edge, a rounded edge model is established to obtain edge points that better match the 3D model of the workpiece. To obtain the edge search box corresponding to each edge, a contour separation method based on an adaptive threshold region growing method is proposed. By preprocesses on the edge points of each edge, the efficiency and the accuracy of RANSAC linear fitting are improved. The experimental result shows that the proposed pose estimation method is highly reliable and can meet the measurement accuracy requirements in practical engineering applications.

The Method of Creel Positioning Based on Monocular Vision.

Automatic replacement of bobbins on the frame is one of the core problems that must be solved in the textile industry. In the process of changing the bobbin, it is faced with problems such as the position offset of the bobbin, the loosening and deformation of the bobbin, which will lead to the failure of replacing the bobbin. Therefore, it is necessary to initialize the creel coordinates regularly, also considering the labor intensity and poor reliability of manual positioning. This paper proposes an automatic creel positioning method based on monocular vision. Firstly, the industrial camera was installed on the drum changing manipulator controlled by the truss system, and each yarn frame in the yarn area was inspected. Secondly, the end face image of the creel was collected at a reasonable distance, and the collected images were transmitted to the computer in real time through the Ethernet bus. Thirdly, the center coordinates (x, y) and radius r of the creel were marked by the improved Hough circle detection algorithm. Finally, the coordinate deviation of the creel was calculated and transmitted to the system controller to realize deviation correction. Before the test, the creel positioning markers were specially designed and the camera was calibrated. Thus, the influence of image complex background, creel end roughness, reflection and other factors can be reduced, and the image processing speed and positioning accuracy can be improved. The results show that the positioning effect of this method is fine when the distance between the center of the camera lens and the center of the end face of the creel is 170~190 mm. Moreover, when the distance is 190 mm, the positioning effect is the best, with an average error of only 0.51 mm. In addition, the deviation between the center coordinate and the radius of the end face of the marker is also very small, which is better than the requirements of bobbin yarn changing accuracy.

Power Tower Inspection Simultaneous Localization and Mapping: A Monocular Semantic Positioning Approach for UAV Transmission Tower Inspection.

Realizing autonomous unmanned aerial vehicle (UAV) inspection is of great significance for power line maintenance. This paper introduces a scheme of using the structure of a tower to realize visual geographical positioning of UAV for tower inspection and presents a monocular semantic simultaneous localization and mapping (SLAM) framework termed PTI-SLAM (power tower inspection SLAM) to cope with the challenge of a tower inspection scene. The proposed scheme utilizes prior knowledge of tower component geolocation and regards geographical positioning as the estimation of transformation between SLAM and the geographic coordinates. To accomplish the robust positioning and semi-dense semantic mapping with limited computing power, PTI-SLAM combines the feature-based SLAM method with a fusion-based direct method and conveys a loosely coupled architecture of a semantic task and a SLAM task. The fusion-based direct method is specially designed to overcome the fragility of the direct method against adverse conditions concerning the inspection scene. Experiment results show that PTI-SLAM inherits the robustness advantage of the feature-based method and the semi-dense mapping ability of the direct method and achieves decimeter-level real-time positioning in the airborne system. The experiment concerning geographical positioning indicates more competitive accuracy compared to the previous visual approach and artificial UAV operating, demonstrating the potential of PTI-SLAM.

Extraction and Research of Crop Feature Points Based on Computer Vision.

Based on computer vision technology, this paper proposes a method for identifying and locating crops in order to successfully capture crops in the process of automatic crop picking. This method innovatively combines the YOLOv3 algorithm under the DarkNet framework with the point cloud image coordinate matching method, and can achieve the goal of this paper very well. Firstly, RGB (RGB is the color representing the three channels of red, green and blue) images and depth images are obtained by using the Kinect v2 depth camera. Secondly, the YOLOv3 algorithm is used to identify the various types of target crops in the RGB images, and the feature points of the target crops are determined. Finally, the 3D coordinates of the feature points are displayed on the point cloud images. Compared with other methods, this method of crop identification has high accuracy and small positioning error, which lays a good foundation for the subsequent harvesting of crops using mechanical arms. In summary, the method used in this paper can be considered effective.

Accurate Smartphone Indoor Visual Positioning Based on a High-Precision 3D Photorealistic Map.

Indoor positioning is in high demand in a variety of applications, and indoor environment is a challenging scene for visual positioning. This paper proposes an accurate visual positioning method for smartphones. The proposed method includes three procedures. First, an indoor high-precision 3D photorealistic map is produced using a mobile mapping system, and the intrinsic and extrinsic parameters of the images are obtained from the mapping result. A point cloud is calculated using feature matching and multi-view forward intersection. Second, top-K similar images are queried using hamming embedding with SIFT feature description. Feature matching and pose voting are used to select correctly matched image, and the relationship between image points and 3D points is obtained. Finally, outlier points are removed using P3P with the coarse focal length. Perspective-four-point with unknown focal length and random sample consensus are used to calculate the intrinsic and extrinsic parameters of the query image and then to obtain the positioning of the smartphone. Compared with established baseline methods, the proposed method is more accurate and reliable. The experiment results show that 70 percent of the images achieve location error smaller than 0.9 m in a 10 m × 15.8 m room, and the prospect of improvement is discussed.

Visual Positioning Indoors: Human Eyes vs. Smartphone Cameras.

Artificial Intelligence (AI) technologies and their related applications are now developing at a rapid pace. Indoor positioning will be one of the core technologies that enable AI applications because people spend 80% of their time indoors. Humans can locate themselves related to a visually well-defined object, e.g., a door, based on their visual observations. Can a smartphone camera do a similar job when it points to an object? In this paper, a visual positioning solution was developed based on a single image captured from a smartphone camera pointing to a well-defined object. The smartphone camera simulates the process of human eyes for the purpose of relatively locating themselves against a well-defined object. Extensive experiments were conducted with five types of smartphones on three different indoor settings, including a meeting room, a library, and a reading room. Experimental results shown that the average positioning accuracy of the solution based on five smartphone cameras is 30.6 cm, while that for the human-observed solution with 300 samples from 10 different people is 73.1 cm.

Indoor Positioning on Smartphones Using Built-In Sensors and Visual Images.

With the rapid development of mobile Internet technology, localization using visual image information has become a hot problem in the field of indoor localization research, which is not affected by signal multipath and fading and can achieve high accuracy localization in indoor areas with complex electromagnetic environments. However, in practical applications, position estimation using visual images is easily influenced by the user's photo pose. In this paper, we propose a multiple-sensor-assisted visual localization method in which the method constructs a machine learning classifier using multiple smart sensors for pedestrian pose estimation, which improves the retrieval efficiency and localization accuracy. The method mainly combines the advantages of visual image location estimation and pedestrian pose estimation based on multiple smart sensors and considers the effect of pedestrian photographing poses on location estimation. The built-in sensors of smartphones are used as the source of pedestrian pose estimation data, which constitutes a feasible location estimation method based on visual information. Experimental results show that the method proposed in this paper has good localization accuracy and robustness. In addition, the experimental scene in this paper is a common indoor scene and the experimental device is a common smartphone. Therefore, we believe that the proposed method in this paper has the potential to be widely used in future indoor navigation applications in complex scenarios (e.g., mall navigation).

An Indoor Visual Positioning Method with 3D Coordinates Using Built-In Smartphone Sensors Based on Epipolar Geometry.

In the process of determining positioning point by constructing geometric relations on the basis of the positions and poses obtained from multiple pairs of epipolar geometry, the direction vectors will not converge due to the existence of mixed errors. The existing methods to calculate the coordinates of undetermined points directly map the three-dimensional direction vector to the two-dimensional plane and take the intersection points that may be at infinity as the positioning result. To end this, an indoor visual positioning method with three-dimensional coordinates using built-in smartphone sensors based on epipolar geometry is proposed, which transforms the positioning problem into solving the distance from one point to multiple lines in space. It combines the location information obtained by the accelerometer and magnetometer with visual computing to obtain more accurate coordinates. Experimental results show that this positioning method is not limited to a single feature extraction method when the source range of image retrieval results is poor. It can also achieve relatively stable localization results in different poses. Furthermore, 90% of the positioning errors are lower than 0.58 m, and the average positioning error is less than 0.3 m, meeting the accuracy requirements for user localization in practical applications at a low cost.

Indoor Passive Visual Positioning by CNN-Based Pedestrian Detection.

Indoor positioning applications are developing at a rapid pace; active visual positioning is one method that is applicable to mobile platforms. Other methods include Wi-Fi, CSI, and PDR approaches; however, their positioning accuracy usually cannot achieve the positioning performance of the active visual method. Active visual users, however, must take a photo to obtain location information, raising confidentiality and privacy issues. To address these concerns, we propose a solution for passive visual positioning based on pedestrian detection and projection transformation. This method consists of three steps: pretreatment, pedestrian detection, and pose estimation. Pretreatment includes camera calibration and camera installation. In pedestrian detection, features are extracted by deep convolutional neural networks using neighboring frame detection results and the map information as the region of interest attention model (RIAM). Pose estimation computes accurate localization results through projection transformation (PT). This system relies on security cameras installed in non-private areas so that pedestrians do not have to take photos. Experiments were conducted in a hall about 100 square meters in size, with 41 test-points for the localization experiment. The results show that the positioning error was 0.48 m (RMSE) and the 90% error was 0.73 m. Therefore, the proposed passive visual method delivers high positioning performance.