Enhanced Path Planning and Obstacle Avoidance Based on High-Precision Mapping and Positioning.

High-precision positioning and multi-target detection have been proposed as key technologies for robotic path planning and obstacle avoidance. First, the Cartographer algorithm was used to generate high-quality maps. Then, the iterative nearest point (ICP) and the occupation probability algorithms were combined to scan and match the local point cloud, and the positions and attitudes of the robot were obtained. Furthermore, Sparse Matrix Pose Optimization was carried out to improve the positioning accuracy. The positioning accuracy of the robot in x and y directions was kept within 5 cm, the angle error was controlled within 2°, and the positioning time was reduced by 40%. An improved timing elastic band (TEB) algorithm was proposed to guide the robot to move safely and smoothly. A critical factor was introduced to adjust the distance between the waypoints and the obstacle, generating a safer trajectory, and increasing the constraint of acceleration and end speed; thus, smooth navigation of the robot to the target point was achieved. The experimental results showed that, in the case of multiple obstacles being present, the robot could choose the path with fewer obstacles, and the robot moved smoothly when facing turns and approaching the target point by reducing its overshoot. The proposed mapping, positioning, and improved TEB algorithms were effective for high-precision positioning and efficient multi-target detection.

Robust H-K Curvature Map Matching for Patient-to-CT Registration in Neurosurgical Navigation Systems.

Image-to-patient registration is a coordinate system matching process between real patients and medical images to actively utilize medical images such as computed tomography (CT) during surgery. This paper mainly deals with a markerless method utilizing scan data of patients and 3D data from CT images. The 3D surface data of the patient are registered to CT data using computer-based optimization methods such as iterative closest point (ICP) algorithms. However, if a proper initial location is not set up, the conventional ICP algorithm has the disadvantages that it takes a long converging time and also suffers from the local minimum problem during the process. We propose an automatic and robust 3D data registration method that can accurately find a proper initial location for the ICP algorithm using curvature matching. The proposed method finds and extracts the matching area for 3D registration by converting 3D CT data and 3D scan data to 2D curvature images and by performing curvature matching between them. Curvature features have characteristics that are robust to translation, rotation, and even some deformation. The proposed image-to-patient registration is implemented with the precise 3D registration of the extracted partial 3D CT data and the patient's scan data using the ICP algorithm.

Forensic Identification from Three-Dimensional Sphenoid Sinus Images Using the Iterative Closest Point Algorithm.

Forensic identification of human remains is crucial for legal, humanitarian, and civil reasons. Wide heterogeneity in sphenoid sinus morphology can be used for personal identification. This study aimed to propose a new protocol for personal identification based on three-dimensional (3D) reconstruction of sphenoid sinus CT images using Iterative Closest Point (ICP) algorithm. Seven hundred thirty-two patients which consisted of 348 females and 384 males were retrospectively included. The study sample includes 732 previous images as a source point set and 743 later ones as a scene target set. The sphenoid sinus computed tomography (CT) images were processed on a workstation (Dolphin imaging) to obtain 3D images and stored as a file format of Stereo lithography (.STL). Then, a Python library vtkplotter was used to transform the STL format to PLY format, which was adapted to Point Cloud Library (PCL). The ICP algorithm was used for point clouds matching. The metric Rank-N recognition rate was used for evaluation. The scene target set of 743 individuals was compared with the source point set of 732 individual models and achieved Rank-1 accuracy of 96.24%, Rank-2 accuracy of 99.73%, and Rank-3 accuracy of 100%. Our results indicated that the 3D point cloud registration of sphenoid sinuses was useful for assessing personal identification in forensic contexts.

3D Face Point Cloud Reconstruction and Recognition Using Depth Sensor.

Facial recognition has attracted more and more attention since the rapid growth of artificial intelligence (AI) techniques in recent years. However, most of the related works about facial reconstruction and recognition are mainly based on big data collection and image deep learning related algorithms. The data driven based AI approaches inevitably increase the computational complexity of CPU and usually highly count on GPU capacity. One of the typical issues of RGB-based facial recognition is its applicability in low light or dark environments. To solve this problem, this paper presents an effective procedure for facial reconstruction as well as facial recognition via using a depth sensor. For each testing candidate, the depth camera acquires a multi-view of its 3D point clouds. The point cloud sets are stitched for 3D model reconstruction by using the iterative closest point (ICP). Then, a segmentation procedure is designed to separate the model set into a body part and head part. Based on the segmented 3D face point clouds, certain facial features are then extracted for recognition scoring. Taking a single shot from the depth sensor, the point cloud data is going to register with other 3D face models to determine which is the best candidate the data belongs to. By using the proposed feature-based 3D facial similarity score algorithm, which composes of normal, curvature, and registration similarities between different point clouds, the person can be labeled correctly even in a dark environment. The proposed method is suitable for smart devices such as smart phones and smart pads with tiny depth camera equipped. Experiments with real-world data show that the proposed method is able to reconstruct denser models and achieve point cloud-based 3D face recognition.

3D Scene Reconstruction Using Omnidirectional Vision and LiDAR: A Hybrid Approach.

In this paper, we propose a novel approach to obtain accurate 3D reconstructions of large-scale environments by means of a mobile acquisition platform. The system incorporates a Velodyne LiDAR scanner, as well as a Point Grey Ladybug panoramic camera system. It was designed with genericity in mind, and hence, it does not make any assumption about the scene or about the sensor set-up. The main novelty of this work is that the proposed LiDAR mapping approach deals explicitly with the inhomogeneous density of point clouds produced by LiDAR scanners. To this end, we keep track of a global 3D map of the environment, which is continuously improved and refined by means of a surface reconstruction technique. Moreover, we perform surface analysis on consecutive generated point clouds in order to assure a perfect alignment with the global 3D map. In order to cope with drift, the system incorporates loop closure by determining the pose error and propagating it back in the pose graph. Our algorithm was exhaustively tested on data captured at a conference building, a university campus and an industrial site of a chemical company. Experiments demonstrate that it is capable of generating highly accurate 3D maps in very challenging environments. We can state that the average distance of corresponding point pairs between the ground truth and estimated point cloud approximates one centimeter for an area covering approximately 4000 m 2 . To prove the genericity of the system, it was tested on the well-known Kitti vision benchmark. The results show that our approach competes with state of the art methods without making any additional assumptions.

3D modelling method and application to a digital campus by fusing point cloud data and image data.

Objective: The use of single-source data for real-world 3D modelling currently faces problems such as deformation, pulling and fuzzy texture at the bottom of buildings in some feature models because of the lack of images. Moreover, LIDAR generates a huge amount of data, and the massive raw data processing and point cloud parsing puts high demands on the hardware arithmetic and algorithms. Aiming at the deficiencies and defects of the two data sources of inclined photogrammetry and airborne laser point cloud in the construction of high-quality and high-precision city-level 3D models. Methods: this study uses a university library building as an example and proposes the main technical process and method of modelling after fusing the point cloud data acquired by inclined photogrammetry and 3D laser scanning technology. This is accomplished in the reconstruction stage of multi-source data fusion through data spatial alignment, coordinate system unification and data spatial integration. At the stage of multi-source data fusion and reconstruction, through data spatial alignment, coordinate system unification, point cloud coarse alignment and the iterative closest point (ICP) algorithm, a realistic 3D model of a building is constructed to verify the effectiveness of the modelling method. Results: The method can effectively improve the accuracy of the real-life 3D model, repair the deficiencies in the model and optimise the details of the model. It can also significantly improve the fineness of the tilt photography model and perfectly present the geometric and texture information of the building, making it a superior method for fine 3D reconstruction. Conclusion: This 3D reconstruction method of buildings, which integrates low-altitude inclined photogrammetry and airborne light detection and ranging (LiDAR), has high positional accuracy and can provide new methods and new ideas for the construction of digital campuses as well as for other engineering applications.

Grey Wolf Optimizer with Behavior Considerations and Dimensional Learning in Three-Dimensional Tooth Model Reconstruction.

Three-dimensional registration with the affine transform is one of the most important steps in 3D reconstruction. In this paper, the modified grey wolf optimizer with behavior considerations and dimensional learning (BCDL-GWO) algorithm as a registration method is introduced. To refine the 3D registration result, we incorporate the iterative closet point (ICP). The BCDL-GWO with ICP method is implemented on the scanned commercial orthodontic tooth and regular tooth models. Since this is a registration from multi-views of optical images, the hierarchical structure is implemented. According to the results for both models, the proposed algorithm produces high-quality 3D visualization images with the smallest mean squared error of about 7.2186 and 7.3999 µm2, respectively. Our results are compared with the statistical randomization-based particle swarm optimization (SR-PSO). The results show that the BCDL-GWO with ICP is better than those from the SR-PSO. However, the computational complexities of both methods are similar.

Machine learning and registration for automatic seed localization in 3D US images for prostate brachytherapy.

PURPOSE: New radiation therapy protocols, in particular adaptive, focal or boost brachytherapy treatments, require determining precisely the position and orientation of the implanted radioactive seeds from real-time ultrasound (US) images. This is necessary to compare them to the planned one and to adjust automatically the dosimetric plan accordingly for next seeds implantations. The image modality, the small size of the seeds, and the artifacts they produce make it a very challenging problem. The objective of the presented work is to setup and to evaluate a robust and automatic method for seed localization in three-dimensional (3D) US images. METHODS: The presented method is based on a prelocalization of the needles through which the seeds are injected in the prostate. This prelocalization allows focusing the search on a region of interest (ROI) around the needle tip. Seeds localization starts by binarizing the ROI and removing false positives using, respectively, a Bayesian classifier and a support vector machine (SVM). This is followed by a registration stage using first an iterative closest point (ICP) for localizing the connected set of seeds (named strand) inserted through a needle, and secondly refining each seed position using sum of squared differences (SSD) as a similarity criterion. ICP registers a geometric model of the strand to the candidate voxels while SSD compares an appearance model of a single seed to a subset of the image. The method was evaluated both for 3D images of an Agar-agar phantom and a dataset of clinical 3D images. It was tested on stranded and on loose seeds. RESULTS: Results on phantom and clinical images were compared with a manual localization giving mean errors of 1.09 ± 0.61 mm on phantom image and 1.44 ± 0.45 mm on clinical images. On clinical images, the mean errors of individual seeds orientation was 4.33 ± 8 . 51 ∘ . CONCLUSIONS: The proposed algorithm for radioactive seed localization is robust, tested on different US images, accurate, giving small mean error values, and returns the five cylindrical seeds degrees of freedom.

Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios.

In this paper, we present a modular and flexible state estimation framework for legged robots operating in real-world scenarios, where environmental conditions, such as occlusions, low light, rough terrain, and dynamic obstacles can severely impair estimation performance. At the core of the proposed estimation system, called Pronto, is an Extended Kalman Filter (EKF) that fuses IMU and Leg Odometry sensing for pose and velocity estimation. We also show how Pronto can integrate pose corrections from visual and LIDAR and odometry to correct pose drift in a loosely coupled manner. This allows it to have a real-time proprioceptive estimation thread running at high frequency (250-1,000 Hz) for use in the control loop while taking advantage of occasional (and often delayed) low frequency (1-15 Hz) updates from exteroceptive sources, such as cameras and LIDARs. To demonstrate the robustness and versatility of the approach, we have tested it on a variety of legged platforms, including two humanoid robots (the Boston Dynamics Atlas and NASA Valkyrie) and two dynamic quadruped robots (IIT HyQ and ANYbotics ANYmal) for more than 2 h of total runtime and 1.37 km of distance traveled. The tests were conducted in a number of different field scenarios under the conditions described above. The algorithms presented in this paper are made available to the research community as open-source ROS packages.

Geometry Preserving Sampling Method Based on Spectral Decomposition for Large-Scale Environments.

In the context of 3D mapping, larger and larger point clouds are acquired with lidar sensors. Although pleasing to the eye, dense maps are not necessarily tailored for practical applications. For instance, in a surface inspection scenario, keeping geometric information such as the edges of objects is essential to detect cracks, whereas very dense areas of very little information such as the ground could hinder the main goal of the application. Several strategies exist to address this problem by reducing the number of points. However, they tend to underperform with non-uniform density, large sensor noise, spurious measurements, and large-scale point clouds, which is the case in mobile robotics. This paper presents a novel sampling algorithm based on spectral decomposition analysis to derive local density measures for each geometric primitive. The proposed method, called Spectral Decomposition Filter (SpDF), identifies and preserves geometric information along the topology of point clouds and is able to scale to large environments with a non-uniform density. Finally, qualitative and quantitative experiments verify the feasibility of our method and present a large-scale evaluation of SpDF with other seven point cloud sampling algorithms, in the context of the 3D registration problem using the Iterative Closest Point (ICP) algorithm on real-world datasets. Results show that a compression ratio up to 97 % can be achieved when accepting a registration error within the range accuracy of the sensor, here for large scale environments of less than 2 cm.

A Novel Visualization System of Using Augmented Reality in Knee Replacement Surgery: Enhanced Bidirectional Maximum CorrentropyAlgorithm.

BACKGROUND AND AIM: Image registration and alignment are the main limitations of augmented reality-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion and maximize the overlapping parts. METHODOLOGY: markerless image registration method was used for Augmented reality-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusion in right to left direction and non-border occlusion from left to right direction. RESULTS: This study has improved video precision to 0.57 mm ∼ 0.61 mm alignment error. Furthermore, with the use of bidirectional points, i.e. Forwards and backwards directional cloud point, the iteration on image registration was decreased. This has led to improved the processing time as well. The processing time of video frames was improved to 7.4 ∼11.74 fps. CONCLUSIONS: It seems clear that this proposed system has focused on overcoming the misalignment difficulty caused by movement of patient and enhancing the AR visualization during knee replacement surgery. The proposed system was reliable and favourable which helps in eliminating alignment error by ascertaining the optimal rigid transformation between two cloud points and removing the outliers and non-Gaussian noise. The proposed augmented reality system helps in accurate visualization and navigation of anatomy of knee such as femur, tibia, cartilage, blood vessels, etc. This article is protected by copyright. All rights reserved.

Feature-based three-dimensional registration for repetitive geometry in machine vision.

As an important step in three-dimensional (3D) machine vision, 3D registration is a process of aligning two or multiple 3D point clouds that are collected from different perspectives together into a complete one. The most popular approach to register point clouds is to minimize the difference between these point clouds iteratively by Iterative Closest Point (ICP) algorithm. However, ICP does not work well for repetitive geometries. To solve this problem, a feature-based 3D registration algorithm is proposed to align the point clouds that are generated by vision-based 3D reconstruction. By utilizing texture information of the object and the robustness of image features, 3D correspondences can be retrieved so that the 3D registration of two point clouds is to solve a rigid transformation. The comparison of our method and different ICP algorithms demonstrates that our proposed algorithm is more accurate, efficient and robust for repetitive geometry registration. Moreover, this method can also be used to solve high depth uncertainty problem caused by little camera baseline in vision-based 3D reconstruction.