A point cloud-based deep learning strategy for protein-ligand binding affinity prediction.

There is great interest to develop artificial intelligence-based protein-ligand binding affinity models due to their immense applications in drug discovery. In this paper, PointNet and PointTransformer, two pointwise multi-layer perceptrons have been applied for protein-ligand binding affinity prediction for the first time. Three-dimensional point clouds could be rapidly generated from PDBbind-2016 with 3772 and 11 327 individual point clouds derived from the refined or/and general sets, respectively. These point clouds (the refined or the extended set) were used to train PointNet or PointTransformer, resulting in protein-ligand binding affinity prediction models with Pearson correlation coefficients R = 0.795 or 0.833 from the extended data set, respectively, based on the CASF-2016 benchmark test. The analysis of parameters suggests that the two deep learning models were capable to learn many interactions between proteins and their ligands, and some key atoms for the interactions could be visualized. The protein-ligand interaction features learned by PointTransformer could be further adapted for the XGBoost-based machine learning algorithm, resulting in prediction models with an average Rp of 0.827, which is on par with state-of-the-art machine learning models. These results suggest that the point clouds derived from PDBbind data sets are useful to evaluate the performance of 3D point clouds-centered deep learning algorithms, which could learn atomic features of protein-ligand interactions from natural evolution or medicinal chemistry and thus have wide applications in chemistry and biology.

Feasibility evaluation of radiotherapy positioning system guided by augmented reality and point cloud registration.

PURPOSE: To develop a radiotherapy positioning system based on Point Cloud Registration (PCR) and Augmented Reality (AR), and to verify its feasibility. METHODS: The optimal steps of PCR were investigated, and virtual positioning experiments were designed to evaluate its accuracy and speed. AR was implemented by Unity 3D and Vuforia for initial position correction, and PCR for precision registration, to achieve the proposed radiotherapy positioning system. Feasibility of the proposed system was evaluated through phantom positioning tests as well as real human positioning tests. Real human tests involved breath-holding positioning and free-breathing positioning tests. Evaluation metrics included 6 Degree of Freedom (DOF) deviations and Distance (D) errors. Additionally, the interaction between CBCT and the proposed system was envisaged through CBCT and optical cross-source PCR. RESULTS: Point-to-plane iterative Closest Point (ICP), statistical filtering, uniform down-sampling, and optimal sampling ratio were determined for PCR procedure. In virtual positioning tests, a single registration took only 0.111 s, and the average D error for 15 patients was 0.015 ± 0.029 mm., Errors of phantom tests were at the sub-millimeter level, with an average D error of 0.6 ± 0.2 mm. In the real human positioning tests, the average accuracy of breath-holding positioning was still at the sub-millimeter level, where the errors of X, Y and Z axes were 0.59 ± 0.12 mm, 0.54 ± 0.12 mm, and 0.52 ± 0.09 mm, and the average D error was 0.96 ± 0.22 mm. In the free-breathing positioning, the average errors in X, Y, and Z axes were still less than 1 mm. Although the mean D error was 1.93 ± 0.36 mm, it still falls within a clinically acceptable error margin. CONCLUSION: The AR and PCR-guided radiotherapy positioning system enables markerless, radiation-free and high-accuracy positioning, which is feasible in real-world scenarios.

Adaptive Clustering for Point Cloud.

The point cloud segmentation method plays an important role in practical applications, such as remote sensing, mobile robots, and 3D modeling. However, there are still some limitations to the current point cloud data segmentation method when applied to large-scale scenes. Therefore, this paper proposes an adaptive clustering segmentation method. In this method, the threshold for clustering points within the point cloud is calculated using the characteristic parameters of adjacent points. After completing the preliminary segmentation of the point cloud, the segmentation results are further refined according to the standard deviation of the cluster points. Then, the cluster points whose number does not meet the conditions are further segmented, and, finally, scene point cloud data segmentation is realized. To test the superiority of this method, this study was based on point cloud data from a park in Guilin, Guangxi, China. The experimental results showed that this method is more practical and efficient than other methods, and it can effectively segment all ground objects and ground point cloud data in a scene. Compared with other segmentation methods that are easily affected by parameters, this method has strong robustness. In order to verify the universality of the method proposed in this paper, we test a public data set provided by ISPRS. The method achieves good segmentation results for multiple sample data, and it can distinguish noise points in a scene.

Research on the Method for Recognizing Bulk Grain-Loading Status Based on LiDAR.

Grain is a common bulk cargo. To ensure optimal utilization of transportation space and prevent overflow accidents, it is necessary to observe the grain's shape and determine the loading status during the loading process. Traditional methods often rely on manual judgment, which results in high labor intensity, poor safety, and low loading efficiency. Therefore, this paper proposes a method for recognizing the bulk grain-loading status based on Light Detection and Ranging (LiDAR). This method uses LiDAR to obtain point cloud data and constructs a deep learning network to perform target recognition and component segmentation on loading vehicles, extract vehicle positions and grain shapes, and recognize and make known the bulk grain-loading status. Based on the measured point cloud data of bulk grain loading, in the point cloud-classification task, the overall accuracy is 97.9% and the mean accuracy is 98.1%. In the vehicle component-segmentation task, the overall accuracy is 99.1% and the Mean Intersection over Union is 96.6%. The results indicate that the method has reliable performance in the research tasks of extracting vehicle positions, detecting grain shapes, and recognizing loading status.

Receptive Field Space for Point Cloud Analysis.

Similar to convolutional neural networks for image processing, existing analysis methods for 3D point clouds often require the designation of a local neighborhood to describe the local features of the point cloud. This local neighborhood is typically manually specified, which makes it impossible for the network to dynamically adjust the receptive field's range. If the range is too large, it tends to overlook local details, and if it is too small, it cannot establish global dependencies. To address this issue, we introduce in this paper a new concept: receptive field space (RFS). With a minor computational cost, we extract features from multiple consecutive receptive field ranges to form this new receptive field space. On this basis, we further propose a receptive field space attention mechanism, enabling the network to adaptively select the most effective receptive field range from RFS, thus equipping the network with the ability to adjust granularity adaptively. Our approach achieved state-of-the-art performance in both point cloud classification, with an overall accuracy (OA) of 94.2%, and part segmentation, achieving an mIoU of 86.0%, demonstrating the effectiveness of our method.

A Review of Dynamic Object Filtering in SLAM Based on 3D LiDAR.

SLAM (Simultaneous Localization and Mapping) based on 3D LiDAR (Laser Detection and Ranging) is an expanding field of research with numerous applications in the areas of autonomous driving, mobile robotics, and UAVs (Unmanned Aerial Vehicles). However, in most real-world scenarios, dynamic objects can negatively impact the accuracy and robustness of SLAM. In recent years, the challenge of achieving optimal SLAM performance in dynamic environments has led to the emergence of various research efforts, but there has been relatively little relevant review. This work delves into the development process and current state of SLAM based on 3D LiDAR in dynamic environments. After analyzing the necessity and importance of filtering dynamic objects in SLAM, this paper is developed from two dimensions. At the solution-oriented level, mainstream methods of filtering dynamic targets in 3D point cloud are introduced in detail, such as the ray-tracing-based approach, the visibility-based approach, the segmentation-based approach, and others. Then, at the problem-oriented level, this paper classifies dynamic objects and summarizes the corresponding processing strategies for different categories in the SLAM framework, such as online real-time filtering, post-processing after the mapping, and Long-term SLAM. Finally, the development trends and research directions of dynamic object filtering in SLAM based on 3D LiDAR are discussed and predicted.

Comparison of Point Cloud Registration Techniques on Scanned Physical Objects.

This paper presents a comparative analysis of six prominent registration techniques for solving CAD model alignment problems. Unlike the typical approach of assessing registration algorithms with synthetic datasets, our study utilizes point clouds generated from the Cranfield benchmark. Point clouds are sampled from existing CAD models and 3D scans of physical objects, introducing real-world complexities such as noise and outliers. The acquired point cloud scans, including ground-truth transformations, are made publicly available. This dataset includes several cleaned-up scans of nine 3D-printed objects. Our main contribution lies in assessing the performance of three classical (GO-ICP, RANSAC, FGR) and three learning-based (PointNetLK, RPMNet, ROPNet) methods on real-world scans, using a wide range of metrics. These include recall, accuracy and computation time. Our comparison shows a high accuracy for GO-ICP, as well as PointNetLK, RANSAC and RPMNet combined with ICP refinement. However, apart from GO-ICP, all methods show a significant number of failure cases when applied to scans containing more noise or requiring larger transformations. FGR and RANSAC are among the quickest methods, while GO-ICP takes several seconds to solve. Finally, while learning-based methods demonstrate good performance and low computation times, they have difficulties in training and generalizing. Our results can aid novice researchers in the field in selecting a suitable registration method for their application, based on quantitative metrics. Furthermore, our code can be used by others to evaluate novel methods.

Crop Leaf Phenotypic Parameter Measurement Based on the RKM-D Point Cloud Method.

Crop leaf length, perimeter, and area serve as vital phenotypic indicators of crop growth status, the measurement of which is important for crop monitoring and yield estimation. However, processing a leaf point cloud is often challenging due to cluttered, fluctuating, and uncertain points, which culminate in inaccurate measurements of leaf phenotypic parameters. To tackle this issue, the RKM-D point cloud method for measuring leaf phenotypic parameters is proposed, which is based on the fusion of improved Random Sample Consensus with a ground point removal (R) algorithm, the K-means clustering (K) algorithm, the Moving Least Squares (M) method, and the Euclidean distance (D) algorithm. Pepper leaves were obtained from three growth periods on the 14th, 28th, and 42nd days as experimental subjects, and a stereo camera was employed to capture point clouds. The experimental results reveal that the RKM-D point cloud method delivers high precision in measuring leaf phenotypic parameters. (i) For leaf length, the coefficient of determination (R2) surpasses 0.81, the mean absolute error (MAE) is less than 3.50 mm, the mean relative error (MRE) is less than 5.93%, and the root mean square error (RMSE) is less than 3.73 mm. (ii) For leaf perimeter, the R2 surpasses 0.82, the MAE is less than 7.30 mm, the MRE is less than 4.50%, and the RMSE is less than 8.37 mm. (iii) For leaf area, the R2 surpasses 0.97, the MAE is less than 64.66 mm2, the MRE is less than 4.96%, and the RMSE is less than 73.06 mm2. The results show that the proposed RKM-D point cloud method offers a robust solution for the precise measurement of crop leaf phenotypic parameters.

Deep Ordinal Classification in Forest Areas Using Light Detection and Ranging Point Clouds.

Recent advances in Deep Learning and aerial Light Detection And Ranging (LiDAR) have offered the possibility of refining the classification and segmentation of 3D point clouds to contribute to the monitoring of complex environments. In this context, the present study focuses on developing an ordinal classification model in forest areas where LiDAR point clouds can be classified into four distinct ordinal classes: ground, low vegetation, medium vegetation, and high vegetation. To do so, an effective soft labeling technique based on a novel proposed generalized exponential function (CE-GE) is applied to the PointNet network architecture. Statistical analyses based on Kolmogorov-Smirnov and Student's t-test reveal that the CE-GE method achieves the best results for all the evaluation metrics compared to other methodologies. Regarding the confusion matrices of the best alternative conceived and the standard categorical cross-entropy method, the smoothed ordinal classification obtains a more consistent classification compared to the nominal approach. Thus, the proposed methodology significantly improves the point-by-point classification of PointNet, reducing the errors in distinguishing between the middle classes (low vegetation and medium vegetation).

A Coal Mine Tunnel Deformation Detection Method Using Point Cloud Data.

In recent years, the deformation detection technology for underground tunnels has played a crucial role in coal mine safety management. Currently, traditional methods such as the cross method and those employing the roof abscission layer monitoring instrument are primarily used for tunnel deformation detection in coal mines. With the advancement of photogrammetric methods, three-dimensional laser scanners have gradually become the primary method for deformation detection of coal mine tunnels. However, due to the high-risk confined spaces and distant distribution of coal mine tunnels, stationary three-dimensional laser scanning technology requires a significant amount of labor and time, posing certain operational risks. Currently, mobile laser scanning has become a popular method for coal mine tunnel deformation detection. This paper proposes a method for detecting point cloud deformation of underground coal mine tunnels based on a handheld three-dimensional laser scanner. This method utilizes SLAM laser radar to obtain complete point cloud information of the entire tunnel, while projecting the three-dimensional point cloud onto different planes to obtain the coordinates of the tunnel centerline. By using the calculated tunnel centerline, the three-dimensional point cloud data collected at different times are matched to the same coordinate system, and then the tunnel deformation parameters are analyzed separately from the global and cross-sectional perspectives. Through on-site collection of tunnel data, this paper verifies the feasibility of the algorithm and compares it with other centerline fitting and point cloud registration algorithms, demonstrating higher accuracy and meeting practical needs.

Point Cloud Denoising and Feature Preservation: An Adaptive Kernel Approach Based on Local Density and Global Statistics.

Noise removal is a critical stage in the preprocessing of point clouds, exerting a significant impact on subsequent processes such as point cloud classification, segmentation, feature extraction, and 3D reconstruction. The exploration of methods capable of adapting to and effectively handling the noise in point clouds from real-world outdoor scenes remains an open and practically significant issue. Addressing this issue, this study proposes an adaptive kernel approach based on local density and global statistics (AKA-LDGS). This method constructs the overall framework for point cloud denoising using Bayesian estimation theory. It dynamically sets the prior probabilities of real and noise points according to the spatial function relationship, which varies with the distance from the points to the center of the LiDAR. The probability density function (PDF) for real points is constructed using a multivariate Gaussian distribution, while the PDF for noise points is established using a data-driven, non-parametric adaptive kernel density estimation (KDE) approach. Experimental results demonstrate that this method can effectively remove noise from point clouds in real-world outdoor scenes while maintaining the overall structural features of the point cloud.

HALNet: Partial Point Cloud Registration Based on Hybrid Attention and Deep Local Features.

Point cloud registration is an important task in computer vision and robotics which is widely used in 3D reconstruction, target recognition, and other fields. At present, many registration methods based on deep learning have better registration accuracy in complete point cloud registration, but partial registration accuracy is poor. Therefore, a partial point cloud registration network, HALNet, is proposed. Firstly, a feature extraction network consisting mainly of adaptive graph convolution (AGConv), two-dimensional convolution, and convolution block attention (CBAM) is used to learn the features of the initial point cloud. Then the overlapping estimation is used to remove the non-overlapping points of the two point clouds, and the hybrid attention mechanism composed of self-attention and cross-attention is used to fuse the geometric information of the two point clouds. Finally, the rigid transformation is obtained by using the fully connected layer. Five methods with excellent registration performance were selected for comparison. Compared with SCANet, which has the best registration performance among the five methods, the RMSE(R) and MAE(R) of HALNet are reduced by 10.67% and 12.05%. In addition, the results of the ablation experiment verify that the hybrid attention mechanism and fully connected layer are conducive to improving registration performance.

Vibration Position Detection of Robot Arm Based on Feature Extraction of 3D Lidar.

In the process of construction, pouring and vibrating concrete on existing reinforced structures is a necessary process. This paper presents an automatic vibration position detecting method based on the feature extraction of 3D lidar point clouds. Compared with the image-based method, this method has better anti-interference performance to light with reduced computational consumption. First, lidar scans are used to capture multiple frames of local steel bar point clouds. Then, the clouds are stitched by Normal Distribution Transform (NDT) for preliminary matching and Iterative Closest Point (ICP) for fine-matching. The Graph-Based Optimization (g2o) method further refines the precision of the 3D registration. Afterwards, the 3D point clouds are projected into a 2D image. Finally, the locations of concrete vibration points and concrete casting points are discerned through point cloud and image processing technologies. Experiments demonstrate that the proposed automatic method outperforms ICP and NDT algorithms, reducing the mean square error (MSE) by 11.5% and 11.37%, respectively. The maximum discrepancies in identifying concrete vibration points and concrete casting points are 0.059 ± 0.031 m and 0.089 ± 0.0493 m, respectively, fulfilling the requirement for concrete vibration detection.

Korean Cattle 3D Reconstruction from Multi-View 3D-Camera System in Real Environment.

The rapid evolution of 3D technology in recent years has brought about significant change in the field of agriculture, including precision livestock management. From 3D geometry information, the weight and characteristics of body parts of Korean cattle can be analyzed to improve cow growth. In this paper, a system of cameras is built to synchronously capture 3D data and then reconstruct a 3D mesh representation. In general, to reconstruct non-rigid objects, a system of cameras is synchronized and calibrated, and then the data of each camera are transformed to global coordinates. However, when reconstructing cattle in a real environment, difficulties including fences and the vibration of cameras can lead to the failure of the process of reconstruction. A new scheme is proposed that automatically removes environmental fences and noise. An optimization method is proposed that interweaves camera pose updates, and the distances between the camera pose and the initial camera position are added as part of the objective function. The difference between the camera's point clouds to the mesh output is reduced from 7.5 mm to 5.5 mm. The experimental results showed that our scheme can automatically generate a high-quality mesh in a real environment. This scheme provides data that can be used for other research on Korean cattle.

Extraction of Moso Bamboo Parameters Based on the Combination of ALS and TLS Point Cloud Data.

Extracting moso bamboo parameters from single-source point cloud data has limitations. In this article, a new approach for extracting moso bamboo parameters using airborne laser scanning (ALS) and terrestrial laser scanning (TLS) point cloud data is proposed. Using the field-surveyed coordinates of plot corner points and the Iterative Closest Point (ICP) algorithm, the ALS and TLS point clouds were aligned. Considering the difference in point distribution of ALS, TLS, and the merged point cloud, individual bamboo plants were segmented from the ALS point cloud using the point cloud segmentation (PCS) algorithm, and individual bamboo plants were segmented from the TLS and the merged point cloud using the comparative shortest-path (CSP) method. The cylinder fitting method was used to estimate the diameter at breast height (DBH) of the segmented bamboo plants. The accuracy was calculated by comparing the bamboo parameter values extracted by the above methods with reference data in three sample plots. The comparison results showed that by using the merged data, the detection rate of moso bamboo plants could reach up to 97.30%; the R2 of the estimated bamboo height was increased to above 0.96, and the root mean square error (RMSE) decreased from 1.14 m at most to a range of 0.35-0.48 m, while the R2 of the DBH fit was increased to a range of 0.97-0.99, and the RMSE decreased from 0.004 m at most to a range of 0.001-0.003 m. The accuracy of moso bamboo parameter extraction was significantly improved by using the merged point cloud data.

SGRTmreg: A Learning-Based Optimization Framework for Multiple Pairwise Registrations.

Point cloud registration is a fundamental task in computer vision and graphics, which is widely used in 3D reconstruction, object tracking, and atlas reconstruction. Learning-based optimization and deep learning methods have been widely developed in pairwise registration due to their own distinctive advantages. Deep learning methods offer greater flexibility and enable registering unseen point clouds that are not trained. Learning-based optimization methods exhibit enhanced robustness and stability when handling registration under various perturbations, such as noise, outliers, and occlusions. To leverage the strengths of both approaches to achieve a less time-consuming, robust, and stable registration for multiple instances, we propose a novel computational framework called SGRTmreg for multiple pairwise registrations in this paper. The SGRTmreg framework utilizes three components-a Searching scheme, a learning-based optimization method called Graph-based Reweighted discriminative optimization (GRDO), and a Transfer module to achieve multi-instance point cloud registration.Given a collection of instances to be matched, a template as a target point cloud, and an instance as a source point cloud, the searching scheme selects one point cloud from the collection that closely resembles the source. GRDO then learns a sequence of regressors by aligning the source to the target, while the transfer module stores and applies the learned regressors to align the selected point cloud to the target and estimate the transformation of the selected point cloud. In short, SGRTmreg harnesses a shared sequence of regressors to register multiple point clouds to a target point cloud. We conduct extensive registration experiments on various datasets to evaluate the proposed framework. The experimental results demonstrate that SGRTmreg achieves multiple pairwise registrations with higher accuracy, robustness, and stability than the state-of-the-art deep learning and traditional registration methods.

Research on Mobile Robot Navigation Method Based on Semantic Information.

This paper proposes a solution to the problem of mobile robot navigation and trajectory interpolation in dynamic environments with large scenes. The solution combines a semantic laser SLAM system that utilizes deep learning and a trajectory interpolation algorithm. The paper first introduces some open-source laser SLAM algorithms and then elaborates in detail on the general framework of the SLAM system used in this paper. Second, the concept of voxels is introduced into the occupation probability map to enhance the ability of local voxel maps to represent dynamic objects. Then, in this paper, we propose a PointNet++ point cloud semantic segmentation network combined with deep learning algorithms to extract deep features of dynamic point clouds in large scenes and output semantic information of points on static objects. A descriptor of the global environment is generated based on its semantic information. Closed-loop completion of global map optimization is performed to reduce cumulative error. Finally, T-trajectory interpolation is utilized to ensure the motion performance of the robot and improve the smooth stability of the robot trajectory. The experimental results indicate that the combination of the semantic laser SLAM system with deep learning and the trajectory interpolation algorithm proposed in this paper yields better graph-building and loop-closure effects in large scenes at SIASUN large scene campus. The use of T-trajectory interpolation ensures vibration-free and stable transitions between target points.

Robust Point Cloud Registration for Aircraft Engine Pipeline Systems.

Aircraft engine systems are composed of numerous pipelines. It is crucial to regularly inspect these pipelines to detect any damages or failures that could potentially lead to serious accidents. The inspection process typically involves capturing complete 3D point clouds of the pipelines using 3D scanning techniques from multiple viewpoints. To obtain a complete and accurate representation of the aircraft pipeline system, it is necessary to register and align the individual point clouds acquired from different views. However, the structures of aircraft pipelines often appear similar from different viewpoints, and the scanning process is prone to occlusions, resulting in incomplete point cloud data. The occlusions pose a challenge for existing registration methods, as they can lead to missing or wrong correspondences. To this end, we present a novel registration framework specifically designed for aircraft pipeline scenes. The proposed framework consists of two main steps. First, we extract the point feature structure of the pipeline axis by leveraging the cylindrical characteristics observed between adjacent blocks. Then, we design a new 3D descriptor called PL-PPFs (Point Line-Point Pair Features), which combines information from both the pipeline features and the engine assembly line features within the aircraft pipeline point cloud. By incorporating these relevant features, our descriptor enables accurate identification of the structure of the engine's piping system. Experimental results demonstrate the effectiveness of our approach on aircraft engine pipeline point cloud data.

Object Detection and Information Perception by Fusing YOLO-SCG and Point Cloud Clustering.

Robots need to sense information about the external environment before moving, which helps them to recognize and understand their surroundings so that they can plan safe and effective paths and avoid obstacles. Conventional algorithms using a single sensor cannot obtain enough information and lack real-time capabilities. To solve these problems, we propose an information perception algorithm with vision as the core and the fusion of LiDAR. Regarding vision, we propose the YOLO-SCG model, which is able to detect objects faster and more accurately. When processing point clouds, we integrate the detection results of vision for local clustering, improving both the processing speed of the point cloud and the detection effectiveness. Experiments verify that our proposed YOLO-SCG algorithm improves accuracy by 4.06% and detection speed by 7.81% compared to YOLOv9, and our algorithm excels in distinguishing different objects in the clustering of point clouds.

Clique-like Point Cloud Registration: A Flexible Sampling Registration Method Based on Clique-like for Low-Overlapping Point Cloud.

Three-dimensional point cloud registration is a critical task in 3D perception for sensors that aims to determine the optimal alignment between two point clouds by finding the best transformation. Existing methods like RANSAC and its variants often face challenges, such as sensitivity to low overlap rates, high computational costs, and susceptibility to outliers, leading to inaccurate results, especially in complex or noisy environments. In this paper, we introduce a novel 3D registration method, CL-PCR, inspired by the concept of maximal cliques and built upon the SC2-PCR framework. Our approach allows for the flexible use of smaller sampling subsets to extract more local consensus information, thereby generating accurate pose hypotheses even in scenarios with low overlap between point clouds. This method enhances robustness against low overlap and reduces the influence of outliers, addressing the limitations of traditional techniques. First, we construct a graph matrix to represent the compatibility relationships among the initial correspondences. Next, we build clique-likes subsets of various sizes within the graph matrix, each representing a consensus set. Then, we compute the transformation hypotheses for the subsets using the SVD algorithm and select the best hypothesis for registration based on evaluation metrics. Extensive experiments demonstrate the effectiveness of CL-PCR. In comparison experiments on the 3DMatch/3DLoMatch datasets using both FPFH and FCGF descriptors, our Fast-CL-PCRv1 outperforms state-of-the-art algorithms, achieving superior registration performance. Additionally, we validate the practicality and robustness of our method with real-world data.