PointCloud-At: Point Cloud Convolutional Neural Networks with Attention for 3D Data Processing.

The rapid growth in technologies for 3D sensors has made point cloud data increasingly available in different applications such as autonomous driving, robotics, and virtual and augmented reality. This raises a growing need for deep learning methods to process the data. Point clouds are difficult to be used directly as inputs in several deep learning techniques. The difficulty is raised by the unstructured and unordered nature of the point cloud data. So, machine learning models built for images or videos cannot be used directly on point cloud data. Although the research in the field of point clouds has gained high attention and different methods have been developed over the decade, very few research works directly with point cloud data, and most of them convert the point cloud data into 2D images or voxels by performing some pre-processing that causes information loss. Methods that directly work on point clouds are in the early stage and this affects the performance and accuracy of the models. Advanced techniques in classical convolutional neural networks, such as the attention mechanism, need to be transferred to the methods directly working with point clouds. In this research, an attention mechanism is proposed to be added to deep convolutional neural networks that process point clouds directly. The attention module was proposed based on specific pooling operations which are designed to be applied directly to point clouds to extract vital information from the point clouds. Segmentation of the ShapeNet dataset was performed to evaluate the method. The mean intersection over union (mIoU) score of the proposed framework was increased after applying the attention method compared to a base state-of-the-art framework that does not have the attention mechanism.

Multi-Task Foreground-Aware Network with Depth Completion for Enhanced RGB-D Fusion Object Detection Based on Transformer.

Fusing multiple sensor perceptions, specifically LiDAR and camera, is a prevalent method for target recognition in autonomous driving systems. Traditional object detection algorithms are limited by the sparse nature of LiDAR point clouds, resulting in poor fusion performance, especially for detecting small and distant targets. In this paper, a multi-task parallel neural network based on the Transformer is constructed to simultaneously perform depth completion and object detection. The loss functions are redesigned to reduce environmental noise in depth completion, and a new fusion module is designed to enhance the network's perception of the foreground and background. The network leverages the correlation between RGB pixels for depth completion, completing the LiDAR point cloud and addressing the mismatch between sparse LiDAR features and dense pixel features. Subsequently, we extract depth map features and effectively fuse them with RGB features, fully utilizing the depth feature differences between foreground and background to enhance object detection performance, especially for challenging targets. Compared to the baseline network, improvements of 4.78%, 8.93%, and 15.54% are achieved in the difficult indicators for cars, pedestrians, and cyclists, respectively. Experimental results also demonstrate that the network achieves a speed of 38 fps, validating the efficiency and feasibility of the proposed method.

Research on High-Precision Measurement Method for Small-Size Gears with Small-Modulus.

Small-modulus gears, which are essential for motion transmission in precision instruments, present a measurement challenge due to their minuscule gear gaps. A high-precision measurement method under the influence of positioning errors is proposed, enabling precise evaluation of the machining quality of small-modulus gears. Firstly, a compound measurement platform for small-modulus gears is developed. Using a 3D model of the measurement system, the mathematical relationships governing motion transmission between various components are analyzed. Secondly, the formation mechanism of gear positioning error is revealed and its important influence on measurement accuracy is discussed. An optimization method for spatial coordinate transformation matrices under positioning errors of gears is proposed. Thirdly, the study focuses on small-sized gears with a modulus of 0.1 mm and a six-level accuracy. Based on the aforementioned measurement system, the tooth profile measurement points are collected in the actual workpiece coordinate system. Then, gear error parameters are extracted based on the established models for tooth profile deviation and pitch deviation. Finally, the accuracy and effectiveness of the proposed measurement method are verified by comparing the measurement results of the P26 gear measuring center.

Depth-Sensing-Based Algorithm for Chest Morphology Assessment in Children with Cerebral Palsy.

This study introduced a depth-sensing-based approach with robust algorithms for tracking relative morphological changes in the chests of patients undergoing physical therapy. The problem that was addressed was the periodic change in morphological parameters induced by breathing, and since the recording was continuous, the parameters were extracted for the moments of maximum and minimum volumes of the chest (inspiration and expiration moments), and analyzed. The parameters were derived from morphological transverse cross-sections (CSs), which were extracted for the moments of maximal and minimal depth variations, and the reliability of the results was expressed through the coefficient of variation (CV) of the resulting curves. Across all subjects and levels of observed anatomy, the mean CV for CS depth values was smaller than 2%, and the mean CV of the CS area was smaller than 1%. To prove the reproducibility of measurements (extraction of morphological parameters), 10 subjects were recorded in two consecutive sessions with a short interval (2 weeks) where no changes in the monitored parameters were expected and statistical methods show that there was no statistically significant difference between the sessions, which confirms the reproducibility hypothesis. Additionally, based on the representative CSs for inspiration and expirations moments, chest mobility in quiet breathing was examined, and the statistical test showed no difference between the two sessions. The findings justify the proposed algorithm as a valuable tool for evaluating the impact of rehabilitation exercises on chest morphology.

Template-Guided Hierarchical Multi-View Registration Framework of Unordered Bridge Terrestrial Laser Scanning Data.

The registration of bridge point cloud data (PCD) is an important preprocessing step for tasks such as bridge modeling, deformation detection, and bridge health monitoring. However, most existing research on bridge PCD registration only focused on pairwise registration, and payed insufficient attention to multi-view registration. In addition, to recover the overlaps of unordered multiple scans and obtain the merging order, extensive pairwise matching and the creation of a fully connected graph of all scans are often required, resulting in low efficiency. To address these issues, this paper proposes a marker-free template-guided method to align multiple unordered bridge PCD to a global coordinate system. Firstly, by aligning each scan to a given registration template, the overlaps between all the scans are recovered. Secondly, a fully connected graph is created based on the overlaps and scanning locations, and then a graph-partition algorithm is utilized to construct the scan-blocks. Then, the coarse-to-fine registration is performed within each scan-block, and the transformation matrix of coarse registration is obtained using an intelligent optimization algorithm. Finally, global block-to-block registration is performed to align all scans to a unified coordinate reference system. We tested our framework on different bridge point cloud datasets, including a suspension bridge and a continuous rigid frame bridge, to evaluate its accuracy. Experimental results demonstrate that our method has high accuracy.

Combined Filtering Method for Offshore Oil and Gas Platform Point Cloud Data Based on KNN_PCF and Hy_WHF and Its Application in 3D Reconstruction.

With the increasing scale of deep-sea oil exploration and drilling platforms, the assessment, maintenance, and optimization of marine structures have become crucial. Traditional detection and manual measurement methods are inadequate for meeting these demands, but three-dimensional laser scanning technology offers a promising solution. However, the complexity of the marine environment, including waves and wind, often leads to problematic point cloud data characterized by noise points and redundancy. To address this challenge, this paper proposes a method that combines K-Nearest-Neighborhood filtering with a hyperbolic function-based weighted hybrid filtering. The experimental results demonstrate the exceptional performance of the algorithm in processing point cloud data from offshore oil and gas platforms. The method improves noise point filtering efficiency by approximately 11% and decreases the total error by 0.6 percentage points compared to existing technologies. Not only does this method accurately process anomalies in high-density areas-it also removes noise while preserving important details. Furthermore, the research method presented in this paper is particularly suited for processing large point cloud data in complex marine environments. It enhances data accuracy and optimizes the three-dimensional reconstruction of offshore oil and gas platforms, providing reliable dimensional information for land-based prefabrication of these platforms.

Self-Attention Mechanism-Based Head Pose Estimation Network with Fusion of Point Cloud and Image Features.

Head pose estimation serves various applications, such as gaze estimation, fatigue-driven detection, and virtual reality. Nonetheless, achieving precise and efficient predictions remains challenging owing to the reliance on singular data sources. Therefore, this study introduces a technique involving multimodal feature fusion to elevate head pose estimation accuracy. The proposed method amalgamates data derived from diverse sources, including RGB and depth images, to construct a comprehensive three-dimensional representation of the head, commonly referred to as a point cloud. The noteworthy innovations of this method encompass a residual multilayer perceptron structure within PointNet, designed to tackle gradient-related challenges, along with spatial self-attention mechanisms aimed at noise reduction. The enhanced PointNet and ResNet networks are utilized to extract features from both point clouds and images. These extracted features undergo fusion. Furthermore, the incorporation of a scoring module strengthens robustness, particularly in scenarios involving facial occlusion. This is achieved by preserving features from the highest-scoring point cloud. Additionally, a prediction module is employed, combining classification and regression methodologies to accurately estimate head poses. The proposed method improves the accuracy and robustness of head pose estimation, especially in cases involving facial obstructions. These advancements are substantiated by experiments conducted using the BIWI dataset, demonstrating the superiority of this method over existing techniques.

LiDAR Point Cloud Data Combined Structural Analysis Based on Strong Form Meshless Method Using Essential Boundary Condition Capturing.

This study proposes a novel hybrid simulation technique for analyzing structural deformation and stress using light detection and ranging (LiDAR)-scanned point cloud data (PCD) and polynomial regression processing. The method estimates the edge and corner points of the deformed structure from the PCD. It transforms into a Dirichlet boundary condition for the numerical simulation using the particle difference method (PDM), which utilizes nodes only based on the strong formulation, and it is advantageous for handling essential boundaries and nodal rearrangement, including node generation and deletion between analysis steps. Unlike previous studies, which relied on digital images with attached targets, this research uses PCD acquired through LiDAR scanning during the loading process without any target. Essential boundary condition implementation naturally builds a boundary value problem for the PDM simulation. The developed hybrid simulation technique was validated through an elastic beam problem and a three-point bending test on a rubber beam. The results were compared with those of ANSYS analysis, showing that the technique accurately approximates the deformed edge shape leading to accurate stress calculations. The accuracy improved when using a linear strain model and increasing the number of PDM model nodes. Additionally, the error that occurred during PCD processing and edge point extraction was affected by the order of polynomial regression equation. The simulation technique offers advantages in cases where linking numerical analysis with digital images is challenging and when direct mechanical gauge measurement is difficult. In addition, it has potential applications in structural health monitoring and smart construction involving machine leading techniques.

A Novel Type of Boundary Extraction Method and Its Statistical Improvement for Unorganized Point Clouds Based on Concurrent Delaunay Triangular Meshes.

Currently, three-dimensional (3D) laser-scanned point clouds have been broadly applied in many important fields, such as non-contact measurements and reverse engineering. However, it is a huge challenge to efficiently and precisely extract the boundary features of unorganized point cloud data with strong randomness and distinct uncertainty. Therefore, a novel type of boundary extraction method will be developed based on concurrent Delaunay triangular meshes (CDTMs), which adds the vertex-angles of all CDTMs around a common data point together as an evaluation index to judge whether this targeted point will appear at boundary regions. Based on the statistical analyses on the CDTM numbers of every data point, another new type of CDTM-based boundary extraction method will be further improved by filtering out most of potential non-edge points in advance. Then these two CDTM-based methods and popular α-shape method will be employed in conducting boundary extractions on several point cloud datasets for comparatively analyzing and discussing their extraction accuracies and time consumptions in detail. Finally, all obtained results can strongly demonstrate that both these two CDTM-based methods present superior accuracies and strong robustness in extracting the boundary features of various unorganized point clouds, but the statistically improved version can greatly reduce time consumption.

Research on the Health Assessment Method of the Safety Retaining Wall in a Dump Based on UAV Point-Cloud Data.

The safety retaining wall is a critical infrastructure in ensuring the safety of both rock removal vehicles and personnel. However, factors such as precipitation infiltration, tire impact from rock removal vehicles, and rolling rocks can cause local damage to the safety retaining wall of the dump, rendering it ineffective in preventing rock removal vehicles from rolling down and posing a huge safety hazard. To address these issues, this study proposed a safety retaining wall health assessment method based on modeling and analysis of UAV point-cloud data of the safety retaining wall of a dump, which enables hazard warning for the safety retaining wall. The point-cloud data used in this study were obtained from the Qidashan Iron Mine Dump in Anshan City, Liaoning Province, China. Firstly, the point-cloud data of the dump platform and slope were extracted separately using elevation gradient filtering. Then, the point-cloud data of the unloading rock boundary was obtained via the ordered crisscrossed scanning algorithm. Subsequently, the point-cloud data of the safety retaining wall were extracted using the range constraint algorithm, and surface reconstruction was conducted to construct the Mesh model. The safety retaining wall mesh model was isometrically profiled to extract cross-sectional feature information and to compare the standard parameters of the safety retaining wall. Finally, the health assessment of the safety retaining wall was carried out. This innovative method allows for unmanned and rapid inspection of all areas of the safety retaining wall, ensuring the safety of rock removal vehicles and personnel.

A Novel Pallet Detection Method for Automated Guided Vehicles Based on Point Cloud Data.

Automated guided vehicles are widely used in warehousing environments for automated pallet handling, which is one of the fundamental parts to construct intelligent logistics systems. Pallet detection is a critical technology for automated guided vehicles, which directly affects production efficiency. A novel pallet detection method for automated guided vehicles based on point cloud data is proposed, which consists of five modules including point cloud preprocessing, key point extraction, feature description, surface matching and point cloud registration. The proposed method combines the color with the geometric features of the pallet point cloud and constructs a new Adaptive Color Fast Point Feature Histogram (ACFPFH) feature descriptor by selecting the optimal neighborhood adaptively. In addition, a new surface matching method called the Bidirectional Nearest Neighbor Distance Ratio-Approximate Congruent Triangle Neighborhood (BNNDR-ACTN) is proposed. The proposed method overcomes the problems of current methods such as low efficiency, poor robustness, random parameter selection, and being time-consuming. To verify the performance, the proposed method is compared with the traditional and modified Iterative Closest Point (ICP) methods in two real-world cases. The results show that the Root Mean Square Error (RMSE) is reduced to 0.009 and the running time is reduced to 0.989 s, which demonstrates that the proposed method has faster registration speed while maintaining higher registration accuracy.

Feature Analysis of Scanning Point Cloud of Structure and Research on Hole Repair Technology Considering Space-Ground Multi-Source 3D Data Acquisition.

As one of the best means of obtaining the geometry information of special shaped structures, point cloud data acquisition can be achieved by laser scanning or photogrammetry. However, there are some differences in the quantity, quality, and information type of point clouds obtained by different methods when collecting point clouds of the same structure, due to differences in sensor mechanisms and collection paths. Thus, this study aimed to combine the complementary advantages of multi-source point cloud data and provide the high-quality basic data required for structure measurement and modeling. Specifically, low-altitude photogrammetry technologies such as hand-held laser scanners (HLS), terrestrial laser scanners (TLS), and unmanned aerial systems (UAS) were adopted to collect point cloud data of the same special-shaped structure in different paths. The advantages and disadvantages of different point cloud acquisition methods of special-shaped structures were analyzed from the perspective of the point cloud acquisition mechanism of different sensors, point cloud data integrity, and single-point geometric characteristics of the point cloud. Additionally, a point cloud void repair technology based on the TLS point cloud was proposed according to the analysis results. Under the premise of unifying the spatial position relationship of the three point clouds, the M3C2 distance algorithm was performed to extract the point clouds with significant spatial position differences in the same area of the structure from the three point clouds. Meanwhile, the single-point geometric feature differences of the multi-source point cloud in the area with the same neighborhood radius was calculated. With the kernel density distribution of the feature difference, the feature points filtered from the HLS point cloud and the TLS point cloud were fused to enrich the number of feature points in the TLS point cloud. In addition, the TLS point cloud voids were located by raster projection, and the point clouds within the void range were extracted, or the closest points were retrieved from the other two heterologous point clouds, to repair the top surface and façade voids of the TLS point cloud. Finally, high-quality basic point cloud data of the special-shaped structure were generated.

An Entropy Analysis-Based Window Size Optimization Scheme for Merging LiDAR Data Frames.

LiDAR is a useful technology for gathering point cloud data from its environment and has been adapted to many applications. We use a cost-efficient LiDAR system attached to a moving object to estimate the location of the moving object using referenced linear structures. In the stationary state, the accuracy of extracting linear structures is low given the low-cost LiDAR. We propose a merging scheme for the LiDAR data frames to improve the accuracy by using the movement of the moving object. The proposed scheme tries to find the optimal window size by means of an entropy analysis. The optimal window size is determined by finding the minimum point between the entropy indicator of the ideal result and the entropy indicator of the actual result of each window size. The proposed indicator can describe the accuracy of the entire path of the moving object at each window size using a simple single value. The experimental results show that the proposed scheme can improve the linear structure extraction accuracy.

Pavement 3D Data Denoising Algorithm Based on Cell Meshing Ellipsoid Detection.

As a new measuring technique, laser 3D scanning technique has advantages of rapidity, safety, and accuracy. However, the measured result of laser scanning always contains some noise points due to the measuring principle and the scanning environment. These noise points will result in the precision loss during the 3D reconstruction. The commonly used denoising algorithms ignore the strong planarity feature of the pavement, and thus might mistakenly eliminate ground points. This study proposes an ellipsoid detection algorithm to emphasize the planarity feature of the pavement during the 3D scanned data denoising process. By counting neighbors within the ellipsoid neighborhood of each point, the threshold of each point can be calculated to distinguish if it is the ground point or the noise point. Meanwhile, to narrow down the detection space and to reduce the processing time, the proposed algorithm divides the cloud point into cells. The result proves that this denoising algorithm can identify and eliminate the scattered noise points and the foreign body noise points very well, providing precise data for later 3D reconstruction of the scanned pavement.

Miniaturised Low-Cost Gamma Scanning Platform for Contamination Identification, Localisation and Characterisation: A New Instrument in the Decommissioning Toolkit.

Formerly clandestine, abandoned and legacy nuclear facilities, whether associated with civil or military applications, represent a significant decommissioning challenge owing to the lack of knowledge surrounding the existence, location and types of radioactive material(s) that may be present. Consequently, mobile and highly deployable systems that are able to identify, spatially locate and compositionally assay contamination ahead of remedial actions are of vital importance. Deployment imposes constraints to dimensions resulting from small diameter access ports or pipes. Herein, we describe a prototype low-cost, miniaturised and rapidly deployable 'cell characterisation' gamma-ray scanning system to allow for the examination of enclosed (internal) or outdoor (external) spaces for radioactive 'hot-spots'. The readout from the miniaturised and lead-collimated gamma-ray spectrometer, that is progressively rastered through a stepped snake motion, is combined with distance measurements derived from a single-point laser range-finder to obtain an array of measurements in order to yield a 3-dimensional point-cloud, based on a polar coordinate system-scaled for radiation intensity. Existing as a smaller and more cost-effective platform than presently available, we are able to produce a millimetre-accurate 3D volumetric rendering of a space-whether internal or external, onto which fully spectroscopic radiation intensity data can be overlain to pinpoint the exact positions at which (even low abundance) gamma-emitting materials exist.

Adaptive and Efficient Mixture-Based Representation for Range Data.

Modern range sensors generate millions of data points per second, making it difficult to utilize all incoming data effectively in real time for devices with limited computational resources. The Gaussian mixture model (GMM) is a convenient and essential tool commonly used in many research domains. In this paper, an environment representation approach based on the hierarchical GMM structure is proposed, which can be utilized to model environments with weighted Gaussians. The hierarchical structure accelerates training by recursively segmenting local environments into smaller clusters. By adopting the information-theoretic distance and shape of probabilistic distributions, weighted Gaussians can be dynamically allocated to local environments in an arbitrary scale, leading to a full adaptivity in the number of Gaussians. Evaluations are carried out in terms of time efficiency, reconstruction, and fidelity using datasets collected from different sensors. The results demonstrate that the proposed approach is superior with respect to time efficiency while maintaining the high fidelity as compared to other state-of-the-art approaches.

Cross-Section Deformation Analysis and Visualization of Shield Tunnel Based on Mobile Tunnel Monitoring System.

With the ongoing developments in laser scanning technology, applications for describing tunnel deformation using rich point cloud data have become a significant topic of investigation. This study describes the independently developed CNU-TS-2 mobile tunnel monitoring system for data acquisition, which has an electric system to control its forward speed and is compatible with various laser scanners such as the Faro and Leica models. A comparison with corresponding data acquired by total station data demonstrates that the data collected by CNU-TS-2 is accurate. Following data acquisition, the overall and local deformation of the tunnel is determined by denoising and 360° deformation analysis of the point cloud data. To enhance the expression of the analysis results, this study proposes an expansion of the tunnel point cloud data into a two-dimensional image via cylindrical projection, followed by an expression of the tunnel deformation through color difference to visualize the deformation. Compared with the three-dimensional modeling method of visualization, this method is easier to implement and facilitates storage. In addition, it is conducive to the performance of comprehensive analysis of problems such as water leakage in the tunnel, thereby achieving the effect of multiple uses for a single image.

A LiDAR Point Cloud Data-Based Method for Evaluating Strain on a Curved Steel Plate Subjected to Lateral Pressure.

Structural health monitoring (SHM) and safety assessment are very important areas for evaluating the behavior of structures. Various wired and wireless sensors can measure the physical responses of structures, such as displacement or strain. One recently developed wireless technique is a light imaging detection and ranging (LiDAR) system that can remotely acquire three-dimensional (3D) high-precision coordinate information using 3D laser scanning. LiDAR systems have been previously used in geographic information systems (GIS) to collect information on geography and terrain. Recently, however, LiDAR is used in the SHM field to analyze structural behavior, as it can remotely detect the surface and deformation shape of structures without the need for attached sensors. This study demonstrates a strain evaluation method using a LiDAR system in order to analyze the behavior of steel structures. To evaluate the strains of structures from the initial and deformed shape, a combination of distributed 3D point cloud data and finite element methods (FEM) was used. The distributed 3D point cloud data were reconstructed into a 3D mesh model, and strains were calculated using the FEM. By using the proposed method, the strain could be calculated at any point on a structure for SHM and safety assessment during construction.

On-Site 4-in-1 Alignment: Visualization and Interactive CAD Model Retrofitting Using UAV, LiDAR's Point Cloud Data, and Video.

Acquisition of 3D point cloud data (PCD) using a laser scanner and aligning it with a video frame is a new approach that is efficient for retrofitting comprehensive objects in heavy pipeline industrial facilities. This work contributes a generic framework for interactive retrofitting in a virtual environment and an unmanned aerial vehicle (UAV)-based sensory setup design to acquire PCD. The framework adopts a 4-in-1 alignment using a point cloud registration algorithm for a pre-processed PCD alignment with the partial PCD, and frame-by-frame registration method for video alignment. This work also proposes a virtual interactive retrofitting framework that uses pre-defined 3D computer-aided design models (CAD) with a customized graphical user interface (GUI) and visualization of a 4-in-1 aligned video scene from a UAV camera in a desktop environment. Trials were carried out using the proposed framework in a real environment at a water treatment facility. A qualitative and quantitative study was conducted to evaluate the performance of the proposed generic framework from participants by adopting the appropriate questionnaire and retrofitting task-oriented experiment. Overall, it was found that the proposed framework could be a solution for interactive 3D CAD model retrofitting on a combination of UAV sensory setup-acquired PCD and real-time video from the camera in heavy industrial facilities.

Virtualizing living and working spaces: Proof of concept for a biomedical space-replication methodology.

The physical spaces within which the work of health occurs - the home, the intensive care unit, the emergency room, even the bedroom - influence the manner in which behaviors unfold, and may contribute to efficacy and effectiveness of health interventions. Yet the study of such complex workspaces is difficult. Health care environments are complex, chaotic workspaces that do not lend themselves to the typical assessment approaches used in other industrial settings. This paper provides two methodological advances for studying internal health care environments: a strategy to capture salient aspects of the physical environment and a suite of approaches to visualize and analyze that physical environment. We used a Faro™ laser scanner to obtain point cloud data sets of the internal aspects of home environments. The point cloud enables precise measurement, including the location of physical boundaries and object perimeters, color, and light, in an interior space that can be translated later for visualization on a variety of platforms. The work was motivated by vizHOME, a multi-year program to intensively examine the home context of personal health information management in a way that minimizes repeated, intrusive, and potentially disruptive in vivo assessments. Thus, we illustrate how to capture, process, display, and analyze point clouds using the home as a specific example of a health care environment. Our work presages a time when emerging technologies facilitate inexpensive capture and efficient management of point cloud data, thus enabling visual and analytical tools for enhanced discharge planning, new insights for designers of consumer-facing clinical informatics solutions, and a robust approach to context-based studies of health-related work environments.