Advanced Image Analytics for Mobile Robot-Based Condition Monitoring in Hazardous Environments: A Comprehensive Thermal Defect Processing Framework.

In hazardous environments like mining sites, mobile inspection robots play a crucial role in condition monitoring (CM) tasks, particularly by collecting various kinds of data, such as images. However, the sheer volume of collected image samples and existing noise pose challenges in processing and visualizing thermal anomalies. Recognizing these challenges, our study addresses the limitations of industrial big data analytics for mobile robot-generated image data. We present a novel, fully integrated approach involving a dimension reduction procedure. This includes a semantic segmentation technique utilizing the pre-trained VGG16 CNN architecture for feature selection, followed by random forest (RF) and extreme gradient boosting (XGBoost) classifiers for the prediction of the pixel class labels. We also explore unsupervised learning using the PCA-K-means method for dimension reduction and classification of unlabeled thermal defects based on anomaly severity. Our comprehensive methodology aims to efficiently handle image-based CM tasks in hazardous environments. To validate its practicality, we applied our approach in a real-world scenario, and the results confirm its robust performance in processing and visualizing thermal data collected by mobile inspection robots. This affirms the effectiveness of our methodology in enhancing the overall performance of CM processes.

Analysis of Lidar Actuator System Influence on the Quality of Dense 3D Point Cloud Obtained with SLAM.

Mobile mapping technologies, based on techniques such as simultaneous localization and mapping (SLAM) and surface-from-motion (SfM), are being vigorously developed both in the scientific community and in industry. They are crucial concepts for automated 3D surveying and autonomous vehicles. For various applications, rotating multiline scanners, manufactured, for example, by Velodyne and Ouster, are utilized as the main sensor of the mapping hardware system. However, their principle of operation has a substantial drawback, as their scanning pattern creates natural gaps between the scanning lines. In some models, the vertical lidar field of view can also be severely limited. To overcome these issues, more sensors could be employed, which would significantly increase the cost of the mapping system. Instead, some investigators have added a tilting or rotating motor to the lidar. Although the effectiveness of such a solution is usually clearly visible, its impact on the quality of the acquired 3D data has not yet been investigated. This paper presents an adjustable mapping system, which allows for switching between a stable, tilting or fully rotating lidar position. A simple experiment in a building corridor was performed, simulating the conditions of a mobile robot passing through a narrow tunnel: a common setting for applications, such as mining surveying or industrial facility inspection. A SLAM algorithm is utilized to create a coherent 3D point cloud of the mapped corridor for three settings of the sensor movement. The extent of improvement in the 3D data quality when using the tilting and rotating lidar, compared to keeping a stable position, is quantified. Different metrics are proposed to account for different aspects of the 3D data quality, such as completeness, density and geometry coherence. The ability of SLAM algorithms to faithfully represent selected objects appearing in the mapped scene is also examined. The results show that the fully rotating solution is optimal in terms of most of the metrics analyzed. However, the improvement observed from a horizontally mounted sensor to a tilting sensor was the most significant.

Automated Identification of Overheated Belt Conveyor Idlers in Thermal Images with Complex Backgrounds Using Binary Classification with CNN.

Mechanical industrial infrastructures in mining sites must be monitored regularly. Conveyor systems are mechanical systems that are commonly used for safe and efficient transportation of bulk goods in mines. Regular inspection of conveyor systems is a challenging task for mining enterprises, as conveyor systems' lengths can reach tens of kilometers, where several thousand idlers need to be monitored. Considering the harsh environmental conditions that can affect human health, manual inspection of conveyor systems can be extremely difficult. Hence, the authors proposed an automatic robotics-based inspection for condition monitoring of belt conveyor idlers using infrared images, instead of vibrations and acoustic signals that are commonly used for condition monitoring applications. The first step in the whole process is to segment the overheated idlers from the complex background. However, classical image segmentation techniques do not always deliver accurate results in the detection of target in infrared images with complex backgrounds. For improving the quality of captured infrared images, preprocessing stages are introduced. Afterward, an anomaly detection method based on an outlier detection technique is applied to the preprocessed image for the segmentation of hotspots. Due to the presence of different thermal sources in mining sites that can be captured and wrongly identified as overheated idlers, in this research, we address the overheated idler detection process as an image binary classification task. For this reason, a Convolutional Neural Network (CNN) was used for the binary classification of the segmented thermal images. The accuracy of the proposed condition monitoring technique was compared with our previous research. The metrics for the previous methodology reach a precision of 0.4590 and an F1 score of 0.6292. The metrics for the proposed method reach a precision of 0.9740 and an F1 score of 0.9782. The proposed classification method considerably improved our previous results in terms of the true identification of overheated idlers in the presence of complex backgrounds.

Fuzzy Risk-Based Maintenance Strategy with Safety Considerations for the Mining Industry.

Enterprises today are increasingly seeking maintenance management strategies to ensure that their machines run faultlessly. This problem is particularly relevant in the mining sector, due to the demanding working conditions of underground mines and machines and equipment-operating regimes. Therefore, in this article, the authors proposed a new approach to mining machinery maintenance management, based on the concept of risk-based maintenance (RBM) and taking into account safety issues. The proposed method includes five levels of analysis, of which the first level focuses on hazard analysis, while the next three are connected with a risk evaluation. The final level relates to determining the RBM recommendations. The recommendations are defined in relation to the three main improvement areas: maintenance, safety, and resource availability/allocation. The proposed approach is based on the use of fuzzy logic. To present the possibilities of implementing our method, a case study covering the operation of selected mining machinery in a selected Polish underground mine is presented. In the case of mining machinery, fourteen adverse-event scenarios were identified and investigated; general recommendations were also given. The authors have also indicated further directions of research work to optimize system maintenance strategies, based on the concept of risk-based maintenance. Additionally, the discussion about the implementation possibilities of the approach developed herein is provided.

Divergence-Based Segmentation Algorithm for Heavy-Tailed Acoustic Signals with Time-Varying Characteristics.

Many real-world systems change their parameters during the operation. Thus, before the analysis of the data, there is a need to divide the raw signal into parts that can be considered as homogeneous segments. In this paper, we propose a segmentation procedure that can be applied for the signal with time-varying characteristics. Moreover, we assume that the examined signal exhibits impulsive behavior, thus it corresponds to the so-called heavy-tailed class of distributions. Due to the specific behavior of the data, classical algorithms known from the literature cannot be used directly in the segmentation procedure. In the considered case, the transition between parts corresponding to homogeneous segments is smooth and non-linear. This causes that the segmentation algorithm is more complex than in the classical case. We propose to apply the divergence measures that are based on the distance between the probability density functions for the two examined distributions. The novel segmentation algorithm is applied to real acoustic signals acquired during coffee grinding. Justification of the methodology has been performed experimentally and using Monte-Carlo simulations for data from the model with heavy-tailed distribution (here the stable distribution) with time-varying parameters. Although the methodology is demonstrated for a specific case, it can be extended to any process with time-changing characteristics.

Vibration-Based Diagnostics of Radial Clearances and Bolts Loosening in the Bearing Supports of the Heavy-Duty Gearboxes.

The problem solved in this research is the diagnosis of the radial clearances in bearing supports and the loosening of fastening bolts due to their plastic elongation (creep) or weak tightening using vibration signals. This is an important issue for the maintenance of the heavy-duty gearboxes of powerful mining machines and rolling mills working in non-stationary regimes. Based on a comprehensive overview of bolted joint diagnostic methods, a solution to this problem based on a developed nonlinear dynamical model of bearing supports is proposed. Diagnostic rules are developed by comparing the changes of natural frequency and its harmonics, the amplitudes and phases of shaft transient oscillations. Then, the vibration signals are measured on real gearboxes while the torque is increasing in the transmission during several series of industrial trials under changing bearings and bolts conditions. In parallel, dynamical torque is measured and its interrelation with vibration is determined. It is concluded that the radial clearances are the most influencing factors among the failure parameters in heavy-duty gearboxes of industrial machines working under impulsive and step-like loading. The developed diagnostics algorithm allows condition monitoring of bearings and fastening bolts, allowing one to undertake timely maintenance actions to prevent failures.

Decision Tree-Based Classification for Planetary Gearboxes' Condition Monitoring with the Use of Vibration Data in Multidimensional Symptom Space.

Monitoring the condition of rotating machinery, especially planetary gearboxes, is a challenging problem. In most of the available approaches, diagnostic procedures are related to advanced signal pre-processing/feature extraction methods or advanced data (features) analysis by using artificial intelligence. In this paper, the second approach is explored, so an application of decision trees for the classification of spectral-based 15D vectors of diagnostic data is proposed. The novelty of this paper is that by a combination of spectral analysis and the application of decision trees to a set of spectral features, we are able to take advantage of the multidimensionality of diagnostic data and classify/recognize the gearbox condition almost faultlessly even in non-stationary operating conditions. The diagnostics of time-varying systems are a complicated issue due to time-varying probability densities estimated for features. Using multidimensional data instead of an aggregated 1D feature, it is possible to improve the efficiency of diagnostics. It can be underlined that in comparison to previous work related to the same data, where the aggregated 1D variable was used, the efficiency of the proposed approach is around 99% (ca. 19% better). We tested several algorithms: classification and regression trees with the Gini index and entropy, as well as the random tree. We compare the obtained results with the K-nearest neighbors classification algorithm and meta-classifiers, namely: random forest and AdaBoost. As a result, we created the decision tree model with 99.74% classification accuracy on the test dataset.

Local Defect Detection in Bearings in the Presence of Heavy-Tailed Noise and Spectral Overlapping of Informative and Non-Informative Impulses.

The problem of the informative frequency band (IFB) selection for local fault detection is considered in the paper. There are various approaches that are very effective in this issue. Most of the techniques are vibration-based and they are related to the cyclic impulses detection (associated with the local fault) in the background noise. However, when the background noise in the vibration signal has non-Gaussian impulsive behavior, the classical methods seem to be insufficient. Recently, new techniques were proposed by several authors and interesting approaches were tested for different non-Gaussian signals. We demonstrate the comparative analysis related to the results for three selected techniques proposed in recent years, namely the Alpha selector, Conditional Variance-based selector, and Spearman selector. The techniques seem to be effective for the IFB selection for the non-Gaussian distributed vibration signals. The main purpose of this article is to investigate how spectral overlapping of informative and non-informative impulsive components will affect diagnostic procedures. According to our knowledge, this problem was not considered in the literature for the non-Gaussian signals. Nevertheless, as we demonstrated by the simulations, the level of overlapping and the location of a center frequency of the mentioned frequency bands have a significant influence on the behavior of the considered selectors. The discussion about the effectiveness of each analyzed method is conducted. The considered problem is supported by real-world examples.

Accuracy Evaluation of Selected Mobile Inspection Robot Localization Techniques in a GNSS-Denied Environment.

Locating an inspection robot is an essential task for inspection missions and spatial data acquisition. Giving a spatial reference to measurements, especially those concerning environmental parameters, e.g., gas concentrations may make them more valuable by enabling more insightful analyses. Thus, an accurate estimation of sensor position and orientation is a significant topic in mobile measurement systems used in robotics, remote sensing, or autonomous vehicles. Those systems often work in urban or underground conditions, which are lowering or disabling the possibility of using Global Navigation Satellite Systems (GNSS) for this purpose. Alternative solutions vary significantly in sensor configuration requirements, positioning accuracy, and computational complexity. The selection of the optimal solution is difficult. The focus here is put on the assessment, using the criterion of the positioning accuracy of the mobile robot with no use of GNSS signals. Automated geodetic surveying equipment is utilized for acquiring precise ground truth data of the robot's movement. The results obtained, with the use of several methods, compared: Wheel odometry, inertial measurement-based dead-reckoning, visual odometry, and trilateration of ultra-wideband signals. The suitability, pros, and cons of each method are discussed in the context of their application in autonomous robotic systems, operating in an underground mine environment.

Model of the Vibration Signal of the Vibrating Sieving Screen Suspension for Condition Monitoring Purposes.

Diagnostics of industrial machinery is a topic related to the need for damage detection, but it also allows to understand the process itself. Proper knowledge about the operational process of the machine, as well as identification of the underlying components, is critical for its diagnostics. In this paper, we present a model of the signal, which describes vibrations of the sieving screen. This particular type is used in the mining industry for the classification of ore pieces in the material stream by size. The model describes the real vibration signal measured on the spring set being the suspension of this machine. This way, it is expected to help in better understanding how the overall motion of the machine can impact the efforts of diagnostics. The analysis of real vibration signals measured on the screen allowed to identify and parameterize the key signal components, which carry valuable information for the following stages of diagnostic process of that machine. In the proposed model we take into consideration deterministic components related to shaft rotation, stochastic Gaussian component related to external noise, stochastic α-stable component as a model of excitations caused by falling rocks pieces, and identified machine response to unitary excitations.

Identification, Decomposition and Segmentation of Impulsive Vibration Signals with Deterministic Components-A Sieving Screen Case Study.

Condition monitoring is a well-established field of research; however, for industrial applications, one may find some challenges. They are mostly related to complex design, a specific process performed by the machine, time-varying load/speed conditions, and the presence of non-Gaussian noise. A procedure for vibration analysis from the sieving screen used in the raw material industry is proposed in the paper. It is more for pre-processing than the damage detection procedure. The idea presented here is related to identification and extraction of two main types of components: (i) deterministic (D)-related to the unbalanced shaft(s) and (ii) high amplitude, impulsive component randomly (R) appeared in the vibration due to pieces of ore falling down of moving along the deck. If we could identify these components, then we will be able to perform classical diagnostic procedures for local damage detection in rolling element bearing. As deterministic component may be AM/FM modulated and each impulse may appear with different amplitude and damping, there is a need for an automatic procedure. We propose a method for signal processing that covers two main steps: (a) related to R/D decomposition and including signal segmentation to neglect AM/FM modulations, iterative sine wave fitting using the least square method (for each segment), signal filtering technique by subtraction fitted sine from the raw signal, the definition of the criterion to stop iteration by residuals analysis, (b) impulse segmentation and description (beginning, end, max amplitude) that contains: detection of the number of impulses in a decomposed random part of the raw signal, detection of the max value of each impulse, statistical analysis (probability density function) of max value to find regime-switching), modeling of the envelope of each impulse for samples that protrude from the signal, extrapolation (forecasting) envelope shape for samples hidden in the signal. The procedure is explained using simulated and real data. Each step is very easy to implement and interpret thus the method may be used in practice in a commercial system.

A Simple Condition Monitoring Method for Gearboxes Operating in Impulsive Environments.

Reliable condition indicators are necessary to perform effective diagnosis and prognosis. However, the vibration signals are often corrupted with non-Gaussian noise and rotating machines may operate under time-varying operating conditions. This impedes the application of conventional condition indicators. The synchronous average of the squared envelope is a relatively simple yet effective method to perform fault detection, fault identification and fault trending under constant and time-varying operating conditions. However, its performance is impeded by the presence of impulsive signal components attributed to impulsive noise or the presence of other damage modes in the machine. In this work, it is proposed that the synchronous median of the squared envelope should be used instead of the synchronous average of the squared envelope for gearbox fault diagnosis. It is shown on numerical and experimental datasets that the synchronous median is more robust to the presence of impulsive signal components and is therefore more reliable for estimating the condition of specific machine components.

Identification and Statistical Analysis of Impulse-Like Patterns of Carbon Monoxide Variation in Deep Underground Mines Associated with the Blasting Procedure.

The quality of the air in underground mines is a challenging issue due to many factors, such as technological processes related to the work of miners (blasting, air conditioning, and ventilation), gas release by the rock mass and geometry of mine corridors. However, to allow miners to start their work, it is crucial to determine the quality of the air. One of the most critical parameters of the air quality is the carbon monoxide (CO) concentration. Thus, in this paper, we analyze the time series describing CO concentration. Firstly, the signal segmentation is proposed, then segmented data (daily patterns) is visualized and statistically analyzed. The method for blasting moment localization, with no prior knowledge, has been presented. It has been found that daily patterns differ and CO concentration values reach a safe level at a different time after blasting. The waiting time to achieve the safe level after blasting moment (with a certain probability) has been calculated based on the historical data. The knowledge about the nature of the CO variability and sources of a high CO concentration can be helpful in creating forecasting models, as well as while planning mining activities.

Semantic segmentation of thermal defects in belt conveyor idlers using thermal image augmentation and U-Net-based convolutional neural networks.

The belt conveyor (BC) is the main means of horizontal transportation of bulk materials at mining sites. The sudden fault in BC modules may cause unexpected stops in production lines. With the increasing number of applications of inspection mobile robots in condition monitoring (CM) of industrial infrastructure in hazardous environments, in this article we introduce an image processing pipeline for automatic segmentation of thermal defects in thermal images captured from BC idlers using a mobile robot. This study follows the fact that CM of idler temperature is an important task for preventing sudden breakdowns in BC system networks. We compared the performance of three different types of U-Net-based convolutional neural network architectures for the identification of thermal anomalies using a small number of hand-labeled thermal images. Experiments on the test data set showed that the attention residual U-Net with binary cross entropy as the loss function handled the semantic segmentation problem better than our previous research and other studied U-Net variations.

Diagnostics of Bolted Joints in Vibrating Screens Based on a Multi-Body Dynamical Model.

The condition-based maintenance of vibrating screens requires new methods of their elements' diagnostics due to severe disturbances in measured signals from vibrators and falling pieces of material. The bolted joints of the sieving deck, when failed, require a lot of time and workforce for repair. In this research, the authors proposed the model-based diagnostic method based on modal analysis of the 2-DOF system, which accounts for the interaction of the screen body and the upper deck under conditions of bolted joint degradation. It is shown that the second natural mode with an out-of-phase motion of the upper deck against the main screen housing may coincide with the excitation frequency or its higher harmonics, which appear when vibrators' bearings are in bad condition. This interaction speeds up bolt loosening and joint opening by the dynamical loading of higher amplitude. The proposed approach can be used to detune the system from resonance and anti-resonance to reduce maintenance costs and energy consumption. To prevent abrupt failures, such parameters as second natural mode frequency, damping factor, and phase space plot (PSP) distortion measures are proposed as bolt health indicators, and these are verified on the laboratory vibrating screen. Also, the robustness is tested by the impulsive non-Gaussian noise addition to the measurement data. A special diagram was proposed for the bolted joints' strength capacity assessment and maintenance actions planning (tightening, replacement), depending on clearance in the joints.

Condition Monitoring of Horizontal Sieving Screens-A Case Study of Inertial Vibrator Bearing Failure in Calcium Carbonate Production Plant.

Predictive maintenance is increasingly popular in many branches, as well as in the mining industry; however, there is a lack of spectacular examples of its practice efficiency. Close collaboration between Omya Group and Wroclaw University of Science and Technology allowed investigation of the failure of the inertial vibrator's bearing. The signals of vibration are captured from the sieving screen just before bearing failure and right after repair, when it was visually inspected after replacement. The additional complication was introduced by the loss of stable attachment of the vibrator's shield, which produced great periodical excitation in each place of measurement on the machine. Such anomalies in the signals, in addition to falling pieces of material, made impossible the diagnostics by standard methods. However, the implementation of advanced signal processing techniques such as time-frequency diagrams, envelope spectrum, cyclic spectral coherence, orbits analysis, and phase space plots allowed to undermine defects (pitting on the inner ring). After repair, the amplitudes of vibration from the damaged bearing side were reduced by five times, while sound pressure was only two times lower. The quantitative parameters of vibrations showed significant changes: time series RMS (-68%) median energy of spectrograms (89%), frequencies ratio of cyclic spectral coherence (-85%), and average amplitude of harmonics in envelope spectrum (-80%). The orbits demonstrated changes in inclination angle (16%) and sizes (-48, … -96%), as well as phase space plots sizes (-28, … -67%). Directions of further research are considered.

Application of spectral decomposition of ²²²Rn activity concentration signal series measured in Niedzwiedzia Cave to identification of mechanisms responsible for different time-period variations.

The authors present an application of spectral decomposition of (222)Rn activity concentration signal series as a mathematical tool used for distinguishing processes determining temporal changes of radon concentration in cave air. The authors demonstrate that decomposition of monitored signal such as (222)Rn activity concentration in cave air facilitates characterizing the processes affecting changes in the measured concentration of this gas. Thanks to this, one can better correlate and characterize the influence of various processes on radon behaviour in cave air. Distinguishing and characterising these processes enables the understanding of radon behaviour in cave environment and it may also enable and facilitate using radon as a precursor of geodynamic phenomena in the lithosphere. Thanks to the conducted analyses, the authors confirmed the unquestionable influence of convective air exchange between the cave and the atmosphere on seasonal and short-term (diurnal) changes in (222)Rn activity concentration in cave air. Thanks to the applied methodology of signal analysis and decomposition, the authors also identified a third process affecting (222)Rn activity concentration changes in cave air. This is a deterministic process causing changes in radon concentration, with a distribution different from the Gaussian one. The authors consider these changes to be the effect of turbulent air movements caused by the movement of visitors in caves. This movement is heterogeneous in terms of the number of visitors per group and the number of groups visiting a cave per day and per year. Such a process perfectly elucidates the observed character of the registered changes in (222)Rn activity concentration in one of the decomposed components of the analysed signal. The obtained results encourage further research into precise relationships between the registered (222)Rn activity concentration changes and factors causing them, as well as into using radon as a precursor of geodynamic phenomena in the lithosphere.