GCN-Based LSTM Autoencoder with Self-Attention for Bearing Fault Diagnosis.

The manufacturing industry has been operating within a constantly evolving technological environment, underscoring the importance of maintaining the efficiency and reliability of manufacturing processes. Motor-related failures, especially bearing defects, are common and serious issues in manufacturing processes. Bearings provide accurate and smooth movements and play essential roles in mechanical equipment with shafts. Given their importance, bearing failure diagnosis has been extensively studied. However, the imbalance in failure data and the complexity of time series data make diagnosis challenging. Conventional AI models (convolutional neural networks (CNNs), long short-term memory (LSTM), support vector machine (SVM), and extreme gradient boosting (XGBoost)) face limitations in diagnosing such failures. To address this problem, this paper proposes a bearing failure diagnosis model using a graph convolution network (GCN)-based LSTM autoencoder with self-attention. The model was trained on data extracted from the Case Western Reserve University (CWRU) dataset and a fault simulator testbed. The proposed model achieved 97.3% accuracy on the CWRU dataset and 99.9% accuracy on the fault simulator dataset.

Objective Falls Risk Assessment Using Markerless Motion Capture and Representational Machine Learning.

Falls are a major issue for those over the age of 65 years worldwide. Objective assessment of fall risk is rare in clinical practice. The most common methods of assessment are time-consuming observational tests (clinical tests). Computer-aided diagnosis could be a great help. A popular clinical test for fall risk is the five times sit-to-stand. The time taken to complete the test is the most commonly used metric to identify the most at-risk patients. However, tracking the movement of skeletal joints can provide much richer insights. We use markerless motion capture, allied with a representational model, to identify those at risk of falls. Our method uses an LSTM autoencoder to derive a distance measure. Using this measure, we introduce a new scoring system, allowing individuals with differing falls risks to be placed on a continuous scale. Evaluating our method on the KINECAL dataset, we achieved an accuracy of 0.84 in identifying those at elevated falls risk. In addition to identifying potential fallers, our method could find applications in rehabilitation. This aligns with the goals of the KINECAL Dataset. KINECAL contains the recordings of 90 individuals undertaking 11 movements used in clinical assessments. KINECAL is labelled to disambiguate age-related decline and falls risk.

Anomaly detection in groundwater monitoring data using LSTM-Autoencoder neural networks.

Groundwater monitoring data can be prone to errors and biases due to various factors like borehole and equipment malfunctions, or human mistakes. These inaccuracies can jeopardize the groundwater system, leading to reduced efficiency and potentially causing partial or complete failures in the monitoring system. Traditional anomaly detection methods, which rely on statistical and time-variant techniques, struggle to handle the complex and dynamic nature of anomalies. With advancements in artificial intelligence and the growing need for effective anomaly detection and prevention across different sectors, artificial neural network methods are emerging as capable of identifying more intricate anomalies by considering both temporal and contextual aspects. Nonetheless, there is still a shortage of comprehensive studies on groundwater anomaly detection. The intricate patterns of sequential data from groundwater present numerous challenges, necessitating sophisticated modeling techniques that combine mathematics, statistics, and machine learning for viable solutions. This paper introduces a model designed for high accuracy and efficient computation in detecting anomalies in groundwater monitoring data through a probabilistic approach. We employed the Monte Carlo method and SEAWAT numerical simulation to ascertain the uncertainty in groundwater salinity. Subsequently, a Long Short-Term Memory (LSTM)-Autoencoder model was trained and evaluated, forming the basis of an anomaly detection framework. Each piece of training data was assessed by the LSTM-Autoencoder using the Negative Log Likelihood (NLL) score and a predefined threshold to determine the data's abnormality percentage. The accuracy evaluation of the proposed LSTM-Autoencoder algorithm revealed that this approach achieved commendable performance, with an accuracy of 98.47% in anomaly detection.

LSTM-Autoencoder Based Anomaly Detection Using Vibration Data of Wind Turbines.

The problem of energy depletion has brought wind energy under consideration to replace oil- or chemical-based energy. However, the breakdown of wind turbines is a major concern. Accordingly, unsupervised learning was performed using the vibration signal of a wind power generator to achieve an outlier detection performance of 97%. We analyzed the vibration data through wavelet packet conversion and identified a specific frequency band that showed a large difference between the normal and abnormal data. To emphasize these specific frequency bands, high-pass filters were applied to maximize the difference. Subsequently, the dimensions of the data were reduced through principal component analysis, giving unique characteristics to the data preprocessing process. Normal data collected from a wind farm located in northern Sweden was first preprocessed and trained using a long short-term memory (LSTM) autoencoder to perform outlier detection. The LSTM Autoencoder is a model specialized for time-series data that learns the patterns of normal data and detects other data as outliers. Therefore, we propose a method for outlier detection through data preprocessing and unsupervised learning, utilizing the vibration signals from wind generators. This will facilitate the quick and accurate detection of wind power generator failures and provide alternatives to the problem of energy depletion.

The Prediction of the Remaining Useful Life of Rotating Machinery Based on an Adaptive Maximum Second-Order Cyclostationarity Blind Deconvolution and a Convolutional LSTM Autoencoder.

The prediction of the remaining useful life (RUL) is important for the conditions of rotating machinery to maintain reliability and decrease losses. This study proposes an efficient approach based on an adaptive maximum second-order cyclostationarity blind deconvolution (ACYCBD) and a convolutional LSTM autoencoder to achieve the feature extraction, health index analysis, and RUL prediction for rotating machinery. First, the ACYCBD is used to filter noise from the vibration signals. Second, based on the peak value properties, a novel health index (HI) is designed to analyze the health conditions for the denoising signal, showing a high sensitivity for the degradation of bearings. Finally, for better prognostics and health management of the rotating machinery, based on convolutional layers and LSTM, an autoencoder can achieve a transform convolutional LSTM network to develop a convolutional LSTM autoencoder (ALSTM) model that can be applied to forecast the health trend for rotating machinery. Compared with the SVM, CNN, LSTM, GRU, and DTGRU methods, our experiments demonstrate that the proposed approach has the greatest performance for the prediction of the remaining useful life of rotating machinery.

TimeTector: A Twin-Branch Approach for Unsupervised Anomaly Detection in Livestock Sensor Noisy Data (TT-TBAD).

Unsupervised anomaly detection in multivariate time series sensor data is a complex task with diverse applications in different domains such as livestock farming and agriculture (LF&A), the Internet of Things (IoT), and human activity recognition (HAR). Advanced machine learning techniques are necessary to detect multi-sensor time series data anomalies. The primary focus of this research is to develop state-of-the-art machine learning methods for detecting anomalies in multi-sensor data. Time series sensors frequently produce multi-sensor data with anomalies, which makes it difficult to establish standard patterns that can capture spatial and temporal correlations. Our innovative approach enables the accurate identification of normal, abnormal, and noisy patterns, thus minimizing the risk of misinterpreting models when dealing with mixed noisy data during training. This can potentially result in the model deriving incorrect conclusions. To address these challenges, we propose a novel approach called "TimeTector-Twin-Branch Shared LSTM Autoencoder" which incorporates several Multi-Head Attention mechanisms. Additionally, our system now incorporates the Twin-Branch method which facilitates the simultaneous execution of multiple tasks, such as data reconstruction and prediction error, allowing for efficient multi-task learning. We also compare our proposed model to several benchmark anomaly detection models using our dataset, and the results show less error (MSE, MAE, and RMSE) in reconstruction and higher accuracy scores (precision, recall, and F1) against the baseline models, demonstrating that our approach outperforms these existing models.

Compression of EEG signals with the LSTM-autoencoder via domain adaptation approach.

The successful implementation of neural network-based EEG signal compression has led to significant cost reductions in data transmission. However, a major obstacle in this process arises from the decline in performance when compressing EEG signals from multiple subjects. This challenge arises due to the notable feature shift of EEG signals between subjects, which poses an impediment to the neural network's efficient concurrent acquisition of information from multiple subjects. To address this limitation and enable more effective utilization of data for improving the performance on target domain, we propose a Domain Adaptation (DA) framework based on LSTM-autoencoder. Our experiments encompassed the following: (1) A comparison between LSTM-autoencoder, GRU-autoencoder, and the commonly used convolutional autoencoder (CAE) in EEG compression. (2) A comparison between our proposed DA method and the MMD-based DA method, as well as Fine-tuning transfer learning. The results demonstrate the following: (1) LSTM-autoencoder outperforms other models in both subject-specific and cross-subject scenarios. (2) Using transfer learning improves the performance of LSTM-autoencoder on the target subject. (3) Our proposed method outperforms maximum mean discrepancy (MMD)-based domain adaptation and fine-tuning approaches, resulting in a more significant enhancement.

Proactive Fault Diagnosis of a Radiator: A Combination of Gaussian Mixture Model and LSTM Autoencoder.

Radiator reliability is crucial in environments characterized by high temperatures and friction, where prompt interventions are often required to prevent system failures. This study introduces a proactive approach to radiator fault diagnosis, leveraging the integration of the Gaussian Mixture Model and Long-Short Term Memory autoencoders. Vibration signals from radiators were systematically collected through randomized durability vibration bench tests, resulting in four operating states-two normal, one unknown, and one faulty. Time-domain statistical features of these signals were extracted and subjected to Principal Component Analysis to facilitate efficient data interpretation. Subsequently, this study discusses the comparative effectiveness of the Gaussian Mixture Model and Long Short-Term Memory in fault detection. Gaussian Mixture Models are deployed for initial fault classification, leveraging their clustering capabilities, while Long-Short Term Memory autoencoders excel in capturing time-dependent sequences, facilitating advanced anomaly detection for previously unencountered faults. This alignment offers a potent and adaptable solution for radiator fault diagnosis, particularly in challenging high-temperature or high-friction environments. Consequently, the proposed methodology not only provides a robust framework for early-stage fault diagnosis but also effectively balances diagnostic capabilities during operation. Additionally, this study presents the foundation for advancing reliability life assessment in accelerated life testing, achieved through dynamic threshold adjustments using both the absolute log-likelihood distribution of the Gaussian Mixture Model and the reconstruction error distribution of the Long-Short Term Memory autoencoder model.

A graph empowered insider threat detection framework based on daily activities.

While threats from outsiders are easier to alleviate, effective ways seldom exist to handle threats from insiders. The key to managing insider threats lies in engineering behavioral features efficiently and classifying them correctly. To handle challenges in feature engineering, we propose an integrated feature engineering solution based on daily activities, combining manually-selected features and automatically-extracted features together. Particularly, an LSTM auto-encoder is introduced for automatic feature engineering from sequential activities. To improve detection, a residual hybrid network (ResHybnet) containing GNN and CNN components is also proposed along with an organizational graph, taking a user-day combination as a node. Experimental results show that the proposed LSTM auto-encoder could extract hidden patterns from sequential activities efficiently, improving F1 score by 0.56%. Additionally, with the designed residual link, our ResHybnet model works well to boost performance and has outperformed the best of other models by 1.97% on the same features. We published our code on GitHub: https://github.com/Wayne-on-the-road/ResHybnet.

A deep LSTM autoencoder-based framework for predictive maintenance of a proton radiotherapy delivery system.

INTRODUCTION: Unscheduled machine downtime can cause treatment interruptions and adversely impact patient treatment outcomes. Conventional Quality Assurance (QA) programs of a proton Pencil Beam Scanning (PBS) system ensure its operational performance by keeping the beam parameters within clinical tolerances but often do not reveal the underlying issues of the device prior to a machine malfunction event. In this study, we propose a Predictive Maintenance (PdM) approach that leverages an advanced analytical tool built on a deep neural network to detect treatment delivery machine issues early. METHODS: Beam delivery log file data from daily QA performed at the Burr Proton Center of Massachusetts General Hospital were collected. A novel PdM framework consisting of long short-term memory-based autoencoder (LSTM-AE) modeling of the proton PBS delivery system and a Mahalanobis distance-based error metric evaluation was constructed to detect rare anomalous machine events. These included QA beam pauses, clinical operational issues, and treatment interruptions. The model was trained in an unsupervised fashion on the QA data of normal sessions so that the model learned characteristics of normal machine operation. The anomaly is quantified as the multivariate deviation between the model predicted data and the measured data of the day using Mahalanobis distance (M-Score). Two-layer and three-layer Long short-term memory-based stacked autoencoder (LSTM-SAE) models were optimized for exploring model performance improvement. Model validation was performed with two clinical datasets and was analyzed using the area under the precision-recall curve (AUPRC) and the area under the receiver operating characteristic (AUROC). RESULTS: LSTM-SAE models showed strong performance in predicting QA beam pauses for both clinical validation datasets. Despite severe skew in the dataset, the model achieved AUPRC of 0.60 and 0.82 and AUROC of 0.75 and 0.92 in the respective 2018 and 2020 datasets. Moreover, these amount to 2.8-fold and 10.7-fold enhancement compared to the respective baseline event rates. In addition, in terms of treatment interruption events, model prediction enabled 3.88-fold and 51.2-fold detection improvement, while the detection improvement for clinical operational issues was 1.04-fold and 1.37-fold, respectively, in the 2018 and 2020 datasets. CONCLUSION: Our novel deep LSTM-SAE-based framework allows for highly discriminative prediction of anomalous machine events and demonstrates great promise for enabling PdM for proton PBS beam delivery.

A Novel Approach on Deep Learning-Based Decision Support System Applying Multiple Output LSTM-Autoencoder: Focusing on Identifying Variations by PHSMs' Effect over COVID-19 Pandemic.

Following the outbreak of the COVID-19 pandemic, the continued emergence of major variant viruses has caused enormous damage worldwide by generating social and economic ripple effects, and the importance of PHSMs (Public Health and Social Measures) is being highlighted to cope with this severe situation. Accordingly, there has also been an increase in research related to a decision support system based on simulation approaches used as a basis for PHSMs. However, previous studies showed limitations impeding utilization as a decision support system for policy establishment and implementation, such as the failure to reflect changes in the effectiveness of PHSMs and the restriction to short-term forecasts. Therefore, this study proposes an LSTM-Autoencoder-based decision support system for establishing and implementing PHSMs. To overcome the limitations of existing studies, the proposed decision support system used a methodology for predicting the number of daily confirmed cases over multiple periods based on multiple output strategies and a methodology for rapidly identifying varies in policy effects based on anomaly detection. It was confirmed that the proposed decision support system demonstrated excellent performance compared to models used for time series analysis such as statistical models and deep learning models. In addition, we endeavored to increase the usability of the proposed decision support system by suggesting a transfer learning-based methodology that can efficiently reflect variations in policy effects. Finally, the decision support system proposed in this study provides a methodology that provides multi-period forecasts, identifying variations in policy effects, and efficiently reflects the effects of variation policies. It was intended to provide reasonable and realistic information for the establishment and implementation of PHSMs and, through this, to yield information expected to be highly useful, which had not been provided in the decision support systems presented in previous studies.

Anomaly Detection in COVID-19 Time-Series Data.

Anomaly detection and explanation in big volumes of real-world medical data, such as those pertaining to COVID-19, pose some challenges. First, we are dealing with time-series data. Typical time-series data describe behavior of a single object over time. In medical data, we are dealing with time-series data belonging to multiple entities. Thus, there may be multiple subsets of records such that records in each subset, which belong to a single entity are temporally dependent, but the records in different subsets are unrelated. Moreover, the records in a subset contain different types of attributes, some of which must be grouped in a particular manner to make the analysis meaningful. Anomaly detection techniques need to be customized for time-series data belonging to multiple entities. Second, anomaly detection techniques fail to explain the cause of outliers to the experts. This is critical for new diseases and pandemics where current knowledge is insufficient. We propose to address these issues by extending our existing work called IDEAL, which is an LSTM-autoencoder based approach for data quality testing of sequential records, and provides explanations of constraint violations in a manner that is understandable to end-users. The extension (1) uses a novel two-level reshaping technique that splits COVID-19 data sets into multiple temporally-dependent subsequences and (2) adds a data visualization plot to further explain the anomalies and evaluate the level of abnormality of subsequences detected by IDEAL. We performed two systematic evaluation studies for our anomalous subsequence detection. One study uses aggregate data, including the number of cases, deaths, recovered, and percentage of hospitalization rate, collected from a COVID tracking project, New York Times, and Johns Hopkins for the same time period. The other study uses COVID-19 patient medical records obtained from Anschutz Medical Center health data warehouse. The results are promising and indicate that our techniques can be used to detect anomalies in large volumes of real-world unlabeled data whose accuracy or validity is unknown.

Detection of price bubbles in Istanbul housing market using LSTM autoencoders: a district-based approach.

Since the early 2000s, there has been a long-term price increase trend in the Istanbul housing market, and this situation also has led to price bubble speculations. Since the housing sector was caught with a high level of unsold housing stock to the economic slowdown emerging in the second half of 2018, housing price bubble speculations have increased even more, especially for the Istanbul market. In this period, housing loan interest reduction campaigns were implemented by the government through state banks to stimulate the housing demand, and a probable collapse in the housing market was prevented. On the other hand, house prices continued to rise during this period due to the stimulated demand. In this paper, we perform a price bubble research on the selected districts in the Istanbul housing market over the 2007-2019 period using LSTM autoencoder model. The first analysis on monthly data is performed by using housing price index, housing rent index, consumer prices index, stock market index, return on government debt securities, USD/TRY exchange rates, BIST price index, monthly deposit interest rates, mortgage interest rates and consumer confidence index, and the second analysis on quarterly data is carried out by adding building construction cost index and GDP data to the previous dataset. In the first analysis, the bubble formations differ regionally and periodically and disappeared toward the end of 2019 in some districts, while in the second analysis, the housing bubble formations have a more common and continuous appearance. Experimental results show that LSTM autoencoder model can be used to detect housing bubbles effectively.

Sensor and Component Fault Detection and Diagnosis for Hydraulic Machinery Integrating LSTM Autoencoder Detector and Diagnostic Classifiers.

Anomaly occurrences in hydraulic machinery might lead to massive system shut down, jeopardizing the safety of the machinery and its surrounding human operator(s) and environment, and the severe economic implications following the faults and their associated damage. Hydraulics are mostly placed in ruthless environments, where they are consistently vulnerable to many faults. Hence, not only are the machines and their components prone to anomalies, but also the sensors attached to them, which monitor and report their health and behavioral changes. In this work, a comprehensive applicational analysis of anomalies in hydraulic systems extracted from a hydraulic test rig was thoroughly achieved. First, we provided a combination of a new architecture of LSTM autoencoders and supervised machine and deep learning methodologies, to perform two separate stages of fault detection and diagnosis. The two phases were condensed by-the detection phase using the LSTM autoencoder. Followed by the fault diagnosis phase represented by the classification schema. The previously mentioned framework was applied to both component and sensor faults in hydraulic systems, deployed in the form of two in-depth applicational experiments. Moreover, a thorough literature review of related work from the past decade, for autoencoders related fault detection and diagnosis in hydraulic systems, was successfully conducted in this study.