Deep belief improved bidirectional LSTM for multivariate time series forecasting.

Multivariate time series (MTS) play essential roles in daily life because most real-world time series datasets are multivariate and rich in time-dependent information. Traditional forecasting methods for MTS are time-consuming and filled with complicated limitations. One efficient method being explored within the dynamical systems is the extended short-term memory networks (LSTMs). However, existing MTS models only partially use the hidden spatial relationship as effectively as LSTMs. Shallow LSTMs are inadequate in extracting features from high-dimensional MTS; however, the multilayer bidirectional LSTM (BiLSTM) can learn more MTS features in both directions. This study tries to generate a novel and improved BiLSTM network (DBI-BiLSTM) based on a deep belief network (DBN), bidirectional propagation technique, and a chained structure. The deep structures are constructed by a DBN layer and multiple stacked BiLSTM layers, which increase the feature representation of DBI-BiLSTM and allow for the model to further learn the extended features in two directions. First, the input is processed by DBN to obtain comprehensive features. Then, the known features, divided into clusters based on a global sensitivity analysis method, are used as the inputs of every BiLSTM layer. Meanwhile, the previous outputs of the shallow layer are combined with the clustered features to reconstitute new input signals for the next deep layer. Four experimental real-world time series datasets illustrate our one-step-ahead prediction performance. The simulating results confirm that the DBI-BiLSTM not only outperforms the traditional shallow artificial neural networks (ANNs), deep LSTMs, and some recently improved LSTMs, but also learns more features of the MTS data. As compared with conventional LSTM, the percentage improvement of DBI-BiLSTM on the four MTS datasets is 85.41, 75.47, 61.66 and 30.72%, respectively.

Statistical monitoring for non-Gaussian processes based on MICA-KDR method.

The focus of the current work attempts to propose a purely data-based model for generating residuals for non-Gaussian process monitoring purposes, the idea of residual generation is borrowed from the field of model-based fault detection and applied in statistical monitoring, the generated residual instead of the measured variables is thus modeled and monitored. The proposed approach first employs the modified independent component analysis (MICA) algorithm to extract independent components (ICs) from a given dataset. Secondly, through assuming but only one variable is missing at one time, the known data regression (KDR) method dealing with missing data problem is then used for estimating the corresponding ICs. The inconsistency between the actual and estimated ICs is called residual and may present much lower level of non-Gaussianity, in contrast to the actual ICs. Thirdly, a principal component analysis based statistical monitoring model can be utilized for online fault detection based on the generated residual. Finally, the superiority and efficiency of the MICA-KDR approach over its counterparts are validated by implementing comparisons on two industrial processes, the proposed MICA-KDR method is demonstrated to be a comparative alternative in monitoring non-Gaussian processes.

A missing variable approach for decentralized statistical process monitoring.

The main focus of the current work is on the investigation and application of a missing variable approach in principal component analysis (PCA) model for decentralized process monitoring purpose. Given that the widely studied PCA algorithm can recover the correlations between measured variables, a missing variable approach is employed for computing score estimation error and residual estimation error from the developed PCA model. Through assuming but only one variable is missing in sequence, the residual between the actual and estimated components is generated and then monitored instead of the original data. The presented method implements a missing variable based offline modeling and online monitoring in a decentralized manner. Generally, the generated residual is expected to follow or at least become much closer to a Gaussian distribution, the resulted model has no restriction on Gaussian distributed dataset and can achieve salient monitoring performance in contrast to its counterparts. Finally, its superiority and effectiveness have been demonstrated by conducting comparisons on two industrial examples.

Double-layer ensemble monitoring of non-gaussian processes using modified independent component analysis.

The modified independent component analysis (MICA) was proposed mainly to obtain a consistent solution that cannot be ensured in the original ICA algorithm and has been widely investigated in multivariate statistical process monitoring (MSPM). Within the MICA-based non-Gaussian process monitoring circle, there are two main problems, i.e., the selection of a proper non-quadratic function for measuring non-Gaussianity and the determination of dominant ICs for constructing latent subspace, have not been well attempted so far. Given that the MICA method as well as other MSPM approaches are usually implemented in an unsupervised manner, the two problems are always solved by some empirical criteria without respect to enhancing fault detectability. The current work aims to address the challenging issues involved in the MICA-based approach and propose a double-layer ensemble monitoring method based on MICA (abbreviated as DEMICA) for non-Gaussian processes. Instead of proposing an approach for selecting a proper non-quadratic function and determining the dominant ICs, the DEMICA method combines all possible base MICA models developed with different non-quadratic functions and different sets of dominant ICs into an ensemble, and a double-layer Bayesian inference is formulated as a decision fusion method to form a unique monitoring index for online fault detection. The effectiveness of the proposed approach is then validated on two systems, and the achieved results clearly demonstrate its superior proficiency.

Statistical process monitoring based on orthogonal multi-manifold projections and a novel variable contribution analysis.

Multivariate statistical methods have been widely applied to develop data-based process monitoring models. Recently, a multi-manifold projections (MMP) algorithm was proposed for modeling and monitoring chemical industrial processes, the MMP is an effective tool for preserving the global and local geometric structure of the original data space in the reduced feature subspace, but it does not provide orthogonal basis functions for data reconstruction. Recognition of this issue, an improved version of MMP algorithm named orthogonal MMP (OMMP) is formulated. Based on the OMMP model, a further processing step and a different monitoring index are proposed to model and monitor the variation in the residual subspace. Additionally, a novel variable contribution analysis is presented for fault diagnosis by integrating the nearest in-control neighbor calculation and reconstruction-based contribution analysis. The validity and superiority of the proposed fault detection and diagnosis strategy are then validated through case studies on the Tennessee Eastman benchmark process.