A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures.

Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

Lifetime Estimation for Multi-Phase Deteriorating Process with Random Abrupt Jumps.

Owing to operating condition changing, physical mutation, and sudden shocks, degradation trajectories usually exhibit multi-phase features, and the abrupt jump often appears at the changing time, which makes the traditional methods of lifetime estimation unavailable. In this paper, we mainly focus on how to estimate the lifetime of the multi-phase degradation process with abrupt jumps at the change points under the concept of the first passage time (FPT). Firstly, a multi-phase degradation model with jumps based on the Wiener process is formulated to describe the multi-phase degradation pattern. Then, we attain the lifetime's closed-form expression for the two-phase model with fixed jump relying on the distribution of the degradation state at the change point. Furthermore, we continue to investigate the lifetime estimation of the degradation process with random effect caused by unit-to-unit variability and the multi-phase degradation process. We extend the results of the two-phase case with fixed parameters to these two cases. For better implementation, a model identification method with off-line and on-line parts based on Expectation Maximization (EM) algorithm and Bayesian rule is proposed. Finally, a numerical case study and a practical example of gyro are provided for illustration.

Interactive Prognosis Framework Between Deep Learning and a Stochastic Process Model for Remaining Useful Life Prediction.

Uncertainty quantification of the remaining useful life (RUL) for degraded systems under the big data era has been a hot topic in recent years. A general idea is to execute two separate steps: deep-learning-based health indicator (HI) construction and stochastic process-based degradation modeling. However, there exists a critical matching defect between the constructed HI and a degradation model, which seriously affects the RUL prediction accuracy. Toward this end, this article proposes an interactive prognosis framework between deep learning and a stochastic process model for the RUL prediction. First, we resort to stacked contractive autoencoders to fuse multiple sensor information of historical systems for constructing the HI in a typical unsupervised manner. Then, considering the nonlinear characteristic of the constructed HI, an exponential-like degradation model is introduced to construct its degradation evolving model, and theoretical expressions of the prediction results are derived under the concept of the first hitting time. Furthermore, we design an optimization objective function by integrating the HI construction and degradation modeling for the RUL prediction. To minimize the designed objective function of the proposed interactive prognosis framework, a gradient descent algorithm is employed to update the model parameters. Based on the well-trained interactive prognosis model, we can obtain the HI of a field system from stacked contractive autoencoders with sensor data and the probability density function (pdf) of the predicted RUL on the basis of the estimated parameters. Finally, the effectiveness and superiority of the proposed interactive prognosis method are verified by two case studies associated with turbofan engines.

Robust Sliding Mode-Based Learning Control for MIMO Nonlinear Nonminimum Phase System in General Form.

The tracking control of a multi-input multioutput nonlinear nonminimum phase system in general form is discussed. This system is assumed to be suffering from parameter uncertainties and unmodeled dynamics, and the priori information of them is unknown. By considering both the exact model and uncertain model, the sliding mode-based learning controller is proposed. By designing an appropriate sliding surface and a learning controller, the stability of the closed-loop system is guaranteed for both the exact model and uncertain model. To overcome the disadvantage caused by parameter uncertainties and unmodeled dynamics, a fuzzy logical system is adopted here. A numerical simulation result carried on vertical takeoff and landing aircraft is taken as an example to validate the effectiveness of the presented controller.

A Novel Unified and Self-Stabilizing Algorithm for Generalized Eigenpairs Extraction.

Generalized eigendecomposition problem has been widely employed in many signal processing applications. In this paper, we propose a unified and self-stabilizing algorithm, which is able to extract the first principal and minor generalized eigenvectors of a matrix pencil of two vector sequences adaptively. Furthermore, we extend the proposed algorithm to extract multiple generalized eigenvectors. The performance analysis shows that only the desired equilibrium point of the proposed algorithm is stable and all others are (unstable) repellers or saddle points. Convergence analysis based on the deterministic discrete-time approach shows that, for a step size within a certain range, the norm of the principal/minor state vector converges to a fixed value that relates to the corresponding principal/minor generalized eigenvalue. Thus, the proposed algorithm is a generalized eigenpairs (eigenvectors and eigenvalues) extraction algorithm. Finally, the simulation experiments are carried to further demonstrate the efficiency of the proposed algorithm.

An Integrated Reliability Estimation Approach With Stochastic Filtering and Degradation Modeling for Phased-Mission Systems.

Reliability estimation is central to enhance safety, availability, and effectiveness of phased-mission systems (PMSs). With the development of information and sensing technologies, condition monitoring (CM) data are now available in many real-world PMSs, and then a more interesting question: how can we dynamically estimate the reliability of PMSs using the in-situ CM data, is of considerable significance to industrial practitioners. In this paper, using the CM data and degradation data of PMS, we present a novel condition-based approach to resolve this question under dynamic operating scenarios. This paper differs from most existing methods which only consider the static scenario without using real-time information, and estimate the reliability only for a population of PMSs but not for an individual PMS in service. To establish a linkage between the historical data and real-time data of the individual PMS, a stochastic filtering model is first utilized to model the phase duration. As such, the updated estimation of the mission time can be obtained by Bayesian law at each phase. To account for the dependency of the degradation progression of PMS on the mission process, the degradation process of PMS is modeled by a Brownian motion with a mission phase-dependent drift coefficient. The corresponding lifetime is derived and the lifetime distribution of PMS can be updated under Bayesian framework once new information is available. Unique to this paper is the union of the CM data and degradation data of PMS to real-time estimate the mission reliability through the estimated distribution of the mission time in conjunction with the estimated lifetime distribution, in which the estimated lifetime considers the dependency of the degradation rate of PMS on mission phase. The effectiveness of the proposed approach is verified by a numerical simulation and a case study.