Co-training based virtual sample generation for solving the small sample size problem in process industry.

With the development of industrialization, the production scale and complexity of process industries are getting larger and larger. But, limited by the small amounts of samples and the uneven sample distribution in the process industry, it is difficult to establish accurate and efficient data-driven soft sensor models to predict some variables. To further develop the application of soft sensor models, generating new virtual samples based on the original sample distribution to extend the sample set is an ideal approach to solve this problem. In this paper, a novel virtual sample generation method based on the co-training of two K-Nearest Neighbor (KNN) models is proposed. First, according to the sparse parameter, sparse regions in each dimension of the feature space are identified. Second, the input features of virtual samples are generated in these sparse regions by performing interpolation operations. Third, the outputs of virtual samples are predicted by double KNN regressors based on co-training. The qualified virtual samples are screened and the model is updated using these virtual samples to improve the prediction accuracy of the double KNN models. To verify the effectiveness and superiority of the proposed virtual sample generation method based on the co-training (CTVSG), case studies are conducted using two standard functions and a Purified Terephthalic Acid (PTA) industrial dataset, where the effectiveness of CTVSG is confirmed.

Enhanced virtual sample generation based on manifold features: Applications to developing soft sensor using small data.

In the process industry, it is essential to establish a data-driven soft sensor to predict the key variable that is difficult to online measure directly. The accuracy performance of data-driven soft sensors relies heavily on data. Unfortunately, it is hard to acquire sufficient and informative data from the samples with limited number, which is called as the small sample problem. For handling the small sample problem, it is a good solution to generating virtual samples according to the distribution of original data. This paper proposes an enhanced method of virtual sample generation utilizing manifold features to develop soft sensors using small data. First, T-Distribution Stochastic Neighbor Embedding (t-SNE) is utilized to extract the features of input data. The main idea of generating virtual samples is to use the interpolation algorithm to obtain virtual t-SNE input features and then the random forest algorithm is utilized to get the virtual outputs using virtual t-SNE input features. Finally, virtual samples using the proposed t-SNE based virtual sample generation (t-SNE-VSG) can be achieved. For the sake of confirming the effectiveness and feasibility of the presented t-SNE-VSG, a standard data set is first used. What is more, a small data set from an actual industrial process of Purified Terephthalic Acid is used to establish a soft sensor model. The results from simulations show that the accuracy performance of the soft sensor established with small data can be effectively improved by adding the virtual samples generated by t-SNE-VSG. In addition, t-SNE-VSG achieves superior accuracy to state-of-the-art virtual sample generation methods.

Novel double-layer bidirectional LSTM network with improved attention mechanism for predicting energy consumption.

For power generation management and power system dispatching, it is of big significance to predict the consumption of electric energy accurately. For the sake of improving the prediction accuracy of power consumption, taking the complex features of time series data into consideration, a novel neural network sandwich structure with an improved attention mechanism is inserted into the double-layer bidirectional long short-term memory network shortened as A-DBLSTM is put forward in this article. In A-DBLSTM, compared with traditional attention mechanism, the presented attention mechanism focuses on different features in each time unit and the A-DBLLSTM network extracts time information in sequence. The parameter optimization of A-DBLSTM is based on the method of particle swarm optimization (PSO). For confirming the effectiveness and feasibility of A-DBLSTM, case studies using two datasets of the hourly temperature values and power loads between 2012 and 2014 and the electric energy consumption are carried out. The experimental results indicate that the presented A-DBLSTM with the novel sandwich network structure achieves superior performance in the aspects of the mean square error, root mean square, the average absolute error and the mean absolute percentage error to other advanced methods. What is more, the factors that have the greatest impact on the prediction performance can be found through analyzing the heatmap of the attention layer.

Novel manifold learning based virtual sample generation for optimizing soft sensor with small data.

Due to the extremely complex mechanism and strong non-linear characteristics of industrial processes, data-driven soft sensor technologies play a key role in the intelligent measurement of process industries. However, the information of the collected process data in the steady stage is quite limited and unreliable, causing the small sample problem. As a result, it becomes an intractable challenge to catch the nature of the process and build accurate soft sensor models. To solve this problem, this paper proposes a novel manifold learning based virtual sample generation method (Isomap-VSG) to generate feasible virtual samples in the information gaps for supplementing the original small sample space. To find data sparse regions reasonably, one kind of manifold learning methods called Isomap is used to visualize process data with high dimension. Then virtual samples can be generated by the interpolation method and extreme learning machine. The simulation results on a standard dataset and a real-world application demonstrate that, compared with other advanced methods, the proposed Isomap-VSG method can achieve better performance in terms of generating feasible virtual samples and improving the accuracy of soft sensor models using limited samples.

Hybrid robust model based on an improved functional link neural network integrating with partial least square (IFLNN-PLS) and its application to predicting key process variables.

In this paper, a hybrid robust model based on an improved functional link neural network integrating with partial least square (IFLNN-PLS) is proposed. Firstly, an improved functional link neural network with small norm of expanded weights and high input-output correlation (SNEWHIOC-FLNN) was proposed for enhancing the generalization performance of FLNN. Unlike the traditional FLNN, the expanded variables of the original inputs are not directly used as the inputs in the proposed SNEWHIOC-FLNN model. The original inputs are attached to some small norm of expanded weights. As a result, the correlation coefficient between some of the expanded variables and the outputs is enhanced. The larger the correlation coefficient is, the more relevant the expanded variables tend to be. In the end, the expanded variables with larger correlation coefficient are selected as the inputs to improve the performance of the traditional FLNN. In order to test the proposed SNEWHIOC-FLNN model, three UCI (University of California, Irvine) regression datasets named Housing, Concrete Compressive Strength (CCS), and Yacht Hydro Dynamics (YHD) are selected. Then a hybrid model based on the improved FLNN integrating with partial least square (IFLNN-PLS) was built. In IFLNN-PLS model, the connection weights are calculated using the partial least square method but not the error back propagation algorithm. Lastly, IFLNN-PLS was developed as an intelligent measurement model for accurately predicting the key variables in the Purified Terephthalic Acid (PTA) process and the High Density Polyethylene (HDPE) process. Simulation results illustrated that the IFLNN-PLS could significant improve the prediction performance.

A robust hybrid model integrating enhanced inputs based extreme learning machine with PLSR (PLSR-EIELM) and its application to intelligent measurement.

In this paper, a robust hybrid model integrating an enhanced inputs based extreme learning machine with the partial least square regression (PLSR-EIELM) was proposed. The proposed PLSR-EIELM model can overcome two main flaws in the extreme learning machine (ELM), i.e. the intractable problem in determining the optimal number of the hidden layer neurons and the over-fitting phenomenon. First, a traditional extreme learning machine (ELM) is selected. Second, a method of randomly assigning is applied to the weights between the input layer and the hidden layer, and then the nonlinear transformation for independent variables can be obtained from the output of the hidden layer neurons. Especially, the original input variables are regarded as enhanced inputs; then the enhanced inputs and the nonlinear transformed variables are tied together as the whole independent variables. In this way, the PLSR can be carried out to identify the PLS components not only from the nonlinear transformed variables but also from the original input variables, which can remove the correlation among the whole independent variables and the expected outputs. Finally, the optimal relationship model of the whole independent variables with the expected outputs can be achieved by using PLSR. Thus, the PLSR-EIELM model is developed. Then the PLSR-EIELM model served as an intelligent measurement tool for the key variables of the Purified Terephthalic Acid (PTA) process and the High Density Polyethylene (HDPE) process. The experimental results show that the predictive accuracy of PLSR-EIELM is stable, which indicate that PLSR-EIELM has good robust character. Moreover, compared with ELM, PLSR, hierarchical ELM (HELM), and PLSR-ELM, PLSR-EIELM can achieve much smaller predicted relative errors in these two applications.