Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View.

ABSTRACT

An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the "black-box" nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity-entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework.