Online continual learning with declarative memory.

Deep neural networks are enjoying unprecedented attention and success in recent years. However, catastrophic forgetting undermines the performance of deep models when the training data are arrived sequentially in an online multi-task learning fashion. To address this issue, we propose a novel method named continual learning with declarative memory (CLDM) in this paper. Specifically, our idea is inspired by the structure of human memory. Declarative memory is a major component of long-term memory which helps human beings memorize past experiences and facts. In this paper, we propose to formulate declarative memory as task memory and instance memory in neural networks to overcome catastrophic forgetting. Intuitively, the instance memory recalls the input-output relations (fact) in previous tasks, which is implemented by jointly rehearsing previous samples and learning current tasks as replaying-based methods act. In addition, the task memory aims to capture long-term task correlation information across task sequences to regularize the learning of the current task, thus preserving task-specific weight realizations (experience) in high task-specific layers. In this work, we implement a concrete instantiation of the proposed task memory by leveraging a recurrent unit. Extensive experiments on seven continual learning benchmarks verify that our proposed method is able to outperform previous approaches with tremendous improvements by retaining the information of both samples and tasks.

An Analysis on Optimal Attack Schedule Based on Channel Hopping Scheme in Cyber-Physical Systems.

In this paper, we investigate the issue of security on the remote state estimation in cyber-physical systems (CPSs), where a wireless sensor utilizes the channel hopping scheme to transmit the data to the remote estimator over multiple channels in the presence of periodic denial-of-service attacks. Assume that the jammer can interfere with a subset of channels at each attack time in active period. For an energy-constraint jammer, the problem of how to select the number of channels at each attack time to maximally deteriorate the CPS performance is investigated. Based on the index of average estimation error, we introduce two different attack strategies, which include selecting identical number of channels and unequal number of channels at each attack time, and further show theoretically that the attack effect by selecting unequal number of channels is better than that of selecting identical number of channels. By formulating the problem of selecting the number of channels as integer programming problems, we present the corresponding algorithm to approximate the optimal attack schedule for both cases. The numerical results are presented to validate the theoretical results and the effectiveness of the proposed algorithms.