MEOL: A Maximum-Entropy Framework for Options Learning.

Options, the temporally extended courses of actions that can be taken at varying time scale, have provided a concrete, key framework for learning levels of temporal abstraction in hierarchical tasks. While methods of learning options end-to-end is well researched, how to explore good options and actions simultaneously is still challenging. We address this issue by maximizing reward augmented with entropies of both option and action selection policy in options learning. To this end, we reveal our novel optimization objective by reformulating options learning from perspective of probabilistic inference and propose a soft options iteration method to guarantee convergence to the optimum. In implementation, we propose an off-policy algorithm called the maximum-entropy options critic (MEOC) and evaluate it on series of continuous control benchmarks. Comparative results demonstrate that our method outperforms baselines in efficiency and final result on most benchmarks, and the performance exhibits superiority and robustness especially on complex tasks. Ablated studies further explain that entropy maximization on hierarchical exploration promotes learning performance through efficient options specialization and multimodality in action level.

Fuzzy Boundary Sampled-Data Control for Nonlinear Parabolic DPSs.

For a nonlinear parabolic distributed parameter system (DPS), a fuzzy boundary sampled-data (SD) control method is introduced in this article, where distributed SD measurement and boundary SD measurement are respected. Initially, this nonlinear parabolic DPS is represented precisely by a Takagi-Sugeno (T-S) fuzzy parabolic partial differential equation (PDE) model. Subsequently, under distributed SD measurement and boundary SD measurement, a fuzzy boundary SD control design is obtained via linear matrix inequalities (LMIs) on the basis of the T-S fuzzy parabolic PDE model to guarantee exponential stability for closed-loop parabolic DPS by using inequality techniques and a LF. Furthermore, respecting the property of membership functions, we present some LMI-based fuzzy boundary SD control design conditions. Finally, the effectiveness of the designed fuzzy boundary SD controller is demonstrated via two simulation examples.

Adaptive event-triggered extended dissipative synchronization of delayed reaction-diffusion neural networks under deception attacks.

Under spatially averaged measurements (SAMs) and deception attacks, this article mainly studies the problem of extended dissipativity output synchronization of delayed reaction-diffusion neural networks via an adaptive event-triggered sampled-data (AETSD) control strategy. Compared with the existing ETSD control methods with constant thresholds, our scheme can be adaptively adjusted according to the current sampling and latest transmitted signals and is realized based on limited sensors and actuators. Firstly, an AETSD control scheme is proposed to save the limited transmission channel. Secondly, some synchronization criteria under SAMs and deception attacks are established by utilizing Lyapunov-Krasovskii functional and inequality techniques. Then, by solving linear matrix inequalities (LMIs), we obtain the desired AETSD controller, which can satisfy the specified level of extended dissipativity behaviors. Lastly, one numerical example is given to demonstrate the validity of the proposed method.

Fuzzy Boundary Control for Nonlinear Delayed DPSs Under Boundary Measurements.

For nonlinear delayed distributed parameter systems (DDPSs), this article considers a fuzzy boundary control (FBC) under boundary measurements (BMs). Initially, we accurately describe the nonlinear DDPS through a Takagi-Sugeno (T-S) fuzzy partial differential-difference equation (PDDE). Then, in accordance with the T-S fuzzy PDDE model, an FBC design under BMs ensuring the exponential stability for closed-loop DDPS is subsequently presented by spatial linear matrix inequalities (SLMIs) via using Wirtinger's inequality, Halanay's inequality, and the Lyapunov direct method, which respects the fast-varying and slow-varying delays. Moreover, we formulate SLMIs as LMIs for solving the fuzzy boundary controller design of nonlinear DDPSs under BMs. Finally, the effectiveness of the proposed FBC strategy is presented via simulation examples.

Pinning Spatiotemporal Sampled-Data Synchronization of Coupled Reaction-Diffusion Neural Networks Under Deception Attacks.

In this article, we investigate the pinning spatiotemporal sampled-data (SD) synchronization of coupled reaction-diffusion neural networks (CRDNNs), which are directed networks with SD in time and space communications under random deception attacks. In order to handle with the random deception attacks, we establish a directed CRDNN model, which respects the impacts of variable sampling and random deception attacks within a unified framework. Through the designed pinning spatiotemporal SD controller, sufficient conditions are obtained by linear matrix inequalities (LMIs) that guarantee the mean square exponential stability of the synchronization error system (SES) derived by utilizing inequality techniques, the stochastic analysis technique, and Lyapunov-Krasovskii functional (LKF). Finally, a numerical example is utilized to support the presented pinning spatiotemporal SD synchronization method.

Fuzzy Control Under Spatially Local Averaged Measurements for Nonlinear Distributed Parameter Systems With Time-Varying Delay.

This paper introduces a fuzzy control (FC) under spatially local averaged measurements (SLAMs) for nonlinear-delayed distributed parameter systems (DDPSs) represented by parabolic partial differential-difference equations (PDdEs), where the fast-varying time delay and slow-varying one are considered. A Takagi-Sugeno (T-S) fuzzy PDdE model is first derived to exactly describe the nonlinear DDPSs. Then, by virtue of the T-S fuzzy PDdE model and a Lyapunov-Krasovskii functional, an FC design under SLAMs, where the membership functions of the proposed FC law are determined by the measurement output and independent of the fuzzy PDdE plant model, is developed on basis of spatial linear matrix inequalities (SLMIs) to guarantee the exponential stability for the resulting closed-loop DDPSs. Lastly, a numerical example is offered to support the presented approach.

Quantized Sampled-Data Synchronization of Delayed Reaction-Diffusion Neural Networks Under Spatially Point Measurements.

This article considers the synchronization problem of delayed reaction-diffusion neural networks via quantized sampled-data (SD) control under spatially point measurements (SPMs), where distributed and discrete delays are considered. The synchronization scheme, which takes into account the communication limitations of quantization and variable sampling, is based on SPMs and only available in a finite number of fixed spatial points. By utilizing inequality techniques and Lyapunov-Krasovskii functional, some synchronization criteria via a quantized SD controller under SPMs are established and presented by linear matrix inequalities, which can ensure the exponential stability of the synchronization error system containing the drive and response dynamics. Finally, two numerical examples are offered to support the proposed quantized SD synchronization method.

Sampled-data control for linear time-delay distributed parameter systems.

Some real systems have spatiotemporal dynamics and are time-delay distributed parameter systems (DPSs). The existence of time-delay may lead to system instability. The analysis and design of DPSs with time-delay is essentially more complicated. To take into account the factor of time-delay and fully enjoy the benefits of the digital technology in control engineering, it is a theoretical and practical value to consider the sampled-data control (SDC) problem of DPSs with time-delay. However, there are few attempts to solve the SDC problem of time-delay DPSs. In this paper, we introduce a SDC for linear time-delay DPSs described by parabolic partial differential equations (PDEs). A SDC design is developed in the formulation of spatial linear matrix inequalities (LMIs) by constructing an appropriate Lyapunov functional, which can stabilize exponentially the time-delay DPSs. This stabilization condition can be applied to either slowing-varying time delay or fast-varying one. Finally, simulation results of a numerical example are provided to illustrate the effectiveness of the proposed method.

Sampled-Data Fuzzy Control for Nonlinear Coupled Parabolic PDE-ODE Systems.

In this paper, a sampled-data fuzzy control problem is addressed for a class of nonlinear coupled systems, which are described by a parabolic partial differential equation (PDE) and an ordinary differential equation (ODE). Initially, the nonlinear coupled system is accurately represented by the Takagi-Sugeno (T-S) fuzzy coupled parabolic PDE-ODE model. Then, based on the T-S fuzzy model, a novel time-dependent Lyapunov functional is used to design a sampled-data fuzzy controller such that the closed-loop coupled system is exponentially stable, where the sampled-data fuzzy controller consists of the ODE state feedback and the PDE static output feedback under spatially averaged measurements. The stabilization condition is presented in terms of a set of linear matrix inequalities. Finally, simulation results on the control of a hypersonic rocket car are given to illustrate the effectiveness of the proposed design method.

On fuzzy sampled-data control of chaotic systems via a time-dependent Lyapunov functional approach.

In this paper, a novel approach to fuzzy sampled-data control of chaotic systems is presented by using a time-dependent Lyapunov functional. The advantage of the new method is that the Lyapunov functional is continuous at sampling times but not necessarily positive definite inside the sampling intervals. Compared with the existing works, the constructed Lyapunov functional makes full use of the information on the piecewise constant input and the actual sampling pattern. In terms of a new parameterized linear matrix inequality (LMI) technique, a less conservative stabilization condition is derived to guarantee the exponential stability for the closed-loop fuzzy sampled-data system. By solving a set of LMIs, the fuzzy sampled-data controller can be easily obtained. Finally, the chaotic Lorenz system and Rössler's system are employed to illustrate the feasibility and effectiveness of the proposed method.