Reinforcement Learning-Based Decentralized Safety Control for Constrained Interconnected Nonlinear Safety-Critical Systems.

This paper addresses the problem of decentralized safety control (DSC) of constrained interconnected nonlinear safety-critical systems under reinforcement learning strategies, where asymmetric input constraints and security constraints are considered. To begin with, improved performance functions associated with the actuator estimates for each auxiliary subsystem are constructed. Then, the decentralized control problem with security constraints and asymmetric input constraints is transformed into an equivalent decentralized control problem with asymmetric input constraints using the barrier function. This approach ensures that safety-critical systems operate and learn optimal DSC policies within their safe global domains. Then, the optimal control strategy is shown to ensure that the entire system is uniformly ultimately bounded (UUB). In addition, all signals in the closed-loop auxiliary subsystem, based on Lyapunov theory, are uniformly ultimately bounded, and the effectiveness of the designed method is verified by practical simulation.

Critic Learning-Based Safe Optimal Control for Nonlinear Systems with Asymmetric Input Constraints and Unmatched Disturbances.

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton-Jacobi-Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor-critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach.

Robust Trajectory Tracking Control for Continuous-Time Nonlinear Systems with State Constraints and Uncertain Disturbances.

In this paper, a robust trajectory tracking control method with state constraints and uncertain disturbances on the ground of adaptive dynamic programming (ADP) is proposed for nonlinear systems. Firstly, the augmented system consists of the tracking error and the reference trajectory, and the tracking control problems with uncertain disturbances is described as the problem of robust control adjustment. In addition, considering the nominal system of the augmented system, the guaranteed cost tracking control problem is transformed into the optimal control problem by using the discount coefficient in the nominal system. A new safe Hamilton-Jacobi-Bellman (HJB) equation is proposed by combining the cost function with the control barrier function (CBF), so that the behavior of violating the safety regulations for the system states will be punished. In order to solve the new safe HJB equation, a critic neural network (NN) is used to approximate the solution of the safe HJB equation. According to the Lyapunov stability theory, in the case of state constraints and uncertain disturbances, the system states and the parameters of the critic neural network are guaranteed to be uniformly ultimately bounded (UUB). At the end of this paper, the feasibility of the proposed method is verified by a simulation example.

A Clustering Scheme Based on the Binary Whale Optimization Algorithm in FANET.

With the continuous development of Unmanned Aerial Vehicle (UAV) technology, UAVs are widely used in military and civilian fields. Multi-UAV networks are often referred to as flying ad hoc networks (FANET). Dividing multiple UAVs into clusters for management can reduce energy consumption, maximize network lifetime, and enhance network scalability to a certain extent, so UAV clustering is an important direction for UAV network applications. However, UAVs have the characteristics of limited energy resources and high mobility, which bring challenges to UAV cluster communication networking. Therefore, this paper proposes a clustering scheme for UAV clusters based on the binary whale optimization (BWOA) algorithm. First, the optimal number of clusters in the network is calculated based on the network bandwidth and node coverage constraints. Then, the cluster heads are selected based on the optimal number of clusters using the BWOA algorithm, and the clusters are divided based on the distance. Finally, the cluster maintenance strategy is set to achieve efficient maintenance of clusters. The experimental simulation results show that the scheme has better performance in terms of energy consumption and network lifetime compared with the BPSO and K-means-based schemes.

Adaptive critic design for safety-optimal FTC of unknown nonlinear systems with asymmetric constrained-input.

Safe fault tolerant control is one of the key technologies to improve the reliability of dynamic complex nonlinear systems with limited inputs, which is hard to solve and definitely a great challenge to tackle. Thus the paper presents a novel safety-optimal FTC (Fault Tolerant Control) approach for a category of completely unknown nonlinear systems incorporating actuator fault and asymmetric constrained-input, which can guarantee the system's operation within a safe range while showcasing optimal performance. Firstly, a CBF (Control Barrier Function) is incorporated into the cost function to penalize unsafe behaviors, and then we translate the intractable safety-optimal FTC problem into a differential ZSG (Zero-Sum Game) problem by defining the control input and the actuator fault as two opposing sides. Secondly, a neural-network-based identifier is employed to reconstruct system dynamics using system data, and the resolution of handling asymmetric constrained-input with the introduced non-quadratic cost function is achieved through the design of an adaptive critic scheme, aiming to reduce computational expenses accordingly. Finally, through the theoretical stability analysis, it is demonstrated that all signals in the closed-loop system are consistently UUB (Uniformly Ultimately Bounded). Furthermore, the proposed method's effectiveness is also verified in the simulation experiments conducted on a model of a single-link robotic arm system with actuator failure. The result shows that the algorithm can fulfill the safety-optimal demand of fault tolerant control in fault system with asymmetric constrained-input.

Online adaptive policy learning algorithm for H∞ state feedback control of unknown affine nonlinear discrete-time systems.

The problem of H∞ state feedback control of affine nonlinear discrete-time systems with unknown dynamics is investigated in this paper. An online adaptive policy learning algorithm (APLA) based on adaptive dynamic programming (ADP) is proposed for learning in real-time the solution to the Hamilton-Jacobi-Isaacs (HJI) equation, which appears in the H∞ control problem. In the proposed algorithm, three neural networks (NNs) are utilized to find suitable approximations of the optimal value function and the saddle point feedback control and disturbance policies. Novel weight updating laws are given to tune the critic, actor, and disturbance NNs simultaneously by using data generated in real-time along the system trajectories. Considering NN approximation errors, we provide the stability analysis of the proposed algorithm with Lyapunov approach. Moreover, the need of the system input dynamics for the proposed algorithm is relaxed by using a NN identification scheme. Finally, simulation examples show the effectiveness of the proposed algorithm.