Two-Layer Reinforcement Learning for Output Consensus of Multiagent Systems Under Switching Topology.

In this article, the data-based output consensus of discrete-time multiagent systems under switching topology (ST) is studied via reinforcement learning. Due to the existence of ST, the kernel matrix of value function is switching-varying, which cannot be applied to existing algorithms. To overcome the inapplicability of varying kernel matrix, a two-layer reinforcement learning algorithm is proposed in this article. To further implement the proposed algorithm, a data-based distributed control policy is presented, which is applicable to both fixed topology and ST. Besides, the proposed method does not need assumptions on the eigenvalues of leader's dynamic matrix, it avoids the assumptions in the previous method. Subsequently, the convergence of algorithm is analyzed. Finally, three simulation examples are provided to verify the proposed algorithm.

Optimized Backstepping Cooperative Control for Output-Constrained Stochastic Nonlinear Network Systems via a Multibridge-Hole Function.

In this article, a new leader-following tracking control approach is investigated for stochastic multiagent systems with multibridge-hole output constraints. The multibridge-hole output constraints mean that the output of the system is constrained in some intervals and unconstrained in other intervals. The constrained and unconstrained intervals can be set arbitrarily. By designing a new shift function to construct the barrier Lyapunov function, the optimal controller is constructed by combining the backstepping technique with the adaptive dynamic programming technique. The model network is used to estimate the unknown disturbances and uncertainty terms in the system. The critic network and the actor network are constructed such that the designed controller adheres to the Bellman optimality principle and gives the optimal solution of the system. The proposed control method is versatile and compatible with various types of output constrained control problems, such as unconstrained control problems, constrained control problems, and delay constrained problems without changing the structure of the controller. Finally, some simulation results are given to verify the effectiveness of the method.

Adaptive Virotherapy Strategy for Organism With Constrained Input Using Medicine Dosage Regulation Mechanism.

In this article, the constrained adaptive control strategy based on virotherapy is investigated for organism using the medicine dosage regulation mechanism (MDRM). First, the tumor-virus-immune interaction dynamics is established to model the relations among the tumor cells (TCs), virus particles, and the immune response. The adaptive dynamic programming (ADP) method is extended to approximately obtain the optimal strategy for the interaction system to reduce the populations of TCs. Due to the consideration of asymmetric control constraints, the nonquadratic functions are proposed to formulate the value function such that the corresponding Hamilton-Jacobi-Bellman equation (HJBE) is derived which can be deemed as the cornerstone of ADP algorithms. Then, the ADP method of a single-critic network architecture which integrates MDRM is proposed to obtain the approximate solutions of HJBE and eventually derive the optimal strategy. The design of MDRM makes it possible for the dosage of the agentia containing oncolytic virus particles to be regulated timely and necessarily. Furthermore, the uniform ultimate boundedness of the system states and critic weight estimation errors is validated by Lyapunov stability analysis. Finally, simulation results are given to show the effectiveness of the derived therapeutic strategy.

Convergence Analysis of Value Iteration Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems.

This article is concerned with the convergence property and error bounds analysis of value iteration (VI) adaptive dynamic programming for continuous-time (CT) nonlinear systems. The size relationship between the total value function and the single integral step cost is described by assuming a contraction assumption. Then, the convergence property of VI is proved while the initial condition is an arbitrary positive semidefinite function. Moreover, the accumulated effects of approximation errors generated in each iteration are taken into consideration while using approximators to implement the algorithm. Based on the contraction assumption, the error bounds condition is proposed, which ensures the approximated iterative results converge to a neighborhood of the optimum, and the relation between the optimal solution and approximated iterative results is also derived. To make the contraction assumption more concrete, an estimation way is proposed to derive a conservative value of the assumption. Finally, three simulation cases are given to validate the theoretical results.

Cooperative Control for Stochastic Multiagent Systems With Deferred Dynamic Constraints via a Novel Universal Barrier Function Approach.

This article investigates the cooperative control problem for stochastic multiagent systems (MASs) with dynamic constraints. A new universal barrier function is proposed, which is applicable to many systems with different types of constraint functions, even unconstrained systems. Several mapping functions are constructed to constrain the state variables directly without feasibility conditions, and the tracking control is achieved for stochastic MASs with deferred full-state constraints under the backstepping framework. In order to regulate the tracking error more precisely, the funnel error transformation is improved and the deferred funnel controller is developed by introducing a preassigned finite-time function. Based on the deferred funnel controller, the tracking error can be maintained within the predetermined funnel in the preassigned time. The convergence time can be defined according to the actual requirements, and it is independent of the design controller parameters and initial conditions. Finally, some simulation results are given to demonstrate the effectiveness of the proposed control algorithm.

Dynamic Threshold Finite-Time Prescribed Performance Control for Nonlinear Systems With Dead-Zone Output.

This article investigates the tracking control problem for nonlinear systems. An adaptive model is proposed to represent the dead-zone phenomenon and solve its control challenge with a Nussbaum function in conjunction. Drawing inspiration from the existing prescribed performance control schemes, a novel dynamic threshold scheme is developed that fuses a proposed continuous function with a finite-time performance function. A dynamic event-triggered strategy is applied to reduce the redundant transmission. The proposed time-varying threshold control strategy has fewer updates than the traditional fixed threshold and improves the efficiency of resource utilization. A command filter backstepping approach is employed to prevent the complexity explosion faced by the computation. The suggested control strategy ensures that all system signals are bounded. The validity of the simulation results has been verified.

Fault-Tolerant Fuzzy-Resilient Control for Fractional-Order Stochastic Underactuated System With Unmodeled Dynamics and Actuator Saturation.

This article is considered on underactuated fractional-order stochastic systems (FOSSs) with actuator saturation and incrementally conic nonlinear terms, whose fractional-order α ∈ (0,1) . First, to bring FO dynamic signals, solving the unmodeled dynamics, in the meantime, the saturated nonlinear term of the control input is taken into account. At the time, to cope with the stability issue of FOSS under such situation, the fault tolerant resilient controller based on underactuated condition is designed. Then, according to the method of the Lyapunov and It∧ o differential formulation to design proper multiple Lyapunov-Krasovskii (L-K) functions, such that, a novel sufficient condition of the robustly asymptotically stability of fuzzy FOSS under underactuated conditions is rigorously proved in terms of linear matrix inequality (LMI). Furthermore, in order to research the mean square stability of the above-mentioned system, so the solution of FOSS is obtained to achieve this purpose. By applying the above method, which is proposed in this work that the controlled system can be obtained with faster response and higher control accuracy. At last, to display the superiority of the above-mentioned scheme is effective, tethered satellite system and numerical results are presented.

Adaptive Dynamic Event-Triggered Distributed Output Observer for Leader-Follower Multiagent Systems Under Directed Graphs.

Leader-follower consensus problem for multiagent systems (MASs) is an important research hotspot. However, the existing methods take the leader system matrix as a priori knowledge for each agent to design the controller and use the leader's state information. In fact, only the output information may be available in some practical applications. On this basis, this article first designs a novel adaptive distributed dynamic event-triggered observer for each follower to learn the minimum polynomial coefficients of the leader system matrix instead of the leader system matrix. The proposed method is scalable and suitable for large-scale MASs and can reduce the information transmission dimension in observer design. Then, an adaptive dynamic event-triggered compensator based on the observer and leader output information is designed for each follower, thereby solving the leader-follower consensus problem. Finally, several simulation examples are given to verify the effectiveness of the proposed scheme.

A Dynamic Weights-Based Wavelet Attention Neural Network for Defect Detection.

Automatic defect detection plays an important role in industrial production. Deep learning-based defect detection methods have achieved promising results. However, there are still two challenges in the current defect detection methods: 1) high-precision detection of weak defects is limited and 2) it is difficult for current defect detection methods to achieve satisfactory results dealing with strong background noise. This article proposes a dynamic weights-based wavelet attention neural network (DWWA-Net) to address these issues, which can enhance the feature representation of defects and simultaneously denoise the image, thereby improving the detection accuracy of weak defects and defects under strong background noise. First, wavelet neural networks and dynamic wavelet convolution networks (DWCNets) are presented, which can effectively filter background noise and improve model convergence. Second, a multiview attention module is designed, which can direct the network attention toward potential targets, thereby guaranteeing the accuracy for detecting weak defects. Finally, a feature feedback module is proposed, which can enhance the feature information of defects to further improve the weak defect detection accuracy. The DWWA-Net can be used for defect detection in multiple industrial fields. Experiment results illustrate that the proposed method outperforms the state-of-the-art methods (mean precision: GC10-DET: 6.0%; NEU: 4.3%). The code is made in https://github.com/781458112/DWWA.

Distributed Multiagent-Based Event-Driven Fault-Tolerant Control of Islanded Microgrids.

This article proposes an observer-based event-driven fault-tolerant (OBEDFT) secondary control strategy for AC microgrids (MGs) to achieve load voltage regulation. First, the input-output feedback linearization method transforms the voltage regulation issue into an output feedback tracking problem for linear multiagent systems (MASs) with nonlinear dynamics. This transformation provides the necessary preprocessing for load voltage regulation. Then, an OBEDFT secondary control protocol that considers full-state immeasurability is proposed. The actuators of distributed generators (DGs) may experience partial loss of effectiveness (PLOE) and bias faults, and these fault parameters may be heterogeneous and time-varying. The protocol introduces adaptive techniques to avoid information related to fault parameters while using event-driven mechanisms to achieve discrete measurements of neighboring DG. Additionally, the protocol uses boundary layers to construct smooth controllers that prevent the chattering effect caused by nonsmooth controllers. Finally, simulation results confirm the effectiveness of this load voltage regulation strategy.

Cooperative Containment Control for Multiagent Systems With Reduced-Order Protocols.

This article addresses the problem of containment control for continuous-time multiagent systems. A containment error is first given to show the coordination between the outputs of leaders and followers. Then, an observer is designed based on the neighbor observable convex hull state. Under the assumption that the designed reduced-order observer is subject to external disturbances, a reduced-order protocol is designed to realize the containment coordination. In order to ensure the designed control protocol can achieve the effect of the main theories, a corresponding Sylvester equation is given with a novel approach which proves that the Sylvester equation is solvable. Finally, a numerical example is given to verify the validity of the main results.

Preassigned Time Adaptive Neural Tracking Control for Stochastic Nonlinear Multiagent Systems With Deferred Constraints.

This article studies a preassigned time adaptive tracking control problem for stochastic multiagent systems (MASs) with deferred full state constraints and deferred prescribed performance. A modified nonlinear mapping is designed, which incorporates a class of shift functions, to eliminate the constraints on the initial value conditions. By virtue of this nonlinear mapping, the feasibility conditions of the full state constraints for stochastic MASs can also be circumvented. In addition, the Lyapunov function codesigned by the shift function and the fixed-time prescribed performance function is constructed. The unknown nonlinear terms of the converted systems are handled based on the approximation property of the neural networks. Furthermore, a preassigned time adaptive tracking controller is established, which can achieve deferred prescribed performance for stochastic MASs that provide only local information. Finally, a numerical example is given to demonstrate the effectiveness of the proposed scheme.

Distributed Observer-Based Robust Fault Estimation Design for Discrete-Time Interconnected Systems With Disturbances.

This article focuses on the distributed robust fault estimation problem for a kind of discrete-time interconnected systems with input and output disturbances. For each subsystem, by letting the fault as a special state, an augmented system is constructed. Particularly, the dimensions of system matrices after augmentation are lower than some existing related results, which may help to reduce calculation amount, especially, for linear matrix inequality-based conditions. Then, a distributed fault estimation observer design scheme that utilizes the associated information among subsystems is presented to not only reconstruct faults, but also suppress disturbances in the sense of robust H∞ optimization. Besides, to improve the fault estimation performance, a common Lyapunov matrix-based multiconstrained design method is first given to solve the observer gain, which is further extended to the different Lyapunov matrices-based multiconstrained calculation method. Thus, the conservatism is reduced. Finally, simulation experiments are shown to verify the validity of our distributed fault estimation scheme.

Sliding Mode Control Based on Reinforcement Learning for T-S Fuzzy Fractional-Order Multiagent System With Time-Varying Delays.

This article researches the sliding mode control (SMC) for fuzzy fractional-order multiagent system (FOMAS) subject to time-varying delays over directed networks based on reinforcement learning (RL), α ∈ (0,1). First, since there is information communication between an agent and another agent, a new distributed control policy ξi(t) is introduced so that the sharing of signals is implemented through RL, whose propose is to minimize the error variables with learning. Then, different from the existed papers studying normal fuzzy MASs, a new stability basis of fuzzy FOMASs with time-varying delay terms is presented to guarantee that the states of each agent eventually converge to the smallest possible domain of 0 using Lyapunov-Krasovskii functionals, free weight matrix, and linear matrix inequality (LMI). Furthermore, in order to provide appropriate parameters for SMC, the RL algorithm is combined with SMC strategy, and the constraints on the initial conditions of the control input ui(t) are eliminated, so that the sliding motion satisfy the reachable condition within a finite time. Finally, to illustrate that the proposed protocol is valid, the results of the simulation and numerical examples are presented.

Adaptive Event-Triggered Control Approach to the Cooperative Output Regulation of Heterogeneous Multiagent Systems Under Digraphs.

The cooperative output regulation (COR) problem of heterogeneous linear multiagent systems under digraphs has been considered under the assumption that continuous communication between neighbors and continuous update of controllers. To get rid of this assumption, that is, to realize event-triggered communication between neighbors and discrete update of controllers, this article proposes the fully distributed event-triggered observers to estimate the matrix and the state for the exosystem, and the event-triggered controllers to solve the COR problem. Moreover, the Zeno behavior is excluded by proving that the interevent times of each agent are strictly greater than zero under the design triggering conditions. Finally, two examples are given to verify the effectiveness and advantages of the proposed methods.

Formation Tracking Control for Heterogeneous Multiagent Systems With Multiple Nonautonomous Leaders via Dynamic Event-Triggered Mechanisms.

This article considers the time-varying formation (TVF) tracking issue of heterogeneous multiagent systems (HMASs) with the dynamic event-triggered control. The HMASs contain heterogeneous multiple leaders, all of which have the input signals to generate flexible reference, and only the output information can be measured. All leaders do not have access to the same followers, that is, the well-informed follower assumption is removed in this article. In this setting, the adaptive multileader state compensator is designed for each follower to estimate the integrated state information of all leaders, which can equip with two kinds of dynamic event-triggered mechanisms, that is, node-based event-triggered mechanism and edge-based event-triggered mechanism, to save communication bandwidth. Then, the TVF controllers are built by some estimation values to regulate the followers to achieve and maintain the geometric shape while tracking the reference which is the convex combination of outputs of leaders. The event-triggered compensator and TVF controller constitute the control protocol of HMASs, which are independent of global information with the fully distributed manner. The stability analysis and numerical simulations are given to verify the presented control protocol.

Neural-Network-Based Immune Optimization Regulation Using Adaptive Dynamic Programming.

This article investigates optimal regulation scheme between tumor and immune cells based on the adaptive dynamic programming (ADP) approach. The therapeutic goal is to inhibit the growth of tumor cells to allowable injury degree and maximize the number of immune cells in the meantime. The reliable controller is derived through the ADP approach to make the number of cells achieve the specific ideal states. First, the main objective is to weaken the negative effect caused by chemotherapy and immunotherapy, which means that the minimal dose of chemotherapeutic and immunotherapeutic drugs can be operational in the treatment process. Second, according to the nonlinear dynamical mathematical model of tumor cells, chemotherapy and immunotherapeutic drugs can act as powerful regulatory measures, which is a closed-loop control behavior. Finally, states of the system and critic weight errors are proved to be ultimately uniformly bounded with the appropriate optimization control strategy and the simulation results are shown to demonstrate the effectiveness of the cybernetics methodology.

Data-Driven Finite-Horizon H∞ Tracking Control With Event-Triggered Mechanism for the Continuous-Time Nonlinear Systems.

In this article, the neural network (NN)-based adaptive dynamic programming (ADP) event-triggered control method is presented to obtain the near-optimal control policy for the model-free finite-horizon H∞ optimal tracking control problem with constrained control input. First, using available input-output data, a data-driven model is established by a recurrent NN (RNN) to reconstruct the unknown system. Then, an augmented system with event-triggered mechanism is obtained by a tracking error system and a command generator. We present a novel event-triggering condition without Zeno behavior. On this basis, the relationship between event-triggered Hamilton-Jacobi-Isaacs (HJI) equation and time-triggered HJI equation is given in Theorem 3. Since the solution of the HJI equation is time-dependent for the augmented system, the time-dependent activation functions of NNs are considered. Moreover, an extra error is incorporated to satisfy the terminal constraints of cost function. This adaptive control pattern finds, in real time, approximations of the optimal value while also ensuring the uniform ultimate boundedness of the closed-loop system. Finally, the effectiveness of the proposed near-optimal control pattern is verified by two simulation examples.

Synchronization of Generally Uncertain Markovian Inertial Neural Networks With Random Connection Weight Strengths and Image Encryption Application.

This article focuses on the synchronization problem of delayed inertial neural networks (INNs) with generally uncertain Markovian jumping and their applications in image encryption. The random connection weight strengths and generally uncertain Markovian are discussed in the INNs model. Compared with most existing INNs models that have constant connection weight strengths, our model is more practical because connection weight strengths of INNs may randomly vary due to the external and internal environment and human factor. The delay-range-dependent synchronization conditions (DRDSCs) could be obtained by adopting the delay-product-term Lyapunov-Krasovskii functional (DPTLKF) and higher order polynomial-based relaxed inequality (HOPRII). In addition, the desired controllers are obtained by solving a set of linear matrix inequalities. Finally, two examples are shown to demonstrate the effectiveness of the proposed results.

Resilient Model Free Adaptive Distributed LFC for Multi-Area Power Systems Against Jamming Attacks.

This article is concerned with distributed resilient load frequency control (LFC) for multi-area power interconnection systems against jamming attacks. First, considering uncertainties and high dimension nonlinearity, the model-free adaptive control (MFAC) model is adopted for the power system, in which only input and output (I/O) data are used. Second, jamming attacks are modeled in a stochastic process, and a multistep predictive compensation algorithm is developed to mitigate the impact of jamming attacks. Then, the distributed MFAC protocol with predictive compensation algorithm is designed such that the frequency tracking errors under the predictive compensation algorithm of multi-area power interconnection systems converge consensually into a small neighborhood of origin in the mean square sense. Simulation results show the effectiveness of the approach.