Neurodynamic approaches for multi-agent distributed optimization.

This paper considers a class of multi-agent distributed convex optimization with a common set of constraints and provides several continuous-time neurodynamic approaches. In problem transformation, l1 and l2 penalty methods are used respectively to cast the linear consensus constraint into the objective function, which avoids introducing auxiliary variables and only involves information exchange among primal variables in the process of solving the problem. For nonsmooth cost functions, two differential inclusions with projection operator are proposed. Without convexity of the differential inclusions, the asymptotic behavior and convergence properties are explored. For smooth cost functions, by harnessing the smoothness of l2 penalty function, finite- and fixed-time convergent algorithms are provided via a specifically designed average consensus estimator. Finally, several numerical examples in the multi-agent simulation environment are conducted to illustrate the effectiveness of the proposed neurodynamic approaches.

Approximating Nash equilibrium for anti-UAV jamming Markov game using a novel event-triggered multi-agent reinforcement learning.

In the downlink communication, it is currently challenging for ground users to cope with the uncertain interference from aerial intelligent jammers. The cooperation and competition between ground users and unmanned aerial vehicle (UAV) jammers leads to a Markov game problem of anti-UAV jamming. Therefore, a model-free method is adopted based on multi-agent reinforcement learning (MARL) to handle the Markov game. However, the benchmark MARL strategies suffer from dimension explosion and local optimal convergence. To solve these issues, a novel event-triggered multi-agent proximal policy optimization algorithm with Beta strategy (ETMAPPO) is proposed in this paper, which aims to reduce the dimension of information transmission and improve the efficiency of policy convergence. In this event-triggering mechanism, agents can learn to obtain appropriate observation in different moment, thereby reducing the transmission of valueless information. Beta operator is used to optimize the action search. It expands the search scope of policy space. Ablation simulations show that the proposed strategy achieves better global benefits with fewer dimension of information than benchmark algorithms. In addition, the convergence performance verifies that the well-trained ETMAPPO has the capability to achieve stable jamming strategies and stable anti-jamming strategies. This approximately constitutes the Nash equilibrium of the anti-jamming Markov game.

Multi-agent based optimal equilibrium selection with resilience constraints for traffic flow.

Traffic guidance and traffic control are effective means to alleviate traffic problems. Formulating effective traffic guidance measures can improve the utilization rate of road resources and optimize the performance of the entire traffic network. Assuming that the traffic coordinator can capture traffic flow information in real-time utilizing sensors installed on each road, we consider the strong resilience constraints to construct an optimal selection problem of balanced flow in the traffic network. Based on multi-agent modeling, each agent has access to the corresponding traffic information of each link. We design a distributed optimization algorithm to tackle this optimization problem. In addition to the inherent advantages of distributed multi-agent algorithms, the communication topology among the sensors is allowed to be time-varying, which is more consistent with reality. To prove the effectiveness of the proposed algorithm, we also give a numerical simulation in the multi-agent environment. It should be reiterated that the optimization problem studied in this paper mainly focuses on traffic managers' perspectives. The goal of the studied optimization problem is to minimize the overall cost of the traffic network by adjusting the optimal equilibrium traffic flow. This study provides a reference for solving congestion optimization in today's cities.