An Improved Approach towards Multi-Agent Pursuit-Evasion Game Decision-Making Using Deep Reinforcement Learning.

A pursuit-evasion game is a classical maneuver confrontation problem in the multi-agent systems (MASs) domain. An online decision technique based on deep reinforcement learning (DRL) was developed in this paper to address the problem of environment sensing and decision-making in pursuit-evasion games. A control-oriented framework developed from the DRL-based multi-agent deep deterministic policy gradient (MADDPG) algorithm was built to implement multi-agent cooperative decision-making to overcome the limitation of the tedious state variables required for the traditionally complicated modeling process. To address the effects of errors between a model and a real scenario, this paper introduces adversarial disturbances. It also proposes a novel adversarial attack trick and adversarial learning MADDPG (A2-MADDPG) algorithm. By introducing an adversarial attack trick for the agents themselves, uncertainties of the real world are modeled, thereby optimizing robust training. During the training process, adversarial learning was incorporated into our algorithm to preprocess the actions of multiple agents, which enabled them to properly respond to uncertain dynamic changes in MASs. Experimental results verified that the proposed approach provides superior performance and effectiveness for pursuers and evaders, and both can learn the corresponding confrontational strategy during training.

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments.

Autonomous motion planning (AMP) of unmanned aerial vehicles (UAVs) is aimed at enabling a UAV to safely fly to the target without human intervention. Recently, several emerging deep reinforcement learning (DRL) methods have been employed to address the AMP problem in some simplified environments, and these methods have yielded good results. This paper proposes a multiple experience pools (MEPs) framework leveraging human expert experiences for DRL to speed up the learning process. Based on the deep deterministic policy gradient (DDPG) algorithm, a MEP-DDPG algorithm was designed using model predictive control and simulated annealing to generate expert experiences. On applying this algorithm to a complex unknown simulation environment constructed based on the parameters of the real UAV, the training experiment results showed that the novel DRL algorithm resulted in a performance improvement exceeding 20% as compared with the state-of-the-art DDPG. The results of the experimental testing indicate that UAVs trained using MEP-DDPG can stably complete a variety of tasks in complex, unknown environments.

Mobility Control of Unmanned Aerial Vehicle as Communication Relay to Optimize Ground-to-Air Uplinks.

In recent years, unmanned aerial vehicles (UAVs) have been considered an ideal relay platform for enhancing the communication between ground agents, because they fly at high altitudes and are easy to deploy with strong adaptabilities. Their maneuvering allows them to adjust their location to optimize the performance of links, which brings out the relay UAV autonomous mobility control problem. This work addressed the problem in a novel scene with mobile agents and completely unknown wireless channel properties, using only online measured information of received signal strength (RSS) and agent positions. The problem is challenging because of the unknown and dynamic radio frequency (RF) environment cause by agents and UAV maneuvering. We present a framework for both end-to-end communication and multi-agent-inter communication applications, and focus on proposing: (1) least square estimation-based channel approximation with consideration of environment effects and, (2) gradient-based optimal relay position seeking. Simulation results show that considering the environmental effects on channel parameters is meaningful and beneficial in using UAV as relays for the communication of multiple ground agents, and validate that the proposed algorithms optimizes the network performance by controlling the heading of the UAV.