Multi-agent Deep Reinforcement Learning for Countering Uncrewed Aerial Systems

The proliferation of small uncrewed aerial systems (UAS) poses many threats to airspace systems and critical infrastructures. In this paper, we apply deep reinforcement learning (DRL) to intercept rogue UAS in urban airspaces. We train a group of homogeneous friendly UAS, in this paper referred to as agents, to pursue and intercept a faster UAS evading capture while navigating through crowded airspace with several moving non-cooperating interacting entities (NCIEs). The problem is formulated as a multi-agent Markov Decision Process, and we develop the Proximal Policy Optimization based Advantage Actor-Critic (PPO-A2C) method to solve it, where the actor and critic networks are trained in a centralized server and the derived actor network is distributed to the agents to generate the optimal action based their observations. The simulation results show that, as compared to the traditional method, PPO-A2C fosters collaborations among agents to achieve the highest probability of capturing the evader and maintain the collision rate with other agents and NCIEs in the environment.