Delay-Tolerant Multi-agent DRL for Trajectory Planning and Transmission Control in UAV-assisted Wireless Networks

This paper exploits multiple unmanned aerial vehicles (UAVs) to assist energy transfer, data uploading, and transmission in wireless networks, aiming to maximize the network's energy efficiency (EE). The inherent challenge of inaccessible or energy-intensive real-time information exchanges among UAVs results in undesirable delays in acquiring global network information. Such delayed information significantly hinders the transmission control and trajectory planning of the UAVs in multi-UAV-assisted wireless networks. To address this challenge, we propose a delay-tolerant multi-agent deep reinforcement learning (DT-MADRL) algorithm to jointly optimize the UAVs' trajectories and transmission control strategies based on randomly delayed information. In particular, we integrate a delay penalty term in the reward function that forces each UAV to have more regular information exchanges with the base station (BS). This ensures that each UAV can understand the real-time network environment, thereby reducing information delay and fostering more effective multi-agent collaboration. The simulation results reveal that our proposed algorithm reduces the UAVs' average information delay by 68% and improves overall EE by 28% compared to traditional MADRL algorithms.