Caching Placement Optimization in UAV-Assisted Cellular Networks: A Deep Reinforcement Learning-Based Framework

Capable of delivering contents offloaded from the base station (BS) to users, unmanned aerial vehicle (UAV) has emerged as a crucial leverage to compensate for terrestrial BSs-based communication. However, the limited storage capacity of the UAV brings challenges to providing low-latency services for users. In this letter, we investigate the caching placement of the UAV for enhancing the timeliness of services. To overcome the unknown content popularity, proximal policy optimization (PPO) is adopted in the proposed algorithm. To be specific, we first propose a modified timeliness model, named effective age of information (EAoI), to comprehensively evaluate the timeliness of services. Then, we employ PPO to build a deep reinforcement learning framework for finding the optimal caching strategy adaptively. Extensive simulation results are provided to verify the superiority of the proposed scheme, in comparison with the conventional schemes.