Hierarchical DRL-based Throughput Maximization for RSMA in UAV-aided Terrestrial Network

As unmanned aerial vehicles (UAVs) are highly maneuverable, they can be used as aerial base stations (ABS) to provide users with line-of-sight (LoS) channels to improve signal quality and transmission throughput. In this paper, an RSMA-based downlink communication system for UAV-aided terrestrial networks under consideration of quality of service (QoS) and power constraints is investigated. Specifically, the problem of maximizing the total system throughput is proposed in the context of simultaneous optimization of UAV resource block (RB) assignment and power control strategies. For this hybrid discrete-continuous nonconvex optimization problem, we decompose it into two subproblems and design a hierarchical deep reinforcement learning (DRL) framework, where the upper tier uses deep Q-network (DQN) to optimize discrete RB assignment and the lower tier uses proximal policy optimization (PPO) to optimize continuous power control strategies. Further, we adopt a centralized training distributed execution (CTDE) approach that allows all the agents to coordinate and cooperate in the common goal of maximizing the total throughput. Through simulation experiments, we verify that the proposed algorithm significantly outperforms other benchmarks in terms of total throughput performance.