Multi-Agent Transfer Reinforcement Learning for Resource Management in Underwater Acoustic Communication Networks

This paper investigates the application of self-organizing networks in solving the interference problem in underwater acoustic communication networks (UACNs) with the coexistence of multi-node. In this network, each node autonomously adjusts its power based on locally observed information without central controller intervention. Considering the non-convexity of the optimization problem with quality-of-service constraints and the dynamic nature of the underwater environment, we propose a reinforcement learning (RL)-based approach coupled with a distributed coordination mechanism, namely the multi-agent-based transfer Dyna-Q algorithm (MA-TDQ). This algorithm combines Q-learning with Dyna structure and transfer learning, and can quickly obtain optimal intelligent resource management strategies. Furthermore, we rigorously demonstrate the convergence of the MA-TDQ algorithm to Nash equilibrium. Simulation results indicate that the proposed distributed coordination learning algorithm outperforms other existing learning algorithms in terms of learning efficiency, network transmission rate, and communication service quality.