Distributed Reinforcement Learning for Quality-of-Service Routing in Wireless Device-to-device Networks

In this paper, we aim to determine the multi-hop route between a device-to-device (D2D) source-destination pair which meets the quality-of-service (QoS) of services. We model this QoS routing problem in D2D as a Markov decision process (MDP) and proposes a distributed multi-agent reinforcement learning routing algorithm. We consider the QoS requirements in terms of bandwidth, delay, and packet loss rate, and allocate the routing path according to link information averaged over time in dynamic network environments. By decomposing the Q-function into multiple local Q-functions, each agent can compute its own optimal strategy based on local observations, which greatly reduces the costs of learning and searching in large-scale multi-state systems. The simulation results show that the proposed algorithm can significantly reduce the average end-to-end delay, the average packet loss rate and service rejection rate compared with both the minimum hop algorithm and the traditional routing algorithm which only considers static parameters.