Access selection algorithm based on improved DQN for ultra-dense networks

In the ultra-dense network environment, each access point is deployed in the hotspot area, which forms a complex heterogeneous network. Users need to choose the appropriate network to access, so as to achieve the best performance. Network selection problem is to choose the optimal network for the user, so that the user or network performance reaches the best. In order to solve the access selection problem of users in ultra-dense networks, we proposed an ultra-dense network access selection algorithm based on the improved deep Q network (DQN), considering network states, user preferences, and service types, and combining with load balancing strategies. First, by analyzing the influence of network attributes and user preferences on network selection, the appropriate network parameters were selected as the parameters of the access selection algorithm. Then, the problem of network access selection was modeled by Markov decision-making process, and the states, actions, and reward functions of the model were designed. Finally, the optimal network strategy was obtained by using DQN to solve the network selection model. In addition, the target function of traditional DQN was optimized to avoid overestimation of Q value by DQN, and a priority experience replay mechanism was introduced to improve learning efficiency. Simulation results show that the method could well solve the problem of overestimation of traditional DQN, accelerate the convergence of neural network, effectively reduce user congestion, and improve network throughput performance. © 2023 Harbin Institute of Technology. All rights reserved.