Joint Computation Offloading and Resource Configuration in Ultra-Dense Edge Computing Networks: A Deep Reinforcement Learning Solution

The prompt development of wireless communication network and emerging technologies such as Internet of Things (IoT) and 5G have increased the number of various mobile devices (MDs). In order to enlarge the capacity of the system and meet the high computation demands of MDs, the integration of ultra-dense heterogeneous networks (UDN) and mobile edge computing (MEC) is proposed as a promising paradigm. However, when massively deploying edge servers in UDN scenario, the operating expense reduction has become an essential issue to be solved, which can be achieved by computation offloading decision-making optimization and edge servers' computing resource configuration. In consideration of the complicated state information and ever-changing environment in UDN, applying reinforcement learning (RL) to the dynamical systems is envisioned as an effective way. Toward this end, we combine the deep learning with RL and propose a deep Q-network based method to address this high-dimensional problem. The experimental results demonstrate the superior performance of our proposed scheme on reducing the processing delay and enhancing the computing resource utilization.