Lyapunov-guided Deep Reinforcement Learning for service caching and task offloading in Mobile Edge Computing

With the development of Internet of Things (IoT) networks and Mobile Edge Computing (MEC), many computing -intensive applications have been developed in large quantities. Due to the heterogeneity of tasks, different application services are required to perform each task. Caching application services and related data in edge servers is challenging. Hence, we study the service cache placement and task offloading problem in IoT networks. Since IoT devices and edge servers with limited storage resources can only cache a few services at the same time, we formulate the service cache placement and task offloading of IoT devices problem to minimize task service delay with long-term energy constraint of IoT devices, which is a mixed integer nonlinear programming problem. To solve this problem, an online Deep Reinforcement Learning guided by the Lyapunov optimization framework algorithm (LYADRL) is proposed. We first build a virtual queue model to decouple the problem by Lyapunov optimization technique to transform the problem into a single time slot optimization problem. Then, we use Deep Reinforcement Learning techniques to find the optimal edge service caching and task offloading policies for each time slot. Simulation results show that our algorithm can reduce the service delay compared with other benchmark algorithms.