Service caching with multi-agent reinforcement learning in cloud-edge collaboration computing

Edge computing moves application services from the central cloud to the network edge, significantly reducing service latency. Edge service caching presents a more complex challenge than cloud caching, due to the dynamics and diversity of mobile user requests. Consequently, traditional caching strategies are not directly applicable to edge environments. Additionally, the challenge intensifies when considering collaborative caching between adjacent servers. To address these challenge, we propose an edge service caching solution aimed at minimizing the total service delay to ensure high quality user experiences. First, given the limited prior information on user requests in the current time period, we adopt a Transformer-based approach to enhance the accuracy of user request predictions. Since the service caching problem involves both continuous and discrete action spaces, we propose a deep reinforcement learning algorithm based on hybrid Soft actor-critic (SAC) to learn the optimal caching strategy. We then leverage a centralized training and decentralized decision making framework to address multi-agent problems, while selectively reducing agent observation connections to avoid the interference from redundant observations. Finally, extensive simulations demonstrate that our proposed collaborative cloud-edge service caching strategy reduces service latency more effectively than existing approaches.