Optimize the Server Provisioning and Request Dispatching in Distributed Memory Cache Services

The distributed cache system contains a group of servers caching different contents based on consistent hashing. The dynamic provisioning of servers helps to improve the system efficiency, which leads to a reduction of energy cost. We first measure the cache hit rate, request batching effect and cache warm-up time of the system through experiments, considering that they can affect the system performance and efficiency. Then we formulate a stochastic network optimization problem, which aims at achieving objectives on the queue stability, energy cost and cache hit rate simultaneously, through the dynamic control of server activeness and request dispatching. The problem is transformed into a minimization problem in each time slot, which is further addressed through the proposed efficient online algorithm based on dynamic programming. Moreover, we improve the scheme with several practical considerations in the scheme implementation. Finally, the proposed algorithm and the improvements are evaluated through extensive experiments.