An Analytic Model of Traffic Surges for Multi-Server Queues in Cloud Environments

Many computer systems, such as cloud computing environments and Internet datacenters, consist of a multitude of servers that process user requests. The performance and scalability of these environments suffer significantly when the workload surges to levels that cause the arrival rate of requests to exceed the system's capacity to process them. This paper models these systems as G/G/c queues and derives equations that estimate the impact of workload surges on these multiserver systems. The existing queuing literature offers approximations and/or bounds for G/G/c systems in equilibrium, but not when these systems are subject to workload surges. This paper's main contributions are: (1) Generic equations for surges of any shape. (2) A set of equations to estimate the impact of trapezoidal and triangular shaped surges on response time. (3) Extensive validations of the derived equations using a G/G/c simulator developed by the authors (available online) and with Google cluster-usage trace workloads. The results show that the equations estimate with great accuracy the impact of surges on response time. The work presented in this paper can be utilized in the design of autonomic elasticity controllers in the cloud to vary the number of servers to mitigate the impact of workload surges.