An optimization approach to load balancing, scheduling and right sizing of cloud computing systems with data locality

We model a cloud computing infrastructure over a set of locations, with multiple server instances per location. The service rate offered by each server is differentiated by the type of task, depending on whether its data is locally available. Resource allocation issues include: load balancing between locations, scheduling of tasks within each location, and sizing of the active server population at each location. Using a fluid queue model, we first characterize the capacity region of a system with a fixed number of servers at each location, recovering known results on throughput optimality of certain policies. Next we allow the server populations to vary, and pose the problem of minimizing a convex cost function subject to load stabilization. Such right sizing of service capacity must be done dynamically, without knowledge of the load. Invoking Lagrange duality, we propose a primal-dual dynamic control with queues and server populations as state variables, that also embeds the optimal load balancing and scheduling. We prove its stability for fixed, unknown load, and explore by simulation its behavior under time-varying loads.