KubeSC-RTP: Smart scheduler for Kubernetes platform on CPU-GPU heterogeneous systems

Heterogeneous systems composed of multiple CPUs and GPUs are progressively attractive as platforms for high performance computing because of their higher performance. Especially with the use of containers which are rapidly replacing virtual machines as the compute instance of choice in cloud-based deployments such in Kubernetes clusters. The task scheduling in a heterogeneous environment became one of the most important issues considered by the platform providers. The ability to choose the appropriate device, CPU or GPU, has a direct impact on the performance of a particular system. It reduces total processing time and increases customer satisfaction. In heterogeneous systems, optimizing resource consumption is a critical aspect for cloud service providers. Adequate scheduling of an application implies optimization of its execution time, which results in resource consumption for the service provider. The development of algorithms for scheduling applications in heterogeneous computing systems has received a significant amount of attention in recent years. A variety of efforts are dedicated to the design of such scheduling algorithms. This article is one of those efforts. We present in this work, KubeSC-RTP, a scheduler for Kubernetes environment using machine learning based on runtime prediction of the applications in order to better select the appropriate device, CPU or GPU.