360SRL: A SEQUENTIAL REINFORCEMENT LEARNING APPROACH FOR ABR TILE-BASED 360 VIDEO STREAMING

Tile-based 360-degree video (360 video) streaming, employed with adaptive bitrate (ABR) algorithms, is a promising approach to offer high video quality of experience (QoE) within limited network bandwidth. Existing ABR algorithms, however, fail to achieve optimal performance in real-world fluctuated network conditions as they heavily rely on unbiased bandwidth predictions. Recently, reinforcement learning (RL) has shown promising potential in generating better ABR algorithms in 2D video streaming. However, unlike existed work in 2D video streaming, directly applying RL in the tile-based 360 video streaming is infeasible due to the resulting exponential decision space. To overcome these limitations, we propose in this paper 360SRL, an improved ABR algorithm employing Sequential RL (360SRL). Firstly, we reduce the decision space of 360SRL from exponential to linear by introducing a sequential ABR decision structure, thus making it feasible to be employed with RL. Secondly, instead of relying on accurate bandwidth predictions, 360SRL learns to make ABR decisions solely through observations of the resulting QoE performance of past decisions. Finally, we compare 360SRL to state-of-the-art ABR algorithms using trace-driven experiments. The experiment results demonstrate that 360SRL outperforms state-of-the-art algorithms with around 12% improvement in average QoE.