NOVA: Neural-Optimized Viewport Adaptive 360-Degree Video Streaming at the Edge

The 360-degree video streaming service provides a unique immersive viewing experience for users, who can freely change their Field-of-View (FoV) to view different portions of the videos. However, the demands for high throughput and low latency for 360-degree video pose substantial challenges to the current network infrastructure. Super Resolution (SR) is the procedure for reconstructing high-resolution images from low-resolution ones. Hence, caching video content on the network edge in advance, which is near end users, and applying the SR technique can significantly alleviate the transmission latency. In this article, we describe NOVA, an efficient Neural-Optimized Viewport Adaptive 360-degree video streaming system to improve the Quality of Experience (QoE) of users. In NOVA, we first design a foveated rendering SR approach to super-resolve video tiles utilizing computational resources at the edge. Subsequently, we present a meta-learning-based Multi-Agent Reinforcement Learning (MARL) algorithm to select SR depths and video tiles inside users' viewports for agile video tile adaptation to optimize overall QoE under frequent network fluctuations. Finally, we implement the holistic prototype of NOVA and evaluate its performance on various real-world network datasets. Extensive experiments illustrate that compared to the state-of-the-art algorithms, NOVA improves average user-perceived QoE by up to 27%.