SAFR: A Real-Time Communication System with Adaptive Frame Rate

Real-Time Communication (RTC) systems tend to drop some frames during transmission to cope with network congestion. Though frame-dropping is helpful to low-delay performance, it may cause sudden increase in frame loss and noticeable Quality of Experience (QoE) degradation. In this paper, we propose a novel RTC system, SAFR (System of Adaptive Frame Rate), to jointly optimize all the key indicators of QoE performance. Firstly, a Frame Rate Controller (FRC) is designed for making reasonable frame-rate alteration decisions. Then, a reinforcement learning-based bandwidth prediction module, named Target Rate Controller (TRC), is established to improve the bandwidth utilization by cooperating with FRC. Finally, an AI-based frame interpolation module is applied on the receiver side to mitigate the frame-rate degradation. Compared to traditional RTC systems, SAFR achieves 16.3%-26.7% frame-delay reductions, 61.8%-69.9% frame loss rate decreases, 6.4%-11.3% QoE improvements, with up to 25.3% savings of bandwidth at the same time. Especially, SAFR shows greater superiority under worse network conditions. The supplementary materials[10] further discuss the settings of hyper-parameters and pseudo-codes of FRC. Our test demos are available at: https://github.com/xiaosayin/SAFR.