STFE-VC: Spatio-temporal feature enhancement for learned video compression

With the increasing growth of video data, limited bandwidth and hardware resource constraints demand more efficient video compression. Current learned video compression methods have shown promising performance. However, these methods mainly rely on the optical flow networks to perform temporal prediction, which may suffer from inaccurate motion estimation and introduce extra artifacts to reconstructed frames. In this paper, we propose a spatio-temporal feature enhancement method for learned video compression to better model the inter-frame motion patterns and reduce compression artifacts. Specifically, we introduce a spatio-temporal motion enhancement module that further extracts the feature representation of original motion vector to enhance corresponding spatial and temporal components. Then, we introduce an in-loop filtering enhancement module that employs cascaded residual blocks to progressively enhance feature textures and provide higher- quality temporal domain reference signals for subsequent reconstruction. More importantly, our proposed method can be integrated into the widely-used residual coding and contextual coding schemes. Comprehensive experiments demonstrate that our integrated methods are superior to the previous learned methods on JCTVC, UVG and MCL-JCV benchmark datasets. In addition, our integrated methods also outperform the latest generalized video coding standard (H.266/VVC) by a larger margin in terms of MS-SSIM metric.