Deformable Attention Network for Efficient Space-Time Video Super-Resolution

被引:0
|
作者
Wang, Hua [1 ,2 ]
Chamchong, Rapeeporn [1 ]
Chomphuwiset, Phatthanaphong [3 ]
Pawara, Pornntiwa [1 ]
机构
[1] Mahasarakham Univ, Fac Informat, Dept Comp Sci, Maha Sarakham, Thailand
[2] Putian Univ, New Engn Ind Coll, Putian, Peoples R China
[3] MQ Sq, Bangkok, Thailand
关键词
image enhancement; image processing; image resolution;
D O I
10.1049/ipr2.70026
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Space-time video super-resolution (STVSR) aims to construct high space-time resolution video sequences from low frame rate and low-resolution video sequences. While recent STVSR works combine temporal interpolation and spatial super-resolution in a unified framework, they face challenges in computational complexity across both temporal and spatial dimensions, particularly in achieving accurate intermediate frame interpolation and efficient temporal information utilisation. To address these, we propose a deformable attention network for efficient STVSR. Specifically, we introduce a deformable interpolation block that employs hierarchical feature fusion to effectively handle complex inter-frame motions at multiple scales, enabling more accurate intermediate frame generation. To fully utilise temporal information, we design a temporal feature shuffle block (TFSB) to efficiently exchange complementary information across multiple frames. Additionally, we develop a motion feature enhancement block incorporating channel attention mechanism to selectively emphasise motion-related features, further boosting TFSB's effectiveness. Experimental results on benchmark datasets definitively demonstrate that our proposed method achieves competitive performance in STVSR tasks.
引用
收藏
页数:13
相关论文
共 50 条
  • [31] RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution
    Geng, Zhicheng
    Liang, Luming
    Ding, Tianyu
    Zharkov, Ilya
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 17420 - 17430
  • [32] Video super-resolution with phase-aided deformable alignment network
    Cai, Zhuojun
    Chen, Yaowu
    Tian, Xiang
    Jiang, Rongxin
    JOURNAL OF ELECTRONIC IMAGING, 2020, 29 (03)
  • [33] An Efficient Network Design for Face Video Super-resolution
    Yu, Feng
    Li, He
    Bian, Sige
    Tang, Yongming
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021, : 1513 - 1520
  • [34] TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
    Tian, Yapeng
    Zhang, Yulun
    Fu, Yun
    Xu, Chenliang
    2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 3357 - 3366
  • [35] Space-Time Super-Resolution for Light Field Videos
    Xiao, Zeyu
    Cheng, Zhen
    Xiong, Zhiwei
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2023, 32 : 4785 - 4799
  • [36] VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution
    Chen, Zeyuan
    Chen, Yinbo
    Liu, Jingwen
    Xu, Xingqian
    Goel, Vidit
    Wang, Zhangyang
    Shi, Humphrey
    Wang, Xiaolong
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 2037 - 2047
  • [37] Controllable Space-Time Video Super-Resolution via Enhanced Bidirectional Flow Warping
    Zhang, Yuantong
    Wang, Huairui
    Chen, Zhenzhong
    2022 IEEE INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2022,
  • [38] Space-Time Super-Resolution for Light Field Videos
    Xiao, Zeyu
    Cheng, Zhen
    Xiong, Zhiwei
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2023, 32 : 4785 - 4799
  • [39] Understanding Deformable Alignment in Video Super-Resolution
    Chan, Kelvin C. K.
    Wang, Xintao
    Yu, Ke
    Dong, Chao
    Loy, Chen Change
    THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 973 - 981
  • [40] Deformable transformer for endoscopic video super-resolution
    Song, Xiaowei
    Tang, Hui
    Yang, Chunfeng
    Zhou, Guangquan
    Wang, Yangang
    Huang, Xinjun
    Hua, Jie
    Coatrieux, Gouenou
    He, Xiaopu
    Chen, Yang
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2022, 77