Deformable Attention Network for Efficient Space-Time Video Super-Resolution

被引:0
|
作者
Wang, Hua [1 ,2 ]
Chamchong, Rapeeporn [1 ]
Chomphuwiset, Phatthanaphong [3 ]
Pawara, Pornntiwa [1 ]
机构
[1] Mahasarakham Univ, Fac Informat, Dept Comp Sci, Maha Sarakham, Thailand
[2] Putian Univ, New Engn Ind Coll, Putian, Peoples R China
[3] MQ Sq, Bangkok, Thailand
关键词
image enhancement; image processing; image resolution;
D O I
10.1049/ipr2.70026
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Space-time video super-resolution (STVSR) aims to construct high space-time resolution video sequences from low frame rate and low-resolution video sequences. While recent STVSR works combine temporal interpolation and spatial super-resolution in a unified framework, they face challenges in computational complexity across both temporal and spatial dimensions, particularly in achieving accurate intermediate frame interpolation and efficient temporal information utilisation. To address these, we propose a deformable attention network for efficient STVSR. Specifically, we introduce a deformable interpolation block that employs hierarchical feature fusion to effectively handle complex inter-frame motions at multiple scales, enabling more accurate intermediate frame generation. To fully utilise temporal information, we design a temporal feature shuffle block (TFSB) to efficiently exchange complementary information across multiple frames. Additionally, we develop a motion feature enhancement block incorporating channel attention mechanism to selectively emphasise motion-related features, further boosting TFSB's effectiveness. Experimental results on benchmark datasets definitively demonstrate that our proposed method achieves competitive performance in STVSR tasks.
引用
收藏
页数:13
相关论文
共 50 条
  • [21] Bidirectional Multi-scale Deformable Attention for Video Super-Resolution
    Zhou, Zhenghua
    Xue, Boxiang
    Wang, Hai
    Zhao, Jianwei
    Multimedia Tools and Applications, 83 (09): : 27809 - 27830
  • [22] Deformable Non-Local Network for Video Super-Resolution
    Wang, Hua
    Su, Dewei
    Liu, Chuangchuang
    Jin, Longcun
    Sun, Xianfang
    Peng, Xinyi
    IEEE ACCESS, 2019, 7 : 177734 - 177744
  • [23] Deformable Spatial-Temporal Attention for Lightweight Video Super-Resolution
    Xue, Tong
    Huang, Xinyi
    Li, Dengshi
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2023, PT X, 2024, 14434 : 482 - 493
  • [24] Bidirectional Multi-scale Deformable Attention for Video Super-Resolution
    Zhenghua Zhou
    Boxiang Xue
    Hai Wang
    Jianwei Zhao
    Multimedia Tools and Applications, 2024, 83 : 27809 - 27830
  • [25] Space-Time Video Super-Resolution Based on Long-Term Time Dependence
    Huang, Xinyi
    Xue, Tong
    Hu, Yiwen
    Lan, Hui
    Sun, Jinshan
    ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING, ICANN 2023, PT VII, 2023, 14260 : 514 - 526
  • [26] Learning Inter-Frame Information for Space-Time Video Super-Resolution
    Zhang, Zhuo
    Guo, Haifa
    Ren, Sheng
    Guo, Kehua
    2022 TENTH INTERNATIONAL CONFERENCE ON ADVANCED CLOUD AND BIG DATA, CBD, 2022, : 139 - 144
  • [27] Fast and Context-Aware Framework for Space-Time Video Super-Resolution
    Zhang, Xueheng
    Chen, Li
    Song, Li
    Gao, Zhiyong
    2021 INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2021,
  • [28] Combining optical flow and Swin Transformer for Space-Time video super-resolution
    Wang, Xin
    Wang, Hua
    Zhang, Mingli
    Zhang, Fan
    Engineering Applications of Artificial Intelligence, 2024, 137
  • [29] 3DAttGAN: A 3D Attention-Based Generative Adversarial Network for Joint Space-Time Video Super-Resolution
    Fu, Congrui
    Yuan, Hui
    Shen, Liquan
    Hamzaoui, Raouf
    Zhang, Hao
    IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE, 2024, 8 (04): : 3117 - 3128
  • [30] Hybrid Domain Attention Network for Efficient Super-Resolution
    Zhang, Qian
    Feng, Linxia
    Liang, Hong
    Yang, Ying
    SYMMETRY-BASEL, 2022, 14 (04):