Spatio-Temporal Inference Transformer Network for Video Inpainting

被引:0
|
作者
Tudavekar, Gajanan [1 ,2 ]
Saraf, Santosh S. [2 ]
Patil, Sanjay R. [3 ]
机构
[1] Angadi Inst Technol & Management, Dept Elect & Commun Engn, Belagavi 590009, Karnataka, India
[2] KLS Gogte Inst Technol, Dept Elect & Commun Engn, Belagavi 590008, Karnataka, India
[3] Dnyanshree Inst Engn & Technol, Dept Elect & Telecommun Engn, Sajjangad Rd, Satara 415013, Maharashtra, India
关键词
Image inpainting; video inpainting; Transformer Network; deep learning; IMAGE QUALITY ASSESSMENT; OBJECT REMOVAL;
D O I
10.1142/S0219467823500079
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Video inpainting aims to complete in a visually pleasing way the missing regions in video frames. Video inpainting is an exciting task due to the variety of motions across different frames. The existing methods usually use attention models to inpaint videos by seeking the damaged content from other frames. Nevertheless, these methods suffer due to irregular attention weight from spatio-temporal dimensions, thus giving rise to artifacts in the inpainted video. To overcome the above problem, Spatio-Temporal Inference Transformer Network (STITN) has been proposed. The STITN aligns the frames to be inpainted and concurrently inpaints all the frames, and a spatio-temporal adversarial loss function improves the STITN. Our method performs considerably better than the existing deep learning approaches in quantitative and qualitative evaluation.
引用
收藏
页数:16
相关论文
共 50 条
  • [21] Inference of network anomaly propagation using spatio-temporal correlation
    Amaral, Alexandre Aguiar
    Zarpelao, Bruno Bogaz
    Mendes, Leonardo de Souza
    Puga Coelho Rodrigues, Joel Jose
    Proenca Junior, Mario Lemes
    JOURNAL OF NETWORK AND COMPUTER APPLICATIONS, 2012, 35 (06) : 1781 - 1792
  • [22] Spatio-Temporal Dynamic Inference Network for Group Activity Recognition
    Yuan, Hangjie
    Ni, Dong
    Wang, Mang
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 7456 - 7465
  • [23] Modeling Temporal Dynamics with Function Approximation in Deep Spatio-Temporal Inference Network
    Karnowski, Thomas P.
    Arel, Itamar
    Young, Steven
    BIOLOGICALLY INSPIRED COGNITIVE ARCHITECTURES 2011, 2011, 233 : 174 - +
  • [24] Spatio-temporal Sampling for Video
    Shankar, Mohan
    Pitsiauis, Nikos P.
    Brady, David
    IMAGE RECONSTRUCTION FROM INCOMPLETE DATA V, 2008, 7076
  • [25] A spatio-temporal network for video semantic segmentation in surgical videos
    Maria Grammatikopoulou
    Ricardo Sanchez-Matilla
    Felix Bragman
    David Owen
    Lucy Culshaw
    Karen Kerr
    Danail Stoyanov
    Imanol Luengo
    International Journal of Computer Assisted Radiology and Surgery, 2024, 19 : 375 - 382
  • [26] Spatio-temporal prediction and reconstruction network for video anomaly detection
    Liu, Ting
    Zhang, Chengqing
    Niu, Xiaodong
    Wang, Liming
    PLOS ONE, 2022, 17 (05):
  • [27] A spatio-temporal network for video semantic segmentation in surgical videos
    Grammatikopoulou, Maria
    Sanchez-Matilla, Ricardo
    Bragman, Felix
    Owen, David
    Culshaw, Lucy
    Kerr, Karen
    Stoyanov, Danail
    Luengo, Imanol
    INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, 2023, 19 (2) : 375 - 382
  • [28] A spatio-temporal network for video semantic segmentation in surgical videos
    Grammatikopoulou, Maria
    Sanchez-Matilla, Ricardo
    Bragman, Felix
    Owen, David
    Culshaw, Lucy
    Kerr, Karen
    Stoyanov, Danail
    Luengo, Imanol
    arXiv, 2023,
  • [29] Video object segmentation using spatio-temporal deep network
    Ramaswamy, Akshaya
    Gubbi, Jayavardhana
    Balamuralidhar, P.
    2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
  • [30] SPATIO-TEMPORAL MOTION AGGREGATION NETWORK FOR VIDEO ACTION DETECTION
    Zhang, Hongcheng
    Zhao, Xu
    2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 2180 - 2184