Temporally Consistent Semantic Video Editing

被引:13
|
作者
Xu, Yiran [1 ]
AlBahar, Badour [2 ]
Huang, Jia-Bin [1 ]
机构
[1] Univ Maryland, College Pk, MD 20742 USA
[2] Virginia Tech, Blacksburg, VA USA
来源
关键词
Video editing; GAN editing; Video consistency;
D O I
10.1007/978-3-031-19784-0_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.
引用
收藏
页码:357 / 374
页数:18
相关论文
共 50 条
  • [41] Temporal Consistent Automatic Video Colorization via Semantic Correspondence
    Beijing University of Posts and Telecommunications, School of Artificial Intelligence, China
    不详
    IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. Workshops, 2160, (1836-1845):
  • [42] Towards Consistent Video Editing with Text-to-Image Diffusion Models
    Zhang, Zicheng
    Li, Bonan
    Nie, Xuecheng
    Han, Congying
    Guo, Tiande
    Liu, Luoqi
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [43] Context-Consistent Semantic Image Editing with Style-Preserved Modulation
    Luo, Wuyang
    Yang, Su
    Wang, Hong
    Long, Bo
    Zhang, Weishan
    COMPUTER VISION - ECCV 2022, PT XVII, 2022, 13677 : 561 - 578
  • [44] Temporally consistent video colorization with deep feature propagation and self-regularization learning
    Yihao Liu
    Hengyuan Zhao
    Kelvin C. K. Chan
    Xintao Wang
    Chen Change Loy
    Yu Qiao
    Chao Dong
    Computational Visual Media, 2024, 10 : 375 - 395
  • [45] Spatio-temporally Consistent Multi-view Video Synthesis for Autostereoscopic Displays
    Lin, Shu-Jyuan
    Cheng, Chia-Ming
    Lai, Shang-Hong
    ADVANCES IN MULTIMEDIA INFORMATION PROCESSING - PCM 2009, 2009, 5879 : 532 - 542
  • [46] Temporally Consistent Depth Map Estimation for 3D Video Generation and Coding
    Lee, Sang-Beom
    Ho, Yo-Sung
    CHINA COMMUNICATIONS, 2013, 10 (05) : 39 - 49
  • [47] Temporally consistent reconstruction from multiple video streams using enhanced belief propagation
    Larsen, E. Scott
    Mordohai, Philippos
    Pollefeys, Marc
    Fuchs, Henry
    2007 IEEE 11TH INTERNATIONAL CONFERENCE ON COMPUTER VISION, VOLS 1-6, 2007, : 1440 - 1447
  • [48] Spatio-Temporally Consistent Color and Structure Optimization for Multiview Video Color Correction
    Lu, Shao-Ping
    Ceulemans, Beerend
    Munteanu, Adrian
    Schelkens, Peter
    IEEE TRANSACTIONS ON MULTIMEDIA, 2015, 17 (05) : 577 - 590
  • [49] Temporally consistent video colorization with deep feature propagation and self-regularization learning
    Liu, Yihao
    Zhao, Hengyuan
    Chan, Kelvin C. K.
    Wang, Xintao
    Loy, Chen Change
    Qiao, Yu
    Dong, Chao
    COMPUTATIONAL VISUAL MEDIA, 2024, 10 (02) : 375 - 395
  • [50] Joint 2D-3D Temporally Consistent Semantic Segmentation of Street Scenes
    Floros, Georgios
    Leibe, Bastian
    2012 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2012, : 2823 - 2830