Temporally Consistent Semantic Video Editing

被引:13
|
作者
Xu, Yiran [1 ]
AlBahar, Badour [2 ]
Huang, Jia-Bin [1 ]
机构
[1] Univ Maryland, College Pk, MD 20742 USA
[2] Virginia Tech, Blacksburg, VA USA
来源
关键词
Video editing; GAN editing; Video consistency;
D O I
10.1007/978-3-031-19784-0_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.
引用
收藏
页码:357 / 374
页数:18
相关论文
共 50 条
  • [31] Region-based Temporally Consistent Video Post-processing
    Dong, Xuan
    Bonev, Boyan
    Zhu, Yu
    Yuille, Alan L.
    2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2015, : 714 - 722
  • [32] Hybrid Skeleton Driven Surface Registration for Temporally Consistent Volumetric Video
    Regateiro, Joao
    Volino, Marco
    Hilton, Adrian
    2018 INTERNATIONAL CONFERENCE ON 3D VISION (3DV), 2018, : 514 - 522
  • [33] Video OWL-ViT: Temporally-consistent open-world localization in video
    Heigold, Georg
    Minderer, Matthias
    Gritsenko, Alexey
    Bewley, Alex
    Keysers, Daniel
    Lucic, Mario
    Yu, Fisher
    Kipf, Thomas
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 13756 - 13765
  • [34] DeepTemporalSeg: Temporally Consistent Semantic Segmentation of 3D LiDAR Scans
    Dewan, Ayush
    Burgard, Wolfram
    2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2020, : 2624 - 2630
  • [35] Bayesian modeling of video editing and structure: Semantic features for video summarization and browsing
    Vasconcelos, N
    Lippman, A
    1998 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING - PROCEEDINGS, VOL 3, 1998, : 153 - 157
  • [36] Generation of Temporally Consistent Depth Maps Using Nosie Removal from Video
    Stankiewicz, Olgierd
    Wegner, Krzysztof
    COMPUTER VISION AND GRAPHICS, PT II, 2010, 6375 : 292 - 299
  • [37] ESTIMATION OF TEMPORALLY-CONSISTENT DEPTH MAPS FROM VIDEO WITH REDUCED NOISE
    Stankiewicz, Olgierd
    Domanski, Marek
    Wegner, Krzysztof
    2015 3DTV-CONFERENCE - TRUE VISION - CAPTURE, TRANSMISSION AND DISPLAY OF 3D VIDEO (3DTV-CON), 2015,
  • [38] Multi-class video segmentation based on temporally consistent energy model
    Bing, Liu
    Advances in Information Sciences and Service Sciences, 2012, 4 (01): : 85 - 92
  • [39] Online Temporally Consistent Indoor Depth Video Enhancement via Static Structure
    Sheng, Lu
    Ngan, King Ngi
    Lim, Chern-Loon
    Li, Songnan
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2015, 24 (07) : 2197 - 2211
  • [40] Temporally Consistent Superpixels
    Reso, Matthias
    Jachalsky, Joern
    Rosenhahn, Bodo
    Ostermann, Joern
    2013 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2013, : 385 - 392