PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Image

被引:5
|
作者
Li, Jianhui [1 ,2 ]
Li, Jianmin [1 ]
Zhang, Haoji [1 ]
Liu, Shilong [1 ]
Wang, Zhengyi [1 ]
Xiao, Zihao [3 ]
Zheng, Kaiwen [1 ]
Zhu, Jun [1 ]
机构
[1] Tsinghua Univ, Inst AI, Dept Comp Sci & Technol, Beijing, Peoples R China
[2] Xian Satellite Control Ctr, State Key Lab Astronaut Dynam, Xian, Peoples R China
[3] RealAI, Beijing, Peoples R China
关键词
D O I
10.1109/CVPR52729.2023.00826
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We study the 3D-aware image attribute editing problem in this paper, which has wide applications in practice. Recent methods solved the problem by training a shared encoder to map images into a 3D generator's latent space or by per-image latent code optimization and then edited images in the latent space. Despite their promising results near the input view, they still suffer from the 3D inconsistency of produced images at large camera poses and imprecise image attribute editing, like affecting unspecified attributes during editing. For more efficient image inversion, we train a shared encoder for all images. To alleviate 3D inconsistency at large camera poses, we propose two novel methods, an alternating training scheme and a multi-view identity loss, to maintain 3D consistency and subject identity. As for imprecise image editing, we attribute the problem to the gap between the latent space of real images and that of generated images. We compare the latent space and inversion manifold of GAN models and demonstrate that editing in the inversion manifold can achieve better results in both quantitative and qualitative evaluations. Extensive experiments show that our method produces more 3D consistent images and achieves more precise image editing than previous work. Source code and pretrained models can be found on our project page: https://mybabyyh.github.io/Preim3D/.
引用
收藏
页码:8549 / 8558
页数:10
相关论文
共 50 条
  • [31] 3D corrective nose reconstruction from a single image
    Yanlong Tang
    Yun Zhang
    Xiaoguang Han
    Fang-Lue Zhang
    Yu-Kun Lai
    Ruofeng Tong
    Computational Visual Media, 2022, 8 : 225 - 237
  • [32] PushNet: 3D reconstruction from a single image by pushing
    Guiju Ping
    Han Wang
    Neural Computing and Applications, 2024, 36 : 6629 - 6641
  • [33] Interactive 3D model extraction from a single image
    François, ARJ
    Medioni, GG
    IMAGE AND VISION COMPUTING, 2001, 19 (06) : 317 - 328
  • [34] Video supervised for 3D reconstruction from single image
    Zhong, Yijie
    Sun, Zhengxing
    Luo, Shoutong
    Sun, Yunhan
    Wang, Yi
    MULTIMEDIA TOOLS AND APPLICATIONS, 2022, 81 (11) : 15061 - 15083
  • [35] Localizing object parts in 3D from a single image
    Shen Yin
    Bin Zhou
    Mingjia Yang
    Yu Zhang
    Science China Information Sciences, 2019, 62
  • [36] From Single Image Query to Detailed 3D Reconstruction
    Schonberger, Johannes L.
    Radenovic, Filip
    Chum, Ondrej
    Frahm, Jan-Michael
    2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2015, : 5126 - 5134
  • [37] 3D corrective nose reconstruction from a single image
    Tang, Yanlong
    Zhang, Yun
    Han, Xiaoguang
    Zhang, Fang-Lue
    Lai, Yu-Kun
    Tong, Ruofeng
    COMPUTATIONAL VISUAL MEDIA, 2022, 8 (02) : 225 - 237
  • [38] DeepHuman: 3D Human Reconstruction From a Single Image
    Zheng, Zerong
    Yu, Tao
    Wei, Yixuan
    Dai, Qionghai
    Liu, Yebin
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 7738 - 7748
  • [39] DEPTH PREDICTION FROM A SINGLE IMAGE WITH 3D CONSISTENCY
    Tian, Hu
    Li, Fei
    2018 25TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2018, : 111 - 115
  • [40] Localizing object parts in 3D from a single image
    Yin, Shen
    Zhou, Bin
    Yang, Mingjia
    Zhang, Yu
    SCIENCE CHINA-INFORMATION SCIENCES, 2019, 62 (07)