PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Image

被引:5
|
作者
Li, Jianhui [1 ,2 ]
Li, Jianmin [1 ]
Zhang, Haoji [1 ]
Liu, Shilong [1 ]
Wang, Zhengyi [1 ]
Xiao, Zihao [3 ]
Zheng, Kaiwen [1 ]
Zhu, Jun [1 ]
机构
[1] Tsinghua Univ, Inst AI, Dept Comp Sci & Technol, Beijing, Peoples R China
[2] Xian Satellite Control Ctr, State Key Lab Astronaut Dynam, Xian, Peoples R China
[3] RealAI, Beijing, Peoples R China
关键词
D O I
10.1109/CVPR52729.2023.00826
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We study the 3D-aware image attribute editing problem in this paper, which has wide applications in practice. Recent methods solved the problem by training a shared encoder to map images into a 3D generator's latent space or by per-image latent code optimization and then edited images in the latent space. Despite their promising results near the input view, they still suffer from the 3D inconsistency of produced images at large camera poses and imprecise image attribute editing, like affecting unspecified attributes during editing. For more efficient image inversion, we train a shared encoder for all images. To alleviate 3D inconsistency at large camera poses, we propose two novel methods, an alternating training scheme and a multi-view identity loss, to maintain 3D consistency and subject identity. As for imprecise image editing, we attribute the problem to the gap between the latent space of real images and that of generated images. We compare the latent space and inversion manifold of GAN models and demonstrate that editing in the inversion manifold can achieve better results in both quantitative and qualitative evaluations. Extensive experiments show that our method produces more 3D consistent images and achieves more precise image editing than previous work. Source code and pretrained models can be found on our project page: https://mybabyyh.github.io/Preim3D/.
引用
收藏
页码:8549 / 8558
页数:10
相关论文
共 50 条
  • [41] Efficiently Modeling 3D Scenes from a Single Image
    Iizuka, Satoshi
    Kanamori, Yoshihiro
    Mitani, Jun
    Fukui, Yukio
    IEEE COMPUTER GRAPHICS AND APPLICATIONS, 2012, 32 (06) : 18 - 25
  • [42] Video supervised for 3D reconstruction from single image
    Yijie Zhong
    Zhengxing Sun
    Shoutong Luo
    Yunhan Sun
    Yi Wang
    Multimedia Tools and Applications, 2022, 81 : 15061 - 15083
  • [43] Reconstructing a 3D line from a single catadioptric image
    Lanman, Douglas
    Wachs, Megan
    Taubin, Gabriel
    Cukierman, Fernando
    THIRD INTERNATIONAL SYMPOSIUM ON 3D DATA PROCESSING, VISUALIZATION, AND TRANSMISSION, PROCEEDINGS, 2007, : 89 - 96
  • [44] Technical Perspective 3D Image Editing Made Easy
    Igarashi, Takeo
    COMMUNICATIONS OF THE ACM, 2016, 59 (12) : 120 - 120
  • [45] Vista3D: Unravel the 3D Darkside of a Single Image
    Shen, Qiuhong
    Yang, Xingyi
    Mi, Michael Bi
    Wang, Xinchao
    COMPUTER VISION-ECCV 2024, PT XXII, 2025, 15080 : 405 - 421
  • [46] Extracting 3D Layout From a Single Image Using Global Image Structures
    Lou, Zhongyu
    Gevers, Theo
    Hu, Ninghang
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2015, 24 (10) : 3098 - 3108
  • [47] ConRad: Image Constrained Radiance Fields for 3D Generation from a Single Image
    Purushwalkam, Senthil
    Naik, Nikhil
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [48] A PROTOCOL FOR 3D IMAGE-RECONSTRUCTION FROM A SINGLE IMAGE OF AN OBLIQUE SECTION
    TAYLOR, KA
    CROWTHER, RA
    ULTRAMICROSCOPY, 1991, 38 (01) : 85 - 103
  • [49] 3D Snapshot: Invertible Embedding of 3D Neural Representations in a Single Image
    Lu, Yuqin
    Deng, Bailin
    Zhong, Zhixuan
    Zhang, Tianle
    Quan, Yuhui
    Cai, Hongmin
    He, Shengfeng
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2024, 46 (12) : 11524 - 11531
  • [50] Instruct Pix-to-3D: Instructional 3D object generation from a single image
    Cai, Weiwei
    Liu, Wen
    Li, Wanzhang
    Zhao, Zibo
    Yin, Fukun
    Chen, Xin
    Zhao, Lei
    Chen, Tao
    NEUROCOMPUTING, 2024, 600