PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Image

被引:5
|
作者
Li, Jianhui [1 ,2 ]
Li, Jianmin [1 ]
Zhang, Haoji [1 ]
Liu, Shilong [1 ]
Wang, Zhengyi [1 ]
Xiao, Zihao [3 ]
Zheng, Kaiwen [1 ]
Zhu, Jun [1 ]
机构
[1] Tsinghua Univ, Inst AI, Dept Comp Sci & Technol, Beijing, Peoples R China
[2] Xian Satellite Control Ctr, State Key Lab Astronaut Dynam, Xian, Peoples R China
[3] RealAI, Beijing, Peoples R China
关键词
D O I
10.1109/CVPR52729.2023.00826
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We study the 3D-aware image attribute editing problem in this paper, which has wide applications in practice. Recent methods solved the problem by training a shared encoder to map images into a 3D generator's latent space or by per-image latent code optimization and then edited images in the latent space. Despite their promising results near the input view, they still suffer from the 3D inconsistency of produced images at large camera poses and imprecise image attribute editing, like affecting unspecified attributes during editing. For more efficient image inversion, we train a shared encoder for all images. To alleviate 3D inconsistency at large camera poses, we propose two novel methods, an alternating training scheme and a multi-view identity loss, to maintain 3D consistency and subject identity. As for imprecise image editing, we attribute the problem to the gap between the latent space of real images and that of generated images. We compare the latent space and inversion manifold of GAN models and demonstrate that editing in the inversion manifold can achieve better results in both quantitative and qualitative evaluations. Extensive experiments show that our method produces more 3D consistent images and achieves more precise image editing than previous work. Source code and pretrained models can be found on our project page: https://mybabyyh.github.io/Preim3D/.
引用
收藏
页码:8549 / 8558
页数:10
相关论文
共 50 条
  • [1] Single Image 3D Without a Single 3D Image
    Fouhey, David F.
    Hussain, Wajahat
    Gupta, Abhinav
    Hebert, Martial
    2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, : 1053 - 1061
  • [2] PR3D: Precise and realistic 3D face reconstruction from a single image
    Huang, Zhangjin
    Wu, Xing
    COMPUTER ANIMATION AND VIRTUAL WORLDS, 2024, 35 (03)
  • [3] 3D Cinemagraphy from a Single Image
    Li, Xingyi
    Cao, Zhiguo
    Sun, Huiqiang
    Zhang, Jianming
    Xian, Ke
    Ling, Guosheng
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, : 4595 - 4605
  • [4] 3D Reconstruction from A Single Image
    Ping, Guiju
    Wang, Han
    PROCEEDINGS OF THE IEEE 2019 9TH INTERNATIONAL CONFERENCE ON CYBERNETICS AND INTELLIGENT SYSTEMS (CIS) ROBOTICS, AUTOMATION AND MECHATRONICS (RAM) (CIS & RAM 2019), 2019, : 47 - 52
  • [5] HIERARCHICAL 3D PERCEPTION FROM A SINGLE IMAGE
    Luo, Ping
    He, Jiajie
    Lin, Liang
    Chao, Hongyang
    2009 16TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1-6, 2009, : 4265 - +
  • [6] Nonrigid 3D Reconstruction from a Single Image
    Ma, Wen-juan
    2016 INTERNATIONAL CONFERENCE ON INFORMATION SYSTEM AND ARTIFICIAL INTELLIGENCE (ISAI 2016), 2016, : 138 - 142
  • [7] Interactive 3D editing tools for image segmentation
    Kang, Y
    Engelke, K
    Kalender, WA
    MEDICAL IMAGE ANALYSIS, 2004, 8 (01) : 35 - 46
  • [8] Fast and Precise Face Alignment and 3D Shape Reconstruction from a Single 2D Image
    Zhao, Ruiqi
    Wang, Yan
    Benitez-Quiroz, C. Fabian
    Liu, Yaojie
    Martinez, Aleix M.
    COMPUTER VISION - ECCV 2016 WORKSHOPS, PT II, 2016, 9914 : 590 - 603
  • [9] HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image
    Wu, Tong
    Li, Zhibing
    Yang, Shuai
    Zhang, Pan
    Pan, Xingang
    Wang, Jiaqi
    Lin, Dahua
    Liu, Ziwei
    PROCEEDINGS OF THE SIGGRAPH ASIA 2023 CONFERENCE PAPERS, 2023,
  • [10] 3D Visual Proxemics: Recognizing Human Interactions in 3D from a Single Image
    Chakraborty, Ishani
    Cheng, Hui
    Javed, Omar
    2013 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2013, : 3406 - 3413