ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation

被引:4
|
作者
Liu, Yahui [1 ]
Chen, Yajing [2 ]
Bao, Linchao [2 ]
Sebe, Nicu [1 ]
Lepri, Bruno [3 ]
De Nadai, Marco [3 ]
机构
[1] Univ Trento, Dept Informat Engn & Comp Sci, I-38123 Trento, Italy
[2] Tencent AI Lab, Shenzhen 518063, Peoples R China
[3] Fdn Bruno Kessler, I-38123 Povo, Italy
基金
欧盟地平线“2020”;
关键词
Face editing; generative adversarial networks (GANs); unsupervised image-to-image translation; GENERATIVE ADVERSARIAL NETWORKS;
D O I
10.1109/TMM.2022.3159115
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recently, there has been an increasing interest in image editing methods that employ pre-trained unconditional image generators (e.g., StyleGAN). However, applying these methods to translate images to multiple visual domains remains challenging. Existing works do not often preserve the domain-invariant part of the image (e.g., the identity in human face translations), or they do not usually handle multiple domains or allow for multi-modal translations. This work proposes an implicit style function (ISF) to straightforwardly achieve multi-modal and multi-domain image-to-image translation from pre-trained unconditional generators. The ISF manipulates the semantics of a latent code to ensure that the image generated from the manipulated code lies in the desired visual domain. Our human faces and animal image manipulations show significantly improved results over the baselines. Our model enables cost-effective multi-modal unsupervised image-to-image translations at high resolution using pre-trained unconditional GANs. The code and data are available at: https://github.com/yhlleo/stylegan-mmuit.
引用
收藏
页码:3343 / 3353
页数:11
相关论文
共 50 条
  • [1] High-Resolution Semantically Consistent Image-to-Image Translation
    Sokolov, Mikhail
    Henry, Christopher
    Storie, Joni
    Storie, Christopher
    Alhassan, Victor
    Turgeon-Pelchat, Mathieu
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2023, 16 : 482 - 492
  • [2] Unsupervised Image-to-Image Translation with Style Consistency
    Lai, Binxin
    Wang, Yuan-Gen
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2023, PT VI, 2024, 14430 : 322 - 334
  • [3] Image-to-image registration to produce high-resolution images
    Sheikho, KM
    Al-Arafi, F
    EARTH OBSERVATION AND REMOTE SENSING, 1997, 14 (05): : 763 - +
  • [4] Consistent Embedded GAN for Image-to-Image Translation
    Xiong, Feng
    Wang, Qianqian
    Gao, Quanxue
    IEEE ACCESS, 2019, 7 : 126651 - 126661
  • [5] Asymmetric GAN for Unpaired Image-to-Image Translation
    Li, Yu
    Tang, Sheng
    Zhang, Rui
    Zhang, Yongdong
    Li, Jintao
    Yan, Shuicheng
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (12) : 5881 - 5896
  • [6] Implicit pairs for boosting unpaired image-to-image translation
    Ginger, Yiftach
    Danon, Dov
    Averbuch-Elor, Hadar
    Cohen-Or, Daniel
    VISUAL INFORMATICS, 2020, 4 (04): : 50 - 58
  • [7] Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation
    Richardson, Elad
    Alaluf, Yuval
    Patashnik, Or
    Nitzan, Yotam
    Azar, Yaniv
    Shapiro, Stav
    Cohen-Or, Daniel
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 2287 - 2296
  • [8] OmniStyleGAN for Style-Guided Image-to-Image Translation
    Zhao, Qianyi
    Wang, Mengyin
    Zhang, Qing
    Wang, Fasheng
    Sun, Fuming
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2024, PT XI, 2025, 15041 : 351 - 365
  • [9] SHUNIT: Style Harmonization for Unpaired Image-to-Image Translation
    Song, Seokbeom
    Lee, Suhyeon
    Seong, Hongje
    Min, Kyoungwon
    Kim, Euntai
    THIRTY-SEVENTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 37 NO 2, 2023, : 2292 - 2302
  • [10] Image-to-image Translation via Hierarchical Style Disentanglement
    Li, Xinyang
    Zhang, Shengchuan
    Hu, Jie
    Cao, Liujuan
    Hong, Xiaopeng
    Mao, Xudong
    Huang, Feiyue
    Wu, Yongjian
    Ji, Rongrong
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 8635 - 8644