ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation

被引:4
|
作者
Liu, Yahui [1 ]
Chen, Yajing [2 ]
Bao, Linchao [2 ]
Sebe, Nicu [1 ]
Lepri, Bruno [3 ]
De Nadai, Marco [3 ]
机构
[1] Univ Trento, Dept Informat Engn & Comp Sci, I-38123 Trento, Italy
[2] Tencent AI Lab, Shenzhen 518063, Peoples R China
[3] Fdn Bruno Kessler, I-38123 Povo, Italy
基金
欧盟地平线“2020”;
关键词
Face editing; generative adversarial networks (GANs); unsupervised image-to-image translation; GENERATIVE ADVERSARIAL NETWORKS;
D O I
10.1109/TMM.2022.3159115
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recently, there has been an increasing interest in image editing methods that employ pre-trained unconditional image generators (e.g., StyleGAN). However, applying these methods to translate images to multiple visual domains remains challenging. Existing works do not often preserve the domain-invariant part of the image (e.g., the identity in human face translations), or they do not usually handle multiple domains or allow for multi-modal translations. This work proposes an implicit style function (ISF) to straightforwardly achieve multi-modal and multi-domain image-to-image translation from pre-trained unconditional generators. The ISF manipulates the semantics of a latent code to ensure that the image generated from the manipulated code lies in the desired visual domain. Our human faces and animal image manipulations show significantly improved results over the baselines. Our model enables cost-effective multi-modal unsupervised image-to-image translations at high resolution using pre-trained unconditional GANs. The code and data are available at: https://github.com/yhlleo/stylegan-mmuit.
引用
收藏
页码:3343 / 3353
页数:11
相关论文
共 50 条
  • [31] ISF-GAN: Imagine, Select, and Fuse with GPT-based Text Enrichment for Text-to-image Synthesis
    Sheng, Yefei
    Tao, Ming
    Wang, Jie
    Bao, Bing-Kun
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (07) : 1 - 17
  • [32] GiGAN: Gate in GAN, could gate mechanism filter the features in image-to-image translation?
    Nie, Xuan
    Jia, Jianchao
    Ding, Haoxuan
    Wong, Edward K.
    NEUROCOMPUTING, 2021, 462 (462) : 376 - 388
  • [33] Swin-UNIT: Transformer-based GAN for High-resolution Unpaired Image Translation
    Li, Yifan
    Li, Yaochen
    Tang, Wenneng
    Zhu, Zhifeng
    Yang, Jinhuo
    Liu, Yuehu
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 4657 - 4665
  • [34] Multimodal image-to-image translation between domains with high internal variability
    Wang, Jian
    Lv, Jiancheng
    Yang, Xue
    Tang, Chenwei
    Peng, Xi
    SOFT COMPUTING, 2020, 24 (23) : 18173 - 18184
  • [35] Multimodal image-to-image translation between domains with high internal variability
    Jian Wang
    Jiancheng Lv
    Xue Yang
    Chenwei Tang
    Xi Peng
    Soft Computing, 2020, 24 : 18173 - 18184
  • [36] LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images
    Lin, Shan
    Qin, Fangbo
    Li, Yangming
    Bly, Randall A.
    Moe, Kris S.
    Hannaford, Blake
    2020 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2020, : 2914 - 2920
  • [37] UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation
    Torbunov, Dmitrii
    Huang, Yi
    Yu, Haiwang
    Huang, Jin
    Yoo, Shinjae
    Lin, Meifeng
    Viren, Brett
    Ren, Yihui
    2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, : 702 - 712
  • [38] High-Resolution SAR-to-Multispectral Image Translation Based on S2MS-GAN
    Liu, Yang
    Han, Qingcen
    Yang, Hong
    Hu, Huizhu
    REMOTE SENSING, 2024, 16 (21)
  • [39] Image Translation Between High-Resolution Remote Sensing Optical and SAR Data Using Conditional GAN
    Niu, Xin
    Yang, Di
    Yang, Ke
    Pan, Hengyue
    Dou, Yong
    ADVANCES IN MULTIMEDIA INFORMATION PROCESSING, PT III, 2018, 11166 : 245 - 255
  • [40] Image Disentanglement and Uncooperative Re-Entanglement for High-Fidelity Image-to-Image Translation
    Harley, Adam W.
    Wei, Shih-En
    Saragih, Jason
    Fragkiadaki, Katerina
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), 2019, : 3324 - 3332