ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation

被引:4
|
作者
Liu, Yahui [1 ]
Chen, Yajing [2 ]
Bao, Linchao [2 ]
Sebe, Nicu [1 ]
Lepri, Bruno [3 ]
De Nadai, Marco [3 ]
机构
[1] Univ Trento, Dept Informat Engn & Comp Sci, I-38123 Trento, Italy
[2] Tencent AI Lab, Shenzhen 518063, Peoples R China
[3] Fdn Bruno Kessler, I-38123 Povo, Italy
基金
欧盟地平线“2020”;
关键词
Face editing; generative adversarial networks (GANs); unsupervised image-to-image translation; GENERATIVE ADVERSARIAL NETWORKS;
D O I
10.1109/TMM.2022.3159115
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recently, there has been an increasing interest in image editing methods that employ pre-trained unconditional image generators (e.g., StyleGAN). However, applying these methods to translate images to multiple visual domains remains challenging. Existing works do not often preserve the domain-invariant part of the image (e.g., the identity in human face translations), or they do not usually handle multiple domains or allow for multi-modal translations. This work proposes an implicit style function (ISF) to straightforwardly achieve multi-modal and multi-domain image-to-image translation from pre-trained unconditional generators. The ISF manipulates the semantics of a latent code to ensure that the image generated from the manipulated code lies in the desired visual domain. Our human faces and animal image manipulations show significantly improved results over the baselines. Our model enables cost-effective multi-modal unsupervised image-to-image translations at high resolution using pre-trained unconditional GANs. The code and data are available at: https://github.com/yhlleo/stylegan-mmuit.
引用
收藏
页码:3343 / 3353
页数:11
相关论文
共 50 条
  • [41] Multiple-ResNet GAN: An enhanced high-resolution image generation method for translation from fundus structure image to fluorescein angiography
    Yuan, Jiahui
    Gao, Weiwei
    Fang, Yu
    Zhang, Haifeng
    Song, Nan
    MEDICAL & BIOLOGICAL ENGINEERING & COMPUTING, 2025, 63 (01) : 181 - 194
  • [42] Enhancing Style-Guided Image-to-Image Translation via Self-Supervised Metric Learning
    Mao, Qi
    Ma, Siwei
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 8511 - 8526
  • [43] Segmentation mask and feature similarity loss guided GAN for object-oriented image-to-image translation
    Qin, Zhen
    Chen, Qingya
    Ding, Yi
    Zhuang, Tianming
    Qin, Zhiguang
    Choo, Kim-Kwang Raymond
    INFORMATION PROCESSING & MANAGEMENT, 2022, 59 (03)
  • [44] Image-to-image translation using an offset-based multi-scale codes GAN encoder
    Zihao Guo
    Mingwen Shao
    Shunhang Li
    The Visual Computer, 2024, 40 (2) : 699 - 715
  • [45] Style-Guided Inference of Transformer for High-resolution Image Synthesis
    Yim, Jonghwa
    Kim, Minjae
    2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, : 1745 - 1755
  • [46] GP-GAN: Towards Realistic High-Resolution Image Blending
    Wu, Huikai
    Zheng, Shuai
    Zhang, Junge
    Huang, Kaiqi
    PROCEEDINGS OF THE 27TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA (MM'19), 2019, : 2487 - 2495
  • [47] StyleSwin: Transformer-based GAN for High-resolution Image Generation
    Zhang, Bowen
    Gu, Shuyang
    Zhang, Bo
    Bao, Jianmin
    Chen, Dong
    Wen, Fang
    Wang, Yong
    Guo, Baining
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, : 11294 - 11304
  • [48] Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation
    Wang, Chao
    Zheng, Haiyong
    Yu, Zhibin
    Zheng, Ziqiang
    Gu, Zhaorui
    Zheng, Bing
    COMPUTER VISION - ECCV 2018, PT I, 2018, 11205 : 796 - 812
  • [49] Discriminative Region Proposal Adversarial Network for High-Quality Image-to-Image Translation
    Chao Wang
    Wenjie Niu
    Yufeng Jiang
    Haiyong Zheng
    Zhibin Yu
    Zhaorui Gu
    Bing Zheng
    International Journal of Computer Vision, 2020, 128 : 2366 - 2385
  • [50] Discriminative Region Proposal Adversarial Network for High-Quality Image-to-Image Translation
    Wang, Chao
    Niu, Wenjie
    Jiang, Yufeng
    Zheng, Haiyong
    Yu, Zhibin
    Gu, Zhaorui
    Zheng, Bing
    INTERNATIONAL JOURNAL OF COMPUTER VISION, 2020, 128 (10-11) : 2366 - 2385