LVAR-CZSL: Learning Visual Attributes Representation for Compositional Zero-Shot Learning

被引:0
|
作者
Ma, Xingjiang [1 ]
Yang, Jing [1 ,2 ]
Lin, Jiacheng [3 ]
Zheng, Zhenzhe [4 ]
Li, Shaobo [1 ]
Hu, Bingqi [1 ]
Tang, Xianghong [1 ]
机构
[1] Guizhou Univ, State Key Lab Publ Big Data, Guiyang 550025, Peoples R China
[2] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
[3] Hunan Univ, Coll Comp Sci & Elect Engn, Changsha 410082, Peoples R China
[4] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
基金
中国国家自然科学基金;
关键词
Visualization; Feature extraction; Dogs; Task analysis; Attention mechanisms; Zero-shot learning; Circuits and systems; Compositional zero-shot learning; visual attributes; objects and attributes; inter-class connectivity; OBJECTS;
D O I
10.1109/TCSVT.2024.3444782
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Compositional Zero-Shot Learning (CZSL) has been applied to various scenarios, including scene understanding, visual-language representation, and domain adaptation. Despite numerous endeavours and significant advancements, the crucial issues of fuzzy conceptualization of visual attributes and insufficient inter-class connectivity, have remained insufficiently addressed. To address these issues, we propose Learning Visual Attributes Representation for Compositional Zero-Shot Learning (LVAR-CZSL), which has the ability to learn visual attributes and inter-class dependencies. LVAR-CZSL is mainly composed of two key components: the Visual Attribute Representation Module (VARM) and the Connected Learning Module (CLM). Specifically, VARM extracts detailed attributes and object visual features from global visual features, resolving the issue of fuzzy visual attribute concepts. Moreover, CLM endows LVAR-CZSL with the capability to perceive connectivity between different attributes and objects, effectively enhancing inter-class connectivity. To establish a close connection between VARM and CLM and minimize the gap between image and text features, we introduce the composition-attribute-object Joint Scoring Function (JSF). Additionally, we propose Joint Loss Function (JLF) to optimize the learning process of VARM and CLM. The experiment results on four datasets show that LVAR-CZSL achieves state-of-the-art performance. The code is available at https://github.com/mxjmxj1/LVAR-CZSL.
引用
收藏
页码:13311 / 13323
页数:13
相关论文
共 50 条
  • [31] Complementary Attributes: A New Clue to Zero-Shot Learning
    Xu, Xiaofeng
    Tsang, Ivor W.
    Liu, Chuancai
    IEEE TRANSACTIONS ON CYBERNETICS, 2021, 51 (03) : 1519 - 1530
  • [32] Learning Discriminative Latent Attributes for Zero-Shot Classification
    Jiang, Huajie
    Wang, Ruiping
    Shan, Shiguang
    Yang, Yi
    Chen, Xilin
    2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, : 4233 - 4242
  • [33] Task guided representation learning using compositional models for zero-shot domain adaptation
    Liu, Shuang
    Ozay, Mete
    NEURAL NETWORKS, 2023, 165 : 370 - 380
  • [34] Zero-Shot Learning via Visual Abstraction
    Antol, Stanislaw
    Zitnick, C. Lawrence
    Parikh, Devi
    COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 : 401 - 416
  • [35] Zero-shot visual grounding via coarse-to-fine representation learning
    Mi, Jinpeng
    Jin, Shaofei
    Chen, Zhiqian
    Liu, Dan
    Wei, Xian
    Zhang, Jianwei
    NEUROCOMPUTING, 2024, 610
  • [36] Zero-shot learning via discriminative representation extraction
    Long, Teng
    Xu, Xing
    Shen, Fumin
    Liu, Li
    Xie, Ning
    Yang, Yang
    PATTERN RECOGNITION LETTERS, 2018, 109 : 27 - 34
  • [37] Zero-Shot Classification with Discriminative Semantic Representation Learning
    Ye, Meng
    Guo, Yuhong
    30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, : 5103 - 5111
  • [38] Generalized Zero-Shot Learning via Disentangled Representation
    Li, Xiangyu
    Xu, Zhe
    Wei, Kun
    Deng, Cheng
    THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 1966 - 1974
  • [39] Dissimilarity Representation Learning for Generalized Zero-Shot Recognition
    Yang, Gang
    Liu, Jinlu
    Xu, Jieping
    Li, Xirong
    PROCEEDINGS OF THE 2018 ACM MULTIMEDIA CONFERENCE (MM'18), 2018, : 2032 - 2039
  • [40] Knowledge Guided Transformer Network for Compositional Zero-Shot Learning
    Panda, Aditya
    Prasad, Dipti
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (11)