LVAR-CZSL: Learning Visual Attributes Representation for Compositional Zero-Shot Learning

被引:0
|
作者
Ma, Xingjiang [1 ]
Yang, Jing [1 ,2 ]
Lin, Jiacheng [3 ]
Zheng, Zhenzhe [4 ]
Li, Shaobo [1 ]
Hu, Bingqi [1 ]
Tang, Xianghong [1 ]
机构
[1] Guizhou Univ, State Key Lab Publ Big Data, Guiyang 550025, Peoples R China
[2] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
[3] Hunan Univ, Coll Comp Sci & Elect Engn, Changsha 410082, Peoples R China
[4] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
基金
中国国家自然科学基金;
关键词
Visualization; Feature extraction; Dogs; Task analysis; Attention mechanisms; Zero-shot learning; Circuits and systems; Compositional zero-shot learning; visual attributes; objects and attributes; inter-class connectivity; OBJECTS;
D O I
10.1109/TCSVT.2024.3444782
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Compositional Zero-Shot Learning (CZSL) has been applied to various scenarios, including scene understanding, visual-language representation, and domain adaptation. Despite numerous endeavours and significant advancements, the crucial issues of fuzzy conceptualization of visual attributes and insufficient inter-class connectivity, have remained insufficiently addressed. To address these issues, we propose Learning Visual Attributes Representation for Compositional Zero-Shot Learning (LVAR-CZSL), which has the ability to learn visual attributes and inter-class dependencies. LVAR-CZSL is mainly composed of two key components: the Visual Attribute Representation Module (VARM) and the Connected Learning Module (CLM). Specifically, VARM extracts detailed attributes and object visual features from global visual features, resolving the issue of fuzzy visual attribute concepts. Moreover, CLM endows LVAR-CZSL with the capability to perceive connectivity between different attributes and objects, effectively enhancing inter-class connectivity. To establish a close connection between VARM and CLM and minimize the gap between image and text features, we introduce the composition-attribute-object Joint Scoring Function (JSF). Additionally, we propose Joint Loss Function (JLF) to optimize the learning process of VARM and CLM. The experiment results on four datasets show that LVAR-CZSL achieves state-of-the-art performance. The code is available at https://github.com/mxjmxj1/LVAR-CZSL.
引用
收藏
页码:13311 / 13323
页数:13
相关论文
共 50 条
  • [21] Fine-grained Primitive Representation Learning for Compositional Zero-shot Classification
    Jiang, Han
    Yang, Xiaoshan
    Chen, Chaofan
    Xu, Changsheng
    2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 456 - 461
  • [22] Hierarchical Prompt Learning for Compositional Zero-Shot Recognition
    Wang, Henan
    Yang, Muli
    Wei, Kun
    Deng, Cheng
    PROCEEDINGS OF THE THIRTY-SECOND INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2023, 2023, : 1470 - 1478
  • [23] Reference-Limited Compositional Zero-Shot Learning
    Huang, Siteng
    Wei, Qiyao
    Wang, Donglin
    PROCEEDINGS OF THE 2023 ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2023, 2023, : 443 - 451
  • [24] Environment Generation for Zero-Shot Compositional Reinforcement Learning
    Gur, Izzeddin
    Jaques, Natasha
    Miao, Yingjie
    Choi, Jongwook
    Tiwari, Manoj
    Lee, Honglak
    Faust, Aleksandra
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021, 34
  • [25] Adaptive Fusion Learning for Compositional Zero-Shot Recognition
    Min, Lingtong
    Fan, Ziman
    Wang, Shunzhou
    Dou, Feiyang
    Li, Xin
    Wang, Binglu
    IEEE TRANSACTIONS ON MULTIMEDIA, 2025, 27 : 1193 - 1204
  • [26] A Decomposable Causal View of Compositional Zero-Shot Learning
    Yang, Muli
    Xu, Chenghao
    Wu, Aming
    Deng, Cheng
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 5892 - 5902
  • [27] Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
    Mancini, Massimiliano
    Naeem, Muhammad Ferjad
    Xian, Yongqin
    Akata, Zeynep
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2024, 46 (03) : 1545 - 1560
  • [28] Robust Zero-Shot Learning with Source Attributes Noise
    Yu, Jun
    Wu, Songsong
    Wang, Lu
    Jing, Xiao-Yuan
    PROCEEDINGS OF THE 2016 INTERNATIONAL CONFERENCE ON PROGRESS IN INFORMATICS AND COMPUTING (PIC), VOL 1, 2016, : 205 - 209
  • [29] Discriminative deep attributes for generalized zero-shot learning
    Kim, Hoseong
    Lee, Jewook
    Byun, Hyeran
    PATTERN RECOGNITION, 2022, 124
  • [30] Zero-Shot Learning with a Partial Set of Observed Attributes
    Wang, Yaqing
    Kwok, James T.
    Yao, Quanming
    Ni, Lionel M.
    2017 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2017, : 3777 - 3784