LVAR-CZSL: Learning Visual Attributes Representation for Compositional Zero-Shot Learning

被引:0
|
作者
Ma, Xingjiang [1 ]
Yang, Jing [1 ,2 ]
Lin, Jiacheng [3 ]
Zheng, Zhenzhe [4 ]
Li, Shaobo [1 ]
Hu, Bingqi [1 ]
Tang, Xianghong [1 ]
机构
[1] Guizhou Univ, State Key Lab Publ Big Data, Guiyang 550025, Peoples R China
[2] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
[3] Hunan Univ, Coll Comp Sci & Elect Engn, Changsha 410082, Peoples R China
[4] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
基金
中国国家自然科学基金;
关键词
Visualization; Feature extraction; Dogs; Task analysis; Attention mechanisms; Zero-shot learning; Circuits and systems; Compositional zero-shot learning; visual attributes; objects and attributes; inter-class connectivity; OBJECTS;
D O I
10.1109/TCSVT.2024.3444782
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Compositional Zero-Shot Learning (CZSL) has been applied to various scenarios, including scene understanding, visual-language representation, and domain adaptation. Despite numerous endeavours and significant advancements, the crucial issues of fuzzy conceptualization of visual attributes and insufficient inter-class connectivity, have remained insufficiently addressed. To address these issues, we propose Learning Visual Attributes Representation for Compositional Zero-Shot Learning (LVAR-CZSL), which has the ability to learn visual attributes and inter-class dependencies. LVAR-CZSL is mainly composed of two key components: the Visual Attribute Representation Module (VARM) and the Connected Learning Module (CLM). Specifically, VARM extracts detailed attributes and object visual features from global visual features, resolving the issue of fuzzy visual attribute concepts. Moreover, CLM endows LVAR-CZSL with the capability to perceive connectivity between different attributes and objects, effectively enhancing inter-class connectivity. To establish a close connection between VARM and CLM and minimize the gap between image and text features, we introduce the composition-attribute-object Joint Scoring Function (JSF). Additionally, we propose Joint Loss Function (JLF) to optimize the learning process of VARM and CLM. The experiment results on four datasets show that LVAR-CZSL achieves state-of-the-art performance. The code is available at https://github.com/mxjmxj1/LVAR-CZSL.
引用
收藏
页码:13311 / 13323
页数:13
相关论文
共 50 条
  • [1] Learning Conditional Attributes for Compositional Zero-Shot Learning
    Wang, Qingsheng
    Liu, Lingqiao
    Jing, Chenchen
    Chen, Hao
    Liang, Guoqiang
    Wang, Peng
    Shen, Chunhua
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, : 11197 - 11206
  • [2] Learning Invariant Visual Representations for Compositional Zero-Shot Learning
    Zhang, Tian
    Liang, Kongming
    Du, Ruoyi
    Sun, Xian
    Ma, Zhanyu
    Guo, Jun
    COMPUTER VISION, ECCV 2022, PT XXIV, 2022, 13684 : 339 - 355
  • [3] Zero-shot recognition with latent visual attributes learning
    Xie, Yurui
    He, Xiaohai
    Zhang, Jing
    Luo, Xiaodong
    MULTIMEDIA TOOLS AND APPLICATIONS, 2020, 79 (37-38) : 27321 - 27335
  • [4] Zero-shot recognition with latent visual attributes learning
    Yurui Xie
    Xiaohai He
    Jing Zhang
    Xiaodong Luo
    Multimedia Tools and Applications, 2020, 79 : 27321 - 27335
  • [5] Deep Representation of Hierarchical Semantic Attributes for Zero-shot Learning
    Zhang, Zhaocheng
    Yang, Gang
    2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
  • [6] Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning
    Kim, Hanjae
    Lee, Jiyoung
    Park, Seongheon
    Sohn, Kwanghoon
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 5652 - 5662
  • [7] Zero-Shot Compositional Concept Learning
    Xu, Guangyue
    Kordjamshidi, Parisa
    Chai, Joyce Y.
    1ST WORKSHOP ON META LEARNING AND ITS APPLICATIONS TO NATURAL LANGUAGE PROCESSING (METANLP 2021), 2021, : 19 - 27
  • [8] Learning the Compositional Domains for Generalized Zero-shot Learning
    Dong, Hanze
    Fu, Yanwei
    Hwang, Sung Ju
    Sigal, Leonid
    Xue, Xiangyang
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2022, 221
  • [9] Learning Attention Propagation for Compositional Zero-Shot Learning
    Khan, Muhammad Gul Zain Ali
    Naeem, Muhammad Ferjad
    Van Gool, Luc
    Pagani, A.
    Stricker, Didier
    Afzal, Muhammad Zeshan
    2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, : 3817 - 3826
  • [10] Learning Attention as Disentangler for Compositional Zero-shot Learning
    Hao, Shaozhe
    Han, Kai
    Wong, Kwan-Yee K.
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, : 15315 - 15324