LVAR-CZSL: Learning Visual Attributes Representation for Compositional Zero-Shot Learning

被引:0
|
作者
Ma, Xingjiang [1 ]
Yang, Jing [1 ,2 ]
Lin, Jiacheng [3 ]
Zheng, Zhenzhe [4 ]
Li, Shaobo [1 ]
Hu, Bingqi [1 ]
Tang, Xianghong [1 ]
机构
[1] Guizhou Univ, State Key Lab Publ Big Data, Guiyang 550025, Peoples R China
[2] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
[3] Hunan Univ, Coll Comp Sci & Elect Engn, Changsha 410082, Peoples R China
[4] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
基金
中国国家自然科学基金;
关键词
Visualization; Feature extraction; Dogs; Task analysis; Attention mechanisms; Zero-shot learning; Circuits and systems; Compositional zero-shot learning; visual attributes; objects and attributes; inter-class connectivity; OBJECTS;
D O I
10.1109/TCSVT.2024.3444782
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Compositional Zero-Shot Learning (CZSL) has been applied to various scenarios, including scene understanding, visual-language representation, and domain adaptation. Despite numerous endeavours and significant advancements, the crucial issues of fuzzy conceptualization of visual attributes and insufficient inter-class connectivity, have remained insufficiently addressed. To address these issues, we propose Learning Visual Attributes Representation for Compositional Zero-Shot Learning (LVAR-CZSL), which has the ability to learn visual attributes and inter-class dependencies. LVAR-CZSL is mainly composed of two key components: the Visual Attribute Representation Module (VARM) and the Connected Learning Module (CLM). Specifically, VARM extracts detailed attributes and object visual features from global visual features, resolving the issue of fuzzy visual attribute concepts. Moreover, CLM endows LVAR-CZSL with the capability to perceive connectivity between different attributes and objects, effectively enhancing inter-class connectivity. To establish a close connection between VARM and CLM and minimize the gap between image and text features, we introduce the composition-attribute-object Joint Scoring Function (JSF). Additionally, we propose Joint Loss Function (JLF) to optimize the learning process of VARM and CLM. The experiment results on four datasets show that LVAR-CZSL achieves state-of-the-art performance. The code is available at https://github.com/mxjmxj1/LVAR-CZSL.
引用
收藏
页码:13311 / 13323
页数:13
相关论文
共 50 条
  • [41] Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning
    Li, Xiangyu
    Yang, Xu
    Wei, Kun
    Deng, Cheng
    Yang, Muli
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, : 9316 - 9325
  • [42] Adversarial Training of Variational Auto-encoders for Continual Zero-shot Learning(A-CZSL)
    Ghosh, Subhankar
    2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [43] Swap-Reconstruction Autoencoder for Compositional Zero-Shot Learning
    Guo, Ting
    Liang, Jiye
    Xie, Guo-Sen
    2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 438 - 443
  • [44] Fine-Grained Attribute-Object Feature Representation in Compositional Zero-Shot Learning
    Shabbir, Nazir
    Rout, Ranjeet Kr.
    Umer, Saiyed
    Mohanta, Partha Pratim
    PATTERN RECOGNITION AND MACHINE INTELLIGENCE, PREMI 2023, 2023, 14301 : 157 - 165
  • [45] Learning Latent Semantic Attributes for Zero-Shot Object Detection
    Wang, Kang
    Zhang, Lu
    Tan, Yifan
    Zhao, Jiajia
    Zhou, Shuigeng
    2020 IEEE 32ND INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE (ICTAI), 2020, : 230 - 237
  • [46] Zero-Shot Learning with Missing Attributes using Semantic Correlations
    Braytee, Ali
    Naji, Mohamad
    Anaissi, Ali
    Chaturvedi, Kunal
    Prasad, Mukesh
    2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [47] INCREMENTAL ZERO-SHOT LEARNING BASED ON ATTRIBUTES FOR IMAGE CLASSIFICATION
    Xue, Nan
    Wang, Yi
    Fan, Xin
    Min, Maomao
    2017 24TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2017, : 850 - 854
  • [48] Grouping attributes zero-shot learning for tongue constitution recognition
    Wen, Guihua
    Ma, Jiajiong
    Hu, Yang
    Li, Huihui
    Jiang, Lijun
    ARTIFICIAL INTELLIGENCE IN MEDICINE, 2020, 109
  • [49] Joint Visual and Semantic Optimization for zero-shot learning
    Wu, Hanrui
    Yan, Yuguang
    Chen, Sentao
    Huang, Xiangkang
    Wu, Qingyao
    Ng, Michael K.
    KNOWLEDGE-BASED SYSTEMS, 2021, 215 (215)
  • [50] Hyperbolic Visual Embedding Learning for Zero-Shot Recognition
    Liu, Shaoteng
    Chen, Jingjing
    Pan, Liangming
    Ngo, Chong-Wah
    Chua, Tat-Seng
    Jiang, Yu-Gang
    2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, : 9270 - 9278