Prompt-supervised dynamic attention graph convolutional network for skeleton-based action recognition

被引:0
|
作者
Zhu, Shasha [1 ]
Sun, Lu [1 ]
Ma, Zeyuan [1 ]
Li, Chenxi [1 ]
He, Dongzhi [1 ]
机构
[1] Beijing Univ Technol, Coll Comp Sci, Beijing, Peoples R China
关键词
Skeleton-based action recognition; Graph convolutional network; Attention mechanism; Dynamic convolution; Prompt learning;
D O I
10.1016/j.neucom.2024.128623
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Skeleton-based action recognition is a core task in the field of video understanding. Skeleton sequences are characterized by high information density, low redundancy, and clear structural information, thereby facilitating the analysis of complex relationships among human behaviors more readily than other modalities. Although existing studies have encoded skeleton data and achieved positive outcomes, they have often overlooked the precise high-level semantic information inherent in the action descriptions. To address this issue, this paper proposes a prompt-supervised dynamic attention graph convolutional network (PDA-GCN). Specifically, the PDA-GCN incorporates a prompt supervision (PS) module that leverages a pre-trained large-scale language model (LLM) as a knowledge engine and retains the generated text features as prompts to provide additional supervision during model training, enhancing the model's ability to discern analogous actions with negligible computational cost. In addition, for the purpose of bolstering the learning of discriminative features, a dynamic attention graph convolution (DA-GC) module is presented. This module utilizes self-attention mechanism to adaptively infer intrinsic relationships between joints and integrates dynamic convolution to strengthen the emphasis on local information. This dual focus on both global context and local details further amplifies the efficiency and effectiveness of the model. Extensive experiments, conducted on the widely-used skeleton-based action recognition datasets NTU RGB+D 60 and NTU RGB+D 120, demonstrate that the PDA-GCN surpasses known state-of-the-art methods, achieving accuracies of 93.4% on the NTU RGB+D 60 cross-subject split and 90.7% on the NTU RGB+D 120 cross-subject split.
引用
收藏
页数:11
相关论文
共 50 条
  • [41] Richly Activated Graph Convolutional Network for Robust Skeleton-Based Action Recognition
    Song, Yi-Fan
    Zhang, Zhang
    Shan, Caifeng
    Wang, Liang
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2021, 31 (05) : 1915 - 1925
  • [42] Temporal Receptive Field Graph Convolutional Network for Skeleton-based Action Recognition
    Zhang, Qingqi
    Wu, Ren
    Nakata, Mitsuru
    Ge, Qi-Wei
    2024 INTERNATIONAL TECHNICAL CONFERENCE ON CIRCUITS/SYSTEMS, COMPUTERS, AND COMMUNICATIONS, ITC-CSCC 2024, 2024,
  • [43] Focus on temporal graph convolutional networks with unified attention for skeleton-based action recognition
    Gao, Bing-Kun
    Dong, Le
    Bi, Hong-Bo
    Bi, Yun-Ze
    APPLIED INTELLIGENCE, 2022, 52 (05) : 5608 - 5616
  • [44] Focus on temporal graph convolutional networks with unified attention for skeleton-based action recognition
    Bing-Kun Gao
    Le Dong
    Hong-Bo Bi
    Yun-Ze Bi
    Applied Intelligence, 2022, 52 : 5608 - 5616
  • [45] Multi-scale skeleton simplification graph convolutional network for skeleton-based action recognition
    Fan, Zhang
    Ding, Chongyang
    Kai, Liu
    Liu, Hongjin
    IET COMPUTER VISION, 2024, 18 (07) : 992 - 1003
  • [46] Spatial–Temporal gated graph attention network for skeleton-based action recognition
    Mrugendrasinh Rahevar
    Amit Ganatra
    Pattern Analysis and Applications, 2023, 26 (3) : 929 - 939
  • [47] Graph transformer network with temporal kernel attention for skeleton-based action recognition
    Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming
    650504, China
    Knowl Based Syst,
  • [48] Graph transformer network with temporal kernel attention for skeleton-based action recognition
    Liu, Yanan
    Zhang, Hao
    Xu, Dan
    He, Kangjian
    KNOWLEDGE-BASED SYSTEMS, 2022, 240
  • [49] Convolutional relation network for skeleton-based action recognition
    Zhu, Jiagang
    Zou, Wei
    Zhu, Zheng
    Hu, Yiming
    NEUROCOMPUTING, 2019, 370 : 109 - 117
  • [50] Attention-Guided and Topology-Enhanced Shift Graph Convolutional Network for Skeleton-Based Action Recognition
    Lu, Chenghong
    Chen, Hongbo
    Li, Menglei
    Jing, Lei
    ELECTRONICS, 2024, 13 (18)