Cross-modal pedestrian re-recognition based on attention mechanism

被引:3
|
作者
Zhao, Yuyao [1 ]
Zhou, Hang [1 ]
Cheng, Hai [1 ]
Huang, Chunguang [1 ]
机构
[1] Heilongjiang Univ, Coll Elect & Engn, Harbin, Heilongjiang, Peoples R China
来源
VISUAL COMPUTER | 2024年 / 40卷 / 04期
基金
中国国家自然科学基金;
关键词
Person re-identification; Cross-modality; Attention mechanisms; Data integration;
D O I
10.1007/s00371-023-02926-7
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Person re-identification, as an essential research direction in intelligent security, has gained the focus of researchers and scholars. In practical scenarios, visible light cameras depend highly on lighting conditions and have limited detection capability in poor light. Therefore, many scholars have gradually shifted their research goals to cross-modality person re-identification. However, there are few relevant studies, and challenges remain in resolving the differences in the images of different modalities. In order to solve these problems, this paper will use the research method based on the attention mechanism to narrow the difference between the two modes and guide the network in a more appropriate direction to improve the recognition performance of the network. Aiming at the problem of using the attention mechanism method can improve training efficiency. However, it is easy to cause the model training instability. This paper proposes a cross-modal pedestrian re-recognition method based on the attention mechanism. A new attention mechanism module is designed to allow the network to use less time to focus on more critical features of a person. In addition, a cross-modality hard center triplet loss is designed to supervise the model training better. The paper has conducted extensive experiments on the above two methods on two publicly available datasets, which obtained better performance than similar current methods and verified the effectiveness and feasibility of the proposed methods in this paper.
引用
收藏
页码:2405 / 2418
页数:14
相关论文
共 50 条
  • [21] Multimodal Fusion with Cross-Modal Attention for Action Recognition in Still Images
    Tsai, Jia-Hua
    Chu, Wei-Ta
    PROCEEDINGS OF THE 4TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA IN ASIA, MMASIA 2022, 2022,
  • [22] Cascaded information enhancement and cross-modal attention feature fusion for multispectral pedestrian detection
    Yang, Yang
    Xu, Kaixiong
    Wang, Kaizheng
    FRONTIERS IN PHYSICS, 2023, 11
  • [23] Cross-modal links in spatial attention
    Driver, J
    Spence, C
    PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES, 1998, 353 (1373) : 1319 - 1331
  • [24] Cross-modal decoupling in temporal attention
    Muehlberg, Stefanie
    Oriolo, Giovanni
    Soto-Faraco, Salvador
    EUROPEAN JOURNAL OF NEUROSCIENCE, 2014, 39 (12) : 2089 - 2097
  • [25] Cross-modal orienting of visual attention
    Hillyard, Steven A.
    Stoermer, Viola S.
    Feng, Wenfeng
    Martinez, Antigona
    McDonald, John J.
    NEUROPSYCHOLOGIA, 2016, 83 : 170 - 178
  • [26] Cross-modal synergies in spatial attention
    Driver, J
    Eimer, M
    Macaluso, E
    Van Velzen, J
    PERCEPTION, 2003, 32 : 15 - 15
  • [27] Multi-corpus emotion recognition method based on cross-modal gated attention fusion
    Ryumina, Elena
    Ryumin, Dmitry
    Axyonov, Alexandr
    Ivanko, Denis
    Karpov, Alexey
    PATTERN RECOGNITION LETTERS, 2025, 190 : 192 - 200
  • [28] Visual question answering with attention transfer and a cross-modal gating mechanism
    Li, Wei
    Sun, Jianhui
    Liu, Ge
    Zhao, Linglan
    Fang, Xiangzhong
    PATTERN RECOGNITION LETTERS, 2020, 133 (133) : 334 - 340
  • [29] Cross-Modal Attention Mechanism for Weakly Supervised Video Anomaly Detection
    Sun, Wenwen
    Cao, Lin
    Guo, Yanan
    Du, Kangning
    BIOMETRIC RECOGNITION, CCBR 2023, 2023, 14463 : 437 - 446
  • [30] Multimodal Humor Detection Based on Cross-Modal Attention and Modal Maximum Correlation
    Quan, Zhibang
    Sun, Tao
    Su, Mengli
    Wei, Jishu
    2022 IEEE 9TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA), 2022, : 1064 - 1065