Text-Guided Knowledge Transfer for Remote Sensing Image-Text Retrieval

被引:4
|
作者
Liu, An-An [1 ,2 ,3 ]
Yang, Bo [1 ]
Li, Wenhui [1 ]
Song, Dan [1 ]
Sun, Zhengya [4 ]
Ren, Tongwei [5 ]
Wei, Zhiqiang [6 ]
机构
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] Chinese Acad Sci, Inst Artificial Intelligence, Hefei Comprehens Natl Sci Ctr, Beijing 100045, Peoples R China
[3] Chinese Acad Sci, Key Lab Electromagnet Space Informat, Beijing 100045, Peoples R China
[4] Chinese Acad Sci, Inst Automat, Beijing 100045, Peoples R China
[5] Nanjing Univ, State Key Lab Novel Software Technol, Nanjing 210093, Jiangsu, Peoples R China
[6] Ocean Univ China, Fac Informat Sci & Engn, Qingdao 266005, Shandong, Peoples R China
关键词
CLIP; knowledge transfer; remote sensing image-text retrieval;
D O I
10.1109/LGRS.2024.3374381
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Remote sensing text-image retrieval aims to retrieve valuable information from diverse and complex remote sensing data, attracting significant attention. However, the performance is limited due to the complexity of scenes and their substantial content differences from natural domain images. To address these issues, we propose a simple but effective text-guided knowledge transfer (TGKT) method for remote sensing image-text retrieval. TGKT utilizes CLIP to encode remote sensing data and transfer its rich semantic knowledge from natural to remote sensing domain. The textual information without significant domain differences is employed to bridge the semantic gap between these two domains, thereby enhancing image features. The extensive experimental results on both RSICD and RSITMD datasets demonstrate the effectiveness of our method.
引用
收藏
页码:1 / 5
页数:5
相关论文
共 50 条
  • [21] The Style Transformer with Common Knowledge Optimization for Image-Text Retrieval
    Li W.
    Ma Z.
    Shi J.
    Fan X.
    IEEE Signal Processing Letters, 2023, 30 : 1197 - 1201
  • [22] SIRS: Multitask Joint Learning for Remote Sensing Foreground-Entity Image-Text Retrieval
    Zhu, Zicong
    Kang, Jian
    Diao, Wenhui
    Feng, Yingchao
    Li, Junxi
    Ni, Jingen
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 15
  • [23] Spatial-Channel Attention Transformer With Pseudo Regions for Remote Sensing Image-Text Retrieval
    Wu, Dongqing
    Li, Huihui
    Hou, Yinxuan
    Xu, Cuili
    Cheng, Gong
    Guo, Lei
    Liu, Hang
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 15
  • [24] Text-guided visual representation learning for medical image retrieval systems
    Serieys, Guillaume
    Kurtz, Camille
    Fournier, Laure
    Cloppet, Florence
    2022 26TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2022, : 593 - 598
  • [25] Text-Guided Neural Image Inpainting
    Zhang, Lisai
    Chen, Qingcai
    Hu, Baotian
    Jiang, Shuoran
    MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, 2020, : 1302 - 1310
  • [26] A Deep Semantic Alignment Network for the Cross-Modal Image-Text Retrieval in Remote Sensing
    Cheng, Qimin
    Zhou, Yuzhuo
    Fu, Peng
    Xu, Yuan
    Zhang, Liang
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2021, 14 : 4284 - 4297
  • [27] Toward Efficient and Accurate Remote Sensing Image-Text Retrieval With a Coarse-to-Fine Approach
    Zhou, Wenqian
    Wu, Hanlin
    Deng, Pei
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2025, 22
  • [28] Strong and Weak Prompt Engineering for Remote Sensing Image-Text Cross-Modal Retrieval
    Sun, Tianci
    Zheng, Chengyu
    Li, Xiu
    Nie, Jie
    Gao, Yanli
    Huang, Lei
    Wei, Zhiqiang
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2025, 18 : 6968 - 6980
  • [29] Text-Guided Portrait Image Matting
    Xu Y.
    Yao X.
    Liu B.
    Quan Y.
    Ji H.
    IEEE Transactions on Artificial Intelligence, 2024, 5 (08): : 1 - 13
  • [30] Text-Guided Generative Adversarial Network for Image Emotion Transfer
    Zhu, Siqi
    Qing, Chunmei
    Xu, Xiangmin
    ADVANCED INTELLIGENT COMPUTING TECHNOLOGY AND APPLICATIONS, ICIC 2023, PT II, 2023, 14087 : 506 - 522