MMYFnet: Multi-Modality YOLO Fusion Network for Object Detection in Remote Sensing Images

被引:0
|
作者
Guo, Huinan [1 ]
Sun, Congying [1 ,2 ]
Zhang, Jing [2 ]
Zhang, Wuxia [3 ]
Zhang, Nengshuang [1 ,2 ]
机构
[1] Chinese Acad Sci, Xian Inst Opt & Fine Mech, Xian 710119, Peoples R China
[2] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[3] Xian Univ Posts & Telecommun, Sch Comp Sci & Technol, Xian 710121, Peoples R China
基金
中国国家自然科学基金;
关键词
cross-modality; cosine similarity; feature fusion; multi-spectral remote sensing imagery; dual-branch; object detection; SIMILARITY;
D O I
10.3390/rs16234451
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Object detection in remote sensing images is crucial for airport management, hazard prevention, traffic monitoring, and more. The precise ability for object localization and identification enables remote sensing imagery to provide early warnings, mitigate risks, and offer strong support for decision-making processes. While traditional deep learning-based object detection techniques have achieved significant results in single-modal environments, their detection capabilities still encounter challenges when confronted with complex environments, such as adverse weather conditions or situations where objects are obscured. To overcome the limitations of existing fusion methods in terms of complexity and insufficient information utilization, we innovatively propose a Cosine Similarity-based Image Feature Fusion (CSIFF) module and integrate it into a dual-branch YOLOv8 network, constructing a lightweight and efficient target detection network called Multi-Modality YOLO Fusion Network (MMYFNet). This network utilizes cosine similarity to divide the original features into common features and specific features, which are then refined and fused through specific modules. Experimental and analytical results show that MMYFNet performs excellently on both the VEDAI and FLIR datasets, achieving mAP values of 80% and 76.8%, respectively. Further validation through parameter sensitivity experiments, ablation studies, and visual analyses confirms the effectiveness of the CSIFF module. MMYFNet achieves high detection accuracy with fewer parameters, and the CSIFF module, as a plug-and-play module, can be integrated into other CNN-based cross-modality network models, providing a new approach for object detection in remote sensing image fusion.
引用
收藏
页数:19
相关论文
共 50 条
  • [1] MA-YOLO: a multi-attention object detection network for remote sensing images
    Song, Qingzeng
    Hou, Maorui
    Xue, Yongjiang
    Yu, Jing
    JOURNAL OF ELECTRONIC IMAGING, 2024, 33 (01)
  • [2] Multi-Component Fusion Network for Small Object Detection in Remote Sensing Images
    Liu, Jing
    Yang, Shuojin
    Tian, Liang
    Guo, Wei
    Zhou, Bingyin
    Jia, Jianqing
    Ling, Haibin
    IEEE ACCESS, 2019, 7 : 128339 - 128352
  • [3] An Interpretable Fusion Siamese Network for Multi-Modality Remote Sensing Ship Image Retrieval
    Xiong, Wei
    Xiong, Zhenyu
    Cui, Yaqi
    Huang, Linzhou
    Yang, Ruining
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2023, 33 (06) : 2696 - 2712
  • [4] Multi-Modality and Multi-Scale Attention Fusion Network for Land Cover Classification from VHR Remote Sensing Images
    Lei, Tao
    Li, Linze
    Lv, Zhiyong
    Zhu, Mingzhe
    Du, Xiaogang
    Nandi, Asoke K.
    REMOTE SENSING, 2021, 13 (18)
  • [5] Transfer Learning for Object Detection in Remote Sensing Images with YOLO
    Devi, A.
    Reddy, K. Venkateswara
    Bangare, Sunil L.
    Pande, Deepti S.
    Balaji, S. R.
    Badhoutiya, Arti
    Shrivastava, Anurag
    JOURNAL OF ELECTRICAL SYSTEMS, 2024, 20 (03) : 980 - 989
  • [6] SeMo-YOLO: A Multiscale Object Detection Network in Satellite Remote Sensing Images
    Li, Peng
    Che, Cheng
    2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [7] A Novel Multi-Model Decision Fusion Network for Object Detection in Remote Sensing Images
    Ma, Wenping
    Guo, Qiongqiong
    Wu, Yue
    Zhao, Wei
    Zhang, Xiangrong
    Jiao, Licheng
    REMOTE SENSING, 2019, 11 (07)
  • [8] Multi-Modality Sensing and Data Fusion for Multi-Vehicle Detection
    Roy, Debashri
    Li, Yuanyuan
    Jian, Tong
    Tian, Peng
    Chowdhury, Kaushik
    Ioannidis, Stratis
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 2280 - 2295
  • [9] A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images
    Cheng, Yong
    Wang, Wei
    Zhang, Wenjie
    Yang, Ling
    Wang, Jun
    Ni, Huan
    Guan, Tingzhao
    He, Jiaxin
    Gu, Yakang
    Tran, Ngoc Nguyen
    REMOTE SENSING, 2023, 15 (08)
  • [10] MULTI-SCALE FEATURE FUSION NETWORK FOR OBJECT DETECTION IN VHR OPTICAL REMOTE SENSING IMAGES
    Zhang, Wenhua
    Jiao, Licheng
    Liu, Xu
    Liu, Jia
    2019 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS 2019), 2019, : 330 - 333