HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

被引:0
|
作者
Wang, Wenqing [1 ,2 ]
Li, Lingzhou [1 ]
Yang, Yifei [1 ]
Liu, Han [1 ,2 ]
Guo, Runyuan [1 ,2 ]
机构
[1] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[2] Xian Univ Technol, Shaanxi Key Lab Complex Syst Control & Intelligen, Xian 710048, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; infrared image; visible image; Transformer; CNN; PERFORMANCE;
D O I
10.3390/s24237729
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
The purpose of infrared and visible image fusion is to combine the advantages of both and generate a fused image that contains target information and has rich details and contrast. However, existing fusion algorithms often overlook the importance of incorporating both local and global feature extraction, leading to missing key information in the fused image. To address these challenges, this paper proposes a dual-branch fusion network combining convolutional neural network (CNN) and Transformer, which enhances the feature extraction capability and motivates the fused image to contain more information. Firstly, a local feature extraction module with CNN as the core is constructed. Specifically, the residual gradient module is used to enhance the ability of the network to extract texture information. Also, jump links and coordinate attention are used in order to relate shallow features to deeper ones. In addition, a global feature extraction module based on Transformer is constructed. Through the powerful ability of Transformer, the global context information of the image can be captured and the global features are fully extracted. The effectiveness of the proposed method in this paper is verified on different experimental datasets, and it is better than most of the current advanced fusion algorithms.
引用
收藏
页数:20
相关论文
共 50 条
  • [1] A Dual-branch Network for Infrared and Visible Image Fusion
    Fu, Yu
    Wu, Xiao-Jun
    2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2021, : 10675 - 10680
  • [2] A dual-branch and dual attention transformer and CNN hybrid network for ultrasound image segmentation
    Zhang, Chong
    Wang, Lingtong
    Wei, Guohui
    Kong, Zhiyong
    Qiu, Min
    FRONTIERS IN PHYSIOLOGY, 2024, 15
  • [3] DBIF: Dual-Branch Feature Extraction Network for Infrared and Visible Image Fusion
    Zhang, Haozhe
    Cui, Rongpu
    Zheng, Zhuohang
    Gao, Shaobing
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2024, PT VIII, 2025, 15038 : 309 - 323
  • [4] MDAN: Multilevel dual-branch attention network for infrared and visible image fusion
    Wang, Jiawei
    Jiang, Min
    Kong, Jun
    OPTICS AND LASERS IN ENGINEERING, 2024, 176
  • [5] HaarFuse: A dual-branch infrared and visible light image fusion network based on Haar wavelet transform
    Wang, Yuequn
    Liu, Jie
    Wang, Jianli
    Yang, Leqiang
    Dong, Bo
    Li, Zhengwei
    PATTERN RECOGNITION, 2025, 164
  • [6] DDFNet-A: Attention-Based Dual-Branch Feature Decomposition Fusion Network for Infrared and Visible Image Fusion
    Wei, Qiancheng
    Liu, Ying
    Jiang, Xiaoping
    Zhang, Ben
    Su, Qiya
    Yu, Muyao
    REMOTE SENSING, 2024, 16 (10)
  • [7] DCTNet: A Heterogeneous Dual-Branch Multi-Cascade Network for Infrared and Visible Image Fusion
    Li, Jinfu
    Liu, Lei
    Song, Hong
    Huang, Yuqi
    Jiang, Junjun
    Yang, Jian
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2023, 72
  • [8] DDRF: Dual-branch decomposition and reconstruction architecture for infrared and visible image fusion
    Zhang, Lei
    Zhou, Qiming
    Tang, Mingliang
    Ding, Xin
    Yang, Chengwei
    Wei, Chuyuan
    Zhou, Zhimiao
    OPTICS AND LASER TECHNOLOGY, 2025, 181
  • [9] A dual-branch infrared and visible image fusion network using progressive image-wise feature transfer
    Xu, Shaoping
    Zhou, Changfei
    Xiao, Jian
    Tao, Wuyong
    Dai, Tianyu
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2024, 102
  • [10] Dual-branch network object detection algorithm based on dual-modality fusion of visible and infrared images
    Hou, Zhiqiang
    Li, Xinyue
    Yang, Chen
    Ma, Sugang
    Yu, Wangsheng
    Wang, Yunchen
    MULTIMEDIA SYSTEMS, 2024, 30 (06)