HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

被引:0
|
作者
Wang, Wenqing [1 ,2 ]
Li, Lingzhou [1 ]
Yang, Yifei [1 ]
Liu, Han [1 ,2 ]
Guo, Runyuan [1 ,2 ]
机构
[1] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[2] Xian Univ Technol, Shaanxi Key Lab Complex Syst Control & Intelligen, Xian 710048, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; infrared image; visible image; Transformer; CNN; PERFORMANCE;
D O I
10.3390/s24237729
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
The purpose of infrared and visible image fusion is to combine the advantages of both and generate a fused image that contains target information and has rich details and contrast. However, existing fusion algorithms often overlook the importance of incorporating both local and global feature extraction, leading to missing key information in the fused image. To address these challenges, this paper proposes a dual-branch fusion network combining convolutional neural network (CNN) and Transformer, which enhances the feature extraction capability and motivates the fused image to contain more information. Firstly, a local feature extraction module with CNN as the core is constructed. Specifically, the residual gradient module is used to enhance the ability of the network to extract texture information. Also, jump links and coordinate attention are used in order to relate shallow features to deeper ones. In addition, a global feature extraction module based on Transformer is constructed. Through the powerful ability of Transformer, the global context information of the image can be captured and the global features are fully extracted. The effectiveness of the proposed method in this paper is verified on different experimental datasets, and it is better than most of the current advanced fusion algorithms.
引用
收藏
页数:20
相关论文
共 50 条
  • [41] ENSO-Former: spatiotemporal fusion network based on multivariate and dual-branch transformer for ENSO prediction
    Fang, Wei
    Sha, Yu
    CLIMATE DYNAMICS, 2025, 63 (02)
  • [42] Attention based dual UNET network for infrared and visible image fusion
    Wang, Xuejiao
    Hua, Zhen
    Li, Jinjiang
    MULTIMEDIA TOOLS AND APPLICATIONS, 2024, 83 (25) : 66959 - 66980
  • [43] DHCT-DT: Dual-branch Hybrid CNN-Transformer Combined with Dual-Teacher Semi-Supervised Network for Kidney Ultrasound Image Segmentation
    Luo, Yang
    Liu, Jun
    Ding, Mengqian
    Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, 2024, : 5224 - 5229
  • [44] Ship Recognition for Complex SAR Images via Dual-Branch Transformer Fusion Network
    Sun, Zhongzhen
    Leng, Xiangguang
    Zhang, Xianghui
    Xiong, Boli
    Ji, Kefeng
    Kuang, Gangyao
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2024, 21
  • [45] Infrared and Visible Image Fusion with Convolutional Neural Network and Transformer
    Yang, Yang
    Ren, Zhennan
    Li, Beichen
    LASER & OPTOELECTRONICS PROGRESS, 2023, 60 (16)
  • [46] Cnn-trans model: A parallel dual-branch network for fundus image classification
    Liu, Shuxian
    Wang, Wei
    Deng, Le
    Xu, Huan
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2024, 96
  • [47] Infrared image super-resolution method based on dual-branch deep neural network
    Huang Zhijian
    Hui Bingwei
    Sun Shujin
    Ma Yanxin
    The Visual Computer, 2024, 40 : 1673 - 1684
  • [48] TBRAFusion: Infrared and visible image fusion based on two-branch residual attention Transformer
    Zhang, Wangwei
    Sun, Hao
    Zhou, Bin
    ELECTRONIC RESEARCH ARCHIVE, 2024, 33 (01): : 158 - 180
  • [49] A dual-branch network for ultrasound image segmentation
    Zhu, Zhiqin
    Zhang, Zimeng
    Qi, Guanqiu
    Li, Yuanyuan
    Li, Yuzhen
    Mu, Lan
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2025, 103
  • [50] Infrared image super-resolution method based on dual-branch deep neural network
    Huang, Zhijian
    Hui, Bingwei
    Sun, Shujin
    Ma, Yanxin
    VISUAL COMPUTER, 2024, 40 (03): : 1673 - 1684