HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

被引:0
|
作者
Wang, Wenqing [1 ,2 ]
Li, Lingzhou [1 ]
Yang, Yifei [1 ]
Liu, Han [1 ,2 ]
Guo, Runyuan [1 ,2 ]
机构
[1] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[2] Xian Univ Technol, Shaanxi Key Lab Complex Syst Control & Intelligen, Xian 710048, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; infrared image; visible image; Transformer; CNN; PERFORMANCE;
D O I
10.3390/s24237729
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
The purpose of infrared and visible image fusion is to combine the advantages of both and generate a fused image that contains target information and has rich details and contrast. However, existing fusion algorithms often overlook the importance of incorporating both local and global feature extraction, leading to missing key information in the fused image. To address these challenges, this paper proposes a dual-branch fusion network combining convolutional neural network (CNN) and Transformer, which enhances the feature extraction capability and motivates the fused image to contain more information. Firstly, a local feature extraction module with CNN as the core is constructed. Specifically, the residual gradient module is used to enhance the ability of the network to extract texture information. Also, jump links and coordinate attention are used in order to relate shallow features to deeper ones. In addition, a global feature extraction module based on Transformer is constructed. Through the powerful ability of Transformer, the global context information of the image can be captured and the global features are fully extracted. The effectiveness of the proposed method in this paper is verified on different experimental datasets, and it is better than most of the current advanced fusion algorithms.
引用
收藏
页数:20
相关论文
共 50 条
  • [31] TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion
    Lu, Siyu
    Ye, Xiangzhou
    Rao, Junmin
    Li, Fanming
    Liu, Shijian
    SENSORS, 2024, 24 (01)
  • [32] Cross-UNet: dual-branch infrared and visible image fusion framework based on cross-convolution and attention mechanism
    Wang, Xuejiao
    Hua, Zhen
    Li, Jinjiang
    VISUAL COMPUTER, 2023, 39 (10): : 4801 - 4818
  • [33] Cross-UNet: dual-branch infrared and visible image fusion framework based on cross-convolution and attention mechanism
    Xuejiao Wang
    Zhen Hua
    Jinjiang Li
    The Visual Computer, 2023, 39 : 4801 - 4818
  • [34] DPACFuse: Dual-Branch Progressive Learning for Infrared and Visible Image Fusion with Complementary Self-Attention and Convolution
    Zhu, Huayi
    Wu, Heshan
    Wang, Xiaolong
    He, Dongmei
    Liu, Zhenbing
    Pan, Xipeng
    SENSORS, 2023, 23 (16)
  • [35] Rain removal method for single image of dual-branch joint network based on sparse transformer
    Qin, Fangfang
    Jia, Zongpu
    Pang, Xiaoyan
    Zhao, Shan
    COMPLEX & INTELLIGENT SYSTEMS, 2025, 11 (01)
  • [36] THFuse: An infrared and visible image fusion network using transformer and hybrid feature extractor
    Chen, Jun
    Ding, Jianfeng
    Yu, Yang
    Gong, Wenping
    NEUROCOMPUTING, 2023, 527 : 71 - 82
  • [37] A Two-Branch Fusion Network for Infrared and Visible Image Fusion
    Zhang, Weihao
    Li, Zhilin
    Li, Bin
    Zhang, Mingliang
    PATTERN RECOGNITION AND COMPUTER VISION, PT IX, PRCV 2024, 2025, 15039 : 42 - 55
  • [38] DDT: Dual-branch Deformable Transformer for Image Denoising
    Liu, Kangliang
    Du, Xiangcheng
    Liu, Sijie
    Zheng, Yingbin
    Wu, Xingjiao
    Jin, Cheng
    2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 2765 - 2770
  • [39] Hyperspectral Image Classification Based on Hybrid Depth-Wise Separable Convolution and Dual-Branch Feature Fusion Network
    Dai, Hualin
    Yue, Yingli
    Liu, Qi
    APPLIED SCIENCES-BASEL, 2025, 15 (03):
  • [40] Dual-branch hybrid encoding embedded network for histopathology image classification
    Li, Mingshuai
    Hu, Zhiqiu
    Qiu, Song
    Zhou, Chenhao
    Weng, Jialei
    Dong, Qiongzhu
    Sheng, Xia
    Ren, Ning
    Zhou, Mei
    PHYSICS IN MEDICINE AND BIOLOGY, 2023, 68 (19):