Infrared and Visible Image Fusion Using Dual-stream Generative Adversarial Network with Multiple Discriminators

To address the problem of insufficiently retaining the source information during the process of fusing the infrared and visible images based on the existing methods, an algorithm for fusing the infrared and visible images using dual-stream generative adversarial networks (GAN) with multiple discriminators is improved. The improved GAN-based fusion framework consists of one generator and four discriminators, and utilizes the differential image as the auxiliary information to further improve the performance of the fusion network. The differential imagein the algorithm is not only used as the auxiliary information of source image to guide the generator to focus on the unique information of different modal images, but also used as the real data distribution to assist the differential discriminator in competitively training with the generator. In the improved network model, the generator adopts a dual encoder-single decoder structure, where the encoder aims to extract the features from different modalities mainly via a densely connected structure combined with an attention module, and the decoder is used to reconstruct the fused image based on the concatenated high-dimensional features. The discriminator evaluates whether the input image is the real image or the fusion image, and constrainedly optimizes the generator based on the evaluated results. Experimental results show that, compared with the other algorithms, the improved algorithm achieves better fusion results both in the subjective assessments and in the objective effects evaluated by the quantitative metrics. © 2024 China Ordnance Industry Corporation. All rights reserved.