DACFusion: Dual Asymmetric Cross-Attention guided feature fusion for multispectral object detection

Effective fusion of unique features from different spectra plays a crucial role in multispectral object detection. Recent research has focused on transplanting advanced methods from other multimodal fusion fields to multispectral object detection tasks. These fusion methods focus on the fusion of features and ignore the spatial correspondence between multispectral images. This lack of correspondence in turn limits the full utilization of the complementarities between the different modalities, which affects the accuracy of object detection. To address this problem, we creatively propose a dual asymmetric cross-attention multispectral fusion (DACFusion) method, which is able to process features interactively based on the positional correspondence between two spectra, and then asymmetrically fuses the multispectral data according to the characteristics of each spectrum to take advantage of their complementary strengths. Meanwhile, we introduce a large selective kernel network to expand the receptive field for object detection, which further improves the detection accuracy. Experimental results on the VEDAI and LLVIP datasets validate the significant performance advantages of our proposed method and show its applicability to a variety of practical application scenarios. Code will be available at https://github.com/wood-fish/DACFusion.