External-attention dual-modality fusion network for RGBT tracking

Due to the unique complementarity of RGB and thermal (RGBT) images, RGBT tracking has gradually become a crucial area of research. To achieve robust tracking performance, how to leverage both local and global information becomes a crucial issue for the RGBT tracking. Inspired by external-attention mechanism, we designed an external-attention dual-modality fusion network (EDFNet) equipped with external-attention guided module (EGM). The EGM based on two external memorized units generates the external attention maps that help reallocate the weights according to the correlations. To avoid feature deterioration, EDFNet introduces shortcuts to make detours and adaptively fuses the features from detours and external attention with adaptive weights. Furthermore, considering the difference of RGBT image, we design an asymmetric feature enhancement approach consisting of detailed information guidance (DiG) and structural information enhancement. DiG aims to optimize the detailed and textural features of RGB feature by axial detail optimization. SiE leverages the accumulated-addtion feature to enhance the structural features. Simultaneously, we deploy a loss function named partial weight enhanced loss in EDFNet to accommodate this new architecture. The evaluation results based on RGBT234 and GTOT, respectively, validate that EDFNet achieves a better tracking performance compared with the other trackers.