Two-stage Unidirectional Fusion Network for RGBT tracking

RGB and Thermal (RGBT) tracking has recently attracted significant attention for its ability to accurately localize targets in complex scenarios. However, the creation of large-scale RGBT tracking datasets is both resource-intensive and laborious, motivating researchers to develop prompt tuning methods to adapt upstream RGB trackers to multimodal data with minimal additional parameters. Nevertheless, these methods do not fully exploit the supplementary modality information and tend to overlook the dynamic advantages between the two modalities in challenging scenarios. To address these issues, we propose a Two-stage Unidirectional Fusion (TUF) algorithm for RGBT tracking. This approach maximizes knowledge retention from upstream models while effectively leveraging the complementarity between the two modalities. It allows the powerful RGB feature extraction backbone from the upstream model to guide TIR image feature extraction through a two-stage unidirectional fusion strategy. Additionally, we have introduced an autoregressive decoder into RGBT tracking as a replacement for traditional bounding box prediction heads. This streamlines the framework of our RGBT tracker and improves tracking accuracy. Extensive experiments conducted on four widely used RGBT tracking benchmarks validate that our method surpasses existing state-of-the-art prompt tuning approaches, achieving a superior balance between performance and efficiency.