Learning Multifrequency Integration Network for RGBT Tracking

RGBT tracking is an attractive topic that benefits from the complementarity of visible and thermal sensors to better handle tracking tasks in atrocious scenarios. Existing RGBT trackers typically introduce self-attention (SA) to capture long-range dependencies. However, recent findings suggest that SA is a low-pass filter, meaning that high-frequency clues involving local edges and texture may be repressed. Aiming at the problem, this article comprehensively considers the multifrequency knowledge of heterogeneous modalities and proposes a learning multifrequency integration network (LMINet) for RGBT tracking to effectively implement adaptive extraction, enhancement, and integration of multifrequency cues. The proposed LMINet primarily benefits from the deployment of three crucial components: pattern-aware reinforcement (PR), multifrequency enhancement (ME), and MI. Specifically, the PR part consists of a carefully designed reinforcement unit (RU) and learnable weighting strategy 1 (LWS1). The former extracts information from the data flow to enhance the backbone, while the latter is a data-driven regulation mechanism that adaptively adjusts the enhancement intensity via learning the input. Then, the ME component separates high- and low-frequency knowledge via high-level branch (HB) and common unit (CU) and further adjusts the improvement intensity of multifrequency cues via the learning of LWS2 to achieve intramodal refinement. Moreover, the MI part first extracts high- and low-frequency signals via HB and low-level branch (LB) and implements cross-modal integration of high- and low-frequency cues through LWS3, respectively. Extensive experimental results on GTOT, RGBT234, and LasHeR demonstrate that the proposed LMINet is effective and competitive with state-of-the-art algorithms. The code will be open-sourced at https://github.com/mjt1312/Lminet.