CaRA-MOT: 3D Multi-Object Tracking with Memory Mechanism using Camera and Radar

In the dynamic and rapidly evolving domain of autonomous driving, the quest for sophisticated 3D Multi-Object Tracking (3D-MOT) solutions remains a critical endeavor. The integration of camera and radar data presents a promising pathway to address the inherent challenges of perception in varying environmental conditions, a necessity for the safe operation of Autonomous Vehicle (AV). This work introduces CaRA-MOT, a novel Tracking-By-Detection (TBD) framework augmented with a memory mechanism, establishing a new state-of-the-art in the domain of Camera-Radar 3D-MOT. The proposed memory mechanism, designed to facilitate the Re-Identification (Re-ID) of unmatched tracklets through object class match identification, spatial proximity, and velocity similarity, has improved IDS and AMOTA metrics, underscoring the effectiveness of this approach. Moreover, the utilization of a class-based score filter in the pre-processing phase has demonstrated its utility in optimizing the tracking performance across various object classes, further enhancing the robustness of CaRA-MOT. Achieving AMOTA of 58.3%, AMOTP of 0.875m, and 336 in IDS on nuScenes test dataset, CaRA-MOT sets a new benchmark in the field.