3D Multi-Object Tracking Based on Radar-Camera Fusion

Multi-target tracking is the cornerstone of autonomous driving. Only on this basis can we achieve tracking and path planning. Detection problem is an important part of environment perception system. Therefore, it has become a hot spot in related fields such as computer vision and autonomous driving. 3D object detection adds more information than 2D object detection, such as size, depth, pose, etc. However, the 3D information obtained based on the image, especially the depth, is very inaccurate, so that the detection frame information has a large deviation. The trajectory tracking ability based on these detection results is also reduced accordingly. Our research proposes a method to detect 3D information by combining data from cameras and mmWave radar. We then apply this result to the multi-object tracking problem. It achieves a MOTA of 70.0 and a IDF1 of 73.1 on the NuScenes dataset.