Siam-Sort: Multi-Target Tracking in Video SAR Based on Tracking by Detection and Siamese Network

Shadows are widely used in the tracking of moving targets by video synthetic aperture radar (video SAR). However, they always appear in groups in video SAR images. In such cases, track effects produced by existing single-target tracking methods are no longer satisfactory. To this end, an effective way to obtain the capability of multiple target tracking (MTT) is in urgent demand. Note that tracking by detection (TBD) for MTT in optical images has achieved great success. However, TBD cannot be utilized in video SAR MTT directly. The reasons for this is that shadows of moving target are quite different from in video SAR image than optical images as they are time-varying and their pixel sizes are small. The aforementioned characteristics make shadows in video SAR images hard to detect in the process of TBD and lead to numerous matching errors in the data association process, which greatly affects the final tracking performance. Aiming at the above two problems, in this paper, we propose a multiple target tracking method based on TBD and the Siamese network. Specifically, to improve the detection accuracy, the multi-scale Faster-RCNN is first proposed to detect the shadows of moving targets. Meanwhile, dimension clusters are used to accelerate the convergence speed of the model in the training process as well as to obtain better network weights. Then, SiamNet is proposed for data association to reduce matching errors. Finally, we apply a Kalman filter to update the tracking results. The experimental results on two real video SAR datasets demonstrate that the proposed method outperforms other state-of-art methods, and the ablation experiment verifies the effectiveness of multi-scale Faster-RCNN and SimaNet.