Deep Matching Online Video Stabilization Using TSNet

With the continuous progress of unmanned aerial vehicle (UAV) target recognition and detection technology, the application of UAVs in measurement and inspection is becoming more and more extensive. However, UAVs are affected by external disturbance during flight, which leads to the degradation of the video quality recorded by the camera and thus affects the effect of target detection and recognition. To solve this problem, this article proposes a video stabilization framework based on deep matching and TSNet, which consists of three phases: motion estimation, trajectory smoothing, and image warping. In the motion estimation stage, this article adopts the keypoint matching method to estimate the camera motion, instead of using optical flow to estimate the trajectory as most of the video stabilization methods. In the trajectory smoothing phase, this article designs a plug-and-play trajectory smoothing network called TSNet and extracts the camera paths of stable and unstable videos from a public video stabilization dataset as a training set. To better evaluate the stabilization methods, this article proposes an evaluation metric called Smooth Ratio by attaching weights to several video stabilization evaluation metrics followed in most papers through the analytic hierarchy process (AHP). The experimental results show that the average stabilization time of our method is 0.067 s for one frame, and the stabilization effect is significantly compared to representative methods.