Learned Good Features to Track

Tracking features in image sequences suffers from varying illumination and viewpoints. In recent years, learningbased features have achieved higher repeatability in challenging scenes and are considered to have the potential to solve this problem. However, features with high repeatability are not always easy to track. There is a gap between repeatability and trackability. To obtain features that are easily tracked in illumination and viewpoints, a data-driven approach expands the definition of good features. Trackability is defined end-toend as the tracking error. According to this definition, a complete feature tracking process is used to compute the tracking loss and train the network. A four-layer convolutional network is used to extract low dimensional image information and obtain features. To validate the proposed method, we compare the tracking errors of several mainstream methods on a challenging test dataset, and the proposed method shows significant advantages. Then, fundamental matrix estimation and visual odometry experiments demonstrate the feature excels in practical tasks. Finally, the features were used in a visual inertial odometry system and achieved a 43% improvement in absolute trajectory error on the challenging dataset. All code will be open source for the benefit of the community.