A Siamese network-based tracking framework for hyperspectral video

With the rapid development of hyperspectral imaging techniques, hyperspectral video visual tracking comes to a breakthrough because its abundant material-based spectral information has strong discrimination ability in complex background. Most existing hyperspectral trackers use hand-craft features to represent the appearances of targets, but their performances are limited for the lack of semantic information in those low-level features. Despite the successes of deep networks on color video, the limited training samples bring difficulties to train a deep learning model-based hyperspectral tracker. To handle well with above problems, we present a novel deep hyperspectral tracker based on Siamese network (SiamHT). In our proposed method, heterogeneous encoder-decoder (HED) and spectral semantic representation (SSR) modules are designed to extract the spatial and spectral semantic features, respectively. After that parameters in HED and SSR modules are learned with a designed two-stage training strategy. Finally, the well-learned spatial and spectral semantic representations are fused to estimate the state of a target. Extensive comparison experiments on hyperspectral object tracking dataset are performed to prove the robustness of our method.