A Siamese network-based tracking framework for hyperspectral video

With the rapid development of hyperspectral imaging techniques, hyperspectral video visual tracking comes to a breakthrough because its abundant material-based spectral information has strong discrimination ability in complex background. Most existing hyperspectral trackers use hand-craft features to represent the appearances of targets, but their performances are limited for the lack of semantic information in those low-level features. Despite the successes of deep networks on color video, the limited training samples bring difficulties to train a deep learning model-based hyperspectral tracker. To handle well with above problems, we present a novel deep hyperspectral tracker based on Siamese network (SiamHT). In our proposed method, heterogeneous encoder–decoder (HED) and spectral semantic representation (SSR) modules are designed to extract the spatial and spectral semantic features, respectively. After that parameters in HED and SSR modules are learned with a designed two-stage training strategy. Finally, the well-learned spatial and spectral semantic representations are fused to estimate the state of a target. Extensive comparison experiments on hyperspectral object tracking dataset are performed to prove the robustness of our method.