Attention-based multiple siamese networks with primary representation guiding for offline signature verification

In the area of biometrics and document forensics, handwritten signatures are one of the most commonly accepted symbols. Thus, financial and commercial institutions usually use them to verify the identity of an individual. However, offline signature verification is still a challenging task due to the difficulties in discriminating the minute but significant details between genuine and skilled forged signatures. To tackle this issue, we propose a novel writer-independent offline signature verification approach using attention-based multiple siamese networks with primary representation guiding. The proposed multiple siamese networks regard the reference signature images, query signature images, and their corresponding inverse images as inputs. These images are fed to four weight-shared parallel branches, respectively. We present an efficient and reliable mutual attention module to discover prominent stroke information from both original and inverse branches. In each branch, feature maps of the first convolution are utilized to guide the combination with deeper features, named as primary representation guiding, which guides the model into concerning the shallow stroke information. The four branches are concatenated in an ordered way and are put into four contrastive pairs, which is helpful to obtain useful representations by comparing reference and query samples. Four contrastive pairs generate four preliminary decisions independently. Then, the eventual verification result is created based on the four preliminary decisions using a voting mechanism. In order to assess the performance of the proposed method, extensive experiments on four widely used public datasets are conducted. The experimental results demonstrate that the proposed method outperforms existing approaches in most cases and can be applied to various language scenarios.