VPCFormer: A transformer-based multi-view finger vein recognition model and a new benchmark

In the past decade, finger vein authentication garners significant interest. However, most existing databases and algorithms predominantly focused on single-view finger vein recognition. The current projection of vein patterns actually maps a 3D network topology into a 2D plane, which inevitably leads to 3D feature loss and topological ambiguity in 2D images. Additionally, single-view based methods are sensitive to finger rotation and translation in practical applications. So far, there are currently few dedicated studies and public databases on multi-view finger vein recognition. To address these issues, we first establish a benchmark for future research by constructing the multi-view finger vein database, named Tsinghua Multi-View Finger Vein-3 Views (THUMVFV-3V) Database , which is collected over two sessions. THUMVFV-3V provides three types of Regions of Interest (ROIs) and includes unified preprocessing operations, catering to the majority of existing methods. Furthermore, we propose a novel Transformer-based model named Vein Pattern Constrained Transformer (VPCFormer) for multi-view finger vein recognition, primarily composed of multiple Vein Pattern Constrained Encoders (VPC-Encoders) and Neighborhood-Perspective Modules (NPMs). Specifically, the VPC-Encoder incorporates a novel Vein Pattern Attention Module (VPAM) and an Integrative Feed-Forward Network (IFFN). Motivated by the fact that the strong correlations veins exhibit across different views, we devise the VPAM. Assisted by a vein mask, VPAM is meticulously designed to exclusively extract intra-and inter-view dependencies between vein patterns. Further, we propose IFFN to efficiently aggregate the preceding attention and contextual information of VPAM. In addition, the NPM is utilized to capture the correlations within a single view, enhancing the final multi-view finger vein representation. Extensive experiments demonstrate the superiority of our VPCFormer. The THUMVFV-3V database is available at https://github.com/Pengyang233/ THUMVFV-3V-Database.