SANet: Face super-resolution based on self-similarity prior and attention integration

Recent deep learning techniques, especially CNN (Convolutional Neural Network), have been driving advancements in face super-resolution (FSR) technologies, achieving unprecedented breakthroughs. However, most existing FSR approaches fail to effectively explore and exploit the inherent self-similarity information of face images, deteriorating the FSR performance. In this paper, we propose a novel attention integration network (SANet) incorporating self-similarity information to model non-local pixel-level dependencies of features. The SANet mainly consists of Hybrid Attention Integration Modules (HAIMs), Self-similarity Information Mining Modules (SIMMs), and a CycleMLP-based Reconstruction Unit (CRU). The HAIM is designed to adaptively bootstrap features relevant to informative facial regions through the customized attention aggregation mechanism, enabling more discriminative feature extraction. The SIMM is dedicated to constructing enhanced features by thoroughly mining the self-similarity information and modeling feature-wise correlations. This is achieved with the help of the clever implementation of the well-designed Symmetric Nearest Neighbor Sampling (SNNS) strategy and Non-local Aggregated Sparse Attention (NASA) mechanism. Based on the iterative interaction between HAMIs and SIMMs, crucial facial feature information can be progressively aggregated. The CRU-based reconstruction module is crafted to restore facial details with greater pixel-wise precision more efficiently. Comprehensive experimental results on three face benchmark datasets demonstrate the superiority of the proposed SANet over current state-of-the-art methods.