Face super resolution with a high frequency highway

Face shape priors such as landmarks, heatmaps, and parsing maps are widely used to improve face super resolution (SR). It is observed that face priors provide locations of high-frequency details in key facial areas such as the eyes and mouth. However, existing methods fail to effectively exploit the high-frequency information by using the priors as either constraints or inputs. This paper proposes a novel high frequency highway (H2F${\rm H}_2{\rm F}$) framework to better utilize prior information for face SR, which dynamically decomposes the final SR face into a coarse SR face and a high frequency (HF) face. The coarse SR face is reconstructed from a low-resolution face via a texture branch, using only pixel-wise reconstruction loss. Meanwhile, the HF face is directly generated from face priors via an HF branch that employs the proposed inception-hourglass model. As a result, H2F${\rm H}_2{\rm F}$ allows the face priors to have a direct impact on the SR face by adding the outputs of both branches as the final result and provides an extra face editing function. Extensive experiments show that H2F${\rm H}_2{\rm F}$ significantly outperforms state-of-the-art face SR methods, is general for different texture branch models and face priors, and is robust to dataset mismatch and pose variations. The method is the first to construct an HF face directly from the face priors via a high-frequency highway for face super-resolution, making it easy to understand the HF information gain. The method dynamically decomposes a final SR face into a coarse SR face and an HF face, making it possible to prevent the smoothing of HF details during learning. image