Hallucination of facial details from degraded images using 3D face models

The goals of this paper are: (1) to enhance the quality of images of faces, (2) to enable 3D Morphable Models (3DMMs) to cope with severely degraded images, and (3) to reconstruct textured 3D faces with details that are not in the input images. Details that are lost in the input images due to blur, low resolution or occlusions, are filled in by the 3DMM and an additional texture enhancement algorithm that adds high-resolution details from example faces. By leveraging class-specific knowledge, this restoration process goes beyond what general image operations such as deblurring or inpainting can achieve. The benefit of the 3DMM for image restoration is that it can be applied to any pose and illumination, unlike image-based methods. However, it is only with the new fitting algorithm that 3DMMs can produce realistic faces from severely degraded images. The new method includes the blurring or downsampling operator explicitly into the analysis-by-synthesis algorithm. (C) 2015 Elsevier B.V. All rights reserved.