Appearance Harmonization for Single Image Shadow Removal

Shadow removal is a challenging problem and previous approaches often produce de-shadowed regions that are visually inconsistent with the rest of the image. We propose an automatic shadow region harmonization approach that makes the appearance of a de-shadowed region (produced using any previous technique) compatible with the rest of the image. We use a shadow-guided patch-based image synthesis approach that reconstructs the shadow region using patches sampled from non-shadowed regions. This result is then refined based on the reconstruction confidence to handle unique textures. Qualitative comparisons over a wide range of images, and a quantitative evaluation on a benchmark dataset show that our technique significantly improves upon the state-of-the-art.