Contrastive feature decomposition for single image layer separation

The key challenge of image layer separation stems from recognizing different components in a single image. Typical methods optimize the modeling of different components by performing low-level supervision on the separated image to minimize its per-pixel difference from the groundtruth, which relies on substantial training samples to learn diverse components robustly and avoid overfitting spurious coupled patterns. In this work, we perform supervision on the contrastive distribution between the predicted separated images. Specifically, our proposed method separates components in parallel and seeks to maximize the distribution consistency between the separated components' contrast and their corresponding groundtruth contrast in the latent space. Such supervision pushes the model to focus on contrastive modeling between different components of the input image. Besides, the learned latent representations of different components directly guide the weight optimization of convolution kernels in the decoder, which achieves more comprehensive separation than traditional skip connection while reconstructing target images by the decoder. We validate the effectiveness and generalization of our CFDNet on two single image layer separation tasks, including image reflection separation and intrinsic image decomposition. Extensive experiments demonstrate that our CFDNet consistently outperforms other state-of-the-art methods specifically designed for either of image separation applications.