Image Inpainting Algorithm with Diverse Aggregation of Contextual Information

被引:0
|
作者
Li H. [1 ]
Chao Y. [1 ]
Yu P. [1 ]
Li H. [1 ]
Zhang Y. [1 ]
机构
[1] School of Information Science and Engineering, Yunnan University, Kunming
[2] Yunnan Communications Investment and Construction Group Company Limited, Kunming
关键词
diverse aggregation of contextual information; encoding and decoding information fusion; image inpainting; mask matching discriminator;
D O I
10.13190/j.jbupt.2021-317
中图分类号
学科分类号
摘要
To effectively solve the defects of structural distortion and blurry texture when repairing large and irregular semantic missing area images by using the existing algorithms, a diverse aggregation image restoration algorithm based on contextual information is proposed. First, the information on the damaged image is extracted by the encoder to estimate the missing content. Thereafter, the context information from various receptive fields is merged through the multi-information aggregation block to enhance the structure and texture information of the missing area. Then, the original image features are restored through the decoder. Finally, the mask matching discriminator is adopted to perform discrimination training on the generated image, and the model is optimized by combining the counter loss, reconstruction loss, perception loss and style loss to promote the generator to synthesize clear textures. The proposed algorithm is trained and tested on the public data set. The experimental results show that the proposed algorithm can obtain clearer and more reasonable structure and texture details than state-of-the-art when inpainting randomly irregular and large missing areas. Its objective indices such as peak signal-to-noise ratio and structural similarity are superior over the compared algorithms. © 2023 Beijing University of Posts and Telecommunications. All rights reserved.
引用
收藏
页码:19 / 25
页数:6
相关论文
共 16 条
  • [1] BERTALMIO M, BERTOZZI A L, SAPIRO G., Navierstokes, fluid dynamics, and image and video inpainting, 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 355-362, (2001)
  • [2] BERTALMIO M, VESE L, SAPIRO G, Et al., Simultaneous structure and texture image inpainting [J], IEEE Transactions on Image Processing, 12, 8, pp. 882-889, (2003)
  • [3] JIANG R, MA L Z, NIE D D., Method for inpainting large block, Application Research of Computers, 23, 12, pp. 338-340, (2006)
  • [4] SUN J, YUAN L, JIA J Y, Et al., Image completion with structure propagation, ACM Transactions on Graphics, 24, 3, pp. 861-868, (2005)
  • [5] XIE J Y, XU L L, CHEN E H., Image denoising and inpainting with deep neural networks, Advances in Neural Information Processing Systems, pp. 341-349, (2012)
  • [6] KHLER R, SCHULER C, SCHLKOPF B, Et al., Mask-specific inpainting with deep neural networks, German Conference on Pattern Recognition(GCPR), pp. 523-534, (2014)
  • [7] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, Et al., Generative adversarial networks [J], Advances in Neural Information Processing Systems, 11, 3, pp. 2672-2680, (2014)
  • [8] PATHAK D, DONAHUE J, DARRELL T, Et al., Context encoders: feature learning by inpainting, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2536-2544, (2016)
  • [9] YU J H, LIN Z, YANG J, Et al., Generative image inpainting with contextual attention, ACM Transactions on Graphics, 21, 3, pp. 1-15, (2018)
  • [10] LIU G, REDA F A, SHIH K, Et al., Image inpainting for irregular holes using partial convolutions, European Conference on Computer Vision, pp. 89-105, (2018)