Multi-prototype collaborative perception enhancement network for few-shot semantic segmentation

Few-shot semantic segmentation (FSS) aims at learning to segment an unseen object from the query image with only a few densely annotated support images. Most existing methods rely on capturing the similarity between support and query features to facilitate segmentation for the unseen object. However, the segmentation performance of these methods remains an open problem because of the diversity between objects in the support and query images. To alleviate this problem, we propose a multi-prototype collaborative perception enhancement network. More specifically, we develop the feature recombination module to recombine the support foreground features. Second, the superpixel-guided multi-prototype generation strategy is employed to generate multiple prototypes in the support foreground and the whole query feature by aggregating similar semantic information. Meanwhile, the Vision Transformer (ViT) is used to generate background prototypes from the support background features. Third, we devise a prototype collaborative perception enhancement module to establish the interaction by exploring correspondence relations between support and query prototypes. Finally, a nonparametric metric is used to match the features and prototypes. Extensive experiments on two benchmarks, PASCAL-5i\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$<^>{i}$$\end{document} and COCO-20i\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$<^>{i}$$\end{document}, demonstrate that the proposed model has superior segmentation performance compared to baseline methods and is competitive with previous FSS methods. The code is released on https://github.com/GS-Chang-Hn/CPENet-Fss.