Hybrid two-stage cascade for instance segmentation of overlapping objects

Although two-stage methods of instance segmentation achieve better performance than one-stage counterparts, the segmentation results on overlapping objects are unsatisfactory. We found that occlusion significantly impacts the location of adjacent objects and produces coarse masks without adequate refinements. To circumvent the issue, we propose a hybrid model for instance segmentation called HTCIS, which iteratively forms the detection and segmentation. The main idea is to improve overall performance by optimizing every component based on a two-stage cascade structure. Compared with existing models, our approach decreases the loss of feature information, including semantic and detailed features. The detection branch prioritizes location accuracy when ranking bounding boxes, while the segmentation branch explores more contextual information and segments pixels in a multi-view fashion with the guide of an attention mechanism. Experimental results demonstrate that HTCIS is capable of processing occlusion. We conclude that multi-refinement of two-stage cascade is essential for accurate segmentation of overlapping objects, and our optimization is efficient in achieving this goal.