One Model Is Enough: Toward Multiclass Weakly Supervised Remote Sensing Image Semantic Segmentation

Semantic segmentation of remote sensing images (RSIs) is effective for large-scale land cover mapping, which heavily relies on a large amount of training data with laborious pixel-level labeling. Due to the easy availability of image-level labels, weakly supervised semantic segmentation (WSSS) based on them has attracted intensive attention. However, existing image-level WSSS methods for RSIs mainly focus on binary segmentation, which are difficult to apply to multiclass scenarios. This study proposes a comprehensive framework for image-level multiclass WSSS of RSIs, consisting of appropriate image-level label generation, high-quality pixel-level pseudo mask generation, and segmentation network iterative training. Specifically, a training sample filtering method, as well as a dataset co-occurrence evaluation metric, is proposed to demonstrate proper image-level training samples. Leveraging multiclass class activation maps (CAMs), an uncertainty-driven pixel-level weighted mask is proposed to relieve the overfitting of labeling noise in pseudo masks when training the segmentation network. Extensive experiments demonstrate that the proposed framework can achieve high-quality multiclass WSSS performance with image-level labels, which can attain 94.23% and 90.77% of the mean intersection over union (mIoU) from pixel-level labels for the ISPRS Potsdam and Vaihingen datasets, respectively. Beyond that, for the DeepGlobe dataset with more complex landscapes, the WSSS framework can achieve an accuracy close to 99% of the fully supervised case. In addition, we further demonstrate that compared to adopting multiple binary WSSS models, directly training a multiclass WSSS model can achieve better results, which can provide new thoughts to achieve WSSS of RSIs for multiclass application scenarios. Our code is publically available at https://github.com/NJU-LHRS/OME.