MSG-SR-Net: A Weakly Supervised Network Integrating Multi-Scale Generation and Super-Pixel Refinement for Building Extraction from High-Resolution Remotely Sensed Imageries

Weakly supervised semantic segmentation (WSSS) methods based on image-level labels can relieve the tedious pixel-level annotation burden, and these methods are mainly based on class activation maps (CAMs). However, it is challenging to generate high-quality CAMs for high-resolution remotely sensed imagery (HRSI). In this paper, we propose a WSSS method for building extraction from HRSI using image-level labels. The proposed method, termed as the MSG-SR-Net, integrates two novel modules, i.e., multi-scale generation (MSG) and super-pixel refinement (SR), to obtain high-quality CAMs so as to provide reliable pixel-level training samples for subsequent semantic segmentation steps. The MSG module is proposed to use global semantic information to guide the learning of multiple features across different levels, and then respectively to utilize multi-level features for generating multi-scale CAMs. This component can effectively suppress the interference of the class-irrelevant noise and strengthen the use of profitable information in multi-level features. The SR module is designed to take advantage of super-pixels to improve multi-scale CAMs in target integrity and details preserving. Extensive experiments on two public building datasets demonstrated that the proposed modules made the MSG-SR-Net obtain more integral and accurate CAMs for building extraction. Moreover, experimental results also showed the proposed method achieved excellent performance with over 67% in F1-score, and outperformed other weakly supervised methods in effectiveness and generalization ability.