Unsupervised Salient Object Detection with Spectral Cluster Voting

23 Mar 2022  ·  Gyungin Shin, Samuel Albanie, Weidi Xie ·

In this paper, we tackle the challenging task of unsupervised salient object detection (SOD) by leveraging spectral clustering on self-supervised features. We make the following contributions: (i) We revisit spectral clustering and demonstrate its potential to group the pixels of salient objects; (ii) Given mask proposals from multiple applications of spectral clustering on image features computed from various self-supervised models, e.g., MoCov2, SwAV, DINO, we propose a simple but effective winner-takes-all voting mechanism for selecting the salient masks, leveraging object priors based on framing and distinctiveness; (iii) Using the selected object segmentation as pseudo groundtruth masks, we train a salient object detector, dubbed SelfMask, which outperforms prior approaches on three unsupervised SOD benchmarks. Code is publicly available at https://github.com/NoelShin/selfmask.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Saliency Detection DUT-OMRON SelfMask Accuracy 91.9 # 2
IoU 65.5 # 2
maximal F-measure 85.2 # 1
Unsupervised Saliency Detection DUTS SelfMask Accuracy 93.3 # 2
IoU 66 # 2
maximal F-measure 88.2 # 1
Unsupervised Saliency Detection ECSSD SelfMask Accuracy 95.5 # 2
IoU 81.8 # 2
maximal F-measure 95.6 # 1