DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering

12 Dec 2022  ·  Amit Aflalo, Shai Bagon, Tamar Kashti, Yonina Eldar ·

Image segmentation is a fundamental task in computer vision. Data annotation for training supervised methods can be labor-intensive, motivating unsupervised methods. Current approaches often rely on extracting deep features from pre-trained networks to construct a graph, and classical clustering methods like k-means and normalized-cuts are then applied as a post-processing step. However, this approach reduces the high-dimensional information encoded in the features to pair-wise scalar affinities. To address this limitation, this study introduces a lightweight Graph Neural Network (GNN) to replace classical clustering methods while optimizing for the same clustering objective function. Unlike existing methods, our GNN takes both the pair-wise affinities between local image features and the raw features as input. This direct connection between the raw features and the clustering objective enables us to implicitly perform classification of the clusters between different graphs, resulting in part semantic segmentation without the need for additional post-processing steps. We demonstrate how classical clustering objectives can be formulated as self-supervised loss functions for training an image segmentation GNN. Furthermore, we employ the Correlation-Clustering (CC) objective to perform clustering without defining the number of clusters, allowing for k-less clustering. We apply the proposed method for object localization, segmentation, and semantic part segmentation tasks, surpassing state-of-the-art performance on multiple benchmarks.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Object Localization COCO_20k DeepCut CorLoc 61.6 # 1
Unsupervised Object Segmentation DUTS DeepCut mIoU 59.5 # 1
Unsupervised Object Segmentation ECSSD DeepCut mIoU 74.6 # 1
Unsupervised Object Localization PASCAL VOC 2007 DeepCut CorLoc 69.8 # 1
Unsupervised Object Localization PASCAL VOC 2012 DeepCut CorLoc 72.2 # 1