GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation

CVPR 2022  ·  Xingzhe He, Bastian Wandt, Helge Rhodin ·

Segmenting an image into its parts is a frequent preprocess for high-level vision tasks such as image editing. However, annotating masks for supervised training is expensive. Weakly-supervised and unsupervised methods exist, but they depend on the comparison of pairs of images, such as from multi-views, frames of videos, and image augmentation, which limits their applicability. To address this, we propose a GAN-based approach that generates images conditioned on latent masks, thereby alleviating full or weak annotations required in previous approaches. We show that such mask-conditioned image generation can be learned faithfully when conditioning the masks in a hierarchical manner on latent keypoints that define the position of parts explicitly. Without requiring supervision of masks or points, this strategy increases robustness to viewpoint and object positions changes. It also lets us generate image-mask pairs for training a segmentation network, which outperforms the state-of-the-art unsupervised segmentation methods on established benchmarks.

PDF Abstract CVPR 2022 PDF CVPR 2022 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Keypoint Estimation CUB GANSeg NME 12.1 # 3
Unsupervised Facial Landmark Detection MAFL Unaligned GANSeg NME 6.18 # 2
Unsupervised Human Pose Estimation Tai-Chi-HD GANSeg MAE 417.17 # 5

Results from Other Papers

Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Unsupervised Human Pose Estimation DeepFashion GANSeg PCK 59 # 3


No methods listed for this paper. Add relevant methods here