Unsupervised Multi-object Segmentation Using Attention and Soft-argmax

26 May 2022  ·  Bruno Sauvalle, Arnaud de La Fortelle ·

We introduce a new architecture for unsupervised object-centric representation learning and multi-object detection and segmentation, which uses a translation-equivariant attention mechanism to predict the coordinates of the objects present in the scene and to associate a feature vector to each object. A transformer encoder handles occlusions and redundant detections, and a convolutional autoencoder is in charge of background reconstruction. We show that this architecture significantly outperforms the state of the art on complex synthetic benchmarks.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Object Segmentation ClevrTex AST-Seg-B3-CT mIoU 79.58±0.54 # 1
MSE 139±7 # 1
Unsupervised Object Segmentation ClevrTex AST mIoU 66.62± 0.80 # 2
MSE 167± 1 # 3
Unsupervised Object Segmentation ObjectsRoom AST ARI-FG 0.87 # 1
Unsupervised Object Segmentation ShapeStacks AST ARI-FG 0.82 # 1