Invariant Information Clustering for Unsupervised Image Classification and Segmentation

ICCV 2019  ·  Xu Ji, João F. Henriques, Andrea Vedaldi ·

We present a novel clustering objective that learns a neural network classifier from scratch, given only unlabelled data samples. The model discovers clusters that accurately match semantic classes, achieving state-of-the-art results in eight unsupervised clustering benchmarks spanning image classification and segmentation. These include STL10, an unsupervised variant of ImageNet, and CIFAR10, where we significantly beat the accuracy of our closest competitors by 6.6 and 9.5 absolute percentage points respectively. The method is not specialised to computer vision and operates on any paired dataset samples; in our experiments we use random transforms to obtain a pair from each image. The trained network directly outputs semantic labels, rather than high dimensional representations that need external processing to be usable for semantic clustering. The objective is simply to maximise mutual information between the class assignments of each pair. It is easy to implement and rigorously grounded in information theory, meaning we effortlessly avoid degenerate solutions that other clustering methods are susceptible to. In addition to the fully unsupervised mode, we also test two semi-supervised settings. The first achieves 88.8% accuracy on STL10 classification, setting a new global state-of-the-art over all existing methods (whether supervised, semi-supervised or unsupervised). The second shows robustness to 90% reductions in label coverage, of relevance to applications that wish to make use of small amounts of labels.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Unsupervised Image Classification CIFAR-10 IIC Accuracy 61.7 # 5
Image Clustering CIFAR-10 IIC Accuracy 0.617 # 20
NMI 0.511 # 17
Train set Train+Test # 1
ARI 0.411 # 17
Backbone ResNet-34 # 1
Unsupervised Image Classification CIFAR-20 IIC Accuracy 25.7 # 7
Unsupervised Semantic Segmentation COCO-Stuff-15 IIC Pixel Accuracy 27.7 # 3
Unsupervised Semantic Segmentation COCO-Stuff-171 IIC (ResNet-50) mIoU 2.2 # 4
Pixel Accuracy 15.7 # 4
Unsupervised Semantic Segmentation COCO-Stuff-27 IIC Accuracy 21.8 # 13
Unsupervised Semantic Segmentation COCO-Stuff-3 IIC Pixel Accuracy 72.3 # 6
Unsupervised MNIST MNIST IIC Accuracy 99.3 # 1
Unsupervised Image Classification MNIST IIC Accuracy 99.3 # 1
Unsupervised Semantic Segmentation Potsdam-3 IIC Accuracy 45.4 # 3
Semantic Segmentation RSMSS IIC mIoU 20.3% # 10
Image Classification STL-10 IIC Percentage correct 88.8 # 43
Unsupervised Image Classification STL-10 IIC Accuracy 61.00 # 7
Semi-Supervised Image Classification STL-10 IIC Accuracy 88.8 # 2


No methods listed for this paper. Add relevant methods here