Unsupervised learning of object landmarks by factorized spatial embeddings
Learning automatically the structure of object categories remains an important open problem in computer vision. In this paper, we propose a novel unsupervised approach that can discover and learn landmarks in object categories, thus characterizing their structure. Our approach is based on factorizing image deformations, as induced by a viewpoint change or an object deformation, by learning a deep neural network that detects landmarks consistently with such visual effects. Furthermore, we show that the learned landmarks establish meaningful correspondences between different object instances in a category without having to impose this requirement explicitly. We assess the method qualitatively on a variety of object types, natural and man-made. We also show that our unsupervised landmarks are highly predictive of manually-annotated landmarks in face benchmark datasets, and can be used to regress these with a high degree of accuracy.
PDF Abstract ICCV 2017 PDF ICCV 2017 AbstractResults from the Paper
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Unsupervised Facial Landmark Detection | 300W | FSE | NME | 7.97 | # 3 | |
Unsupervised Facial Landmark Detection | AFLW-MTFL | FSE | NME | 10.53 | # 2 | |
Unsupervised Facial Landmark Detection | MAFL | FSE | NME | 6.67 | # 12 | |
Unsupervised Facial Landmark Detection | MAFL Unaligned | ULD | NME | 31.3 | # 7 |