Unsupervised learning of object landmarks by factorized spatial embeddings

ICCV 2017  ·  James Thewlis, Hakan Bilen, Andrea Vedaldi ·

Learning automatically the structure of object categories remains an important open problem in computer vision. In this paper, we propose a novel unsupervised approach that can discover and learn landmarks in object categories, thus characterizing their structure. Our approach is based on factorizing image deformations, as induced by a viewpoint change or an object deformation, by learning a deep neural network that detects landmarks consistently with such visual effects. Furthermore, we show that the learned landmarks establish meaningful correspondences between different object instances in a category without having to impose this requirement explicitly. We assess the method qualitatively on a variety of object types, natural and man-made. We also show that our unsupervised landmarks are highly predictive of manually-annotated landmarks in face benchmark datasets, and can be used to regress these with a high degree of accuracy.

PDF Abstract ICCV 2017 PDF ICCV 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Unsupervised Facial Landmark Detection 300W FSE NME 7.97 # 3
Unsupervised Facial Landmark Detection AFLW-MTFL FSE NME 10.53 # 2
Unsupervised Facial Landmark Detection MAFL FSE NME 6.67 # 12
Unsupervised Facial Landmark Detection MAFL Unaligned ULD NME 31.3 # 7

Results from Other Papers

Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Unsupervised Human Pose Estimation Human3.6M Thewlis2017unsupervised NME 7.51 # 6
Unsupervised Facial Landmark Detection MAFL Thewlis2017unsupervised NME 6.32 # 11


No methods listed for this paper. Add relevant methods here