Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Features

ICCV 2019  ·  Juhong Min, Jongmin Lee, Jean Ponce, Minsu Cho ·

Establishing visual correspondences under large intra-class variations requires analyzing images at different levels, from features linked to semantics and context to local patterns, while being invariant to instance-specific details. To tackle these challenges, we represent images by "hyperpixels" that leverage a small number of relevant features selected among early to late layers of a convolutional neural network. Taking advantage of the condensed features of hyperpixels, we develop an effective real-time matching algorithm based on Hough geometric voting. The proposed method, hyperpixel flow, sets a new state of the art on three standard benchmarks as well as a new dataset, SPair-71k, which contains a significantly larger number of image pairs than existing datasets, with more accurate and richer annotations for in-depth analysis.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semantic correspondence Caltech-101 HPF LT-ACC 87 # 1
IoU 63 # 1
Semantic correspondence PF-PASCAL HPF PCK 88.3 # 9
Semantic correspondence PF-WILLOW HPF PCK 76.3 # 7
Semantic correspondence SPair-71k HPF PCK 28.2 # 14

Methods


No methods listed for this paper. Add relevant methods here