Large-Scale Image Retrieval with Attentive Deep Local Features

We propose an attentive local feature descriptor suitable for large-scale image retrieval, referred to as DELF (DEep Local Feature). The new feature is based on convolutional neural networks, which are trained only with image-level annotations on a landmark image dataset. To identify semantically useful local features for image retrieval, we also propose an attention mechanism for keypoint selection, which shares most network layers with the descriptor. This framework can be used for image retrieval as a drop-in replacement for other keypoint detectors and descriptors, enabling more accurate feature matching and geometric verification. Our system produces reliable confidence scores to reject false positives---in particular, it is robust against queries that have no correct match in the database. To evaluate the proposed descriptor, we introduce a new large-scale dataset, referred to as Google-Landmarks dataset, which involves challenges in both database and query such as background clutter, partial occlusion, multiple landmarks, objects in variable scales, etc. We show that DELF outperforms the state-of-the-art global and local descriptors in the large-scale setting by significant margins. Code and dataset can be found at the project webpage: https://github.com/tensorflow/models/tree/master/research/delf .

PDF Abstract ICCV 2017 PDF ICCV 2017 Abstract

Datasets


Introduced in the Paper:

Google Landmarks

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Retrieval Oxf105k DELF+FT+ATT MAP 82.6% # 5
Image Retrieval Oxf105k DELF+FT+ATT+DIR+QE MAP 88.5% # 2
Image Retrieval Oxf5k DELF+FT+ATT MAP 83.8% # 5
Image Retrieval Oxf5k DELF+FT+ATT+DIR+QE MAP 90.0% # 3
Image Retrieval Par106k DELF+FT+ATT+DIR+QE mAP 92.8% # 2
Image Retrieval Par106k DELF+FT+ATT mAP 81.7% # 4
Image Retrieval Par6k DELF+FT+ATT+DIR+QE mAP 95.7% # 2
Image Retrieval Par6k DELF+FT+ATT mAP 85.0% # 6
Image Retrieval ROxford (Hard) DELF–HQE+SP mAP 50.3 # 8
Image Retrieval ROxford (Hard) DELF–ASMK*+SP mAP 43.1 # 10
Image Retrieval ROxford (Medium) DELF–ASMK*+SP mAP 67.8 # 9
Image Retrieval ROxford (Medium) DELF–HQE+SP mAP 73.4 # 7
Image Retrieval RParis (Hard) DELF–ASMK*+SP mAP 55.4 # 11
Image Retrieval RParis (Hard) DELF–HQE+SP mAP 69.3 # 7
Image Retrieval RParis (Medium) DELF–ASMK*+SP mAP 76.9 # 10
Image Retrieval RParis (Medium) DELF–HQE+SP mAP 84.0 # 6

Methods


No methods listed for this paper. Add relevant methods here