Robust and Decomposable Average Precision for Image Retrieval

In image retrieval, standard evaluation metrics rely on score ranking, e.g. average precision (AP). In this paper, we introduce a method for robust and decomposable average precision (ROADMAP) addressing two major challenges for end-to-end training of deep neural networks with AP: non-differentiability and non-decomposability. Firstly, we propose a new differentiable approximation of the rank function, which provides an upper bound of the AP loss and ensures robust training. Secondly, we design a simple yet effective loss function to reduce the decomposability gap between the AP in the whole training set and its averaged batch approximation, for which we provide theoretical guarantees. Extensive experiments conducted on three image retrieval datasets show that ROADMAP outperforms several recent AP approximation methods and highlight the importance of our two contributions. Finally, using ROADMAP for training deep models yields very good performances, outperforming state-of-the-art results on the three datasets.

PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Retrieval CUB-200-2011 ROADMAP (ResNet-50) R@1 68.5 # 4
Image Retrieval CUB-200-2011 ROADMAP (Deit-B) R@1 77.4 # 2
Image Retrieval iNaturalist ROADMAP (DeiT-S) R@1 73.6 # 2
R@5 86.2 # 2
R@16 93.1 # 2
R@32 95.2 # 2
Image Retrieval iNaturalist ROADMAP (ResNet-50) R@1 69.1 # 6
R@5 83.1 # 4
R@16 91.3 # 4
R@32 93.9 # 4
Image Retrieval SOP ROADMAP (DeiT-B) R@1 86.0 # 1
Image Retrieval SOP ROADMAP (ResNet-50) R@1 83.1 # 3
Metric Learning Stanford Online Products ROADMAP (ResNet-50) R@1 83.1 # 6
Metric Learning Stanford Online Products ROADMAP (DeiT-S) R@1 86.0 # 2

Methods


No methods listed for this paper. Add relevant methods here