Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval

Optimising a ranking-based metric, such as Average Precision (AP), is notoriously challenging due to the fact that it is non-differentiable, and hence cannot be optimised directly using gradient-descent methods. To this end, we introduce an objective that optimises instead a smoothed approximation of AP, coined Smooth-AP. Smooth-AP is a plug-and-play objective function that allows for end-to-end training of deep networks with a simple and elegant implementation. We also present an analysis for why directly optimising the ranking based metric of AP offers benefits over other deep metric learning losses. We apply Smooth-AP to standard retrieval benchmarks: Stanford Online products and VehicleID, and also evaluate on larger-scale datasets: INaturalist for fine-grained category retrieval, and VGGFace2 and IJB-C for face retrieval. In all cases, we improve the performance over the state-of-the-art, especially for larger-scale datasets, thus demonstrating the effectiveness and scalability of Smooth-AP to real-world scenarios.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Retrieval iNaturalist Smooth-AP R@1 67.2 # 7
R@5 81.8 # 5
R@16 90.3 # 5
R@32 93.1 # 5
Image Retrieval SOP Smooth-AP R@1 80.1 # 7
Vehicle Re-Identification VehicleID Large Smooth-AP Rank-1 91.9 # 5
Rank-5 96.2 # 5
Vehicle Re-Identification VehicleID Medium Smooth-AP Rank-1 93.3 # 4
Rank-5 96.4 # 5
Vehicle Re-Identification VehicleID Small Smooth-AP Rank-1 94.9 # 5
Rank-5 97.6 # 5


No methods listed for this paper. Add relevant methods here