Revisiting IM2GPS in the Deep Learning Era
Image geolocalization, inferring the geographic location of an image, is a challenging computer vision problem with many potential applications. The recent state-of-the-art approach to this problem is a deep image classification approach in which the world is spatially divided into cells and a deep network is trained to predict the correct cell for a given image. We propose to combine this approach with the original Im2GPS approach in which a query image is matched against a database of geotagged images and the location is inferred from the retrieved set. We estimate the geographic location of a query image by applying kernel density estimation to the locations of its nearest neighbors in the reference database. Interestingly, we find that the best features for our retrieval task are derived from networks trained with classification loss even though we do not use a classification approach at test time. Training with classification loss outperforms several deep feature learning methods (e.g. Siamese networks with contrastive of triplet loss) more typical for retrieval applications. Our simple approach achieves state-of-the-art geolocalization accuracy while also requiring significantly less training data.
PDF Abstract ICCV 2017 PDF ICCV 2017 AbstractDatasets
Results from the Paper
Ranked #1 on Photo geolocation estimation on Im2GPS (Reference images metric)
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Photo geolocation estimation | Im2GPS | Im2GPS (... 28m database) | Street level (1 km) | 14.4 | # 5 | |
City level (25 km) | 33.3 | # 6 | ||||
Region level (200 km) | 47.7 | # 5 | ||||
Country level (750 km) | 61.6 | # 7 | ||||
Continent level (2500 km) | 73.4 | # 7 | ||||
Training images | 6M | # 6 | ||||
Reference images | 28M | # 11 | ||||
Photo geolocation estimation | Im2GPS | Im2GPS ([L] 7011C) | Street level (1 km) | 6.8 | # 9 | |
City level (25 km) | 21.9 | # 10 | ||||
Region level (200 km) | 34.6 | # 10 | ||||
Country level (750 km) | 49.4 | # 10 | ||||
Continent level (2500 km) | 63.7 | # 11 | ||||
Training images | 6M | # 6 | ||||
Reference images | 0 | # 1 | ||||
Photo geolocation estimation | Im2GPS | Im2GPS ([L] KNN, sigma=4) | Street level (1 km) | 12.2 | # 7 | |
City level (25 km) | 33.3 | # 6 | ||||
Region level (200 km) | 44.3 | # 8 | ||||
Country level (750 km) | 57.4 | # 8 | ||||
Continent level (2500 km) | 71.3 | # 8 | ||||
Training images | 6M | # 6 | ||||
Reference images | 0 | # 1 | ||||
Photo geolocation estimation | Im2GPS3k | Im2GPS ([M] 7011C) | Street level (1 km) | 3.7 | # 11 | |
City level (25 km) | 14.2 | # 12 | ||||
Region level (200 km) | 21.3 | # 12 | ||||
Country level (750 km) | 33.5 | # 11 | ||||
Continent level (2500 km) | 52.7 | # 11 | ||||
Training Images | 6M | # 2 | ||||
Photo geolocation estimation | Im2GPS3k | Im2GPS ([L] 7011C) | Street level (1 km) | 4.0 | # 10 | |
City level (25 km) | 14.8 | # 11 | ||||
Region level (200 km) | 21.4 | # 11 | ||||
Country level (750 km) | 32.6 | # 12 | ||||
Continent level (2500 km) | 52.4 | # 12 | ||||
Training Images | 6M | # 2 | ||||
Photo geolocation estimation | Im2GPS3k | Im2GPS (kNN, sigma = 4) | Street level (1 km) | 7.2 | # 9 | |
City level (25 km) | 19.4 | # 10 | ||||
Region level (200 km) | 26.9 | # 10 | ||||
Country level (750 km) | 38.9 | # 10 | ||||
Continent level (2500 km) | 55.9 | # 10 | ||||
Training Images | 6M | # 2 | ||||
Photo geolocation estimation | YFCC4k | [L]kNN, ฯ = 4 | Street (1 km) | 2.3 | # 2 | |
City (25 km) | 5.7 | # 2 | ||||
Region (200 km) | 11.0 | # 2 | ||||
Country (750 km) | 23.5 | # 2 | ||||
Continent (2500 km) | 42.0 | # 2 |