3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions

Matching local geometric features on real-world depth images is a challenging task due to the noisy, low-resolution, and incomplete nature of 3D scan data. These difficulties limit the performance of current state-of-art methods, which are typically based on histograms over geometric properties. In this paper, we present 3DMatch, a data-driven model that learns a local volumetric patch descriptor for establishing correspondences between partial 3D data. To amass training data for our model, we propose a self-supervised feature learning method that leverages the millions of correspondence labels found in existing RGB-D reconstructions. Experiments show that our descriptor is not only able to match local geometry in new scenes for reconstruction, but also generalize to different tasks and spatial scales (e.g. instance-level object model alignment for the Amazon Picking Challenge, and mesh surface correspondence). Results show that 3DMatch consistently outperforms other state-of-the-art approaches by a significant margin. Code, data, benchmarks, and pre-trained models are available online at http://3dmatch.cs.princeton.edu

PDF Abstract CVPR 2017 PDF CVPR 2017 Abstract


Introduced in the Paper:


Used in the Paper:


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Point Cloud Registration 3DMatch Benchmark 3DMatch + RANSAC Feature Matching Recall 66.8 # 12
Point Cloud Registration ETH (trained on 3DMatch) 3DMatch Recall 0.169 # 9
3D Reconstruction Scan2CAD 3DMatch Average Accuracy 10.29% # 2


No methods listed for this paper. Add relevant methods here