Unsupervised Point Cloud Pre-Training via Occlusion Completion
We describe a simple pre-training approach for point clouds. It works in three steps: 1. Mask all points occluded in a camera view; 2. Learn an encoder-decoder model to reconstruct the occluded points; 3. Use the encoder weights as initialisation for downstream point cloud tasks. We find that even when we construct a single pre-training dataset (from ModelNet40), this pre-training method improves accuracy across different datasets and encoders, on a wide range of downstream tasks. Specifically, we show that our method outperforms previous pre-training methods in object classification, and both part-based and semantic segmentation tasks. We study the pre-trained features and find that they lead to wide downstream minima, have high transformation invariance, and have activations that are highly correlated with part labels. Code and data are available at: https://github.com/hansen7/OcCo
PDF Abstract ICCV 2021 PDF ICCV 2021 AbstractCode
Datasets
ModelNet
S3DIS
ScanObjectNN
SensatUrban
PointCloud-C
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| 3D Point Cloud Linear Classification | ModelNet40 | OcCo | Overall Accuracy | 89.2 | # 14 | |
| Few-Shot 3D Point Cloud Classification | ModelNet40 10-way (10-shot) | OcCo+DGCNN | Overall Accuracy | 82.9 | # 17 | |
| Standard Deviation | 1.3 | # 2 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 10-way (10-shot) | OcCo+PointNet | Overall Accuracy | 83.9 | # 16 | |
| Standard Deviation | 1.8 | # 3 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 10-way (20-shot) | OcCo+DGCNN | Overall Accuracy | 86.5 | # 17 | |
| Standard Deviation | 2.2 | # 6 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 10-way (20-shot) | OcCo+PointNet | Overall Accuracy | 89.7 | # 16 | |
| Standard Deviation | 1.5 | # 1 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 5-way (10-shot) | OcCo+PointNet | Overall Accuracy | 89.7 | # 16 | |
| Standard Deviation | 1.9 | # 6 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 5-way (10-shot) | OcCo+DGCNN | Overall Accuracy | 90.6 | # 15 | |
| Standard Deviation | 2.8 | # 15 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 5-way (20-shot) | OcCo+PointNet | Overall Accuracy | 92.4 | # 16 | |
| Standard Deviation | 1.6 | # 12 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 5-way (20-shot) | OcCo+DGCNN | Overall Accuracy | 92.5 | # 15 | |
| Standard Deviation | 1.9 | # 16 | ||||
| Point Cloud Segmentation | PointCloud-C | OcCo-PCN | mean Corruption Error (mCE) | 1.173 | # 10 | |
| Point Cloud Segmentation | PointCloud-C | OcCo-DGCNN | mean Corruption Error (mCE) | 0.977 | # 4 | |
| Point Cloud Segmentation | PointCloud-C | OcCo-PointNet | mean Corruption Error (mCE) | 1.130 | # 9 | |
| Point Cloud Classification | PointCloud-C | OcCo-DGCNN | mean Corruption Error (mCE) | 1.047 | # 18 | |
| 3D Point Cloud Linear Classification | ScanObjectNN | OcCo | Overall Accuracy | 78.3 | # 3 | |
| Few-Shot 3D Point Cloud Classification | ScanObjectNN 10-way (10-shot) | OcCo | Overall Accuracy | 57.0 | # 3 |
