SageMix: Saliency-Guided Mixup for Point Clouds

13 Oct 2022  ·  Sanghyeok Lee, Minkyu Jeon, Injae Kim, Yunyang Xiong, Hyunwoo J. Kim ·

Data augmentation is key to improving the generalization ability of deep learning models. Mixup is a simple and widely-used data augmentation technique that has proven effective in alleviating the problems of overfitting and data scarcity. Also, recent studies of saliency-aware Mixup in the image domain show that preserving discriminative parts is beneficial to improving the generalization performance. However, these Mixup-based data augmentations are underexplored in 3D vision, especially in point clouds. In this paper, we propose SageMix, a saliency-guided Mixup for point clouds to preserve salient local structures. Specifically, we extract salient regions from two point clouds and smoothly combine them into one continuous shape. With a simple sequential sampling by re-weighted saliency scores, SageMix preserves the local structure of salient regions. Extensive experiments demonstrate that the proposed method consistently outperforms existing Mixup methods in various benchmark point cloud datasets. With PointNet++, our method achieves an accuracy gain of 2.6% and 4.0% over standard training in 3D Warehouse dataset (MN40) and ScanObjectNN, respectively. In addition to generalization performance, SageMix improves robustness and uncertainty calibration. Moreover, when adopting our method to various tasks including part segmentation and standard 2D image classification, our method achieves competitive performance.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Classification CIFAR-100 PreActResNet-18 + SageMix Percentage correct 80.16 # 124
3D Point Cloud Classification ModelNet40 DGCNN + SageMix Overall Accuracy 93.6 # 40
3D Point Cloud Classification ModelNet40 PointNet++ + SageMix Overall Accuracy 93.3 # 54
3D Point Cloud Classification ModelNet40 PointNet + SageMix Overall Accuracy 90.3 # 89
3D Point Cloud Classification ScanObjectNN DGCNN + SageMix Overall Accuracy 83.6 # 42
3D Point Cloud Classification ScanObjectNN PointNet++ + SageMix Overall Accuracy 83.7 # 41
3D Part Segmentation ShapeNet-Part PointNet++ + SageMix Instance Average IoU 85.7 # 40
3D Part Segmentation ShapeNet-Part DGCNN + SageMix Instance Average IoU 85.4 # 44