CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features

Regional dropout strategies have been proposed to enhance the performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to attend on less discriminative parts of objects (e.g. leg as opposed to head of a person), thereby letting the network generalize better and have better object localization capabilities. On the other hand, current methods for regional dropout remove informative pixels on training images by overlaying a patch of either black pixels or random noise. Such removal is not desirable because it leads to information loss and inefficiency during training. We therefore propose the CutMix augmentation strategy: patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches. By making efficient use of training pixels and retaining the regularization effect of regional dropout, CutMix consistently outperforms the state-of-the-art augmentation strategies on CIFAR and ImageNet classification tasks, as well as on the ImageNet weakly-supervised localization task. Moreover, unlike previous augmentation methods, our CutMix-trained ImageNet classifier, when used as a pretrained model, results in consistent performance gains in Pascal detection and MS-COCO image captioning benchmarks. We also show that CutMix improves the model robustness against input corruptions and its out-of-distribution detection performances. Source code and pretrained models are available at \href{}{}.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Semantic Segmentation ACDC Scribbles CutMix Dice (Average) 70.5% # 3
Image Classification CIFAR-10 PyramidNet-200 + CutMix Percentage correct 97.12 # 89
Image Classification CIFAR-100 PyramidNet-200 + Shakedrop + Cutmix Percentage correct 86.19 # 55
Image Captioning COCO NIC (ResNet-50, CutMix) BLEU-1 64.2 # 1
BLEU-2 46.3 # 1
BLEU-3 33.6 # 1
BLEU-4 24.9 # 3
CIDEr 77.6 # 10
METEOR 23.1 # 1
ROUGE 49 # 1
Image Classification ImageNet ResNet-50 (CutMix) Top 1 Accuracy 78.4% # 723
Top 5 Accuracy 94.10% # 177
Image Classification ImageNet ResNeXt-101 (CutMix) Top 1 Accuracy 80.53% # 595
Top 5 Accuracy 94.97% # 134
Domain Generalization ImageNet-A CutMix (ResNet-50) Top-1 accuracy % 7.3 # 19
Out-of-Distribution Generalization ImageNet-W CutMix (ResNet-50) IN-W Gap -14.8 # 1
Carton Gap +22 # 1
Image Classification OmniBenchmark CutMix Average Top-1 Accuracy 31.1 # 20
Out-of-Distribution Generalization UrbanCars CutMix BG Gap -45.0 # 1
CoObj Gap -4.8 # 1
BG+CoObj Gap -86.5 # 1