ShakeDrop regularization
This paper proposes a powerful regularization method named \textit{ShakeDrop regularization}. ShakeDrop is inspired by Shake-Shake regularization that decreases error rates by disturbing learning. While Shake-Shake can be applied to only ResNeXt which has multiple branches, ShakeDrop can be applied to not only ResNeXt but also ResNet, Wide ResNet and PyramidNet in a memory efficient way. Important and interesting feature of ShakeDrop is that it strongly disturbs learning by multiplying even a negative factor to the output of a convolutional layer in the forward training pass. The effectiveness of ShakeDrop is confirmed by experiments on CIFAR-10/100 and Tiny ImageNet datasets.
PDF AbstractTasks
CIFAR-10
CIFAR-100
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
1x1 Convolution •
Average Pooling •
Batch Normalization •
Bottleneck Residual Block •
Convolution •
Dropout •
Global Average Pooling •
Grouped Convolution •
Kaiming Initialization •
Max Pooling •
Pyramidal Bottleneck Residual Unit •
Pyramidal Residual Unit •
PyramidNet •
ReLU •
Residual Block •
Residual Connection •
ResNet •
ResNeXt •
ResNeXt Block •
Shake-Shake Regularization •
Wide Residual Block •
WideResNet •
Zero-padded Shortcut Connection
