Grouped Pointwise Convolutions Reduce Parameters in Convolutional Neural Networks
In Deep Convolutional Neural Networks (DCNNs), the parameter count in pointwise convolutions quickly grows due to the multiplication of the filters and input channels from the preceding layer. To handle this growth, we propose a new technique that makes pointwise convolutions parameter-efficient via employing parallel branching, where each branch contains a group of filters and processes a fraction of the input channels. To avoid degrading the learning capability of DCNNs, we propose interleaving the filters' output from separate branches at intermediate layers of successive pointwise convolutions. To demonstrate the efficacy of the proposed technique, we apply it to various state-of-the-art DCNNs, namely EfficientNet, DenseNet-BC L100, MobileNet and MobileNet V3 Large. The performance of these DCNNs with and without the proposed method is compared on CIFAR-10, CIFAR-100, Cropped-PlantDoc and Oxford-IIIT Pet datasets. The experimental results demonstrated that DCNNs with the proposed technique, when trained from scratch, obtained similar test accuracies to the original EfficientNet and MobileNet V3 Large architectures while saving up to 90% of the parameters and 63% of the floating-point computations.
PDFCode
Colab
Tasks
CIFAR-10
CIFAR-100
Oxford-IIIT Pet Dataset
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Image Classification | CIFAR-10 | kEffNet-B0 32ch | Percentage correct | 93.75 | # 163 | ||
| Image Classification | CIFAR-10 | kDenseNet-BC L100 12ch | Percentage correct | 90.83 | # 192 | ||
| Image Classification | CIFAR-10 | kMobileNet V3 Large 16ch | Percentage correct | 92.74 | # 173 | ||
| Image Classification | CIFAR-10 | kMobileNet 16ch | Percentage correct | 89.81 | # 201 | ||
| Image Classification | CIFAR-100 | kMobileNet V3 Large 16ch | Percentage correct | 71.36 | # 165 | ||
| PARAMS | 0.52M | # 183 | |||||
| Image Classification | Oxford-IIIT Pet Dataset | kMobileNet V3 Large 16ch | Accuracy | 60.4 | # 4 | ||
| PARAMS | 0.36M | # 4 | |||||
| FLOPS | 81.0M | # 5 | |||||
| Image Classification | PlantDoc | kMobileNet V3 Large 16ch | PARAMS | 0.43M | # 1 |
