Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network

17 Jan 2020 Jungkyu Lee Taeryun Won Tae Kwan Lee Hyemin Lee Geonmo Gu Kiho Hong

Recent studies in image classification have demonstrated a variety of techniques for improving the performance of Convolutional Neural Networks (CNNs). However, attempts to combine existing techniques to create a practical model are still uncommon... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Fine-Grained Image Classification FGVC Aircraft Assemble-ResNet-FGVC-50 Accuracy 92.4% # 21
Fine-Grained Image Classification Food-101 Assemble-ResNet-FGVC-50 Accuracy 92.5 # 5
Image Classification ImageNet Assemble-ResNet152 Top 1 Accuracy 84.2% # 71
Image Classification ImageNet ReaL Assemble ResNet-50 Accuracy 87.82% # 19
Image Classification ImageNet ReaL Assemble-ResNet152 Accuracy 88.65% # 16
Fine-Grained Image Classification Oxford 102 Flowers Assemble-ResNet Accuracy 98.9% # 9
Fine-Grained Image Classification Oxford-IIIT Pets Assemble-ResNet-FGVC-50 Top-1 Error Rate 5.7% # 5
Accuracy 94.3% # 10
Fine-Grained Image Classification SOP Assemble-ResNet-FGVC-50 Recall@1 85.9 # 1
Fine-Grained Image Classification Stanford Cars Assemble-ResNet-FGVC-50 Accuracy 94.4% # 27

Methods used in the Paper


METHOD TYPE
Cosine Annealing
Learning Rate Schedules
Pointwise Convolution
Convolutions
Depthwise Convolution
Convolutions
Depthwise Separable Convolution
Convolutions
MobileNetV1
Convolutional Neural Networks
Sigmoid Activation
Activation Functions
Tanh Activation
Activation Functions
Channel-wise Soft Attention
Attention Mechanisms
Linear Layer
Feedforward Networks
Dilated Convolution
Convolutions
Softmax
Output Functions
Anti-Alias Downsampling
Downsampling
Selective Kernel Convolution
Convolutions
Selective Kernel
Image Model Blocks
Big-Little Module
Skip Connection Blocks
Dense Connections
Feedforward Networks
Xavier Initialization
Initialization
DropBlock
Regularization
Mixup
Image Data Augmentation
Label Smoothing
Regularization
LSTM
Recurrent Neural Networks
ColorJitter
Image Data Augmentation
Cutout
Image Data Augmentation
AutoAugment
Image Data Augmentation
Assemble-ResNet
Convolutional Neural Networks
Global Average Pooling
Pooling Operations
Kaiming Initialization
Initialization
Max Pooling
Pooling Operations
ReLU
Activation Functions
Bottleneck Residual Block
Skip Connection Blocks
Residual Block
Skip Connection Blocks
Average Pooling
Pooling Operations
1x1 Convolution
Convolutions
ResNet-D
Convolutional Neural Networks
Linear Warmup With Cosine Annealing
Learning Rate Schedules
Batch Normalization
Normalization
Weight Decay
Regularization
SGD with Momentum
Stochastic Optimization
Random Horizontal Flip
Image Data Augmentation
Random Resized Crop
Image Data Augmentation
Residual Connection
Skip Connections
Convolution
Convolutions