TResNet: High Performance GPU-Dedicated Architecture

Many deep learning models, developed in recent years, reach higher ImageNet accuracy than ResNet50, with fewer or comparable FLOPS count. While FLOPs are often seen as a proxy for network efficiency, when measuring actual GPU training and inference throughput, vanilla ResNet50 is usually significantly faster than its recent competitors, offering better throughput-accuracy trade-off... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Image Classification CIFAR-10 TResNet-XL Percentage correct 99 # 12
Image Classification CIFAR-100 TResNet-L-V2 Percentage correct 92.6 # 8
Image Classification Flowers-102 TResNet-L Accuracy 99.1% # 10
Image Classification ImageNet TResNet-XL Top 1 Accuracy 84.3% # 68
Number of params 77M # 60
Multi-Label Classification MS-COCO TResNet-L mAP 86.4 # 5
Fine-Grained Image Classification Oxford 102 Flowers TResNet-L Accuracy 99.1% # 6

Methods used in the Paper


METHOD TYPE
Softplus
Activation Functions
Mish
Activation Functions
Tanh Activation
Activation Functions
Average Pooling
Pooling Operations
Dense Connections
Feedforward Networks
Sigmoid Activation
Activation Functions
Batch Normalization
Normalization
Anti-Alias Downsampling
Downsampling
InPlace-ABN
Normalization
Squeeze-and-Excitation Block
Image Model Blocks
Global Average Pooling
Pooling Operations
Residual Connection
Skip Connections
ReLU
Activation Functions
Leaky ReLU
Activation Functions
1x1 Convolution
Convolutions
Convolution
Convolutions
Weight Decay
Regularization
Label Smoothing
Regularization
LSTM
Recurrent Neural Networks
ColorJitter
Image Data Augmentation
Cutout
Image Data Augmentation
AutoAugment
Image Data Augmentation
TResNet
Convolutional Neural Networks