Convolutional Neural Networks

EfficientNetV2

Introduced by Tan et al. in EfficientNetV2: Smaller Models and Faster Training

EfficientNetV2 is a type convolutional neural network that has faster training speed and better parameter efficiency than previous models. To develop these models, the authors use a combination of training-aware neural architecture search and scaling, to jointly optimize training speed. The models were searched from the search space enriched with new ops such as Fused-MBConv.

Architecturally the main differences are:

  • EfficientNetV2 extensively uses both MBConv and the newly added fused-MBConv in the early layers.
  • EfficientNetV2 prefers smaller expansion ratio for MBConv since smaller expansion ratios tend to have less memory access overhead.
  • EfficientNetV2 prefers smaller 3x3 kernel sizes, but it adds more layers to compensate the reduced receptive field resulted from the smaller kernel size.
  • EfficientNetV2 completely removes the last stride-1 stage in the original EfficientNet, wperhaps due to its large parameter size and memory access overhead.
Source: EfficientNetV2: Smaller Models and Faster Training

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
DeepFake Detection 1 10.00%
Face Swapping 1 10.00%
Self-Supervised Learning 1 10.00%
Denoising 1 10.00%
Image Denoising 1 10.00%
Knowledge Distillation 1 10.00%
Network Pruning 1 10.00%
Super-Resolution 1 10.00%
Quantization 1 10.00%

Categories