High-Performance Large-Scale Image Recognition Without Normalization

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Image Classification	ImageNet	NFNet-F6 w/ SAM	Top 1 Accuracy	86.5%	# 14
Image Classification	ImageNet	NFNet-F6 w/ SAM	Top 5 Accuracy	97.9%	# 8
Image Classification	ImageNet	NFNet-F6 w/ SAM	Number of params	438.4M	# 8
Image Classification	ImageNet	NFNet-F4+	Top 1 Accuracy	89.2%	# 3
Image Classification	ImageNet	NFNet-F4+	Number of params	527M	# 4
Image Classification	ImageNet	NFNet-F5 w/ SAM	Top 1 Accuracy	86.3%	# 16
Image Classification	ImageNet	NFNet-F5 w/ SAM	Number of params	377.2M	# 10
Image Classification	ImageNet	NFNet-F5	Top 1 Accuracy	86.0%	# 18
Image Classification	ImageNet	NFNet-F5	Top 5 Accuracy	97.6%	# 12
Image Classification	ImageNet	NFNet-F0	Top 1 Accuracy	83.6%	# 40
Image Classification	ImageNet	NFNet-F0	Top 5 Accuracy	96.8%	# 20
Image Classification	ImageNet	NFNet-F0	Number of params	71.5M	# 33
Image Classification	ImageNet	NFNet-F4	Top 1 Accuracy	85.9%	# 19
Image Classification	ImageNet	NFNet-F4	Number of params	316.1M	# 11
Image Classification	ImageNet	NFNet-F1	Top 1 Accuracy	84.7%	# 30
Image Classification	ImageNet	NFNet-F1	Top 5 Accuracy	97.1%	# 17
Image Classification	ImageNet	NFNet-F1	Number of params	132.6M	# 19
Image Classification	ImageNet	NFNet-F2	Top 1 Accuracy	85.1%	# 27
Image Classification	ImageNet	NFNet-F2	Top 5 Accuracy	97.3%	# 15
Image Classification	ImageNet	NFNet-F2	Number of params	193.8M	# 15
Image Classification	ImageNet	NFNet-F3	Top 1 Accuracy	85.7%	# 21
Image Classification	ImageNet	NFNet-F3	Top 5 Accuracy	97.5%	# 13
Image Classification	ImageNet	NFNet-F3	Number of params	254.9M	# 13
Image Classification	ImageNet		Top 1 Accuracy	83.6%	# 40

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/high-performance-large-scale-image/image-classification-on-imagenet)](https://paperswithcode.com/sota/image-classification-on-imagenet?p=high-performance-large-scale-image)`

Batch normalization is a key component of most image classification models, but it has many undesirable properties stemming from its dependence on the batch size and interactions between examples. Although recent work has succeeded in training deep ResNets without normalization layers, these models do not match the test accuracies of the best batch-normalized networks, and are often unstable for large learning rates or strong data augmentations... (read more)

PDF Abstract

Results from the Paper

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK	BENCHMARK
Image Classification	ImageNet	NFNet-F6 w/ SAM	Top 1 Accuracy	86.5%	# 14	Compare
			Top 5 Accuracy	97.9%	# 8	Compare
			Number of params	438.4M	# 8	Compare
Image Classification	ImageNet	NFNet-F4+	Top 1 Accuracy	89.2%	# 3	Compare
Image Classification	ImageNet	NFNet-F4+	Number of params	527M	# 4	Compare
Image Classification	ImageNet	NFNet-F5 w/ SAM	Top 1 Accuracy	86.3%	# 16	Compare
Image Classification	ImageNet	NFNet-F5 w/ SAM	Number of params	377.2M	# 10	Compare
Image Classification	ImageNet	NFNet-F5	Top 1 Accuracy	86.0%	# 18	Compare
Image Classification	ImageNet	NFNet-F5	Top 5 Accuracy	97.6%	# 12	Compare
Image Classification	ImageNet	NFNet-F0	Top 1 Accuracy	83.6%	# 40	Compare
			Top 5 Accuracy	96.8%	# 20	Compare
			Number of params	71.5M	# 33	Compare
Image Classification	ImageNet	NFNet-F4	Top 1 Accuracy	85.9%	# 19	Compare
Image Classification	ImageNet	NFNet-F4	Number of params	316.1M	# 11	Compare
Image Classification	ImageNet	NFNet-F1	Top 1 Accuracy	84.7%	# 30	Compare
			Top 5 Accuracy	97.1%	# 17	Compare
			Number of params	132.6M	# 19	Compare
Image Classification	ImageNet	NFNet-F2	Top 1 Accuracy	85.1%	# 27	Compare
			Top 5 Accuracy	97.3%	# 15	Compare
			Number of params	193.8M	# 15	Compare
Image Classification	ImageNet	NFNet-F3	Top 1 Accuracy	85.7%	# 21	Compare
			Top 5 Accuracy	97.5%	# 13	Compare
			Number of params	254.9M	# 13	Compare
Image Classification	ImageNet		Top 1 Accuracy	83.6%	# 40	Compare

METHOD	TYPE
GELU	Activation Functions
Gradient Clipping	Optimization

METHOD

TYPE

GELU

Activation Functions

Gradient Clipping

Optimization

High-Performance Large-Scale Image Recognition Without Normalization

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods used in the Paper

Add Remove

High-Performance Large-Scale Image Recognition Without Normalization

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods used in the Paper Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods used in the Paper

Add Remove