TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Fine-Grained Image Classification	Birdsnap	GPIPE	Accuracy	83.6%	# 3
Image Classification	CIFAR-10	GPIPE + transfer learning	Percentage correct	99	# 7
Image Classification	CIFAR-10	GPIPE + transfer learning	Percentage error	1	# 6
Image Classification	CIFAR-100	GPIPE	Percentage correct	91.3	# 6
Image Classification	ImageNet	GPIPE	Top 1 Accuracy	84.4%	# 32
Image Classification	ImageNet	GPIPE	Top 5 Accuracy	97%	# 18
Fine-Grained Image Classification	Stanford Cars	GPipe	Accuracy	94.6%	# 12

🦡 Badges

Include the markdown at the top of your GitHub README.md file to showcase the performance of the model.

Badges are live and will be dynamically updated with the latest ranking of this paper.

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/gpipe-efficient-training-of-giant-neural/fine-grained-image-classification-on-birdsnap)](https://paperswithcode.com/sota/fine-grained-image-classification-on-birdsnap?p=gpipe-efficient-training-of-giant-neural)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/gpipe-efficient-training-of-giant-neural/image-classification-on-cifar-100)](https://paperswithcode.com/sota/image-classification-on-cifar-100?p=gpipe-efficient-training-of-giant-neural)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/gpipe-efficient-training-of-giant-neural/image-classification-on-cifar-10)](https://paperswithcode.com/sota/image-classification-on-cifar-10?p=gpipe-efficient-training-of-giant-neural)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/gpipe-efficient-training-of-giant-neural/fine-grained-image-classification-on-stanford)](https://paperswithcode.com/sota/fine-grained-image-classification-on-stanford?p=gpipe-efficient-training-of-giant-neural)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/gpipe-efficient-training-of-giant-neural/image-classification-on-imagenet)](https://paperswithcode.com/sota/image-classification-on-imagenet?p=gpipe-efficient-training-of-giant-neural)`

GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

NeurIPS 2019 • Yanping Huang • Youlong Cheng • Ankur Bapna • Orhan Firat • Mia Xu Chen • Dehao Chen • HyoukJoong Lee • Jiquan Ngiam • Quoc V. Le • Yonghui Wu • Zhifeng Chen

Scaling up deep neural network capacity has been known as an effective approach to improving model quality for several different machine learning tasks. In many cases, increasing model capacity beyond the memory limit of a single accelerator has required developing special algorithms or infrastructure... (read more)

PDF Abstract NeurIPS 2019 PDF NeurIPS 2019 Abstract

Code

Add Remove Mark official

tensorflow/lingvo

↳ Quickstart in

Colab

2,200

qubvel/efficientnet

1,722

KakaoBrain/torchgpipe

542

alondj/Pytorch-Gpipe

pikkaay/efficientnet_gpu

See all 7 implementations

Tasks

Add Remove

FINE-GRAINED IMAGE CLASSIFICATION

IMAGE CLASSIFICATION

MACHINE TRANSLATION

Datasets

ImageNet

Results from the Paper

Edit

Ranked #3 on Fine-Grained Image Classification on Birdsnap (using extra training data)

Get a GitHub badge

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK	BENCHMARK
Fine-Grained Image Classification	Birdsnap	GPIPE	Accuracy	83.6%	# 3	Compare
Image Classification	CIFAR-10	GPIPE + transfer learning	Percentage correct	99	# 7	Compare
Image Classification	CIFAR-10	GPIPE + transfer learning	Percentage error	1	# 6	Compare
Image Classification	CIFAR-100	GPIPE	Percentage correct	91.3	# 6	Compare
Image Classification	ImageNet	GPIPE	Top 1 Accuracy	84.4%	# 32	Compare
Image Classification	ImageNet	GPIPE	Top 5 Accuracy	97%	# 18	Compare
Fine-Grained Image Classification	Stanford Cars	GPipe	Accuracy	94.6%	# 12	Compare

Methods used in the Paper

Add Remove

METHOD	TYPE
Spatially Separable Convolution	Convolutions
Max Pooling	Pooling Operations
Convolution	Convolutions
Average Pooling	Pooling Operations
AmoebaNet	Convolutional Neural Networks
Residual Connection	Skip Connections
BPE	Subword Segmentation
Dense Connections	Feedforward Networks
Label Smoothing	Regularization
ReLU	Activation Functions
Adam	Stochastic Optimization
Softmax	Output Functions
Dropout	Regularization
Multi-Head Attention	Attention Modules
Layer Normalization	Normalization
Scaled Dot-Product Attention	Attention Mechanisms
Transformer	Transformers

GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods used in the Paper Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods used in the Paper

Add Remove