TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Cross-Modal Retrieval	COCO 2014	ALIGN	Image-to-text R@1	77	# 1
Cross-Modal Retrieval	COCO 2014	ALIGN	Image-to-text R@10	96.9	# 1
Cross-Modal Retrieval	COCO 2014	ALIGN	Image-to-text R@5	93.5	# 1
Cross-Modal Retrieval	COCO 2014	ALIGN	Text-to-image R@1	59.9	# 1
Cross-Modal Retrieval	COCO 2014	ALIGN	Text-to-image R@10	89.8	# 1
Cross-Modal Retrieval	COCO 2014	ALIGN	Text-to-image R@5	83.3	# 1
Cross-Modal Retrieval	Flickr30k	ALIGN	Image-to-text R@1	95.3	# 1
Cross-Modal Retrieval	Flickr30k	ALIGN	Image-to-text R@10	100	# 1
Cross-Modal Retrieval	Flickr30k	ALIGN	Image-to-text R@5	99.8	# 1
Cross-Modal Retrieval	Flickr30k	ALIGN	Text-to-image R@1	84.9	# 1
Cross-Modal Retrieval	Flickr30k	ALIGN	Text-to-image R@10	98.6	# 1
Cross-Modal Retrieval	Flickr30k	ALIGN	Text-to-image R@5	97.4	# 1
Image Classification	Flowers-102	ALIGN	Accuracy	99.65%	# 5
Fine-Grained Image Classification	Food-101	ALIGN	Accuracy	95.88	# 2
Image Classification	ImageNet	ALIGN (EfficientNet-L2)	Top 1 Accuracy	88.64%	# 4
Image Classification	ImageNet	ALIGN (EfficientNet-L2)	Top 5 Accuracy	98.67%	# 5
Image Classification	ImageNet	ALIGN (EfficientNet-L2)	Number of params	480M	# 6
Fine-Grained Image Classification	Oxford-IIIT Pets	ALIGN	Accuracy	96.19%	# 5
Fine-Grained Image Classification	Stanford Cars	ALIGN	Accuracy	96.13%	# 3
Image Classification	VTAB-1k	ALIGN (50 hypers/task)	Top-1 Accuracy	79.99	# 1

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/cross-modal-retrieval-on-coco-2014)](https://paperswithcode.com/sota/cross-modal-retrieval-on-coco-2014?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/cross-modal-retrieval-on-flickr30k)](https://paperswithcode.com/sota/cross-modal-retrieval-on-flickr30k?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/image-classification-on-vtab-1k-1)](https://paperswithcode.com/sota/image-classification-on-vtab-1k-1?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/fine-grained-image-classification-on-food-101)](https://paperswithcode.com/sota/fine-grained-image-classification-on-food-101?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/fine-grained-image-classification-on-stanford)](https://paperswithcode.com/sota/fine-grained-image-classification-on-stanford?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/image-classification-on-imagenet)](https://paperswithcode.com/sota/image-classification-on-imagenet?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/image-classification-on-flowers-102)](https://paperswithcode.com/sota/image-classification-on-flowers-102?p=scaling-up-visual-and-vision-language)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/scaling-up-visual-and-vision-language/fine-grained-image-classification-on-oxford-1)](https://paperswithcode.com/sota/fine-grained-image-classification-on-oxford-1?p=scaling-up-visual-and-vision-language)`

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

11 Feb 2021 • Chao Jia • Yinfei Yang • Ye Xia • Yi-Ting Chen • Zarana Parekh • Hieu Pham • Quoc V. Le • YunHsuan Sung • Zhen Li • Tom Duerig

Pre-trained representations are becoming crucial for many NLP and perception tasks. While representation learning in NLP has transitioned to training on raw text without human annotations, visual and vision-language representations still rely heavily on curated training datasets that are expensive or require expert knowledge... (read more)

PDF Abstract

Results from the Paper

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK	BENCHMARK
Cross-Modal Retrieval	COCO 2014	ALIGN	Image-to-text R@1	77	# 1	Compare
			Image-to-text R@10	96.9	# 1	Compare
			Image-to-text R@5	93.5	# 1	Compare
			Text-to-image R@1	59.9	# 1	Compare
			Text-to-image R@10	89.8	# 1	Compare
			Text-to-image R@5	83.3	# 1	Compare
Cross-Modal Retrieval	Flickr30k	ALIGN	Image-to-text R@1	95.3	# 1	Compare
			Image-to-text R@10	100	# 1	Compare
			Image-to-text R@5	99.8	# 1	Compare
			Text-to-image R@1	84.9	# 1	Compare
			Text-to-image R@10	98.6	# 1	Compare
			Text-to-image R@5	97.4	# 1	Compare
Image Classification	Flowers-102	ALIGN	Accuracy	99.65%	# 5	Compare
Fine-Grained Image Classification	Food-101	ALIGN	Accuracy	95.88	# 2	Compare
Image Classification	ImageNet	ALIGN (EfficientNet-L2)	Top 1 Accuracy	88.64%	# 4	Compare
			Top 5 Accuracy	98.67%	# 5	Compare
			Number of params	480M	# 6	Compare
Fine-Grained Image Classification	Oxford-IIIT Pets	ALIGN	Accuracy	96.19%	# 5	Compare
Fine-Grained Image Classification	Stanford Cars	ALIGN	Accuracy	96.13%	# 3	Compare
Image Classification	VTAB-1k	ALIGN (50 hypers/task)	Top-1 Accuracy	79.99	# 1	Compare

METHOD	TYPE
🤖 No Methods Found	Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet

METHOD

TYPE

🤖 No Methods Found

Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods used in the Paper

Add Remove

Edit Social Preview

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Edit social preview

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods used in the Paper Edit Add Remove

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods used in the Paper

Add Remove