MS COCO (Microsoft Common Objects in Context)

Introduced by Lin et al. in Microsoft COCO: Common Objects in Context

The MS COCO (Microsoft Common Objects in Context) dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images.

Splits: The first version of MS COCO dataset was released in 2014. It contains 164K images split into training (83K), validation (41K) and test (41K) sets. In 2015 additional test set of 81K images was released, including all the previous test images and 40K new images.

Based on community feedback, in 2017 the training/validation split was changed from 83K/41K to 118K/5K. The new split uses the same images and annotations. The 2017 test set is a subset of 41K images of the 2015 test set. Additionally, the 2017 release contains a new unannotated dataset of 123K images.

Task	Dataset Variant	Best Model
Object Detection	COCO minival	PE_spatial
Instance Segmentation	COCO minival	Co-DETR
Object Detection	COCO test-dev	Co-DETR
Instance Segmentation	COCO test-dev	Co-DETR
Real-Time Object Detection	MS COCO	DEIM-D-FINE-X+
Text-to-Image Generation	MS COCO	Parti Finetuned
Pose Estimation	COCO test-dev	ViTPose
Panoptic Segmentation	COCO test-dev	Mask DINO
Cross-Modal Retrieval	COCO 2014	VAST
Panoptic Segmentation	COCO minival	HyperSeg
Multi-Label Classification	MS-COCO	ADDS
Open Vocabulary Object Detection	MSCOCO	Cooperative Foundational Models
Few-Shot Object Detection	MS-COCO (10-shot)	CD-ViTO
Keypoint Detection	MS COCO	4xRSN-50
Object Detection	COCO 2017	CP-DETR-L Swin-L
Real-time Instance Segmentation	MSCOCO	RTMDet-Ins-x
Zero-Shot Cross-Modal Retrieval	COCO 2014	InternVL-G
Image Captioning	MS COCO	ExpansionNet v2
Keypoint Detection	COCO test-dev	HRNet*
Multi-Person Pose Estimation	COCO test-dev	SCIO
Multi-Person Pose Estimation	MS COCO	RSN
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 1.0 open ended	SAN
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 1.0 multiple choice	MCB 7 att.
Pose Estimation	MS COCO	OmniPose
Single-object discovery	COCO_20k	IMST
Zero-Shot Composed Image Retrieval (ZS-CIR)	MS COCO	iSEARLE-XL-OTI
Zero-shot Text-to-Image Retrieval	MS COCO	BLIP-2
Zero-Shot Object Detection	MS-COCO	UniFa
Semantic Segmentation	MS COCO	HyperSeg
Image-to-Text Retrieval	MS COCO	BLIP-2
Keypoint Detection	COCO test-challenge	4×RSN-50
Generalized Zero-Shot Object Detection	MS-COCO	SeeDS
Box-supervised Instance Segmentation	COCO test-dev	Box2Mask-T
Object Counting	COCO count-test	ens
Weakly-supervised instance segmentation	COCO test-dev	DiscoBox
Object Detection	MSCOCO	YOLOv5s
Image-level Supervised Instance Segmentation	COCO test-dev	WeakSAM-Mask2Former
Zero-Shot Object Detection	MSCOCO	Grounding DINO 1.6 Pro
Unsupervised Semantic Segmentation	COCO-Stuff-3	IIC
Image Retrieval	MS COCO	BLIP-2 ViT-G
Layout-to-Image Generation	COCO-Stuff 256x256	LayoutDiffusion
Weakly Supervised Object Detection	MS COCO	MSLPD
Question Generation	COCO Visual Question Answering (VQA) real images 1.0 open ended	MDN
Multi-Label Image Classification	MSCOCO	IDA-R101
One-Shot Object Detection	MS COCO	Siamese Mask R-CNN
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 2.0 open ended	HDU-USYD-UNCC
Multi-Person Pose Estimation	COCO minival	HRNet-W32
Region Proposal	COCO test-dev	RPN+Focal Loss
Knowledge Distillation	MS COCO	ADLIK-Faster
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) abstract images 1.0 open ended	Graph VQA
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) abstract 1.0 multiple choice	Graph VQA
Weakly Supervised Object Detection	COCO test-dev	wetectron
Object Detection	MS COCO	MOAT-3 22K+1K
Image Retrieval	MSCOCO	HADA
mage-to-Text Retrieval	MSCOCO	BLIP
Open World Object Detection	COCO 2017 (Outdoor, Accessories, Appliance, Truck)	ORE
Robust Object Detection	MS COCO	Faster R-CNN with Stylized Training Data
Conditional Image Generation	COCO-Animals	U-Net GAN
Open World Object Detection	COCO 2017 (Sports, Food)	ORE
Open World Object Detection	COCO 2017 (Electronic, Indoor, Kitchen, Furniture)	ORE
Text-to-Image Generation	MS-COCO	AttnGAN
Pose Estimation	DensePose-COCO	Parsing R-CNN + ResNext101
Cross-Modal Retrieval	MSCOCO-1k	NAPReg
Panoptic Segmentation	COCO panoptic	VAN-B6*
Paraphrase Generation	MSCOCO	HRQ-VAE
Active Object Detection	MS COCO	RetinaNet
Point-Supervised Instance Segmentation	COCO test-dev	BESTIE
Real-time Instance Segmentation	MSCOCO-1k	RTMDet-Ins-x
Interactive Segmentation	MS COCO	IOG
Quantization	MS COCO	SSD ResNet50 V1 FPN 640x640
One-Shot Instance Segmentation	MS COCO	Siamese Mask R-CNN
Homography Estimation	COCO 2014	PFNet
Unsupervised Semantic Segmentation with Language-image Pre-training	MS COCO	CLIPpy ViT-B
Weakly Supervised Object Detection	MSCOCO	CASD
Object Detection	COCO+	RepPoints + Self-adaptation
Activeness Detection	COCO test-dev	Lightweight OpenPose
Unsupervised Object Localization	COCO_20k	DeepCut
Image Outpainting	MSCOCO	NUWA-3D
Image Captioning	MSCOCO	CapDec
Cross-Modal Retrieval	MSCOCO	3SHNet
Visual Question Answering	COCO Visual Question Answering (VQA) real images 2.0 open ended	MaMMUT
Question Answering	COCO Visual Question Answering (VQA) real images 1.0 open ended	MaMMUT
Multi-Label Learning	COCO 2014	SADCL
Multi-object discovery	COCO_20k	Large-scale rOSD
Visual Question Answering	MS COCO	BenchLMM
Few Shot Open Set Object Detection	MSCOCO	FOODv2
Object Detection	MSCOCO	DAS
Interactive Segmentation	COCO minival	ViT-B+MST+CL
Scene Graph Generation	MS-COCO	NeuSyRE
Image Captioning	MS-COCO	NeuSyRE
Pose Estimation	MS-COCO	UniHCP
Pose Estimation	COCO minival	MSPN
Few-Shot Object Detection	COCO 2017	DETReg
Semi Supervised Learning for Image Captioning	MS COCO	Perturb, Predict & Paraphrase
Object Proposal Generation	MS COCO	MDef-DETR
Instance Segmentation	coco minval	R3-CNN

MS COCO (Microsoft Common Objects in Context)

Benchmarks Edit Add a new result Link an existing benchmark

Papers

Dataset Loaders Edit Add Remove

Tasks Edit

Similar Datasets

ADE20K

LVIS

RefCOCO

Flickr30k

Usage

License Edit

Modalities Edit

Languages Edit

Benchmarks

Add a new result Link an existing benchmark

Dataset Loaders

Add Remove

Tasks

License

Modalities

Languages