COCO (Microsoft Common Objects in Context)

Introduced by Lin et al. in Microsoft COCO: Common Objects in Context

The MS COCO (Microsoft Common Objects in Context) dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images.

Splits: The first version of MS COCO dataset was released in 2014. It contains 164K images split into training (83K), validation (41K) and test (41K) sets. In 2015 additional test set of 81K images was released, including all the previous test images and 40K new images.

Based on community feedback, in 2017 the training/validation split was changed from 83K/41K to 118K/5K. The new split uses the same images and annotations. The 2017 test set is a subset of 41K images of the 2015 test set. Additionally, the 2017 release contains a new unannotated dataset of 123K images.

Task	Dataset Variant	Best Model
Object Detection	COCO minival	Co-DETR
Object Detection	COCO test-dev	Co-DETR
Instance Segmentation	COCO minival	InternImage-H
Instance Segmentation	COCO test-dev	InternImage-H
Text-to-Image Generation	COCO	Parti Finetuned
Pose Estimation	COCO test-dev	ViTPose
Panoptic Segmentation	COCO test-dev	Mask DINO
Cross-Modal Retrieval	COCO 2014	OURS-COMBINED-VAL
Multi-Label Classification	MS-COCO	ADDS
Panoptic Segmentation	COCO minival	OneFormer
Few-Shot Object Detection	MS-COCO (10-shot)	DETReg-ft-full DDETR
Real-Time Object Detection	COCO	YOLOv6-L6
Keypoint Detection	COCO	4xRSN-50
Real-time Instance Segmentation	MSCOCO	RTMDet-Ins-x
Open Vocabulary Object Detection	MSCOCO	CORA+
Keypoint Detection	COCO test-dev	HRNet*
Multi-Person Pose Estimation	COCO	RSN
Multi-Person Pose Estimation	COCO test-dev	SCIO
Image Captioning	COCO	ExpansionNet v2
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 1.0 open ended	MCB 7 att.
Zero-Shot Cross-Modal Retrieval	COCO 2014	TCL
Single-object discovery	COCO_20k	IMST
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 1.0 multiple choice	MCB 7 att.
Object Detection	COCO 2017	MaxViT-B
Image-to-Text Retrieval	COCO	BLIP-2 ViT-G
Keypoint Detection	COCO test-challenge	4×RSN-50
Pose Estimation	COCO	OmniPose
Weakly-supervised instance segmentation	COCO test-dev	DiscoBox
Zero-Shot Object Detection	MS-COCO	ZSD-SCR
Generalized Zero-Shot Object Detection	MS-COCO	ZSD-SCR
Object Detection	MSCOCO	PP-PicoDet-L
Box-supervised Instance Segmentation	COCO test-dev	Box2Mask-T
Object Counting	COCO count-test	ens
Image Retrieval	COCO	BLIP-2 ViT-G
Unsupervised Semantic Segmentation	COCO-Stuff-3	SAN
Semantic Segmentation	COCO	OneFormer
Image-level Supervised Instance Segmentation	COCO test-dev	CIM + Mask R-CNN
Layout-to-Image Generation	COCO-Stuff 256x256	LayoutDiffusion
Weakly Supervised Object Detection	COCO	MSLPD
Region Proposal	COCO test-dev	GA-RPN
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) abstract images 1.0 open ended	Graph VQA
Question Generation	COCO Visual Question Answering (VQA) real images 1.0 open ended	MDN
Multi-Person Pose Estimation	COCO minival	HRNet-W48plus
Weakly Supervised Object Detection	COCO test-dev	wetectron
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) abstract 1.0 multiple choice	Graph VQA
Knowledge Distillation	COCO	ADLIK-Faster
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 2.0 open ended	HDU-USYD-UNCC
Multi-Label Image Classification	MSCOCO	IDA-R101
One-Shot Object Detection	COCO	OWL-ViT
Object Detection	COCO	MOAT-3
Open World Object Detection	COCO 2017 (Sports, Food)	ORE
Pose Estimation	DensePose-COCO	SwinV2-L 1K-MIM
Robust Object Detection	COCO	Faster R-CNN with Stylized Training Data
Conditional Image Generation	COCO-Animals	U-Net GAN
Open World Object Detection	COCO 2017 (Outdoor, Accessories, Appliance, Truck)	ORE
Open World Object Detection	COCO 2017 (Electronic, Indoor, Kitchen, Furniture)	ORE
Zero-Shot Object Detection	MSCOCO	Grounding DINO
object-detection	COCO	darknet-coco-object_detection
Few-Shot Object Detection	COCO 2017	DETReg
Active Object Detection	COCO	RetinaNet
Weakly Supervised Object Detection	MSCOCO	CASD
Class-agnostic Object Detection	COCO	MDef-DETR
Object Proposal Generation	COCO	MDef-DETR
Object Detection	COCO+	RepPoints + Self-adaptation
Paraphrase Generation	MSCOCO	HRQ-VAE
Activeness Detection	COCO test-dev	Lightweight OpenPose
Point-Supervised Instance Segmentation	COCO test-dev	BESTIE
Question Answering	COCO Visual Question Answering (VQA) real images 1.0 open ended	MaMMUT
Multi-object discovery	COCO_20k	Large-scale rOSD
Real-Time Object Detection	COCO test-dev	YOLOR-P6
Multi-Label Learning	COCO 2014	SADCL
Quantization	COCO	SSD ResNet50 V1 FPN 640x640
Interactive Segmentation	COCO	IOG
Pose Estimation	COCO minival	MSPN
Unsupervised Semantic Segmentation with Language-image Pre-training	COCO	CLIPpy ViT-B
Panoptic Segmentation	COCO panoptic	VAN-B6*
Homography Estimation	COCO 2014	PFNet
One-Shot Instance Segmentation	COCO	Siamese Mask R-CNN
Unsupervised Object Localization	COCO_20k	DeepCut
Image Outpainting	MSCOCO	NUWA-3D
Image Captioning	MSCOCO	CapDec
Cross-Modal Retrieval	MSCOCO-1k	NAPReg
Visual Question Answering	COCO Visual Question Answering (VQA) real images 2.0 open ended	MaMMUT
Instance Segmentation	coco minval	R3-CNN
Semi Supervised Learning for Image Captioning	COCO	Perturb, Predict & Paraphrase

COCO (Microsoft Common Objects in Context)

Benchmarks Edit Add a new result Link an existing benchmark

Papers

Dataset Loaders Edit Add Remove

Tasks Edit

Similar Datasets

ADE20K

LVIS

RefCOCO

Flickr30k

Usage

License Edit

Modalities Edit

Languages Edit

Benchmarks

Add a new result Link an existing benchmark

Dataset Loaders

Add Remove

Tasks

License

Modalities

Languages