The MS COCO (Microsoft Common Objects in Context) dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images.

Splits: The first version of MS COCO dataset was released in 2014. It contains 164K images split into training (83K), validation (41K) and test (41K) sets. In 2015 additional test set of 81K images was released, including all the previous test images and 40K new images.

Based on community feedback, in 2017 the training/validation split was changed from 83K/41K to 118K/5K. The new split uses the same images and annotations. The 2017 test set is a subset of 41K images of the 2015 test set. Additionally, the 2017 release contains a new unannotated dataset of 123K images.

Annotations: The dataset has annotations for

object detection: bounding boxes and per-instance segmentation masks with 80 object categories,
captioning: natural language descriptions of the images (see MS COCO Captions),
keypoints detection: containing more than 200,000 images and 250,000 person instances labeled with keypoints (17 possible keypoints, such as left eye, nose, right hip, right ankle),
stuff image segmentation – per-pixel segmentation masks with 91 stuff categories, such as grass, wall, sky (see MS COCO Stuff),
panoptic: full scene segmentation, with 80 thing categories (such as person, bicycle, elephant) and a subset of 91 stuff categories (grass, sky, road),
dense pose: more than 39,000 images and 56,000 person instances labeled with DensePose annotations – each labeled person is annotated with an instance id and a mapping between image pixels that belong to that person body and a template 3D model. The annotations are publicly available only for training and validation images.

Source: https://cocodataset.org/

Homepage

Benchmarks

Add a new result Link an existing benchmark

TASK	DATASET VARIANT	BEST MODEL
Object Detection	COCO minival	Cascade Eff-B7 NAS-FPN
Instance Segmentation	COCO minival	Cascade Eff-B7 NAS-FPN
Object Detection	COCO test-dev	Cascade Eff-B7 NAS-FPN
Instance Segmentation	COCO test-dev	Cascade Eff-B7 NAS-FPN
Real-Time Object Detection	COCO	YOLOv4-CSP-P7
Panoptic Segmentation	COCO test-dev	REFINE
Keypoint Detection	COCO	EvoPose2D-L
Pose Estimation	COCO test-dev	HRNet-W48+DARK
Keypoint Detection	COCO test-dev	HRNet*
Visual Question Answering	COCO Visual Question Answering (VQA) real images 1.0 open ended	MCB 7 att.
Text-to-Image Generation	COCO	OP-GAN
Multi-Person Pose Estimation	COCO test-dev	HigherHRNet
Cross-Modal Retrieval	COCO 2014	ALIGN
Real-time Instance Segmentation	MSCOCO	SipMask++
Multi-Person Pose Estimation	COCO	DarkPose
Visual Question Answering	COCO Visual Question Answering (VQA) real images 1.0 multiple choice	MCB 7 att.
Few-Shot Object Detection	MS-COCO (10-shot)	FSDetView
Panoptic Segmentation	COCO panoptic	MaX-DeepLab-L
Object Counting	COCO count-test	ens
Multi-Label Classification	MS-COCO	TResNet-XL
Keypoint Detection	COCO test-challenge	MSPN+*
Weakly Supervised Object Detection	COCO	MSLPD
Panoptic Segmentation	COCO minival	Panoptic FCN*
Visual Question Answering	COCO Visual Question Answering (VQA) abstract images 1.0 open ended	Graph VQA
Question Generation	COCO Visual Question Answering (VQA) real images 1.0 open ended	MDN
Visual Question Answering	COCO Visual Question Answering (VQA) real images 2.0 open ended	HDU-USYD-UNCC
Visual Question Answering	COCO Visual Question Answering (VQA) abstract 1.0 multiple choice	Graph VQA
Weakly Supervised Object Detection	COCO test-dev	wetectron
Region Proposal	COCO test-dev	GA-RPN
Text-Image Retrieval	COCO (image as query)	Oscar
Multi-Person Pose Estimation	COCO minival	HRNet-W48plus
Pose Estimation	DensePose-COCO	Parsing R-CNN + ResNext101
One-Shot Object Detection	COCO	One-Shot Object Detection
Zero-Shot Object Detection	MS-COCO	Ours
Robust Object Detection	COCO	Faster R-CNN with Stylized Training Data
Image Captioning	COCO	M2 Transformer
Text-Image Retrieval	COCO	DVSA
Real-Time Object Detection	COCO minival	Mask R-CNN X-152-32x8d
Conditional Image Generation	COCO-Animals	U-Net GAN
Layout-to-Image Generation	COCO-Stuff 256x256	OC-GAN
Multi-object discovery	COCO_20k	Large-scale rOSD
Open World Object Detection	COCO 2017 (Sports, Food)	ORE
Activeness Detection	COCO test-dev	Lightweight OpenPose
Object Detection	COCO+	RepPoints + Self-adaptation
Pose Estimation	COCO minival	MSPN
Open World Object Detection	COCO 2017 (Electronic, Indoor, Kitchen, Furniture)	ORE
Weakly Supervised Object Detection	MSCOCO	CASD
Homography Estimation	COCO 2014	PFNet
Generalized Zero-Shot Object Detection	MS-COCO	Ours
One-Shot Instance Segmentation	COCO	Siamese Mask R-CNN
Unsupervised Semantic Segmentation	COCO-Stuff-3	IIC
Open World Object Detection	COCO 2017 (Outdoor, Accessories, Appliance, Truck)	ORE
Interactive Segmentation	COCO	IOG
Single-object discovery	COCO_20k	Large-scale rOSD