COCO (Microsoft Common Objects in Context)
Introduced by Lin et al. in Microsoft COCO: Common Objects in Context
The MS COCO (Microsoft Common Objects in Context) dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images.
Splits: The first version of MS COCO dataset was released in 2014. It contains 164K images split into training (83K), validation (41K) and test (41K) sets. In 2015 additional test set of 81K images was released, including all the previous test images and 40K new images.
Based on community feedback, in 2017 the training/validation split was changed from 83K/41K to 118K/5K. The new split uses the same images and annotations. The 2017 test set is a subset of 41K images of the 2015 test set. Additionally, the 2017 release contains a new unannotated dataset of 123K images.
Annotations: The dataset has annotations for
- object detection: bounding boxes and per-instance segmentation masks with 80 object categories,
- captioning: natural language descriptions of the images (see MS COCO Captions),
- keypoints detection: containing more than 200,000 images and 250,000 person instances labeled with keypoints (17 possible keypoints, such as left eye, nose, right hip, right ankle),
- stuff image segmentation – per-pixel segmentation masks with 91 stuff categories, such as grass, wall, sky (see MS COCO Stuff),
- panoptic: full scene segmentation, with 80 thing categories (such as person, bicycle, elephant) and a subset of 91 stuff categories (grass, sky, road),
- dense pose: more than 39,000 images and 56,000 person instances labeled with DensePose annotations – each labeled person is annotated with an instance id and a mapping between image pixels that belong to that person body and a template 3D model. The annotations are publicly available only for training and validation images.
Benchmarks
| Trend | Task | Dataset Variant | Best Model | Paper | Code |
|---|---|---|---|---|---|
|
COCO minival
|
DINO
|
|||
|
COCO test-dev
|
DINO
|
|||
|
COCO minival
|
SwinV2-G
|
|||
|
COCO test-dev
|
SwinV2-G
|
|||
|
COCO
|
Imagen
|
|||
|
COCO test-dev
|
Mask2Former
|
|||
|
COCO test-dev
|
ViTPose
|
|||
|
MS-COCO (10-shot)
|
imTED+ViT-B
|
|||
|
COCO
|
4xRSN-50
|
|||
|
MS-COCO
|
ML-Decoder
|
|||
|
COCO minival
|
Mask2Former
|
|||
|
COCO 2014
|
X-VLM
|
|||
|
COCO test-dev
|
HRNet*
|
|||
|
COCO
|
OmniPose
|
|||
|
COCO test-dev
|
HRNet-W48plus
|
|||
|
COCO Visual Question Answering (VQA) real images 1.0 open ended
|
MCB 7 att.
|
|||
|
MSCOCO
|
SparseInst-608
|
|||
|
COCO Visual Question Answering (VQA) real images 1.0 multiple choice
|
MCB 7 att.
|
|||
|
COCO 2014
|
TCL
|
|||
|
COCO
|
YOLOR‑D6
|
|||
|
COCO_20k
|
TokenCut + CAD
|
|||
|
COCO test-challenge
|
4×RSN-50
|
|||
|
COCO count-test
|
ens
|
|||
|
COCO 256 x 256
|
XMC-GAN
|
|||
|
COCO test-dev
|
DiscoBox
|
|||
|
MSCOCO
|
PP-PicoDet-L
|
|||
|
COCO
|
MSLPD
|
|||
|
COCO test-dev
|
wetectron
|
|||
|
COCO Visual Question Answering (VQA) abstract images 1.0 open ended
|
Graph VQA
|
|||
|
COCO Visual Question Answering (VQA) abstract 1.0 multiple choice
|
Graph VQA
|
|||
|
COCO Visual Question Answering (VQA) real images 2.0 open ended
|
HDU-USYD-UNCC
|
|||
|
COCO Visual Question Answering (VQA) real images 1.0 open ended
|
MDN
|
|||
|
COCO
|
M2 Transformer
|
|||
|
COCO test-dev
|
GA-RPN
|
|||
|
COCO minival
|
HRNet-W48plus
|
|||
|
COCO (image as query)
|
Oscar
|
|||
|
COCO test-dev
|
LIID
|
|||
|
COCO minival
|
Mask R-CNN X-152-32x8d
|
|||
|
COCO
|
Faster R-CNN with Stylized Training Data
|
|||
|
COCO
|
Oscar
|
|||
|
COCO
|
One-Shot Object Detection
|
|||
|
COCO-Animals
|
U-Net GAN
|
|||
|
COCO-Stuff 256x256
|
OC-GAN
|
|||
|
COCO
|
FLAVA
|
|||
|
COCO 2017 (Outdoor, Accessories, Appliance, Truck)
|
ORE
|
|||
|
COCO 2017 (Sports, Food)
|
ORE
|
|||
|
COCO 2017 (Electronic, Indoor, Kitchen, Furniture)
|
ORE
|
|||
|
COCO-Stuff-3
|
IIC
|
|||
|
COCO 2017
|
UM-MAE（HTC++, Swin-L, IN1K）
|
|||
|
DensePose-COCO
|
Parsing R-CNN + ResNext101
|
|||
|
COCO
|
LSHFM
|
|||
|
COCO
|
FLAVA
|
|||
|
COCO
|
MDef-DETR
|
|||
|
COCO test-dev
|
BESTIE
|
|||
|
COCO minival
|
MSPN
|
|||
|
COCO
|
RetinaNet
|
|||
|
coco minval
|
R3-CNN
|
|||
|
MSCOCO
|
HRQ-VAE
|
|||
|
COCO
|
Perturb, Predict & Paraphrase
|
|||
|
COCO 2014
|
PFNet
|
|||
|
COCO
|
IOG
|
|||
|
COCO 2017
|
DETReg
|
|||
|
COCO_20k
|
Large-scale rOSD
|
|||
|
MS-COCO
|
ZSD-Polarity Loss
|
|||
|
COCO test-dev
|
Lightweight OpenPose
|
|||
|
COCO+
|
RepPoints + Self-adaptation
|
|||
|
COCO
|
Siamese Mask R-CNN
|
|||
|
COCO
|
PEFORMER-Xcit-dino-p8
|
|||
|
MSCOCO
|
CASD
|
|||
|
COCO
|
MDef-DETR
|
|||
|
MS-COCO
|
Ours
|
Papers
| Paper | Code | Results | Date | Stars |
|---|
Dataset Loaders
Tasks
-
Pose Estimation
-
Keypoint Detection
-
Object Detection
-
Instance Segmentation
-
Visual Question Answering
-
Image Captioning
-
Multi-Label Classification
-
Object Localization
-
Weakly Supervised Object Detection
-
Unsupervised Semantic Segmentation
-
Interactive Segmentation
-
Multi-Person Pose Estimation
-
Panoptic Segmentation
-
Text-Image Retrieval
-
Text-to-Image Generation
-
Question Generation
-
Conditional Image Generation
-
Real-Time Object Detection
-
Object Counting
-
Few-Shot Object Detection
-
Layout-to-Image Generation
-
Cross-Modal Retrieval
-
Single-object discovery
-
Class-agnostic Object Detection
-
Object Proposal Generation
-
Homography Estimation
-
Real-time Instance Segmentation
-
Open World Object Detection
-
Paraphrase Generation
-
Zero-Shot Object Detection
-
Weakly-supervised instance segmentation
-
Robust Object Detection
-
Multi-object discovery
-
Knowledge Distillation
-
Image-level Supervised Instance Segmentation
-
Point-Supervised Instance Segmentation
-
One-Shot Object Detection
-
Active Object Detection
-
Zero-Shot Cross-Modal Retrieval
-
One-Shot Instance Segmentation
-
Region Proposal
-
Activeness Detection
-
Generalized Zero-Shot Object Detection
-
Semi Supervised Learning for Image Captioning
-
Zero-shot Text Retrieval
-
Zero-shot Image Retrieval
-
Zero-Shot Text-to-Image Generation
Similar Datasets
-
Pose Estimation
-
Keypoint Detection
-
Object Detection
-
Instance Segmentation
-
Visual Question Answering
-
Image Captioning
-
Multi-Label Classification
-
Object Localization
-
Weakly Supervised Object Detection
-
Unsupervised Semantic Segmentation
-
Interactive Segmentation
-
Multi-Person Pose Estimation
-
Panoptic Segmentation
-
Text-Image Retrieval
-
Text-to-Image Generation
-
Question Generation
-
Conditional Image Generation
-
Real-Time Object Detection
-
Object Counting
-
Few-Shot Object Detection
-
Layout-to-Image Generation
-
Cross-Modal Retrieval
-
Single-object discovery
-
Class-agnostic Object Detection
-
Object Proposal Generation
-
Homography Estimation
-
Real-time Instance Segmentation
-
Open World Object Detection
-
Paraphrase Generation
-
Zero-Shot Object Detection
-
Weakly-supervised instance segmentation
-
Robust Object Detection
-
Multi-object discovery
-
Knowledge Distillation
-
Image-level Supervised Instance Segmentation
-
Point-Supervised Instance Segmentation
-
One-Shot Object Detection
-
Active Object Detection
-
Zero-Shot Cross-Modal Retrieval
-
One-Shot Instance Segmentation
-
Region Proposal
-
Activeness Detection
-
Generalized Zero-Shot Object Detection
-
Semi Supervised Learning for Image Captioning
-
Zero-shot Text Retrieval
-
Zero-shot Image Retrieval
-
Zero-Shot Text-to-Image Generation
