MS COCO (Microsoft Common Objects in Context)

Introduced by Lin et al. in Microsoft COCO: Common Objects in Context

The MS COCO (Microsoft Common Objects in Context) dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images.

Splits: The first version of MS COCO dataset was released in 2014. It contains 164K images split into training (83K), validation (41K) and test (41K) sets. In 2015 additional test set of 81K images was released, including all the previous test images and 40K new images.

Based on community feedback, in 2017 the training/validation split was changed from 83K/41K to 118K/5K. The new split uses the same images and annotations. The 2017 test set is a subset of 41K images of the 2015 test set. Additionally, the 2017 release contains a new unannotated dataset of 123K images.

Task	Dataset Variant	Best Model
Object Detection	COCO minival	Co-DETR
Object Detection	COCO test-dev	Co-DETR
Instance Segmentation	COCO minival	InternImage-H
Instance Segmentation	COCO test-dev	EVA
Text-to-Image Generation	MS COCO	Parti Finetuned
Pose Estimation	COCO test-dev	ViTPose
Real-Time Object Detection	MS COCO	YOLOv6-L6
Panoptic Segmentation	COCO test-dev	Mask DINO
Cross-Modal Retrieval	COCO 2014	OURS-COMBINED-VAL
Multi-Label Classification	MS-COCO	ADDS
Panoptic Segmentation	COCO minival	OneFormer
Few-Shot Object Detection	MS-COCO (10-shot)	DE-ViT
Open Vocabulary Object Detection	MSCOCO	Cooperative Foundational Models
Real-time Instance Segmentation	MSCOCO	RTMDet-Ins-x
Keypoint Detection	MS COCO	4xRSN-50
Object Detection	COCO 2017	MaxViT-B
Image Captioning	MS COCO	ExpansionNet v2
Keypoint Detection	COCO test-dev	HRNet*
Multi-Person Pose Estimation	MS COCO	RSN
Zero-Shot Cross-Modal Retrieval	COCO 2014	InternVL-G
Multi-Person Pose Estimation	COCO test-dev	SCIO
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 1.0 open ended	SAN
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 1.0 multiple choice	MCB 7 att.
Single-object discovery	COCO_20k	IMST
Pose Estimation	MS COCO	OmniPose
Image-to-Text Retrieval	MS COCO	BLIP-2 ViT-G
Keypoint Detection	COCO test-challenge	Simple Base+*
Zero-Shot Object Detection	MS-COCO	SeeDS
Generalized Zero-Shot Object Detection	MS-COCO	SeeDS
Weakly-supervised instance segmentation	COCO test-dev	DiscoBox
Box-supervised Instance Segmentation	COCO test-dev	Box2Mask-T
Object Detection	MSCOCO	YOLOv5s
Object Counting	COCO count-test	ens
Image Retrieval	MS COCO	BLIP-2 ViT-G
Semantic Segmentation	MS COCO	OneFormer
Unsupervised Semantic Segmentation	COCO-Stuff-3	IIC
Image-level Supervised Instance Segmentation	COCO test-dev	CIM + Mask R-CNN
Layout-to-Image Generation	COCO-Stuff 256x256	LayoutDiffusion
Weakly Supervised Object Detection	MS COCO	MSLPD
Region Proposal	COCO test-dev	RPN+Focal Loss
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) abstract images 1.0 open ended	Graph VQA
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) abstract 1.0 multiple choice	Graph VQA
Visual Question Answering (VQA)	COCO Visual Question Answering (VQA) real images 2.0 open ended	HDU-USYD-UNCC
One-Shot Object Detection	MS COCO	Siamese Mask R-CNN
Question Generation	COCO Visual Question Answering (VQA) real images 1.0 open ended	MDN
Weakly Supervised Object Detection	COCO test-dev	wetectron
Multi-Person Pose Estimation	COCO minival	HRNet-W32
Knowledge Distillation	MS COCO	ADLIK-Faster
Multi-Label Image Classification	MSCOCO	IDA-R101
Object Detection	MS COCO	MOAT-2
Open World Object Detection	COCO 2017 (Sports, Food)	ORE
Text-to-Image Generation	MS-COCO	AttnGAN
Zero-Shot Composed Image Retrieval (ZS-CIR)	MS COCO	Context-I2W
Pose Estimation	DensePose-COCO	Parsing R-CNN + ResNext101
Zero-Shot Object Detection	MSCOCO	Grounding DINO
Robust Object Detection	MS COCO	Faster R-CNN with Stylized Training Data
Open World Object Detection	COCO 2017 (Electronic, Indoor, Kitchen, Furniture)	ORE
Cross-Modal Retrieval	MSCOCO-1k	NAPReg
Open World Object Detection	COCO 2017 (Outdoor, Accessories, Appliance, Truck)	ORE
Conditional Image Generation	COCO-Animals	U-Net GAN
Image Captioning	MS-COCO	NeuSyRE
Paraphrase Generation	MSCOCO	HRQ-VAE
Panoptic Segmentation	COCO panoptic	VAN-B6*
Point-Supervised Instance Segmentation	COCO test-dev	BESTIE
Quantization	MS COCO	SSD ResNet50 V1 FPN 640x640
Homography Estimation	COCO 2014	PFNet
One-Shot Instance Segmentation	MS COCO	Siamese Mask R-CNN
Unsupervised Semantic Segmentation with Language-image Pre-training	MS COCO	CLIPpy ViT-B
Pose Estimation	MS-COCO	UniHCP
Unsupervised Object Localization	COCO_20k	DeepCut
Image Outpainting	MSCOCO	NUWA-3D
Image Captioning	MSCOCO	CapDec
Visual Question Answering	COCO Visual Question Answering (VQA) real images 2.0 open ended	MaMMUT
Question Answering	COCO Visual Question Answering (VQA) real images 1.0 open ended	MaMMUT
Multi-Label Learning	COCO 2014	SADCL
Visual Question Answering	MS COCO	BenchLMM
Weakly Supervised Object Detection	MSCOCO	CASD
Object Detection	MSCOCO	DAS
Interactive Segmentation	COCO minival	ViT-B+MST+CL
Object Detection	COCO+	RepPoints + Self-adaptation
Activeness Detection	COCO test-dev	Lightweight OpenPose
Few Shot Open Set Object Detection	MSCOCO	FOODv2
Scene Graph Generation	MS-COCO	NeuSyRE
Multi-object discovery	COCO_20k	Large-scale rOSD
Active Object Detection	MS COCO	RetinaNet
Instance Segmentation	coco minval	R3-CNN
Interactive Segmentation	MS COCO	IOG
Semi Supervised Learning for Image Captioning	MS COCO	Perturb, Predict & Paraphrase
Few-Shot Object Detection	COCO 2017	DETReg
Pose Estimation	COCO minival	MSPN
Class-agnostic Object Detection	MS COCO	MDef-DETR
Object Proposal Generation	MS COCO	MDef-DETR

MS COCO (Microsoft Common Objects in Context)

Benchmarks Edit Add a new result Link an existing benchmark

Papers

Dataset Loaders Edit Add Remove

Tasks Edit

Similar Datasets

ADE20K

LVIS

RefCOCO

Flickr30k

Usage

License Edit

Modalities Edit

Languages Edit

Benchmarks

Add a new result Link an existing benchmark

Dataset Loaders

Add Remove

Tasks

License

Modalities

Languages