Open Vocabulary Object Detection
55 papers with code • 4 benchmarks • 6 datasets
Open-vocabulary detection (OVD) aims to generalize beyond the limited number of base classes labeled during the training phase. The goal is to detect novel classes defined by an unbounded (open) vocabulary at inference.
Benchmarks
These leaderboards are used to track progress in Open Vocabulary Object Detection
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Open Vocabulary Object Detection models and implementations
MS COCO
LVIS
Objects365
MSCOCO
OVAD benchmark
Description Detection Dataset
Most implemented papers
Training-free Boost for Open-Vocabulary Object Detection with Confidence Aggregation
Specifically, in the region-proposal stage, proposals that contain novel instances showcase lower objectness scores, since they are treated as background proposals during the training phase.
Open-Vocabulary Object Detection Using Captions
alirezazareian/ovr-cnn
•
•
CVPR 2021
Weakly supervised and zero-shot learning techniques have been explored to scale object detectors to more categories with less supervision, but they have not been as successful and widely adopted as supervised models.
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels
To enlarge the set of base classes, we propose a method to automatically generate pseudo bounding-box annotations of diverse objects from large-scale image-caption pairs.
RegionCLIP: Region-based Language-Image Pretraining
microsoft/regionclip
•
•
CVPR 2022
However, we show that directly applying such models to recognize image regions for object detection leads to poor performance due to a domain shift: CLIP was trained to match an image as a whole to a text description, without capturing the fine-grained alignment between image regions and text spans.
Detecting Twenty-thousand Classes using Image-level Supervision
For the first time, we train a detector with all the twenty-one-thousand classes of the ImageNet dataset and show that it generalizes to new datasets without finetuning.
Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation
mengqidyangge/hierkd
•
•
CVPR 2022
Open-vocabulary object detection aims to detect novel object categories beyond the training set.
Open-Vocabulary DETR with Conditional Matching
To this end, we propose a novel open-vocabulary detector based on DETR -- hence the name OV-DETR -- which, once trained, can detect any object given its class name or an exemplar image.
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model
In this paper, we introduce a novel method, detection prompt (DetPro), to learn continuous prompt representations for open-vocabulary object detection based on the pre-trained vision-language model.
Localized Vision-Language Matching for Open-vocabulary Object Detection
In this work, we propose an open-vocabulary object detection method that, based on image-caption pairs, learns to detect novel object classes along with a given set of known classes.
GLIPv2: Unifying Localization and Vision-Language Understanding
microsoft/GLIP
•
•
We present GLIPv2, a grounded VL understanding model, that serves both localization tasks (e. g., object detection, instance segmentation) and Vision-Language (VL) understanding tasks (e. g., VQA, image captioning).
