OmniCount: Multi-label Object Counting with Semantic-Geometric Priors
Object counting is pivotal for understanding the composition of scenes. Previously, this task was dominated by class-specific methods, which have gradually evolved into more adaptable class-agnostic strategies. However, these strategies come with their own set of limitations, such as the need for manual exemplar input and multiple passes for multiple categories, resulting in significant inefficiencies. This paper introduces a new, more practical approach enabling simultaneous counting of multiple object categories using an open vocabulary framework. Our solution, OmniCount, stands out by using semantic and geometric insights from pre-trained models to count multiple categories of objects as specified by users, all without additional training. OmniCount distinguishes itself by generating precise object masks and leveraging point prompts via the Segment Anything Model for efficient counting. To evaluate OmniCount, we created the OmniCount-191 benchmark, a first-of-its-kind dataset with multi-label object counts, including points, bounding boxes, and VQA annotations. Our comprehensive evaluation in OmniCount-191, alongside other leading benchmarks, demonstrates OmniCount's exceptional performance, significantly outpacing existing solutions and heralding a new era in object counting technology.
PDF AbstractDatasets
Introduced in the Paper:
Omnicount-191Used in the Paper:
MS COCO Visual Question Answering PASCAL VOC PASCAL VOC 2007 CARPK FSC147Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Training-free Object Counting | FSC147 | Omnicount | MAE | 18.63 | # 1 | |
Object Counting | FSC147 | Omnicount (Open vocabulary, multi-label, without training) | MAE(test) | 18.63 | # 15 | |
RMSE(test) | 112 | # 15 | ||||
Object Counting | Omnicount-191 | Omnicount | mRMSE | 0.0023 | # 1 | |
Training-free Object Counting | Omnicount-191 | Omnicount | mRMSE | 0.7 | # 1 | |
Object Counting | Pascal VOC 2007 count-test | Omnicount | mRMSE | 0.0023 | # 1 | |
mRMSE-nz | 0.009 | # 1 |