DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations

20 Jun 2022  ·  Ximeng Sun, Ping Hu, Kate Saenko ·

Solving multi-label recognition (MLR) for images in the low-label regime is a challenging task with many real-world applications. Recent work learns an alignment between textual and visual spaces to compensate for insufficient image labels, but loses accuracy because of the limited amount of available MLR annotations. In this work, we utilize the strong alignment of textual and visual features pretrained with millions of auxiliary image-text pairs and propose Dual Context Optimization (DualCoOp) as a unified framework for partial-label MLR and zero-shot MLR. DualCoOp encodes positive and negative contexts with class names as part of the linguistic input (i.e. prompts). Since DualCoOp only introduces a very light learnable overhead upon the pretrained vision-language framework, it can quickly adapt to multi-label recognition tasks that have limited annotations and even unseen classes. Experiments on standard multi-label recognition benchmarks across two challenging low-label settings demonstrate the advantages of our approach over state-of-the-art methods.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Multi-label Image Recognition with Partial Labels MS-COCO-2014 DualCoOp Average mAP 81.9 # 2
Multi-label Image Recognition with Partial Labels PASCAL VOC 2007 DualCoOp Average mAP 93.2 # 2


No methods listed for this paper. Add relevant methods here