Resolving Semantic Confusions for Improved Zero-Shot Detection

Zero-shot detection (ZSD) is a challenging task where we aim to recognize and localize objects simultaneously, even when our model has not been trained with visual samples of a few target ("unseen") classes. Recently, methods employing generative models like GANs have shown some of the best results, where unseen-class samples are generated based on their semantics by a GAN trained on seen-class data, enabling vanilla object detectors to recognize unseen objects. However, the problem of semantic confusion still remains, where the model is sometimes unable to distinguish between semantically-similar classes. In this work, we propose to train a generative model incorporating a triplet loss that acknowledges the degree of dissimilarity between classes and reflects them in the generated samples. Moreover, a cyclic-consistency loss is also enforced to ensure that generated visual samples of a class highly correspond to their own semantics. Extensive experiments on two benchmark ZSD datasets - MSCOCO and PASCAL-VOC - demonstrate significant gains over the current ZSD methods, reducing semantic confusion and improving detection for the unseen classes.

PDF Abstract British Machine 2022 PDF British Machine 2022 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Generalized Zero-Shot Object Detection MS-COCO ZSD-SCR HM(mAP) 26.15 # 2
HM(Recall) 61.18 # 2
Zero-Shot Object Detection MS-COCO ZSD-SCR mAP 20.10 # 2
Recall 65.10 # 1
Zero-Shot Object Detection PASCAL VOC'07 ZSD-SCR mAP 62.70 # 4