Cut and Learn for Unsupervised Object Detection and Instance Segmentation

We propose Cut-and-LEaRn (CutLER), a simple approach for training unsupervised object detection and segmentation models. We leverage the property of self-supervised models to 'discover' objects without supervision and amplify it to train a state-of-the-art localization model without any human labels. CutLER first uses our proposed MaskCut approach to generate coarse masks for multiple objects in an image and then learns a detector on these masks using our robust loss function. We further improve the performance by self-training the model on its predictions. Compared to prior work, CutLER is simpler, compatible with different detection architectures, and detects multiple objects. CutLER is also a zero-shot unsupervised detector and improves detection performance AP50 by over 2.7 times on 11 benchmarks across domains like video frames, paintings, sketches, etc. With finetuning, CutLER serves as a low-shot detector surpassing MoCo-v2 by 7.3% APbox and 6.6% APmask on COCO when training with 5% labels.

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Object Detection Clipart1k CutLER (Cascade+DINO) AP50 21.1 # 1
AR 41.3 # 1
Unsupervised Instance Segmentation COCO val2017 CutLER (Cascade+DINO) AP50 18.9 # 1
AP75 9.7 # 1
AP 9.2 # 1
Unsupervised Panoptic Segmentation COCO val2017 CutLER+STEGO PQ 12.4 # 2
SQ 36.1 # 2
RQ 15.2 # 2
Unsupervised Zero-Shot Instance Segmentation COCO val2017 CutLER AP 5.3 # 2
AP75 5.5 # 2
AP50 8.6 # 2
AR100 9.3 # 2
Unsupervised Object Detection COCO val2017 CutLER (Cascade+DINO) AP50 21.9 # 1
AP75 11.8 # 1
AP 12.3 # 1
Unsupervised Object Detection Comic2k CutLER (Cascade+DINO) AP50 30.4 # 1
AR 38.6 # 1
Unsupervised Object Detection KITTI CutLER (Cascade+DINO) AP50 18.4 # 1
AR 27.5 # 1
Unsupervised Object Detection LVIS v1.0 val CutLER (Cascade+DINO) AP50 8.4 # 1
AR 21.8 # 1
Unsupervised Object Detection Objects365 CutLER (Cascade+DINO) AP50 21.6 # 1
AR 34.2 # 1
Unsupervised Object Detection OpenImages-v6 CutLER (Cascade+DINO) AP50 17.3 # 1
AR 29.6 # 1
Unsupervised Object Detection PASCAL VOC 2007 CutLER (Cascade+DINO) AP50 36.9 # 1
AP75 19.2 # 1
AP 20.2 # 1
APS 1.3 # 1
APM 6.5 # 1
APL 32.2 # 1
Unsupervised Object Detection UVO CutLER (Cascade+DINO) AP50 31.7 # 1
AP75 14.1 # 1
AP 16.1 # 1
Unsupervised Instance Segmentation UVO CutLER (Cascade+DINO) AP50 22.8 # 1
AP75 8 # 1
AP 10.1 # 1
Unsupervised Object Detection Watercolor2k CutLER (Cascade+DINO) AP50 37.5 # 1
AR 44.6 # 1

Methods


No methods listed for this paper. Add relevant methods here