Instances as Queries

5 May 2021  ·  Yuxin Fang, Shusheng Yang, Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, Wenyu Liu ·

Recently, query based object detection frameworks achieve comparable performance with previous state-of-the-art object detectors. However, how to fully leverage such frameworks to perform instance segmentation remains an open problem... In this paper, we present QueryInst (Instances as Queries), a query based instance segmentation method driven by parallel supervision on dynamic mask heads. The key insight of QueryInst is to leverage the intrinsic one-to-one correspondence in object queries across different stages, as well as one-to-one correspondence between mask RoI features and object queries in the same stage. This approach eliminates the explicit multi-stage mask head connection and the proposal distribution inconsistency issues inherent in non-query based multi-stage instance segmentation methods. We conduct extensive experiments on three challenging benchmarks, i.e., COCO, CityScapes, and YouTube-VIS to evaluate the effectiveness of QueryInst in instance segmentation and video instance segmentation (VIS) task. Specifically, using ResNet-101-FPN backbone, QueryInst obtains 48.1 box AP and 42.8 mask AP on COCO test-dev, which is 2 points higher than HTC in terms of both box AP and mask AP, while runs 2.4 times faster. For video instance segmentation, QueryInst achieves the best performance among all online VIS approaches and strikes a decent speed-accuracy trade-off. Code is available at \url{https://github.com/hustvl/QueryInst}. read more

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Object Detection COCO minival QueryInst (single scale) box AP 56.1 # 12
AP50 75.8 # 4
AP75 61.7 # 3
APS 40.2 # 4
APM 59.8 # 5
APL 71.5 # 5
Instance Segmentation COCO minival QueryInst (single scale) mask AP 48.9 # 8
AP50 74.0 # 1
AP75 53.9 # 1
APL 68.3 # 1
APM 52.6 # 3
APS 30.8 # 3
Instance Segmentation COCO test-dev QueryInst (single scale) mask AP 49.1 # 7
AP50 74.2 # 2
AP75 53.8 # 2
APS 31.5 # 3
APM 51.8 # 3
APL 63.2 # 3
Object Detection COCO test-dev QueryInst (single-scale) box AP 56.1 # 13
AP50 75.9 # 3
AP75 61.9 # 3
APS 37.4 # 9
APM 58.9 # 8
APL 70.3 # 3

Methods