YOLACT: Real-time Instance Segmentation

ICCV 2019  ·  Daniel Bolya, Chong Zhou, Fanyi Xiao, Yong Jae Lee ·

We present a simple, fully-convolutional model for real-time instance segmentation that achieves 29.8 mAP on MS COCO at 33.5 fps evaluated on a single Titan Xp, which is significantly faster than any previous competitive approach. Moreover, we obtain this result after training on only one GPU. We accomplish this by breaking instance segmentation into two parallel subtasks: (1) generating a set of prototype masks and (2) predicting per-instance mask coefficients. Then we produce instance masks by linearly combining the prototypes with the mask coefficients. We find that because this process doesn't depend on repooling, this approach produces very high-quality masks and exhibits temporal stability for free. Furthermore, we analyze the emergent behavior of our prototypes and show they learn to localize instances on their own in a translation variant manner, despite being fully-convolutional. Finally, we also propose Fast NMS, a drop-in 12 ms faster replacement for standard NMS that only has a marginal performance penalty.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract

Results from the Paper

Ranked #21 on Real-time Instance Segmentation on MSCOCO (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Instance Segmentation COCO minival YOLACT-550 (ResNet-50) mask AP 29.9 # 89
Instance Segmentation COCO test-dev YOLACT (ResNet-50-FPN) mask AP 29.8% # 102
Real-time Instance Segmentation MSCOCO YOLACT-550 (ResNet-101-FPN) Frame (fps) 33.3 (Titan Xp) # 6
mask AP 28.2 # 21
AP50 46.6 # 14
AP75 29.2 # 14
APS 9.2 # 13
APM 29.3 # 14
APL 44.8 # 15
Real-time Instance Segmentation MSCOCO YOLACT Frame (fps) 45.3 (Titan Xp) # 3
mask AP 24.9 # 22
AP50 42.0 # 15
AP75 25.4 # 15
APS 5.0 # 15
APM 25.3 # 15
APL 45.0 # 14