ViDT: An Efficient and Effective Fully Transformer-based Object Detector

Transformers are transforming the landscape of computer vision, especially for recognition tasks. Detection transformers are the first fully end-to-end learning systems for object detection, while vision transformers are the first fully transformer-based architecture for image classification... In this paper, we integrate Vision and Detection Transformers (ViDT) to build an effective and efficient object detector. ViDT introduces a reconfigured attention module to extend the recent Swin Transformer to be a standalone object detector, followed by a computationally efficient transformer decoder that exploits multi-scale features and auxiliary techniques essential to boost the detection performance without much increase in computational load. Extensive evaluation results on the Microsoft COCO benchmark dataset demonstrate that ViDT obtains the best AP and latency trade-off among existing fully transformer-based object detectors, and achieves 49.2AP owing to its high scalability for large models. We will release the code and trained models athttps://github.com/naver-ai/vidt read more

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Object Detection COCO 2017 val ViDT Swin-nano AP 40.4 # 4
AP50 59.6 # 4
AP75 43.3 # 4
APS 23.2 # 4
APM 42.5 # 4
APL 55.8 # 4
Param. 16M # 2
Object Detection COCO 2017 val ViDT Swin-tiny AP 44.8 # 3
AP50 64.5 # 3
AP75 48.7 # 3
APS 25.9 # 3
APM 47.6 # 3
APL 62.1 # 3
Param. 38M # 3
Object Detection COCO 2017 val ViDT Swin-base AP 49.2 # 1
AP50 69.4 # 1
AP75 53.1 # 1
APS 30.6 # 1
APM 52.6 # 1
APL 66.9 # 1
Param. 0.1B # 1
Object Detection COCO 2017 val ViDT Swin-small AP 47.5 # 2
AP50 67.7 # 2
AP75 51.4 # 2
APS 29.2 # 2
APM 50.7 # 2
APL 64.8 # 2
Param. 61M # 4

Methods