Spatial Transform Decoupling for Oriented Object Detection

21 Aug 2023  ·  Hongtian Yu, Yunjie Tian, Qixiang Ye, Yunfan Liu ·

Vision Transformers (ViTs) have achieved remarkable success in computer vision tasks. However, their potential in rotation-sensitive scenarios has not been fully explored, and this limitation may be inherently attributed to the lack of spatial invariance in the data-forwarding process. In this study, we present a novel approach, termed Spatial Transform Decoupling (STD), providing a simple-yet-effective solution for oriented object detection with ViTs. Built upon stacked ViT blocks, STD utilizes separate network branches to predict the position, size, and angle of bounding boxes, effectively harnessing the spatial transform potential of ViTs in a divide-and-conquer fashion. Moreover, by aggregating cascaded activation masks (CAMs) computed upon the regressed parameters, STD gradually enhances features within regions of interest (RoIs), which complements the self-attention mechanism. Without bells and whistles, STD achieves state-of-the-art performance on the benchmark datasets including DOTA-v1.0 (82.24% mAP) and HRSC2016 (98.55% mAP), which demonstrates the effectiveness of the proposed method. Source code is available at

PDF Abstract

Results from the Paper

 Ranked #1 on Object Detection In Aerial Images on DOTA (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Object Detection In Aerial Images DOTA STD+HiViT-B mAP 82.24% # 1
Object Detection In Aerial Images HRSC2016 STD+ViT-B mAP-07 90.67 # 2
mAP-12 98.55 # 2