Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation

Despite great progress in supervised semantic segmentation,a large performance drop is usually observed when deploying the model in the wild. Domain adaptation methods tackle the issue by aligning the source domain and the target domain. However, most existing methods attempt to perform the alignment from a holistic view, ignoring the underlying class-level data structure in the target domain. To fully exploit the supervision in the source domain, we propose a fine-grained adversarial learning strategy for class-level feature alignment while preserving the internal structure of semantics across domains. We adopt a fine-grained domain discriminator that not only plays as a domain distinguisher, but also differentiates domains at class level. The traditional binary domain labels are also generalized to domain encodings as the supervision signal to guide the fine-grained feature alignment. An analysis with Class Center Distance (CCD) validates that our fine-grained adversarial strategy achieves better class-level alignment compared to other state-of-the-art methods. Our method is easy to implement and its effectiveness is evaluated on three classical domain adaptation tasks, i.e., GTA5 to Cityscapes, SYNTHIA to Cityscapes and Cityscapes to Cross-City. Large performance gains show that our method outperforms other global feature alignment based and class-wise alignment based counterparts. The code is publicly available at https://github.com/JDAI-CV/FADA.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Synthetic-to-Real Translation GTAV-to-Cityscapes Labels FADA mIoU 50.1 # 46
Image-to-Image Translation SYNTHIA-to-Cityscapes FADA (ResNet-101) mIoU (13 classes) 52.5 # 15
Synthetic-to-Real Translation SYNTHIA-to-Cityscapes FADA(ResNet-101) MIoU (13 classes) 52.5 # 30
MIoU (16 classes) 45.2 # 29
Domain Adaptation SYNTHIA-to-Cityscapes FADA (ResNet-101) mIoU 45.2 # 21
Domain Adaptation SYNTHIA-to-Cityscapes FADA (VGG-16) mIoU 39.5 # 28

Methods


No methods listed for this paper. Add relevant methods here