Domain-Adversarial Training of Neural Networks

We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient descent, and can thus be implemented with little effort using any of the deep learning packages. We demonstrate the success of our approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. We also validate the approach for descriptor learning task in the context of person re-identification application.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Domain Adaptation EPIC-KITCHENS-100 DANN Average Accuracy 39.2 # 4
Unsupervised Domain Adaptation HMDB-UCF DANN Accuracy 88.09 # 6
Unsupervised Domain Adaptation Jester DANN Accuracy 55.4 # 4
Domain Adaptation MNIST-to-MNIST-M DANN [ganin2016domain] Accuracy 77.4 # 3
Sentiment Analysis Multi-Domain Sentiment Dataset DANN DVD 75.40 # 6
Books 71.43 # 6
Electronics 77.67 # 5
Kitchen 80.53 # 6
Average 76.26 # 6
Unsupervised Domain Adaptation Office-Home DANN [cite:JMLR16RevGrad] Accuracy 76.8 # 2
Domain Adaptation SVNH-to-MNIST DANN [ganin2016domain] Accuracy 70.7 # 8
Unsupervised Domain Adaptation UCF-HMDB DANN Accuracy 80.83 # 5

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Synthetic-to-Real Translation Syn2Real-C DANN Accuracy 57.4 # 5
Domain Adaptation Synth Digits-to-SVHN DANN [ganin2016domain] Accuracy 90.3 # 2

Methods


No methods listed for this paper. Add relevant methods here