U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

ICLR 2020  ·  Junho Kim, Minjae Kim, Hyeonwoo Kang, Kwanghee Lee ·

We propose a novel method for unsupervised image-to-image translation, which incorporates a new attention module and a new learnable normalization function in an end-to-end manner. The attention module guides our model to focus on more important regions distinguishing between source and target domains based on the attention map obtained by the auxiliary classifier. Unlike previous attention-based method which cannot handle the geometric changes between domains, our model can translate both images requiring holistic changes and images requiring large shape changes. Moreover, our new AdaLIN (Adaptive Layer-Instance Normalization) function helps our attention-guided model to flexibly control the amount of change in shape and texture by learned parameters depending on datasets. Experimental results show the superiority of the proposed method compared to the existing state-of-the-art models with a fixed network architecture and hyper-parameters. Our code and datasets are available at https://github.com/taki0112/UGATIT or https://github.com/znxlwm/UGATIT-pytorch.

PDF Abstract ICLR 2020 PDF ICLR 2020 Abstract


Introduced in the Paper:

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image-to-Image Translation anime-to-selfie U-GAT-IT Kernel Inception Distance 11.52 # 2
Image-to-Image Translation cat2dog U-GAT-IT Kernel Inception Distance 7.07 # 1
Image-to-Image Translation dog2cat U-GAT-IT Kernel Inception Distance 8.15 # 1
Fundus to Angiography Generation Fundus Fluorescein Angiogram Photographs & Colour Fundus Images of Diabetic Patients U-GAT-IT FID 24.5 # 3
Kernel Inception Distance 0.00131 # 4
Image-to-Image Translation horse2zebra U-GAT-IT Kernel Inception Distance 7.06 # 1
Image-to-Image Translation photo2portrait U-GAT-IT Kernel Inception Distance 1.79 # 1
Image-to-Image Translation photo2vangogh U-GAT-IT Kernel Inception Distance 4.28 # 1
Image-to-Image Translation portrait2photo U-GAT-IT Kernel Inception Distance 1.69 # 1
Image-to-Image Translation selfie-to-anime U-GAT-IT Kernel Inception Distance 11.61 # 2
Image-to-Image Translation vangogh2photo U-GAT-IT Kernel Inception Distance 5.61 # 1
Image-to-Image Translation zebra2horse U-GAT-IT Kernel Inception Distance 7.47 # 1


No methods listed for this paper. Add relevant methods here