Multimodal Unsupervised Image-to-Image Translation

ECCV 2018  ·  Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz ·

Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any pairs of corresponding images. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image Translation (MUNIT) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to the state-of-the-art approaches further demonstrates the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image. Code and pretrained models are available at https://github.com/nvlabs/MUNIT

PDF Abstract ECCV 2018 PDF ECCV 2018 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Multimodal Unsupervised Image-To-Image Translation AFHQ MUNIT FID 41.5 # 2
Multimodal Unsupervised Image-To-Image Translation Cats-and-Dogs MUNIT CIS 1.039 # 1
IS 1.050 # 1
Multimodal Unsupervised Image-To-Image Translation CelebA-HQ MUNIT FID 31.4 # 2
Multimodal Unsupervised Image-To-Image Translation Edge-to-Handbags MUNIT Quality 50.0% # 2
Diversity 0.175 # 1
Multimodal Unsupervised Image-To-Image Translation Edge-to-Shoes MUNIT Quality 50.0% # 2
Diversity 0.109 # 1

Methods


No methods listed for this paper. Add relevant methods here