Improving MMD-GAN Training with Repulsive Loss Function

ICLR 2019  ·  Wei Wang, Yuan Sun, Saman Halgamuge ·

Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. First, we argue that the existing MMD loss function may discourage the learning of fine details in data as it attempts to contract the discriminator outputs of real data. To address this issue, we propose a repulsive loss function to actively learn the difference among the real data by simply rearranging the terms in MMD. Second, inspired by the hinge loss, we propose a bounded Gaussian kernel to stabilize the training of MMD-GAN with the repulsive loss function. The proposed methods are applied to the unsupervised image generation tasks on CIFAR-10, STL-10, CelebA, and LSUN bedroom datasets. Results show that the repulsive loss function significantly improves over the MMD loss at no additional computational cost and outperforms other representative loss functions. The proposed methods achieve an FID score of 16.21 on the CIFAR-10 dataset using a single DCGAN network and spectral normalization.

PDF Abstract ICLR 2019 PDF ICLR 2019 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Image Generation CIFAR-10 MMD-GAN-rep Inception score 8.29 # 55
FID 16.21 # 127

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Image Generation STL-10 Improving MMD GAN FID 37.63 # 25
Inception score 9.34 # 16

Methods