Understanding and Stabilizing GANs' Training Dynamics with Control Theory

29 Sep 2019  ·  Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang ·

Generative adversarial networks (GANs) are effective in generating realistic images but the training is often unstable. There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods. To this end, we present a conceptually novel perspective from control theory to directly model the dynamics of GANs in the function space and provide simple yet effective methods to stabilize GANs' training. We first analyze the training dynamic of a prototypical Dirac GAN and adopt the widely-used closed-loop control (CLC) to improve its stability. We then extend CLC to stabilize the training dynamic of normal GANs, where CLC is implemented as a squared $L2$ regularizer on the output of the discriminator. Empirical results show that our method can effectively stabilize the training and obtain state-of-the-art performance on data generation tasks.

PDF Abstract

Datasets


Results from the Paper


Ranked #31 on Image Generation on CIFAR-10 (Inception score metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Image Generation CIFAR-10 CLC-GAN Inception score 8.54 # 31

Methods


No methods listed for this paper. Add relevant methods here