StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

19 Oct 2017  ·  Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas ·

Although Generative Adversarial Networks (GANs) have shown remarkable success in various tasks, they still face challenges in generating high quality images. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images. First, we propose a two-stage generative adversarial network architecture, StackGAN-v1, for text-to-image synthesis. The Stage-I GAN sketches the primitive shape and colors of the object based on given text description, yielding low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. Second, an advanced multi-stage generative adversarial network architecture, StackGAN-v2, is proposed for both conditional and unconditional generative tasks. Our StackGAN-v2 consists of multiple generators and discriminators in a tree-like structure; images at multiple scales corresponding to the same scene are generated from different branches of the tree. StackGAN-v2 shows more stable training behavior than StackGAN-v1 by jointly approximating multiple distributions. Extensive experiments demonstrate that the proposed stacked generative adversarial networks significantly outperform other state-of-the-art methods in generating photo-realistic images.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Text-to-Image Generation CUB StackGAN-v2 FID 15.3 # 11
Inception score 3.82 # 12
Text-to-Image Generation CUB StackGAN-v1 FID 51.89 # 13
Inception score 3.7 # 13
Image Generation LSUN Bedroom 256 x 256 StackGAN-v2 FID 35.61 # 19
Text-to-Image Generation MS COCO StackGAN-v1 FID 74.05 # 66
Inception score 8.45 # 25
Text-to-Image Generation Oxford 102 Flowers StackGAN-v1 FID 55.28 # 7
Inception score 3.2 # 4
Text-to-Image Generation Oxford 102 Flowers StackGAN-v2 FID 48.68 # 6
Inception score 3.26 # 3

Methods