Wavelet Diffusion Models are fast and scalable Image Generators

CVPR 2023  ·  Hao Phung, Quan Dao, Anh Tran ·

Diffusion models are rising as a powerful solution for high-fidelity image generation, which exceeds GANs in quality in many circumstances. However, their slow training and inference speed is a huge bottleneck, blocking them from being used in real-time applications. A recent DiffusionGAN method significantly decreases the models' running time by reducing the number of sampling steps from thousands to several, but their speeds still largely lag behind the GAN counterparts. This paper aims to reduce the speed gap by proposing a novel wavelet-based diffusion scheme. We extract low-and-high frequency components from both image and feature levels via wavelet decomposition and adaptively handle these components for faster processing while maintaining good generation quality. Furthermore, we propose to use a reconstruction term, which effectively boosts the model training convergence. Experimental results on CelebA-HQ, CIFAR-10, LSUN-Church, and STL-10 datasets prove our solution is a stepping-stone to offering real-time and high-fidelity diffusion models. Our code and pre-trained checkpoints are available at \url{https://github.com/VinAIResearch/WaveDiff.git}.

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Generation CelebA-HQ 1024x1024 WaveDiff FID 5.98 # 4
NFE 2 # 1
Image Generation CelebA-HQ 256x256 WaveDiff FID 5.94 # 5
Recall 0.37 # 2
NFE 2 # 1
Image Generation CelebA-HQ 512x512 WaveDiff FID 6.40 # 1
Recall 0.35 # 1
NFE 2 # 1
Image Generation CIFAR-10 WaveDiff FID 4.01 # 55
Recall 0.55 # 4
NFE 4 # 9
Image Generation LSUN Churches 256 x 256 WaveDiff FID 5.06 # 16
Recall 0.40 # 1
NFE 4 # 1
Image Generation STL-10 WaveDiff FID 12.93 # 4
Recall 0.41 # 1
NFE 4 # 1