Alias-Free Generative Adversarial Networks

We observe that despite their hierarchical convolutional nature, the synthesis process of typical generative adversarial networks depends on absolute pixel coordinates in an unhealthy manner. This manifests itself as, e.g., detail appearing to be glued to image coordinates instead of the surfaces of depicted objects. We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hierarchical synthesis process. The resulting networks match the FID of StyleGAN2 but differ dramatically in their internal representations, and they are fully equivariant to translation and rotation even at subpixel scales. Our results pave the way for generative models better suited for video and animation.

PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Image Generation AFHQV2 Alias-Free-R FID 4.40 # 3
EQ-T 64.89 # 1
EQ-R 40.34 # 1
Image Generation AFHQV2 Alias-Free-T FID 4.04 # 2
EQ-T 60.15 # 2
EQ-R 13.51 # 2
Image Generation AFHQV2 StyleGAN2 FID 4.62 # 4
EQ-T 13.83 # 3
EQ-R 11.50 # 3
Image Generation FFHQ 1024 x 1024 StyleGAN3-R FID 3.07 # 7
Image Generation FFHQ 1024 x 1024 StyleGAN3-T FID 2.79 # 4
Image Generation FFHQ-U StyleGAN2 + Simplified generator FID 5.21 # 11
EQ-T 19.47 # 9
EQ-R 10.41 # 12
Image Generation FFHQ-U StyleGAN2 + No noise inputs FID 4.54 # 5
EQ-T 15.81 # 12
EQ-R 10.84 # 6
Image Generation FFHQ-U StyleGAN2 FID 5.14 # 10
Image Generation FFHQ-U StyleGAN2 + Transformed Fourier features FID 4.64 # 7
EQ-T 45.20 # 5
EQ-R 10.61 # 11
Image Generation FFHQ-U StyleGAN2 + Fourier features FID 4.79 # 9
EQ-T 16.23 # 10
EQ-R 10.81 # 8
Image Generation FFHQ-U Alias-Free-R FID 3.66 # 1
EQ-T 64.78 # 2
EQ-R 47.64 # 1
Image Generation FFHQ-U Alias-Free-T FID 3.67 # 2
EQ-T 61.69 # 4
EQ-R 13.95 # 3
Image Generation FFHQ-U StyleGAN2 (70000 img, 1024^2, train from scratch) FID 3.79 # 3
EQ-T 15.89 # 11
EQ-R 10.79 # 10
Image Generation FFHQ-U StyleGAN2 + Rotation equiv. (Alias-Free-R) FID 4.50 # 4
EQ-T 66.65 # 1
EQ-R 40.48 # 2
Image Generation FFHQ-U StyleGAN2 + Flexible layers (Alias-Free-T) FID 4.62 # 6
EQ-T 63.01 # 3
EQ-R 13.12 # 4
Image Generation FFHQ-U StyleGAN2 + Non-critical sampling FID 4.78 # 8
EQ-T 43.90 # 6
EQ-R 10.84 # 6
Image Generation FFHQ-U StyleGAN2 + Filtered nonlinearities FID 6.35 # 13
EQ-T 30.60 # 7
EQ-R 10.81 # 8
Image Generation FFHQ-U StyleGAN2 + Boundaries & upsampling FID 6.02 # 12
EQ-T 24.62 # 8
EQ-R 10.97 # 5
Image Generation MetFaces-U Alias-Free-R FID 18.75 # 1
EQ-T 66.34 # 1
EQ-R 48.57 # 1
Image Generation MetFaces-U Alias-Free-T FID 18.75 # 1
EQ-T 64.11 # 2
EQ-R 16.63 # 2
Image Generation MetFaces-U StyleGAN2 FID 18.98 # 3
EQ-T 18.77 # 3
EQ-R 13.19 # 3

Methods