Signature and Log-signature for the Study of Empirical Distributions Generated with GANs

7 Mar 2022  ·  Joaquim de Curtò, Irene de Zarzà, Hong Yan, Carlos T. Calafate ·

In this paper, we bring forward the use of the recently developed Signature Transform as a way to measure the similarity between image distributions and provide detailed acquaintance and extensive evaluations. We are the first to pioneer RMSE and MAE Signature, along with log-signature as an alternative to measure GAN convergence, a problem that has been extensively studied. We are also forerunners to introduce analytical measures based on statistics to study the goodness of fit of the GAN sample distribution that are both efficient and effective. Current GAN measures involve lots of computation normally done at the GPU and are very time consuming. In contrast, we diminish the computation time to the order of seconds and computation is done at the CPU achieving the same level of goodness. Lastly, a PCA adaptive t-SNE approach, which is novel in this context, is also proposed for data visualization.

PDF Abstract

Datasets


Introduced in the Paper:

NASA Perseverance

Used in the Paper:

AFHQ MetFaces

Results from the Paper


 Ranked #1 on Image Generation on AFHQ Wild (RMSE Signature metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Generation AFHQ Cat Stylegan2-ada (NVIDIA pre-trained) RMSE Signature 61450 # 1
RMSE log-signature 29201 # 1
MAE Signature 45968 # 1
MAE log-signature 22297 # 1
Image Generation AFHQ Dog Stylegan2-ada (NVIDIA pre-trained) RMSE Signature 38861 # 1
RMSE log-signature 31686 # 1
MAE Signature 30441 # 1
MAE log-signature 24612 # 1
Image Generation AFHQ Wild Stylegan2-ada (NVIDIA pre-trained) RMSE Signature 33306 # 1
RMSE log-signature 26622 # 1
MAE Signature 25578 # 1
MAE log-signature 20359 # 1
Image Generation MetFaces Stylegan2-ada (NVIDIA pre-trained) RMSE Signature 33247 # 2
RMSE log-signature 25685 # 3
MAE Signature 23428 # 3
MAE log-signature 18071 # 3
Image Generation MetFaces t-Stylegan3-ada (NVIDIA pre-trained) RMSE Signature 30894 # 1
RMSE log-signature 21560 # 1
MAE Signature 19872 # 1
MAE log-signature 13761 # 1
Image Generation MetFaces r-Stylegan3-ada (NVIDIA pre-trained) RMSE Signature 34977 # 3
RMSE log-signature 24707 # 2
MAE Signature 22799 # 2
MAE log-signature 16539 # 2
Image Generation NASA Perseverance Stylegan2-ada RMSE Signature 11601 # 1
RMSE log-signature 7397 # 1
MAE Signature 9086 # 1
MAE log-signature 5717 # 1

Methods