Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

Benchmark Model Rank Results
image-generation-on-ffhq-256-x-256StyleNAT (Exposing)#4FID: 2.11FD: 229.42Precision: 0.79Recall: 0.41Density: 0.77…
image-generation-on-ffhq-256-x-256StyleGAN-XL (Exposing)#6FID: 2.26FD: 240.07Precision: 0.77Recall: 0.43Density: 0.68…
image-generation-on-ffhq-256-x-256StyleSwin (Exposing)#12FID: 2.89FD: 303.21Precision: 0.79Recall: 0.28Density: 0.71…
image-generation-on-ffhq-256-x-256InsGen (Exposing)#19FID: 3.46FD: 436.26Precision: 0.64Recall: 0.13Coverage: 0.51
image-generation-on-ffhq-256-x-256Projected-GAN (Exposing)#22FID: 4.29FD: 589.20Precision: 0.57Recall: 0.07Density: 0.31…
image-generation-on-ffhq-256-x-256StyleGAN2-ada (Exposing)#27FID: 5.30FD: 514.78Precision: 0.59Recall: 0.06Density: 0.36…
image-generation-on-ffhq-256-x-256LDM (Exposing)#33FID: 8.11FD: 226.72Precision: 0.81Recall: 0.44Density: 0.83…
image-generation-on-ffhq-256-x-256Unleash-Trans (Exposing)#34FID: 9.02FD: 393.45Precision: 0.76Recall: 0.24Density: 0.61…
image-generation-on-ffhq-256-x-256Efficient-vdVAE (Exposing)#40FID: 34.88FD: 514.16Precision: 0.86Recall: 0.14Density: 1.04…