Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

Benchmark Model Rank Results
image-generation-on-ffhq-256-x-256StyleNAT (Exposing)#3FID: 2.11FD: 229.42Precision: 0.79Recall: 0.41Density: 0.77
image-generation-on-ffhq-256-x-256StyleGAN-XL (Exposing)#5FID: 2.26FD: 240.07Precision: 0.77Recall: 0.43Density: 0.68
image-generation-on-ffhq-256-x-256StyleSwin (Exposing)#11FID: 2.89FD: 303.21Precision: 0.79Recall: 0.28Density: 0.71
image-generation-on-ffhq-256-x-256InsGen (Exposing)#18FID: 3.46FD: 436.26Precision: 0.64Recall: 0.13Coverage: 0.51
image-generation-on-ffhq-256-x-256Projected-GAN (Exposing)#21FID: 4.29FD: 589.20Precision: 0.57Recall: 0.07Density: 0.31
image-generation-on-ffhq-256-x-256StyleGAN2-ada (Exposing)#26FID: 5.30FD: 514.78Precision: 0.59Recall: 0.06Density: 0.36
image-generation-on-ffhq-256-x-256LDM (Exposing)#32FID: 8.11FD: 226.72Precision: 0.81Recall: 0.44Density: 0.83
image-generation-on-ffhq-256-x-256Unleash-Trans (Exposing)#33FID: 9.02FD: 393.45Precision: 0.76Recall: 0.24Density: 0.61
image-generation-on-ffhq-256-x-256Efficient-vdVAE (Exposing)#39FID: 34.88FD: 514.16Precision: 0.86Recall: 0.14Density: 1.04