Understanding Gaussian Attention Bias of Vision Transformers Using Effective Receptive Fields

Benchmark Model Rank Results
fine-grained-image-classification-on-caltechViT-S/16 (RPE w/ GAB)#11Top-1 Error Rate: 9.798%
fine-grained-image-classification-on-stanford-1ViT-B/16 (RPE w/ GAB)#12Accuracy: 90.185%
image-classification-on-imagenetViT-B/16 (RPE w/ GAB)#615Top 1 Accuracy: 81.484%
image-classification-on-stanford-carsViT-B/16 (RPE w/ GAB)#7Accuracy: 93.743
image-classification-on-stanford-carsViT-M/16 (RPE w/ GAB)#20Accuracy: 83.89
object-detection-on-cocoSwin-S (RPE w/ GAB)#107box mAP: 48.23
semantic-segmentation-on-ade20k-valSwin-S (RPE w/ GAB)#65mIoU: 46.41