Training data-efficient image transformers & distillation through attention

Benchmark Model Rank Results
document-image-classification-on-rvl-cdipDeiT-B#24Accuracy: 90.32%Parameters: 87M
document-layout-analysis-on-publaynet-valDeiT-B#6Overall: 0.932Text: 0.934Title: 0.874List: 0.921Table: 0.972
efficient-vits-on-imagenet-1k-with-deit-sBase (DeiT-S)#4Top 1 Accuracy: 79.8GFLOPs: 4.6
efficient-vits-on-imagenet-1k-with-deit-tBase (DeiT-T)#6Top 1 Accuracy: 72.2GFLOPs: 1.2
fine-grained-image-classification-on-oxfordDeiT-B#9Accuracy: 98.8%PARAMS: 86M
fine-grained-image-classification-on-stanfordDeiT-B#49Accuracy: 93.3%PARAMS: 86M
image-classification-on-cifar-10DeiT-B#14Percentage correct: 99.1
image-classification-on-cifar-100DeiT-B#22Percentage correct: 90.8PARAMS: 86M
image-classification-on-flowers-102DeiT-B#17Accuracy: 98.8%PARAMS: 86M
image-classification-on-imagenetDeiT-B 384#241Top 1 Accuracy: 85.2%Number of params: 87M
image-classification-on-imagenetDeiT-B#326Top 1 Accuracy: 84.2%Number of params: 86M
image-classification-on-imagenetDeiT-B#505Top 1 Accuracy: 82.6%Number of params: 22M
image-classification-on-imagenetDeiT-B#853Top 1 Accuracy: 76.6%Number of params: 5M
image-classification-on-imagenet-realDeiT-B-384#24Accuracy: 89.3%Params: 86M
image-classification-on-imagenet-realDeiT-B#26Accuracy: 88.7%Params: 86M
image-classification-on-imagenet-realDeiT-S#36Accuracy: 86.8%Params: 22M
image-classification-on-imagenet-realDeiT-Ti#48Accuracy: 82.1%Params: 5M
image-classification-on-inaturalist-2018DeiT-B#15Top-1 Accuracy: 79.5%