Training data-efficient image transformers & distillation through attention

Benchmark Model Rank Results
efficient-vits-on-imagenet-1k-with-deit-sBase (DeiT-S)#4Top 1 Accuracy: 79.8GFLOPs: 4.6
document-layout-analysis-on-publaynet-valDeiT-B#6Overall: 0.932Text: 0.934Title: 0.874List: 0.921Table: 0.972…
efficient-vits-on-imagenet-1k-with-deit-tBase (DeiT-T)#6Top 1 Accuracy: 72.2GFLOPs: 1.2
fine-grained-image-classification-on-oxford-102-flowersDeiT-B#9Accuracy: 98.8%PARAMS: 86M
image-classification-on-inaturalist-2018DeiT-B#15Top-1 Accuracy: 79.5%
image-classification-on-cifar-10DeiT-B#16Percentage correct: 99.1
image-classification-on-flowers-102DeiT-B#23Accuracy: 98.8%PARAMS: 86M
document-image-classification-on-rvl-cdipDeiT-B#26Accuracy: 90.32%Parameters: 87M
image-classification-on-imagenet-realDeiT-B-384#29Accuracy: 89.3%Params: 86M
image-classification-on-imagenet-realDeiT-B#32Accuracy: 88.7%Params: 86M
image-classification-on-imagenet-realDeiT-S#42Accuracy: 86.8%Params: 22M
image-classification-on-imagenet-realDeiT-Ti#54Accuracy: 82.1%Params: 5M
image-classification-on-imagenet-v2DeiT-B Distilled 384 (1000 epochs)#30Top 1 Accuracy: 75.2
image-classification-on-cifar-100DeiT-B#33Percentage correct: 90.8PARAMS: 86M
fine-grained-image-classification-on-stanford-carsDeiT-B#49Accuracy: 93.3%PARAMS: 86M
image-classification-on-imagenetDeiT-B 384#263Top 1 Accuracy: 85.2%Number of params: 87M
image-classification-on-imagenetDeiT-B#360Top 1 Accuracy: 84.2%Number of params: 86M
image-classification-on-imagenetDeiT-B#553Top 1 Accuracy: 82.6%Number of params: 22M
image-classification-on-imagenetDeiT-B#951Top 1 Accuracy: 76.6%Number of params: 5M