Asymmetric Masked Distillation for Pre-Training Small Foundation Models

Benchmark Model Rank Results
action-classification-on-kinetics-400AMD(ViT-B/16)Acc@1: 82.2Acc@5: 95.3FLOPs (G) x views: 180x15Parameters (M): 87
action-classification-on-kinetics-400AMD(ViT-S/16)Acc@1: 80.1Acc@5: 94.5FLOPs (G) x views: 57X15Parameters (M): 22
action-recognition-in-videos-on-hmdb-51AMD(ViT-B/16)Average accuracy of 3 splits: 79.6
action-recognition-in-videos-on-somethingAMD(ViT-B/16)Top-1 Accuracy: 73.3Top-5 Accuracy: 94.0Parameters: 87
action-recognition-in-videos-on-somethingAMD(ViT-S/16)Top-1 Accuracy: 70.2Top-5 Accuracy: 92.5Parameters: 22GFLOPs: 57x6
action-recognition-in-videos-on-ucf101AMD(ViT-B/16)3-fold Accuracy: 97.1
action-recognition-on-ava-v2-2AMD(ViT-B/16)mAP: 33.5
image-classification-on-imagenetAMD(ViT-B/16)Top 1 Accuracy: 84.6%Number of params: 87M
image-classification-on-imagenetAMD(ViT-S/16)Top 1 Accuracy: 82.1%Number of params: 22M