VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

Benchmark Model Rank Results
action-classification-on-kinetics-400VideoMAE (no extra data, ViT-H, 32x320x320)#32Acc@1: 87.4Acc@5: 97.6
action-classification-on-kinetics-400VideoMAE (no extra data, ViT-H)#40Acc@1: 86.6Acc@5: 97.1
action-classification-on-kinetics-400VideoMAE (no extra data, ViT-L, 32x320x320)#44Acc@1: 86.1Acc@5: 97.3
action-classification-on-kinetics-400VideoMAE (no extra data, ViT-L, 16x4)#50Acc@1: 85.2Acc@5: 96.8
action-classification-on-kinetics-400VideoMAE (no extra data, ViT-B, 16x4)#74Acc@1: 81.5Acc@5: 95.1
action-recognition-in-videos-on-somethingVideoMAE (no extra data, ViT-L, 32x2)#9Top-1 Accuracy: 75.4Top-5 Accuracy: 95.2Parameters: 305
action-recognition-in-videos-on-somethingVideoMAE (no extra data, ViT-L, 16frame)#15Top-1 Accuracy: 74.3Top-5 Accuracy: 94.6Parameters: 305
action-recognition-in-videos-on-somethingVideoMAE (no extra data, ViT-B, 16frame)#33Top-1 Accuracy: 70.8Top-5 Accuracy: 92.4Parameters: 87
action-recognition-on-ava-v2-2VideoMAE (K400 pretrain+finetune, ViT-H, 16x4)#9mAP: 39.5
action-recognition-on-ava-v2-2VideoMAE (K700 pretrain+finetune, ViT-L, 16x4)#10mAP: 39.3
action-recognition-on-ava-v2-2VideoMAE (K400 pretrain+finetune, ViT-L, 16x4)#12mAP: 37.8
action-recognition-on-ava-v2-2VideoMAE (K400 pretrain, ViT-H, 16x4)#14mAP: 36.5
action-recognition-on-ava-v2-2VideoMAE (K700 pretrain, ViT-L, 16x4)#15mAP: 36.1
action-recognition-on-ava-v2-2VideoMAE (K400 pretrain, ViT-L, 16x4)#18mAP: 34.3
action-recognition-on-ava-v2-2VideoMAE (K400 pretrain+finetune, ViT-B, 16x4)#21mAP: 31.8
action-recognition-on-ava-v2-2VideoMAE (K400 pretrain, ViT-B, 16x4)#31mAP: 26.7
self-supervised-action-recognition-on-hmdb51VideoMAE#5Top-1 Accuracy: 73.3Pre-Training Dataset: Kinetics400Frozen: false
self-supervised-action-recognition-on-hmdb51VideoMAE(no extra data)#21Top-1 Accuracy: 62.6Pre-Training Dataset: no extra dataFrozen: false
self-supervised-action-recognition-on-ucf101VideoMAE#53-fold Accuracy: 96.1Pre-Training Dataset: Kinetics400Frozen: false
self-supervised-action-recognition-on-ucf101VideoMAE(no extra data)#193-fold Accuracy: 91.3Pre-Training Dataset: no extra dataFrozen: false