Unmasked Teacher: Towards Training-Efficient Video Foundation Models

Benchmark Model Rank Results
action-classification-on-kinetics-400Unmasked Teacher (ViT-L)#5Acc@1: 90.6Acc@5: 98.7FLOPs (G) x views: 1434×3×4
action-classification-on-kinetics-400UMT-L (ViT-L/16)#6Acc@1: 90.6Acc@5: 98.7
action-classification-on-kinetics-600UMT-L (ViT-L/16)#7Top-1 Accuracy: 90.5Top-5 Accuracy: 98.8
action-classification-on-kinetics-700UMT-L (ViT-L/16)#5Top-1 Accuracy: 83.6Top-5 Accuracy: 96.7
action-classification-on-moments-in-timeUMT-L (ViT-L/16)#2Top 1 Accuracy: 48.7Top 5 Accuracy: 78.2
action-recognition-on-ava-v2-2UMT-L (ViT-L/16)#8mAP: 39.8
video-question-answering-on-activitynet-qaUMT-L (ViT-L/16)#6Accuracy: 47.9
video-retrieval-on-activitynetUMT-L (ViT-L/16)#6text-to-video R@1: 66.8text-to-video R@5: 89.1
video-retrieval-on-didemoUMT-L (ViT-L/16)#5text-to-video R@1: 70.4text-to-video R@5: 90.1
video-retrieval-on-lsmdcUMT-L (ViT-L/16)#3text-to-video R@1: 43.0text-to-video R@5: 65.5
video-retrieval-on-msr-vttUMT-L (ViT-L/16)#5text-to-video R@1: 58.8text-to-video R@5: 81.0
video-retrieval-on-ssv2-label-retrievalUMT-L (ViT-L/16)#1text-to-video R@1: 73.3text-to-video R@5: 92.7
video-retrieval-on-ssv2-template-retrievalUMT-L (ViT-L/16)#1text-to-video R@1: 90.8text-to-video R@10: 100.0
video-retrieval-on-vatexUnmasked Teacher#5text-to-video R@1: 72text-to-video R@5: 95.1text-to-video R@10: 97.8
visual-question-answering-on-msrvtt-qa-1UMT-L (ViT-L/16)#5Accuracy: 0.471
visual-question-answering-on-msvd-qa-1UMT-L (ViT-L/16)#10Accuracy: 0.552
zero-shot-video-retrieval-on-activitynetUMT-L (ViT-L/16)#4text-to-video R@1: 42.8text-to-video R@5: 69.6
zero-shot-video-retrieval-on-didemoUMT-L (ViT-L/16)#6text-to-video R@1: 48.6text-to-video R@5: 72.9
zero-shot-video-retrieval-on-lsmdcUMT-L (ViT-L/16)#4text-to-video R@1: 25.2text-to-video R@5: 43.0
zero-shot-video-retrieval-on-msr-vttUMT-L (ViT-L/16)#11text-to-video R@1: 42.6text-to-video R@5: 64.4
zero-shot-video-retrieval-on-msvdUMT-L (ViT-L/16)#7text-to-video R@1: 49.0text-to-video R@5: 76.9