LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

Benchmark Model Rank Results
temporal-relation-extraction-on-vinogroundLanguageBind#15Text Score: 10.6Video Score: 5Group Score: 1.2
zero-shot-action-recognition-on-kineticsLanguageBind#8Top-1 Accuracy: 64.1Top-5 Accuracy: 85.7
zero-shot-video-retrieval-on-activitynetLanguageBind(ViT-H/14)#6text-to-video R@1: 41.0text-to-video R@5: 68.4
zero-shot-video-retrieval-on-activitynetLanguageBind(ViT-L/14)#7text-to-video R@1: 38.4text-to-video R@5: 66.6
zero-shot-video-retrieval-on-didemoLanguageBind(ViT-H/14)#8text-to-video R@1: 39.9text-to-video R@5: 66.1
zero-shot-video-retrieval-on-didemoLanguageBind(ViT-L/14)#9text-to-video R@1: 39.7text-to-video R@5: 65.5
zero-shot-video-retrieval-on-msr-vttLanguageBind(ViT-H/14)#9text-to-video R@1: 44.8text-to-video R@5: 70.0
zero-shot-video-retrieval-on-msr-vttLanguageBind(ViT-L/14)#10text-to-video R@1: 42.8text-to-video R@5: 67.5
zero-shot-video-retrieval-on-msvdLanguageBind(ViT-L/14)#4text-to-video R@1: 54.1text-to-video R@5: 81.1
zero-shot-video-retrieval-on-msvdLanguageBind(ViT-H/14)#5text-to-video R@1: 53.9text-to-video R@5: 80.4