InternVideo: General Video Foundation Models via Generative and Discriminative Learning

Benchmark Model Rank Results
action-recognition-in-videos-on-hmdb-51InternVideo#1Average accuracy of 3 splits: 89.3
action-recognition-in-videos-on-something-something-v1InternVideo#1Top 1 Accuracy: 70.0
action-recognition-in-videos-on-something-something-v2InternVideo#2Top-1 Accuracy: 77.2
spatio-temporal-action-localization-on-ava-kineticsInternVideo#2val mAP: 41.01
video-retrieval-on-msvdInternVideo#2text-to-video R@1: 58.4video-to-text R@1: 76.3
action-classification-on-kinetics-400InternVideo#3Acc@1: 91.1
action-classification-on-kinetics-700InternVideo-T#3Top-1 Accuracy: 84.0
video-question-answering-on-starInternVideo#4Average Accuracy: 58.7
zero-shot-video-question-answer-on-starInternVideo#4Accuracy: 41.6
action-classification-on-kinetics-600InternVideo-T#5Top-1 Accuracy: 91.3
action-recognition-on-ava-v2-2InternVideo#5mAP: 41.01
temporal-action-localization-on-thumos14ActionFormer (InternVideo features)#5Avg mAP (0.3:0.7): 71.58
visual-question-answering-on-msrvtt-qaInternVideo#5Accuracy: 0.471
zero-shot-action-recognition-on-kinetics-400-zero-shot-action-classificationInternVideo (zero-shot)#5Accuracy: 64.25
zero-shot-video-retrieval-on-vatexInternVideo#5text-to-video R@1: 49.5video-to-text R@1: 69.5
temporal-action-localization-on-fineactionInternVideo#6mAP: 17.57
video-retrieval-on-vatexInternVideo#6text-to-video R@1: 71.1video-to-text R@1: 87.2
temporal-action-localization-on-hacsInternVideo#7Average-mAP: 41.55
video-retrieval-on-lsmdcInternVideo#7text-to-video R@1: 34.0video-to-text R@1: 34.9
zero-shot-video-retrieval-on-lsmdcInternVideo#7text-to-video R@1: 17.6text-to-video R@5: 32.4…
video-retrieval-on-activitynetInternVideo#8text-to-video R@1: 62.2video-to-text R@1: 62.8
zero-shot-video-question-answer-on-tvqaInternVideo (no speech)#8Accuracy: 35.9
temporal-action-localization-on-activitynet-1-3InternVideo#9mAP: 39.00
video-retrieval-on-msr-vttInternVideo#9text-to-video R@1: 55.2video-to-text R@1: 57.9
visual-question-answering-on-msvd-qaInternVideo#9Accuracy: 0.555
video-retrieval-on-didemoInternVideo#10text-to-video R@1: 57.9video-to-text R@1: 59.1
zero-shot-video-retrieval-on-activitynetInternVideo#10text-to-video R@1: 30.7video-to-text R@1: 31.4
zero-shot-video-retrieval-on-msvdInternVideo#11text-to-video R@1: 43.4video-to-text R@1: 67.6
zero-shot-video-retrieval-on-didemoInternVideo#16text-to-video R@1: 31.5text-to-video R@5: 57.6…
zero-shot-video-retrieval-on-msr-vttInternVideo#17text-to-video R@1: 40.7video-to-text R@1: 39.6
zero-shot-video-question-answer-on-egoschema-fullsetInternVideo#51Accuracy: 32.1