InternVideo2: Scaling Foundation Models for Multimodal Video Understanding

Benchmark Model Rank Results
action-classification-on-kinetics-400InternVideo2-6B#1Acc@1: 92.1
action-classification-on-kinetics-400InternVideo2-1B#2Acc@1: 91.6
action-classification-on-kinetics-600InternVideo2-6B#1Top-1 Accuracy: 91.9
action-classification-on-kinetics-600InternVideo2-1B#3Top-1 Accuracy: 91.6
action-classification-on-kinetics-700InternVideo2-6B#1Top-1 Accuracy: 85.9
action-classification-on-kinetics-700InternVideo2-1B#2Top-1 Accuracy: 85.4
action-classification-on-moments-in-timeInternVideo2-1B#1Top 1 Accuracy: 50.9
action-recognition-in-videos-on-activitynetInternVideo2-6B#3mAP: 95.9
action-recognition-in-videos-on-somethingInternVideo2-1B#4Top-1 Accuracy: 77.1
action-recognition-in-videos-on-somethingInternVideo2-6B#99Top-1 Accuracy: 1Top-5 Accuracy: 12Parameters: 2131GFLOPs: 13321
action-recognition-on-hacsInternVideo2-6B#1Top 1 Accuracy: 97.0
audio-classification-on-esc-50InternVideo2#1Top-1 Accuracy: 98.6PRE-TRAINING DATASET: Multiple
moment-retrieval-on-charades-staInternVideo2-6B#5R@1 IoU=0.5: 70.03R@1 IoU=0.7: 48.95
moment-retrieval-on-charades-staInternVideo2-1B#6R@1 IoU=0.5: 68.36R@1 IoU=0.7: 45.03
moment-retrieval-on-qvhighlightsInternVideo2-6B#5mAP: 49.24R@1 IoU=0.5: 71.42R@1 IoU=0.7: 56.45
temporal-action-localization-on-activitynetInternVideo2-6B#5mAP: 41.2
temporal-action-localization-on-activitynetInternVideo2-1B#6mAP: 40.4
temporal-action-localization-on-fineactionInternVideo2-6B#3mAP: 27.7
temporal-action-localization-on-hacsInternVideo2-6B#4Average-mAP: 43.3
temporal-action-localization-on-hacsInternVideo2-1B#6Average-mAP: 42.4
temporal-action-localization-on-thumos14InternVideo2-6B#4Avg mAP (0.3:0.7): 72.0
temporal-action-localization-on-thumos14InternVideo2-1B#7Avg mAP (0.3:0.7): 69.8
text-to-audio-retrieval-on-audiocapsInternVideo2-6B#1R@1: 55.2
text-to-audio-retrieval-on-clothoInternVideo2-6B#2R@1: 27.2
video-grounding-on-qvhighlightsInternVideo2-6B#1R@1,IoU=0.7: 56.45R@1,IoU=0.5: 71.42
video-grounding-on-qvhighlightsInternVideo2-1B#2R@1,IoU=0.7: 54.45R@1,IoU=0.5: 70.00
video-question-answering-on-mvbenchInternVideo2#3Avg.: 67.2
video-question-answering-on-perception-testInternVideo2 (8B)#3Accuracy (Top-1): 63.4
video-retrieval-on-activitynetInternVideo2-6B#1text-to-video R@1: 74.1video-to-text R@1: 69.7
video-retrieval-on-didemoInternVideo2-6B#1text-to-video R@1: 74.2video-to-text R@1: 71.9
video-retrieval-on-lsmdcInternVideo2-6B#1text-to-video R@1: 46.4video-to-text R@1: 46.7
video-retrieval-on-msr-vttInternVideo2-6B#3text-to-video R@1: 62.8video-to-text R@1: 60.2
video-retrieval-on-msvdInternVideo2-6B#1text-to-video R@1: 61.4video-to-text R@1: 85.2
video-retrieval-on-vatexInternVideo2-6B#4text-to-video R@1: 75.5video-to-text R@1: 89.3
zero-shot-video-question-answer-on-egoschema-1InternVideo2-6B#10Accuracy: 60.2
zero-shot-video-retrieval-on-activitynetInternVideo2-6B#1text-to-video R@1: 63.2text-to-video R@5: 85.6
zero-shot-video-retrieval-on-activitynetInternVideo2-1B#2text-to-video R@1: 60.4text-to-video R@5: 83.9
zero-shot-video-retrieval-on-didemoInternVideo2-6B#1text-to-video R@1: 57.9text-to-video R@5: 80.0
zero-shot-video-retrieval-on-didemoInternVideo2-1B#2text-to-video R@1: 57.0text-to-video R@5: 80.0
zero-shot-video-retrieval-on-lsmdcInternVideo2-6B#1text-to-video R@1: 33.8text-to-video R@5: 55.9
zero-shot-video-retrieval-on-lsmdcInternVideo2-1B#2text-to-video R@1: 32.0text-to-video R@5: 52.4
zero-shot-video-retrieval-on-msr-vttInternVideo2-6B#1text-to-video R@1: 55.9text-to-video R@5: 78.3
zero-shot-video-retrieval-on-msr-vttInternVideo2-1B#3text-to-video R@1: 51.9text-to-video R@5: 75.3
zero-shot-video-retrieval-on-msvdInternVideo2-6B#1text-to-video R@1: 59.3text-to-video R@5: 84.4
zero-shot-video-retrieval-on-msvdInternVideo2-1B#2text-to-video R@1: 58.1text-to-video R@5: 83.0
zero-shot-video-retrieval-on-vatexInternVideo2-6B#2text-to-video R@1: 71.5text-to-video R@5: 94.0
zero-shot-video-retrieval-on-vatexInternVideo2-1B#3text-to-video R@1: 70.4text-to-video R@5: 93.4