MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Benchmark Model Rank Results
3d-question-answering-3d-qa-on-scanqa-test-wVideoChat2#11Exact Match: 19.2BLEU-4: 9.6ROUGE: 28.2METEOR: 9.5CIDEr: 49.2
3d-question-answering-3d-qa-on-sqa3dVideoChat2#7Exact Match: 37.3
vcgbench-diverse-on-videoinstructVideoChat2#3mean: 2.20Correctness of Information: 2.13Detail Orientation: 2.42
video-based-generative-performanceVideoChat2_HD_mistral#10mean: 3.10Correctness of Information: 3.40Detail Orientation: 2.91
video-based-generative-performanceVideoChat2#15mean: 2.98Correctness of Information: 3.02Detail Orientation: 2.88
video-based-generative-performance-1VideoChat2_HD_mistral#5gpt-score: 3.40
video-based-generative-performance-1VideoChat2#9gpt-score: 3.02
video-based-generative-performance-2VideoChat2#7gpt-score: 2.81
video-based-generative-performance-2VideoChat2_HD_mistral#10gpt-score: 2.62
video-based-generative-performance-3VideoChat2_HD_mistral#7gpt-score: 3.64
video-based-generative-performance-3VideoChat2#9gpt-score: 3.51
video-based-generative-performance-4VideoChat2#11gpt-score: 2.88
video-based-generative-performance-4VideoChat2_HD_mistral#12gpt-score: 2.86
video-based-generative-performance-5VideoChat2#7gpt-score: 2.66
video-based-generative-performance-5VideoChat2_HD_mistral#8gpt-score: 2.65
video-question-answering-on-activitynet-qaVideoChat2#4Accuracy: 49.1Confidence score: 3.3
video-question-answering-on-intentqaVideoChat2_HD_mistral#1Accuarcy: 83.4CW: 84.0CH: 90.0TP&TN: 77.3
video-question-answering-on-intentqaVideoChat2_mistral#2Accuarcy: 81.9CW: 82.6CH: 86.9TP&TN: 77.0
video-question-answering-on-mvbenchVideoChat2#13Avg.: 51.9
video-question-answering-on-next-qaVideoChat2_HD_mistral#13Accuracy: 79.5
video-question-answering-on-next-qaVideoChat2_mistral#16Accuracy: 78.6
video-question-answering-on-next-qaVideoChat2#28Accuracy: 68.6
video-question-answering-on-tvbenchVideoChat2#21Average Accuracy: 35.0
zero-shot-video-question-answer-on-egoschemaVideoChat2_HD_mistral#6Accuracy: 65.6
zero-shot-video-question-answer-on-egoschemaVideoChat2_mistral#7Accuracy: 63.6
zero-shot-video-question-answer-on-egoschema-1VideoChat2_phi3#12Accuracy: 56.7
zero-shot-video-question-answer-on-egoschema-1VideoChat2_HD_mistral#13Accuracy: 55.8
zero-shot-video-question-answer-on-egoschema-1VideoChat2_mistral#14Accuracy: 54.4
zero-shot-video-question-answer-on-next-qaVideoChat2#17Accuracy: 61.7
zero-shot-video-question-answer-on-starVideoChat2#1Accuracy: 59.0
zero-shot-video-question-answer-on-tvqaVideoChat_HD_mistral (no speech)#4Accuracy: 50.6
zero-shot-video-question-answer-on-tvqaVideoChat_mistral (no speech)#5Accuracy: 46.4
zero-shot-video-question-answer-on-tvqaVideoChat2 (no speech)#6Accuracy: 40.6
zeroshot-video-question-answer-on-activitynetVideoChat2#14Accuracy: 49.1Confidence Score: 3.3
zeroshot-video-question-answer-on-msrvtt-qaVideoChat2#22Accuracy: 54.1Confidence Score: 3.3
zeroshot-video-question-answer-on-msvd-qaVideoChat2#18Accuracy: 70.0Confidence Score: 3.9