MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

Benchmark Model Rank Results
question-answering-on-next-qa-open-ended-videoqaMovieChat#5Accuracy: 49.9Confidence Score: 2.7
video-based-generative-performance-benchmarking-detail-orientation-on-videoinstructMovieChat#9gpt-score: 2.93
video-question-answering-on-activitynet-qaMovieChat#11Accuracy: 45.7Confidence score: 3.1
zeroshot-video-question-answer-on-msvd-qaMovieChat#11Accuracy: 75.2Confidence Score: 2.9
video-based-generative-performance-benchmarking-correctness-of-information-on-videoinstructMovieChat#12gpt-score: 2.76
video-based-generative-performance-benchmarking-consistency-on-videoinstructMovieChat#13gpt-score: 2.42
video-based-generative-performance-benchmarking-contextual-understanding-on-videoinstructMovieChat#13gpt-score: 3.01
video-based-generative-performance-benchmarking-temporal-understanding-on-videoinstructMovieChat#13gpt-score: 2.24
video-question-answering-on-ovbenchMovieChat (7B)#13AVG: 30.9
zeroshot-video-question-answer-on-activitynet-qaMovieChat#22Accuracy: 45.7Confidence Score: 3.1
zeroshot-video-question-answer-on-msrvtt-qaMovieChat#24Accuracy: 52.7Confidence Score: 2.6