Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

Benchmark Model Rank Results
science-question-answering-on-scienceqaChat-UniVi-13B#4Avg. Accuracy: 90.99Natural Science: 90.41Social Science: 95.05
vcgbench-diverse-on-videoinstructChat-UniVi#2mean: 2.29Correctness of Information: 2.29Detail Orientation: 2.56
video-based-generative-performanceChat-UniVi#13mean: 2.99Correctness of Information: 2.89Detail Orientation: 2.91
video-based-generative-performance-1Chat-UniVi#10gpt-score: 2.89
video-based-generative-performance-2Chat-UniVi#6gpt-score: 2.81
video-based-generative-performance-3Chat-UniVi#10gpt-score: 3.46
video-based-generative-performance-4Chat-UniVi#10gpt-score: 2.91
video-based-generative-performance-5Chat-UniVi#11gpt-score: 2.39
video-question-answering-on-activitynet-qaChat-UniVi-13B#9Accuracy: 46.4Confidence score: 3.3
zeroshot-video-question-answer-on-activitynetChat-UniVi-13B#17Accuracy: 46.4Confidence Score: 3.6
zeroshot-video-question-answer-on-activitynetChat-UniVi#20Accuracy: 46.1Confidence Score: 3.3
zeroshot-video-question-answer-on-msrvtt-qaChat-UniVi-7B#21Accuracy: 55.0Confidence Score: 3.1
zeroshot-video-question-answer-on-msvd-qaChat-UniVi-7B#21Accuracy: 69.3Confidence Score: 3.7
zeroshot-video-question-answer-on-tgif-qaChat-UniVi-7B#10Accuracy: 69.0Confidence Score: 3.8