Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

Benchmark Model Rank Results
vcgbench-diverse-on-videoinstructChat-UniVi#2mean: 2.29Correctness of Information: 2.29Detail Orientation: 2.56…
video-based-generative-performance-benchmarking-consistency-on-videoinstructChat-UniVi#6gpt-score: 2.81
science-question-answering-on-scienceqaChat-UniVi-13B#7Avg. Accuracy: 90.99Natural Science: 90.41Social Science: 95.05…
video-question-answering-on-activitynet-qaChat-UniVi-13B#9Accuracy: 46.4Confidence score: 3.3
video-based-generative-performance-benchmarking-contextual-understanding-on-videoinstructChat-UniVi#10gpt-score: 3.46
video-based-generative-performance-benchmarking-correctness-of-information-on-videoinstructChat-UniVi#10gpt-score: 2.89
video-based-generative-performance-benchmarking-detail-orientation-on-videoinstructChat-UniVi#10gpt-score: 2.91
video-based-generative-performance-benchmarking-temporal-understanding-on-videoinstructChat-UniVi#11gpt-score: 2.39
zeroshot-video-question-answer-on-tgif-qaChat-UniVi-7B#11Accuracy: 69.0Confidence Score: 3.8
video-based-generative-performance-benchmarking-on-videoinstructChat-UniVi#13mean: 2.99Correctness of Information: 2.89Detail Orientation: 2.91…
zeroshot-video-question-answer-on-activitynet-qaChat-UniVi-13B#18Accuracy: 46.4Confidence Score: 3.6
zeroshot-video-question-answer-on-activitynet-qaChat-UniVi#21Accuracy: 46.1Confidence Score: 3.3
zeroshot-video-question-answer-on-msrvtt-qaChat-UniVi-7B#22Accuracy: 55.0Confidence Score: 3.1
zeroshot-video-question-answer-on-msvd-qaChat-UniVi-7B#22Accuracy: 69.3Confidence Score: 3.7