Tarsier: Recipes for Training and Evaluating Large Video Description Models

Benchmark Model Rank Results
zero-shot-video-question-answer-on-egoschema-subsetTarsier (34B)#1Accuracy: 68.6
zeroshot-video-question-answer-on-activitynet-qaTarsier (34B)#1Accuracy: 61.6Confidence Score: 3.7
zeroshot-video-question-answer-on-tgif-qaTarsier (34B)#1Accuracy: 82.5Confidence Score: 4.4
zero-shot-video-question-answer-on-next-qaTarsier (34B)#2Accuracy: 79.2
zeroshot-video-question-answer-on-msvd-qaTarsier (34B)#2Accuracy: 80.3Confidence Score: 4.2
video-question-answering-on-tvbenchTarsier-34B#4Average Accuracy: 55.5
video-question-answering-on-tvbenchTarsier-7B#12Average Accuracy: 46.9
zeroshot-video-question-answer-on-msrvtt-qaTarsier (34B)#5Accuracy: 66.4Confidence Score: 3.7
zero-shot-video-question-answer-on-egoschema-fullsetTarsier (34B)#25Accuracy: 61.7
video-question-answering-on-mvbenchTarsier (34B)#43Avg.: 67.6