BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning

Benchmark Model Rank Results
vcgbench-diverse-on-videoinstructBT-Adapter#4mean: 2.19Correctness of Information: 2.20Detail Orientation: 2.62…
zero-shot-action-recognition-on-kinetics-400-zero-shot-action-classificationBT-Adapter#4Accuracy: 64.3
zero-shot-video-retrieval-on-lsmdcBT-Adapter#6text-to-video R@1: 19.5text-to-video R@5: 35.9…
zero-shot-video-retrieval-on-activitynetBT-Adapter#8text-to-video R@1: 37.0text-to-video R@5: 66.7…
video-question-answering-on-activitynet-qaBT-Adapter (zero-shot)#10Accuracy: 46.1Confidence score: 3.6
video-based-generative-performance-benchmarking-consistency-on-videoinstructBT-Adapter#12gpt-score: 2.46
video-based-generative-performance-benchmarking-consistency-on-videoinstructBT-Adapter (zero-shot)#16gpt-score: 2.2
video-based-generative-performance-benchmarking-contextual-understanding-on-videoinstructBT-Adapter#12gpt-score: 3.27
video-based-generative-performance-benchmarking-contextual-understanding-on-videoinstructBT-Adapter (zero-shot)#14gpt-score: 2.89
video-based-generative-performance-benchmarking-temporal-understanding-on-videoinstructBT-Adapter#12gpt-score: 2.34
video-based-generative-performance-benchmarking-temporal-understanding-on-videoinstructBT-Adapter (zero-shot)#14gpt-score: 2.13
video-based-generative-performance-benchmarking-correctness-of-information-on-videoinstructBT-Adapter#13gpt-score: 2.68
video-based-generative-performance-benchmarking-correctness-of-information-on-videoinstructBT-Adapter (zero-shot)#16gpt-score: 2.16
video-based-generative-performance-benchmarking-detail-orientation-on-videoinstructBT-Adapter#13gpt-score: 2.69
video-based-generative-performance-benchmarking-detail-orientation-on-videoinstructBT-Adapter (zero-shot)#16gpt-score: 2.46
zero-shot-video-retrieval-on-didemoBT-Adapter#15text-to-video R@1: 35.6text-to-video R@5: 61.9…
zero-shot-video-retrieval-on-msr-vttBT-Adapter#16text-to-video R@1: 40.9text-to-video R@5: 64.7…
video-based-generative-performance-benchmarking-on-videoinstructBT-Adapter#18mean: 2.69Correctness of Information: 2.68Detail Orientation: 2.69…
video-based-generative-performance-benchmarking-on-videoinstructBT-Adapter (zero-shot)#19mean: 2.46Correctness of Information: 2.16Detail Orientation: 2.46…
zeroshot-video-question-answer-on-activitynet-qaBT-Adapter (zero-shot)#20Accuracy: 46.1Confidence Score: 3.2
zeroshot-video-question-answer-on-msvd-qaBT-Adapter (zero-shot)#23Accuracy: 67.0Confidence Score: 3.6
zeroshot-video-question-answer-on-msvd-qaBT-Adapter (zero-shot)#24Accuracy: 67.0Confidence Score: 3.6
zeroshot-video-question-answer-on-msrvtt-qaBT-Adapter (zero-shot)#25Accuracy: 51.2Confidence Score: 2.9
zeroshot-video-question-answer-on-msrvtt-qaBT-Adapter (zero-shot)#26Accuracy: 51.2Confidence Score: 2.9