VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners

Benchmark Model Rank Results
video-captioning-on-activitynet-captionsVideoCoCaBLEU4: 14.7CIDEr: 39.3ROUGE-L: 35.0
video-captioning-on-msr-vtt-1VideoCoCaCIDEr: 73.2ROUGE-L: 68.0BLEU-4: 53.8
video-captioning-on-vatex-1VideoCoCaBLEU-4: 39.7CIDEr: 77.8ROUGE-L: 54.5
video-captioning-on-youcook2VideoCoCaBLEU-4: 14.2CIDEr: 1.28ROUGE-L: 37.7
video-question-answering-on-activitynet-qaVideoCoCaAccuracy: 56.1
video-question-answering-on-ivqaVideoCoCaAccuracy: 39.0
video-retrieval-on-msr-vttVideoCoCa (zero-shot)text-to-video R@1: 34.3text-to-video R@5: 57.8
video-retrieval-on-youcook2VideoCoCa (zero-shot)text-to-video R@1: 21.7text-to-video R@5: 43.9
visual-question-answering-on-msrvtt-qa-1VideoCoCaAccuracy: 0.463
visual-question-answering-on-msvd-qa-1VideoCoCaAccuracy: 0.569
zero-shot-action-recognition-on-charades-1VideoCoCamAP: 25.8
zero-shot-action-recognition-on-hmdb51VideoCoCaTop-1 Accuracy: 58.7Top-5 Accuracy: 84.5
zero-shot-action-recognition-on-kineticsVideoCoCaTop-1 Accuracy: 70.1Top-5 Accuracy: 88.9
zero-shot-action-recognition-on-ucf101VideoCoCaTop-1 Accuracy: 86.6Top-5 accuracy: 98.4
zero-shot-video-retrieval-on-activitynetVideoCoCatext-to-video R@1: 34.5text-to-video R@5: 63.2
zero-shot-video-retrieval-on-msr-vtt-fullVideoCoCatext-to-video R@1: 34.3text-to-video R@5: 57.8
zero-shot-video-retrieval-on-vatexVideoCoCatext-to-video R@1: 53.2text-to-video R@5: 83.3
zero-shot-video-retrieval-on-youcook2VideoCOcatext-to-video R@1: 20.3text-to-video R@5: 43.0