HowToCaption: Prompting LLMs to Transform Video Annotations at Scale

Benchmark Model Rank Results
zero-shot-video-retrieval-on-lsmdcVAST, HowToCaption-finetuned#3text-to-video R@1: 27.7text-to-video R@5: 46.5…
zero-shot-video-retrieval-on-lsmdcHowToCaption#8text-to-video R@1: 17.3text-to-video R@5: 31.7…
zero-shot-video-retrieval-on-msvdVAST, HowToCaption-finetuned#3text-to-video R@1: 54.8text-to-video R@5: 80.9…
zero-shot-video-retrieval-on-msvdHowToCaption#8text-to-video R@1: 44.5text-to-video R@5: 73.3…
zero-shot-video-retrieval-on-youcook2VAST, HowToCaption-finetuned#3text-to-video R@1: 19.7text-to-video R@5: 43.6…
zero-shot-video-retrieval-on-youcook2HowToCaption#5text-to-video R@1: 13.4text-to-video R@5: 33.1…
video-captioning-on-msvdHowToCaption#5CIDEr: 154.2BLEU-4: 70.4METEOR: 46.4ROUGE-L: 83.2
zero-shot-video-retrieval-on-msr-vttVAST, HowToCaption-finetuned#7text-to-video R@1: 50text-to-video R@5: 73.2text-to-video R@10: 81.4…
zero-shot-video-retrieval-on-msr-vttHowToCaption#20text-to-video R@1: 37.6text-to-video R@5: 62text-to-video R@10: 73.3…
video-captioning-on-msr-vttHowToCaption#8CIDEr: 65.3METEOR: 32.2ROUGE-L: 66.3BLEU-4: 49.8
video-captioning-on-youcook2HowToCaption#8BLEU-4: 8.8CIDEr: 116.4ROUGE-L: 37.3METEOR: 15.9