MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models

Benchmark Model Rank Results
multimodal-sentiment-analysis-on-cmu-mosiUniVL + MELTR#3F1: 85.4Acc-2: 85.3Corr: 0.789MAE: 0.759
video-captioning-on-msr-vtt-1UniVL + MELTR#16CIDEr: 52.77METEOR: 29.26ROUGE-L: 62.35BLEU-4: 44.17
video-captioning-on-youcook2UniVL + MELTR#2BLEU-4: 17.92BLEU-3: 24.12CIDEr: 1.90ROUGE-L: 47.04
video-retrieval-on-msr-vttAll-in-one + MELTR#11text-to-video R@1: 38.6text-to-video R@5: 74.4
video-retrieval-on-msr-vttVIOLET + MELTR#14text-to-video R@1: 33.6text-to-video R@5: 63.7
video-retrieval-on-msr-vttUniVL + MELTR#20text-to-video R@1: 28.5text-to-video R@5: 55.5
video-retrieval-on-msr-vtt-1kaAll-in-one + MELTR#32text-to-video R@1: 41.3text-to-video R@5: 73.5
video-retrieval-on-msr-vtt-1kaVIOLET + MELTR#38text-to-video R@1: 35.5text-to-video R@5: 67.2
video-retrieval-on-msr-vtt-1kaUniVL + MELTR#40text-to-video R@1: 31.1text-to-video R@5: 55.7
video-retrieval-on-youcook2UniVL + MELTR#2text-to-video R@1: 33.7text-to-video R@5: 63.1
visual-question-answering-on-msvd-qa-1VIOLET + MELTR#16Accuracy: 0.517