MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models

Benchmark Model Rank Results
video-captioning-on-youcook2UniVL + MELTR#2BLEU-4: 17.92BLEU-3: 24.12CIDEr: 1.90ROUGE-L: 47.04…
multimodal-sentiment-analysis-on-cmu-mosiUniVL + MELTR#3F1: 85.4Acc-2: 85.3Corr: 0.789MAE: 0.759
video-retrieval-on-youcook2UniVL + MELTR#3text-to-video R@1: 33.7text-to-video R@5: 63.1…
video-retrieval-on-msr-vttAll-in-one + MELTR#13text-to-video R@1: 38.6text-to-video R@5: 74.4…
video-retrieval-on-msr-vttVIOLET + MELTR#16text-to-video R@1: 33.6text-to-video R@5: 63.7…
video-retrieval-on-msr-vttUniVL + MELTR#22text-to-video R@1: 28.5text-to-video R@5: 55.5…
video-captioning-on-msr-vttUniVL + MELTR#16CIDEr: 52.77METEOR: 29.26ROUGE-L: 62.35BLEU-4: 44.17
visual-question-answering-on-msvd-qaVIOLET + MELTR#16Accuracy: 0.517
video-retrieval-on-msr-vtt-1kaAll-in-one + MELTR#36text-to-video R@1: 41.3text-to-video R@5: 73.5…
video-retrieval-on-msr-vtt-1kaVIOLET + MELTR#42text-to-video R@1: 35.5text-to-video R@5: 67.2…
video-retrieval-on-msr-vtt-1kaUniVL + MELTR#44text-to-video R@1: 31.1text-to-video R@5: 55.7…