ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

Benchmark Model Rank Results
action-classification-on-kinetics-400ONE-PEACE#22Acc@1: 88.1Acc@5: 97.8
audio-classification-on-fsd50kONE-PEACE#1mAP: 69.7
audio-classification-on-vggsoundONE-PEACE (Audio-Visual)#2Top 1 Accuracy: 68.2
audio-classification-on-vggsoundONE-PEACE (Audio-Only)#7Top 1 Accuracy: 59.6
image-classification-on-imagenetONE-PEACE#986Number of params: 1520M
image-to-text-retrieval-on-cocoONE-PEACE (ViT-G, w/o ranking)#2Recall@1: 84.1Recall@5: 96.3Recall@10: 98.3
image-to-text-retrieval-on-flickr30kONE-PEACE (finetuned, w/o ranking)#3Recall@1: 97.6Recall@5: 100Recall@10: 100
semantic-segmentation-on-ade20kONE-PEACE#2Validation mIoU: 63.0Params (M): 1500
text-to-audio-retrieval-on-audiocapsONE-PEACE#3R@1: 42.5R@5: 77.5R@10: 88.4
text-to-audio-retrieval-on-clothoONE-PEACE#5R@1: 22.4R@5: 49.0R@10: 62.7
visual-question-answering-on-vqa-v2-test-devONE-PEACE#4Accuracy: 82.6
visual-question-answering-on-vqa-v2-test-stdONE-PEACE#3overall: 82.52yes/no: 94.85number: 72.24other: 74.15