OmniVec2 - A Novel Transformer based Network for Large Scale Multimodal and Multitask Learning

Benchmark Model Rank Results
3d-point-cloud-classification-on-modelnet40-cOmniVec2Error Rate: 0.142
3d-point-cloud-classification-on-scanobjectnnOmniVec2Overall Accuracy: 97.2
action-classification-on-kinetics-400OmniVec2Acc@1: 93.6
action-classification-on-moments-in-timeOmniVec2Top 1 Accuracy: 53.1
action-classification-on-moments-in-time-2OmniVec2Top 1 Accuracy: 53.1
action-recognition-in-videos-on-ucf101OmniVec23-fold Accuracy: 99.6
audio-classification-on-audiosetOmniVec2Test mAP: 0.558
audio-classification-on-esc-50OmniVec2Top-1 Accuracy: 99.1PRE-TRAINING DATASET: Multiple
fine-grained-image-classification-on-oxford-1OmniVec2Accuracy: 99.6
image-classification-on-imagenetOmniVec2Top 1 Accuracy: 89.3%
image-classification-on-inaturalist-2018OmniVec2Top-1 Accuracy: 94.6
image-classification-on-places365OmniVec2Top 1 Accuracy: 65.1
semantic-segmentation-on-nyu-depth-v2OmniVec2Mean IoU: 63.6
text-summarization-on-dialogsumOmniVec2Rouge1: 47.6Rouge2: 22.1RougeL: 41.4BertScore: 72.8
text-summarization-on-samsum-corpusOmniVec2ROUGE-1: 59.1ROUGE-2: 34.1ROUGE-L: 63.7BertScoreF1: 65.1
zero-shot-video-retrieval-on-youcook2OmniVec2text-to-video R@1: 26.1text-to-video R@5: 54.1